Compare commits

..
Author SHA1 Message Date
Pratik Mankawde c4adc416e1 Merge the Boost.Coroutine cleanup branch
Bring in Stdcoroutine-4, which now carries the migration plan and the
latest develop. Keep develop's new strategy matrix layout and re-add the
thread sanitizer to its sanitizer list.
2026-10-02 18:04:32 +01:00
Pratik Mankawde be6f5f8602 Merge the test migration branch
Bring in Stdcoroutine-3, which now carries the migration plan and the
latest develop.
2026-10-02 17:55:54 +01:00
Pratik Mankawde 4034c866a3 Merge the entry-point migration branch
Bring in Stdcoroutine-2, which now carries the migration plan and the
latest develop.
2026-10-02 17:49:17 +01:00
Pratik Mankawde 4935cbaeb3 style: Remove unused Zero.h include from PayChan_test
misc-include-cleaner flags it now that this branch edits the file.
2026-10-02 17:47:30 +01:00
Pratik Mankawde 302a486c5f Merge the coroutine primitives branch
Bring in Stdcoroutine-1, which now carries the migration plan and the
latest develop.
2026-10-02 17:40:57 +01:00
Pratik Mankawde 754e0b90b7 Merge the coroutine migration plan and develop
Merge Stdcoroutine-0 (the migration plan docs), which now includes the
latest develop. Keep both sides' cspell words and drop the duplicate
stackful entry.
2026-10-02 17:33:42 +01:00
Pratik Mankawde 0ec6648c5f Merge branch 'develop'
Resolve the cspell word list conflict after develop renamed the file to
.cspell.config.yaml.
2026-10-02 16:34:27 +01:00
Pratik Mankawde 13b2d82645 Merge branch 'pratik/std-coro/cleanup-boost-coroutine' into pratik/std-coro/tsan-fixes 2026-08-19 17:47:42 +01:00
Pratik Mankawde bacfb38233 Merge branch 'pratik/std-coro/migrate-test-code' into pratik/std-coro/cleanup-boost-coroutine 2026-08-19 17:47:42 +01:00
Pratik Mankawde bec82ee704 Merge branch 'pratik/std-coro/migrate-entry-points' into pratik/std-coro/migrate-test-code 2026-08-19 17:47:41 +01:00
Pratik Mankawde c6224c48ed Merge branch 'pratik/std-coro/add-coroutine-primitives' into pratik/std-coro/migrate-entry-points
Bring in develop via the upstream branch. Two classes of collision needed
manual resolution:

Namespace case refactor (develop #7933) renamed RPC -> rpc and
xrpl::RPC::Tuning -> xrpl::rpc::tuning. Conflicts were resolved by taking
develop's naming together with this stack's coroutine API (postCoroTask,
CoroTask<void>, no Context::coro):
  - src/test/app/Path_test.cpp
  - src/test/app/PathMPT_test.cpp
  - src/test/jtx/impl/TestHelpers.cpp
  - src/xrpld/rpc/handlers/orderbook/RipplePathFind.cpp (no textual
    conflict; stale spellings carried forward on stack-authored lines)

New old-API caller: develop added postCoro and Context::coro uses in
src/test/app/PayChan_test.cpp, which this stack does not otherwise touch,
so it merged without conflict but would not compile once Context::coro is
removed here. Both call sites are migrated to postCoroTask and the
.coro = {} designated initialisers dropped. The kFeeHeavyBurdenRpc
assertions that motivated develop's change are preserved unchanged.
2026-08-19 17:47:30 +01:00
Pratik Mankawde 5fb1429e3a Merge remote-tracking branch 'origin/develop' into pratik/std-coro/add-coroutine-primitives
# Conflicts:
#	.cspell.config.yaml
2026-08-19 17:41:56 +01:00
Pratik Mankawde 039e855127 Merge branch 'pratik/std-coro/cleanup-boost-coroutine' into pratik/std-coro/tsan-fixes 2026-07-27 13:48:08 +01:00
Pratik Mankawde b54c500bb2 Merge branch 'pratik/std-coro/migrate-test-code' into pratik/std-coro/cleanup-boost-coroutine 2026-07-27 13:48:08 +01:00
Pratik Mankawde b6526522fe Merge branch 'pratik/std-coro/migrate-entry-points' into pratik/std-coro/migrate-test-code 2026-07-27 13:48:08 +01:00
Pratik Mankawde 61041155d2 Merge branch 'pratik/std-coro/add-coroutine-primitives' into pratik/std-coro/migrate-entry-points 2026-07-27 13:48:08 +01:00
Pratik Mankawde 5ea72b45f1 Merge remote-tracking branch 'origin/develop' into pratik/std-coro/add-coroutine-primitives 2026-07-27 13:47:54 +01:00
Pratik Mankawde 65c45fa02d Merge branch 'pratik/std-coro/cleanup-boost-coroutine' into pratik/std-coro/tsan-fixes 2026-07-27 13:06:43 +01:00
Pratik Mankawde 7b95a8ea8b Merge branch 'pratik/std-coro/migrate-test-code' into pratik/std-coro/cleanup-boost-coroutine 2026-07-27 13:06:43 +01:00
Pratik Mankawde 7385004952 Move coroutine-capture NOLINTNEXTLINE comments next to the lambdas
clang-format placed the suppression comments two lines above the lambda
expressions (before the postCoroTask call), so NOLINTNEXTLINE suppressed
the wrong line and clang-tidy still reported
cppcoreguidelines-avoid-capturing-lambda-coroutines at the lambda. Move
the comments inside the argument list, immediately before each lambda,
matching the pattern already used in threadSpecificStorage.
2026-07-27 13:06:13 +01:00
Pratik Mankawde f172a56c4b Merge branch 'pratik/std-coro/cleanup-boost-coroutine' into pratik/std-coro/tsan-fixes 2026-07-27 12:36:46 +01:00
Pratik Mankawde fce702d5c3 Merge branch 'pratik/std-coro/migrate-test-code' into pratik/std-coro/cleanup-boost-coroutine 2026-07-27 12:36:46 +01:00
Pratik Mankawde a6fea1227f Suppress clang-tidy coroutine-capture warnings in core tests
The JobQueue_test and Coroutine_test coroutine lambdas capture pointers
to test-scope locals, but each test drives the coroutine to completion
(or, for the stopped-queue case, guarantees it never starts) before the
locals go out of scope. Add NOLINTNEXTLINE for
cppcoreguidelines-avoid-capturing-lambda-coroutines with comments
explaining the lifetime guarantee, and drop the unused <memory> include
flagged by misc-include-cleaner.
2026-07-27 12:36:10 +01:00
Pratik Mankawde 968f9e1bf3 Merge branch 'pratik/std-coro/migrate-entry-points' into pratik/std-coro/migrate-test-code 2026-07-27 12:34:16 +01:00
Pratik Mankawde 70813b336a Suppress clang-tidy coroutine-capture warnings in pathfinding tests
The Path_test and TestHelpers coroutine lambdas capture locals by
reference, but the caller blocks on Gate::waitFor() until the coroutine
signals completion, so the captures cannot dangle. Add NOLINTNEXTLINE
for cppcoreguidelines-avoid-capturing-lambda-coroutines with a comment
explaining the lifetime guarantee.
2026-07-27 12:33:49 +01:00
Pratik Mankawde ae98fc30d0 Merge branch 'pratik/std-coro/cleanup-boost-coroutine' into pratik/std-coro/tsan-fixes 2026-07-27 12:06:41 +01:00
Pratik Mankawde 47d84d2e21 Merge branch 'pratik/std-coro/migrate-test-code' into pratik/std-coro/cleanup-boost-coroutine 2026-07-27 12:06:41 +01:00
Pratik Mankawde ab8350b558 Merge branch 'pratik/std-coro/migrate-entry-points' into pratik/std-coro/migrate-test-code 2026-07-27 12:06:41 +01:00
Pratik Mankawde 072e11ecd1 Address clang-tidy findings in entry-point migration
- Suppress cppcoreguidelines-avoid-capturing-lambda-coroutines with
  NOLINT where lifetime is guaranteed: GRPCServer (thisShared keeps
  CallData alive), ServerHandler RPC/WS clients (captures by value,
  handler outlives JobQueue jobs), and PathMPT_test (test blocks on
  Gate::waitFor until the coroutine completes).
- Change ServerHandler::processRequest to take Output const& to fix
  cppcoreguidelines-rvalue-reference-param-not-moved.
- Replace std::lock_guard with std::scoped_lock in RipplePathFind.
- Remove redundant includes and default member initializer in
  Context.h, GRPCServer.h, RipplePathFind.cpp.
2026-07-27 12:06:25 +01:00
Pratik Mankawde 1d51ba21b5 Merge branch 'pratik/std-coro/cleanup-boost-coroutine' into pratik/std-coro/tsan-fixes
# Conflicts:
#	include/xrpl/core/JobQueue.h
2026-07-27 11:31:31 +01:00
Pratik Mankawde 3ac48e297f Merge branch 'pratik/std-coro/migrate-test-code' into pratik/std-coro/cleanup-boost-coroutine 2026-07-27 11:30:34 +01:00
Pratik Mankawde f669b70451 Merge branch 'pratik/std-coro/migrate-entry-points' into pratik/std-coro/migrate-test-code 2026-07-27 11:30:34 +01:00
Pratik Mankawde 62839b1531 Merge branch 'pratik/std-coro/add-coroutine-primitives' into pratik/std-coro/migrate-entry-points 2026-07-27 11:30:34 +01:00
Pratik Mankawde 9bddc54722 Fix clang-tidy violations in coroutine primitives
- NOLINT the compiler-mandated coroutine protocol names (promise_type,
  await_ready, ...) that conflict with readability-identifier-naming and
  readability-convert-member-functions-to-static
- Add [[nodiscard]] to handle(), done(), await_ready()
- Replace std::lock_guard with std::scoped_lock const
- Pass CoroTaskRunner name by value and std::move it; default-init
  runCount_ in-class
- Drop unused <coroutine> include from JobQueue.h; include
  instrumentation.h directly in JobQueueAwaiter.h
- CoroTask_test: kN constant naming, const locals, file-level NOLINT for
  cppcoreguidelines-avoid-capturing-lambda-coroutines (lifetimes are
  gated and joined)
2026-07-27 11:30:16 +01:00
Pratik Mankawde 5899912a76 Tighten TSAN runtime options and suppressions
- runtime-tsan-options.txt: halt_on_error=true so CI fails fast on the
  first unsuppressed report instead of only relying on the exit-code-66
  behavior at process end.
- tsan.supp: drop entries that were redundant with the boost/asio/ and
  boost/context/ path patterns (executor.hpp, boost::asio,
  fiber::resume, spawned_fiber_thread*), and remove over-broad ones:
  race:__tsan_memcpy (matches any race whose top frame is the memcpy
  interceptor), race:pipe (substring-matches any function containing
  'pipe'; narrowed to __pipe), called_from_lib:libclang_rt (GCC builds
  link libtsan, and it would blanket-suppress interceptor reports),
  signal:execute_native_thread_routine (matches every std::thread entry
  point), and the duplicate std::__cxx11::basic_stringbuf. Document
  that path-pattern suppressions match ANY frame in a report.
- workflow: add a lower-bound guard so BUILD_NPROC never drops below 1
  after the coverage decrement on small runners.
- docs/build/sanitizers.md: document the halt_on_error=true default and
  how to collect all reports locally.
2026-07-27 10:35:22 +01:00
Pratik Mankawde 9857842179 Merge branch 'pratik/std-coro/cleanup-boost-coroutine' into pratik/std-coro/tsan-fixes
# Conflicts:
#	include/xrpl/core/CoroTaskRunner.ipp
2026-07-27 10:31:08 +01:00
Pratik Mankawde 598bda0ebc Merge branch 'pratik/std-coro/migrate-test-code' into pratik/std-coro/cleanup-boost-coroutine 2026-07-27 10:30:16 +01:00
Pratik Mankawde 25b9b4d27d Fix JobQueue_test resume() contract violation and Coroutine_test timeout hazards
- PostCoroTest2 called runner->resume() directly from the test thread,
  violating the documented precondition on CoroTaskRunner::resume() (a
  post() must precede every resume) and driving runCount_ negative.
  Rewrite the loop as post()+join(); the non-atomic yieldCount now also
  verifies the happens-before edge join() provides via mutexRun_.
- PostCoroTest3 wrote false into an already-false flag, so it could
  not detect the coroutine running after stop(). Write true and assert
  the flag stays false.
- Coroutine_test: early-return when a Gate waitFor() times out instead
  of falling through to c->join()/a[i]->join(), which would deref a
  null shared_ptr (correctOrder, threadSpecificStorage first loop) or
  block indefinitely (second loop).
2026-07-27 10:29:57 +01:00
Pratik Mankawde 22ed85c3ad Merge branch 'pratik/std-coro/migrate-entry-points' into pratik/std-coro/migrate-test-code 2026-07-27 10:26:00 +01:00
Pratik Mankawde 8c7a27d721 Name the ripple_path_find wait timeout and fire completion on exception
- Replace the magic 30s in doRipplePathFind with
  RPC::Tuning::kPathfindCompletionTimeout.
- In PathRequestManager::updateAll, invoke updateComplete() via a
  ScopeExit guard on the one-shot (hasCompletion) path so the blocked
  RPC handler is released immediately even if doUpdate throws, instead
  of waiting out the full timeout.
2026-07-25 15:05:08 +01:00
Pratik Mankawde 9266c23ef6 Bound concurrent blocking ripple_path_find calls to prevent worker starvation
The no-ledger branch of doRipplePathFind parks its JobQueue worker on a
condition_variable for up to 30s, waiting for a completion that is fired
from a JtUpdatePf job -- which itself needs a free worker to run. With
unbounded JtClientRpc concurrency, enough simultaneous ripple_path_find
calls could park every worker, stalling the entire JobQueue until the
30s timeouts expired. The old Boost.Coroutine implementation suspended
and released the worker, so it did not have this failure mode.

Reuse the RPC::LegacyPathFind guard (already applied on the
ledger-specified branch) to admit at most kMaxPathfindsInProgress
non-admin blocking requests, returning rpcTOO_BUSY beyond that. The
guard stays in scope across the wait.
2026-07-25 12:39:04 +01:00
Pratik Mankawde 5e9ce16de6 Merge branch 'pratik/std-coro/add-coroutine-primitives' into pratik/std-coro/migrate-entry-points 2026-07-25 12:37:14 +01:00
Pratik Mankawde 8d4ea00453 Strengthen CoroTask_test assertions and timeout handling
- testExceptionPropagation now co_awaits an inner CoroTask<void> that
  throws and asserts the rethrown message, covering the
  CoroTask<void>::await_resume rethrow path; the old version could not
  distinguish a throw from a normal return.
- testJobQueueAwaiter now actually awaits the JobQueueAwaiter struct
  (single use per coroutine, per the GCC-12 note) and asserts the full
  ordered step sequence instead of only the terminal value.
- testValueException asserts the caught exception's message.
- All waitFor() timeouts early-return on failure instead of falling
  through to join()/state reads (null-deref and TSAN-race hazards on
  timeout, plus the nSuspend_ assert in ~Env).
- Remove dead shared_ptr locals in testCorrectOrder/testMultipleYields.
- Add missing includes (<array>, <stdexcept>, <string>, <vector>,
  LocalValue.h, CoroTask.h).
2026-07-25 12:36:30 +01:00
Pratik Mankawde bdcf094171 Guard resume() against completed coroutines and log unhandled exceptions
- resume() now skips the handle resume when the task is null or done
  (duplicate external post() after completion), matching the old
  Coro::resume() 'if (coro_)' guard instead of invoking UB in release
  builds. The runCount_ bookkeeping still runs to balance post().
- Exceptions escaping a top-level coroutine body were captured by
  unhandled_exception() and destroyed unobserved with the frame; they
  are now logged at error level before the frame is released.
- Document that join() may return via the finished_ disjunct while the
  final resume() is still completing its bookkeeping.
2026-07-25 12:33:51 +01:00
Pratik Mankawde a42e8174d9 Fix check-rename CI failure
- Remove BoostToStdCoroutineSwitchPlan.md and BoostToStdCoroutineTaskList.md
  working documents from the repo root; the rename script rewrites their
  'rippled' references and the job fails on the resulting dirty tree.
- Reword two CoroTask.h comments to use 'xrpld'.
- Drop cspell words (cppcoro, gantt, Pratik, Mankawde) that existed only
  for the removed documents.
2026-07-25 12:30:38 +01:00
Pratik Mankawde de4d93b3c5 Merge branch 'pratik/std-coro/cleanup-boost-coroutine' into pratik/std-coro/tsan-fixes
Forward-merge PR 6429 (and develop) into the TSAN branch.

Conflict resolutions:
- JobQueue.h: keep the TSAN-safe postCoroTask (jobCounter_.wrap slot
  reservation) on top of the legacy Coro removal.
- CoroTaskRunner.ipp: keep atomic finished_ (acquire loads) with the
  renamed mutexRun_ from develop.
- reusable-build-test-config.yml: keep the single sanitizer-options step
  and fold the sanitizer single-threaded test parallelism into the
  coverage parallelism logic.
- linux.json: add "thread" to the amd64 sanitizer matrix (replaces the
  removed generate.py logic).
- CoroTask_test.cpp: rename create_t/jtCLIENT to CreateT/JtClient in
  testExpectEarlyExit.
2026-07-24 21:13:11 +01:00
Pratik Mankawde 791c061a98 Merge branch 'pratik/std-coro/migrate-test-code' into pratik/std-coro/cleanup-boost-coroutine
Forward-merge the test-code migration (carrying the latest develop) into the
Boost.Coroutine cleanup.

Conflict resolutions: keep this branch's removal of JobQueue::Coro,
CoroCreateT, postCoro, Coro.ipp and the Boost.Coroutine2 includes, while
taking develop's include set, Doxygen comment style and formatting for the
retained CoroTaskRunner / postCoroTask declarations.
2026-07-24 21:02:25 +01:00
Pratik Mankawde 91004209e9 Merge branch 'pratik/std-coro/migrate-entry-points' into pratik/std-coro/migrate-test-code
Forward-merge the entry-point migration (carrying the latest develop) into
the test-code migration.

Conflict resolutions in Coroutine_test.cpp and JobQueue_test.cpp: keep the
postCoroTask / CoroTaskRunner form from this branch, with develop's renames
(JtClient, Gate::waitFor, kN) and develop's initialization of the
unprotected flag.
2026-07-24 20:57:01 +01:00
Pratik Mankawde 4c18dd867c Merge branch 'pratik/std-coro/add-coroutine-primitives' into pratik/std-coro/migrate-entry-points
Forward-merge the updated coroutine primitives (which themselves carry the
latest develop) into the entry-point migration.

Conflict resolutions:
- .cspell.config.yaml: keep develop's dotfile name, merge word lists.
- Context.h: drop the coro member; infoSub gets a default initializer.
- Application.cpp, ServerHandler.cpp, TestHelpers.cpp, Path_test.cpp,
  PathMPT_test.cpp: adopt develop's designated-initializer JsonContext and
  renamed identifiers (JtClient, kApiVersionIfUnspecified, kMaxSrcCur, Gate).
- GRPCServer: process(coro) becomes processRequest(), posted with
  postCoroTask.
- RipplePathFind.cpp: follow develop's move to handlers/orderbook/ and
  PathRequestManager; the completion state shared with the path-finding
  continuation is now heap-allocated so it outlives an early return.
- AMMTest.cpp: develop moved find_paths_request into TestHelpers.cpp, so the
  local copy is dropped and the migration applied there instead.
2026-07-24 20:52:28 +01:00
Pratik Mankawde fb6ece56f2 Merge remote-tracking branch 'origin/develop' into pratik/std-coro/add-coroutine-primitives
Resolves the develop rename of cspell.config.yaml and the JobQueue.h
conflicts, and aligns the new coroutine primitives with develop naming
(CreateT, mutex_/mutexRun_, JtClient, forceMultiThread,
beast::unit_test::Suite).

Also addresses two review findings:

- postCoroTask now holds a jobCounter_ reservation for the whole
  function. JobQueue::stop() joins jobCounter_ before asserting
  nSuspend_ == 0, so the reservation closes the window between the
  ++nSuspend_ and the balancing post()/expectEarlyExit(), and doubles
  as the shutdown check.

- YieldPostAwaiter::await_suspend returns a coroutine_handle<>
  (symmetric transfer) instead of resuming inline. This keeps a
  yield loop against a stopping JobQueue from growing the stack
  without bound and avoids touching a frame that resume() may have
  already destroyed.
2026-07-24 20:39:41 +01:00
Pratik Mankawde c85cd77b2f cspell update
Signed-off-by: Pratik Mankawde <[email protected]>
2026-04-22 14:14:16 +01:00
Pratik MankawdeandClaude Opus 4.6 cb5259d7c5 docs: Add Boost to C++20 coroutine migration plan
Comprehensive migration plan documenting the switch from
Boost.Coroutine2 to C++20 standard coroutines in rippled, including
research analysis, implementation phases, risk assessment, and
testing strategy.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-04-22 13:18:04 +01:00
Pratik MankawdeandClaude Opus 4.6 01fe13dd54 fix: TSAN data-race fixes and sanitizer CI configuration
- Fix TSAN data races in CoroTaskRunner (atomic operations, mutex guards)
- Enable TSAN CI builds with proper ucontext support
- Add TSAN suppressions for pre-existing rippled issues
- Remove -fno-pie flags for sanitizer compatibility

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-25 15:48:17 +00:00
Pratik Mankawde b53df32334 feat: Remove Boost.Coroutine dependency and old Coro API
Remove the legacy Boost.Coroutine infrastructure now that all callers
have been migrated to C++20 std::coroutine:

- Delete Coro.ipp: Remove the Boost.Coroutine-based Coro class and
  its suspend/resume/post/runOnJobQueue helpers.
- Remove Coro API from JobQueue.h: Remove postCoro(), Coro class
  declaration, and all Boost.Coroutine-related methods.
- Remove Boost.Coroutine from build: Drop coroutine and context from
  Boost components in cmake and conanfile.py.
- Remove BOOST_COROUTINES_NO_DEPRECATION_WARNING compile definition.
2026-03-25 15:48:17 +00:00
Pratik Mankawde 42cced50fb feat: Migrate coroutine tests from Boost.Coroutine to C++20 coroutines
Migrate Coroutine_test and JobQueue_test from Boost.Coroutine to
C++20 std::coroutine using CoroTask/CoroTaskRunner:

- Coroutine_test: Replace Coro-based coroutine tests with CoroTask
  equivalents using co_await runner->yieldAndPost().
- JobQueue_test: Replace Coro suspend/resume patterns with CoroTask
  equivalents, use pointer-by-value captures in coroutine lambdas
  to avoid dangling reference issues.
2026-03-25 15:48:17 +00:00
Pratik Mankawde 0e815aa1ac feat: Migrate production entry points from Boost.Coroutine to C++20 coroutines
Migrate all production coroutine entry points from Boost.Coroutine
to C++20 std::coroutine using the CoroTask/CoroTaskRunner primitives:

- RipplePathFind: Replace Coro suspend/resume with co_await pattern,
  add cv timeout for graceful shutdown.
- ServerHandler: Replace Coro-based processRequest with CoroTask,
  simplify coroutine lifecycle management.
- GRPCServer: Replace Coro with CoroTask for streaming RPC handlers.
- Remove Coro usage from Context.h aggregate initialization.
- Add exception handling in coroutine bodies to prevent unhandled
  exceptions from escaping the coroutine frame.
2026-03-25 15:48:10 +00:00
Pratik Mankawde 21149a81e3 feat: Add C++20 coroutine primitives: CoroTask, CoroTaskRunner, JobQueueAwaiter
Add C++20 std::coroutine based task primitives for the JobQueue:

- CoroTask<T>: A coroutine return type with RAII ownership semantics
  and symmetric transfer for efficient resumption.
- CoroTaskRunner: Manages coroutine lifecycle on the JobQueue with
  suspend/resume tracking, LocalValue preservation, and graceful
  shutdown support.
- JobQueueAwaiter: External awaiter combining yield+post atomically.
- yieldAndPost(): Inline awaiter workaround for GCC-12 codegen bug
  where external awaiters at multiple co_await points corrupt the
  coroutine state machine resume index.
- CoroTask_test: Comprehensive test suite covering task lifecycle,
  suspend/resume, shutdown, and value-returning coroutines.
- BoostToStdCoroutineSwitchPlan.md: Migration plan documentation.
2026-03-25 15:46:44 +00:00
Pratik MankawdeandClaude Opus 4.6 b78202a99a docs: Add Boost to C++20 coroutine migration plan
Comprehensive migration plan documenting the switch from
Boost.Coroutine2 to C++20 standard coroutines in rippled, including
research analysis, implementation phases, risk assessment, and
testing strategy.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-25 15:44:19 +00:00
317 changed files with 9819 additions and 9869 deletions

No files matched your search

+9 -1
View File
@@ -83,6 +83,7 @@ words:
- coeffs
- coldwallet
- compr
- cppcoro
- conanfile
- conanrun
- confs
@@ -119,6 +120,7 @@ words:
- enablerepo
- endmacro
- envrc
- eventfd
- exceptioned
- EXPECT_STREQ
- Falco
@@ -129,6 +131,7 @@ words:
- fsanitize
- funclets
- Gamal
- gantt
- gcov
- gcovr
- ghead
@@ -178,6 +181,7 @@ words:
- ltype
- mathbunnyru
- mcmodel
- Mankawde
- MEMORYSTATUSEX
- MPTAMM
- MPTDEX
@@ -247,6 +251,7 @@ words:
- protobuf
- protos
- ptrs
- Pratik
- pushd
- pyenv
- pyparsing
@@ -255,10 +260,11 @@ words:
- queuable
- Raphson
- rcflags
- reencrypted
- replayer
- repodata
- repomd
- repost
- reposts
- rerandomize
- rerandomization
- rerandomized
@@ -306,6 +312,7 @@ words:
- sponsees
- SRPMS
- sslws
- stackful
- statsd
- STATSDCOLLECTOR
- stissue
@@ -326,6 +333,7 @@ words:
- takerpays
- ters
- TMEndpointv2
- TOCTOU
- toolchain
- tparam
- trixie
-4
View File
@@ -8,7 +8,3 @@ watch_file rust-toolchain.toml
watch_dir conan
use flake
# Optional, untracked local overrides. To use a different shell, put e.g.
# `use flake .#formal-verification` in .envrc.local.
source_env_if_exists .envrc.local
-12
View File
@@ -5,18 +5,6 @@
# This file is sorted in reverse chronological order, with the most recent commits at the top.
# The commits listed here are ignored by git blame, which is useful for formatting-only commits that would otherwise obscure the history of changes to a file.
# chore: CamelCase for typedef/using in `clang-tidy` (#8177)
97a1824537d20d82d25bf151a37a7a4532ebf085
# chore: Rename CamelCase namespaces to snake_case (#7933)
06488c1318d96f56d0536251bee08ac85fa7fdd3
# style: Unify style for all Doxygen comments (#7776)
73b6852a122854140336e6e6bc30a3a4b41aa5fd
# style: More clang-tidy identifier renaming (#7290)
a830ab10efed8d3e59ef2fc15d66efdf9c6bb0d8
# refactor: Rename static constants (#7120)
5b6e8b6f93b19c1e3f6a3467a25639031d9d9a53
# chore: More fixes for bad renames (#7092)
7afdd71a54d562b32a50b29a5aa00bb997dc9053
# refactor: Enable clang-tidy `readability-identifier-naming` check (#6571)
8995564ed6b9e453e144bb663303072a3c1ba305
# refactor: Enable remaining clang-tidy `cppcoreguidelines` checks (#6538)
+1 -1
View File
@@ -36,4 +36,4 @@ runs:
- name: Determine release channel and package release
id: release_info
uses: XRPLF/actions/release-info@a9f2eeca6fb3980ba3a84cf68566f1c69ad30674
uses: XRPLF/actions/release-info@ebcf6cea14eee258697308a51fea55cad777581b
@@ -1,40 +0,0 @@
#!/usr/bin/env bash
# Exit the script as soon as an error occurs.
set -euo pipefail
# This script fails if <head> merges a release back into <base>,
# but also adds commits that aren't on a release or staging branch, see RELEASING.md.
# Merge commits are allowed.
# Usage: .github/scripts/releasing/check-merge-back-commits.sh <base> <head>
if [ "$#" -ne 2 ]; then
echo "Usage: $0 <base> <head>"
exit 1
fi
BASE=$1
HEAD=$2
SCRIPT_DIR=$(dirname "${BASH_SOURCE[0]}")
# shellcheck source=.github/scripts/releasing/common.sh
source "${SCRIPT_DIR}/common.sh"
load_release_branches
# A PR is a merge-back if some of its commits are on a release or staging branch.
PR_COUNT=$(git rev-list --no-merges --count "${BASE}..${HEAD}")
NEW_COUNT=$(git rev-list --no-merges --count "${BASE}..${HEAD}" --not "${BRANCHES[@]}")
if ((NEW_COUNT == PR_COUNT)); then
echo "This PR doesn't merge a release back."
exit 0
fi
if ((NEW_COUNT == 0)); then
echo "This merge-back adds no commits of its own."
exit 0
fi
echo "This PR merges a release back, but also adds commits that aren't on a release or staging branch:"
git log --no-merges --format=' %h %s' "${BASE}..${HEAD}" --not "${BRANCHES[@]}"
echo
echo "Make these changes in a separate PR, see RELEASING.md."
exit 1
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#!/bin/bash
# Exit the script as soon as an error occurs.
set -euo pipefail
@@ -21,10 +21,11 @@ patch_ids() {
git log --no-merges --patch --no-color --no-ext-diff "$@" | git patch-id --stable | sort
}
SCRIPT_DIR=$(dirname "${BASH_SOURCE[0]}")
# shellcheck source=.github/scripts/releasing/common.sh
source "${SCRIPT_DIR}/common.sh"
load_release_branches
mapfile -t BRANCHES < <(git for-each-ref --format='%(refname)' 'refs/remotes/*/release/*' 'refs/remotes/*/staging/*')
if [ "${#BRANCHES[@]}" -eq 0 ]; then
echo "Error: No release or staging branches found."
exit 1
fi
# Each line is "<patch-id> <commit>".
RELEASE_PATCHES=$(patch_ids "${BRANCHES[@]}" --not "${HEAD}")
@@ -1,38 +0,0 @@
#!/usr/bin/env bash
# Exit the script as soon as an error occurs.
set -euo pipefail
# This script fails if <head> contains release or staging commits that <base> does not,
# i.e. if <head> merges a release back into <base>.
# Used in the merge queue, which squashes PRs and would drop the merge commit, see RELEASING.md.
# Usage: .github/scripts/releasing/check-no-merge-back.sh <base> <head>
if [ "$#" -ne 2 ]; then
echo "Usage: $0 <base> <head>"
exit 1
fi
BASE=$1
HEAD=$2
SCRIPT_DIR=$(dirname "${BASH_SOURCE[0]}")
# shellcheck source=.github/scripts/releasing/common.sh
source "${SCRIPT_DIR}/common.sh"
load_release_branches
RELEASE_COMMITS=$(git rev-list "${BRANCHES[@]}" --not "${BASE}")
HEAD_COMMITS=$(git rev-list "${BASE}..${HEAD}")
# The release commits in <head>, newest first.
MERGED=$(grep -xF -f <(echo "${RELEASE_COMMITS}") <<<"${HEAD_COMMITS}" || true)
if [ -z "${MERGED}" ]; then
echo "No release commits are merged back."
exit 0
fi
echo "This PR merges $(wc -l <<<"${MERGED}" | tr -d ' ') release commits back, e.g.:"
head -5 <<<"${MERGED}" | xargs git log --no-walk --format=' %h %s'
echo
echo "Merge-backs must not go through the merge queue, which squashes them."
echo "Fast-forward develop to the PR branch instead, see RELEASING.md."
exit 1
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#!/bin/bash
# Exit the script as soon as an error occurs.
set -euo pipefail
-12
View File
@@ -1,12 +0,0 @@
# shellcheck shell=bash
# Helpers shared by the release checks in this directory, see RELEASING.md.
# Sets BRANCHES to all release and staging branches, and fails if there are none.
load_release_branches() {
mapfile -t BRANCHES < <(git for-each-ref --format='%(refname)' 'refs/remotes/*/release/*' 'refs/remotes/*/staging/*')
if [ "${#BRANCHES[@]}" -eq 0 ]; then
echo "Error: No release or staging branches found."
exit 1
fi
}
+1 -1
View File
@@ -36,7 +36,7 @@
"build_type": ["Debug", "Release"],
"arch": ["amd64"],
"minimal": false,
"sanitizers": ["address", "undefinedbehavior"]
"sanitizers": ["address", "undefinedbehavior", "thread"]
},
{
+2 -23
View File
@@ -166,31 +166,10 @@ jobs:
fetch-depth: 0
persist-credentials: false
- name: Check for copied release commits
if: ${{ github.event_name == 'pull_request' }}
env:
BASE: ${{ github.event.pull_request.base.sha }}
HEAD: ${{ github.event.pull_request.head.sha }}
BASE: ${{ github.event.pull_request.base.sha || github.event.merge_group.base_sha }}
HEAD: ${{ github.event.pull_request.head.sha || github.event.merge_group.head_sha }}
run: .github/scripts/releasing/check-no-copied-release-commits.sh "${BASE}" "${HEAD}"
# Runs even if the previous check fails, so that both problems are reported at once.
- name: Check merge-back has no new commits
if: ${{ !cancelled() && github.event_name == 'pull_request' }}
env:
BASE: ${{ github.event.pull_request.base.sha }}
HEAD: ${{ github.event.pull_request.head.sha }}
run: .github/scripts/releasing/check-merge-back-commits.sh "${BASE}" "${HEAD}"
# The queue squashes PRs, so check the PR's own branch, named in the queue branch.
- name: Check the merge queue doesn't merge a release back
if: ${{ github.event_name == 'merge_group' }}
env:
BASE: ${{ github.event.merge_group.base_sha }}
HEAD_REF: ${{ github.event.merge_group.head_ref }}
run: |
if ! [[ "${HEAD_REF}" =~ /pr-([0-9]+)-[0-9a-f]+$ ]]; then
echo "Error: Can't find the PR number in '${HEAD_REF}'."
exit 1
fi
git fetch --no-tags origin "refs/pull/${BASH_REMATCH[1]}/head"
.github/scripts/releasing/check-no-merge-back.sh "${BASE}" FETCH_HEAD
clang-tidy:
needs: should-run
@@ -230,8 +230,11 @@ jobs:
run: |
SUPP="${GITHUB_WORKSPACE}/sanitizers/suppressions"
ASAN_OPTS="include=${SUPP}/runtime-asan-options.txt:suppressions=${SUPP}/asan.supp"
# GCC ASAN's detect_stack_use_after_return produces false positives with
# Boost.Context fiber stack switching (used by boost::asio::spawn).
# See: https://github.com/google/sanitizers/issues/856
if [[ "${CONFIG_NAME}" == *gcc* ]]; then
ASAN_OPTS="${ASAN_OPTS}:alloc_dealloc_mismatch=0"
ASAN_OPTS="${ASAN_OPTS}:alloc_dealloc_mismatch=0:detect_stack_use_after_return=0"
fi
echo "ASAN_OPTIONS=${ASAN_OPTS}" >>${GITHUB_ENV}
echo "TSAN_OPTIONS=include=${SUPP}/runtime-tsan-options.txt:suppressions=${SUPP}/tsan.supp" >>${GITHUB_ENV}
@@ -361,10 +364,15 @@ jobs:
working-directory: ${{ runner.os == 'Windows' && format('{0}/{1}', env.BUILD_DIR, inputs.build_type) || env.BUILD_DIR }}
env:
BUILD_NPROC: ${{ steps.nproc.outputs.nproc }}
SANITIZERS_ON: ${{ env.SANITIZERS_ENABLED }}
run: |
set -o pipefail
# Coverage builds are slower due to instrumentation; use fewer parallel jobs to avoid flakiness
[ "$COVERAGE_ENABLED" = "true" ] && BUILD_NPROC=$((BUILD_NPROC - 2))
# Sanitizer builds are memory hungry; run the tests single-threaded
[ "$SANITIZERS_ON" = "true" ] && BUILD_NPROC=1
# Never drop below one job (small runners could go to 0 or negative above)
[ "$BUILD_NPROC" -ge 1 ] || BUILD_NPROC=1
# The resolver/preload workaround is only correct for the ASan build:
# a regular build doesn't hit the __dn_expand interceptor bug, and must
+3 -12
View File
@@ -269,23 +269,14 @@ jobs:
env:
CONTEXT: image-context
IMAGE: ${{ matrix.target == 'voidstar' && 'xrpld-voidstar' || 'xrplf/xrpld' }}:${{ github.ref_type == 'tag' && github.ref_name || 'develop' }}
# Docker Hub is public, so a private build never reaches it;
# the Antithesis registry is not.
PUSH: ${{ inputs.publish && (matrix.target == 'voidstar' || github.event.repository.visibility == 'public') }}
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Determine release info
id: release_info
uses: ./.github/actions/release-info
# Docker Hub is public, so it only gets builds whose packages are public:
# those of a public codebase, and stable releases.
# The Antithesis registry is private, so it gets every build.
- name: Decide whether to push
env:
PUSH: ${{ inputs.publish && (matrix.target == 'voidstar' || github.event.repository.visibility == 'public' || steps.release_info.outputs.channel == 'stable') }}
run: echo "PUSH=${PUSH}" | tee -a "${GITHUB_ENV}"
- name: Download package artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
-3
View File
@@ -92,9 +92,6 @@ target/
# Direnv's directory
/.direnv
# Direnv's local, per-developer overrides
/.envrc.local
# clangd cache
/.cache
-13
View File
@@ -22,19 +22,6 @@ API version 2 is available in `xrpld` version 2.0.0 and later. See [API-VERSION-
This version is supported by all `xrpld` versions. For WebSocket and HTTP JSON-RPC requests, it is currently the default API version used when no `api_version` is specified.
## XRP Ledger server version 3.5.0
Version 3.5.0 is not yet released.
### Additions in 3.5.0
- `subscribe`, `unsubscribe`: Added an optional `mpt_issuances` request field, an array of MPT issuance IDs (hex strings). Subscribers receive a message with `type` `mptTransaction` for each validated transaction whose metadata affects a subscribed issuance; the message has the same fields as the `transactions` stream. MPT issuance subscriptions count toward the per-connection subscription limit. An empty array, a non-array value, or an invalid ID returns `invalidParams`. ([#5671](https://github.com/XRPLF/rippled/pull/5671))
### Bugfixes in 3.5.0
- `channel_authorize`: The `channel_id` field now returns an `invalidParams` error if the value is not a string. [#7582](https://github.com/XRPLF/rippled/pull/7582)
- `channel_verify`: The `channel_id` and `signature` fields now return an `invalidParams` error if the value is not a string. [#7582](https://github.com/XRPLF/rippled/pull/7582)
## XRP Ledger server version 3.4.0
Version 3.4.0 is not yet released. These changes are available in the 3.4.0 beta releases.
+5 -3
View File
@@ -41,13 +41,15 @@ branch.
git checkout develop
```
For a release or release candidate, check out its [tag](https://github.com/XRPLF/rippled/releases), e.g.:
For a release candidate, choose the relevant release branch, e.g.
`release/3.2.x`.
```bash
git checkout 3.4.0
git checkout release/3.2.x
```
See [RELEASING.md](./RELEASING.md) for how branches and releases are organized.
For a stable release, choose one of the [tagged
releases](https://github.com/XRPLF/rippled/releases).
A build reports `0.0.0-dev` with its short commit hash as build metadata, e.g.
`0.0.0-dev+0123abc`. Only builds of a release, from a tag in CI or a versioned
File diff suppressed because it is too large. Load diff
+142
View File
@@ -0,0 +1,142 @@
# Boost.Coroutine to C++20 Migration — Task List
> Parent document: [BoostToStdCoroutineSwitchPlan.md](BoostToStdCoroutineSwitchPlan.md)
---
## Milestone 1: New Coroutine Primitives
- [ ] **1.1** Design `CoroTask<T>` class with `promise_type`
- Define `promise_type` with `initial_suspend`, `final_suspend`, `unhandled_exception`, `return_value`/`return_void`
- Implement `FinalAwaiter` for continuation support
- Implement move-only RAII handle wrapper
- Support both `CoroTask<T>` and `CoroTask<void>`
- [ ] **1.2** Design and implement `JobQueueAwaiter`
- `await_suspend()` calls `jq_.addJob(type, name, [h]{ h.resume(); })`
- Handle `addJob()` failure (shutdown) — resume with error flag or throw
- Integrate `nSuspend_` counter increment/decrement
- [ ] **1.3** Implement `LocalValues` swap in new coroutine resume path
- Before `handle.resume()`: save thread-local, install coroutine-local
- After `handle.resume()` returns: restore thread-local
- Ensure this works when coroutine migrates between threads
- [ ] **1.4** Add `postCoroTask()` template to `JobQueue`
- Accept callable returning `CoroTask<void>`
- Schedule initial execution on JobQueue (mirror `postCoro()` behavior)
- Return a handle/shared_ptr for join/cancel
- [ ] **1.5** Write unit tests (`src/test/core/CoroTask_test.cpp`)
- Test `CoroTask<void>` runs to completion
- Test `CoroTask<int>` returns value
- Test exception propagation across co_await
- Test coroutine destruction before completion
- Test `JobQueueAwaiter` schedules on correct thread
- Test `LocalValue` isolation across 4+ coroutines
- Test shutdown rejection (addJob returns false)
- Test `correct_order` equivalent (yield → join → post → complete)
- Test `incorrect_order` equivalent (post → yield → complete)
- Test multiple sequential co_await points
- [ ] **1.6** Verify build on GCC 12+, Clang 16+
- [ ] **1.7** Run ASAN + TSAN on new tests
- [ ] **1.8** Run full `--unittest` suite (no regressions)
- [ ] **1.9** Self-review and create PR #1
---
## Milestone 2: Entry Point Migration
- [ ] **2.1** Migrate `ServerHandler::onRequest()` (`ServerHandler.cpp:287`)
- Replace `m_jobQueue.postCoro(jtCLIENT_RPC, ...)` with `postCoroTask()`
- Update lambda to return `CoroTask<void>` (add `co_return`)
- Update `processSession` to accept new coroutine type
- [ ] **2.2** Migrate `ServerHandler::onWSMessage()` (`ServerHandler.cpp:325`)
- Replace `m_jobQueue.postCoro(jtCLIENT_WEBSOCKET, ...)` with `postCoroTask()`
- Update lambda signature
- [ ] **2.3** Migrate `GRPCServer::CallData::process()` (`GRPCServer.cpp:102`)
- Replace `app_.getJobQueue().postCoro(JobType::jtRPC, ...)` with `postCoroTask()`
- Update `process(shared_ptr<Coro> coro)` overload signature
- [ ] **2.4** Update `RPC::Context` (`Context.h:27`)
- Replace `std::shared_ptr<JobQueue::Coro> coro{}` with new coroutine wrapper type
- Ensure all code that accesses `context.coro` compiles
- [ ] **2.5** Update `ServerHandler.h` signatures
- `processSession()` and `processRequest()` parameter types
- [ ] **2.6** Update `GRPCServer.h` signatures
- `process()` method parameter types
- [ ] **2.7** Run full `--unittest` suite
- [ ] **2.8** Manual smoke test: HTTP + WS + gRPC RPC requests
- [ ] **2.9** Run ASAN + TSAN
- [ ] **2.10** Self-review and create PR #2
---
## Milestone 3: Handler Migration
- [ ] **3.1** Migrate `doRipplePathFind()` (`RipplePathFind.cpp`)
- Replace `context.coro->yield()` with `co_await PathFindAwaiter{...}`
- Replace continuation lambda's `coro->post()` / `coro->resume()` with awaiter scheduling
- Handle shutdown case (post failure) in awaiter
- [ ] **3.2** Create `PathFindAwaiter` (or use generic `JobQueueAwaiter`)
- Encapsulate the continuation + yield pattern from `RipplePathFind.cpp` lines 108-132
- [ ] **3.3** Update `Path_test.cpp`
- Replace `postCoro` usage with `postCoroTask`
- Ensure `context.coro` usage matches new type
- [ ] **3.4** Update `AMMTest.cpp`
- Replace `postCoro` usage with `postCoroTask`
- [ ] **3.5** Rewrite `Coroutine_test.cpp` for new API
- `correct_order`: postCoroTask → co_await → join → resume → complete
- `incorrect_order`: post before yield equivalent
- `thread_specific_storage`: 4 coroutines with LocalValue isolation
- [ ] **3.6** Update `JobQueue_test.cpp` `testPostCoro`
- Migrate to `postCoroTask` API
- [ ] **3.7** Verify `ripple_path_find` works end-to-end with new coroutines
- [ ] **3.8** Test shutdown-during-pathfind scenario
- [ ] **3.9** Run full `--unittest` suite
- [ ] **3.10** Run ASAN + TSAN
- [ ] **3.11** Self-review and create PR #3
---
## Milestone 4: Cleanup & Validation
- [ ] **4.1** Delete `include/xrpl/core/Coro.ipp`
- [ ] **4.2** Remove from `JobQueue.h`:
- `#include <boost/coroutine2/all.hpp>`
- `struct Coro_create_t`
- `class Coro` (entire class)
- `postCoro()` template
- Comment block (lines 322-377) describing old race condition
- [ ] **4.3** Update `cmake/deps/Boost.cmake`:
- Remove `coroutine` from `find_package(Boost REQUIRED COMPONENTS ...)`
- Remove `Boost::coroutine` from `target_link_libraries`
- [ ] **4.4** Update `cmake/XrplInterface.cmake`:
- Remove `BOOST_COROUTINES2_NO_DEPRECATION_WARNING`
- [ ] **4.5** Run memory benchmark
- Create N=1000 coroutines, compare RSS: before vs after
- Document results
- [ ] **4.6** Run context switch benchmark
- 100K yield/resume cycles, compare latency: before vs after
- Document results
- [ ] **4.7** Run RPC throughput benchmark
- Concurrent `ripple_path_find` requests, compare throughput
- Document results
- [ ] **4.8** Run full `--unittest` suite
- [ ] **4.9** Run ASAN, TSAN, UBSan
- Confirm `__asan_handle_no_return` warnings are gone
- [ ] **4.10** Verify build on all supported compilers
- [ ] **4.11** Self-review and create PR #4
- [ ] **4.12** Document final benchmark results in PR description
+586 -20
View File
@@ -14,13 +14,9 @@ The following branches exist in the main project repository:
- `develop`: The latest set of unreleased features, and the most common
starting point for contributions.
- `staging/*` (e.g. `staging/3.4.x`): Staging branches, one per release line,
where fixes for that line are developed.
- `release/*` (e.g. `release/3.4.x`): Release branches, one per release line,
holding the latest public release candidate or release for that line.
Releases are published as [tagged releases](https://github.com/XRPLF/rippled/releases).
See [RELEASING.md](./RELEASING.md) for how these branches are used.
- `release/*` (e.g. `release/3.2.x`): Release branches, one per release line,
holding the latest release candidate, or stable release for that line.
Stable releases are published as [tagged releases](https://github.com/XRPLF/rippled/releases).
The tip of each branch must be signed. In order for GitHub to sign a
squashed commit that it builds from your pull request, GitHub must know
@@ -149,8 +145,8 @@ tl;dr
In general, pull requests use `develop` as the base branch.
The exceptions are fixes for an existing release line,
which use that line's staging branch (e.g. `staging/3.4.x`) as the base.
The exceptions are fixes, improvements, and hotfixes for an existing release,
which use that release's branch (e.g. `release/3.2.x`) as the base.
If your changes are not quite ready, but you want to make it easily available
for preliminary examination or review, you can create a "Draft" pull request.
@@ -595,16 +591,15 @@ the suggested commit message, or modify it as needed.
#### Slightly more complicated pull requests
Some pull requests need to be pushed to their base branch (usually `develop`)
as more than one commit.
A PR author may _request_ to merge as separate commits. They
Some pull requests need to be pushed to `develop` as more than one
commit. A PR author may _request_ to merge as separate commits. They
must _justify_ why separate commits are needed, and _specify_ how they
would like the commits to be merged. If you disagree with the author,
discuss it with them directly.
If the process is reasonable, follow it. The simplest option is to do a
fast forward only merge (`--ff-only`) on the command line
and push to the base branch.
fast forward only merge (`--ff-only`) on the command line and push to
`develop`.
Some examples of when separate commits are worthwhile are:
@@ -617,10 +612,9 @@ Some examples of when separate commits are worthwhile are:
Either way, check that:
- The commits are based on the current tip of the base branch.
- The commits are clean:
No merge commits (except when merging a release, see [RELEASING.md](./RELEASING.md)),
no "[FOLD]" or "fixup!" messages.
- The commits are based on the current tip of `develop`.
- The commits are clean: No merge commits (except when reverse
merging), no "[FOLD]" or "fixup!" messages.
- All commits are signed. If the commits are not signed by the author, use
`git commit --amend -S` to sign them yourself.
- At least one (but preferably all) of the commits has the PR number
@@ -632,8 +626,578 @@ use them!**
### Releases
Releases, release branches, and merging releases back into `develop`
are described in [RELEASING.md](./RELEASING.md).
All releases, including release candidates and betas, are handled
differently from typical PRs. Most importantly, never use
the Github UI to merge a release.
Xrpld uses a linear workflow model that can be summarized as:
1. In between releases, developers work against the `develop` branch.
2. Periodically, a maintainer will build and tag a beta version from
`develop`, which is pushed to `release`.
- Betas are usually released every two to three weeks, though that
schedule can vary depending on progress, availability, and other
factors.
3. When the changes in `develop` are considered stable and mature enough
to be ready to release, a release candidate (RC) is built and tagged
from `develop`, and merged to `release`.
- Further development for that release (primarily fixes) then
continues against `release`, while other development continues on
`develop`. Effectively, `release` is forked from `develop`. Changes
to `release` must be reverse merged to `develop`.
4. When the candidate has passed testing and is ready for release, the
final release is merged to `master`.
5. If any issues are found post-release, a hotfix / point release may be
created, which is merged to `master`, and then reverse merged to
`develop`.
#### Betas, and the first release candidate
##### Preparing the `develop` branch
1. Optimally, the `develop` branch will be ready to go, with all
relevant PRs already merged.
2. If there are any PRs pending, merge them **BEFORE** preparing the beta.
1. If only one or two PRs need to be merged, merge those PRs [as
normal](#when-and-how-to-merge-pull-requests), updating the second
one, and waiting for CI to finish in between.
2. If there are several pending PRs, do not use the Github UI,
because the delays waiting for CI in between each merge will be
unnecessarily onerous. (Incidentally, this process can also be
used to merge if the Github UI has issues.) Merge each PR branch
directly to a `release-next` on your local machine and create a single
PR, then push your branch to `develop`.
1. Squash the changes from each PR, one commit each (unless more
are needed), being sure to sign each commit and update the
commit message to include the PR number. You may be able to use
a fast-forward merge for the first PR.
2. Push your branch.
3. Continue to [Making the release](#making-the-release) to update
the version number, etc.
The workflow may look something like:
```
git fetch --multiple upstreams user1 user2 user3 [...]
git checkout -B release-next --no-track upstream/develop
# Only do an ff-only merge if pr-branch1 is either already
# squashed, or needs to be merged with separate commits,
# and has no merge commits.
# Use -S on the ff-only merge if pr-branch1 isn't signed.
git merge [-S] --ff-only user1/pr-branch1
git merge --squash user2/pr-branch2
git commit -S # Use the commit message provided on the PR
git merge --squash user3/pr-branch3
git commit -S # Use the commit message provided on the PR
[...]
# Make sure the commits look right
git log --show-signature "upstream/develop..HEAD"
git push --set-upstream origin
# Continue to "Making the release" to update the version number, so
# everything can be done in one PR.
```
You can also use the [squash-branches] script.
You may also need to manually close the open PRs after the changes are
merged to `develop`. Be sure to include the commit ID.
##### Making the release
This includes, betas, and the first release candidate (RC).
1. If you didn't create one [preparing the `develop`
branch](#preparing-the-develop-branch), Ensure there is no old
`release-next` branch hanging around. Then make a `release-next`
branch that only changes the version number. e.g.
```
git fetch upstreams
git checkout --no-track -B release-next upstream/develop
v="A.B.C-bD"
build=$( find -name BuildInfo.cpp )
sed 's/\(^.*versionString =\).*$/\1 "'${v}'"/' ${build} > version.cpp && mv -vi version.cpp ${build}
git diff
git add ${build}
git commit -S -m "Set version to ${v}"
# You could use your "origin" repo, but some CI tests work better on upstream.
git push upstream-push
git fetch upstreams
git branch --set-upstream-to=upstream/release-next
```
You can also use the [update-version] script. 2. Create a Pull Request for `release-next` with **`develop`** as
the base branch.
1. Use the title "[TRIVIAL] Set version to X.X.X-bX".
2. Instead of the default description template, use the following:
```
## High Level Overview of Change
This PR only changes the version number. It will be merged as
soon as Github CI actions successfully complete.
```
3. Wait for CI to successfully complete, and get someone to approve
the PR. (It is safe to ignore known CI issues.)
4. Push the updated `develop` branch using your `release-next`
branch. **Do not use the Github UI. It's important to preserve
commit IDs.**
```
git push upstream-push release-next:develop
```
5. In the unlikely event that the push fails because someone has merged
something else in the meantime, rebase your branch onto the updated
`develop` branch, push again, and go back to step 3.
6. Ensure that your PR against `develop` is closed. Github should do it
automatically.
7. Once this is done, forward progress on `develop` can continue
(other PRs may be merged).
8. Now create a Pull Request for `release-next` with **`release`** as
the base branch. Instead of the default template, reuse and update
the message from the previous release. Include the following verbiage
somewhere in the description:
```
The base branch is `release`. [All releases (including
betas)](https://github.com/XRPLF/rippled/blob/develop/CONTRIBUTING.md#before-you-start)
go in `release`. This PR branch will be pushed directly to `release` (not
squashed or rebased, and not using the GitHub UI).
```
7. Sign-offs for the three platforms (Linux, Mac, Windows) usually occur
offline, but at least one approval will be needed on the PR.
- If issues are discovered during testing, simply abandon the
release. It's easy to start a new release, it should be easy to
abandon one. **DO NOT REUSE THE VERSION NUMBER.** e.g. If you
abandon 2.4.0-b1, the next attempt will be 2.4.0-b2.
8. Once everything is ready to go, push to `release`.
```
git fetch upstreams
# Just to be safe, do a dry run first:
git push --dry-run upstream-push release-next:release
# If everything looks right, push the branch
git push upstream-push release-next:release
# Check that all of the branches are updated
git fetch upstreams
git log -1 --oneline
# The output should look like:
# 0123456789 (HEAD -> upstream/release-next, upstream/release,
# upstream/develop) Set version to 2.4.0-b1
# Note that upstream/develop may not be on this commit, but
# upstream/release must be.
# Other branches, including some from upstream-push, may also be
# present.
```
9. Tag the release, too.
```
git tag <version number>
git push upstream-push <version number>
```
10. Delete the `release-next` branch on the repo. Use the Github UI or:
```
git push --delete upstream-push release-next
```
11. Finally [create a new release on
Github](https://github.com/XRPLF/rippled/releases).
#### Release candidates after the first
Once the first release candidate is [merged into
release](#making-the-release), then `release` and `develop` _are allowed
to diverge_.
If a bug or issue is discovered in a version that has a release
candidate being tested, any fix and new version will need to be applied
against `release`, then reverse-merged to `develop`. This helps keep git
history as linear as possible.
A `release-next` branch will be created from `release`, and any further
work for that release must be based on `release-next`. Specifically,
PRs must use `release-next` as the base, and those PRs will be merged
directly to `release-next` when approved. Changes should be restricted
to bug fixes, but other changes may be necessary from time to time.
1. Open any PRs for the pending release using `release-next` as the base,
so they can be merged directly in to it. Unlike `develop`, though,
`release-next` can be thrown away and recreated if necessary.
2. Once a new release candidate is ready, create a version commit as in
step 1 [above](#making-the-release) on `release-next`. You can use
the [update-version] script for this, too.
3. Jump to step 8 ("Now create a Pull Request for `release-next` with
**`release`** as the base") from the process
[above](#making-the-release) to merge `release-next` into `release`.
##### Follow up: reverse merge
Once the RC is merged and tagged, it needs to be reverse merged into
`develop` as soon as possible.
1. Create a branch, based on `upstream/develop`.
The branch name is not important, but could include "mergeNNNrcN".
E.g. For release A.B.C-rcD, use `mergeABCrcD`.
```
git fetch upstreams
git checkout --no-track -b mergeABCrcD upstream/develop
```
2. Merge `release` into your branch.
```
# I like the "--edit --log --verbose" parameters, but they are
# not required.
git merge upstream/release
```
3. `BuildInfo.cpp` will have a conflict with the version number.
Resolve it with the version from `develop` - the higher version.
4. Push your branch to your repo (or `upstream` if you have permission),
and open a normal PR against `develop`. The "High level overview" can
simply indicate that this is a merge of the RC. The "Context" should
summarize the changes from the RC. Include the following text
prominently:
```
This PR must be merged manually using a push. Do not use the Github UI.
```
5. Depending on the complexity of the changes, and/or merge conflicts,
the PR may need a thorough review, or just a sign-off that the
merge was done correctly.
6. If `develop` is updated before this PR is merged, do not merge
`develop` back into your branch. Instead rebase preserving merges,
or do the merge again. (See also the `rerere` git config setting.)
```
git rebase --rebase-merges upstream/develop
# OR
git reset --hard upstream/develop
git merge upstream/release
```
7. When the PR is ready, push it to `develop`.
```
git fetch upstreams
# Make sure the commits look right
git log --show-signature "upstream/develop^..HEAD"
git push upstream-push mergeABCrcD:develop
git fetch upstreams
```
Development on `develop` can proceed as normal.
#### Final releases
A final release is any release that is not a beta or RC, such as 2.2.0.
Only code that has already been tested and vetted across all three
platforms should be included in a final release. Most of the time, that
means that the commit immediately preceding the commit setting the
version number will be an RC. Occasionally, there may be last-minute bug
fixes included as well. If so, those bug fixes must have been tested
internally as if they were RCs (at minimum, ensuring unit tests pass,
and the app starts, syncs, and stops cleanly across all three
platforms.)
_If in doubt, make an RC first._
The process for building a final release is very similar to [the process
for building a beta](#making-the-release), except the code will be
moving from `release` to `master` instead of from `develop` to
`release`, and both branches will be pushed at the same time.
1. Ensure there is no old `master-next` branch hanging around.
Then make a `master-next` branch that only changes the version
number. As above, or using the
[update-version] script.
2. Create a Pull Request for `master-next` with **`master`** as
the base branch. Instead of the default template, reuse and update
the message from the previous final release. Include the following verbiage
somewhere in the description:
```
The base branch is `master`. This PR branch will be pushed directly to
`release` and `master` (not squashed or rebased, and not using the
GitHub UI).
```
7. Sign-offs for the three platforms (Linux, Mac, Windows) usually occur
offline, but at least one approval will be needed on the PR.
- If issues are discovered during testing, close the PR, delete
`master-next`, and move development back to `release`, [issuing
more RCs as necessary](#release-candidates-after-the-first)
8. Once everything is ready to go, push to `release` and `master`.
```
git fetch upstreams
# Just to be safe, do dry runs first:
git push --dry-run upstream-push master-next:release
git push --dry-run upstream-push master-next:master
# If everything looks right, push the branch
git push upstream-push master-next:release
git push upstream-push master-next:master
# Check that all of the branches are updated
git fetch upstreams
git log -1 --oneline
# The output should look like:
# 0123456789 (HEAD -> upstream/master-next, upstream/master,
# upstream/release) Set version to A.B.0
# Note that both upstream/release and upstream/master must be on this
# commit.
# Other branches, including some from upstream-push, may also be
# present.
```
9. Tag the release, too.
```
git tag <version number>
git push upstream-push <version number>
```
10. Delete the `master-next` branch on the repo. Use the Github UI or:
```
git push --delete upstream-push master-next
```
11. [Create a new release on
Github](https://github.com/XRPLF/rippled/releases). Be sure that
"Set as the latest release" is checked.
12. Open a PR to update the [API-CHANGELOG](API-CHANGELOG.md) and `API-VERSION-[n].md` with the changes for this release (if any are missing).
13. Finally, [reverse merge the release into `develop`](#follow-up-reverse-merge).
#### Special cases: point releases, hotfixes, etc.
On occasion, a bug or issue is discovered in a version that already
had a final release. Most of the time, development will have started
on the next version, and will usually have changes in `develop`
and often in `release`.
Because git history is kept as linear as possible, any fix and new
version will need to be applied against `master`.
The process for building a hotfix release is very similar to [the
process for building release candidates after the
first](#release-candidates-after-the-first) and [for building a final
release](#final-releases), except the changes will be done against
`master` instead of `release`.
If there is only a single issue for the hotfix, the work can be done in
any branch. When it's ready to merge, jump to step 3 using your branch
instead of `master-next`.
1. Create a `master-next` branch from `master`.
```
git checkout --no-track -b master-next upstream/master
git push upstream-push
git fetch upstreams
```
2. Open any PRs for the pending hotfix using `master-next` as the base,
so they can be merged directly in to it. Unlike `develop`, though,
`master-next` can be thrown away and recreated if necessary.
3. Once the hotfix is ready, create a version commit using the same
steps as above, or use the
[update-version] script.
4. Create a Pull Request for `master-next` with **`master`** as
the base branch. Instead of the default template, reuse and update
the message from the previous final release. Include the following verbiage
somewhere in the description:
```
The base branch is `master`. This PR branch will be pushed directly to
`master` (not squashed or rebased, and not using the GitHub UI).
```
7. Sign-offs for the three platforms (Linux, Mac, Windows) usually occur
offline, but at least one approval will be needed on the PR.
- If issues are discovered during testing, update `master-next` as
needed, but ensure that the changes are properly squashed, and the
version setting commit remains last
8. Once everything is ready to go, push to `master` **only**.
```
git fetch upstreams
# Just to be safe, do a dry run first:
git push --dry-run upstream-push master-next:master
# If everything looks right, push the branch
git push upstream-push master-next:master
# Check that all of the branches are updated
git fetch upstreams
git log -1 --oneline
# The output should look like:
# 0123456789 (HEAD -> upstream/master-next, upstream/master) Set version
# to 2.4.1
# Note that upstream/master must be on this commit. upstream/release and
# upstream/develop should not.
# Other branches, including some from upstream-push, may also be
# present.
```
9. Tag the release, too.
```
git tag <version number>
git push upstream-push <version number>
```
9. Delete the `master-next` branch on the repo.
```
git push --delete upstream-push master-next
```
10. [Create a new release on
Github](https://github.com/XRPLF/rippled/releases). Be sure that
"Set as the latest release" is checked.
Once the hotfix is released, it needs to be reverse merged into
`develop` as soon as possible. It may also need to be merged into
`release` if a release candidate is under development.
1. Create a branch in your own repo, based on `upstream/develop`.
The branch name is not important, but could include "mergeNNN".
E.g. For release 2.2.3, use `merge223`.
```
git fetch upstreams
git checkout --no-track -b merge223 upstream/develop
```
2. Merge master into your branch.
```
# I like the "--edit --log --verbose" parameters, but they are
# not required.
git merge upstream/master
```
3. `BuildInfo.cpp` will have a conflict with the version number.
Resolve it with the version from `develop` - the higher version.
4. Push your branch to your repo, and open a normal PR against
`develop`. The "High level overview" can simply indicate that this
is a merge of the hotfix version. The "Context" should summarize
the changes from the hotfix. Include the following text
prominently:
```
This PR must be merged manually using a --ff-only merge. Do not use the Github UI.
```
5. Depending on the complexity of the hotfix, and/or merge conflicts,
the PR may need a thorough review, or just a sign-off that the
merge was done correctly.
6. If `develop` is updated before this PR is merged, do not merge
`develop` back into your branch. Instead rebase preserving merges,
or do the merge again. (See also the `rerere` git config setting.)
```
git rebase --rebase-merges upstream/develop
# OR
git reset --hard upstream/develop
git merge upstream/master
```
7. When the PR is ready, push it to `develop`.
```
git fetch upstreams
# Make sure the commits look right
git log --show-signature "upstream/develop..HEAD"
git push upstream-push HEAD:develop
```
Development on `develop` can proceed as normal. It is recommended to
create a beta (or RC) immediately to ensure that everything worked as
expected.
##### An even rarer scenario: A hotfix on an old release
Historically, once a final release is tagged and packages are released,
versions older than the latest final release are no longer supported.
However, there is a possibility that a very high severity bug may occur
in a non-amendment blocked version that is still being run by
a significant fraction of users, which would necessitate a hotfix / point
release to that version as well as any later versions.
This scenario would follow the same basic procedure as above,
except that _none_ of `develop`, `release`, or `master`
would be touched during the release process.
In this example, consider if version 2.1.1 needed to be patched.
1. Create two branches in the main (`upstream`) repo.
```
git fetch upstreams
# Create a base branch off the tag
git checkout --no-track -b master-2.1.2 2.1.1
git push upstream-push
# Create a working branch
git checkout --no-track -b master212-next master-2.1.2
git push upstream-push
git fetch upstreams
```
2. Work continues as above, except using `master-2.1.2`as
the base branch for any merging, packaging, etc.
3. After the release is tagged and packages are built, you could
potentially delete both branches, e.g. `master-2.1.2` and
`master212-next`. However, it may be useful to keep `master-2.1.2`
around indefinitely for reference.
4. Assuming that a hotfix is also released for the latest
version in parallel with this one, or if the issue is
already fixed in the latest version, do no do any
reverse merges. However, if it is not, it probably makes
sense to reverse merge `master-2.1.2` into `master`,
release a hotfix for _that_ version, then reverse merge
from `master` to `develop`. (Please don't do this unless absolutely
necessary.)
[contrib]: https://docs.github.com/en/get-started/quickstart/contributing-to-projects
[squash]: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/incorporating-changes-from-a-pull-request/about-pull-request-merges#squash-and-merge-your-commits
@@ -641,3 +1205,5 @@ are described in [RELEASING.md](./RELEASING.md).
[xrpld]: https://github.com/XRPLF/rippled
[signing]: https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification
[setup-upstreams]: ./bin/git/setup-upstreams.sh
[squash-branches]: ./bin/git/squash-branches.sh
[update-version]: ./bin/git/update-version.sh
-152
View File
@@ -1,152 +0,0 @@
# Branching and Release Management
This document describes how we branch, release, and merge releases back into `develop`.
It does not define version naming
(e.g., what constitutes a major-minor, patch, or beta release).
Examples use `X.Y` for a release line:
`X` and `Y` are placeholders, while the trailing `x` is literal,
e.g., `release/X.Y.x` is `release/3.4.x` for the `3.4` line.
## Branches
| Branch | Purpose |
| :-------------- | :---------------------------------------------------------------------------------------- |
| `develop` | Main development branch. Betas and the first RC of a major-minor release are tagged here. |
| `staging/X.Y.x` | Where fixes for the `X.Y` line are developed and RCs are prepared. |
| `release/X.Y.x` | The last public RC or release of the `X.Y` line. |
A release line is named `X.Y.x`
because one branch serves every patch release of that line (`X.Y.0`, `X.Y.1`, `X.Y.2`, ...).
The `/` groups branches hierarchically,
so tools can filter them and protection rules can target `release/*` and `staging/*`.
## Principles
- **Releases are merged back into `develop`, never cherry-picked or rebased onto it.**
Cherry-picked and rebased commits get new hashes,
so Git can't tell that `develop` already has them.
Future merges then replay them and produce artificial conflicts,
and it's hard to verify that every fix actually reached `develop`.
Merging keeps a single history:
Git knows exactly which release commits `develop` contains,
and no fix is left behind.
- **Branches only move forward.**
`develop`, `staging/X.Y.x`, and `release/X.Y.x` are never rewritten.
- **Cherry-picking only goes from `develop` to a staging branch**,
for fixes that must get into a release after the code freeze
(see [Emergency Fixes From `develop`](#emergency-fixes-from-develop)).
- **Security fixes are prepared privately and published with the release that contains them**,
so vulnerabilities are not disclosed prematurely.
## Release Lifecycle
This diagram shows a major-minor release and its first patch release.
The staging and release branches are drawn as one line.
```text
develop staging/X.Y.x & release/X.Y.x
│
├── Tag: X.Y.0-b1
├── Tag: X.Y.0-bN
├── Tag: X.Y.0-rc1 ───────────────┐ (branches created)
│ │
│ (development continues) ├── Fixes
│ ├── Tag: X.Y.0-rcN
│ ├── Tag: X.Y.0 (final)
◀──── (merge) ────────────────────┤
│ ├── Fixes
│ ├── Tag: X.Y.1-rcN
│ ├── Tag: X.Y.1 (final)
◀──── (merge) ────────────────────┤
│
▼
```
### Betas, First RC & Branching
> [!NOTE]
> This phase applies only to a new major-minor release (e.g., `X.Y.0`).
> Patch releases work on the existing branches of the line.
1. **Betas:** All beta versions (e.g., `X.Y.0-b1`) are built and tagged directly on `develop`.
2. **First RC:** We release the first RC (`X.Y.0-rc1`)
once everything that should be included in the release has been merged.
3. **Branches:** `staging/X.Y.x` and `release/X.Y.x` are created from `develop`
at the commit tagged `X.Y.0-rc1`.
The first RC also kicks off the QE process.
4. **Code freeze:** No new features or unrelated changes are pulled from `develop`
into `staging/X.Y.x` or `release/X.Y.x`.
Only critical stabilization fixes go into the line.
5. **No large changes on `develop`:** Until `X.Y.0` is [merged back](#merging-back-into-develop),
large changes (e.g., big refactors, moving or renaming many files, mass reformatting)
are not merged into `develop`,
so that fixes on the line and the merge back don't run into conflicts.
### Release Candidates
1. Fixes are developed against `staging/X.Y.x`.
2. When ready, a new RC is created on `staging/X.Y.x`,
and `release/X.Y.x` is fast-forwarded to it.
RCs that contain unpublished security fixes are not published,
and don't touch the public branches.
RCs are not merged back into `develop`:
`X.Y.0-rc1` is tagged on `develop` itself,
and all later changes reach `develop` with the [final release](#final-release).
### Final Release
Security fixes become public as soon as they reach the public repo,
so these steps happen only once the release is ready to be published,
one right after the other.
1. Unpublished security fixes, if any, are merged into `staging/X.Y.x`.
2. `release/X.Y.x` is fast-forwarded to `staging/X.Y.x`,
and the release is tagged on it.
3. The release is immediately [merged back into `develop`](#merging-back-into-develop).
For a major-minor release, this lifts the freeze on large changes in `develop`.
### Merging Back Into `develop`
The merge back is a regular PR into `develop`
whose branch contains a real merge commit of the release tag:
1. Create a branch from `develop`, run `git merge --no-ff <tag>`, and resolve any conflicts.
2. Once the PR is approved, `develop` is fast-forwarded to the PR branch,
so the merge commit lands as it is.
Never squash or rebase it.
Never add it to the merge queue either: the queue squashes PRs, which would drop the merge commit.
3. If `develop` moves while the PR is open, redo the merge on top of the new `develop`.
After the merge, `git log develop..<tag>` must be empty.
## Special Cases
### Emergency Fixes From `develop`
If a commit was merged to `develop`
and needs to be included in a release after the code freeze:
1. Create a PR that cherry-picks the commit onto `staging/X.Y.x`.
2. Leave `develop` as it is, with no reverts.
3. Follow the [release candidates](#release-candidates) process as usual.
When the release is later merged back into `develop`,
both sides already contain the same change,
so the merge usually resolves it cleanly or with a trivial conflict.
From Git's perspective, the cherry-picked commit then becomes part of `develop` too.
### Several Supported Lines
When a fix must ship in more than one supported line (e.g., `X.Y` and `X.(Y+1)`),
the lines are merged upwards rather than cherry-picked between:
1. The fix goes into the oldest line first.
2. After that line's release,
its `release/X.Y.x` is merged into the staging branch of the next newer line,
and so on up to the newest line.
3. The newest line is merged back into `develop` as usual.
This way every newer line, and eventually `develop`, contains the history of the older lines.
-12
View File
@@ -24,13 +24,9 @@
# `g++-15`, ...) are probed under both names: a suffixed name can break while
# the plain one still works (see mkVersionedToolLinks in nix/packages.nix).
#
# Tools scoped to a single dev shell rather than to commonPackages are checked
# only in that shell, keyed off XRPL_DEVSHELL.
#
# Environment variables:
# CI if set, skip the tools above when on macOS.
# CHECK_TOOLS_SKIP_CLONE if set, skip the git-over-HTTPS connectivity check.
# XRPL_DEVSHELL active dev shell; selects shell-specific tools.
set -uo pipefail
@@ -167,14 +163,6 @@ if [ "${os}" = "linux" ] || [ "${os}" = "macos" ]; then
check rustfmt
fi
# Lean4 is in the formal-verification shell only, not in commonPackages.
if [ "${XRPL_DEVSHELL:-}" = "formal-verification" ]; then
echo
echo "Formal verification toolchain:"
check lean
check lake
fi
# GCC is the default compiler on Linux. macOS uses the system Apple Clang
# instead, so GCC/g++/gcov are not expected there.
if [ "${os}" = "linux" ]; then
+11 -15
View File
@@ -39,25 +39,21 @@ if(Boost_COMPILER)
target_link_libraries(xrpl_boost INTERFACE Boost::disable_autolinking)
endif()
# GCC 14+ has a false positive -Wuninitialized warning in Boost.Coroutine2's
# state.hpp when compiled with -O3. This is due to GCC's intentional behavior
# change (Bug #98871, #119388) where warnings from inlined system header code
# are no longer suppressed by -isystem. The warning occurs in operator|= in
# boost/coroutine2/detail/state.hpp when inlined from push_control_block::destroy().
# See: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119388
if(is_gcc AND CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 14)
target_compile_options(xrpl_boost INTERFACE -Wno-uninitialized)
endif()
# Boost.Context's ucontext backend has ASAN fiber-switching annotations
# (start/finish_switch_fiber) that are compiled in when BOOST_USE_ASAN is defined.
# This tells ASAN about coroutine stack switches, preventing false positive
# stack-use-after-scope errors. BOOST_USE_UCONTEXT ensures the ucontext backend
# is selected (fcontext does not support ASAN annotations).
# Boost.Context's ucontext backend has sanitizer fiber-switching annotations
# that are compiled in when BOOST_USE_ASAN/BOOST_USE_TSAN is defined.
# This tells the sanitizer about fiber stack switches used by boost::asio::spawn,
# preventing false positive errors.
# BOOST_USE_UCONTEXT ensures the ucontext backend is selected (fcontext does
# not support sanitizer annotations).
# These defines must match what Boost was compiled with (see conan/profiles/sanitizers).
if(enable_asan)
target_compile_definitions(
xrpl_boost
INTERFACE BOOST_USE_ASAN BOOST_USE_UCONTEXT
)
elseif(enable_tsan)
target_compile_definitions(
xrpl_boost
INTERFACE BOOST_USE_TSAN BOOST_USE_UCONTEXT
)
endif()
@@ -31,7 +31,7 @@ namespace xrpl::ledger_entries {
// builder's STObject and the wrapper's SLE.
TEST(${name}Tests, BuilderSettersRoundTrip)
{
constexpr UInt256 index{1};
UInt256 const index{1u};
% for field in fields:
auto const ${field["paramName"]}Value = ${canonical_expr(field)};
@@ -85,7 +85,7 @@ TEST(${name}Tests, BuilderSettersRoundTrip)
// from that SLE, build a new wrapper, and verify all fields (and validate()).
TEST(${name}Tests, BuilderFromSleRoundTrip)
{
constexpr UInt256 index{2};
UInt256 const index{2u};
% for field in fields:
auto const ${field["paramName"]}Value = ${canonical_expr(field)};
@@ -146,7 +146,7 @@ TEST(${name}Tests, BuilderFromSleRoundTrip)
// 3) Verify wrapper throws when constructed from wrong ledger entry type.
TEST(${name}Tests, WrapperThrowsOnWrongEntryType)
{
constexpr UInt256 index{3};
UInt256 const index{3u};
// Build a valid ledger entry of a different type
// Ticket requires: Account, OwnerNode, TicketSequence, PreviousTxnID, PreviousTxnLgrSeq
@@ -177,7 +177,7 @@ TEST(${name}Tests, WrapperThrowsOnWrongEntryType)
// 4) Verify builder throws when constructed from wrong ledger entry type.
TEST(${name}Tests, BuilderThrowsOnWrongEntryType)
{
constexpr UInt256 index{4};
UInt256 const index{4u};
// Build a valid ledger entry of a different type
% if wrong_le_include == "Ticket":
@@ -207,7 +207,7 @@ TEST(${name}Tests, BuilderThrowsOnWrongEntryType)
// 5) Build with only required fields and verify optional fields return nullopt.
TEST(${name}Tests, OptionalFieldsReturnNullopt)
{
constexpr UInt256 index{3};
UInt256 const index{3u};
% for field in required_fields:
auto const ${field["paramName"]}Value = ${canonical_expr(field)};
+1 -1
View File
@@ -67,7 +67,7 @@ class Xrpl(ConanFile):
"boost/*:without_cobalt": True,
"boost/*:without_context": False,
"boost/*:without_coroutine": True,
"boost/*:without_coroutine2": False,
"boost/*:without_coroutine2": True,
"date/*:header_only": True,
"ed25519/*:shared": False,
"grpc/*:shared": False,
+5
View File
@@ -97,6 +97,11 @@ export TSAN_OPTIONS="include=sanitizers/suppressions/runtime-tsan-options.txt:su
./xrpld --unittest --unittest-jobs=5
```
`runtime-tsan-options.txt` sets `halt_on_error=true`, so the process aborts on
the first unsuppressed report. To collect all reports in one run instead (e.g.
when triaging locally), append `:halt_on_error=false` to `TSAN_OPTIONS`; the
exit code is still non-zero (66) if any report was produced.
More details [here](https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual).
### LeakSanitizer (LSan)
Generated
+6 -6
View File
@@ -2,11 +2,11 @@
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1791130267,
"narHash": "sha256-1sjwcQMcgAbmiip/VivBSiK8p2bTkQGOvCuO7vnwfx4=",
"lastModified": 1781173989,
"narHash": "sha256-fnzKKPvS+oieI/pTzotA5tkoM47EB1NpaBcgk4R97hE=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "9013764fcc0ea99fa16cf7aa7decf8a5c3889dd2",
"rev": "8c91a71d13451abc40eb9dae8910f972f979852f",
"type": "github"
},
"original": {
@@ -47,11 +47,11 @@
]
},
"locked": {
"lastModified": 1791189933,
"narHash": "sha256-Y8eN7ki0Urx4Ojf4w1xSpWg8uk/3lJqK+E0Oth77rH0=",
"lastModified": 1784611586,
"narHash": "sha256-OfqgY+0hp/zseZB7uyH0U8kIDPS4scZZCyAurEplvG0=",
"owner": "oxalica",
"repo": "rust-overlay",
"rev": "e60029353d0c48d216bc4b065168ccd4079c166f",
"rev": "14f58845249f3552a89b07772626b8d3c632fa86",
"type": "github"
},
"original": {
+56 -147
View File
@@ -1,31 +1,29 @@
#pragma once
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/beast/type_name.h>
#include <boost/core/type_name.hpp>
#include <algorithm>
#include <atomic>
#include <cstddef>
#include <cstdint>
#include <iterator>
#include <string>
#include <type_traits>
#include <utility>
#include <vector>
namespace xrpl {
/**
* Manages all counted object types.
*
* Counters register themselves on a lock-free intrusive list maintained
* by this object when constructed. Because counters are never destroyed
* or removed, the ABA problem does not apply.
*
* The registry is iterable as a forward range.
*/
class CountedObjects
{
public:
static CountedObjects&
getInstance() noexcept;
using Entry = std::pair<std::string, int>;
using List = std::vector<Entry>;
[[nodiscard]] List
getCounts(int minimumThreshold) const;
public:
/**
* Implementation for @ref CountedObject.
@@ -35,147 +33,68 @@ public:
class Counter
{
public:
Counter(std::string name) noexcept;
Counter(std::string name) noexcept : name_(std::move(name)), count_(0)
{
// Insert ourselves at the front of the lock-free linked list
CountedObjects& instance = CountedObjects::getInstance();
Counter* head = nullptr;
// Counters are intrusive list nodes whose addresses are published
// in the registry; they must never be copied or moved. The atomic
// members already force this, but we make it explicit.
Counter(Counter const&) = delete;
Counter&
operator=(Counter const&) = delete;
Counter(Counter&&) = delete;
Counter&
operator=(Counter&&) = delete;
do
{
head = instance.head_.load();
next_ = head;
} while (instance.head_.exchange(this) != head);
std::uint32_t
++instance.count_;
}
~Counter() noexcept = default;
int
increment() noexcept
{
auto const newCount = count_.fetch_add(1, std::memory_order::relaxed) + 1;
XRPL_ASSERT(newCount != 0, "xrpl::CountedObjects::Counter::increment : no overflow");
auto maxCount = maxCount_.load(std::memory_order::relaxed);
while (newCount > maxCount &&
!maxCount_.compare_exchange_weak(maxCount, newCount, std::memory_order::relaxed))
{
}
return newCount;
return ++count_;
}
std::uint32_t
int
decrement() noexcept
{
auto const prev = count_.fetch_sub(1, std::memory_order::relaxed);
XRPL_ASSERT(prev != 0, "xrpl::CountedObjects::Counter::decrement : no underflow");
return prev - 1;
return --count_;
}
[[nodiscard]] std::uint32_t
count() const noexcept
[[nodiscard]] int
getCount() const noexcept
{
return count_.load(std::memory_order::relaxed);
return count_.load();
}
[[nodiscard]] std::uint32_t
max() const noexcept
[[nodiscard]] Counter*
getNext() const noexcept
{
return std::max(
count_.load(std::memory_order::relaxed),
maxCount_.load(std::memory_order::relaxed));
return next_;
}
[[nodiscard]] std::string const&
name() const noexcept
getName() const noexcept
{
return name_;
}
private:
friend class CountedObjects;
Counter* next_ = nullptr;
std::atomic<std::uint32_t> count_ = 0;
std::atomic<std::uint32_t> maxCount_ = 0;
std::string const name_;
std::atomic<int> count_;
Counter* next_;
};
class Iterator
{
public:
using value_type = Counter const;
using reference = value_type&;
using pointer = value_type*;
using difference_type = std::ptrdiff_t;
using iterator_category = std::forward_iterator_tag;
explicit Iterator(Counter* c = nullptr) noexcept : current_(c)
{
}
reference
operator*() const noexcept
{
return *current_;
}
pointer
operator->() const noexcept
{
return current_;
}
Iterator&
operator++() noexcept
{
current_ = current_->next_;
return *this;
}
Iterator
operator++(int) noexcept
{
auto tmp = *this;
++*this;
return tmp;
}
bool
operator==(Iterator const&) const noexcept = default;
private:
Counter* current_;
};
constexpr CountedObjects() noexcept = default;
[[nodiscard]] auto
begin() const noexcept
{
return Iterator{head_.load(std::memory_order::acquire)};
}
[[nodiscard]] auto
end() const noexcept
{
return Iterator{};
}
private:
std::atomic<Counter*> head_ = nullptr;
CountedObjects() noexcept;
~CountedObjects() noexcept = default;
private:
std::atomic<int> count_;
std::atomic<Counter*> head_;
};
/** The global counted object registry. */
inline constinit CountedObjects gCountedObjects;
inline CountedObjects::Counter::Counter(std::string name) noexcept
: next_(gCountedObjects.head_.load(std::memory_order::relaxed)), name_(std::move(name))
{
while (!gCountedObjects.head_.compare_exchange_weak(
next_, this, std::memory_order::release, std::memory_order::relaxed))
;
}
//------------------------------------------------------------------------------
/**
@@ -184,37 +103,27 @@ inline CountedObjects::Counter::Counter(std::string name) noexcept
* Derived classes have their instances counted automatically. This is used
* for reporting purposes.
*
* The constructors are private and `Object` is befriended so that the
* CRTP parameter must be the deriving class itself: a copy-paste error
* like `class B : public CountedObject<A>` fails to compile instead of
* silently polluting A's count.
*
* @note This class has no move operations by design: a derived class's
* move constructor falls back to the copy constructor for this
* base, so the newly created instance is counted. This keeps the
* invariant that count is the number of outstanding subobjects.
*
* @warning Counted objects constructed during dynamic initialization of
* other translation units may have their increments discarded when
* counter itself is dynamically initialized. Do not create counted
* objects before main() begins.
*
* @ingroup basics
*/
template <class Object>
requires std::is_class_v<Object>
class CountedObject
{
static inline CountedObjects::Counter counter{boost::core::type_name<Object>()};
private:
static auto&
getCounter() noexcept
{
static CountedObjects::Counter kC{beast::typeName<Object>()};
return kC;
}
CountedObject() noexcept
{
counter.increment();
getCounter().increment();
}
CountedObject(CountedObject const&) noexcept
{
counter.increment();
getCounter().increment();
}
CountedObject&
@@ -223,7 +132,7 @@ class CountedObject
public:
~CountedObject() noexcept
{
counter.decrement();
getCounter().decrement();
}
friend Object;
+12 -12
View File
@@ -110,10 +110,10 @@ static_assert(
*
* However, it does not have sufficient precision to represent the full integer
* range of int64_t values (-2^63 to 2^63-1), which are needed for XRP and MPT
* values. The implementation of SingleAssetVault, LendingProtocol, and
* MPTokensV2 need to represent those integer values accurately and precisely,
* both for the STNumber field type, and for internal calculations. That
* necessitated the "large" scale.
* values. The implementation of SingleAssetVault, and LendingProtocol need to
* represent those integer values accurately and precisely, both for the
* STNumber field type, and for internal calculations. That necessitated the
* "large" scale.
*
* The "Large" scales are intended to represent all values that can be represented
* by an STAmount - IOUs, XRP, and MPTs. It has a min value of 10^18, and a max
@@ -134,8 +134,8 @@ struct MantissaRange final
// NOLINTBEGIN(readability-enum-initial-value)
// The values don't matter, except for Large
enum class MantissaScale {
// Small can be removed when any of featureSingleAssetVault, featureLendingProtocol, or
// featureMPTokensV2 are retired
// Small can be removed when either featureSingleAssetVault or featureLendingProtocol are
// retired
Small,
// LargeLegacy can be removed when fixCleanup3_2_0 is retired
LargeLegacy,
@@ -311,10 +311,10 @@ concept Integral64 = std::is_same_v<T, std::int64_t> || std::is_same_v<T, std::u
*
* The mantissa range may be changed at runtime via setMantissaScale(). The
* default mantissa range is "large". The range is updated whenever transaction
* processing begins, based on whether SingleAssetVault, LendingProtocol, or
* MPTokensV2 are enabled. If any is enabled, the mantissa range is set to
* "large". If not, it is set to "small", preserving backward compatibility and
* correct "amendment-gating".
* processing begins, based on whether SingleAssetVault or LendingProtocol are
* enabled. If either is enabled, the mantissa range is set to "large". If not,
* it is set to "small", preserving backward compatibility and correct
* "amendment-gating".
*
* It is extremely unlikely that any more calls to setMantissaScale() will be
* needed outside of unit tests.
@@ -344,8 +344,8 @@ concept Integral64 = std::is_same_v<T, std::int64_t> || std::is_same_v<T, std::u
* set/getMantissaScale() functions may be most appropriate. However, if the
* test has anything to do with transaction processing, it should enable or
* disable the amendments that control the mantissa range choice
* (SingleAssetVault, LendingProtocol, and MPTokensV2), and/or check if any of
* those amendments are enabled to determine which result to expect.
* (SingleAssetVault and LendingProtocol), and/or check if either of those
* amendments are enabled to determine which result to expect.
*/
class Number final
{
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+2 -3
View File
@@ -1,8 +1,7 @@
#pragma once
#include <xrpl/basics/sanitizers.h>
#include <boost/core/type_name.hpp>
#include <xrpl/beast/type_name.h>
#include <exception>
#include <string>
@@ -56,7 +55,7 @@ Throw(Args&&... args)
std::is_convertible_v<E*, std::exception*>, "Exception must derive from std::exception.");
E e(std::forward<Args>(args)...);
logThrow("Throwing exception of type " + boost::core::type_name<E>() + ": " + e.what());
logThrow(std::string("Throwing exception of type " + beast::typeName<E>() + ": ") + e.what());
throw std::move(e);
}
-200
View File
@@ -1,200 +0,0 @@
#pragma once
#include <concepts>
#include <type_traits>
#include <utility>
namespace xrpl {
/**
* Opt-in bitwise operators for scoped enumerations.
*
* Provides `&`, `|`, `^`, `~` and the corresponding compound assignment forms
* for any scoped enumeration with an unsigned underlying type that opts in by
* specializing @ref OptIn:
*
* @code
* namespace xrpl {
*
* enum class MyFlags : std::uint32_t { a = 1, b = 2, c = 4 };
*
* template <>
* struct enum_bitops::OptIn<MyFlags> : std::true_type {};
*
* } // namespace xrpl
* @endcode
*
* The operators are constexpr and noexcept, and return the enumeration type.
*
* @par Where the specialization may be declared
* In namespace xrpl, or at global scope with full qualification, and after
* the enumeration is defined but before the operators are first used. This
* cannot be done in a nested namespace or at class scope; for enumerations
* nested in a class this means after the class definition.
*
* @par Where the operators are found
* The operators are declared in namespace xrpl, so argument-dependent lookup
* will find them only for enumerations declared in xrpl or nested in a class
* declared directly in xrpl. Enumerations in a nested namespace will only be
* found from code inside xrpl, and only if no enclosing scope declares an
* operator of the same name; elsewhere they require using-declarations.
*/
namespace enum_bitops {
/**
* Types that may be opted in to the bitwise operators.
*
* Satisfied by a scoped enumeration whose underlying type is an unsigned
* integer type.
*
* @tparam T The type to test.
*/
template <typename T>
concept Eligible = std::is_scoped_enum_v<T> && std::unsigned_integral<std::underlying_type_t<T>>;
/**
* Opt-in switch for the bitwise operators.
*
* The primary template derives from std::false_type. Specialize it to derive
* from std::true_type to enable the operators for an @ref Eligible enumeration.
*
* @tparam T The scoped enumeration to opt in.
*/
template <Eligible T>
struct OptIn : std::false_type
{
};
/**
* Enumerations for which the bitwise operators are enabled.
*
* Satisfied when @p T satisfies @ref Eligible and @ref OptIn has been
* specialized for it to derive from std::true_type.
*
* @tparam T The type to test.
*/
template <typename T>
concept Candidate = Eligible<T> && OptIn<T>::value;
} // namespace enum_bitops
// These are declared in xrpl, not in enum_bitops, so that argument-dependent
// lookup finds them for enumerations whose associated namespace is xrpl.
/**
* @name Bitwise operators for opted-in scoped enumerations
*
* Each operator applies the corresponding built-in operator to the
* underlying values and converts the result back to the enumeration type.
* Both operands must have the same enumeration type; there is no implicit
* conversion to or from the underlying type.
*/
/** @{ */
/**
* Bitwise AND.
*
* @param lhs The left operand.
* @param rhs The right operand.
* @return The bits set in both @p lhs and @p rhs.
*/
template <enum_bitops::Candidate T>
constexpr T
operator&(T lhs, T rhs) noexcept
{
return static_cast<T>(std::to_underlying(lhs) & std::to_underlying(rhs));
}
/**
* Bitwise OR.
*
* @param lhs The left operand.
* @param rhs The right operand.
* @return The bits set in @p lhs, in @p rhs, or in both.
*/
template <enum_bitops::Candidate T>
constexpr T
operator|(T lhs, T rhs) noexcept
{
return static_cast<T>(std::to_underlying(lhs) | std::to_underlying(rhs));
}
/**
* Bitwise exclusive OR.
*
* @param lhs The left operand.
* @param rhs The right operand.
* @return The bits set in exactly one of @p lhs and @p rhs.
*/
template <enum_bitops::Candidate T>
constexpr T
operator^(T lhs, T rhs) noexcept
{
return static_cast<T>(std::to_underlying(lhs) ^ std::to_underlying(rhs));
}
/**
* Bitwise complement.
*
* The result has every bit of the underlying type that is clear in
* @p val, including bits that no enumerator names. It is intended for
* clearing flags, as in `flags & ~flag`.
*
* @param val The operand.
* @return The complement of @p val.
*/
template <enum_bitops::Candidate T>
constexpr T
operator~(T val) noexcept
{
return static_cast<T>(~std::to_underlying(val));
}
/**
* Bitwise AND assignment.
*
* @param lhs The value to modify.
* @param rhs The right operand.
* @return A reference to @p lhs.
*/
template <enum_bitops::Candidate T>
constexpr T&
operator&=(T& lhs, T rhs) noexcept
{
lhs = lhs & rhs;
return lhs;
}
/**
* Bitwise OR assignment.
*
* @param lhs The value to modify.
* @param rhs The right operand.
* @return A reference to @p lhs.
*/
template <enum_bitops::Candidate T>
constexpr T&
operator|=(T& lhs, T rhs) noexcept
{
lhs = lhs | rhs;
return lhs;
}
/**
* Bitwise exclusive OR assignment.
*
* @param lhs The value to modify.
* @param rhs The right operand.
* @return A reference to @p lhs.
*/
template <enum_bitops::Candidate T>
constexpr T&
operator^=(T& lhs, T rhs) noexcept
{
lhs = lhs ^ rhs;
return lhs;
}
/** @} */
} // namespace xrpl
+46 -151
View File
@@ -2,212 +2,107 @@
#include <xrpl/beast/utility/instrumentation.h> // IWYU pragma: keep
#include <concepts>
#include <cstdint>
#include <limits>
#include <memory>
#include <type_traits>
#include <utility>
namespace xrpl {
/** Every value of @p Src can be represented by @p Dest.
// safe_cast adds compile-time checks to a static_cast to ensure that
// the destination can hold all values of the source. This is particularly
// handy when the source or destination is an enumeration type.
Given two integral types, the cast is safe when the destination can
hold every possible value of the source.
template <class Src, class Dest>
concept SafeToCast = (std::is_integral_v<Src> && std::is_integral_v<Dest>) &&
(std::is_signed_v<Src> || std::is_unsigned_v<Dest>) &&
(std::is_signed_v<Src> != std::is_signed_v<Dest> ? sizeof(Dest) > sizeof(Src)
: sizeof(Dest) >= sizeof(Src));
Comparing the bounds requires care: we use @c std::cmp_less_equal and
@c std::cmp_greater_equal; the plain relational operators would apply
arithmetic conversions, resulting in incorrect results when comparing
across signedness.
Because @c std::cmp_* requires standard signed or unsigned integer
arguments, which excludes character types and bool, we first widen
all type bounds to the maximum-width integer type while preserving
signedness.
@note Extended integer types, like __int128 on gcc, cannot be safely
widened to a standard integer type, so the concept will reject
them.
*/
template <typename Src, typename Dest>
concept SafeToCast = std::is_integral_v<Src> && std::is_integral_v<Dest> && []() consteval {
using WideSrc = std::conditional_t<std::is_signed_v<Src>, std::intmax_t, std::uintmax_t>;
using WideDest = std::conditional_t<std::is_signed_v<Dest>, std::intmax_t, std::uintmax_t>;
// Note: this guard must be an evaluated branch, and not a
// static_assert. The lambda body is outside the immediate
// context, so a substitution-time failure here would be a
// hard error rather than leaving the concept unsatisfied.
if constexpr (sizeof(Src) > sizeof(WideSrc) || sizeof(Dest) > sizeof(WideDest))
{
return false;
}
else
{
return std::cmp_less_equal(
static_cast<WideDest>(std::numeric_limits<Dest>::min()),
static_cast<WideSrc>(std::numeric_limits<Src>::min())) &&
std::cmp_greater_equal(
static_cast<WideDest>(std::numeric_limits<Dest>::max()),
static_cast<WideSrc>(std::numeric_limits<Src>::max()));
}
}();
/** Compile-time-checked static_cast that rejects non-value preserving casts.
@note There is deliberately no enum-to-enum overload, and no overload
returning the underlying type of an enum. For the latter, use
@c std::to_underlying.
*/
/** @{ */
template <typename Dest, typename Src>
requires(std::is_integral_v<Dest> && std::is_integral_v<Src>)
template <class Dest, class Src>
constexpr Dest
safeCast(Src s) noexcept
requires(std::is_integral_v<Dest> && std::is_integral_v<Src>)
{
static_assert(
SafeToCast<Src, Dest>, "This cast is not value-preserving. Please use unsafeCast instead.");
std::is_signed_v<Dest> || std::is_unsigned_v<Src>, "Cannot cast signed to unsigned");
constexpr unsigned kNotSame = std::is_signed_v<Dest> != std::is_signed_v<Src>;
static_assert(
sizeof(Dest) >= sizeof(Src) + kNotSame,
"Destination is too small to hold all values of source");
return static_cast<Dest>(s);
}
template <typename Dest, typename Src>
requires(std::is_enum_v<Dest> && std::is_integral_v<Src>)
template <class Dest, class Src>
constexpr Dest
safeCast(Src s) noexcept
requires(std::is_enum_v<Dest> && std::is_integral_v<Src>)
{
return static_cast<Dest>(safeCast<std::underlying_type_t<Dest>>(s));
}
template <typename Dest, typename Src>
requires(std::is_integral_v<Dest> && std::is_enum_v<Src>)
template <class Dest, class Src>
constexpr Dest
safeCast(Src s) noexcept
requires(std::is_integral_v<Dest> && std::is_enum_v<Src>)
{
return safeCast<Dest>(std::to_underlying(s));
return safeCast<Dest>(static_cast<std::underlying_type_t<Src>>(s));
}
/** @} */
/** Integral-to-integral cast that is known to be narrowing or sign-erasing.
// unsafe_cast explicitly flags a static_cast as not necessarily able to hold
// all values of the source. It includes a compile-time check so that if
// underlying types become safe, it can be converted to a safe_cast.
This explicitly flags a conversion that can lose information for some
values of the source type, where the call site accepts that loss (or
truncation is the intended behavior).
The compile-time check ensures the cast remains "unsafe": if the types
involved later change such that the conversion becomes inherently
value-preserving, the static assertion fires with instructions to
migrate the call site to @ref safeCast.
If the conversion's safety depends on a runtime precondition rather
than on the types, or varies across instantiations of generic code,
use @ref checkedCast instead.
*/
/** @{ */
template <typename Dest, typename Src>
requires(std::is_integral_v<Dest> && std::is_integral_v<Src>)
template <class Dest, class Src>
constexpr Dest
unsafeCast(Src s) noexcept
requires(std::is_integral_v<Dest> && std::is_integral_v<Src>)
{
static_assert(
!SafeToCast<Src, Dest>, "This cast is value-preserving. Please use safeCast instead.");
!SafeToCast<Src, Dest>,
"Only unsafe if casting signed to unsigned or "
"destination is too small");
return static_cast<Dest>(s);
}
template <typename Dest, typename Src>
requires(std::is_enum_v<Dest> && std::is_integral_v<Src>)
template <class Dest, class Src>
constexpr Dest
unsafeCast(Src s) noexcept
requires(std::is_enum_v<Dest> && std::is_integral_v<Src>)
{
return static_cast<Dest>(unsafeCast<std::underlying_type_t<Dest>>(s));
}
template <typename Dest, typename Src>
requires(std::is_integral_v<Dest> && std::is_enum_v<Src>)
template <class Dest, class Src>
constexpr Dest
unsafeCast(Src s) noexcept
{
return unsafeCast<Dest>(std::to_underlying(s));
}
/** @} */
/** Integral-to-integral cast when the caller has performed a bounds check.
This documents that a runtime precondition or external invariant, which
is not necessarily visible to the compiler, guarantees that the requested
conversion is value-preserving for the values that can actually occur.
This is primarily meant for generic code, where the same expression may
be value-preserving for one instantiation but not for another, making
both @ref safeCast and @ref unsafeCast unusable.
Unlike @ref safeCast and @ref unsafeCast, this imposes no static check
on the type relationship: the caller's claim is about runtime values,
not about types.
*/
/** @{ */
template <typename Dest, typename Src>
requires(std::is_integral_v<Dest> && std::is_integral_v<Src>)
constexpr Dest
checkedCast(Src s) noexcept
{
return static_cast<Dest>(s);
}
template <typename Dest, typename Src>
requires(std::is_enum_v<Dest> && std::is_integral_v<Src>)
constexpr Dest
checkedCast(Src s) noexcept
{
return static_cast<Dest>(checkedCast<std::underlying_type_t<Dest>>(s));
}
template <typename Dest, typename Src>
requires(std::is_integral_v<Dest> && std::is_enum_v<Src>)
constexpr Dest
checkedCast(Src s) noexcept
{
return checkedCast<Dest>(std::to_underlying(s));
return unsafeCast<Dest>(static_cast<std::underlying_type_t<Src>>(s));
}
/** @} */
/** Downcast within a class hierarchy, verified in debug builds.
Performs a static_cast down a hierarchy, but in debug builds verifies
via dynamic_cast that the object's dynamic type actually permits the
downcast. Both build modes execute the same conversion; debug builds
merely add the check.
The pointer form passes null through unchanged, as dynamic_cast does.
@note The check requires @p Src to be polymorphic; in release builds
an invalid downcast is undefined behavior on use, exactly as
with a bare static_cast.
*/
/** @{ */
template <class Dest, class Src>
requires(
std::is_pointer_v<Dest> && std::is_polymorphic_v<Src> &&
std::derived_from<std::remove_pointer_t<Dest>, Src> &&
(std::is_const_v<std::remove_pointer_t<Dest>> || !std::is_const_v<Src>) &&
(std::is_volatile_v<std::remove_pointer_t<Dest>> || !std::is_volatile_v<Src>))
requires std::is_pointer_v<Dest>
inline Dest
safeDowncast(Src* s) noexcept
{
XRPL_ASSERT(
s == nullptr || dynamic_cast<Dest>(s) != nullptr, "xrpl::safeDowncast : valid downcast");
#ifdef NDEBUG
return static_cast<Dest>(s); // NOLINT(cppcoreguidelines-pro-type-static-cast-downcast)
#else
auto* result = dynamic_cast<Dest>(s);
XRPL_ASSERT(result != nullptr, "xrpl::safeDowncast : pointer downcast is valid");
return result;
#endif
}
template <class Dest, class Src>
requires(
std::is_lvalue_reference_v<Dest> && std::is_polymorphic_v<Src> &&
std::derived_from<std::remove_reference_t<Dest>, Src>)
requires std::is_lvalue_reference_v<Dest>
inline Dest
safeDowncast(Src& s) noexcept
{
return *safeDowncast<std::add_pointer_t<std::remove_reference_t<Dest>>>(std::addressof(s));
#ifndef NDEBUG
XRPL_ASSERT(
dynamic_cast<std::add_pointer_t<std::remove_reference_t<Dest>>>(&s) != nullptr,
"xrpl::safeDowncast : reference downcast is valid");
#endif
return static_cast<Dest>(s); // NOLINT(cppcoreguidelines-pro-type-static-cast-downcast)
}
/** @} */
} // namespace xrpl
+152 -208
View File
@@ -4,215 +4,88 @@
#include <xrpl/beast/utility/instrumentation.h>
#include <boost/predef/architecture.h>
#include <atomic>
#include <concepts>
#include <limits>
#include <type_traits>
#if BOOST_ARCH_X86
#ifndef __aarch64__
#include <immintrin.h>
#endif
namespace xrpl {
/** An unsigned integral type suitable for use as a spinlock.
The type must be always lock-free when wrapped in std::atomic, so
that lock operations cannot themselves take a (library-level) lock.
*/
template <typename T>
concept SpinlockValueType = std::is_unsigned_v<T> && std::atomic<T>::is_always_lock_free;
/** A spinlock value type that additionally supports the atomic bitwise
operations required to pack multiple locks into a single integer.
*/
template <typename T>
concept PackedSpinlockValueType = SpinlockValueType<T> && requires(std::atomic<T>& a, T v) {
{ a.fetch_or(v) } -> std::same_as<T>;
{ a.fetch_and(v) } -> std::same_as<T>;
};
namespace detail {
/** Inform the processor that we are in a tight spin-wait loop.
Spinlocks caught in tight loops can result in the processor's pipeline
filling up with comparison operations, resulting in a misprediction at
the time the lock is finally acquired, necessitating pipeline flushing
which is ridiculously expensive and results in very high latency.
This function instructs the processor to "pause" for some architecture
specific amount of time, to prevent this.
/**
* Inform the processor that we are in a tight spin-wait loop.
*
* Spinlocks caught in tight loops can result in the processor's pipeline
* filling up with comparison operations, resulting in a misprediction at
* the time the lock is finally acquired, necessitating pipeline flushing
* which is ridiculously expensive and results in very high latency.
*
* This function instructs the processor to "pause" for some architecture
* specific amount of time, to prevent this.
*/
inline void
spinPause() noexcept
{
#if BOOST_ARCH_X86
_mm_pause();
#elif BOOST_ARCH_ARM
asm volatile("yield" ::: "memory");
#ifdef __aarch64__
asm volatile("yield");
#else
#error No implementation available for spinPause to use
_mm_pause();
#endif
}
} // namespace detail
//------------------------------------------------------------------------------
/** Attempt to acquire a spinlock without blocking.
@note This interface is primarily intended for one-shot attempts to
acquire the lock. Avoid calling this function directly from a
loop and use @ref spinLock instead.
@tparam T An unsigned integral type.
@param lock The atomic variable used as the lock.
@return true if the lock was acquired, false if it was already held.
/** @{ */
/**
* Classes to handle arrays of spinlocks packed into a single atomic integer:
*
* Packed spinlocks allow for tremendously space-efficient lock-sharding
* but they come at a cost.
*
* First, the implementation is necessarily low-level and uses advanced
* features like memory ordering and highly platform-specific tricks to
* maximize performance. This imposes a significant and ongoing cost to
* developers.
*
* Second, and perhaps most important, is that the packing of multiple
* locks into a single integer which, albeit space-efficient, also has
* performance implications stemming from data dependencies, increased
* cache-coherency traffic between processors and heavier loads on the
* processor's load/store units.
*
* To be sure, these locks can have advantages but they are definitely
* not general purpose locks and should not be thought of or used that
* way. The use cases for them are likely few and far between; without
* a compelling reason to use them, backed by profiling data, it might
* be best to use one of the standard locking primitives instead. Note
* that in most common platforms, `std::mutex` is so heavily optimized
* that it can, usually, outperform spinlocks.
*
* @tparam T An unsigned integral type (e.g. std::uint16_t)
*/
template <SpinlockValueType T>
[[nodiscard]] bool
spinTryLock(std::atomic<T>& lock) noexcept
{
// A compare-exchange is required here, not an unconditional exchange:
// a failed attempt must not modify the lock word, in case the atomic
// is shared with PackedSpinlock).
T expected = 0;
return lock.compare_exchange_strong(
expected,
std::numeric_limits<T>::max(),
std::memory_order::acquire,
std::memory_order::relaxed);
;
}
/** Acquire a spinlock, blocking until available.
Uses a TTAS (test-and-test-and-set) pattern, so waiters share the cache
line read-only, helping to avoid unnecessary coherency traffic.
@tparam T An unsigned integral type.
@param lock The atomic variable used as the lock.
/**
* A class that grabs a single packed spinlock from an atomic integer.
*
* This class meets the requirements of Lockable:
* https://en.cppreference.com/w/cpp/named_req/Lockable
*/
template <SpinlockValueType T>
void
spinLock(std::atomic<T>& lock) noexcept
{
do
{
// Relaxed ordering is sufficient for the spin: this load is only
// a filter. The acquire on the successful exchange in spinTryLock
// is what synchronizes the critical section.
while (lock.load(std::memory_order::relaxed) != 0)
detail::spinPause();
} while (!spinTryLock(lock));
}
/** Release a spinlock.
@tparam T An unsigned integral type.
@param lock The atomic variable used as the lock.
*/
template <SpinlockValueType T>
void
spinUnlock(std::atomic<T>& lock) noexcept
{
lock.store(0, std::memory_order::release);
}
//------------------------------------------------------------------------------
/** A Lockable interface to a spinlock implemented on top of an atomic.
@tparam T An unsigned integral type.
@note Using `PackedSpinlock` and `Spinlock` against the same underlying
atomic integer is possible but can result in `Spinlock` not being
able to acquire the lock during periods of high contention due to
the way the two locks operate: `Spinlock` spins and tries to grab
all the bits at once, whereas any given `PackedSpinlock` instance
only tries to grab one bit at a time. Caveat emptor.
This class meets the requirements of Lockable:
https://en.cppreference.com/w/cpp/named_req/Lockable
*/
template <SpinlockValueType T>
class Spinlock
{
std::atomic<T>& lock_;
public:
Spinlock(Spinlock const&) = delete;
Spinlock&
operator=(Spinlock const&) = delete;
/** Construct a spinlock handle.
@param lock The atomic integer to spin against.
@note For performance reasons, you should strive to have `lock` be
on a cacheline by itself.
*/
explicit Spinlock(std::atomic<T>& lock) noexcept : lock_(lock)
{
}
[[nodiscard]] bool
try_lock() noexcept // NOLINT(readability-identifier-naming)
{
return spinTryLock(lock_);
}
void
lock() noexcept
{
spinLock(lock_);
}
void
unlock() noexcept
{
spinUnlock(lock_);
}
};
//------------------------------------------------------------------------------
/** A Lockable interface to a packed spinlock implemented on top of an atomic.
Packed spinlocks offer tremendous space-efficient lock-sharding but
they come at a cost.
First, the implementation is necessarily low-level and uses advanced
features like memory ordering and highly platform-specific tricks to
maximize performance. This imposes a significant and ongoing cost to
developers.
Second, and perhaps most important, is that the packing of multiple
locks into a single integer which, albeit space-efficient, also has
performance implications stemming from data dependencies, increased
cache-coherency traffic between processors and heavier loads on the
processor's load/store units.
To be sure, these locks can have advantages but they are definitely
not general purpose locks and should not be thought of or used that
way. The use cases for them are likely few and far between; without
a compelling reason to use them, backed by profiling data, it might
be best to use one of the standard locking primitives instead. Note
that in most common platforms, `std::mutex` is so heavily optimized
that it can, usually, outperform spinlocks.
@tparam T An unsigned integral type (e.g. std::uint16_t)
This class meets the requirements of Lockable:
https://en.cppreference.com/w/cpp/named_req/Lockable
*/
template <PackedSpinlockValueType T>
template <class T>
class PackedSpinlock
{
// clang-format off
static_assert(std::is_unsigned_v<T>);
static_assert(std::atomic<T>::is_always_lock_free);
static_assert(
std::is_same_v<decltype(std::declval<std::atomic<T>&>().fetch_or(0)), T> &&
std::is_same_v<decltype(std::declval<std::atomic<T>&>().fetch_and(0)), T>,
"std::atomic<T>::fetch_and(T) and std::atomic<T>::fetch_and(T) are required by packed_spinlock");
// clang-format on
private:
std::atomic<T>& bits_;
T const mask_;
@@ -221,49 +94,120 @@ public:
PackedSpinlock&
operator=(PackedSpinlock const&) = delete;
/** Construct a packed spinlock handle for a single bit.
@param lock The atomic integer inside which the spinlock is packed.
@param index The index of the spinlock this object acquires.
@note For performance reasons, you should strive to have `lock` be
on a cacheline by itself.
/**
* A single spinlock packed inside the specified atomic
*
* @param lock The atomic integer inside which the spinlock is packed.
* @param index The index of the spinlock this object acquires.
*
* @note For performance reasons, you should strive to have `lock` be
* on a cacheline by itself.
*/
PackedSpinlock(std::atomic<T>& lock, int index) noexcept
: bits_(lock), mask_([index]() {
XRPL_ASSERT(
index >= 0 && index < std::numeric_limits<T>::digits,
"xrpl::PackedSpinlock::PackedSpinlock : valid index");
return static_cast<T>(T{1} << index);
}())
PackedSpinlock(std::atomic<T>& lock, int index) : bits_(lock), mask_(static_cast<T>(1) << index)
{
XRPL_ASSERT(
index >= 0 && (mask_ != 0),
"xrpl::PackedSpinlock::PackedSpinlock : valid index and mask");
}
[[nodiscard]] bool
try_lock() // NOLINT(readability-identifier-naming)
{
return (bits_.fetch_or(mask_, std::memory_order_acquire) & mask_) == 0;
}
void
lock()
{
while (!try_lock())
{
// The use of relaxed memory ordering here is intentional and
// serves to help reduce cache coherency traffic during times
// of contention by avoiding writes that would definitely not
// result in the lock being acquired.
while ((bits_.load(std::memory_order_relaxed) & mask_) != 0)
detail::spinPause();
}
}
void
unlock()
{
bits_.fetch_and(~mask_, std::memory_order_release);
}
};
/**
* A spinlock implemented on top of an atomic integer.
*
* @note Using `packed_spinlock` and `spinlock` against the same underlying
* atomic integer can result in `spinlock` not being able to actually
* acquire the lock during periods of high contention, because of how
* the two locks operate: `spinlock` will spin trying to grab all the
* bits at once, whereas any given `packed_spinlock` will only try to
* grab one bit at a time. Caveat emptor.
*
* This class meets the requirements of Lockable:
* https://en.cppreference.com/w/cpp/named_req/Lockable
*/
template <class T>
class Spinlock
{
static_assert(std::is_unsigned_v<T>);
static_assert(std::atomic<T>::is_always_lock_free);
private:
std::atomic<T>& lock_;
public:
Spinlock(Spinlock const&) = delete;
Spinlock&
operator=(Spinlock const&) = delete;
/**
* Grabs the
*
* @param lock The atomic integer to spin against.
*
* @note For performance reasons, you should strive to have `lock` be
* on a cacheline by itself.
*/
Spinlock(std::atomic<T>& lock) : lock_(lock)
{
}
[[nodiscard]] bool
try_lock() noexcept // NOLINT(readability-identifier-naming)
try_lock() // NOLINT(readability-identifier-naming)
{
return (bits_.fetch_or(mask_, std::memory_order::acquire) & mask_) == 0;
T expected = 0;
return lock_.compare_exchange_weak(
expected,
std::numeric_limits<T>::max(),
std::memory_order_acquire,
std::memory_order_relaxed);
}
void
lock() noexcept
lock()
{
do
while (!try_lock())
{
// The use of relaxed memory ordering here is intentional and
// serves to help reduce cache coherency traffic during times
// of contention by avoiding writes that are unlikely to grab
// the requested lock.
while ((bits_.load(std::memory_order::relaxed) & mask_) != 0)
// of contention by avoiding writes that would definitely not
// result in the lock being acquired.
while (lock_.load(std::memory_order_relaxed) != 0)
detail::spinPause();
} while (!try_lock());
}
}
void
unlock() noexcept
unlock()
{
bits_.fetch_and(~mask_, std::memory_order::release);
lock_.store(0, std::memory_order_release);
}
};
/** @} */
} // namespace xrpl
+47
View File
@@ -0,0 +1,47 @@
#pragma once
#include <cstdlib>
#include <string>
#include <type_traits>
#include <typeinfo>
#ifndef _MSC_VER
#include <cxxabi.h>
#endif
namespace beast {
template <typename T>
std::string
typeName()
{
using TR = std::remove_reference_t<T>;
std::string name = typeid(TR).name();
#ifndef _MSC_VER
if (auto s = abi::__cxa_demangle(name.c_str(), nullptr, nullptr, nullptr))
{
name = s;
// NOLINTNEXTLINE(cppcoreguidelines-no-malloc)
std::free(s);
}
#endif
if (std::is_const_v<TR>)
name += " const";
if (std::is_volatile_v<TR>)
name += " volatile";
if (std::is_lvalue_reference_v<T>)
{
name += "&";
}
else if (std::is_rvalue_reference_v<T>)
{
name += "&&";
}
return name;
}
} // namespace beast
+109 -36
View File
@@ -2,24 +2,26 @@
#pragma once
#include <compare>
#include <concepts>
namespace beast {
/**
* Zero allows classes to offer efficient comparisons to zero.
*
* Zero is a struct to allow classes to efficiently compare with zero without
* requiring an rvalue construction.
*
* It's often the case that we have classes which combine a number and a unit.
* In such cases, comparisons like t > 0 or t != 0 make sense, but comparisons
* like t > 1 or t != 1 do not. Comparing against kZero expresses exactly that,
* without constructing a T.
* like t > 1 or t != 1 do not.
*
* A type T participates if either `t.signum()` or an unqualified `signum(t)`
* found by argument-dependent lookup returns an integer that is negative,
* zero, or positive according to the sign of t. Both `t == kZero` and
* `kZero == t` work, as do all six relational operators in either order.
* The class Zero allows such comparisons to be easily made.
*
* The comparing class T either needs to have a method called signum() which
* returns a positive number, 0, or a negative; or there needs to be a signum
* function which resolves in the namespace which takes an instance of T and
* returns a positive, zero or negative number.
*/
struct Zero
{
explicit Zero() = default;
@@ -28,44 +30,115 @@ struct Zero
inline constexpr Zero kZero{};
/**
* Default implementation of signum: call the member function.
* Default implementation of signum calls the method on the class.
*/
template <class T>
requires requires(T const& t) {
{ t.signum() } -> std::integral;
}
[[nodiscard]] constexpr auto
signum(T const& t) noexcept(noexcept(t.signum()))
template <typename T>
auto
signum(T const& t)
{
return t.signum();
}
namespace detail {
namespace detail::zero_helper {
/**
* A type with a usable signum: either the member-based default above, or a
* `signum(t)` overload in T's own namespace, found by ADL. A user overload
* that is a better match than the template wins, as usual.
*/
// For argument dependent lookup to function properly, calls to signum must
// be made from a namespace that does not include overloads of the function..
template <class T>
concept HasSignum = requires(T const& t) {
{ signum(t) } -> std::integral;
};
} // namespace detail
template <detail::HasSignum T>
[[nodiscard]] constexpr bool
operator==(T const& t, Zero) noexcept(noexcept(signum(t)))
auto
callSignum(T const& t)
{
return signum(t) == 0;
return signum(t);
}
template <detail::HasSignum T>
[[nodiscard]] constexpr std::strong_ordering
operator<=>(T const& t, Zero) noexcept(noexcept(signum(t)))
} // namespace detail::zero_helper
// Handle operators where T is on the left side using signum.
template <typename T>
bool
operator==(T const& t, Zero)
{
return signum(t) <=> 0;
return detail::zero_helper::callSignum(t) == 0;
}
template <typename T>
bool
operator!=(T const& t, Zero)
{
return detail::zero_helper::callSignum(t) != 0;
}
template <typename T>
bool
operator<(T const& t, Zero)
{
return detail::zero_helper::callSignum(t) < 0;
}
template <typename T>
bool
operator>(T const& t, Zero)
{
return detail::zero_helper::callSignum(t) > 0;
}
template <typename T>
bool
operator>=(T const& t, Zero)
{
return detail::zero_helper::callSignum(t) >= 0;
}
template <typename T>
bool
operator<=(T const& t, Zero)
{
return detail::zero_helper::callSignum(t) <= 0;
}
// Handle operators where T is on the right side by
// reversing the operation, so that T is on the left side.
template <typename T>
bool
operator==(Zero, T const& t)
{
return t == kZero;
}
template <typename T>
bool
operator!=(Zero, T const& t)
{
return t != kZero;
}
template <typename T>
bool
operator<(Zero, T const& t)
{
return t > kZero;
}
template <typename T>
bool
operator>(Zero, T const& t)
{
return t < kZero;
}
template <typename T>
bool
operator>=(Zero, T const& t)
{
return t <= kZero;
}
template <typename T>
bool
operator<=(Zero, T const& t)
{
return t >= kZero;
}
} // namespace beast
-136
View File
@@ -1,136 +0,0 @@
#pragma once
#include <utility>
namespace xrpl {
/**
* Coroutine stack size (1.5 MB). Increased from 1 MB because
* ASAN-instrumented deep call stacks exceeded the original limit.
*/
constexpr std::size_t kCoroStackSize = 1536 * 1024;
template <class F>
JobQueue::Coro::Coro(CoroCreateT, JobQueue& jq, JobType type, std::string name, F&& f)
: jq_(jq)
, type_(type)
, name_(std::move(name))
, coro_(
boost::context::protected_fixedsize_stack(kCoroStackSize),
[this, fn = std::forward<F>(f)](boost::coroutines2::coroutine<void>::push_type& doYield) {
yield_ = &doYield;
yield();
fn(shared_from_this());
#ifndef NDEBUG
finished_ = true;
#endif
})
{
}
inline JobQueue::Coro::~Coro()
{
#ifndef NDEBUG
XRPL_ASSERT(finished_, "xrpl::JobQueue::Coro::~Coro : is finished");
#endif
}
inline void
JobQueue::Coro::yield() const
{
{
std::scoped_lock const lock(jq_.mutex_);
++jq_.nSuspend_;
}
(*yield_)();
}
inline bool
JobQueue::Coro::post()
{
{
std::scoped_lock const lk(mutexRun_);
running_ = true;
}
// sp keeps 'this' alive
if (jq_.addJob(type_, name_, [this, sp = shared_from_this()]() { resume(); }))
{
return true;
}
// The coroutine will not run. Clean up running_.
std::scoped_lock const lk(mutexRun_);
running_ = false;
cv_.notify_all();
return false;
}
inline void
JobQueue::Coro::resume()
{
{
std::scoped_lock const lk(mutexRun_);
running_ = true;
}
{
std::scoped_lock const lk(jq_.mutex_);
--jq_.nSuspend_;
}
auto saved = detail::getLocalValues().release();
detail::getLocalValues().reset(&lvs_);
std::scoped_lock const lock(mutex_);
// A late resume() can arrive after the coroutine has already completed.
// This is an expected (if rare) outcome of the race condition documented
// in JobQueue.h:354-377 where post() schedules a resume job before the
// coroutine yields — the mutex serializes access, but by the time this
// resume() acquires the lock the coroutine may have already run to
// completion. Calling operator() on a completed boost::coroutine2 is
// undefined behavior, so we must check and skip invoking the coroutine
// body if it has already completed.
if (coro_)
{
coro_();
}
detail::getLocalValues().release();
detail::getLocalValues().reset(saved);
std::scoped_lock const lk(mutexRun_);
running_ = false;
cv_.notify_all();
}
inline bool
JobQueue::Coro::runnable() const
{
return static_cast<bool>(coro_);
}
inline void
JobQueue::Coro::expectEarlyExit()
{
#ifndef NDEBUG
if (!finished_)
#endif
{
// expectEarlyExit() must only ever be called from outside the
// Coro's stack. It you're inside the stack you can simply return
// and be done.
//
// That said, since we're outside the Coro's stack, we need to
// decrement the nSuspend that the Coro's call to yield caused.
std::scoped_lock const lock(jq_.mutex_);
--jq_.nSuspend_;
#ifndef NDEBUG
finished_ = true;
#endif
}
}
inline void
JobQueue::Coro::join()
{
std::unique_lock<std::mutex> lk(mutexRun_);
cv_.wait(lk, [this]() { return !running_; });
}
} // namespace xrpl
+713
View File
@@ -0,0 +1,713 @@
#pragma once
#include <xrpl/beast/utility/instrumentation.h>
#include <coroutine>
#include <exception>
#include <type_traits>
#include <utility>
#include <variant>
namespace xrpl {
template <typename T = void>
class CoroTask;
/**
* CoroTask<void> -- coroutine return type for void-returning coroutines.
*
* Class / Dependency Diagram
* ==========================
*
* CoroTask<void>
* +-----------------------------------------------+
* | - handle_ : Handle (coroutine_handle<promise>) |
* +-----------------------------------------------+
* | + handle(), done() |
* | + await_ready/suspend/resume (Awaiter iface) |
* +-----------------------------------------------+
* | owns
* v
* promise_type
* +-----------------------------------------------+
* | - exception_ : std::exception_ptr |
* | - continuation_ : std::coroutine_handle<> |
* +-----------------------------------------------+
* | + get_return_object() -> CoroTask |
* | + initial_suspend() -> suspend_always (lazy) |
* | + final_suspend() -> FinalAwaiter |
* | + return_void() |
* | + unhandled_exception() |
* +-----------------------------------------------+
* | returns at final_suspend
* v
* FinalAwaiter
* +-----------------------------------------------+
* | await_suspend(h): |
* | if continuation_ set -> symmetric transfer |
* | else -> noop_coroutine |
* +-----------------------------------------------+
*
* Design Notes
* ------------
* - Lazy start: initial_suspend returns suspend_always, so the coroutine
* body does not execute until the handle is explicitly resumed.
* - Symmetric transfer: await_suspend returns a coroutine_handle instead
* of void/bool, allowing the scheduler to jump directly to the next
* coroutine without growing the call stack.
* - Continuation chaining: when one CoroTask is co_await-ed inside
* another, the caller's handle is stored as continuation_ so
* FinalAwaiter can resume it when this task finishes.
* - Move-only: the handle is exclusively owned; copy is deleted.
*
* Usage Examples
* ==============
*
* 1. Basic void coroutine (the most common case in xrpld):
*
* CoroTask<void> doWork(std::shared_ptr<CoroTaskRunner> runner) {
* // do something
* co_await runner->suspend(); // yield control
* // resumed later via runner->post() or runner->resume()
* co_return;
* }
*
* 2. co_await-ing one CoroTask<void> from another (chaining):
*
* CoroTask<void> inner() {
* // ...
* co_return;
* }
* CoroTask<void> outer() {
* co_await inner(); // continuation_ links outer -> inner
* co_return; // FinalAwaiter resumes outer
* }
*
* 3. Exceptions propagate through co_await:
*
* CoroTask<void> failing() {
* throw std::runtime_error("oops");
* co_return;
* }
* CoroTask<void> caller() {
* try { co_await failing(); }
* catch (std::runtime_error const&) { // caught here }
* }
*
* Caveats / Pitfalls
* ==================
*
* BUG-RISK: Dangling references in coroutine parameters.
* Coroutine parameters are copied into the frame, but references
* are NOT -- they are stored as-is. If the referent goes out of scope
* before the coroutine finishes, you get use-after-free.
*
* // BROKEN -- local dies before coroutine runs:
* CoroTask<void> bad(int& ref) { co_return; }
* void launch() {
* int local = 42;
* auto task = bad(local); // frame stores &local
* } // local destroyed; frame holds dangling ref
*
* // FIX -- pass by value, or ensure lifetime via shared_ptr.
*
* BUG-RISK: GCC 14 corrupts reference captures in coroutine lambdas.
* When a lambda that returns CoroTask captures by reference ([&]),
* GCC 14 may generate a corrupted coroutine frame. Always capture
* by explicit pointer-to-value instead:
*
* // BROKEN on GCC 14:
* jq.postCoroTask(t, n, [&](auto) -> CoroTask<void> { ... });
*
* // FIX -- capture pointers explicitly:
* jq.postCoroTask(t, n, [ptr = &val](auto) -> CoroTask<void> { ... });
*
* BUG-RISK: Resuming a destroyed or completed CoroTask.
* Calling handle().resume() after the coroutine has already run to
* completion (done() == true) is undefined behavior. The CoroTaskRunner
* guards against this with an XRPL_ASSERT, but standalone usage of
* CoroTask must check done() before resuming.
*
* BUG-RISK: Moving a CoroTask that is being awaited.
* If task A is co_await-ed by task B (so A.continuation_ == B), moving
* or destroying A will invalidate the continuation link. Never move
* or reassign a CoroTask while it is mid-execution or being awaited.
*
* LIMITATION: CoroTask is fire-and-forget for the top-level owner.
* There is no built-in notification when the coroutine finishes.
* The caller must use external synchronization (e.g. CoroTaskRunner::join
* or a gate/condition_variable) to know when it is done.
*
* LIMITATION: No cancellation token.
* There is no way to cancel a suspended CoroTask from outside. The
* coroutine body must cooperatively check a flag (e.g. jq_.isStopping())
* after each co_await and co_return early if needed.
*
* LIMITATION: Stackless -- cannot suspend from nested non-coroutine calls.
* If a coroutine calls a regular function that wants to "yield", it
* cannot. Only the immediate coroutine body can use co_await.
* This is acceptable for xrpld because all yield() sites are shallow.
*/
template <>
class CoroTask<void>
{
public:
// The C++ coroutine protocol mandates these names (promise_type,
// initial_suspend, await_ready, ...) and instance-callable awaiter
// methods, which conflict with the project naming/static conventions.
// NOLINTBEGIN(readability-identifier-naming, readability-convert-member-functions-to-static)
struct promise_type;
using Handle = std::coroutine_handle<promise_type>;
/**
* Coroutine promise. Compiler uses this to manage coroutine state.
* Stores the exception (if any) and the continuation handle for
* symmetric transfer back to the awaiting coroutine.
*/
struct promise_type
{
// Captured exception from the coroutine body, rethrown in
// await_resume() when this task is co_await-ed by a caller.
std::exception_ptr exception_;
// Handle to the coroutine that is co_await-ing this task.
// Set by await_suspend(). FinalAwaiter uses it for symmetric
// transfer back to the caller. Null if this is a top-level task.
std::coroutine_handle<> continuation_;
/**
* Create the CoroTask return object.
* Called by the compiler at coroutine creation.
*/
CoroTask
get_return_object()
{
return CoroTask{Handle::from_promise(*this)};
}
/**
* Lazy start. The coroutine body does not execute until the
* handle is explicitly resumed (e.g. by CoroTaskRunner::resume).
*/
std::suspend_always
initial_suspend() noexcept
{
return {};
}
/**
* Awaiter returned by final_suspend(). Uses symmetric transfer:
* if a continuation exists, transfers control directly to it
* (tail-call, no stack growth). Otherwise returns noop_coroutine
* so the coroutine frame stays alive for the owner to destroy.
*/
struct FinalAwaiter
{
/**
* Always false. We need await_suspend to run for
* symmetric transfer.
*/
bool
await_ready() noexcept
{
return false;
}
/**
* Symmetric transfer: returns the continuation handle so
* the compiler emits a tail-call instead of a nested resume.
* If no continuation is set, returns noop_coroutine to
* suspend at final_suspend without destroying the frame.
*
* @param h Handle to this completing coroutine
*
* @return Continuation handle, or noop_coroutine
*/
std::coroutine_handle<>
await_suspend(Handle h) noexcept
{
if (auto cont = h.promise().continuation_)
return cont;
return std::noop_coroutine();
}
void
await_resume() noexcept
{
}
};
/**
* Returns FinalAwaiter for symmetric transfer at coroutine end.
*/
FinalAwaiter
final_suspend() noexcept
{
return {};
}
/**
* Called by the compiler for `co_return;` (void coroutine).
*/
void
return_void()
{
}
/**
* Called by the compiler when an exception escapes the coroutine
* body. Captures it for later rethrowing in await_resume().
*/
void
unhandled_exception()
{
exception_ = std::current_exception();
}
};
// NOLINTEND(readability-identifier-naming, readability-convert-member-functions-to-static)
/**
* Default constructor. Creates an empty (null handle) task.
*/
CoroTask() = default;
/**
* Takes ownership of a compiler-generated coroutine handle.
*
* @param h Coroutine handle to own
*/
explicit CoroTask(Handle h) : handle_(h)
{
}
/**
* Destroys the coroutine frame if this task owns one.
*/
~CoroTask()
{
if (handle_)
handle_.destroy();
}
/**
* Move constructor. Transfers handle ownership, leaves other empty.
*/
CoroTask(CoroTask&& other) noexcept : handle_(std::exchange(other.handle_, {}))
{
}
/**
* Move assignment. Destroys current frame (if any), takes other's.
*/
CoroTask&
operator=(CoroTask&& other) noexcept
{
if (this != &other)
{
if (handle_)
handle_.destroy();
handle_ = std::exchange(other.handle_, {});
}
return *this;
}
CoroTask(CoroTask const&) = delete;
CoroTask&
operator=(CoroTask const&) = delete;
/**
* @return The underlying coroutine_handle
*/
[[nodiscard]] Handle
handle() const
{
return handle_;
}
/**
* @return true if the coroutine has run to completion (or thrown)
*/
[[nodiscard]] bool
done() const
{
return handle_ && handle_.done();
}
// -- Awaiter interface: allows `co_await someCoroTask;` --
/**
* Always false. This task is lazy, so co_await always suspends
* the caller to set up the continuation link.
*/
// NOLINTBEGIN(readability-identifier-naming, readability-convert-member-functions-to-static)
[[nodiscard]] bool
await_ready() const noexcept
{
return false;
}
/**
* Stores the caller's handle as our continuation, then returns
* our handle for symmetric transfer (caller suspends, we resume).
*
* @param caller Handle of the coroutine doing co_await on us
*
* @return Our handle for symmetric transfer
*/
std::coroutine_handle<>
await_suspend(std::coroutine_handle<> caller) noexcept
{
XRPL_ASSERT(handle_, "xrpl::CoroTask<void>::await_suspend : handle is valid");
handle_.promise().continuation_ = caller;
return handle_; // Symmetric transfer
}
/**
* Called in the awaiting coroutine's context after this task
* completes. Rethrows any exception captured by
* unhandled_exception().
*/
void
await_resume()
{
XRPL_ASSERT(handle_, "xrpl::CoroTask<void>::await_resume : handle is valid");
if (auto& ep = handle_.promise().exception_)
std::rethrow_exception(ep);
}
// NOLINTEND(readability-identifier-naming, readability-convert-member-functions-to-static)
private:
// Exclusively-owned coroutine handle. Null after move or default
// construction. Destroyed in the destructor.
Handle handle_;
};
/**
* CoroTask<T> -- coroutine return type for value-returning coroutines.
*
* Class / Dependency Diagram
* ==========================
*
* CoroTask<T>
* +-----------------------------------------------+
* | - handle_ : Handle (coroutine_handle<promise>) |
* +-----------------------------------------------+
* | + handle(), done() |
* | + await_ready/suspend/resume (Awaiter iface) |
* +-----------------------------------------------+
* | owns
* v
* promise_type
* +-----------------------------------------------+
* | - result_ : variant<monostate, T, |
* | exception_ptr> |
* | - continuation_ : std::coroutine_handle<> |
* +-----------------------------------------------+
* | + get_return_object() -> CoroTask |
* | + initial_suspend() -> suspend_always (lazy) |
* | + final_suspend() -> FinalAwaiter |
* | + return_value(T) -> stores in result_[1] |
* | + unhandled_exception -> stores in result_[2] |
* +-----------------------------------------------+
* | returns at final_suspend
* v
* FinalAwaiter (same symmetric-transfer pattern as CoroTask<void>)
*
* Value Extraction
* ----------------
* await_resume() inspects the variant:
* - index 2 (exception_ptr) -> rethrow
* - index 1 (T) -> return value via move
*
* Usage Examples
* ==============
*
* 1. Simple value return:
*
* CoroTask<int> computeAnswer() { co_return 42; }
*
* CoroTask<void> caller() {
* int v = co_await computeAnswer(); // v == 42
* }
*
* 2. Chaining value-returning coroutines:
*
* CoroTask<int> add(int a, int b) { co_return a + b; }
* CoroTask<int> doubleSum(int a, int b) {
* int s = co_await add(a, b);
* co_return s * 2;
* }
*
* 3. Exception propagation from inner to outer:
*
* CoroTask<int> failing() {
* throw std::runtime_error("bad");
* co_return 0; // never reached
* }
* CoroTask<void> caller() {
* try {
* int v = co_await failing(); // throws here
* } catch (std::runtime_error const& e) {
* // e.what() == "bad"
* }
* }
*
* Caveats / Pitfalls (in addition to CoroTask<void> caveats above)
* ================================================================
*
* BUG-RISK: await_resume() moves the value out of the variant.
* Calling co_await on the same CoroTask<T> instance twice is undefined
* behavior -- the second call will see a moved-from T. CoroTask is
* single-shot: one co_return, one co_await.
*
* BUG-RISK: T must be move-constructible.
* return_value(T) takes by value and moves into the variant.
* Types that are not movable cannot be used as T.
*
* LIMITATION: No co_yield support.
* CoroTask<T> only supports a single co_return. It does not implement
* yield_value(), so using co_yield inside a CoroTask<T> coroutine is a
* compile error. For streaming values, a different return type
* (e.g. Generator<T>) would be needed.
*
* LIMITATION: Result is only accessible via co_await.
* There is no .get() or .result() method. The value can only be
* extracted by co_await-ing the CoroTask<T> from inside another
* coroutine. For extracting results in non-coroutine code, pass a
* pointer to the caller and write through it (as the tests do).
*/
template <typename T>
class CoroTask
{
static_assert(
std::is_move_constructible_v<T>,
"CoroTask<T> requires T to be move-constructible");
public:
// The C++ coroutine protocol mandates these names (promise_type,
// initial_suspend, await_ready, ...) and instance-callable awaiter
// methods, which conflict with the project naming/static conventions.
// NOLINTBEGIN(readability-identifier-naming, readability-convert-member-functions-to-static)
struct promise_type;
using Handle = std::coroutine_handle<promise_type>;
/**
* Coroutine promise for value-returning coroutines.
* Stores the result as a variant: monostate (not yet set),
* T (co_return value), or exception_ptr (unhandled exception).
*/
struct promise_type
{
// Tri-state result:
// index 0 (monostate) -- coroutine has not yet completed
// index 1 (T) -- co_return value stored here
// index 2 (exception) -- unhandled exception captured here
std::variant<std::monostate, T, std::exception_ptr> result_;
// Handle to the coroutine co_await-ing this task. Used by
// FinalAwaiter for symmetric transfer. Null for top-level tasks.
std::coroutine_handle<> continuation_;
/**
* Create the CoroTask return object.
* Called by the compiler at coroutine creation.
*/
CoroTask
get_return_object()
{
return CoroTask{Handle::from_promise(*this)};
}
/**
* Lazy start. Coroutine body does not run until explicitly resumed.
*/
std::suspend_always
initial_suspend() noexcept
{
return {};
}
/**
* Symmetric-transfer awaiter at coroutine completion.
* Same pattern as CoroTask<void>::FinalAwaiter.
*/
struct FinalAwaiter
{
bool
await_ready() noexcept
{
return false;
}
/**
* Returns continuation for symmetric transfer, or
* noop_coroutine if this is a top-level task.
*
* @param h Handle to this completing coroutine
*
* @return Continuation handle, or noop_coroutine
*/
std::coroutine_handle<>
await_suspend(Handle h) noexcept
{
if (auto cont = h.promise().continuation_)
return cont;
return std::noop_coroutine();
}
void
await_resume() noexcept
{
}
};
FinalAwaiter
final_suspend() noexcept
{
return {};
}
/**
* Called by the compiler for `co_return value;`.
* Moves the value into result_ at index 1.
*
* @param value The value to store
*/
void
return_value(T value)
{
result_.template emplace<1>(std::move(value));
}
/**
* Captures unhandled exceptions at index 2 of result_.
* Rethrown later in await_resume().
*/
void
unhandled_exception()
{
result_.template emplace<2>(std::current_exception());
}
};
// NOLINTEND(readability-identifier-naming, readability-convert-member-functions-to-static)
/**
* Default constructor. Creates an empty (null handle) task.
*/
CoroTask() = default;
/**
* Takes ownership of a compiler-generated coroutine handle.
*
* @param h Coroutine handle to own
*/
explicit CoroTask(Handle h) : handle_(h)
{
}
/**
* Destroys the coroutine frame if this task owns one.
*/
~CoroTask()
{
if (handle_)
handle_.destroy();
}
/**
* Move constructor. Transfers handle ownership, leaves other empty.
*/
CoroTask(CoroTask&& other) noexcept : handle_(std::exchange(other.handle_, {}))
{
}
/**
* Move assignment. Destroys current frame (if any), takes other's.
*/
CoroTask&
operator=(CoroTask&& other) noexcept
{
if (this != &other)
{
if (handle_)
handle_.destroy();
handle_ = std::exchange(other.handle_, {});
}
return *this;
}
CoroTask(CoroTask const&) = delete;
CoroTask&
operator=(CoroTask const&) = delete;
/**
* @return The underlying coroutine_handle
*/
[[nodiscard]] Handle
handle() const
{
return handle_;
}
/**
* @return true if the coroutine has run to completion (or thrown)
*/
[[nodiscard]] bool
done() const
{
return handle_ && handle_.done();
}
// -- Awaiter interface: allows `T val = co_await someCoroTask;` --
/**
* Always false. co_await always suspends to set up continuation.
*/
// NOLINTBEGIN(readability-identifier-naming, readability-convert-member-functions-to-static)
[[nodiscard]] bool
await_ready() const noexcept
{
return false;
}
/**
* Stores caller as continuation, returns our handle for
* symmetric transfer.
*
* @param caller Handle of the coroutine doing co_await on us
*
* @return Our handle for symmetric transfer
*/
std::coroutine_handle<>
await_suspend(std::coroutine_handle<> caller) noexcept
{
XRPL_ASSERT(handle_, "xrpl::CoroTask<T>::await_suspend : handle is valid");
handle_.promise().continuation_ = caller;
return handle_;
}
/**
* Extracts the result: rethrows if exception, otherwise moves
* the T value out of the variant. Single-shot: calling twice
* on the same task is undefined (moved-from T).
*
* @return The co_return-ed value
*/
T
await_resume()
{
XRPL_ASSERT(handle_, "xrpl::CoroTask<T>::await_resume : handle is valid");
auto& result = handle_.promise().result_;
if (auto* ep = std::get_if<2>(&result))
std::rethrow_exception(*ep);
return std::get<1>(std::move(result));
}
// NOLINTEND(readability-identifier-naming, readability-convert-member-functions-to-static)
private:
// Exclusively-owned coroutine handle. Null after move or default
// construction. Destroyed in the destructor.
Handle handle_;
};
} // namespace xrpl
+452
View File
@@ -0,0 +1,452 @@
#pragma once
/**
* @file CoroTaskRunner.ipp
*
* CoroTaskRunner inline implementation.
*
* This file contains the business logic for managing C++20 coroutines
* on the JobQueue. It is included at the bottom of JobQueue.h.
*
* Data Flow: suspend / post / resume cycle
* =========================================
*
* coroutine body CoroTaskRunner JobQueue
* -------------- -------------- --------
* |
* co_await runner->suspend()
* |
* +--- await_suspend ------> onSuspend()
* | ++nSuspend_ ------------> nSuspend_
* | [coroutine is now suspended]
* |
* . (externally or by yieldAndPost())
* .
* +--- (caller calls) -----> post()
* | ++runCount_
* | addJob(resume) ----------> job enqueued
* | |
* | [worker picks up]
* | |
* +--- <----- resume() <-----------------------------------+
* | --nSuspend_ ------> nSuspend_
* | swap in LocalValues (lvs_)
* | task_.handle().resume()
* | |
* | [coroutine body continues here]
* | |
* | swap out LocalValues
* | --runCount_
* | cv_.notify_all()
* v
*
* Thread Safety
* =============
* - mutex_ : guards task_.handle().resume() so that post()-before-suspend
* races cannot resume the coroutine while it is still running.
* (See the race condition discussion in JobQueue.h)
* - mutexRun_ : guards runCount_ counter; used by join() to wait until
* all in-flight resume operations complete.
* - jq_.mutex_: guards nSuspend_ increments/decrements.
*
* Common Mistakes When Modifying This File
* =========================================
*
* 1. Changing lock ordering.
* resume() acquires locks sequentially (never held simultaneously):
* jq_.mutex_ (released immediately), then mutex_ (held across resume),
* then mutexRun_ (released after decrement). post() acquires only
* mutexRun_. Any new code path must follow the same order.
*
* 2. Removing the shared_from_this() capture in post().
* The lambda passed to addJob captures [this, sp = shared_from_this()].
* If you remove sp, 'this' can be destroyed before the job runs,
* causing use-after-free. The sp capture is load-bearing.
*
* 3. Forgetting to decrement nSuspend_ on a new code path.
* Every ++nSuspend_ must have a matching --nSuspend_. If you add a new
* suspension path (e.g. a new awaiter) and forget to decrement on resume
* or on failure, JobQueue::stop() will hang.
*
* 4. Calling task_.handle().resume() without holding mutex_.
* This allows a race where the coroutine runs on two threads
* simultaneously. Always hold mutex_ around resume().
*
* 5. Swapping LocalValues outside of the mutex_ critical section.
* The swap-in and swap-out of LocalValues must bracket the resume()
* call. If you move the swap-out before the lock_guard(mutex_) is
* released, you break LocalValue isolation for any code that runs
* after the coroutine suspends but before the lock is dropped.
*/
namespace xrpl {
/**
* Construct a CoroTaskRunner. Sets runCount_ to 0; does not
* create the coroutine. Call init() afterwards.
*
* @param jq The JobQueue this coroutine will run on
* @param type Job type for scheduling priority
* @param name Human-readable name for logging
*/
inline JobQueue::CoroTaskRunner::CoroTaskRunner(
CreateT,
JobQueue& jq,
JobType type,
std::string name)
: jq_(jq), type_(type), name_(std::move(name))
{
}
/**
* Initialize with a coroutine-returning callable.
* Stores the callable on the heap (FuncStore) so it outlives the
* coroutine frame. Coroutine frames store a reference to the
* callable's implicit object parameter (the lambda). If the callable
* is a temporary, that reference dangles after the caller returns.
* Keeping the callable alive here ensures the coroutine's captures
* remain valid.
*
* @param f Callable: CoroTask<void>(shared_ptr<CoroTaskRunner>)
*/
template <class F>
void
JobQueue::CoroTaskRunner::init(F&& f)
{
using Fn = std::decay_t<F>;
auto store = std::make_unique<FuncStore<Fn>>(std::forward<F>(f));
task_ = store->func(shared_from_this());
storedFunc_ = std::move(store);
}
/**
* Destructor. Waits for any in-flight resume() to complete, then
* asserts (debug) that the coroutine has finished or
* expectEarlyExit() was called.
*
* The join() call is necessary because with async dispatch the
* coroutine runs on a worker thread. The gate signal (which wakes
* the test thread) can arrive before resume() has set finished_.
* join() synchronizes via mutexRun_, establishing a happens-before
* edge: finished_ = true -> unlock(mutexRun_) in resume() ->
* lock(mutexRun_) in join() -> read finished_.
*/
inline JobQueue::CoroTaskRunner::~CoroTaskRunner()
{
#ifndef NDEBUG
join();
XRPL_ASSERT(
finished_.load(std::memory_order_acquire),
"xrpl::JobQueue::CoroTaskRunner::~CoroTaskRunner : is finished");
#endif
}
/**
* Increment the JobQueue's suspended-coroutine count (nSuspend_).
*/
inline void
JobQueue::CoroTaskRunner::onSuspend()
{
std::scoped_lock const lock(jq_.mutex_);
++jq_.nSuspend_;
}
/**
* Decrement nSuspend_ without resuming.
*/
inline void
JobQueue::CoroTaskRunner::onUndoSuspend()
{
std::scoped_lock const lock(jq_.mutex_);
--jq_.nSuspend_;
}
/**
* Return a SuspendAwaiter whose await_suspend() increments nSuspend_
* before the coroutine actually suspends. The caller must later call
* post() or resume() to continue execution.
*
* @return Awaiter for use with `co_await runner->suspend()`
*/
inline auto
JobQueue::CoroTaskRunner::suspend()
{
/**
* Custom awaiter for suspend(). Always suspends (await_ready
* returns false) and increments nSuspend_ in await_suspend().
*/
// The C++ coroutine protocol mandates these awaiter names and
// instance-callable methods, which conflict with the project
// naming/static conventions.
// NOLINTBEGIN(readability-identifier-naming, readability-convert-member-functions-to-static)
struct SuspendAwaiter
{
CoroTaskRunner& runner_; // The runner that owns this coroutine.
/**
* Always returns false so the coroutine suspends.
*/
[[nodiscard]] bool
await_ready() const noexcept
{
return false;
}
/**
* Called when the coroutine suspends. Increments nSuspend_
* so the JobQueue knows a coroutine is waiting.
*/
void
await_suspend(std::coroutine_handle<>) const
{
runner_.onSuspend();
}
void
await_resume() const noexcept
{
}
};
// NOLINTEND(readability-identifier-naming, readability-convert-member-functions-to-static)
return SuspendAwaiter{*this};
}
/**
* Suspend and immediately repost on the JobQueue. Equivalent to
* `co_await JobQueueAwaiter{runner}` but uses an inline struct
* to work around a GCC-12 codegen bug (see declaration in JobQueue.h).
*
* If the JobQueue is stopping (post fails), the suspend count is
* undone and the coroutine continues immediately via symmetric
* transfer back to its own handle.
*
* @return An inline YieldPostAwaiter
*/
inline auto
JobQueue::CoroTaskRunner::yieldAndPost()
{
// The C++ coroutine protocol mandates these awaiter names and
// instance-callable methods, which conflict with the project
// naming/static conventions.
// NOLINTBEGIN(readability-identifier-naming, readability-convert-member-functions-to-static)
struct YieldPostAwaiter
{
CoroTaskRunner& runner_;
[[nodiscard]] bool
await_ready() const noexcept
{
return false;
}
/**
* Returns a coroutine_handle<> (symmetric transfer) rather than
* void + h.resume(). Two reasons:
*
* 1. h.resume() runs the coroutine nested inside this frame. A
* coroutine that yields in a loop against a stopping JobQueue
* fails post() every iteration, so the stack grows without
* bound. Symmetric transfer is a tail call and does not nest.
*
* 2. After h.resume() returns, the coroutine may have completed
* and destroyed its frame -- the frame this awaiter lives in.
* Returning from await_suspend would then touch freed memory.
*
* A bool return would also avoid nesting, but GCC-12 miscompiles
* bool-returning await_suspend (see JobQueueAwaiter.h).
*
* @return noop_coroutine() to stay suspended (job posted);
* the caller's handle to continue now (JQ stopping)
*/
std::coroutine_handle<>
await_suspend(std::coroutine_handle<> h)
{
runner_.onSuspend();
if (!runner_.post())
{
runner_.onUndoSuspend();
return h;
}
return std::noop_coroutine();
}
void
await_resume() const noexcept
{
}
};
// NOLINTEND(readability-identifier-naming, readability-convert-member-functions-to-static)
return YieldPostAwaiter{*this};
}
/**
* Schedule coroutine resumption as a job on the JobQueue.
* A shared_ptr capture (sp) prevents this CoroTaskRunner from being
* destroyed while the job is queued but not yet executed.
*
* @return false if the JobQueue rejected the job (shutting down)
*/
inline bool
JobQueue::CoroTaskRunner::post()
{
{
std::scoped_lock const lk(mutexRun_);
++runCount_;
}
// sp prevents 'this' from being destroyed while the job is pending
if (jq_.addJob(type_, name_, [this, sp = shared_from_this()]() { resume(); }))
{
return true;
}
// The coroutine will not run. Undo the runCount_ increment.
std::scoped_lock const lk(mutexRun_);
--runCount_;
cv_.notify_all();
return false;
}
/**
* Resume the coroutine on the current thread.
*
* Steps:
* 1. Decrement nSuspend_ (under jq_.mutex_)
* 2. Swap in this coroutine's LocalValues for thread-local isolation
* 3. Resume the coroutine handle (under mutex_)
* 4. Swap out LocalValues, restoring the thread's previous state
* 5. Decrement runCount_ and notify join() waiters
*
* @pre post() must have been called before resume(). Direct calls
* without a prior post() will corrupt runCount_ and break join().
* Note: runCount_ is NOT incremented here — post() already did that.
* This ensures join() stays blocked for the entire post->resume lifetime.
*/
inline void
JobQueue::CoroTaskRunner::resume()
{
{
std::scoped_lock const lock(jq_.mutex_);
--jq_.nSuspend_;
}
auto saved = detail::getLocalValues().release();
detail::getLocalValues().reset(&lvs_);
std::scoped_lock const lock(mutex_);
XRPL_ASSERT(
task_.handle() && !task_.done(),
"xrpl::JobQueue::CoroTaskRunner::resume : task handle is valid and not done");
if (task_.handle() && !task_.done())
{
task_.handle().resume();
}
else
{
// A resume() with no coroutine to run (e.g. a duplicate external
// post() after completion). Resuming a null or finished handle is
// undefined behavior, so skip it -- this matches the old
// Coro::resume() `if (coro_)` guard. The bookkeeping below still
// runs to balance the ++runCount_ done by the post() that
// scheduled this call.
JLOG(jq_.journal_.warn())
<< "CoroTaskRunner::resume called for coroutine '" << name_
<< "' with no runnable coroutine (duplicate post or already completed)";
}
detail::getLocalValues().release();
detail::getLocalValues().reset(saved);
if (task_.done())
{
finished_.store(true, std::memory_order_release);
// An exception that escapes a top-level coroutine body is captured
// by promise_type::unhandled_exception() but has no awaiter to
// rethrow it, so it would vanish with the frame. Surface it in the
// log. (The old Boost path propagated it out of resume() instead.)
if (auto const& ep = task_.handle().promise().exception_)
{
try
{
std::rethrow_exception(ep);
}
catch (std::exception const& e)
{
JLOG(jq_.journal_.error())
<< "Unhandled exception in coroutine '" << name_ << "': " << e.what();
}
catch (...)
{
JLOG(jq_.journal_.error())
<< "Unhandled non-standard exception in coroutine '" << name_ << "'";
}
}
// Break the shared_ptr cycle: frame -> shared_ptr<runner> -> this.
// Use std::move (not task_ = {}) so task_.handle_ is null BEFORE the
// frame is destroyed. operator= would destroy the frame while handle_
// still holds the old value -- a re-entrancy hazard on GCC-12 if
// frame destruction triggers runner cleanup.
[[maybe_unused]] auto completed = std::move(task_);
}
std::scoped_lock const lk(mutexRun_);
--runCount_;
cv_.notify_all();
}
/**
* @return true if the coroutine has not yet run to completion.
*
* Uses the atomic finished_ flag instead of reading task_ directly,
* because task_ is modified in resume() under mutex_ and reading it
* here without a lock would be a data race visible to TSAN.
*/
inline bool
JobQueue::CoroTaskRunner::runnable() const
{
return !finished_.load(std::memory_order_acquire);
}
/**
* Handle early termination when the coroutine never ran (e.g. JobQueue
* is stopping). Decrements nSuspend_ and destroys the coroutine frame
* to break the shared_ptr cycle: frame -> lambda -> runner -> frame.
*/
inline void
JobQueue::CoroTaskRunner::expectEarlyExit()
{
if (!finished_.load(std::memory_order_acquire))
{
std::scoped_lock const lock(jq_.mutex_);
--jq_.nSuspend_;
finished_.store(true, std::memory_order_release);
}
// Break the shared_ptr cycle: frame -> shared_ptr<runner> -> this.
// The coroutine is at initial_suspend and never ran user code, so
// destroying it is safe. Use std::move (not task_ = {}) so
// task_.handle_ is null before the frame is destroyed.
{
[[maybe_unused]] auto completed = std::move(task_);
}
storedFunc_.reset();
}
/**
* Block until all pending/active resume operations complete.
* Uses cv_ + mutexRun_ to wait until runCount_ reaches 0 or
* finished_ becomes true. The finished_ check handles the case
* where resume() is called directly (without post()), which
* decrements runCount_ below zero. In that scenario runCount_
* never returns to 0, but finished_ becoming true guarantees
* the coroutine is done and no more resumes will occur.
*
* Note: when join() returns via the finished_ disjunct, the final
* resume() call may still be executing its post-completion
* bookkeeping (the --runCount_ / notify after finished_ is set).
* That is safe -- the coroutine body has fully completed and the
* runner is kept alive by the resume job's shared_ptr -- but
* callers must not assume resume() itself has returned.
*/
inline void
JobQueue::CoroTaskRunner::join()
{
std::unique_lock<std::mutex> lk(mutexRun_);
cv_.wait(lk, [this]() { return runCount_ == 0 || finished_.load(std::memory_order_acquire); });
}
} // namespace xrpl
+464 -147
View File
@@ -1,25 +1,17 @@
#pragma once
#include <xrpl/basics/LocalValue.h>
#include <xrpl/core/ClosureCounter.h>
#include <xrpl/core/JobTypeData.h>
#include <xrpl/core/detail/Workers.h>
#include <xrpl/json/json_value.h>
// Include only the specific Boost.Coroutine2 headers actually used here.
// Avoid `boost/coroutine2/all.hpp` because it transitively pulls in
// `boost/context/pooled_fixedsize_stack.hpp`, whose `.malloc()` / `.free()`
// member calls on `boost::pool` collide with MSVC's `_CRTDBG_MAP_ALLOC` macros
// in Debug builds (see cmake/XrplCompiler.cmake).
#include <xrpl/beast/insight/Collector.h>
#include <xrpl/beast/insight/Gauge.h>
#include <xrpl/beast/insight/Hook.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/core/ClosureCounter.h>
#include <xrpl/core/CoroTask.h>
#include <xrpl/core/Job.h>
#include <xrpl/core/JobTypeData.h>
#include <xrpl/core/LoadEvent.h>
#include <boost/context/protected_fixedsize_stack.hpp>
#include <boost/coroutine2/coroutine.hpp>
#include <xrpl/core/detail/Workers.h>
#include <xrpl/json/json_value.h>
#include <atomic>
#include <chrono>
@@ -40,10 +32,6 @@ class PerfLog;
} // namespace perf
class Logs;
struct CoroCreateT
{
explicit CoroCreateT() = default;
};
/**
* A pool of threads to perform work.
@@ -60,93 +48,417 @@ class JobQueue : private Workers::Callback
{
public:
/**
* Coroutines must run to completion.
* C++20 coroutine lifecycle manager.
*
* Class / Inheritance / Dependency Diagram
* =========================================
*
* std::enable_shared_from_this<CoroTaskRunner>
* ^
* | (public inheritance)
* |
* CoroTaskRunner
* +---------------------------------------------------+
* | - lvs_ : detail::LocalValues |
* | - jq_ : JobQueue& |
* | - type_ : JobType |
* | - name_ : std::string |
* | - runCount_ : int (in-flight resumes) |
* | - mutex_ : std::mutex (coroutine guard) |
* | - mutexRun_ : std::mutex (join guard) |
* | - cv_ : condition_variable |
* | - task_ : CoroTask<void> |
* | - storedFunc_ : unique_ptr<FuncBase> (type-erased)|
* +---------------------------------------------------+
* | + init(F&&) : set up coroutine callable |
* | + onSuspend() : ++jq_.nSuspend_ |
* | + onUndoSuspend() : --jq_.nSuspend_ |
* | + suspend() : returns SuspendAwaiter |
* | + post() : schedule resume on JobQueue |
* | + resume() : resume coroutine on caller |
* | + runnable() : !task_.done() |
* | + expectEarlyExit() : teardown for failed post |
* | + join() : block until not running |
* +---------------------------------------------------+
* | |
* | owns | references
* v v
* CoroTask<void> JobQueue
* (coroutine frame) (thread pool + nSuspend_)
*
* FuncBase / FuncStore<F> (type-erased heap storage
* for the coroutine lambda)
*
* Coroutine Lifecycle (Control Flow)
* ===================================
*
* Caller thread JobQueue worker thread
* ------------- ----------------------
* postCoroTask(f)
* |
* +-- reserve a jobCounter_ slot (reject if JQ shutting down)
* +-- ++nSuspend_ (lazy start counts as suspended)
* +-- make_shared<CoroTaskRunner>
* +-- init(f)
* | +-- store lambda on heap (FuncStore)
* | +-- task_ = f(shared_from_this())
* | [coroutine created, suspended at initial_suspend]
* +-- post()
* | +-- ++runCount_
* | +-- addJob(type_, [resume]{})
* | resume()
* | |
* | +-- --nSuspend_
* | +-- swap in LocalValues
* | +-- task_.handle().resume()
* | | [coroutine body runs]
* | | ...
* | | co_await suspend()
* | | +-- ++nSuspend_
* | | [coroutine suspends]
* | +-- swap out LocalValues
* | +-- --runCount_
* | +-- cv_.notify_all()
* |
* post() <-- called externally or by yieldAndPost()
* +-- ++runCount_
* +-- addJob(type_, [resume]{})
* resume()
* |
* +-- [coroutine body continues]
* +-- co_return
* +-- --runCount_
* +-- cv_.notify_all()
* join()
* +-- cv_.wait([]{runCount_ == 0})
* +-- [done]
*
* Usage Examples
* ==============
*
* 1. Fire-and-forget coroutine (most common pattern):
*
* jq.postCoroTask(JtClient, "MyWork",
* [](auto runner) -> CoroTask<void> {
* doSomeWork();
* co_await runner->suspend(); // yield to other jobs
* doMoreWork();
* co_return;
* });
*
* 2. Manually controlling suspend / resume (external trigger):
*
* auto runner = jq.postCoroTask(JtClient, "ExtTrigger",
* [&result](auto runner) -> CoroTask<void> {
* startAsyncOperation(callback);
* co_await runner->suspend();
* // callback called runner->post() to get here
* result = collectResult();
* co_return;
* });
* // ... later, from the callback:
* runner->post(); // reschedule the coroutine on the JobQueue
*
* 3. Using yieldAndPost() for automatic suspend + repost:
*
* jq.postCoroTask(JtClient, "AutoRepost",
* [](auto runner) -> CoroTask<void> {
* step1();
* co_await runner->yieldAndPost(); // yield + auto-repost
* step2();
* co_await runner->yieldAndPost();
* step3();
* co_return;
* });
*
* 4. Checking shutdown after co_await (cooperative cancellation):
*
* jq.postCoroTask(JtClient, "Cancellable",
* [&jq](auto runner) -> CoroTask<void> {
* while (moreWork()) {
* co_await runner->yieldAndPost();
* if (jq.isStopping())
* co_return; // bail out cleanly
* processNextItem();
* }
* co_return;
* });
*
* Caveats / Pitfalls
* ==================
*
* BUG-RISK: Calling suspend() without a matching post()/resume().
* After co_await runner->suspend(), the coroutine is parked and
* nSuspend_ is incremented. If nothing ever calls post() or
* resume(), the coroutine is leaked and JobQueue::stop() will
* hang forever waiting for nSuspend_ to reach zero.
*
* BUG-RISK: Calling post() on an already-running coroutine.
* post() schedules a resume() job. If the coroutine has not
* actually suspended yet (no co_await executed), the resume job
* will try to call handle().resume() while the coroutine is still
* running on another thread. This is UB. The mutex_ prevents
* data corruption but the logic is wrong — always co_await
* suspend() before calling post(). (The test incorrect_order()
* shows this works only because mutex_ serializes the calls.)
*
* BUG-RISK: Dropping the shared_ptr<CoroTaskRunner> before join().
* The CoroTaskRunner destructor asserts that finished_ is true
* (the coroutine completed). If you let the last shared_ptr die
* while the coroutine is still running or suspended, you get an
* assertion failure in debug and UB in release. Always call
* join() or expectEarlyExit() first.
*
* BUG-RISK: Lambda captures outliving the coroutine frame.
* The lambda passed to postCoroTask is heap-allocated (FuncStore)
* to prevent dangling. But objects captured by pointer still need
* their own lifetime management. If you capture a raw pointer to
* a stack variable, and the stack frame exits before the coroutine
* finishes, the pointer dangles. Use shared_ptr or ensure the
* pointed-to object outlives the coroutine.
*
* BUG-RISK: Forgetting co_return in a void coroutine.
* If the coroutine body falls off the end without co_return,
* the compiler may silently treat it as co_return (per standard),
* but some compilers warn. Always write explicit co_return.
*
* LIMITATION: CoroTaskRunner only supports CoroTask<void>.
* The task_ member is CoroTask<void>. To return values from
* the top-level coroutine, write through a captured pointer
* (as the tests demonstrate), or co_await inner CoroTask<T>
* coroutines that return values.
*
* LIMITATION: One coroutine per CoroTaskRunner.
* init() must be called exactly once. You cannot reuse a
* CoroTaskRunner to run a second coroutine. Create a new one
* via postCoroTask() instead.
*
* LIMITATION: No timeout on join().
* join() blocks indefinitely. If the coroutine is suspended
* and never posted, join() will deadlock. Use timed waits
* on the gate pattern (condition_variable + wait_for) in tests.
*/
class Coro : public std::enable_shared_from_this<Coro>
class CoroTaskRunner : public std::enable_shared_from_this<CoroTaskRunner>
{
private:
// Per-coroutine thread-local storage. Swapped in before resume()
// and swapped out after, so each coroutine sees its own LocalValue
// state regardless of which worker thread executes it.
detail::LocalValues lvs_;
// Back-reference to the owning JobQueue. Used to post jobs,
// increment/decrement nSuspend_, and acquire jq_.mutex_.
JobQueue& jq_;
// Job type passed to addJob() when posting this coroutine.
JobType type_;
// Human-readable name for this coroutine job (for logging).
std::string name_;
bool running_{false};
// Number of in-flight resume operations (pending + active).
// Incremented by post(), decremented when resume() finishes.
// Guarded by mutexRun_. join() blocks until this reaches 0.
//
// A counter (not a bool) is needed because post() can be called
// from within the coroutine body (e.g. via yieldAndPost()),
// enqueuing a second resume while the first is still running.
// A bool would be clobbered: R2.post() sets true, then R1's
// cleanup sets false — losing the fact that R2 is still pending.
int runCount_ = 0;
// Serializes all coroutine resume() calls, preventing concurrent
// execution of the coroutine body on multiple threads. Handles the
// race where post() enqueues a resume before the coroutine has
// actually suspended (post-before-suspend pattern).
std::mutex mutex_;
// Guards runCount_. Used with cv_ for join() to wait
// until all pending/active resume operations complete.
std::mutex mutexRun_;
// Notified when runCount_ reaches zero, allowing
// join() waiters to wake up.
std::condition_variable cv_;
boost::coroutines2::coroutine<void>::push_type* yield_{};
boost::coroutines2::coroutine<void>::pull_type coro_;
#ifndef NDEBUG
bool finished_ = false;
#endif
// The coroutine handle wrapper. Owns the coroutine frame.
// Set by init(). Reset to empty in resume() upon coroutine
// completion (to break the shared_ptr cycle) or in
// expectEarlyExit() on early termination.
CoroTask<void> task_;
/**
* Type-erased base for heap-stored callables.
* Prevents the coroutine lambda from being destroyed before
* the coroutine frame is done with it.
*
* @see FuncStore
*/
struct FuncBase
{
virtual ~FuncBase() = default;
};
/**
* Concrete type-erased storage for a callable of type F.
* The coroutine frame stores a reference to the lambda's implicit
* object parameter. If the lambda is a temporary, that reference
* dangles after the call returns. FuncStore keeps it alive on
* the heap for the lifetime of the CoroTaskRunner.
*/
template <class F>
struct FuncStore : FuncBase
{
F func; // The stored callable (coroutine lambda).
explicit FuncStore(F&& f) : func(std::move(f))
{
}
};
// Heap-allocated callable storage. Set by init(), ensures the
// lambda outlives the coroutine frame that references it.
std::unique_ptr<FuncBase> storedFunc_;
// True once the coroutine has completed or expectEarlyExit() was
// called. Asserted in the destructor (debug) to catch leaked
// runners. Available in all builds to guard expectEarlyExit()
// against double-decrementing nSuspend_.
// Atomic to allow lock-free reads from runnable(), join(), and
// the destructor without requiring the same mutex that guards
// the write in resume().
std::atomic<bool> finished_{false};
public:
template <class F>
Coro(CoroCreateT, JobQueue&, JobType, std::string, F&&);
// Not copy-constructible or assignable
Coro(Coro const&) = delete;
Coro&
operator=(Coro const&) = delete;
~Coro();
/**
* Tag type for private construction. Prevents external code
* from constructing CoroTaskRunner directly. Use postCoroTask().
*/
struct CreateT
{
explicit CreateT() = default;
};
/**
* Suspend coroutine execution.
* Effects:
* The coroutine's stack is saved.
* The associated Job thread is released.
* Note:
* The associated Job function returns.
* Undefined behavior if called consecutively without a corresponding
* post.
* Construct a CoroTaskRunner. Private by convention (CreateT tag).
*
* @param jq The JobQueue this coroutine will run on
* @param type Job type for scheduling priority
* @param name Human-readable name for logging
*/
CoroTaskRunner(CreateT, JobQueue&, JobType, std::string);
CoroTaskRunner(CoroTaskRunner const&) = delete;
CoroTaskRunner&
operator=(CoroTaskRunner const&) = delete;
/**
* Destructor. Asserts (debug) that the coroutine has finished
* or expectEarlyExit() was called.
*/
~CoroTaskRunner();
/**
* Initialize with a coroutine-returning callable.
* Must be called exactly once, after the object is managed by
* shared_ptr (because init uses shared_from_this internally).
* This is handled automatically by postCoroTask().
*
* @param f Callable: CoroTask<void>(shared_ptr<CoroTaskRunner>)
*/
template <class F>
void
init(F&& f);
/**
* Increment the JobQueue's suspended-coroutine count (nSuspend_).
* Called when the coroutine is about to suspend. Every call
* must be balanced by a corresponding decrement (via resume()
* or onUndoSuspend()), or JobQueue::stop() will hang.
*/
void
yield() const;
onSuspend();
/**
* Schedule coroutine execution.
* Effects:
* Returns immediately.
* A new job is scheduled to resume the execution of the coroutine.
* When the job runs, the coroutine's stack is restored and execution
* continues at the beginning of coroutine function or the
* statement after the previous call to yield. Undefined behavior if
* called after the coroutine has completed with a return (as opposed to
* a yield()). Undefined behavior if post() or resume() called
* consecutively without a corresponding yield.
* Decrement nSuspend_ without resuming.
* Used to undo onSuspend() when a scheduled post() fails
* (e.g. JobQueue is stopping).
*/
void
onUndoSuspend();
/**
* Suspend the coroutine.
* The awaiter's await_suspend() increments nSuspend_ before the
* coroutine actually suspends. The caller must later call post()
* or resume() to continue execution.
*
* @return true if the Coro's job is added to the JobQueue.
* @return An awaiter for use with `co_await runner->suspend()`
*/
auto
suspend();
/**
* Suspend the coroutine and immediately repost it on the
* JobQueue. Combines suspend() + post() atomically inside
* await_suspend, so there is no window where an external
* event could race between the two.
*
* Equivalent to JobQueueAwaiter but defined as an inline
* awaiter returned from a member function. This avoids a
* GCC-12 coroutine codegen bug where an external awaiter
* struct (JobQueueAwaiter) used at multiple co_await points
* corrupts the coroutine state machine's resume index,
* causing the coroutine to hang on the third resumption.
*
* @return An awaiter for use with `co_await runner->yieldAndPost()`
*/
auto
yieldAndPost();
/**
* Schedule coroutine resumption as a job on the JobQueue.
* Captures shared_from_this() to prevent this runner from being
* destroyed while the job is queued.
*
* @return true if the job was accepted; false if the JobQueue
* is stopping (caller must handle cleanup)
*/
bool
post();
/**
* Resume coroutine execution.
* Effects:
* The coroutine continues execution from where it last left off
* using this same thread.
* If the coroutine has already completed, returns immediately
* (handles the documented post-before-yield race condition).
* Undefined behavior if resume() or post() called consecutively
* without a corresponding yield.
* Resume the coroutine on the current thread.
* Decrements nSuspend_, swaps in LocalValues, resumes the
* coroutine handle, swaps out LocalValues, and notifies join()
* waiters. Lock ordering (sequential, non-overlapping):
* jq_.mutex_ -> mutex_ -> mutexRun_.
*
* @pre post() must have been called before resume(). Direct
* calls without a prior post() will corrupt runCount_
* and break join().
*/
void
resume();
/**
* Returns true if the Coro is still runnable (has not returned).
* @return true if the coroutine has not yet run to completion
*/
[[nodiscard]] bool
bool
runnable() const;
/**
* Once called, the Coro allows early exit without an assert.
* Handle early termination when the coroutine never ran.
* Decrements nSuspend_ and destroys the coroutine frame to
* break the shared_ptr cycle (frame -> lambda -> runner -> frame).
* Called by postCoroTask() when post() fails.
*/
void
expectEarlyExit();
/**
* Waits until coroutine returns from the user function.
* Block until all pending/active resume operations complete.
* Uses cv_ + mutexRun_ to wait until runCount_ reaches 0.
* Warning: deadlocks if the coroutine is suspended and never posted.
*/
void
join();
@@ -184,18 +496,18 @@ public:
}
/**
* Creates a coroutine and adds a job to the queue which will run it.
* Creates a C++20 coroutine and adds a job to the queue to run it.
*
* @param t The type of job.
* @param name Name of the job.
* @param f Has a signature of void(std::shared_ptr<Coro>). Called when the
* job executes.
* @param f Callable with signature
* CoroTask<void>(std::shared_ptr<CoroTaskRunner>).
*
* @return shared_ptr to posted Coro. nullptr if post was not successful.
* @return shared_ptr to posted CoroTaskRunner. nullptr if not successful.
*/
template <class F>
std::shared_ptr<Coro>
postCoro(JobType t, std::string const& name, F&& f);
std::shared_ptr<CoroTaskRunner>
postCoroTask(JobType t, std::string const& name, F&& f);
/**
* Jobs waiting at this priority.
@@ -256,8 +568,6 @@ public:
isStopped() const;
private:
friend class Coro;
using JobDataMap = std::map<JobType, JobTypeData>;
beast::Journal journal_;
@@ -358,88 +668,95 @@ private:
getJobLimit(JobType type);
};
/*
An RPC command is received and is handled via ServerHandler(HTTP) or
Handler(websocket), depending on the connection type. The handler then calls
the JobQueue::postCoro() method to create a coroutine and run it at a later
point. This frees up the handler thread and allows it to continue handling
other requests while the RPC command completes its work asynchronously.
postCoro() creates a Coro object. When the Coro ctor is called, and its
coro_ member is initialized (a boost::coroutines::pull_type), execution
automatically passes to the coroutine, which we don't want at this point,
since we are still in the handler thread context. It's important to note
here that construction of a boost pull_type automatically passes execution to
the coroutine. A pull_type object automatically generates a push_type that is
passed as a parameter (do_yield) in the signature of the function the
pull_type was created with. This function is immediately called during coro_
construction and within it, Coro::yield_ is assigned the push_type
parameter (do_yield) address and called (yield()) so we can return execution
back to the caller's stack.
postCoro() then calls Coro::post(), which schedules a job on the job
queue to continue execution of the coroutine in a JobQueue worker thread at
some later time. When the job runs, we lock on the Coro::mutex_ and call
coro_ which continues where we had left off. Since we the last thing we did
in coro_ was call yield(), the next thing we continue with is calling the
function param f, that was passed into Coro ctor. It is within this
function body that the caller specifies what he would like to do while
running in the coroutine and allow them to suspend and resume execution.
A task that relies on other events to complete, such as path finding, calls
Coro::yield() to suspend its execution while waiting on those events to
complete and continue when signaled via the Coro::post() method.
There is a potential race condition that exists here where post() can get
called before yield() after f is called. Technically the problem only occurs
if the job that post() scheduled is executed before yield() is called.
If the post() job were to be executed before yield(), undefined behavior
would occur. The lock ensures that coro_ is not called again until we exit
the coroutine. At which point a scheduled resume() job waiting on the lock
would gain entry. resume() checks if the coroutine has already completed
(coro_ converts to false) and, if so, skips invoking operator() since
calling operator() on a completed boost::coroutine2 pull_type is undefined
behavior.
The race condition occurs as follows:
1- The coroutine is running.
2- The coroutine is about to suspend, but before it can do so, it must
arrange for some event to wake it up.
3- The coroutine arranges for some event to wake it up.
4- Before the coroutine can suspend, that event occurs and the
resumption of the coroutine is scheduled on the job queue. 5- Again, before
the coroutine can suspend, the resumption of the coroutine is dispatched. 6-
Again, before the coroutine can suspend, the resumption code runs the
coroutine.
The coroutine is now running in two threads.
The lock prevents this from happening as step 6 will block until the
lock is released which only happens after the coroutine completes.
*/
} // namespace xrpl
#include <xrpl/core/Coro.ipp> // IWYU pragma: keep
#include <xrpl/core/CoroTaskRunner.ipp> // IWYU pragma: keep
namespace xrpl {
// postCoroTask — entry point for launching a C++20 coroutine on the JobQueue.
//
// Control Flow
// ============
//
// postCoroTask(t, name, f)
// |
// +-- 1. Reserve a jobCounter_ slot — reject if JQ shutting down
// |
// +-- 2. ++nSuspend_ (the coroutine uses lazy-start, so it is
// | "suspended" from the JQ's perspective before its first resume.
// | This keeps the JQ shutdown logic correct — it waits for
// | nSuspend_ to reach 0).
// |
// +-- 3. Create CoroTaskRunner (shared_ptr, ref-counted)
// |
// +-- 4. runner->init(f)
// | +-- Heap-allocate the lambda (FuncStore) to prevent
// | | dangling captures in the coroutine frame
// | +-- task_ = f(shared_from_this())
// | [coroutine created but NOT started — lazy initial_suspend]
// |
// +-- 5. runner->post()
// | +-- addJob(type_, [resume]{}) → resume on worker thread
// | +-- failure (JQ stopping):
// | +-- runner->expectEarlyExit()
// | | --nSuspend_, destroy coroutine frame
// | +-- return nullptr
// |
// +-- 6. Release the jobCounter_ slot (on return)
//
// Why reserve a jobCounter_ slot?
// ===============================
// JobQueue::stop() joins jobCounter_ before it acquires mutex_ and asserts
// nSuspend_ == 0. Without a reservation, stop() could run to completion in
// the window between the ++nSuspend_ in step 2 and the balancing post() or
// expectEarlyExit() in step 5, tripping that assert. Holding a slot blocks
// stop()'s join() for the whole function, closing the window. wrap() also
// returns nullopt once the counter is joined, so it doubles as the shutdown
// check — a plain stopping_ read cannot do this, because the read and the
// ++nSuspend_ are not a single atomic step.
//
// Why async post() instead of synchronous resume()?
// ==================================================
// The initial dispatch MUST use async post() so the coroutine body runs on
// a JobQueue worker thread, not the caller's thread. resume() swaps the
// caller's thread-local LocalValues with the coroutine's private copy.
// If the coroutine mutates LocalValues (e.g. thread_specific_storage test),
// those mutations bleed back into the caller's thread-local state after the
// swap-out, corrupting subsequent tests that share the same thread pool.
// Async post() avoids this by running the coroutine on a worker thread whose
// LocalValues are managed by the thread pool, not by the caller.
//
template <class F>
std::shared_ptr<JobQueue::Coro>
JobQueue::postCoro(JobType t, std::string const& name, F&& f)
std::shared_ptr<JobQueue::CoroTaskRunner>
JobQueue::postCoroTask(JobType t, std::string const& name, F&& f)
{
/* First param is a detail type to make construction private.
Last param is the function the coroutine runs. Signature of
void(std::shared_ptr<Coro>).
*/
auto coro = std::make_shared<Coro>(CoroCreateT{}, *this, t, name, std::forward<F>(f));
if (!coro->post())
// Held until this function returns. Null once jobCounter_ is joined,
// which is how a shutting-down JobQueue rejects new coroutines.
auto const shutdownGuard = jobCounter_.wrap([]() {});
if (!shutdownGuard)
return nullptr;
// Account for the initial suspension (CoroTask uses lazy start). Reject
// if the JQ is shutting down and increment nSuspend_ under the same
// lock. Without the lock, a TOCTOU race exists: stopping_ could become
// true between the check and the increment, leaving an orphan nSuspend_
// that causes stop() to hang.
{
// The Coro was not successfully posted. Disable it so it's destructor
// can run with no negative side effects. Then destroy it.
coro->expectEarlyExit();
coro.reset();
std::scoped_lock const lock(mutex_);
if (stopping_)
return nullptr;
++nSuspend_;
}
return coro;
auto runner = std::make_shared<CoroTaskRunner>(CoroTaskRunner::CreateT{}, *this, t, name);
runner->init(std::forward<F>(f));
if (!runner->post())
{
runner->expectEarlyExit();
runner.reset();
}
return runner;
}
} // namespace xrpl
+212
View File
@@ -0,0 +1,212 @@
#pragma once
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/core/JobQueue.h>
#include <coroutine>
#include <memory>
namespace xrpl {
/**
* Awaiter that suspends and immediately reschedules on the JobQueue.
* Equivalent to calling yield() followed by post() in the old Coro API.
*
* Usage:
* co_await JobQueueAwaiter{runner};
*
* What it waits for: The coroutine is re-queued as a job and resumes
* when a worker thread picks it up.
*
* Which thread resumes: A JobQueue worker thread.
*
* What await_resume() returns: void.
*
* Dependency Diagram
* ==================
*
* JobQueueAwaiter
* +----------------------------------------------+
* | + runner : shared_ptr<CoroTaskRunner> |
* +----------------------------------------------+
* | + await_ready() -> false (always suspend) |
* | + await_suspend() -> bool (suspend or cancel) |
* | + await_resume() -> void |
* +----------------------------------------------+
* | |
* | uses | uses
* v v
* CoroTaskRunner JobQueue
* .onSuspend() (via runner->post() -> addJob)
* .onUndoSuspend()
* .post()
*
* Control Flow (await_suspend)
* ============================
*
* co_await JobQueueAwaiter{runner}
* |
* +-- await_ready() -> false
* +-- await_suspend(handle)
* |
* +-- runner->onSuspend() // ++nSuspend_
* +-- runner->post() // addJob to JobQueue
* | |
* | +-- success? return noop_coroutine()
* | | // coroutine stays suspended;
* | | // worker thread will call resume()
* | +-- failure? (JQ stopping)
* | +-- runner->onUndoSuspend() // --nSuspend_
* | +-- return handle // symmetric transfer back
* | // coroutine continues immediately
* | // so it can clean up and co_return
*
* DEPRECATED — prefer `co_await runner->yieldAndPost()`
* =====================================================
*
* GCC-12 has a coroutine codegen bug where using this external awaiter
* struct at multiple co_await points in the same coroutine corrupts the
* state machine's resume index. After the second co_await, the third
* resumption enters handle().resume() but never reaches await_resume()
* or any subsequent user code — the coroutine hangs indefinitely.
*
* The fix is `co_await runner->yieldAndPost()`, which defines the
* awaiter as an inline struct inside a CoroTaskRunner member function.
* GCC-12 handles inline awaiters correctly at multiple co_await points.
*
* This struct is retained for single-use scenarios and documentation
* purposes. For any code that may use co_await in a loop or at
* multiple points, always use `runner->yieldAndPost()`.
*
* Usage Examples
* ==============
*
* 1. Yield and auto-repost (preferred — works on all compilers):
*
* CoroTask<void> handler(auto runner) {
* doPartA();
* co_await runner->yieldAndPost(); // yield + repost
* doPartB(); // runs on a worker thread
* co_return;
* }
*
* 2. Multiple yield points in a loop:
*
* CoroTask<void> batchProcessor(auto runner) {
* for (auto& item : items) {
* process(item);
* co_await runner->yieldAndPost(); // let other jobs run
* }
* co_return;
* }
*
* 3. Graceful shutdown — checking after resume:
*
* CoroTask<void> longTask(auto runner, JobQueue& jq) {
* while (hasWork()) {
* co_await runner->yieldAndPost();
* // If JQ is stopping, await_suspend resumes the coroutine
* // immediately without re-queuing. Always check
* // isStopping() to decide whether to proceed:
* if (jq.isStopping())
* co_return;
* doNextChunk();
* }
* co_return;
* }
*
* Caveats / Pitfalls
* ==================
*
* BUG-RISK: Using a stale or null runner.
* The runner shared_ptr must be valid and point to the CoroTaskRunner
* that owns the coroutine currently executing. Passing a runner from
* a different coroutine, or a default-constructed shared_ptr, is UB.
*
* BUG-RISK: Assuming resume happens on the same thread.
* After co_await, the coroutine resumes on whatever worker thread
* picks up the job. Do not rely on thread-local state unless it is
* managed through LocalValue (which CoroTaskRunner automatically
* swaps in/out).
*
* BUG-RISK: Ignoring the shutdown path.
* When the JobQueue is stopping, post() fails and await_suspend()
* resumes the coroutine immediately (symmetric transfer back to h).
* The coroutine body continues on the same thread. If your code
* after co_await assumes it was re-queued and is running on a worker
* thread, that assumption breaks during shutdown. Always handle the
* "JQ is stopping" case, either by checking jq.isStopping() or by
* letting the coroutine fall through to co_return naturally.
*
* DIFFERENCE from runner->suspend() + runner->post():
* Both JobQueueAwaiter and yieldAndPost() combine suspend + post
* in one atomic operation. With the manual suspend()/post() pattern,
* there is a window between the two calls where an external event
* could race. The atomic awaiters remove that window — onSuspend()
* and post() happen within the same await_suspend() call while the
* coroutine is guaranteed to be suspended. Use yieldAndPost() unless
* you need an external party to decide *when* to call post().
*/
struct JobQueueAwaiter
{
// The CoroTaskRunner that owns the currently executing coroutine.
std::shared_ptr<JobQueue::CoroTaskRunner> runner;
/**
* Always returns false so the coroutine suspends.
*/
// The C++ coroutine protocol mandates these awaiter names and
// instance-callable methods, which conflict with the project
// naming/static conventions.
// NOLINTBEGIN(readability-identifier-naming, readability-convert-member-functions-to-static)
[[nodiscard]] bool
await_ready() const noexcept
{
return false;
}
/**
* Increment nSuspend (equivalent to yield()) and schedule resume
* on the JobQueue (equivalent to post()). If the JobQueue is
* stopping, undoes the suspend count and transfers back to the
* coroutine so it can clean up and co_return.
*
* Returns a coroutine_handle<> (symmetric transfer) instead of
* bool to work around a GCC-12 codegen bug where bool-returning
* await_suspend leaves the coroutine in an invalid state —
* neither properly suspended nor resumed — causing a hang.
*
* WARNING: GCC-12 has an additional codegen bug where using this
* external awaiter struct at multiple co_await points in the same
* coroutine corrupts the state machine's resume index, causing the
* coroutine to hang on the third resumption. Prefer
* `co_await runner->yieldAndPost()` which uses an inline awaiter
* that GCC-12 handles correctly.
*
* @return noop_coroutine() to stay suspended (job posted);
* the caller's handle to resume immediately (JQ stopping)
*/
std::coroutine_handle<>
await_suspend(std::coroutine_handle<> h)
{
XRPL_ASSERT(runner, "xrpl::JobQueueAwaiter::await_suspend : runner is valid");
runner->onSuspend();
if (!runner->post())
{
// JobQueue is stopping. Undo the suspend count and
// transfer back to the coroutine so it can clean up
// and co_return.
runner->onUndoSuspend();
return h;
}
return std::noop_coroutine();
}
void
await_resume() const noexcept
{
}
// NOLINTEND(readability-identifier-naming, readability-convert-member-functions-to-static)
};
} // namespace xrpl
+53 -12
View File
@@ -1,7 +1,7 @@
#pragma once
#include <xrpl/basics/base_uint.h>
#include <xrpl/basics/enum_bitops.h>
#include <xrpl/basics/safe_cast.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/ledger/OwnerCounts.h>
#include <xrpl/ledger/ReadView.h>
@@ -22,39 +22,80 @@
namespace xrpl {
enum class ApplyFlags : std::uint32_t {
None = 0x00,
// Bitwise flag enum with existing operator overloads
// NOLINTNEXTLINE(cppcoreguidelines-use-enum-class)
enum ApplyFlags : std::uint32_t {
TapNone = 0x00,
// This is a local transaction with the
// fail_hard flag set.
FailHard = 0x10,
TapFailHard = 0x10,
// This is not the transaction's last pass
// Transaction can be retried, soft failures allowed
Retry = 0x20,
TapRetry = 0x20,
// Transaction came from a privileged source
Unlimited = 0x400,
TapUnlimited = 0x400,
// Transaction is executing as part of a batch
Batch = 0x800,
TapBatch = 0x800,
// Transaction shouldn't be applied
// Signatures shouldn't be checked
DryRun = 0x1000,
TapDryRun = 0x1000,
// Transaction is being preflighted as the payload of a
// TransactionProposalCreate. Its signatures are collected on-ledger
// afterward, so signature-presence checks (e.g. Batch signer matching)
// are skipped at proposal-creation time (On-Chain Cosigner spec
// §5.3.1.2).
Proposal = 0x2000
TapProposal = 0x2000
};
template <>
struct enum_bitops::OptIn<ApplyFlags> : std::true_type
constexpr ApplyFlags
operator|(ApplyFlags const& lhs, ApplyFlags const& rhs)
{
};
return safeCast<ApplyFlags>(
safeCast<std::underlying_type_t<ApplyFlags>>(lhs) |
safeCast<std::underlying_type_t<ApplyFlags>>(rhs));
}
static_assert((TapFailHard | TapRetry) == safeCast<ApplyFlags>(0x30u), "ApplyFlags operator |");
static_assert((TapRetry | TapFailHard) == safeCast<ApplyFlags>(0x30u), "ApplyFlags operator |");
constexpr ApplyFlags
operator&(ApplyFlags const& lhs, ApplyFlags const& rhs)
{
return safeCast<ApplyFlags>(
safeCast<std::underlying_type_t<ApplyFlags>>(lhs) &
safeCast<std::underlying_type_t<ApplyFlags>>(rhs));
}
static_assert((TapFailHard & TapRetry) == TapNone, "ApplyFlags operator &");
static_assert((TapRetry & TapFailHard) == TapNone, "ApplyFlags operator &");
constexpr ApplyFlags
operator~(ApplyFlags const& flags)
{
return safeCast<ApplyFlags>(~safeCast<std::underlying_type_t<ApplyFlags>>(flags));
}
static_assert(~TapRetry == safeCast<ApplyFlags>(0xFFFFFFDFu), "ApplyFlags operator ~");
inline ApplyFlags
operator|=(ApplyFlags& lhs, ApplyFlags const& rhs)
{
lhs = lhs | rhs;
return lhs;
}
inline ApplyFlags
operator&=(ApplyFlags& lhs, ApplyFlags const& rhs)
{
lhs = lhs & rhs;
return lhs;
}
//------------------------------------------------------------------------------
@@ -1,17 +1,11 @@
#pragma once
#include <xrpl/basics/base_uint.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/ledger/ApplyView.h>
#include <xrpl/ledger/ReadView.h>
#include <xrpl/ledger/entries/SLEBase.h>
#include <xrpl/protocol/Indexes.h>
#include <xrpl/protocol/LedgerFormats.h>
#include <xrpl/protocol/SField.h>
#include <xrpl/protocol/STVector256.h>
#include <cstddef>
#include <optional>
namespace xrpl {
@@ -31,31 +25,6 @@ public:
: Base(keylet::skip(), view, j)
{
}
/**
* Looks up a hash `diff` slots back from the most recent entry in the
* sfHashes vector (diff == 0 is the most recent entry).
*
* Callers decide which skip list entry to read and how to translate a
* target ledger sequence into `diff`; this only does the bounds check
* and vector indexing shared by both the recent (stride 1) and distant
* (stride 256) skip lists.
*
* @param diff how many slots back from the most recent hash to look up;
* 0 is the most recent hash.
* @return the hash at that slot, or std::nullopt if the entry does not
* exist or `diff` is out of range.
*/
[[nodiscard]] std::optional<UInt256>
hashAt(std::size_t diff) const
{
if (!this->exists())
return std::nullopt;
STVector256 const vec = (*this)->getFieldV256(sfHashes);
if (vec.size() > diff)
return vec[vec.size() - diff - 1];
return std::nullopt;
}
};
using LedgerHashesEntryR = LedgerHashesEntry<ReadView>;
+3 -5
View File
@@ -1,10 +1,8 @@
#pragma once
#include <xrpl/basics/base_uint.h>
#include <xrpl/nodestore/NodeObject.h>
#include <memory>
#include <optional>
namespace xrpl::node_store {
@@ -25,7 +23,7 @@ public:
/**
* Construct the decoded blob from raw data.
*/
DecodedBlob(std::optional<uint256> key, void const* value, int valueBytes);
DecodedBlob(void const* key, void const* value, int valueBytes);
/**
* Determine if the decoding was successful.
@@ -45,10 +43,10 @@ public:
private:
bool success_{false};
uint256 key_;
void const* key_;
NodeObjectType objectType_{NodeObjectType::Unknown};
unsigned char const* objectData_{nullptr};
int dataBytes_ = 0;
int dataBytes_;
};
} // namespace xrpl::node_store
+8 -10
View File
@@ -1,6 +1,5 @@
#pragma once
#include <xrpl/basics/base_uint.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/nodestore/NodeObject.h>
@@ -38,6 +37,11 @@ namespace xrpl::node_store {
class EncodedBlob
{
/**
* The 32-byte key of the serialized object.
*/
std::array<std::uint8_t, 32> key_{};
/**
* A pre-allocated buffer for the serialized object.
*
@@ -61,11 +65,6 @@ class EncodedBlob
*/
std::uint8_t* const ptr_;
/**
* The 32-byte key of the serialized object.
*/
uint256 key_;
public:
explicit EncodedBlob(std::shared_ptr<NodeObject> const& obj)
: size_([&obj]() {
@@ -77,12 +76,11 @@ public:
return obj->getData().size() + 9;
}())
, ptr_((size_ <= payload_.size()) ? payload_.data() : new std::uint8_t[size_])
, key_(obj->getHash())
{
std::fill_n(ptr_, 8, std::uint8_t{0});
ptr_[8] = static_cast<std::uint8_t>(obj->getType());
std::copy_n(obj->getData().data(), obj->getData().size(), ptr_ + 9);
std::copy_n(obj->getHash().data(), obj->getHash().size(), key_.data());
}
~EncodedBlob()
@@ -97,10 +95,10 @@ public:
delete[] ptr_;
}
[[nodiscard]] uint256 const&
[[nodiscard]] void const*
getKey() const noexcept
{
return key_;
return static_cast<void const*>(key_.data());
}
[[nodiscard]] std::size_t
+23 -15
View File
@@ -61,22 +61,14 @@ parseBase58(std::string const& s);
/**
* A special account that's used as the "issuer" for XRP.
*/
constexpr inline AccountID const&
xrpAccount() noexcept
{
static constexpr AccountID kAccount(beast::kZero);
return kAccount;
}
AccountID const&
xrpAccount();
/**
* A placeholder for empty accounts.
*/
constexpr inline AccountID const&
noAccount() noexcept
{
static constexpr AccountID kAccount = xrpAccount().next();
return kAccount;
}
AccountID const&
noAccount();
/**
* Convert hex or base58 string to AccountID.
@@ -88,10 +80,10 @@ bool
toIssuer(AccountID&, std::string const&);
// DEPRECATED Should be checking the currency or native flag
constexpr inline bool
isXRP(AccountID const& c) noexcept
inline bool
isXRP(AccountID const& c)
{
return c == xrpAccount();
return c == beast::kZero;
}
// DEPRECATED
@@ -109,6 +101,22 @@ operator<<(std::ostream& os, AccountID const& x)
return os;
}
/**
* Initialize the global cache used to map AccountID to base58 conversions.
*
* The cache is optional and need not be initialized. But because conversion
* is expensive (it requires a SHA-256 operation) in most cases the overhead
* of the cache is worth the benefit.
*
* @param count The number of entries the cache should accommodate. Zero will
* disable the cache, releasing any memory associated with it.
*
* @note The function will only initialize the cache the first time it is
* invoked. Subsequent invocations do nothing.
*/
void
initAccountIdCache(std::size_t count);
} // namespace xrpl
//------------------------------------------------------------------------------
+2 -2
View File
@@ -1,10 +1,10 @@
#pragma once
#include <xrpl/basics/contract.h>
#include <xrpl/beast/type_name.h>
#include <xrpl/protocol/SOTemplate.h>
#include <boost/container/flat_map.hpp>
#include <boost/core/type_name.hpp>
#include <algorithm>
#include <cstddef>
@@ -84,7 +84,7 @@ public:
* Derived classes will load the object with all the known formats.
*/
private:
KnownFormats() : name_(boost::core::type_name<Derived>())
KnownFormats() : name_(beast::typeName<Derived>())
{
}
+1 -14
View File
@@ -33,20 +33,7 @@ public:
STBitString(SField const& n);
STBitString(value_type const& v);
template <typename Tag>
requires(!std::is_void_v<Tag>)
STBitString(BaseUInt<Bits, Tag> const& v) : value_(v)
{
}
STBitString(SField const& n, value_type const& v);
template <typename Tag>
requires(!std::is_void_v<Tag>)
STBitString(SField const& n, BaseUInt<Bits, Tag> const& v) : STBase(n), value_(v)
{
}
STBitString(SerialIter& sit, SField const& name);
[[nodiscard]] SerializedTypeID
@@ -179,7 +166,7 @@ template <typename Tag>
void
STBitString<Bits>::setValue(BaseUInt<Bits, Tag> const& v)
{
value_ = value_type{v};
value_ = v;
}
template <int Bits>
+51 -56
View File
@@ -2,7 +2,6 @@
#include <xrpl/basics/CountedObject.h>
#include <xrpl/basics/UnorderedContainers.h>
#include <xrpl/basics/enum_bitops.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/json/json_value.h>
#include <xrpl/protocol/AccountID.h>
@@ -16,7 +15,6 @@
#include <cstdint>
#include <memory>
#include <optional>
#include <type_traits>
#include <utility>
#include <vector>
@@ -24,8 +22,18 @@ namespace xrpl {
class STPathElement final : public CountedObject<STPathElement>
{
unsigned int type_;
AccountID accountID_;
PathAsset assetID_;
AccountID issuerID_;
bool isOffer_;
std::size_t hashValue_;
public:
enum class Type : std::uint8_t {
// Bitwise values (typeCurrency | typeMPT)
// NOLINTNEXTLINE(cppcoreguidelines-use-enum-class)
enum Type {
TypeNone = 0x00,
TypeAccount = 0x01, // Rippling through an account (vs taking an offer).
TypeCurrency = 0x10, // Currency follows.
@@ -37,8 +45,6 @@ public:
// Combination of all types.
};
using enum Type;
STPathElement();
STPathElement(STPathElement const&) = default;
STPathElement&
@@ -56,42 +62,42 @@ public:
bool forceAsset = false);
STPathElement(
Type type,
unsigned int uType,
AccountID const& account,
PathAsset const& asset,
AccountID const& issuer);
[[nodiscard]] Type
getNodeType() const noexcept;
[[nodiscard]] std::uint32_t
getNodeType() const;
[[nodiscard]] bool
isOffer() const noexcept;
isOffer() const;
[[nodiscard]] bool
isAccount() const noexcept;
isAccount() const;
[[nodiscard]] bool
hasIssuer() const noexcept;
hasIssuer() const;
[[nodiscard]] bool
hasCurrency() const noexcept;
hasCurrency() const;
[[nodiscard]] bool
hasMPT() const noexcept;
hasMPT() const;
[[nodiscard]] bool
hasAsset() const noexcept;
hasAsset() const;
[[nodiscard]] bool
isNone() const noexcept;
isNone() const;
// Nodes are either an account ID or a offer prefix. Offer prefixs denote a
// class of offers.
[[nodiscard]] AccountID const&
getAccountID() const noexcept;
getAccountID() const;
[[nodiscard]] PathAsset const&
getPathAsset() const noexcept;
getPathAsset() const;
[[nodiscard]] Currency const&
getCurrency() const;
@@ -100,35 +106,17 @@ public:
getMPTID() const;
[[nodiscard]] AccountID const&
getIssuerID() const noexcept;
getIssuerID() const;
[[nodiscard]] bool
isType(Type pe) const noexcept;
isType(Type const& pe) const;
friend bool
operator==(STPathElement const& lhs, STPathElement const& rhs) noexcept
{
return lhs.isType(TypeAccount) == rhs.isType(TypeAccount) &&
lhs.hashValue_ == rhs.hashValue_ && lhs.accountID_ == rhs.accountID_ &&
lhs.assetID_ == rhs.assetID_ && lhs.issuerID_ == rhs.issuerID_;
}
bool
operator==(STPathElement const& t) const;
private:
static std::size_t
getHash(STPathElement const& element);
Type type_;
AccountID accountID_;
PathAsset assetID_;
AccountID issuerID_;
bool isOffer_;
std::size_t hashValue_;
};
template <>
struct enum_bitops::OptIn<STPathElement::Type> : std::true_type
{
};
template <class Hasher>
@@ -136,7 +124,7 @@ void
hash_append(Hasher& h, STPathElement const& e) noexcept
{
using beast::hash_append;
hash_append(h, e.isType(STPathElement::TypeAccount));
hash_append(h, (e.getNodeType() & STPathElement::TypeAccount) != 0u);
hash_append(h, e.getAccountID());
hash_append(h, e.getPathAsset());
hash_append(h, e.getIssuerID());
@@ -416,11 +404,11 @@ inline STPathElement::STPathElement(
}
inline STPathElement::STPathElement(
Type type,
unsigned int uType,
AccountID const& account,
PathAsset const& asset,
AccountID const& issuer)
: type_(type)
: type_(uType)
, accountID_(account)
, assetID_(asset)
, issuerID_(issuer)
@@ -435,56 +423,56 @@ inline STPathElement::STPathElement(
hashValue_ = getHash(*this);
}
inline STPathElement::Type
STPathElement::getNodeType() const noexcept
inline std::uint32_t
STPathElement::getNodeType() const
{
return type_;
}
inline bool
STPathElement::isOffer() const noexcept
STPathElement::isOffer() const
{
return isOffer_;
}
inline bool
STPathElement::isAccount() const noexcept
STPathElement::isAccount() const
{
return !isOffer();
}
inline bool
STPathElement::isType(Type pe) const noexcept
STPathElement::isType(Type const& pe) const
{
return (type_ & pe) != STPathElement::TypeNone;
return (type_ & pe) != 0u;
}
inline bool
STPathElement::hasIssuer() const noexcept
STPathElement::hasIssuer() const
{
return isType(STPathElement::TypeIssuer);
}
inline bool
STPathElement::hasCurrency() const noexcept
STPathElement::hasCurrency() const
{
return isType(STPathElement::TypeCurrency);
}
inline bool
STPathElement::hasMPT() const noexcept
STPathElement::hasMPT() const
{
return isType(STPathElement::TypeMpt);
}
inline bool
STPathElement::hasAsset() const noexcept
STPathElement::hasAsset() const
{
return isType(STPathElement::TypeAsset);
}
inline bool
STPathElement::isNone() const noexcept
STPathElement::isNone() const
{
return getNodeType() == STPathElement::TypeNone;
}
@@ -492,13 +480,13 @@ STPathElement::isNone() const noexcept
// Nodes are either an account ID or a offer prefix. Offer prefixs denote a
// class of offers.
inline AccountID const&
STPathElement::getAccountID() const noexcept
STPathElement::getAccountID() const
{
return accountID_;
}
inline PathAsset const&
STPathElement::getPathAsset() const noexcept
STPathElement::getPathAsset() const
{
return assetID_;
}
@@ -516,11 +504,18 @@ STPathElement::getMPTID() const
}
inline AccountID const&
STPathElement::getIssuerID() const noexcept
STPathElement::getIssuerID() const
{
return issuerID_;
}
inline bool
STPathElement::operator==(STPathElement const& t) const
{
return (type_ & TypeAccount) == (t.type_ & TypeAccount) && hashValue_ == t.hashValue_ &&
accountID_ == t.accountID_ && assetID_ == t.assetID_ && issuerID_ == t.issuerID_;
}
// ------------ STPath ------------
inline STPath::STPath(std::vector<STPathElement> p) : path_(std::move(p))
+2 -2
View File
@@ -596,7 +596,7 @@ template <std::size_t Bits, class Tag>
BaseUInt<Bits, Tag>
SerialIter::getBitString()
{
constexpr auto n = BaseUInt<Bits, Tag>::size();
auto const n = Bits / 8;
if (remain_ < n)
Throw<std::runtime_error>("invalid SerialIter getBitString");
@@ -607,7 +607,7 @@ SerialIter::getBitString()
used_ += n;
remain_ -= n;
return BaseUInt<Bits, Tag>{std::span<std::uint8_t const, n>{x, n}};
return BaseUInt<Bits, Tag>::fromVoid(x);
}
} // namespace xrpl
+4 -3
View File
@@ -14,7 +14,7 @@ namespace xrpl {
// Various protocol and system specific constant globals.
/* The name of the system. */
inline std::string const&
static inline std::string const&
systemName()
{
static std::string const kName = "xrpld";
@@ -52,10 +52,11 @@ isLegalAmountSigned(XRPAmount const& amount)
}
/* The currency code for the native currency. */
inline std::string
static inline std::string const&
systemCurrencyCode()
{
return "XRP";
static std::string const kCode = "XRP";
return kCode;
}
/**
-6
View File
@@ -238,12 +238,6 @@ inline constexpr FlagValue tfUniversalMask = ~tfUniversal;
TF_FLAG(tfSponsorshipEnd, 0x00010000) \
TF_FLAG(tfSponsorshipCreate, 0x00020000) \
TF_FLAG(tfSponsorshipReassign, 0x00040000), \
MASK_ADJ(0)) \
\
TRANSACTION(ConfidentialMPTHolderKeyUpdate, \
TF_FLAG(tfHolderKeyRotation, 0x00010000) \
TF_FLAG(tfHolderKeyRecovery, 0x00020000) \
TF_FLAG(tfCancelRecovery, 0x00040000), \
MASK_ADJ(0))
// clang-format on
-5
View File
@@ -12,12 +12,10 @@
#include <xrpl/protocol/STObject.h>
#include <xrpl/protocol/Serializer.h>
#include <xrpl/protocol/TER.h>
#include <xrpl/protocol/UintTypes.h>
#include <boost/container/flat_set.hpp>
#include <cstdint>
#include <flat_set>
#include <optional>
namespace xrpl {
@@ -117,9 +115,6 @@ public:
parentBatchID_ = id;
}
[[nodiscard]] std::flat_set<MPTID>
getAffectedMPTs() const;
private:
UInt256 transactionID_;
std::uint32_t ledgerSeq_;
+12 -25
View File
@@ -61,40 +61,26 @@ using Domain = BaseUInt<256>;
/**
* XRP currency.
*/
constexpr inline Currency const&
xrpCurrency() noexcept
{
static constexpr Currency const kCurrency(beast::kZero);
return kCurrency;
}
Currency const&
xrpCurrency();
/**
* A placeholder for empty currencies.
*/
constexpr inline Currency const&
noCurrency() noexcept
{
static constexpr Currency const kCurrency(xrpCurrency().next());
return kCurrency;
}
Currency const&
noCurrency();
/**
* We deliberately disallow the currency that looks like "XRP" because too
* many people were using it instead of the correct XRP currency.
*
* Note that this doesn't catch "xRP" or "xrp" or other case variations.
*/
constexpr inline Currency const&
badCurrency() noexcept
{
static constexpr Currency kCurrency{"0000000000000000000000005852500000000000"};
return kCurrency;
}
Currency const&
badCurrency();
constexpr inline bool
isXRP(Currency const& c) noexcept
inline bool
isXRP(Currency const& c)
{
return c == xrpCurrency();
return c == beast::kZero;
}
/**
@@ -112,7 +98,7 @@ to_string(Currency const& c);
* to rewrite some unit test code.
*/
bool
toCurrency(Currency&, std::string_view);
toCurrency(Currency&, std::string const&);
/**
* Tries to convert a string to a Currency, returns noCurrency() on failure.
@@ -121,7 +107,8 @@ toCurrency(Currency&, std::string_view);
* unfortunate; changing this will require very careful checking
* everywhere and may mean having to rewrite some unit test code.
*/
Currency toCurrency(std::string_view);
Currency
toCurrency(std::string const&);
inline std::ostream&
operator<<(std::ostream& os, Currency const& x)
@@ -439,7 +439,6 @@ LEDGER_ENTRY(ltMPTOKEN, 0x007f, MPToken, mptoken, ({
{sfIssuerKeyMirrorEpoch, SoeOptional},
{sfAuditorKeyMirrorEpoch, SoeOptional},
{sfHolderEncryptionKey, SoeOptional},
{sfRecoveryKey, SoeOptional},
}))
/** A ledger object which tracks Oracle
@@ -330,7 +330,6 @@ TYPED_SFIELD(sfAuditorEncryptionKey, VL, 44)
TYPED_SFIELD(sfAmountCommitment, VL, 45)
TYPED_SFIELD(sfBalanceCommitment, VL, 46)
TYPED_SFIELD(sfBytecode, VL, 47)
TYPED_SFIELD(sfRecoveryKey, VL, 48)
// account (common)
TYPED_SFIELD(sfAccount, ACCOUNT, 1)
@@ -1147,27 +1147,12 @@ TRANSACTION(ttCONFIDENTIAL_MPT_MIRROR_UPDATE, 92, ConfidentialMPTMirrorUpdate,
{sfZKProof, SoeRequired},
}))
/** This transaction rotates or recovers a confidential MPT holder's ElGamal encryption key,
or cancels a recovery. */
#if TRANSACTION_INCLUDE
# include <xrpl/tx/transactors/token/ConfidentialMPTHolderKeyUpdate.h>
#endif
TRANSACTION(ttCONFIDENTIAL_MPT_HOLDER_KEY_UPDATE, 93, ConfidentialMPTHolderKeyUpdate,
({.amendment = featureConfidentialMPTKeyRotation}),
({
{sfMPTokenIssuanceID, SoeRequired},
{sfHolderEncryptionKey, SoeOptional},
{sfConfidentialBalanceSpending, SoeOptional},
{sfConfidentialBalanceInbox, SoeOptional},
{sfZKProof, SoeOptional},
}))
/** This transaction posts an unsigned transaction on-ledger as a
TransactionProposal, pending multi-signature collection. */
#if TRANSACTION_INCLUDE
# include <xrpl/tx/transactors/proposal/TransactionProposalCreate.h>
#endif
TRANSACTION(ttTRANSACTION_PROPOSAL_CREATE, 95, TransactionProposalCreate,
TRANSACTION(ttTRANSACTION_PROPOSAL_CREATE, 93, TransactionProposalCreate,
({.amendment = featureCosign}),
({
{sfProposedTransaction, SoeRequired},
+1 -2
View File
@@ -8,7 +8,6 @@
#include <array>
#include <cstddef>
#include <cstdint>
#include <span>
#include <type_traits>
namespace xrpl {
@@ -183,7 +182,7 @@ public:
operator result_type() noexcept
{
auto const digest = Sha512Hasher::result_type(h_);
return result_type{std::span{digest}.first<result_type::size()>()};
return result_type::fromVoid(digest.data());
}
private:
-2
View File
@@ -392,7 +392,6 @@ JSS(max_ledger); // in/out: LedgerCleaner
JSS(max_queue_size); // out: TxQ
JSS(max_spend_drops); // out: AccountInfo
JSS(max_spend_drops_total); // out: AccountInfo
JSS(maximum);
JSS(mean); // out: get_aggregate_price
JSS(median); // out: get_aggregate_price
JSS(median_fee); // out: TxQ
@@ -412,7 +411,6 @@ JSS(minimum_level); // out: TxQ
JSS(missingCommand); // error
JSS(mpt_amount); // out: mpt_holders
JSS(mpt_issuance_id); // in: Payment, mpt_holders
JSS(mpt_issuances); // in: Subscribe, Unsubscribe
JSS(mptoken_index); // out: mpt_holders
JSS(mpt_issuance_id_a); // out: BookChanges
JSS(mpt_issuance_id_b); // out: BookChanges
+1 -3
View File
@@ -8,7 +8,6 @@
#include <cstdint>
#include <cstring>
#include <span>
namespace xrpl::nft {
@@ -99,8 +98,7 @@ getTaxon(UInt256 const& id)
inline AccountID
getIssuer(UInt256 const& id)
{
return AccountID{
std::span<unsigned char const, uint256::size()>{id}.subspan<4, AccountID::size()>()};
return AccountID::fromVoid(id.data() + 4);
}
} // namespace xrpl::nft
+4 -1
View File
@@ -2,10 +2,13 @@
#include <xrpl/basics/base_uint.h>
#include <string_view>
namespace xrpl::nft {
// NFT directory pages order their contents based only on the low 96 bits of
// the NFToken value. This mask provides easy access to the necessary mask.
constexpr UInt256 kPageMask{"0000000000000000000000000000000000000000ffffffffffffffffffffffff"};
constexpr UInt256 kPageMask(
std::string_view("0000000000000000000000000000000000000000ffffffffffffffffffffffff"));
} // namespace xrpl::nft
@@ -339,30 +339,6 @@ public:
{
return this->sle_->isFieldPresent(sfHolderEncryptionKey);
}
/**
* @brief Get sfRecoveryKey (SoeOptional)
* @return The field value, or std::nullopt if not present.
*/
[[nodiscard]]
protocol_autogen::Optional<SF_VL::type::value_type>
getRecoveryKey() const
{
if (hasRecoveryKey())
return this->sle_->at(sfRecoveryKey);
return std::nullopt;
}
/**
* @brief Check if sfRecoveryKey is present.
* @return True if the field is present, false otherwise.
*/
[[nodiscard]]
bool
hasRecoveryKey() const
{
return this->sle_->isFieldPresent(sfRecoveryKey);
}
};
/**
@@ -576,17 +552,6 @@ public:
return *this;
}
/**
* @brief Set sfRecoveryKey (SoeOptional)
* @return Reference to this builder for method chaining.
*/
MPTokenBuilder&
setRecoveryKey(std::decay_t<typename SF_VL::type::value_type> const& value)
{
object_[sfRecoveryKey] = value;
return *this;
}
/**
* @brief Build and return the completed MPToken wrapper.
* @param index The ledger entry index.
@@ -1,279 +0,0 @@
// This file is auto-generated. Do not edit.
#pragma once
#include <xrpl/protocol/STTx.h>
#include <xrpl/protocol/STParsedJSON.h>
#include <xrpl/protocol/jss.h>
#include <xrpl/protocol_autogen/TransactionBase.h>
#include <xrpl/protocol_autogen/TransactionBuilderBase.h>
#include <xrpl/json/json_value.h>
#include <stdexcept>
#include <optional>
namespace xrpl::transactions {
class ConfidentialMPTHolderKeyUpdateBuilder;
/**
* @brief Transaction: ConfidentialMPTHolderKeyUpdate
*
* Type: ttCONFIDENTIAL_MPT_HOLDER_KEY_UPDATE (93)
* Delegable: Delegation::NotDelegable
* Amendment: featureConfidentialMPTKeyRotation
* Privileges: Privilege::NoPriv
*
* Immutable wrapper around STTx providing type-safe field access.
* Use ConfidentialMPTHolderKeyUpdateBuilder to construct new transactions.
*/
class ConfidentialMPTHolderKeyUpdate : public TransactionBase
{
public:
static constexpr xrpl::TxType txType = ttCONFIDENTIAL_MPT_HOLDER_KEY_UPDATE;
/**
* @brief Construct a ConfidentialMPTHolderKeyUpdate transaction wrapper from an existing STTx object.
* @throws std::runtime_error if the transaction type doesn't match.
*/
explicit ConfidentialMPTHolderKeyUpdate(std::shared_ptr<STTx const> tx)
: TransactionBase(std::move(tx))
{
// Verify transaction type
if (tx_->getTxnType() != txType)
{
throw std::runtime_error("Invalid transaction type for ConfidentialMPTHolderKeyUpdate");
}
}
// Transaction-specific field getters
/**
* @brief Get sfMPTokenIssuanceID (SoeRequired)
* @return The field value.
*/
[[nodiscard]]
SF_UINT192::type::value_type
getMPTokenIssuanceID() const
{
return this->tx_->at(sfMPTokenIssuanceID);
}
/**
* @brief Get sfHolderEncryptionKey (SoeOptional)
* @return The field value, or std::nullopt if not present.
*/
[[nodiscard]]
protocol_autogen::Optional<SF_VL::type::value_type>
getHolderEncryptionKey() const
{
if (hasHolderEncryptionKey())
{
return this->tx_->at(sfHolderEncryptionKey);
}
return std::nullopt;
}
/**
* @brief Check if sfHolderEncryptionKey is present.
* @return True if the field is present, false otherwise.
*/
[[nodiscard]]
bool
hasHolderEncryptionKey() const
{
return this->tx_->isFieldPresent(sfHolderEncryptionKey);
}
/**
* @brief Get sfConfidentialBalanceSpending (SoeOptional)
* @return The field value, or std::nullopt if not present.
*/
[[nodiscard]]
protocol_autogen::Optional<SF_VL::type::value_type>
getConfidentialBalanceSpending() const
{
if (hasConfidentialBalanceSpending())
{
return this->tx_->at(sfConfidentialBalanceSpending);
}
return std::nullopt;
}
/**
* @brief Check if sfConfidentialBalanceSpending is present.
* @return True if the field is present, false otherwise.
*/
[[nodiscard]]
bool
hasConfidentialBalanceSpending() const
{
return this->tx_->isFieldPresent(sfConfidentialBalanceSpending);
}
/**
* @brief Get sfConfidentialBalanceInbox (SoeOptional)
* @return The field value, or std::nullopt if not present.
*/
[[nodiscard]]
protocol_autogen::Optional<SF_VL::type::value_type>
getConfidentialBalanceInbox() const
{
if (hasConfidentialBalanceInbox())
{
return this->tx_->at(sfConfidentialBalanceInbox);
}
return std::nullopt;
}
/**
* @brief Check if sfConfidentialBalanceInbox is present.
* @return True if the field is present, false otherwise.
*/
[[nodiscard]]
bool
hasConfidentialBalanceInbox() const
{
return this->tx_->isFieldPresent(sfConfidentialBalanceInbox);
}
/**
* @brief Get sfZKProof (SoeOptional)
* @return The field value, or std::nullopt if not present.
*/
[[nodiscard]]
protocol_autogen::Optional<SF_VL::type::value_type>
getZKProof() const
{
if (hasZKProof())
{
return this->tx_->at(sfZKProof);
}
return std::nullopt;
}
/**
* @brief Check if sfZKProof is present.
* @return True if the field is present, false otherwise.
*/
[[nodiscard]]
bool
hasZKProof() const
{
return this->tx_->isFieldPresent(sfZKProof);
}
};
/**
* @brief Builder for ConfidentialMPTHolderKeyUpdate transactions.
*
* Provides a fluent interface for constructing transactions with method chaining.
* Uses STObject internally for flexible transaction construction.
* Inherits common field setters from TransactionBuilderBase.
*/
class ConfidentialMPTHolderKeyUpdateBuilder : public TransactionBuilderBase<ConfidentialMPTHolderKeyUpdateBuilder>
{
public:
/**
* @brief Construct a new ConfidentialMPTHolderKeyUpdateBuilder with required fields.
* @param account The account initiating the transaction.
* @param mPTokenIssuanceID The sfMPTokenIssuanceID field value.
* @param sequence Optional sequence number for the transaction.
* @param fee Optional fee for the transaction.
*/
ConfidentialMPTHolderKeyUpdateBuilder(SF_ACCOUNT::type::value_type account,
std::decay_t<typename SF_UINT192::type::value_type> const& mPTokenIssuanceID, std::optional<SF_UINT32::type::value_type> sequence = std::nullopt,
std::optional<SF_AMOUNT::type::value_type> fee = std::nullopt
)
: TransactionBuilderBase<ConfidentialMPTHolderKeyUpdateBuilder>(ttCONFIDENTIAL_MPT_HOLDER_KEY_UPDATE, account, sequence, fee)
{
setMPTokenIssuanceID(mPTokenIssuanceID);
}
/**
* @brief Construct a ConfidentialMPTHolderKeyUpdateBuilder from an existing STTx object.
* @param tx The existing transaction to copy from.
* @throws std::runtime_error if the transaction type doesn't match.
*/
ConfidentialMPTHolderKeyUpdateBuilder(std::shared_ptr<STTx const> tx)
{
if (tx->getTxnType() != ttCONFIDENTIAL_MPT_HOLDER_KEY_UPDATE)
{
throw std::runtime_error("Invalid transaction type for ConfidentialMPTHolderKeyUpdateBuilder");
}
object_ = *tx;
}
/**
* @brief Transaction-specific field setters
*/
/**
* @brief Set sfMPTokenIssuanceID (SoeRequired)
* @return Reference to this builder for method chaining.
*/
ConfidentialMPTHolderKeyUpdateBuilder&
setMPTokenIssuanceID(std::decay_t<typename SF_UINT192::type::value_type> const& value)
{
object_[sfMPTokenIssuanceID] = value;
return *this;
}
/**
* @brief Set sfHolderEncryptionKey (SoeOptional)
* @return Reference to this builder for method chaining.
*/
ConfidentialMPTHolderKeyUpdateBuilder&
setHolderEncryptionKey(std::decay_t<typename SF_VL::type::value_type> const& value)
{
object_[sfHolderEncryptionKey] = value;
return *this;
}
/**
* @brief Set sfConfidentialBalanceSpending (SoeOptional)
* @return Reference to this builder for method chaining.
*/
ConfidentialMPTHolderKeyUpdateBuilder&
setConfidentialBalanceSpending(std::decay_t<typename SF_VL::type::value_type> const& value)
{
object_[sfConfidentialBalanceSpending] = value;
return *this;
}
/**
* @brief Set sfConfidentialBalanceInbox (SoeOptional)
* @return Reference to this builder for method chaining.
*/
ConfidentialMPTHolderKeyUpdateBuilder&
setConfidentialBalanceInbox(std::decay_t<typename SF_VL::type::value_type> const& value)
{
object_[sfConfidentialBalanceInbox] = value;
return *this;
}
/**
* @brief Set sfZKProof (SoeOptional)
* @return Reference to this builder for method chaining.
*/
ConfidentialMPTHolderKeyUpdateBuilder&
setZKProof(std::decay_t<typename SF_VL::type::value_type> const& value)
{
object_[sfZKProof] = value;
return *this;
}
/**
* @brief Build and return the ConfidentialMPTHolderKeyUpdate wrapper.
* @param publicKey The public key for signing.
* @param secretKey The secret key for signing.
* @return The constructed transaction wrapper.
*/
ConfidentialMPTHolderKeyUpdate
build(PublicKey const& publicKey, SecretKey const& secretKey)
{
sign(publicKey, secretKey);
return ConfidentialMPTHolderKeyUpdate{std::make_shared<STTx>(std::move(object_))};
}
};
} // namespace xrpl::transactions
@@ -18,7 +18,7 @@ class TransactionProposalCreateBuilder;
/**
* @brief Transaction: TransactionProposalCreate
*
* Type: ttTRANSACTION_PROPOSAL_CREATE (95)
* Type: ttTRANSACTION_PROPOSAL_CREATE (93)
* Delegable: Delegation::NotDelegable
* Amendment: featureCosign
* Privileges: Privilege::NoPriv
+8 -66
View File
@@ -7,7 +7,6 @@
#include <xrpl/protocol/AccountID.h>
#include <xrpl/protocol/Book.h>
#include <xrpl/protocol/ErrorCodes.h>
#include <xrpl/protocol/UintTypes.h>
#include <xrpl/resource/Consumer.h>
#include <xrpl/server/Manifest.h>
@@ -27,11 +26,11 @@ namespace xrpl {
/**
* Maximum number of subscriptions a single client connection may hold at once.
*
* Applies to the account, real-time account, account-history, and MPT issuance
* subscriptions tracked on one InfoSub (the sets counted by
* totalSubscriptionCount), bounding the disconnect-time cleanup of those sets.
* Book subscriptions are tracked separately (OrderBookDB) and are not counted
* here. Generous enough for legitimate power users such as block explorers.
* Applies to the account, real-time account, and account-history subscriptions
* tracked on one InfoSub (the sets counted by totalSubscriptionCount), bounding
* the disconnect-time cleanup of those sets. Book subscriptions are tracked
* separately (OrderBookDB) and are not counted here. Generous enough for
* legitimate power users such as block explorers.
*/
constexpr std::size_t kMaxSubscriptionsPerConnection = 100'000;
@@ -152,21 +151,6 @@ public:
AccountID const& account,
bool historyOnly) = 0;
/**
* Remove an MPT issuance subscription during InfoSub teardown.
*
* Removes only the server-side entry from subMPT_. Does NOT touch
* InfoSub::mptSubscriptions_ because the InfoSub is being destroyed.
* Called by ~InfoSub() for each issuance in mptSubscriptions_.
*
* @param uListener The sequence number of the subscriber being torn down.
* @param mptID The MPT issuance to remove.
*
* @note Thread-safety: acquires mptLock_ internally.
*/
virtual void
unsubMPTInternal(std::uint64_t uListener, MPTID const& mptID) = 0;
/**
* Schedule the server-side teardown of a disconnecting connection's
* account subscriptions off the destructor thread.
@@ -286,11 +270,6 @@ public:
virtual bool
unsubConsensus(std::uint64_t uListener) = 0;
virtual void
subMPT(InfoSub::Ref ispListener, HashSet<MPTID> const& mptIDs) = 0;
virtual void
unsubMPT(InfoSub::Ref ispListener, HashSet<MPTID> const& mptIDs) = 0;
// VFALCO TODO Remove
// This was added for one particular partner, it
// "pushes" subscription data to a particular URL.
@@ -330,9 +309,9 @@ public:
* Return the number of subscriptions currently tracked on this
* connection.
*
* The combined size of the per-connection account, real-time account,
* account-history, and MPT issuance subscription sets. `doSubscribe` reads
* this to enforce the per-connection subscription cap before admitting more.
* The combined size of the per-connection account, real-time account, and
* account-history subscription sets. `doSubscribe` reads this to enforce
* the per-connection subscription cap before admitting more.
*
* @return The total tracked subscription count for this connection.
*
@@ -363,25 +342,6 @@ public:
HashSet<AccountID> const& normalAccounts,
std::size_t cap);
/**
* Enforce the cap and reserve a request's net-new MPT issuances, atomically.
*
* The MPT analogue of tryReserveAccountSubscriptions: under one hold of
* `lock_`, count the net-new issuances, check the total against @p cap, and
* insert them only if it fits. All-or-nothing, so a rejected request records
* nothing. Doing check and insert together stops two concurrent requests
* sharing an InfoSub (the admin subscribe-by-url path) from both passing the
* check before either records its issuances. The server-side map is
* populated afterwards by subMPT, whose re-insert is a no-op.
*
* @param mptIDs The MPT issuance ids to reserve.
* @param cap The effective per-connection cap.
* @return true if reserved; false if the request must be rejected.
* @note Thread-safe: takes `lock_`.
*/
[[nodiscard]] bool
tryReserveMPTSubscriptions(HashSet<MPTID> const& mptIDs, std::size_t cap);
/**
* Whether this connection already tracks an account-history for @p account.
*
@@ -453,29 +413,12 @@ public:
[[nodiscard]] unsigned int
getApiVersion() const noexcept;
void
insertSubMPTInfo(MPTID const& mptID);
void
deleteSubMPTInfo(MPTID const& mptID);
protected:
// Mutable so the read-only totalSubscriptionCount() accessor can lock it
// from a const method; locking semantics are otherwise unchanged.
mutable std::mutex lock_;
private:
// The lock type guarding this instance's subscription sets.
using ScopedLock = std::scoped_lock<decltype(lock_)>;
/**
* The combined tally the per-connection cap is enforced against.
*
* @param lock Proof that `lock_` is held; unused otherwise.
*/
[[nodiscard]] std::size_t
subscriptionCount(ScopedLock const& lock) const;
Consumer consumer_;
Source& source_;
HashSet<AccountID> realTimeSubscriptions_;
@@ -484,7 +427,6 @@ private:
std::uint64_t seq_;
HashSet<AccountID> accountHistorySubscriptions_;
HashSet<Book> bookSubscriptions_;
HashSet<MPTID> mptSubscriptions_;
unsigned int apiVersion_ = 0;
static int
+26 -16
View File
@@ -1,5 +1,6 @@
#pragma once
#include <xrpl/basics/ByteUtilities.h>
#include <xrpl/basics/CountedObject.h>
#include <xrpl/basics/SlabAllocator.h>
#include <xrpl/basics/Slice.h>
@@ -14,13 +15,12 @@
#include <cstdint>
#include <cstring>
#include <memory>
#include <new>
#include <type_traits>
namespace xrpl {
// an item stored in a SHAMap
class alignas(8) SHAMapItem : public CountedObject<SHAMapItem>
class SHAMapItem : public CountedObject<SHAMapItem>
{
// These are used to support boost::intrusive_ptr reference counting
// These functions are used internally by boost::intrusive_ptr to handle
@@ -101,15 +101,15 @@ namespace detail {
// The slab cutoffs and the number of megabytes per allocation are customized
// based on the number of objects of each size we expect to need at any point
// in time and with an eye to minimize the number of slack bytes in a block.
inline constinit slab::Allocator<SHAMapItem, slab::HeapFallback,
slab::Config<1000000, 128>,
slab::Config<1000000, 296>,
slab::Config<125000, 392>,
slab::Config<125000, 520>,
slab::Config<62500, 760>,
slab::Config<62500, 856>,
slab::Config<31250, 1048>
> gSlabber;
inline SlabAllocatorSet<SHAMapItem> gSlabber({
{ 128, megabytes(std::size_t(60)) },
{ 192, megabytes(std::size_t(46)) },
{ 272, megabytes(std::size_t(60)) },
{ 384, megabytes(std::size_t(56)) },
{ 564, megabytes(std::size_t(40)) },
{ 772, megabytes(std::size_t(46)) },
{ 1052, megabytes(std::size_t(60)) },
});
// clang-format on
} // namespace detail
@@ -150,14 +150,24 @@ makeShamapitem(UInt256 const& tag, Slice data)
XRPL_ASSERT(
data.size() <= megabytes<std::size_t>(16), "xrpl::makeShamapitem : maximum input size");
// We do not increment the reference count here on purpose: the
// constructor of SHAMapItem explicitly sets it to 1.
if (auto raw = detail::gSlabber.allocate(data.size())) [[likely]]
return {new (raw) SHAMapItem{tag, data}, false};
// NOLINTNEXTLINE(misc-const-correctness)
std::uint8_t* raw = detail::gSlabber.allocate(data.size());
Throw<std::bad_alloc>();
// If we can't grab memory from the slab allocators, we fall back to
// the standard library and try to grab a precisely-sized memory block:
if (raw == nullptr)
raw = new std::uint8_t[sizeof(SHAMapItem) + data.size()];
// We do not increment the reference count here on purpose: the
// constructor of SHAMapItem explicitly sets it to 1. We use the fact
// that the refcount can never be zero before incrementing as an
// invariant.
return {new (raw) SHAMapItem{tag, data}, false};
}
static_assert(alignof(SHAMapItem) != 40);
static_assert(alignof(SHAMapItem) == 8 || alignof(SHAMapItem) == 4);
inline boost::intrusive_ptr<SHAMapItem>
makeShamapitem(SHAMapItem const& other)
{
+1 -2
View File
@@ -51,8 +51,7 @@ public:
beast::Journal journal = beast::Journal{beast::Journal::getNullSink()})
: ApplyContext(registry, base, std::nullopt, tx, preclaimResult, baseFee, flags, journal)
{
XRPL_ASSERT(
(flags & ApplyFlags::Batch) == ApplyFlags::None, "Batch apply flag should not be set");
XRPL_ASSERT((flags & TapBatch) == 0, "Batch apply flag should not be set");
}
std::reference_wrapper<ServiceRegistry> registry;
+8 -11
View File
@@ -59,11 +59,10 @@ public:
, parentBatchId(parentBatchId)
, j(j)
{
XRPL_ASSERT(
(flags & ApplyFlags::Batch) == ApplyFlags::Batch, "Batch apply flag should be set");
XRPL_ASSERT((flags & TapBatch) == TapBatch, "Batch apply flag should be set");
XRPL_ASSERT_IF(
(flags & ApplyFlags::Proposal) != ApplyFlags::None,
(flags & ApplyFlags::DryRun) != ApplyFlags::None,
(flags & TapProposal) != TapNone,
(flags & TapDryRun) != TapNone,
"xrpl::PreflightContext : proposal preflight implies dry run");
}
@@ -75,11 +74,10 @@ public:
beast::Journal j = beast::Journal{beast::Journal::getNullSink()})
: registry(registry), tx(tx), rules(std::move(rules)), flags(flags), j(j)
{
XRPL_ASSERT(
(flags & ApplyFlags::Batch) == ApplyFlags::None, "Batch apply flag should not be set");
XRPL_ASSERT((flags & TapBatch) == 0, "Batch apply flag should not be set");
XRPL_ASSERT_IF(
(flags & ApplyFlags::Proposal) != ApplyFlags::None,
(flags & ApplyFlags::DryRun) != ApplyFlags::None,
(flags & TapProposal) != TapNone,
(flags & TapDryRun) != TapNone,
"xrpl::PreflightContext : proposal preflight implies dry run");
}
@@ -118,7 +116,7 @@ public:
, j(j)
{
XRPL_ASSERT(
parentBatchId.has_value() == ((flags & ApplyFlags::Batch) == ApplyFlags::Batch),
parentBatchId.has_value() == ((flags & TapBatch) == TapBatch),
"Parent Batch ID should be set if batch apply flag is set");
}
@@ -131,8 +129,7 @@ public:
beast::Journal j = beast::Journal{beast::Journal::getNullSink()})
: PreclaimContext(registry, view, preflightResult, tx, flags, std::nullopt, j)
{
XRPL_ASSERT(
(flags & ApplyFlags::Batch) == ApplyFlags::None, "Batch apply flag should not be set");
XRPL_ASSERT((flags & TapBatch) == 0, "Batch apply flag should not be set");
}
PreclaimContext&
+1 -1
View File
@@ -41,7 +41,7 @@ struct ApplyResult
inline bool
isTecClaimHardFail(TER ter, ApplyFlags flags)
{
return isTecClaim(ter) && ((flags & ApplyFlags::Retry) == ApplyFlags::None);
return isTecClaim(ter) && ((flags & TapRetry) == 0u);
}
/**
+2 -8
View File
@@ -46,9 +46,6 @@ class ValidMPTIssuance
* MPTokens and RippleStates deleted during apply. finalize() checks each
* holder's AccountRoot to detect vault pseudo-account holdings deleted
* outside VaultDelete. All these checks are gated on fixCleanup3_2_0.
*
* Under fixCleanup3_5_0, finalize() also rejects any MPToken erased with
* a non-zero sfMPTAmount.
*/
std::vector<std::shared_ptr<SLE const>> deletedHoldings_;
@@ -141,9 +138,8 @@ public:
* - Convert/ConvertBack symmetry:
* Regular MPToken balance change (±X) == COA (Confidential Outstanding Amount) change (∓X)
* - Cannot delete MPToken with non-zero confidential state:
* Cannot delete if any of sfConfidentialBalanceSpending, sfConfidentialBalanceInbox,
* sfIssuerEncryptedBalance or sfAuditorEncryptedBalance is present, and the issuance's
* sfConfidentialOutstandingAmount is non-zero. Mirrors MPTokenAuthorize::preclaim.
* Cannot delete if sfIssuerEncryptedBalance exists
* Cannot delete if sfConfidentialBalanceInbox and sfConfidentialBalanceSpending exist
* - Privacy flag consistency:
* MPToken confidential balance fields can only be created or changed if
* lsfMPTCanHoldConfidentialBalance is set on the issuance.
@@ -166,8 +162,6 @@ class ValidConfidentialMPToken
std::int64_t outstandingDelta = 0;
SLE::const_pointer issuance;
bool deletedWithEncrypted = false;
// True when an erased MPToken had a non-zero pre-tx public balance.
bool deletedWithBalanceBefore = false;
bool badConsistency = false;
bool badCOA = false;
bool changesConfidentialFields = false;
@@ -1,84 +0,0 @@
#pragma once
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/core/ServiceRegistry.h>
#include <xrpl/ledger/ReadView.h>
#include <xrpl/protocol/STTx.h>
#include <xrpl/protocol/TER.h>
#include <xrpl/protocol/XRPAmount.h>
#include <xrpl/tx/ApplyContext.h>
#include <xrpl/tx/Transactor.h>
#include <cstdint>
namespace xrpl {
/**
* @brief Allows a confidential MPT holder to rotate or recover their ElGamal
* encryption key, or to cancel a pending recovery.
*
* Submitted by the holder. Exactly one of three modes must be selected via
* the transaction flags:
*
* - Rotation (tfHolderKeyRotation): the holder still has their current
* ElGamal private key. They provide a new public key along with their
* current spending/inbox balances re-encrypted under that new key. The
* holder's encryption key and confidential balances are updated
* immediately.
*
* - Recovery (tfHolderKeyRecovery): the holder has lost their current
* private key. They provide a new public key but cannot provide
* re-encrypted balances. The new key is recorded as a pending
* sfRecoveryKey; the confidential balances are left untouched. Completing
* the recovery (rewriting the balances) is done separately by the issuer
* via ConfidentialMPTRecoverBalance.
*
* - Cancel (tfCancelRecovery): the holder revokes a pending recovery
* authorization created by a prior Recovery-mode transaction. No key,
* balances, or proof are carried; authorization is via the holder's
* ordinary XRPL signature. sfRecoveryKey is removed and everything else
* is left untouched. Fails if no recovery is pending.
*
* @note Zero-knowledge proof verification for Rotation and Recovery is
* deferred to a follow-up once the mpt-crypto constructions are finalized.
*/
class ConfidentialMPTHolderKeyUpdate : public Transactor
{
public:
static constexpr auto kConsequencesFactory = ConsequencesFactoryType::Normal;
explicit ConfidentialMPTHolderKeyUpdate(ApplyContext& ctx) : Transactor(ctx)
{
}
static bool
checkExtraFeatures(PreflightContext const& ctx);
static std::uint32_t
getFlagsMask(PreflightContext const& ctx);
static NotTEC
preflight(PreflightContext const& ctx);
static XRPAmount
calculateBaseFee(ReadView const& view, STTx const& tx);
static TER
preclaim(PreclaimContext const& ctx);
TER
doApply() override;
void
visitInvariantEntry(bool isDelete, SLE::ConstRef before, SLE::ConstRef after) override;
[[nodiscard]] bool
finalizeInvariants(
STTx const& tx,
TER result,
XRPAmount fee,
ReadView const& view,
beast::Journal const& j) override;
};
} // namespace xrpl
-4
View File
@@ -26,10 +26,6 @@ The store paths carry their derivation hash, so they change whenever a tool is
rebuilt — a `flake.lock` update generally rewrites most of them even when no
version moves. That is deliberate: it makes tooling changes visible in review.
Tools scoped to a single dev shell rather than to `commonPackages` do not appear
here: `check-tools.sh` keys those off `XRPL_DEVSHELL`, and none of the three
environments above is such a shell. Adding to one needs no snapshot update.
## Regenerating
The two Linux snapshots come from the `nix-ubuntu` image (Docker or a compatible
+73 -73
View File
@@ -2,144 +2,144 @@ Detected OS: macos (Darwin arm64)
Core build tools:
✅ cmake
cmake version 4.4.3
/nix/store/q85csxf4s4shx89zif097h1ql4ax1rrz-cmake-4.4.3/bin/cmake
cmake version 4.1.2
/nix/store/gvabsb4yqb5xsqzqph54rijnn4zpihnp-cmake-4.1.2/bin/cmake
✅ conan
Conan version 2.32.0
/nix/store/921jqsgbilixmr3xchj95si0jl8pi13z-conan-2.32.0/bin/conan
Conan version 2.28.1
/nix/store/9jiyxmkpwmn6dcqs0765s83riw3l5ail-conan-2.28.1/bin/conan
✅ git
git version 2.55.0
/nix/store/gw7c7m0dwca5lg4152x9lpmxpj171bzw-git-2.55.0/bin/git
git version 2.54.0
/nix/store/a14yxcqvv9x2l9mllgpirzhvz93pgprg-git-2.54.0/bin/git
✅ python3
Python 3.14.7
/nix/store/0kcws5blwbdyxqxjd1w8hjipz61xny3k-python3-3.14.7/bin/python3.14
Python 3.13.13
/nix/store/ygxqin6ydzjfawywqpp5pal8wv6sf5bh-python3-3.13.13/bin/python3.13
Development tooling:
✅ ccache
ccache version 4.13.6
/nix/store/ydr5nlzxp1djb5256y0zzqa87vz55gsk-ccache-4.13.6/bin/ccache
/nix/store/57davyvs6p6dkrl3svzwg1ph18wsy4cz-ccache-4.13.6/bin/ccache
✅ clang
clang version 22.1.8
/nix/store/dakcxgwz4ixkk6ps50b7vn3dvblyzsnf-clang-wrapper-22.1.8/bin/clang
clang version 22.1.7
/nix/store/192glrb2cldvziyf3378mzjqbzx3ih4g-clang-wrapper-22.1.7/bin/clang
✅ clang-22
clang version 22.1.8
/nix/store/w826sqb98nnym6kzackg829664iqdlxj-clang-22/bin/clang-22
clang version 22.1.7
/nix/store/rbap7zqq7mw00fyqa02p5rj7gqjp4w5i-clang-22/bin/clang-22
✅ clang++
clang version 22.1.8
/nix/store/dakcxgwz4ixkk6ps50b7vn3dvblyzsnf-clang-wrapper-22.1.8/bin/clang++
clang version 22.1.7
/nix/store/192glrb2cldvziyf3378mzjqbzx3ih4g-clang-wrapper-22.1.7/bin/clang++
✅ clang++-22
clang version 22.1.8
/nix/store/wnwyw3yfqrhpxij65bl3pa1x65q2zg3m-clang++-22/bin/clang++-22
clang version 22.1.7
/nix/store/v9haf787f7bcz0mq1sad4bpyx21pj6li-clang++-22/bin/clang++-22
✅ ClangBuildAnalyzer
ClangBuildAnalyzer 1.6.0
/nix/store/c7jjnw29ra78jdlzjww2m374izhy05n8-clangbuildanalyzer-1.6.0/bin/ClangBuildAnalyzer
/nix/store/4l50ds9fa2mkvh7wg8qzrlbmjs12sb8l-clangbuildanalyzer-1.6.0/bin/ClangBuildAnalyzer
✅ curl
curl 8.22.0 (aarch64-apple-darwin25.6.0) libcurl/8.22.0 OpenSSL/3.5.8 zlib/1.3.2 libssh2/1.11.1 nghttp2/1.70.0 mit-krb5/1.22.2
/nix/store/pq1h17abqy6q5y6k5dn9qj0fwb89flba-curl-8.22.0-bin/bin/curl
curl 8.20.0 (aarch64-apple-darwin25.3.0) libcurl/8.20.0 OpenSSL/3.6.2 zlib/1.3.2 libssh2/1.11.1 nghttp2/1.69.0 mit-krb5/1.22.1
/nix/store/kclq0czaxvsgh4ym9ld7b6iwy50l1snk-curl-8.20.0-bin/bin/curl
✅ file
file-5.48
/nix/store/y44vpziwbzfm8020dzlqqbybyyy7lb6g-file-5.48/bin/file
file-5.47
/nix/store/dax63li7wwcbqxxkkgzc4g2rx7d4w86x-file-5.47/bin/file
✅ less
less 710 (PCRE2 regular expressions)
/nix/store/3yn3dx5fwg5i0inqqzwnxq7f6lwhpjqa-less-710/bin/less
less 692 (PCRE2 regular expressions)
/nix/store/lvr16y75r1pdxpdv0aph5ak2yd0hkvqm-less-692/bin/less
✅ make
GNU Make 4.4.1
/nix/store/8l4wgxmhdxfvc9g18y76bnxl1ghgcp7n-gnumake-4.4.1/bin/make
/nix/store/8wwiw8pwyhrkzyq28hqzxfl4z84lks81-gnumake-4.4.1/bin/make
✅ netstat
present
/nix/store/57fqvgdav8lrmzndkwm7qrgksl01yj23-network_cmds-741.100.2/bin/netstat
/nix/store/qsd1kzqb0ahrk433vmyl245gp623j19s-network_cmds-730.80.3/bin/netstat
✅ ninja
1.13.2
/nix/store/infm4vlx71q83bqz0k559w1gn4b0rq4i-ninja-1.13.2/bin/ninja
/nix/store/bqykhrblarkj4fl0hz2mf8ngwfv6x6bz-ninja-1.13.2/bin/ninja
✅ perl
v5.42.3
/nix/store/7rvr6qjfzk2ybzn1imjw6c6qq1n1a9cr-perl-5.42.3/bin/perl
v5.42.0
/nix/store/js13ri9fvm0ajk1fpd3acigys2a9whdv-perl-5.42.0/bin/perl
✅ pkg-config
0.29.2
/nix/store/nhgylvj98klqggy54h7gbc3b2ia4a00y-pkg-config-wrapper-0.29.2/bin/pkg-config
/nix/store/lzrwr375jqhhbca116kja96xf1md83l8-pkg-config-wrapper-0.29.2/bin/pkg-config
✅ vim
VIM - Vi IMproved 9.2 (2026 Feb 14, compiled Jan 01 1980 00:00:00)
/nix/store/akaa9kqm3b8rlkv61kvsqr2l2j70rvxb-vim-9.2.1001/bin/vim
/nix/store/6vbkykg92w603c0sw3mkk7p7mfaawbns-vim-9.2.0389/bin/vim
✅ zip
Zip 3.0
/nix/store/dvawr36npnad4xa6dxi2ss0fw44aa82r-zip-3.0/bin/zip
/nix/store/z6ph729vcakbvz3wh8ln1wk6mi06w487-zip-3.0/bin/zip
✅ clang-apply-replacements
clang-apply-replacements version 22.1.8
/nix/store/5bij4zn161lagwyndmrr21vmxjg054nv-clang-tools-22.1.8/bin/clang-apply-replacements
clang-apply-replacements version 22.1.7
/nix/store/vzyyjf3cm1hbj9wcr2qcb66x6j98zpy7-clang-tools-22.1.7/bin/clang-apply-replacements
✅ clang-apply-replacements-22
clang-apply-replacements version 22.1.8
/nix/store/w4bz034k4l0w719rrmbwnqcgyjbswy81-clang-apply-replacements-22/bin/clang-apply-replacements-22
clang-apply-replacements version 22.1.7
/nix/store/m3ii69rca4077lf4wlk7m3jcag1fs577-clang-apply-replacements-22/bin/clang-apply-replacements-22
✅ clang-format
clang-format version 22.1.8
/nix/store/5bij4zn161lagwyndmrr21vmxjg054nv-clang-tools-22.1.8/bin/clang-format
clang-format version 22.1.7
/nix/store/vzyyjf3cm1hbj9wcr2qcb66x6j98zpy7-clang-tools-22.1.7/bin/clang-format
✅ clang-format-22
clang-format version 22.1.8
/nix/store/w7z346l0r8y36b6lc5i7jzj3w0581l2y-clang-format-22/bin/clang-format-22
clang-format version 22.1.7
/nix/store/4fawqy6ngqcsqd2ygyyzm93q0xy3f5gs-clang-format-22/bin/clang-format-22
✅ clang-tidy
LLVM version 22.1.8
/nix/store/5bij4zn161lagwyndmrr21vmxjg054nv-clang-tools-22.1.8/bin/clang-tidy
LLVM version 22.1.7
/nix/store/vzyyjf3cm1hbj9wcr2qcb66x6j98zpy7-clang-tools-22.1.7/bin/clang-tidy
✅ clang-tidy-22
LLVM version 22.1.8
/nix/store/0lv25qd3ddgjm3ndn70q61jfr6krlkl7-clang-tidy-22/bin/clang-tidy-22
LLVM version 22.1.7
/nix/store/jqw4280saixaxxihwdba9ldm2fsm6dr3-clang-tidy-22/bin/clang-tidy-22
✅ dot
dot - graphviz version 15.1.1 (0)
/nix/store/mc99a6bpbk2waym2cnw6ifmxli4ndb4x-graphviz-15.1.1/bin/dot
dot - graphviz version 12.2.1 (0)
/nix/store/ijb4fbnqa6wzlpqnhb6q9knqpf7qqn5z-graphviz-12.2.1/bin/dot
✅ doxygen
1.17.0
/nix/store/kz43yaj0wg6fx6cbrcmi2442z1ramy0c-doxygen-1.17.0/bin/doxygen
1.16.1
/nix/store/kbryjdpq9jizjb0ws0nzbf2h2ymbdiwm-doxygen-1.16.1/bin/doxygen
✅ gcovr
gcovr 8.4
/nix/store/czj1vxsizrbba280gg22cys40qxkqjgr-python3.14-gcovr-8.4/bin/gcovr
/nix/store/wn8jiyh9p0bybs96s4163qp3k8vfmczx-python3.13-gcovr-8.4/bin/gcovr
✅ gh
gh version 2.102.0 (2026-09-30)
/nix/store/s8bahl6zssvry93ihivfmk1l6c58byrr-gh-2.102.0/bin/gh
gh version 2.94.0 (nixpkgs)
/nix/store/fhnpw0hs0gjms1ha6ap02jq7rx13gkbp-gh-2.94.0/bin/gh
✅ git-cliff
git-cliff 2.14.2
/nix/store/k4m5yk0qb7hfslwsq1zjajz1d94j12bj-git-cliff-2.14.2/bin/git-cliff
git-cliff 2.13.1
/nix/store/cy0wwhgxa7yvrz97zydbq6sqmixc90fq-git-cliff-2.13.1/bin/git-cliff
✅ git-lfs
git-lfs/3.8.0 (3.8.0; darwin arm64; go 1.26.8)
/nix/store/wzixjr3yb40a13m2q654hm3bdjn6sal3-git-lfs-3.8.0/bin/git-lfs
git-lfs/3.7.1 (3.7.1; darwin arm64; go 1.26.3)
/nix/store/k9r7zjfjplqa4d5s71cqvf2iv73jd9mc-git-lfs-3.7.1/bin/git-lfs
✅ gpg
gpg (GnuPG) 2.4.9
/nix/store/0ldfwnwx385i5kh7cyvxx4mkvgjhs5rh-gnupg-2.4.9/bin/gpg
/nix/store/cgh6iwzz5jgx9z5whka4vgj210i6npc6-gnupg-2.4.9/bin/gpg
✅ pre-commit
pre-commit 4.6.2
/nix/store/ivh54xypg9qvd3ha9lis0i8p4nkf8n7a-pre-commit-4.6.2/bin/pre-commit
pre-commit 4.5.1
/nix/store/z3cca68620w0w10f090szgzdnmh1waf2-pre-commit-4.5.1/bin/pre-commit
✅ run-clang-tidy
usage: run-clang-tidy [-h] [-allow-enabling-alpha-checkers]
/nix/store/3xss1mm3mh1hzcmg182na6ki1p5rnxf1-run-clang-tidy/bin/run-clang-tidy
/nix/store/4x28x911z2f9y7adqlh3qspp4a16dig7-run-clang-tidy/bin/run-clang-tidy
✅ run-clang-tidy-22
usage: run-clang-tidy [-h] [-allow-enabling-alpha-checkers]
/nix/store/w68r9z07hcq4fwbfyq3yvc7qxm8aqfbl-run-clang-tidy-22/bin/run-clang-tidy-22
/nix/store/x8iymrh76sk5q91ryg5pa7i32s6gfh34-run-clang-tidy-22/bin/run-clang-tidy-22
Rust toolchain:
✅ cargo
cargo 1.97.1 (c980f4866 2026-06-30)
/nix/store/gklji8yq8yrx1k039c10kwz60vqf2dxj-rust-minimal-1.97.1/bin/cargo
/nix/store/bnfk1sl4s9angb0vj1cj9a5y5zvqinwy-rust-minimal-1.97.1/bin/cargo
✅ cargo-audit
cargo-audit-audit 0.22.2
/nix/store/7jgrwj557l91940aj0ag4i924ima1cqn-cargo-audit-0.22.2/bin/cargo-audit
cargo-audit-audit 0.22.1
/nix/store/snwkga2f5gyf404h7mmp9wriwxb8v65f-cargo-audit-0.22.1/bin/cargo-audit
✅ cargo-llvm-cov
cargo-llvm-cov 0.9.0
/nix/store/5q1h285z8zzkw2qxig9a0b1avcsk1cf0-cargo-llvm-cov-0.9.0/bin/cargo-llvm-cov
cargo-llvm-cov 0.8.5
/nix/store/fpiqdh91gwyxalqp409ynm0s0g086w7w-cargo-llvm-cov-0.8.5/bin/cargo-llvm-cov
✅ cargo-nextest
cargo-nextest 0.9.146
/nix/store/69v3hx3k597hcxd2zy7yad7fg0aw5yxd-cargo-nextest-0.9.146/bin/cargo-nextest
cargo-nextest 0.9.137
/nix/store/ylz7m947mhkgsp6i7611id3s3gcd58nq-cargo-nextest-0.9.137/bin/cargo-nextest
✅ clippy-driver
clippy 0.1.97 (8bab26f4f6 2026-07-14)
/nix/store/gklji8yq8yrx1k039c10kwz60vqf2dxj-rust-minimal-1.97.1/bin/clippy-driver
/nix/store/bnfk1sl4s9angb0vj1cj9a5y5zvqinwy-rust-minimal-1.97.1/bin/clippy-driver
✅ rust-analyzer
rust-analyzer 1.97.1 (8bab26f4 2026-07-14)
/nix/store/30rbyk8gx7v53fxqhs4pp0fvxfjycspr-rust-analyzer-preview-1.97.1-aarch64-apple-darwin/bin/rust-analyzer
/nix/store/j6apc5pmd0giy15da9p650r8zklslmvi-rust-analyzer-preview-1.97.1-aarch64-apple-darwin/bin/rust-analyzer
✅ rust-nightly
rustc 1.101.0-nightly (282215592 2026-10-04)
/nix/store/a9552rf3pylcnkijc1z0hf1lcnv50djq-rust-nightly/bin/rust-nightly
rustc 1.99.0-nightly (87e5904f5 2026-07-20)
/nix/store/fqpjz4l0nsnji8b2pz57mnj0akbp6hcl-rust-nightly/bin/rust-nightly
✅ rustc
rustc 1.97.1 (8bab26f4f 2026-07-14)
/nix/store/gklji8yq8yrx1k039c10kwz60vqf2dxj-rust-minimal-1.97.1/bin/rustc
/nix/store/bnfk1sl4s9angb0vj1cj9a5y5zvqinwy-rust-minimal-1.97.1/bin/rustc
✅ rustfmt
rustfmt 1.9.0-stable (8bab26f4f6 2026-07-14)
/nix/store/0jmdipyn8266j9kqx2xxm2qkxlrlrjfc-rustfmt-preview-1.97.1-aarch64-apple-darwin/bin/rustfmt
/nix/store/5ymwgr9jqjz7zzbmj0j5vqbwcd3kp0vm-rustfmt-preview-1.97.1-aarch64-apple-darwin/bin/rustfmt
Skipping git-over-HTTPS check (CHECK_TOOLS_SKIP_CLONE is set).
+1 -2
View File
@@ -21,8 +21,7 @@ let
)
);
toolchain =
if pkgs.stdenv.hostPlatform.isLinux then linux.toolchain else (darwin.toolchain ++ [ darwinEnv ]);
toolchain = if pkgs.stdenv.isLinux then linux.toolchain else (darwin.toolchain ++ [ darwinEnv ]);
in
{
default = pkgs.buildEnv {
+6 -20
View File
@@ -20,16 +20,15 @@ let
# Custom-glibc stdenvs, matching the CI environment. darwin has no custom
# glibc, so there they fall back to the plain nixpkgs stdenvs.
customGccStdenv = if pkgs.stdenv.hostPlatform.isLinux then linux.gccStdenv else plainGccStdenv;
customClangStdenv =
if pkgs.stdenv.hostPlatform.isLinux then linux.clangStdenv else plainClangStdenv;
customGccStdenv = if pkgs.stdenv.isLinux then linux.gccStdenv else plainGccStdenv;
customClangStdenv = if pkgs.stdenv.isLinux then linux.clangStdenv else plainClangStdenv;
# gcov matching each gcc shell, so `-Dcoverage=ON` builds work in the shell.
plainGcov = mkGcov {
name = "plain";
cc = gccPackage.cc;
};
customGccGcov = if pkgs.stdenv.hostPlatform.isLinux then linux.gcov else plainGcov;
customGccGcov = if pkgs.stdenv.isLinux then linux.gcov else plainGcov;
# Whole directory: init.sh locates the profiles relative to itself.
conanDir = ../conan;
@@ -52,7 +51,7 @@ let
# Not sdkEnv: a shell's stdenv already sets that up. Prepended so the stub
# beats the nixpkgs libresolv this shell's tooling drags in.
darwinLibresolvHook = pkgs.lib.optionalString pkgs.stdenv.hostPlatform.isDarwin (
darwinLibresolvHook = pkgs.lib.optionalString pkgs.stdenv.isDarwin (
pkgs.lib.concatLines (
pkgs.lib.mapAttrsToList (
name: value: ''export ${name}="${value} ''${${name}:-}"''
@@ -127,7 +126,7 @@ let
in
rec {
# macOS: Nix Clang. Linux: Nix GCC.
default = if pkgs.stdenv.hostPlatform.isDarwin then clang else gcc;
default = if pkgs.stdenv.isDarwin then clang else gcc;
# gcc/clang use the custom-glibc toolchain, matching CI. On darwin there is no
# custom glibc, so they fall back to the plain nixpkgs toolchain.
@@ -148,19 +147,6 @@ rec {
versionedTools = clangVersionedTools;
};
# The gcc shell plus the Lean4 formal verification toolchain
formal-verification = makeShell {
shellName = "formal-verification";
stdenv = customGccStdenv;
compilerName = "gcc";
version = gccVersion;
versionedTools = gccVersionedTools;
extraPackages = [
customGccGcov
pkgs.lean4
];
};
# Nix provides no compiler; use the one from your system (e.g. Apple Clang).
no-compiler = makeShell {
shellName = "no-compiler";
@@ -172,7 +158,7 @@ rec {
# The *-plain shells (stock nixpkgs toolchain) exist only on Linux: on darwin
# gcc/clang are already plain, so these would be redundant and are omitted, which
# makes `nix develop .#gcc-plain` fail there rather than silently aliasing gcc.
// pkgs.lib.optionalAttrs pkgs.stdenv.hostPlatform.isLinux {
// pkgs.lib.optionalAttrs pkgs.stdenv.isLinux {
gcc-plain = makeShell {
shellName = "gcc-plain";
stdenv = plainGccStdenv;
+4 -6
View File
@@ -217,13 +217,12 @@ The `release-info` action decides the channel from the event, and
| tag | `X.Y.Z-bN` | `beta` | `deb-beta` | `rpm-beta-hosted` |
| tag, any other | `xrpld --version` | `custom` | `deb-custom` | `rpm-custom-hosted` |
| push to `develop` | `0.0.0-dev+<hash>` | `develop` | `deb-develop` | `rpm-develop-hosted` |
| tag, non-public codebase | _any but `X.Y.Z`_ | `private` | `deb-private` | `rpm-private-hosted` |
| tag, non-public codebase | _any_ | `private` | `deb-private` | `rpm-private-hosted` |
A variant is published to the same channel under its own name, so
`xrpld-assert` never overwrites `xrpld`.
Only a tag picks a release channel. A final release, `X.Y.Z`, goes to `stable`
even from a non-public codebase. Versions sort in row order, so moving to a
Only a tag picks a release channel. Versions sort in row order, so moving to a
more mature channel never downgrades. A tag matching none of the release
patterns, such as `X.Y.Z-hotfix1`, publishes to `custom`, which sits outside
that order.
@@ -285,9 +284,8 @@ checks that the server starts in each image. A tag's images are tagged with the
tag name, a develop image as `develop`. With `publish: true`:
- `xrpld` is pushed to `xrplf/xrpld` on Docker Hub using the `DOCKERHUB_TOKEN`
secret, an organization access token for `xrplf`. Builds of a non-public
codebase are pushed there only for `stable` releases, whose packages are
public too.
secret, an organization access token for `xrplf`. Private builds are never
pushed there.
- `voidstar` replaces `/usr/bin/xrpld` with the binary of the `voidstar` build
config, adds `libvoidstar.so` and links the binary into `/symbols`, as
Antithesis expects. It is pushed as `xrpld-voidstar` to the Antithesis
@@ -1,3 +1,3 @@
halt_on_error=false
halt_on_error=true
verbosity=1
second_deadlock_stack=1
+59 -87
View File
@@ -1,102 +1,74 @@
# The idea is to empty this file gradually by fixing the underlying issues and removing suppresions.
# TSAN suppression file for rippled.
# Only suppress issues in third-party libraries and TSAN's own instrumentation.
# Races in rippled's own code should be fixed, not suppressed.
# Suppress race in Boost ASIO scheduler detected by GCC-15
# This is a false positive in Boost's internal pipe() synchronization
# Boost ASIO / Boost Context false positives
# These are internal to Boost's reactor and the fiber context switching used
# by boost::asio::spawn (server Spawn.h, test yield_to.h).
# NOTE: a suppression fires if ANY frame in the report matches, so these
# path patterns can also hide rippled races that merely pass through asio
# (e.g. handlers running on an io_context). Narrow them to specific frames
# once TSAN runs cleanly.
race:boost/asio/
race:boost/context/
race:boost/asio/executor.hpp
race:boost::asio
race:__cxxabiv1::manage_exception_state
# Suppress tsan related issues in xrpld code.
race:src/libxrpl/basics/make_SSLContext.cpp
race:src/libxrpl/basics/Number.cpp
race:src/libxrpl/json/json_value.cpp
race:src/libxrpl/json/to_string.cpp
race:src/libxrpl/ledger/OpenView.cpp
race:src/libxrpl/net/HTTPClient.cpp
race:src/libxrpl/nodestore/backend/NuDBFactory.cpp
race:src/libxrpl/protocol/InnerObjectFormats.cpp
race:src/libxrpl/protocol/STParsedJSON.cpp
race:src/libxrpl/resource/ResourceManager.cpp
race:src/test/app/Flow_test.cpp
race:src/test/app/LedgerReplay_test.cpp
race:src/test/app/NFToken_test.cpp
race:src/test/app/Offer_test.cpp
race:src/test/app/ValidatorSite_test.cpp
race:src/test/consensus/NegativeUNL_test.cpp
race:src/test/jtx/impl/Env.cpp
race:src/test/jtx/impl/JSONRPCClient.cpp
race:src/test/jtx/impl/pay.cpp
race:src/test/jtx/impl/token.cpp
race:src/test/rpc/Book_test.cpp
race:src/xrpld/app/ledger/detail/InboundTransactions.cpp
race:src/xrpld/app/main/Application.cpp
race:src/xrpld/app/main/BasicApp.cpp
race:src/xrpld/app/main/GRPCServer.cpp
race:src/xrpld/app/misc/detail/AmendmentTable.cpp
race:src/xrpld/app/misc/FeeVoteImpl.cpp
race:src/xrpld/app/rdb/detail/Wallet.cpp
race:src/xrpld/overlay/detail/OverlayImpl.cpp
race:src/xrpld/peerfinder/detail/PeerfinderManager.cpp
race:src/xrpld/peerfinder/detail/SourceStrings.cpp
race:src/xrpld/rpc/detail/ServerHandler.cpp
race:xrpl/server/detail/Door.h
race:xrpl/server/detail/Spawn.h
race:xrpl/server/detail/ServerImpl.h
race:xrpl/nodestore/detail/DatabaseNodeImp.h
race:src/libxrpl/beast/utility/beast_Journal.cpp
race:src/test/beast/LexicalCast_test.cpp
race:ServerHandler
# More suppressions in external library code.
# TSAN's own syscall interceptors and libc internals
# These are false positives from TSAN's fd tracking in glibc.
# TODO: Tighten these suppressions once TSAN runs cleanly — they may mask
# real fd-lifetime bugs in rippled's networking code.
race:crtstuff.c
race:pipe
race:__pipe
race:epoll_ctl
race:epoll_create
race:closedir
race:__close_nocancel
race:__GI___close
race:__socket
race:__GI___accept4
race:eventfd
# Deadlock / lock-order-inversion suppressions
# Note: GCC's TSAN may not fully support all deadlock suppression patterns
deadlock:src/libxrpl/beast/utility/beast_Journal.cpp
deadlock:src/libxrpl/beast/utility/beast_PropertyStream.cpp
deadlock:src/test/beast/beast_PropertyStream_test.cpp
deadlock:src/xrpld/core/detail/Workers.cpp
deadlock:src/xrpld/app/misc/detail/Manifest.cpp
deadlock:src/xrpld/app/misc/detail/ValidatorList.cpp
deadlock:src/xrpld/app/misc/detail/ValidatorSite.cpp
# C++ standard library internals
race:ostreambuf_iterator
race:basic_ostream
signal:src/libxrpl/beast/utility/beast_Journal.cpp
signal:src/xrpld/core/detail/Workers.cpp
signal:src/xrpld/core/JobQueue.cpp
signal:Workers::Worker
# Aggressive suppressing of deadlock tsan errors
# Deadlock false positives in Boost and threading primitives
deadlock:pthread_create
deadlock:pthread_rwlock_rdlock
deadlock:boost::asio
# Suppress SEGV crashes in TSAN itself during stringbuf operations
# This appears to be a GCC-15 TSAN instrumentation issue with basic_stringbuf::str()
# Commonly triggered in beast::Journal::ScopedStream destructor
signal:std::__cxx11::basic_stringbuf
# Pre-existing rippled issues (suppress until fixed, tracked separately)
# TODO(RIPD-XXXX): beast::Journal::Sink::threshold is not thread-safe.
# The severity field is read/written from multiple threads without synchronization.
race:beast::Journal::Sink::threshold
# TODO(RIPD-XXXX): LedgerReplayTask/LedgerDeltaAcquire lock-order-inversion.
# LedgerDeltaAcquire::notify() calls LedgerReplayTask::deltaReady() while
# holding its own mutex, but deltaReady() acquires LedgerReplayTask's mutex,
# creating an A->B / B->A lock ordering cycle.
deadlock:xrpl::LedgerDeltaAcquire
deadlock:xrpl::LedgerReplayTask
# TODO(RIPD-XXXX): SHAMap copy constructor / setLedgerSeq data race.
race:xrpl::SHAMap::SHAMap
race:xrpl::SHAMap::setLedgerSeq
# TODO(RIPD-XXXX): RCLConsensus / NetworkOPsImp lock-order-inversion.
# doLedgerAccept() acquires NetworkOPsImp mutex then RCLConsensus::startRound()
# acquires consensus mutex (M0→M1), while RCLConsensus::Adaptor::doAccept()
# acquires them in reverse order via std::lock (M1→M0). The std::lock path is
# actually safe (deadlock-free), but TSAN flags the inconsistent ordering.
deadlock:xrpl::RCLConsensus
deadlock:xrpl::doLedgerAccept
# TODO(RIPD-XXXX): beast::PropertyStream::Source lock-order-inversion.
# find_one_deep() acquires parent's recursive_mutex then recursively acquires
# children's mutexes, while find_one() can acquire them in reverse order.
# This is a pre-existing issue in beast's PropertyStream tree traversal.
deadlock:beast::PropertyStream::Source
# Signal/crash suppressions for GCC TSAN instrumentation issues
signal:basic_stringbuf
signal:basic_ostringstream
called_from_lib:libclang_rt
race:ostreambuf_iterator
race:basic_ostream
# Suppress SEGV in Boost ASIO memory allocation with GCC-15 TSAN
signal:boost::asio::aligned_new
signal:boost::asio::detail::memory
# Suppress SEGV in execute_native_thread_routine
signal:execute_native_thread_routine
# Suppress data race in Boost Context fiber management
# This is a false positive in Boost's exception state management during fiber context switching
race:__cxxabiv1::manage_exception_state
race:boost::context::fiber::resume
race:boost::asio::detail::spawned_fiber_thread
race:boost::asio::detail::spawned_fiber_thread::suspend_with
race:boost::asio::detail::spawned_fiber_thread::destroy
# Suppress data race in __tsan_memcpy called from Boost fiber operations
race:__tsan_memcpy
+39
View File
@@ -0,0 +1,39 @@
#include <xrpl/basics/CountedObject.h>
#include <algorithm>
namespace xrpl {
CountedObjects&
CountedObjects::getInstance() noexcept
{
static CountedObjects kInstance;
return kInstance;
}
CountedObjects::CountedObjects() noexcept : count_(0), head_(nullptr)
{
}
CountedObjects::List
CountedObjects::getCounts(int minimumThreshold) const
{
List counts;
// When other operations are concurrent, the count
// might be temporarily less than the actual count.
counts.reserve(count_.load());
for (auto* ctr = head_.load(); ctr != nullptr; ctr = ctr->getNext())
{
if (ctr->getCount() >= minimumThreshold)
counts.emplace_back(ctr->getName(), ctr->getCount());
}
std::ranges::sort(counts);
return counts;
}
} // namespace xrpl
+2 -4
View File
@@ -5,7 +5,6 @@
#include <cerrno>
#include <cstddef>
#include <filesystem>
#include <format>
#include <fstream>
#include <iomanip>
#include <ios>
@@ -100,13 +99,12 @@ uniqueRandomPath(
if (ec)
{
Throw<std::runtime_error>(
std::format("Unable to check path '{}': {}", candidate.string(), ec.message()));
"Unable to check path '" + candidate.string() + "': " + ec.message());
}
if (!exists)
return candidate;
}
Throw<std::runtime_error>(
std::format("Unable to generate a unique path under '{}'", base.string()));
Throw<std::runtime_error>("Unable to generate a unique path under '" + base.string() + "'");
}
TempDir::TempDir() : path_(uniqueRandomPath(std::filesystem::temp_directory_path()))
+2 -2
View File
@@ -80,10 +80,10 @@ BookDirs::const_iterator::operator++()
XRPL_ASSERT(index_ != kZero, "xrpl::BookDirs::const_iterator::operator++ : nonzero index");
if (!cdirNext(*view_, curKey_, sle_, entry_, index_))
{
if (index_ == kZero)
if (index_ == 0)
curKey_ = view_->succ(++curKey_, nextQuality_).value_or(kZero);
if (index_ != kZero || curKey_ == kZero)
if (index_ != 0 || curKey_ == kZero)
{
curKey_ = key_;
entry_ = 0;
+2 -3
View File
@@ -11,7 +11,6 @@
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/ledger/LedgerTiming.h>
#include <xrpl/ledger/ReadView.h>
#include <xrpl/ledger/entries/FeeSettingsEntry.h>
#include <xrpl/nodestore/NodeObject.h>
#include <xrpl/protocol/Feature.h>
#include <xrpl/protocol/Fees.h>
@@ -262,7 +261,7 @@ Ledger::Ledger(Ledger const& prevLedger, NetClock::time_point closeTime)
{
header_.seq = prevLedger.header_.seq + 1;
header_.parentCloseTime = prevLedger.header_.closeTime;
header_.hash = prevLedger.header().hash.next();
header_.hash = prevLedger.header().hash + UInt256(1);
header_.drops = prevLedger.header().drops;
header_.closeTimeResolution = prevLedger.header_.closeTimeResolution;
header_.parentHash = prevLedger.header().hash;
@@ -561,7 +560,7 @@ Ledger::setup()
try
{
if (auto const sle = FeeSettingsEntryR(*this))
if (auto const sle = read(keylet::feeSettings()))
{
bool oldFees = false;
bool newFees = false;
+1 -2
View File
@@ -16,7 +16,6 @@
#include <boost/container/pmr/monotonic_buffer_resource.hpp>
#include <cstddef>
#include <format>
#include <memory>
#include <optional>
#include <stdexcept>
@@ -265,7 +264,7 @@ OpenView::rawTxInsert(
auto const result = txs_.emplace(
std::piecewise_construct, std::forward_as_tuple(key), std::forward_as_tuple(txn, metaData));
if (!result.second)
Throw<std::logic_error>(std::format("rawTxInsert: duplicate TX id: {}", to_string(key)));
Throw<std::logic_error>("rawTxInsert: duplicate TX id: " + to_string(key));
}
} // namespace xrpl
+10 -8
View File
@@ -9,7 +9,6 @@
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/ledger/ApplyView.h>
#include <xrpl/ledger/ReadView.h>
#include <xrpl/ledger/entries/LedgerHashesEntry.h>
#include <xrpl/ledger/helpers/AccountRootHelpers.h>
#include <xrpl/ledger/helpers/CredentialHelpers.h>
#include <xrpl/ledger/helpers/DirectoryHelpers.h>
@@ -362,16 +361,17 @@ hashOfSeq(ReadView const& ledger, LedgerIndex seq, beast::Journal journal)
if (int const diff = ledger.seq() - seq; diff <= 256)
{
// Within 256...
LedgerHashesEntryR const hashIndex(ledger, journal);
auto const hashIndex = ledger.read(keylet::skip());
if (hashIndex)
{
XRPL_ASSERT(
hashIndex->getFieldU32(sfLastLedgerSequence) == (ledger.seq() - 1),
"xrpl::hashOfSeq : matching ledger sequence");
if (auto const hash = hashIndex.hashAt(diff - 1))
return hash;
STVector256 vec = hashIndex->getFieldV256(sfHashes);
if (vec.size() >= diff)
return vec[vec.size() - diff];
JLOG(journal.warn()) << "Ledger " << ledger.seq() << " missing hash for " << seq << " ("
<< hashIndex->getFieldV256(sfHashes).size() << "," << diff << ")";
<< vec.size() << "," << diff << ")";
}
else
{
@@ -387,14 +387,16 @@ hashOfSeq(ReadView const& ledger, LedgerIndex seq, beast::Journal journal)
}
// in skiplist
LedgerHashesEntryR const hashIndex(keylet::skip(seq), ledger, journal);
auto const hashIndex = ledger.read(keylet::skip(seq));
if (hashIndex)
{
auto const lastSeq = hashIndex->getFieldU32(sfLastLedgerSequence);
XRPL_ASSERT(lastSeq >= seq, "xrpl::hashOfSeq : minimum last ledger");
XRPL_ASSERT((lastSeq & 0xff) == 0, "xrpl::hashOfSeq : valid last ledger");
if (auto const hash = hashIndex.hashAt((lastSeq - seq) >> 8))
return hash;
auto const diff = (lastSeq - seq) >> 8;
STVector256 vec = hashIndex->getFieldV256(sfHashes);
if (vec.size() > diff)
return vec[vec.size() - diff - 1];
}
JLOG(journal.warn()) << "Can't get seq " << seq << " from " << ledger.seq() << " error";
return std::nullopt;
@@ -506,7 +506,7 @@ pseudoAccountAddress(ReadView const& view, UInt256 const& pseudoOwnerKey)
RipeshaHasher rsh;
auto const hash = sha512Half(i, view.header().parentHash, pseudoOwnerKey);
rsh(hash.data(), hash.size());
AccountID const ret{static_cast<RipeshaHasher::result_type>(rsh)};
AccountID const ret = AccountID::fromRaw(static_cast<RipeshaHasher::result_type>(rsh));
if (!view.read(keylet::account(ret)))
return ret;
}
+7 -10
View File
@@ -33,7 +33,6 @@
#include <algorithm>
#include <cstddef>
#include <cstdint>
#include <format>
#include <functional>
#include <iterator>
#include <memory>
@@ -395,11 +394,9 @@ removeToken(ApplyView& view, AccountID const& owner, UInt256 const& nftokenID, S
if (!page2)
{
Throw<std::runtime_error>(std::format(
"page {} has a broken {} field pointing to {}",
to_string(page1->key()),
field.getName(),
to_string(*id)));
Throw<std::runtime_error>(
"page " + to_string(page1->key()) + " has a broken " + field.getName() +
" field pointing to " + to_string(*id));
}
}
@@ -600,7 +597,7 @@ removeTokenOffersWithLimit(ApplyView& view, Keylet const& directory, std::size_t
else
{
Throw<std::runtime_error>(
std::format("Offer {} cannot be deleted!", to_string(offerIndexes[i])));
"Offer " + to_string(offerIndexes[i]) + " cannot be deleted!");
}
}
@@ -737,9 +734,9 @@ repairNFTokenDirectoryLinks(ApplyView& view, AccountID const& owner)
if (!newPrev)
{
// LCOV_EXCL_START
Throw<std::runtime_error>(std::format(
"NFTokenPage directory for {} cannot be repaired. Unexpected link problem.",
to_string(owner)));
Throw<std::runtime_error>(
"NFTokenPage directory for " + to_string(owner) +
" cannot be repaired. Unexpected link problem.");
// LCOV_EXCL_STOP
}
newPrev->at(sfNextPageMin) = nextPage->key();
+3 -7
View File
@@ -1,6 +1,7 @@
#include <xrpl/nodestore/detail/DecodedBlob.h>
#include <xrpl/basics/Blob.h>
#include <xrpl/basics/base_uint.h>
#include <xrpl/basics/safe_cast.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/nodestore/NodeObject.h>
@@ -11,13 +12,8 @@
namespace xrpl::node_store {
DecodedBlob::DecodedBlob(std::optional<UInt256> key, void const* value, int valueBytes)
DecodedBlob::DecodedBlob(void const* key, void const* value, int valueBytes) : key_(key)
{
if (!key)
return;
key_ = key.value();
/* Data format:
Bytes
@@ -67,7 +63,7 @@ DecodedBlob::createObject()
{
Blob data(objectData_, objectData_ + dataBytes_);
object = NodeObject::createObject(objectType_, std::move(data), key_);
object = NodeObject::createObject(objectType_, std::move(data), UInt256::fromVoid(key_));
}
return object;
+3 -15
View File
@@ -38,7 +38,6 @@
#include <functional>
#include <memory>
#include <optional>
#include <span>
#include <sstream>
#include <stdexcept>
#include <string>
@@ -217,7 +216,7 @@ public:
[&hash, pno, &status](void const* data, std::size_t size) {
nudb::detail::buffer bf;
auto const result = nodeobjectDecompress(data, size, bf);
DecodedBlob decoded(hash, result.first, result.second);
DecodedBlob decoded(hash.data(), result.first, result.second);
if (!decoded.wasOk())
{
status = Status::DataCorrupt;
@@ -241,7 +240,7 @@ public:
nudb::error_code ec;
nudb::detail::buffer bf;
auto const result = nodeobjectCompress(e.getData(), e.getSize(), bf);
db.insert(e.getKey().data(), result.first, result.second, ec);
db.insert(e.getKey(), result.first, result.second, ec);
if (ec && ec != nudb::error::key_exists)
Throw<nudb::system_error>(ec);
}
@@ -289,25 +288,14 @@ public:
Throw<nudb::system_error>(ec);
nudb::visit(
dp,
[&](void const* keyData,
[&](void const* key,
std::size_t keyBytes,
void const* data,
std::size_t size,
nudb::error_code&) {
auto key = uint256::fromRaw(
std::span{static_cast<unsigned char const*>(keyData), keyBytes});
if (!key) [[unlikely]]
{
ec = make_error_code(nudb::error::invalid_key_size);
return;
}
nudb::detail::buffer bf;
auto const result = nodeobjectDecompress(data, size, bf);
DecodedBlob decoded(key, result.first, result.second);
if (!decoded.wasOk())
{
ec = make_error_code(nudb::error::missing_value);
@@ -35,10 +35,8 @@
#include <atomic>
#include <cstddef>
#include <filesystem>
#include <format>
#include <functional>
#include <memory>
#include <span>
#include <stdexcept>
#include <string>
@@ -112,9 +110,6 @@ public:
if (!getIfExists(keyValues, Keys::kPath, name))
Throw<std::runtime_error>("Missing path in RocksDBFactory backend");
if (keyBytes != uint256::size())
Throw<std::runtime_error>("Incorrect key size: expected 32 bytes");
rocksdb::BlockBasedTableOptions tableOptions;
options.env = env;
@@ -198,7 +193,7 @@ public:
if (!s.ok())
{
Throw<std::runtime_error>(
std::format("Unable to set RocksDB bbt_options: {}", s.ToString()));
std::string("Unable to set RocksDB bbt_options: ") + s.ToString());
}
}
@@ -211,7 +206,7 @@ public:
if (!s.ok())
{
Throw<std::runtime_error>(
std::format("Unable to set RocksDB options: {}", s.ToString()));
std::string("Unable to set RocksDB options: ") + s.ToString());
}
}
@@ -246,7 +241,7 @@ public:
if (!status.ok() || (localDb == nullptr))
{
Throw<std::runtime_error>(
std::format("Unable to open/create RocksDB: {}", status.ToString()));
std::string("Unable to open/create RocksDB: ") + status.ToString());
}
db.reset(localDb);
}
@@ -296,7 +291,7 @@ public:
if (getStatus.ok())
{
DecodedBlob decoded(hash, string.data(), string.size());
DecodedBlob decoded(hash.data(), string.data(), string.size());
if (decoded.wasOk())
{
@@ -321,8 +316,8 @@ public:
}
else
{
status = checkedCast<Status>(
safeCast<int>(Status::CustomCode) + safeCast<int>(getStatus.code()));
status = static_cast<Status>(
static_cast<int>(Status::CustomCode) + unsafeCast<int>(getStatus.code()));
JLOG(journal.error()) << getStatus.ToString();
}
@@ -351,7 +346,7 @@ public:
EncodedBlob const encoded(e);
wb.Put(
rocksdb::Slice(reinterpret_cast<char const*>(encoded.getKey().data()), keyBytes),
rocksdb::Slice(reinterpret_cast<char const*>(encoded.getKey()), keyBytes),
rocksdb::Slice(
reinterpret_cast<char const*>(encoded.getData()), encoded.getSize()));
}
@@ -361,7 +356,7 @@ public:
auto ret = db->Write(options, &wb);
if (!ret.ok())
Throw<std::runtime_error>(std::format("storeBatch failed: {}", ret.ToString()));
Throw<std::runtime_error>("storeBatch failed: " + ret.ToString());
}
void
@@ -379,12 +374,9 @@ public:
for (it->SeekToFirst(); it->Valid(); it->Next())
{
if (it->key().size() == keyBytes && it->key().size() == uint256::size())
if (it->key().size() == keyBytes)
{
DecodedBlob decoded(
uint256::fromRaw(std::span{it->key().data(), it->key().size()}),
it->value().data(),
it->value().size());
DecodedBlob decoded(it->key().data(), it->value().data(), it->value().size());
if (decoded.wasOk())
{
+82 -116
View File
@@ -8,8 +8,6 @@
#include <xrpl/protocol/digest.h>
#include <xrpl/protocol/tokens.h>
#include <algorithm>
#include <array>
#include <atomic>
#include <cstdint>
#include <cstring>
@@ -21,143 +19,95 @@
namespace xrpl {
namespace {
namespace detail {
/** If true, we implement a cache for account ID lookups.
This is generally a good idea and can improve performance, especially
when serving client requests.
The cache may generate warnings under some sanitizers; if you want to
disable caching, set this variable to false.
/**
* Caches the base58 representations of AccountIDs
*/
constexpr bool cacheAccountIDs = true;
/** Represents a cached entry for the given AccountID.
We keep the size and alignment of the structure at 64 bytes to
make sure it fits within a cache line and avoid false sharing.
*/
struct alignas(64) CachedAccountID
class AccountIdCache
{
/** The sequence lock for the entry.
private:
struct CachedAccountID
{
AccountID id;
char encoding[40] = {0};
};
An even value means that the data is stable, whereas an odd
value indicates the data is being actively modified.
// The actual cache
std::vector<CachedAccountID> cache_;
We use a 32-bit value to ensure that the counter will never
return to an observed value during a read window.
*/
std::atomic<std::uint32_t> seq{0};
// We use a hash function designed to resist algorithmic complexity attacks
HardenedHash<> hasher_;
/** The account id. */
AccountID id{};
// 64 spinlocks, packed into a single 64-bit value
std::atomic<std::uint64_t> locks_ = 0;
/** The length of the cached encoding.
public:
AccountIdCache(std::size_t count) : cache_(count)
{
// This is non-binding, but we try to avoid wasting memory that
// is caused by overallocation.
cache_.shrink_to_fit();
}
This field disambiguates collisions between ACCOUNT_ZERO and
never-written-to slots, since they would otherwise share the
same encoded bits.
*/
std::uint8_t len = 0;
std::string
toBase58(AccountID const& id)
{
auto const index = hasher_(id) % cache_.size();
/** The actual string encoding. */
char encoding[39] = {0};
PackedSpinlock sl(locks_, index % 64);
{
std::scoped_lock const lock(sl);
// The check against the first character of the encoding ensures
// that we don't mishandle the case of the all-zero account:
if (cache_[index].encoding[0] != 0 && cache_[index].id == id)
return cache_[index].encoding;
}
auto ret = encodeBase58Token(TokenType::AccountID, id.data(), id.size());
XRPL_ASSERT(ret.size() <= 38, "xrpl::detail::AccountIdCache : maximum result size");
{
std::scoped_lock const lock(sl);
cache_[index].id = id;
std::strcpy(cache_[index].encoding, ret.c_str());
}
return ret;
}
};
static_assert(sizeof(CachedAccountID) == 64 && alignof(CachedAccountID) == 64);
} // namespace detail
// The actual cache to use: a simple, direct-mapped, best-effort construct.
//
// We do not use a hardened hash for this because an AccountID is already
// just cryptographically generated (i.e. uniformly distributed) bits and
// there is no amplification, since collisions do not carry an additional
// cost.
//
// The size is a "nice" power of 2 because it makes the calculation of an
// index easier.
constinit std::array<CachedAccountID, 65536> gAccountIdCache{};
static std::unique_ptr<detail::AccountIdCache> gAccountIdCache;
} // namespace
void
initAccountIdCache(std::size_t count)
{
if (!gAccountIdCache && count != 0)
gAccountIdCache = std::make_unique<detail::AccountIdCache>(count);
}
std::string
toBase58(AccountID const& v)
{
// Since the input bits are generated by a cryptographically secure
// hash construct, we can just pick any subset of them, no muss, no
// fuss.
auto const index = [&v]() -> std::uint16_t {
auto const data = v.data();
if (gAccountIdCache)
return gAccountIdCache->toBase58(v);
return static_cast<std::uint16_t>(data[0]) | (static_cast<std::uint16_t>(data[1]) << 8);
}();
if constexpr (cacheAccountIDs)
{
auto& slot = gAccountIdCache[index];
// This is the optimistic, store-free read path: a collision with a
// concurrent writer will be detected because the seq recheck fails
// and we fall through to the slow path.
//
// These unsynchronized reads of the slot fields are, formally, data
// races and, technically speaking, result in undefined behavior and
// a tool like TSAN might flag them. These should not have an impact
// on the correctness of the returned data: the sequence lock checks
// ensure we discard any data that was in flux.
if (auto const s0 = slot.seq.load(std::memory_order::acquire);
(s0 & 1) == 0 && slot.len != 0 && slot.id == v)
{
char buf[38];
auto const len = std::min<std::size_t>(slot.len, sizeof(buf));
std::memcpy(buf, slot.encoding, len);
// Prevents the loads above from sinking below the recheck; the
// acquire on s0 constrains later operations, not earlier ones.
std::atomic_thread_fence(std::memory_order::acquire);
// Everything read from the slot — the gates above and the bytes in
// buf — precedes this recheck and is validated by it. No slot access
// may follow it.
if (slot.seq.load(std::memory_order::relaxed) == s0)
return std::string(buf, len);
}
}
auto ret = encodeBase58Token(TokenType::AccountID, v.data(), v.size());
XRPL_ASSERT(!ret.empty() && ret.size() <= 38, "xrpl::toBase58 : maximum result size");
if constexpr (cacheAccountIDs)
{
auto& slot = gAccountIdCache[index];
// The CAS to an odd value excludes other writers; if it fails, another
// thread wrote or is writing into this slot. That is not a problem: we
// just don't store our result into the cache this time.
if (auto s0 = slot.seq.load(std::memory_order::relaxed); (s0 & 1) == 0 &&
slot.seq.compare_exchange_strong(s0, s0 + 1, std::memory_order::acquire))
{
slot.id = v;
slot.len = static_cast<std::uint8_t>(ret.size());
std::copy_n(ret.data(), ret.size(), slot.encoding);
// The acquire on the CAS above keeps the writes from becoming visible
// before the seqlock transitions to an odd value, which means readers
// can detect an in-progress write. This release publishes them before
// the seqlock transitions to an even value.
slot.seq.store(s0 + 2, std::memory_order::release);
}
}
return ret;
return encodeBase58Token(TokenType::AccountID, v.data(), v.size());
}
template <>
std::optional<AccountID>
parseBase58(std::string const& s)
{
return AccountID::fromRaw(decodeBase58Token(s, TokenType::AccountID));
auto const result = decodeBase58Token(s, TokenType::AccountID);
if (result.size() != AccountID::kBytes)
return std::nullopt;
return AccountID::fromRaw(result);
}
//------------------------------------------------------------------------------
@@ -198,9 +148,25 @@ parseBase58(std::string const& s)
AccountID
calcAccountID(PublicKey const& pk)
{
static_assert(AccountID::kBytes == sizeof(RipeshaHasher::result_type));
RipeshaHasher rsh;
rsh(pk.data(), pk.size());
return AccountID{static_cast<RipeshaHasher::result_type>(rsh)};
return AccountID::fromRaw(static_cast<RipeshaHasher::result_type>(rsh));
}
AccountID const&
xrpAccount()
{
static AccountID const kAccount(beast::kZero);
return kAccount;
}
AccountID const&
noAccount()
{
static AccountID const kAccount(1);
return kAccount;
}
bool
+5 -6
View File
@@ -185,9 +185,8 @@ getBookBase(Book const& book)
UInt256
getQualityNext(UInt256 const& uBase)
{
static constexpr UInt256 kNextQuality{
"0000000000000000000000000000000000000000000000010000000000000000"};
static constexpr UInt256 kNextQuality(
"0000000000000000000000000000000000000000000000010000000000000000");
return uBase + kNextQuality;
}
@@ -430,9 +429,9 @@ payChannel(AccountID const& src, AccountID const& dst, SeqProxy const& seq) noex
Keylet
nftokenPageMin(AccountID const& owner)
{
std::array<std::uint8_t, UInt256::size()> buf{};
std::ranges::copy(owner, buf.begin());
return {ltNFTOKEN_PAGE, UInt256{buf}};
std::array<std::uint8_t, 32> buf{};
std::memcpy(buf.data(), owner.data(), owner.size());
return {ltNFTOKEN_PAGE, UInt256::fromRaw(buf)};
}
Keylet
+2 -3
View File
@@ -14,7 +14,6 @@
#include <algorithm>
#include <cstdint>
#include <format>
#include <functional>
#include <limits>
#include <optional>
@@ -106,8 +105,8 @@ Permission::Permission()
if (type <= UINT16_MAX)
{
// LCOV_EXCL_START
Throw<std::logic_error>(std::format(
"Granular permission value must exceed the maximum uint16_t value: {}", name));
Throw<std::logic_error>(
"Granular permission value must exceed the maximum uint16_t value: " + name);
// LCOV_EXCL_STOP
}
}
+3 -1
View File
@@ -295,9 +295,11 @@ verify(PublicKey const& publicKey, Slice const& m, Slice const& sig) noexcept
NodeID
calcNodeID(PublicKey const& pk)
{
static_assert(NodeID::kBytes == sizeof(RipeshaHasher::result_type));
RipeshaHasher h;
h(pk.data(), pk.size());
return NodeID{static_cast<RipeshaHasher::result_type>(h)};
return NodeID::fromRaw(static_cast<RipeshaHasher::result_type>(h));
}
} // namespace xrpl
+3 -5
View File
@@ -43,9 +43,8 @@ setCurrentTransactionRules(std::optional<Rules> r)
auto const range = [&r]() {
// If any new conditions with new amendments are added to "enableLargeNumbers", those
// amendments must also be added to useRulesGuards.
bool const enableLargeNumbers = !r ||
(r->enabled(featureSingleAssetVault) || r->enabled(featureLendingProtocol) ||
r->enabled(featureMPTokensV2));
bool const enableLargeNumbers =
!r || (r->enabled(featureSingleAssetVault) || r->enabled(featureLendingProtocol));
// If enableLargeNumbers is true, then useRulesGuards must also return true.
// However, the reverse is not true. Other amendments can cause the rules guard to be used,
// even though large numbers are _not_ used.
@@ -85,8 +84,7 @@ useRulesGuards(Rules const& rules)
// with createGuards, and any other callers, and the first set of guards can be created directly
// at the call site, without using optional.
return rules.enabled(featureSingleAssetVault) || rules.enabled(featureLendingProtocol) ||
rules.enabled(fixCleanup3_2_0) || rules.enabled(fixCleanup3_3_0) ||
rules.enabled(featureMPTokensV2);
rules.enabled(fixCleanup3_2_0) || rules.enabled(fixCleanup3_3_0);
}
void
+7 -6
View File
@@ -37,7 +37,6 @@
#include <cstddef>
#include <cstdint>
#include <exception>
#include <format>
#include <iterator>
#include <limits>
#include <memory>
@@ -148,12 +147,12 @@ STAmount::STAmount(SerialIter& sit, SField const& name) : STBase(name)
}
Issue issue;
issue.currency = Currency{sit.get160()};
issue.currency = sit.get160();
if (isXRP(issue.currency))
Throw<std::runtime_error>("invalid native currency");
issue.account = AccountID{sit.get160()};
issue.account = sit.get160();
if (isXRP(issue.account))
Throw<std::runtime_error>("invalid native account");
@@ -1194,7 +1193,8 @@ muldiv(std::uint64_t multiplier, std::uint64_t multiplicand, std::uint64_t divis
if (ret > std::numeric_limits<std::uint64_t>::max())
{
Throw<std::overflow_error>(
std::format("overflow: ({} * {}) / {}", multiplier, multiplicand, divisor));
"overflow: (" + std::to_string(multiplier) + " * " + std::to_string(multiplicand) +
") / " + std::to_string(divisor));
}
return static_cast<uint64_t>(ret);
@@ -1215,8 +1215,9 @@ muldivRound(
if (ret > std::numeric_limits<std::uint64_t>::max())
{
Throw<std::overflow_error>(std::format(
"overflow: (({} * {}) + {}) / {}", multiplier, multiplicand, rounding, divisor));
Throw<std::overflow_error>(
"overflow: ((" + std::to_string(multiplier) + " * " + std::to_string(multiplicand) +
") + " + std::to_string(rounding) + ") / " + std::to_string(divisor));
}
return static_cast<uint64_t>(ret);
+4 -4
View File
@@ -19,13 +19,13 @@ STCurrency::STCurrency(SField const& name) : STBase{name}
{
}
STCurrency::STCurrency(SField const& name, Currency const& currency)
: STBase{name}, currency_{currency}
STCurrency::STCurrency(SerialIter& sit, SField const& name) : STBase{name}
{
currency_ = sit.get160();
}
STCurrency::STCurrency(SerialIter& sit, SField const& name)
: STCurrency{name, Currency{sit.get160()}}
STCurrency::STCurrency(SField const& name, Currency const& currency)
: STBase{name}, currency_{currency}
{
}
+35 -25
View File
@@ -28,38 +28,48 @@ STIssue::STIssue(SField const& name) : STBase{name}
STIssue::STIssue(SerialIter& sit, SField const& name) : STBase{name}
{
// Either the currency of an Issue or the issuer of an MPT; the
// following 160 bits disambiguate.
auto const currencyOrAccount = sit.get160();
if (isXRP(Currency{currencyOrAccount}))
if (isXRP(Currency::fromRaw(currencyOrAccount)))
{
asset_ = xrpIssue();
return;
}
// MPT is serialized as:
// - 160 bits MPT issuer account
// - 160 bits black hole account
// - 32 bits sequence
AccountID const account{sit.get160()};
// MPT
if (noAccount() == account)
// Check if MPT
else
{
// MPT: issuer (160), noAccount() (160), sequence (32). The sequence
// is stored in little-endian and get32() reads big-endian, so
// reverse it before building the canonical ID.
asset_ = MPTIssue{std::byteswap(sit.get32()), AccountID{currencyOrAccount}};
return;
// MPT is serialized as:
// - 160 bits MPT issuer account
// - 160 bits black hole account
// - 32 bits sequence
AccountID const account = AccountID::fromRaw(sit.get160());
// MPT
if (noAccount() == account)
{
MPTID mptID;
std::uint32_t sequence = sit.get32();
// MPTID stores the sequence in canonical big-endian bytes. STIssue
// ledger bytes are the legacy LE-host encoding, so convert the
// native get32() value to LE bytes before copying into the MPTID.
sequence = boost::endian::native_to_little(sequence);
static_assert(MPTID::size() == sizeof(sequence) + sizeof(currencyOrAccount));
memcpy(mptID.data(), &sequence, sizeof(sequence));
memcpy(
mptID.data() + sizeof(sequence),
currencyOrAccount.data(),
sizeof(currencyOrAccount));
MPTIssue const issue{mptID};
asset_ = issue;
}
else
{
Issue issue;
issue.currency = currencyOrAccount;
issue.account = account;
if (!isConsistent(issue))
Throw<std::runtime_error>("invalid issue: currency and account native mismatch");
asset_ = issue;
}
}
Issue const issue{Currency{currencyOrAccount}, account};
if (!isConsistent(issue))
Throw<std::runtime_error>("invalid issue: currency and account native mismatch");
asset_ = issue;
}
SerializedTypeID
+3 -2
View File
@@ -35,8 +35,9 @@ STLedgerEntry::STLedgerEntry(Keylet const& k) : STObject(sfLedgerEntry), key_(k.
if (format == nullptr)
{
Throw<std::runtime_error>(std::format(
"Attempt to create a SLE of unknown type {}", safeCast<std::uint16_t>(k.type)));
Throw<std::runtime_error>(
"Attempt to create a SLE of unknown type " +
std::to_string(safeCast<std::uint16_t>(k.type)));
}
set(format->getSOTemplate());
+1 -2
View File
@@ -19,7 +19,6 @@
#include <cstddef>
#include <cstdint>
#include <format>
#include <limits>
#include <ostream>
#include <stdexcept>
@@ -171,7 +170,7 @@ partsFromString(std::string const& number)
boost::smatch match;
if (!boost::regex_match(number, match, kReNumber))
Throw<std::runtime_error>(std::format("'{}' is not a number", number));
Throw<std::runtime_error>("'" + number + "' is not a number");
// Match fields:
// 0 = whole input
+47 -75
View File
@@ -5,7 +5,6 @@
#include <xrpl/basics/UnorderedContainers.h>
#include <xrpl/basics/base_uint.h>
#include <xrpl/basics/contract.h>
#include <xrpl/basics/safe_cast.h>
#include <xrpl/beast/hash/uhash.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/json/json_value.h>
@@ -18,13 +17,9 @@
#include <xrpl/protocol/jss.h>
#include <algorithm>
#include <concepts>
#include <cstddef>
#include <cstdint>
#include <memory>
#include <optional>
#include <stdexcept>
#include <type_traits>
#include <utility>
#include <vector>
@@ -41,10 +36,10 @@ STPathElement::getHash(STPathElement const& element)
// important. We don't even really need to fully hash the whole
// base_uint here, as a few bytes would do for our use.
//
// The note above is only true because the result of this function is only
// used by the STPathElement equality operator as a fast-reject prefilter,
// ahead of the field comparisons that then decide the answer. Do not use
// it to key a container.
// The note above is only true because the result of this function reaches
// nothing but STPathElement::operator==, where it is a fast-reject
// prefilter ahead of the field comparisons that decide the answer. Do not
// use it to key a container.
for (auto const x : element.getAccountID())
hashAccount += (hashAccount * 257) ^ x;
@@ -111,38 +106,12 @@ STPathSet::operator=(STPathSet const& other)
STPathSet::STPathSet(SerialIter& sit, SField const& name) : STBase(name)
{
using enum STPathElement::Type;
auto parsePathElementType = [](std::underlying_type_t<STPathElement::Type> byte) noexcept
-> std::optional<STPathElement::Type> {
if (byte == std::to_underlying(TypeBoundary))
return TypeBoundary;
constexpr auto knownBits = static_cast<std::underlying_type_t<STPathElement::Type>>(
std::to_underlying(TypeAccount) | std::to_underlying(TypeCurrency) |
std::to_underlying(TypeIssuer) | std::to_underlying(TypeMpt));
if ((byte & ~knownBits) != 0)
return std::nullopt;
return static_cast<STPathElement::Type>(byte);
};
std::vector<STPathElement> path;
std::optional<STPathElement::Type> type;
do
for (;;)
{
type = parsePathElementType(sit.get8());
int const iType = sit.get8();
if (!type)
{
JLOG(debugLog().error()) << "Bad path element in pathset";
Throw<std::runtime_error>("bad path element");
}
if (type == TypeNone || type == TypeBoundary)
if (iType == STPathElement::TypeNone || iType == STPathElement::TypeBoundary)
{
if (path.empty())
{
@@ -153,11 +122,28 @@ STPathSet::STPathSet(SerialIter& sit, SField const& name) : STBase(name)
// Move rather than converting the vector to an STPath by copy.
value_.emplace_back(std::move(path));
path.clear();
if (iType == STPathElement::TypeNone)
return;
}
else if ((iType & ~STPathElement::TypeAll) != 0)
{
JLOG(debugLog().error()) << "Bad path element " << iType << " in pathset";
Throw<std::runtime_error>("bad path element");
}
else
{
bool const hasCurrency = ((*type & TypeCurrency) == TypeCurrency);
bool const hasMPT = ((*type & TypeMpt) == TypeMpt);
auto const hasAccount = (iType & STPathElement::TypeAccount) != 0u;
auto const hasCurrency = (iType & STPathElement::TypeCurrency) != 0u;
auto const hasIssuer = (iType & STPathElement::TypeIssuer) != 0u;
auto const hasMPT = (iType & STPathElement::TypeMpt) != 0u;
AccountID account;
PathAsset asset;
AccountID issuer;
if (hasAccount)
account = sit.get160();
if (hasCurrency && hasMPT)
{
@@ -165,26 +151,18 @@ STPathSet::STPathSet(SerialIter& sit, SField const& name) : STBase(name)
Throw<std::runtime_error>("bad path element: MPT and Currency");
}
AccountID account;
AccountID issuer;
if ((*type & TypeAccount) == TypeAccount)
account = AccountID{sit.get160()};
PathAsset asset;
if (hasCurrency)
asset = Currency{sit.get160()};
asset = Currency::fromRaw(sit.get160());
if (hasMPT)
asset = sit.get192();
if ((*type & TypeIssuer) == TypeIssuer)
issuer = AccountID{sit.get160()};
if (hasIssuer)
issuer = sit.get160();
path.emplace_back(account, asset, issuer, hasCurrency || hasMPT);
}
} while (type != TypeNone);
}
}
STBase*
@@ -238,22 +216,24 @@ STPath::getJson(JsonOptions) const
for (auto const& it : path_)
{
json::Value elem(json::ValueType::Object);
auto const iType = it.getNodeType();
elem[jss::type] = safeCast<std::uint32_t>(it.getNodeType());
elem[jss::type] = iType;
if (it.isType(STPathElement::TypeAccount))
if ((iType & STPathElement::TypeAccount) != 0u)
elem[jss::account] = to_string(it.getAccountID());
XRPL_ASSERT(
!(it.hasCurrency() && it.hasMPT()),
((iType & STPathElement::TypeCurrency) == 0u) ||
((iType & STPathElement::TypeMpt) == 0u),
"xrpl::STPath::getJson : not type Currency and MPT");
if (it.hasCurrency())
if ((iType & STPathElement::TypeCurrency) != 0u)
elem[jss::currency] = to_string(it.getCurrency());
if (it.hasMPT())
if ((iType & STPathElement::TypeMpt) != 0u)
elem[jss::mpt_issuance_id] = to_string(it.getMPTID());
if (it.hasIssuer())
if ((iType & STPathElement::TypeIssuer) != 0u)
elem[jss::issuer] = to_string(it.getIssuerID());
ret.append(elem);
@@ -285,42 +265,34 @@ STPathSet::add(Serializer& s) const
XRPL_ASSERT(getFName().fieldType == STI_PATHSET, "xrpl::STPathSet::add : valid field type");
bool first = true;
// Workaround, since `Serializer` does not accept strongly-typed enums
// yet. When support is added, the static_assert will trigger, and the
// lambda can be removed entirely.
auto const toByte = [](std::same_as<STPathElement::Type> auto t) noexcept {
static_assert(
!requires(Serializer& ser) { ser.add8(t); },
"Serializer::add8 now accepts STPathElement::Type directly; remove toByte.");
return std::to_underlying(t);
};
for (auto const& spPath : value_)
{
if (!first)
s.add8(toByte(STPathElement::TypeBoundary));
s.add8(STPathElement::TypeBoundary);
for (auto const& speElement : spPath)
{
s.add8(toByte(speElement.getNodeType()));
int const iType = speElement.getNodeType();
if (speElement.isType(STPathElement::TypeAccount))
s.add8(iType);
if ((iType & STPathElement::TypeAccount) != 0u)
s.addBitString(speElement.getAccountID());
if (speElement.hasMPT())
if ((iType & STPathElement::TypeMpt) != 0u)
s.addBitString(speElement.getMPTID());
if (speElement.hasCurrency())
if ((iType & STPathElement::TypeCurrency) != 0u)
s.addBitString(speElement.getCurrency());
if (speElement.hasIssuer())
if ((iType & STPathElement::TypeIssuer) != 0u)
s.addBitString(speElement.getIssuerID());
}
first = false;
}
s.add8(toByte(STPathElement::TypeNone));
s.add8(STPathElement::TypeNone);
}
} // namespace xrpl
Loaded 100 of 317 files, more files were not shown because too many files have changed in this diff. Show more