Compare commits

..

28 Commits

Author SHA1 Message Date
Bart
ae7a39d737 fix: Require a sync filter's answer to hash to the key asked for
SHAMapSyncFilter::getNode() states the postcondition its callers already
depended on: an implementation returns only data that hashes to the key
it was asked for. checkFilter() adopts that hash rather than recomputing
one, and canonicalize() publishes the node to the family's node cache.

ConsensusTransSetSF is the one filter that did not meet it. Its cache
arm computes the digest and drops the entry it read when that disagrees,
so a blob that can never be served is not re-hashed on every later
request. It falls through rather than returning, since the transaction
arm reads a different store and may hold the key legitimately. That arm
compares the transaction's own id, which STTx caches from the same
prefix and the same bytes, so it needs no digest of its own.
2026-10-10 07:53:42 +09:00
Bart
adda315ec6 test: Cover visitDifferences node skipping
visitDifferences() is reached in production only through a file-static
populateFetchPack() in LedgerMaster.cpp, so the private hasLeafNode()
and hasInnerNode() it consults were untested. Four cases pin both
answers: only the nodes on a missing item's path, nothing against an
identical map, every node against a null map, and a callback stop.

The first case uses 64 shared items, so the shared leaves build inner
nodes of their own and whole matching subtrees exist to skip. A
finalize() helper seals each map first, since visitDifferences() returns
early while the root hash is zero, and getHash() is what seals it, by
unsharing such a map. Each call is wrapped in ASSERT_NO_FATAL_FAILURE,
which carries a gtest ASSERT_ failure out of a helper to its caller.
2026-10-10 07:53:42 +09:00
Bart
803ef61048 refactor: Derive a node's position from its path, not a stored ID
NodePathStack stored a SHAMapNodeID beside every node, so the path
carried two answers to where a node sits that could disagree. The path
answers it alone: a SHAMap has no path compression, so entry i sits at
depth i. No consumer wanted the ID, only a depth and the nibbles of a
key it already held, so selectBranch() gains a depth overload,
unshareNode() takes a depth, and SHAMap::isLeafDepth() replaces a
file-local copy so one predicate serves every site that judges a depth.

What a path cannot derive is which branches the caller descended, so
pathKey_ records those and a leaf is judged against every branch above
it through samePositionAtDepth(). SharedIntrusive's move constructors
are noexcept, since the path is a vector of them and move_if_noexcept
would otherwise copy each element.
2026-10-10 07:53:41 +09:00
Bart
1adaf0d368 refactor: Move entries off NodePathStack instead of copying them
NodePathStack::releaseNode() moves the top node out and pops it, where
five sites copied it out and then popped, each copy costing an atomic
increment and the original's release. dirtyUp() and delItem() walk up to
kLeafDepth levels per insert or delete on the ledger write path. The two
sites that read without popping bind a reference into the container,
sound only because it is a std::stack over a std::deque.

staticPointerCast() gains an rvalue overload, spelled with
SharedIntrusive<TT>&& so an lvalue still picks the const-ref one. Three
dynamic casts become static with a live type test, and a fourth static
cast that had none gains one, reporting UNREACHABLE in dirtyUp() and
delItem()'s loop and answering false in updateGiveItem() and delItem()'s
leaf cast, since the public API permits a call whose tag is absent.
2026-10-10 07:53:41 +09:00
Bart
75c85ae5d9 fix: Judge a node's position on both getMissingNodes paths
Both getMissingNodes paths judge a leaf's position: the arm that handles
a node descendAsync has already resolved, and the deferred-read hook for
one an asynchronous read resolves later. The deferred-read hook also
refuses an inner node at kLeafDepth, which only the synchronous arm
judged before. gmnProcessDeferredReads() is no longer static so it can
record the verdict, and it keeps draining every read this pass posted
after reaching it, since each holds a pointer to the walk's state. Both
of its tests skip the node rather than return.

A second belongsAt() overload answers about a child from its parent's
position and the branch taken, so neither site builds the child's ID,
and the type is tested first, which keeps the position test off every
inner child. It is reached only for a child the full-below lookup did
not answer for: that lookup decides which children the walk visits, and
this test decides whether a visited leaf belongs where it was put.

With the node flag no longer trusted below the root, this is also what
closes the shared-subtree case: a root naming a completed subtree one
branch over, or using it as a root, now descends it and condemns the
map at the misplaced leaf. The full-below case asserts that outcome.
2026-10-10 07:53:40 +09:00
Bart
bc2bca949f fix: Judge a node's position at its hook and in both full-below memos
descend()'s filter path and addKnownNode()'s leaf hook judge a node's
position before hooking it. setInvalid() and the trySetState() it routes
through are const, so a read-only descend() can record that verdict.

A full-below entry was keyed on the subtree hash alone, so a hit
answered for a hash without answering for a position. FullBelowKey
carries the masked node ID, from the new childNodeID(), and its depth
beside the hash, so a hit answers for one position. The depth is kept
because SHAMapNodeID masks its id to its own depth, leaving the root and
every all-zero prefix sharing one id, and the position is two plain
fields rather than a SHAMapNodeID, which is a CountedObject a cache
entry has no business entering. The key grows from 32 to 68 bytes, so
about 18 MiB more at the kFullBelowTargetSize of 524288 entries.

SHAMapInnerNode::fullBelowGen_ is the second memo with the same flaw.
The TreeNodeCache shares one node object by hash across maps and
positions, and the walk read the flag in its loop condition ahead of the
cache lookup, so a subtree completed at one position was skipped at
another. The flag is now written and read for a walk's root alone, where
the position is fixed, and the position-keyed cache is the only memo
below it. A backed map already consults the cache first, so a warm cache
costs nothing more; an unbacked map re-walks its resident nodes on each
pass. The new case completes a subtree in one map, then checks that a
root naming it one branch over descends it and that the subtree used as
a root is walked rather than trusted.
2026-10-10 07:53:22 +09:00
Bart
7b85c325fa fix: Refuse a node that cannot sit where a path puts it
NodePathStack::pushChild() refuses a node with no room below it at the
depth offered, and a leaf whose key does not lie under the branch
reached. belongsAt() and pastLeafDepth() are the predicates, and
walkTowardsKey()'s no-path mode shares pastLeafDepth(), so both its
modes refuse at the same depth; the no-path mode reports the key
absent, since its callers ask about a key rather than a position. Each reports SOMETIMES, not UNREACHABLE,
since the two-argument descend() judges neither position nor type.
boundHelper() throws where it answered end(), since an empty map still
leaves its root on the path, so an empty path means a node was refused
while end() claims no key lies on the requested side.

A refusal also reaches the empty-path throws in delItem(), addGiveItem()
and updateGiveItem(). OpenLedger::apply() gains in its retry pass the
guard the first pass already had, counting a transaction it cannot apply
as a failure.
2026-10-10 07:53:21 +09:00
Bart
9362b40566 fix: Bound the duplicate credit a TX-set acquire grants
A duplicate-only reply postpones at most kMaxDuplicateCredits
consecutive timer intervals with no useful node in between, so the
timeout count always resumes. A batch that advances the set earns the
budget back, and stillNeed() clears it on revival. The credit is spent
only while nothing else has answered the interval, so a wide fan-out
costs one credit rather than one per loser.

Two bounds join it: the sender must be a peer already asked, and the
acquisition must still be running. Both are read in takeNodes() before
takeNodesLocked(), which settles the set on some paths and enrolls the
sender in requestedPeers_ on others. takeNodes() also absorbs the
settled-set branch, so both bounds and the late-reply allowance are
judged ahead of takeNodesLocked().
2026-10-10 07:53:21 +09:00
Bart
8671174b90 fix: Keep an acquisition registered when it supplies its own set
giveSet()'s third parameter is now fromAcquire, and the entry's acquire
pointer is reset only when something other than the acquisition supplied
the set. Dropping it cancels an acquisition in flight, which a set
arriving some other way should do and an acquisition completing on its
own should not, so the entry stays registered until newRound() sweeps it
and a late reply still reaches the per-peer allowance. Keying the entry
by hash alone holds because addRootNode() refuses a root that does not
hash to the hash asked for.

wantsReplyFrom() decides ahead of the parse, so a reply turned away is
never built into nodes. It shares chargeLateReply() with
takeNodesLocked(), and only one of the two sees a reply, so one charge.
2026-10-10 07:53:20 +09:00
Bart
e8fbda021c fix: Fail a receiveNode() packet whose map has been abandoned
receiveNode() asks map.isValid() right after the locked receive and
fails the acquisition without charging the sender, since the verdict
belongs to the walk that reached it rather than to this packet. The test
sits ahead of isSynching(), which reads an abandoned map the same way as
one this packet just finished, because addKnownNode() answers every node
offered to such a map as a duplicate.

A second case pins the other outcome at that site: a packet leaving
nothing to fetch settles the ledger and reports it complete from
receiveNode() itself. DeepChain::allNodes() names the node list, root
included, that two suites assembled by hand.
2026-10-10 07:53:20 +09:00
Bart
03c6efe411 fix: Fail an acquisition whose map cannot exist
A node that leaves a map invalid proves the hash being chased belongs to
no valid tree, so TransactionAcquire::takeNodesLocked() and
InboundLedger::receiveNode() fail the acquisition there rather than
leaving the failure to the next trigger(). The whole batch is discarded,
since the nodes hooked in ahead of that one belong to the same tree.
Such a node costs kFeeMalformedData rather than kFeeInvalidData.

stillNeed() reports whether the set is still worth keeping and refuses
to revive one whose map went invalid. InboundTransactions::getSet()
refreshes an entry's retention window only while that answer is yes, so
such an entry is swept rather than kept alive by the asking.
recordPacket() and haveEverything() name the packet tally and the
nothing-left-to-fetch test that several sites repeated.
2026-10-10 07:53:19 +09:00
Bart
749cc6a959 fix: Charge declined TX-set data where it was declined
takeNodesLocked() issues the charge for declined data, under the lock
that reached the verdict: an empty node list, a root that does not hash
to the set asked for, a node that cannot be hooked, and node data that
will not deserialize each cost kFeeInvalidData at the site rejecting it.
A batch whose parse throws is tallied as invalid but not charged, since
what throws there is this code rather than the data.

A reply arriving after the acquisition has failed is free once per peer:
each peer holds one unspent pass, renewed by each request the acquisition
sends it, so overlapping requests to one peer share one pass, and a reply
past that pass costs kFeeUselessData. requestedPeers_ records every peer
a request went to, since trigger() can also send a targeted request to an
unsolicited sender, and it records one where it builds the request, so
selection alone does not enroll a peer. lateReplyGranted_ keys the
allowance by peer identity rather than counting it, since a count cannot
say whose pass a reply spends. recordAsked() renews a peer's pass with
every request that reaches it, targeted or broadcast; a revival on its
own renews nothing. A reply to a completed set still costs
kFeeUselessData, since giveSet() drops the acquisition.

That settled reply reports a duplicate, so takeNodes() reads whether the
set was settled when the call arrived and grants the timer credit only
to a reply the running acquisition earned.
2026-10-10 07:53:19 +09:00
Bart
50afd928bd fix: Count partial batch progress in a TX-set reply
TransactionAcquire::takeNodes() accumulates one SHAMapAddNode across the
whole batch, so a packet that ends on a rejected node still counts the
nodes hooked in ahead of it, as InboundLedger::receiveNode() does. The
body moves to takeNodesLocked() and takeNodes() records progress on the
single exit, since several inner returns stop the batch early.

Progress is recorded for a batch that hooked a node in or answered with
one already held. A batch of nodes already held is what an honest second
responder to trigger()'s fan-out sends, so gotData() reads isGood()
rather than isUseful() when it decides whether to charge for useless
data.
2026-10-10 07:53:18 +09:00
Bart
b28936342a fix: Restart the timer when reviving a timed-out TX-set acquire
TransactionAcquire::stillNeed() restarts the retry timer whenever it
revives a failed acquisition, so a revived object resumes asking rather
than waiting for a peer to send data unprompted. expires_after() cancels
whatever wait was outstanding, so the timer holds at most one wait at a
time. A job already handed to the JobQueue is not canceled and re-arms
this same timer, which folds back into the one chain. A running
acquisition returns early and keeps the wait it has, so a consensus
round that asks for the set again does not restart it. The timeout count
is clamped either way.
2026-10-10 07:53:18 +09:00
Bart
e820f5a8bf fix: Start a ledger built from a header mutable
Ledger(LedgerHeader const&, Rules, Family&) starts at immutable_ false.
Its maps are constructed Synching and filled in afterwards by an
acquisition or a replay, so setImmutable() is what settles the ledger,
and only once both maps are sound. mapHashesFromHeader_ records that
this constructor's transaction and account hashes are input rather than
derived, so setImmutable() leaves them alone: deriving them from the
maps would relabel a map that fell short of its target. The flag is
const, unlike immutable_.

A ledger this constructor built is therefore unsettled until
setImmutable() says otherwise, and LedgerMaster::switchLCL() and
LedgerHistory::insert() each require a settled one.
InboundLedger::getLedger() already reports nothing once the acquisition
has failed, so done()'s posted job reads the accessor: failure and a
null ledger_ are refused by one check, and checkAccept() requires a
non-null one. isFailed()'s comment said the opposite of what it returns.
2026-10-10 07:53:17 +09:00
Bart
07a6f129cf fix: Settle an acquired ledger before reporting it complete
done() owns the publication of complete_ and sets it only once the
ledger is settled, so a caller reading isComplete() without mtx_ may use
the ledger directly. LedgerHistory::insert(), LedgerHolder::set() and
LedgerMaster::switchLCL() each require an immutable ledger, and that
ordering is what their requirement rests on. trigger() and receiveNode()
set the have-flags and leave the verdict to done(), while tryDB()
settles its own result and sets complete_ itself, so that path arrives
with only the reporting left.
2026-10-10 07:53:17 +09:00
Bart
db65266943 fix: Publish an acquisition's outcome flags across threads
complete_ and failed_ are std::atomic<bool> at the default seq_cst
order, so a reader that sees either flag set also sees the work the
writer did before setting it. isComplete() and isFailed() return them
without taking mtx_, and InboundLedgers::acquire() calls both outside
every lock it holds. A static_assert pins both to a lock-free
representation, so neither accessor can block. No access site changes: a
plain assignment and a plain test compile unchanged, and nothing takes a
reference or an address of either field.

Scoped to those two, which are the only fields here reached through an
accessor that takes no lock.
2026-10-10 07:53:16 +09:00
Bart
a86a2651ec fix: Tell a satisfied InboundLedger map from an abandoned one
hasInvalidMap() reports whether either map of the ledger being acquired
has been found invalid, and the three places that read an empty walk
result as nothing left to fetch ask it first: tryDB(), whose two walks
set haveState_ and haveTransactions_ independently, trigger()'s
aggressive-retry branch, whose getNeededHashes() walk can reach the
verdict itself, and trigger()'s state-map walk, the one walk that runs
with mtx_ released.

The state-map walk asks outside the guard that re-reads the flags after
re-locking, since the verdict is about the map rather than the round,
and it withdraws the completeness claim beside the failure.
2026-10-10 07:53:16 +09:00
Bart
08b8f62aa2 test: Pin the getMissingNodes refusal and cover a leaf at kLeafDepth
get_missing_nodes_refuses_invalid_map now counts the filter lookups and
expects none. The result alone could not tell the early refusal from a
walk that reaches the same verdict, since the walk discards what it
collected once the map is invalid, so the early return could be removed
without failing the test.

get_missing_nodes_accepts_leaf_at_leaf_depth is the control for the
depth guard: a real leaf at kLeafDepth is served through the filter and
the walk completes the map. Without it, a guard that also rejected
leaves passed every test.

The two ThreadSanitizer comments no longer name PR 8245, which goes
stale the day it merges, and no longer claim a nightly job that does
not exist yet. Only a ThreadSanitizer build observes the ordering.
2026-10-10 07:52:40 +09:00
Bart
057b5788fb fix: Refuse to walk an invalid SHAMap in getMissingNodes
getMissingNodes() returns an empty list and walks nothing once the map
is Invalid, and a walk that resolves an inner node at a position only a
leaf may occupy records that verdict itself, since no node on that
route passes through addKnownNode() and getChildNodeID() has no answer
at kLeafDepth. The map is judged again after the deferred reads are
drained and once more before the result is returned, since another
thread can write the verdict in between. Every posted read is drained
either way, because each holds a pointer to the walk's state.

A caller must ask isValid() to tell an empty result from a satisfied
map, which the docstring now states. The depth test precedes the
full-below cache lookup, as in addKnownNode().
2026-10-04 06:35:39 -04:00
Bart
88ad25eb0e fix: Refuse to make an invalid SHAMap or Ledger immutable
SHAMap::setImmutable() returns [[nodiscard]] bool and refuses a map
already proven impossible. trySetState() and clearSynching() are the
only writers of state_ past construction, and Invalid outranks and
survives every other state. clearSynching() moves only a Synching map,
so a walk that ends after the ledger settled leaves the map Immutable.

Ledger::setImmutable() and setAccepted() report the same. They check
mapsValid() first and settle each map on its own, so neither is left
mid-sync because the other refused. The hashes are read before the maps
are settled, since getHash() can unshare a dirty tree, and written to
the header only once both have made it. A locally built or loaded ledger
treats false as a broken invariant. One assembled from peer data
recovers, withdrawing complete_ beside the failure. done() settles the
ledger ahead of its reason switch, so a HISTORY ledger reaches
onLedgerFetched() only once settled.

trigger() now calls done() under mtx_, as every other caller already
did, so the flags done() writes are published under one lock.

InboundLedger::getLedger() reports nothing once the acquisition has
failed. A failure found once the header is held, as trigger() reports
for an invalid map, leaves ledger_ populated, and the loadOldLedger
fallbacks read getLedger() without testing isFailed(), so the guard sits
at the one accessor rather than at each caller. ledger_ itself is kept,
since getJson() still reports on the partial maps.
2026-10-04 06:35:38 -04:00
Bart
ccd8838d09 fix: Report a node the sync path refuses as invalid data
addKnownNode() returns invalid() for the two shapes it declines to hook
in: an inner node at kLeafDepth, a depth only a leaf may occupy, and a
node whose claimed ID is not where the hash-verified descent stopped.
The depth case also marks the map Invalid, which is terminal, since
every node above it hash-verified and so the requested root hash itself
commits to a shape no valid tree has. An ID mismatch leaves the map
sound. The depth test precedes the full-below cache lookup, which is
keyed by hash and so answers for no particular depth.

addRootNode() guarantees in every build mode that a root already held is
a duplicate only if it hashes to the hash the map was asked for, and
invalid data otherwise.
2026-10-04 06:35:38 -04:00
Bart
dcd44fa0ae fix: Make the SHAMap sync-path state atomic
SHAMap::state_, ::full_, ::ledgerSeq_ and SHAMapInnerNode::fullBelowGen_
are std::atomic and pinned lock-free, since a nodestore fetch and a
getMissingNodes() walk reach them while the thread driving an
acquisition does, and neither may block. finishFetch() withdraws full_
with an acq_rel exchange, so exactly one reader reports the gap.
ledgerSeq_ and fullBelowGen_ are relaxed both ways: a lookup hint for a
store keyed by hash, and a generation compared only for equality whose
children are published through the node's own lock. Ledger::setFull()
stores each map's sequence before its full flag, the one order that
makes the sequence visible to the thread whose exchange wins.

The shamap tests' TestNodeFamily replaces helpers/TestFamily.h, takes a
readThreads count for its nodestore, and declares its clock ahead of the
members that use it. InnerNode.h adds makeFullInnerNode() and
makeCompressedInnerNode(), which build an inner node from a list of
children.
2026-10-04 06:35:37 -04:00
Bart
a0fd44cc7b fix: Judge a header's account hash on both acquisition routes
No ledger has an empty state map, so a header with a zero account hash
cannot name a ledger. tryDB() judges that inside makeLedger(), next to
the hash and sequence check that already refuses a header that cannot be
a ledger, and takeHeader() judges it through the same helper,
failOnZeroAccountHash(). The helper sets failed_ and drops ledger_, so
neither route stores the header, sets seq_, or sets haveHeader_ for such
a header. A zero transaction hash is still an empty transaction set.

The header is the one asked for, so the peer is not charged.
processData() calls done() when takeHeader() has failed the acquisition,
since trigger() and onTimer() return early once isDone() and nothing else
would signal the waiters or record the hash in recentFailures_.
2026-10-04 06:35:37 -04:00
Bart
516ed4d433 fix: Signal every InboundLedger failure found in local data
init() and trigger() call done() when tryDB() sets failed_, so every
route that ends an acquisition runs the one function that signals
whatever waits on it and records the hash in recentFailures_.
checkLocal() already did. A hash in recentFailures_ is what keeps a
later round from asking for a ledger already judged unobtainable.
2026-10-04 06:35:36 -04:00
Bart
a790fb3527 refactor: Add a reusable peer harness for acquisition tests
DeepChain builds the node chains both acquisition suites need: chains
that run to SHAMap::kLeafDepth, which no valid tree holds, and toLeaf()
chains that complete an acquisition. AcquireTestHelpers.h adds
ChargeRecordingPeer, RequestCountingPeerSet, packetFor(), waitFor() and
tallyIs(), so each suite drives an acquisition through the real
gotData() dispatch rather than reproducing it.

AcquireTestHelpers.h is the first src/test file to include one from
src/tests, so ordering.txt records a test.app > tests.libxrpl edge. That
direction is deliberate: src/test goes away as its suites migrate, so an
edge into the surviving tree outlives the migration. Nothing under
src/tests includes src/test.
2026-10-04 06:35:36 -04:00
Bart
0514203d03 refactor: Open the acquisition seams its tests need
TransactionAcquire and InboundLedger are no longer final and take a
retry interval defaulted to the constant each used before, so a test
subclass can drive a whole timeout chain in a fraction of a second.
TimeoutCounter's constructor already asserts an interval above 10ms and
below 30s, and every production caller takes the default.
TransactionAcquire::map_ and InboundLedger's trigger(), done() and
TriggerReason are protected, which is the only way a subclass reaches
them.

ConsensusTransSetSF::kMinTxNodeBytesToParse names the resubmission floor
as the 4-byte hash prefix plus kMinShaMapItemBytes plus one, which is
the same 17 the old size() > 16 test applied.
2026-10-04 06:35:35 -04:00
Bart
db21f68086 refactor: Expose a SHAMapAddNode verdict as counts
getBad() and getDuplicate() join getGood(), so a caller reads a verdict
as three counts rather than parsing the log string get() builds. The bad
count is per node rejected, not per node offered: a batch that stops at
its first rejected node counts one, and a batch that carries on counts
each. get() is a log format, so its wording is pinned in one place, the
new gtest, and nothing else depends on it.
2026-10-04 06:35:34 -04:00
451 changed files with 15051 additions and 10517 deletions

View File

@@ -5,10 +5,7 @@ Checks: "-*,
-bugprone-exception-escape,
-bugprone-implicit-widening-of-multiplication-result,
-bugprone-narrowing-conversions,
-bugprone-signed-bitwise,
-bugprone-std-exception-baseclass,
-bugprone-throwing-static-initialization,
-bugprone-unhandled-code-paths,
cppcoreguidelines-*,
-cppcoreguidelines-avoid-c-arrays,
@@ -17,7 +14,6 @@ Checks: "-*,
-cppcoreguidelines-avoid-magic-numbers,
-cppcoreguidelines-avoid-non-const-global-variables,
-cppcoreguidelines-c-copy-assignment-signature,
-cppcoreguidelines-explicit-constructor,
-cppcoreguidelines-interfaces-global-init,
-cppcoreguidelines-macro-usage,
-cppcoreguidelines-missing-std-forward,
@@ -36,7 +32,6 @@ Checks: "-*,
llvm-namespace-comment,
misc-*,
-misc-explicit-constructor,
-misc-multiple-inheritance,
-misc-no-recursion,
-misc-non-private-member-variables-in-classes,
@@ -50,8 +45,6 @@ Checks: "-*,
-modernize-avoid-c-style-cast,
-modernize-return-braced-init-list,
-modernize-use-integer-sign-comparison,
-modernize-use-string-view,
-modernize-use-structured-binding,
-modernize-use-trailing-return-type,
performance-*,
@@ -60,7 +53,6 @@ Checks: "-*,
-performance-noexcept-move-constructor,
-performance-unnecessary-copy-initialization,
-performance-unnecessary-value-param,
-performance-use-std-move,
readability-*,
-readability-avoid-const-params-in-decls,
@@ -73,11 +65,7 @@ Checks: "-*,
-readability-named-parameter,
-readability-qualified-auto,
-readability-redundant-access-specifiers,
-readability-redundant-nested-if,
-readability-redundant-qualified-alias,
-readability-static-accessed-through-instance,
-readability-trailing-comma,
-readability-trivial-switch,
-readability-uppercase-literal-suffix
"
# ---
@@ -93,10 +81,6 @@ CheckOptions:
bugprone-unsafe-functions.ReportMoreUnsafeFunctions: true
bugprone-unused-return-value.CheckedReturnTypes: ::std::error_code;::std::error_condition;::std::errc
# New in clang-tidy 23; disabled until the code is updated
misc-const-correctness.AnalyzeAutoVariables: false
misc-const-correctness.AnalyzeLambdas: false
misc-const-correctness.AnalyzeParameters: false
misc-include-cleaner.IgnoreHeaders: ".*/(detail|impl)/.*;.*fwd\\.h(pp)?;time.h;stdlib.h;sqlite3.h;netinet/in\\.h;sys/resource\\.h;sys/sysinfo\\.h;linux/sysinfo\\.h;__chrono/.*;bits/.*;_abort\\.h;boost/.*;openssl/obj_mac\\.h"
readability-braces-around-statements.ShortStatementLines: 2

View File

@@ -103,6 +103,7 @@ words:
- dearmor
- decryptor
- dedented
- dedup
- deleteme
- demultiplexer
- deserializaton
@@ -344,6 +345,7 @@ words:
- unambiguity
- unauthorizes
- unauthorizing
- undeserializable
- unergonomic
- unfetched
- unfindable

View File

@@ -108,75 +108,3 @@ endfunction()
function(patch_nix_binary target)
endfunction()
function(rpcspec_generate_instantiations)
set(options)
set(oneValueArgs OUT_VAR VALUE_TYPE VIEW_HEADER INCLUDE_DIR)
set(multiValueArgs HANDLERS)
cmake_parse_arguments(
THIS_FUNCTION_PREFIX
"${options}"
"${oneValueArgs}"
"${multiValueArgs}"
${ARGN}
)
endfunction()
function(corrosion_import_crate)
set(options
ALL_FEATURES
NO_DEFAULT_FEATURES
NO_STD
NO_LINKER_OVERRIDE
NO_USES_TERMINAL
LOCKED
FROZEN
)
set(oneValueArgs MANIFEST_PATH PROFILE IMPORTED_CRATES)
set(multiValueArgs
CRATE_TYPES
CRATES
FEATURES
FLAGS
OVERRIDE_CRATE_TYPE
)
cmake_parse_arguments(
THIS_FUNCTION_PREFIX
"${options}"
"${oneValueArgs}"
"${multiValueArgs}"
${ARGN}
)
endfunction()
function(corrosion_set_env_vars target_name env_var)
endfunction()
function(corrosion_add_cxxbridge cxx_target)
set(options)
set(oneValueArgs CRATE)
set(multiValueArgs FILES)
cmake_parse_arguments(
THIS_FUNCTION_PREFIX
"${options}"
"${oneValueArgs}"
"${multiValueArgs}"
${ARGN}
)
endfunction()
function(_unlink_libgcc_s crate)
endfunction()
function(add_xrpl_crate name)
set(options)
set(oneValueArgs CRATE)
set(multiValueArgs FILES)
cmake_parse_arguments(
THIS_FUNCTION_PREFIX
"${options}"
"${oneValueArgs}"
"${multiValueArgs}"
${ARGN}
)
endfunction()

View File

@@ -15,9 +15,9 @@ inputs:
required: false
default: "false"
log_verbosity:
description: 'The logging verbosity ("quiet", "verbose"), or empty to use the Conan defaults.'
description: "The logging verbosity."
required: false
default: ""
default: "verbose"
sanitizers:
description: "The sanitizers to enable."
required: false
@@ -35,16 +35,6 @@ runs:
LOG_VERBOSITY: ${{ inputs.log_verbosity }}
SANITIZERS: ${{ inputs.sanitizers }}
run: |
# By default, leave the verbosity unset, so CMake configure output is
# shown, but compile commands and Boost's b2 debug output (~85k lines
# when "verbose") are not.
VERBOSITY_ARGS=()
if [[ -n "${LOG_VERBOSITY}" ]]; then
VERBOSITY_ARGS=(
--conf:all tools.build:verbosity="${LOG_VERBOSITY}"
--conf:all tools.compilation:verbosity="${LOG_VERBOSITY}"
)
fi
conan install \
--profile:all ci \
--build="${BUILD_OPTION}" \
@@ -52,13 +42,6 @@ runs:
--options:host='&:xrpld=True' \
--settings:all build_type="${BUILD_TYPE}" \
--conf:all tools.build:jobs=${BUILD_NPROC} \
"${VERBOSITY_ARGS[@]}" \
--format=json \
. >"${RUNNER_TEMP}/conan-graph.json"
# Tools that run during the build may only load glibc from the Nix store,
# as their package ID survives a GCC runtime update.
- name: Check build-context packages for Nix store dependencies (Linux)
if: ${{ runner.os == 'Linux' }}
shell: bash
run: ./bin/nix/check-build-context-runtime.sh "${RUNNER_TEMP}/conan-graph.json"
--conf:all tools.build:verbosity="${LOG_VERBOSITY}" \
--conf:all tools.compilation:verbosity="${LOG_VERBOSITY}" \
.

View File

@@ -62,6 +62,7 @@ libxrpl.tx > xrpl.protocol
libxrpl.tx > xrpl.server
libxrpl.tx > xrpl.tx
test.app > test.jtx
test.app > tests.libxrpl
test.app > test.unit_test
test.app > xrpl.basics
test.app > xrpl.config

View File

@@ -1,5 +1,5 @@
{
"image_tag": "sha-3d526d4",
"image_tag": "sha-060957e",
"configs": {
"ubuntu": [
{
@@ -74,7 +74,7 @@
"extra_cmake_args": "-Dvalidator_keys=ON",
"package": {
"type": "deb",
"image": "ghcr.io/xrplf/xrpld/packaging-debian:sha-e6055dd"
"image": "ghcr.io/xrplf/xrpld/packaging-debian:sha-3a2d19f"
}
},
{
@@ -86,7 +86,7 @@
"extra_cmake_args": "-Dvalidator_keys=ON -Dassert=ON",
"package": {
"type": "deb",
"image": "ghcr.io/xrplf/xrpld/packaging-debian:sha-e6055dd",
"image": "ghcr.io/xrplf/xrpld/packaging-debian:sha-3a2d19f",
"variant": "assert"
}
}
@@ -101,7 +101,7 @@
"extra_cmake_args": "-Dvalidator_keys=ON",
"package": {
"type": "rpm",
"image": "ghcr.io/xrplf/xrpld/packaging-rhel:sha-e6055dd"
"image": "ghcr.io/xrplf/xrpld/packaging-rhel:sha-3a2d19f"
}
}
]

View File

@@ -12,7 +12,8 @@ on:
- "!nix/docker/README.md"
- "!nix/devshell.nix"
- "!nix/check-tools/**"
- "bin/nix/default-loader-path.sh"
- "bin/default-loader-path.sh"
- "bin/install-sanitizer-libs.sh"
pull_request:
paths:
- ".github/workflows/build-nix-images.yml"
@@ -24,8 +25,8 @@ on:
- "!nix/devshell.nix"
- "!nix/check-tools/**"
- "bin/check-tools.sh"
- "bin/nix/default-loader-path.sh"
- "bin/install/sanitizer-libs.sh"
- "bin/default-loader-path.sh"
- "bin/install-sanitizer-libs.sh"
workflow_dispatch:
concurrency:

View File

@@ -5,12 +5,13 @@ on:
branches:
- develop
paths:
- "bin/install/packaging-tools.sh"
- ".github/workflows/build-packaging-images.yml"
- "bin/install-packaging-tools.sh"
- "package/images/packaging/**"
pull_request:
paths:
- ".github/workflows/build-packaging-images.yml"
- "bin/install/packaging-tools.sh"
- "bin/install-packaging-tools.sh"
- "package/images/packaging/**"
workflow_dispatch:

View File

@@ -5,6 +5,7 @@ on:
branches:
- develop
paths:
- ".github/workflows/build-pre-commit-image.yml"
- "bin/pre-commit/Dockerfile"
- "rust-toolchain.toml"
pull_request:

View File

@@ -34,7 +34,7 @@ permissions:
jobs:
audit:
runs-on: ubuntu-latest
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-3d526d4
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-060957e
permissions:
contents: read
# Needed to open an issue on scheduled failures.

View File

@@ -93,8 +93,9 @@ jobs:
.github/workflows/reusable-upload-recipe.yml
.clang-tidy
.codecov.yml
bin/check-nix-store-refs.sh
bin/check-tools.sh
bin/nix/**
bin/default-loader-path.sh
cfg/**
cmake/**
conan/**

View File

@@ -31,8 +31,9 @@ on:
- ".github/workflows/reusable-upload-recipe.yml"
- ".clang-tidy"
- ".codecov.yml"
- "bin/check-nix-store-refs.sh"
- "bin/check-tools.sh"
- "bin/nix/**"
- "bin/default-loader-path.sh"
- "cfg/**"
- "cmake/**"
- "conan/**"

View File

@@ -17,4 +17,4 @@ jobs:
uses: XRPLF/actions/.github/workflows/pre-commit.yml@279ec358f4a1be4088be3e024b07916fa97c75b6
with:
runs_on: ubuntu-latest
container: '{ "image": "ghcr.io/xrplf/xrpld/pre-commit:sha-70b8fd3" }'
container: '{ "image": "ghcr.io/xrplf/xrpld/pre-commit:sha-473fe44" }'

View File

@@ -41,7 +41,7 @@ env:
jobs:
build:
runs-on: ubuntu-latest
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-3d526d4
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-060957e
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

View File

@@ -186,6 +186,9 @@ jobs:
with:
build_nproc: ${{ steps.nproc.outputs.nproc }}
build_type: ${{ inputs.build_type }}
# Set the verbosity to "quiet" for Windows to avoid an excessive
# amount of logs. For other OSes, the "verbose" logs are more useful.
log_verbosity: ${{ runner.os == 'Windows' && 'quiet' || 'verbose' }}
sanitizers: ${{ inputs.sanitizers }}
- name: Configure CMake
@@ -254,20 +257,20 @@ jobs:
# cache included, since what it holds is what gets uploaded and reused.
- name: Check the build output for Nix store references (Nix toolchain)
if: ${{ inputs.toolchain == 'nix' }}
run: ./bin/nix/check-nix-store-refs.sh "${BUILD_DIR}"
run: ./bin/check-nix-store-refs.sh "${BUILD_DIR}"
- name: Check the Conan cache for Nix store references (Nix toolchain)
if: ${{ inputs.toolchain == 'nix' }}
run: ./bin/nix/check-nix-store-refs.sh "${CONAN_HOME}"
run: ./bin/check-nix-store-refs.sh "${CONAN_HOME}"
# Only what PatchNixBinary.cmake retargets: the toolchain in the Linux
# images always references the store. Same condition it uses.
- name: Check for Nix store references (Linux)
if: ${{ runner.os == 'Linux' && env.SANITIZERS_ENABLED == 'false' }}
run: |
./bin/nix/check-nix-store-refs.sh "${BUILD_DIR}/xrpld"
./bin/nix/check-nix-store-refs.sh "${BUILD_DIR}/xrpl_tests"
./bin/nix/check-nix-store-refs.sh "${BUILD_DIR}/xrpld_tests"
./bin/check-nix-store-refs.sh "${BUILD_DIR}/xrpld"
./bin/check-nix-store-refs.sh "${BUILD_DIR}/xrpl_tests"
./bin/check-nix-store-refs.sh "${BUILD_DIR}/xrpld_tests"
- name: Show ccache statistics
if: ${{ inputs.ccache_enabled }}

View File

@@ -34,7 +34,7 @@ jobs:
needs: [determine-files]
if: ${{ needs.determine-files.outputs.cpp_changed_files != '' || needs.determine-files.outputs.need_full_run == 'true' }}
runs-on: ["self-hosted", "Linux", "X64", "heavy"]
container: "ghcr.io/xrplf/xrpld/nix-debian:sha-3d526d4"
container: "ghcr.io/xrplf/xrpld/nix-debian:sha-060957e"
permissions:
contents: read
issues: write
@@ -73,6 +73,7 @@ jobs:
with:
build_nproc: ${{ steps.nproc.outputs.nproc }}
build_type: ${{ env.BUILD_TYPE }}
log_verbosity: verbose
- name: Configure CMake
working-directory: ${{ env.BUILD_DIR }}

View File

@@ -28,7 +28,7 @@ permissions:
jobs:
clippy:
runs-on: ubuntu-latest
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-3d526d4
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-060957e
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -41,7 +41,7 @@ jobs:
coverage:
runs-on: ubuntu-latest
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-3d526d4
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-060957e
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -70,7 +70,7 @@ jobs:
doc:
runs-on: ubuntu-latest
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-3d526d4
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-060957e
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

View File

@@ -40,7 +40,7 @@ defaults:
jobs:
upload:
runs-on: ubuntu-latest
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-3d526d4
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-060957e
env:
REMOTE_NAME: ${{ inputs.remote_name }}
CONAN_LOGIN_USERNAME_XRPLF: ${{ secrets.remote_username }}

View File

@@ -108,11 +108,14 @@ jobs:
build_nproc: ${{ steps.nproc.outputs.nproc }}
build_type: ${{ matrix.build_type }}
force_build: ${{ github.event_name == 'schedule' || github.event.inputs.force_source_build == 'true' }}
# Set the verbosity to "quiet" for Windows to avoid an excessive
# amount of logs. For other OSes, the "verbose" logs are more useful.
log_verbosity: ${{ runner.os == 'Windows' && 'quiet' || 'verbose' }}
sanitizers: ${{ matrix.sanitizers }}
- name: Check the Conan cache for Nix store references (Nix toolchain)
if: ${{ matrix.toolchain == 'nix' }}
run: ./bin/nix/check-nix-store-refs.sh "${CONAN_HOME}"
run: ./bin/check-nix-store-refs.sh "${CONAN_HOME}"
- name: Log into Conan remote
if: ${{ github.repository == 'XRPLF/rippled' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') }}

View File

@@ -60,7 +60,7 @@ repos:
types_or: [c++, c]
- repo: https://github.com/pre-commit/mirrors-clang-format
rev: a9a8a861f30ed207ead7d5a3b7e8032283ba5da7 # frozen: v23.1.2
rev: e2b496dc2bd8340c2524cb9a2d2a943cde1bb6df # frozen: v23.1.1
hooks:
- id: clang-format
args: [--style=file]
@@ -82,10 +82,9 @@ repos:
files: ^crates/.*\.rs$
- repo: https://github.com/BlankSpruce/gersemi-pre-commit
rev: 28010ddd6016e1a0f7bd232acb6536ef996ae897 # frozen: 0.29.2
rev: f1c4833f8cf23c6d952673abc73525411a5719e8 # frozen: 0.29.1
hooks:
- id: gersemi
args: [-i, --warnings-as-errors]
- repo: https://github.com/rbubley/mirrors-prettier
rev: ef4a397f916211b4a39ccf9d3d9cbb6562157251 # frozen: v3.9.9
@@ -96,19 +95,19 @@ repos:
# Scoped to package/: the rest of the repo's Python has pre-existing findings,
# so widening these is its own change.
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: f12be1ebaa5351c1fc76472de98db2c3446c8253 # frozen: v0.16.10
rev: a56c0b927e6465d37cae3e97d35d4d18ab2b96cd # frozen: v0.16.9
hooks:
- id: ruff-check
args: [--fix]
files: ^package/.*\.py$
- repo: https://github.com/psf/black-pre-commit-mirror
rev: 96ae3e5802f3fe2d551e703e18f0a367d1a81ac2 # frozen: 26.10.0
rev: 4160603246a6b365d4a2af661c6d71b0a0f50478 # frozen: 26.5.1
hooks:
- id: black
- repo: https://github.com/pre-commit/mirrors-mypy
rev: 2834ec6639549dd6796205c8f011dedcd587288b # frozen: v2.4.0
rev: 7ff8d35ae36a7d2b968f2f90b4c723e292e594ee # frozen: v2.3.1
hooks:
- id: mypy
args: [--strict]

View File

@@ -28,17 +28,7 @@ Version 3.5.0 is not yet released.
### Additions in 3.5.0
- `subscribe`, `unsubscribe`: Added an optional `mpt_issuances` request field, an array of MPT issuance IDs (hex strings). Subscribers receive the same `transaction` message as the `transactions` stream for each validated transaction whose metadata affects a subscribed issuance. MPT issuance subscriptions count toward the per-connection subscription limit. An empty array, a non-array value, or an invalid ID returns `invalidParams`. ([#5671](https://github.com/XRPLF/rippled/pull/5671))
- `ledger_entry`: Add full support for checks, NFT offers, payment channels, and signer lists. ([#6319](https://github.com/XRPLF/rippled/pull/6319))
### Bugfixes in 3.5.0
- `channel_authorize`: The `channel_id` field now returns an `invalidParams` error if the value is not a string. [#7582](https://github.com/XRPLF/rippled/pull/7582)
- `channel_verify`: The `channel_id` and `signature` fields now return an `invalidParams` error if the value is not a string. [#7582](https://github.com/XRPLF/rippled/pull/7582)
### Bugfixes in 3.5.0
- `feature`: The admin-only `vetoed` field now returns `invalidParams` unless its value is a boolean. [#7583](https://github.com/XRPLF/rippled/pull/7583)
- `subscribe`, `unsubscribe`: Added an optional `mpt_issuances` request field, an array of MPT issuance IDs (hex strings). Subscribers receive a message with `type` `mptTransaction` for each validated transaction whose metadata affects a subscribed issuance; the message has the same fields as the `transactions` stream. MPT issuance subscriptions count toward the per-connection subscription limit. An empty array, a non-array value, or an invalid ID returns `invalidParams`. ([#5671](https://github.com/XRPLF/rippled/pull/5671))
## XRP Ledger server version 3.4.0
@@ -60,7 +50,6 @@ Version 3.4.0 is not yet released. These changes are available in the 3.4.0 beta
- `account_lines`: The `peer` field now returns an error if the value is not a string. [#7728](https://github.com/XRPLF/rippled/pull/7728)
- `ledger`: `delivered_amount` is now included in the metadata of successful `AccountDelete` transactions when transactions are expanded (`expand`, or admin-only `full`). Previously it was only added for `Payment` and `CheckCash`, which made `ledger` inconsistent with `tx` and `account_tx`. [#5706](https://github.com/XRPLF/rippled/pull/5706)
- `noripple_check`: The `transactions` field is no longer included in error responses; it is still returned (possibly as an empty array) whenever `transactions` is `true` and the request succeeds. A malformed `account` is now rejected before the ledger is looked up, so that error response no longer carries the `ledger_hash`, `ledger_index`, and `validated` fields ([#6303](https://github.com/XRPLF/rippled/pull/6303)).
- `transaction_entry`: An object or an array in `tx_hash` now returns `malformedRequest`, like any other value that is not a hex hash, instead of an `internal` error.
## XRP Ledger server version 3.3.0

View File

@@ -10,7 +10,7 @@
# alone; the scripts in a Conan cache are all git hook samples and autotools
# scratch, 36 false positives to 0 real.
#
# Usage: bin/nix/check-nix-store-refs.sh <path>
# Usage: bin/check-nix-store-refs.sh <path>
set -euo pipefail

View File

@@ -20,11 +20,9 @@
# development setups, but not in the macOS CI environment. They are checked
# everywhere except when running in CI on macOS.
#
# Tools that Nix also exposes under a version-suffixed name
# (`clang-tidy-<v>`, `g++-<v>`, ...) are probed under both names:
# a suffixed name can break while the plain one still works
# (see mkVersionedToolLinks in nix/packages.nix).
# The suffix is the major version of the plain `clang` / `gcc` on PATH.
# Tools that Nix also exposes under a version-suffixed name (`clang-tidy-22`,
# `g++-15`, ...) are probed under both names: a suffixed name can break while
# the plain one still works (see mkVersionedToolLinks in nix/packages.nix).
#
# Tools scoped to a single dev shell rather than to commonPackages are checked
# only in that shell, keyed off XRPL_DEVSHELL.
@@ -36,16 +34,9 @@
set -uo pipefail
# major_version <compiler>
# Major version of a compiler on PATH, or "unknown" when it isn't there.
major_version() {
local version
version="$("$1" -dumpversion 2>/dev/null)" || version=""
version="${version%%.*}"
printf '%s' "${version:-unknown}"
}
llvm_version="$(major_version clang)"
# Version suffixes of the Nix tool links, tracking nix/packages.nix.
gcc_version=15
llvm_version=22
missing=()
checked=0
@@ -121,7 +112,6 @@ if [ "${os}" = "linux" ] || [ "${os}" = "macos" ]; then
check ClangBuildAnalyzer
check curl
check file
check jq
check less
check make
# net-tools netstat reports "net-tools X.Y"; macOS ships BSD netstat with no
@@ -190,7 +180,6 @@ fi
if [ "${os}" = "linux" ]; then
echo
echo "GCC toolchain:"
gcc_version="$(major_version gcc)"
check gcc
check "gcc-${gcc_version}"
check g++

View File

@@ -1,65 +0,0 @@
#!/usr/bin/env bash
# Fail if a binary of a build-context Conan package loads anything from the Nix
# store other than glibc, or cannot resolve a library at all.
#
# Only binaries linked by the Nix toolchain are checked, i.e. those recording a
# store path as their interpreter or RUNPATH. Prebuilt upstream binaries (such
# as the ones the cmake package ships) use the system loader instead.
#
# Build-context packages provide the tools that run during the build (protoc,
# grpc_cpp_plugin, ...). Their package ID does not change when a Nix toolchain
# update moves the GCC runtime to a new store path, so a cached binary has to
# get by with the pinned glibc alone. See docs/build/nix.md.
#
# Usage: bin/nix/check-build-context-runtime.sh <graph.json>
# <graph.json> is the output of `conan install --format=json`.
set -euo pipefail
if [ "$#" -ne 1 ]; then
echo "usage: $0 <graph.json>" >&2
exit 2
fi
if [ "$(uname -s)" != "Linux" ]; then
echo "$0: Linux only" >&2
exit 2
fi
folders="$(jq -r '.graph.nodes[] | select(.context == "build" and .package_folder) | .package_folder' "$1" | sort -u)"
checked=0
failed=0
while IFS= read -r file; do
case "$(file -b "${file}")" in
ELF*) ;;
*) continue ;;
esac
[[ "$(readelf -ldW "${file}")" == */nix/store/* ]] || continue
checked=$((checked + 1))
# `ldd` lists the interpreter and every library as the loader resolves them.
if deps="$(ldd "${file}" 2>&1)"; then
bad="$(printf '%s\n' "${deps}" |
grep -E 'not found|/nix/store/' |
grep -vE '/nix/store/[^/]+-glibc-[^/]+/' || true)"
else
case "${deps}" in
*"not a dynamic executable"*) continue ;;
esac
bad="${deps}"
fi
if [ -n "${bad}" ]; then
failed=$((failed + 1))
echo "::error file=${file}::loads a library from the Nix store other than glibc"
echo "${file}"
echo "${bad}" | sed 's/^/ /'
fi
done < <(
# shellcheck disable=SC2086 # one folder per line, no spaces in Conan paths
[ -z "${folders}" ] || find ${folders} -type f \( -perm -u+x -o -name '*.so*' \)
)
echo "Build-context packages: checked ${checked} binaries, ${failed} failed."
[ "${failed}" -eq 0 ]

View File

@@ -23,7 +23,7 @@ apt-get clean
rm -rf /var/lib/apt/lists/*
EOF
ARG PRE_COMMIT_VERSION=4.6.2
ARG PRE_COMMIT_VERSION=4.6.0
RUN pip install --no-cache --break-system-packages \
pre-commit==${PRE_COMMIT_VERSION}

View File

@@ -26,6 +26,8 @@ import sys
import tempfile
from pathlib import Path
CLANG_TIDY_VERSION = 22
# Extensions run-clang-tidy can analyse: `.cpp` translation units and, thanks to
# the `verify_headers` build option, `.h`/`.hpp` headers (each has its own
# compile_commands.json entry). `.ipp` fragments have no entry and are skipped.
@@ -37,21 +39,8 @@ TIDY_EXTENSIONS = {".cpp", ".h", ".hpp"}
FILEPATH_RE = re.compile(r"^(\s*(?:-\s+)?FilePath:\s*)'((?:[^']|'')*)'\s*$")
def clang_tidy_major() -> str | None:
"""Major version of the `clang-tidy` on PATH, which run-clang-tidy invokes."""
if not (clang_tidy := shutil.which("clang-tidy")):
return None
output = subprocess.run(
[clang_tidy, "--version"], capture_output=True, text=True
).stdout
m = re.search(r"LLVM version (\d+)", output)
return m.group(1) if m else None
def find_tool(name: str, version: str | None) -> str | None:
"""Prefer `<name>-<version>`, so a host tool of another version can't win."""
candidates = ([f"{name}-{version}"] if version else []) + [name]
for candidate in candidates:
def find_tool(name: str) -> str | None:
for candidate in (f"{name}-{CLANG_TIDY_VERSION}", name):
if path := shutil.which(candidate):
return path
return None
@@ -114,9 +103,8 @@ def main():
if not files:
return 0
version = clang_tidy_major()
run_clang_tidy = find_tool("run-clang-tidy", version)
clang_apply_replacements = find_tool("clang-apply-replacements", version)
run_clang_tidy = find_tool("run-clang-tidy")
clang_apply_replacements = find_tool("clang-apply-replacements")
missing = [
name
for name, path in (
@@ -126,10 +114,9 @@ def main():
if not path
]
if missing:
tried = f" (tried the '-{version}' suffix too)" if version else ""
print(
f"clang-tidy check failed: TIDY is enabled but {' and '.join(missing)} "
f"was not found in PATH{tried}.",
f"was not found in PATH (tried the '-{CLANG_TIDY_VERSION}' suffix too).",
file=sys.stderr,
)
return 1

View File

@@ -17,7 +17,7 @@
(runtime libraries resolved through the rpath) are skipped too.
Everywhere else `patch_nix_binary` is a no-op.
The default loader is resolved by bin/nix/default-loader-path.sh.
The default loader is resolved by bin/default-loader-path.sh.
#]===================================================================]
include_guard(GLOBAL)
@@ -25,7 +25,7 @@ include_guard(GLOBAL)
include(CompilationEnv)
# Resolves the system default ELF loader path for the current architecture.
set(_loader_path_script "${CMAKE_SOURCE_DIR}/bin/nix/default-loader-path.sh")
set(_loader_path_script "${CMAKE_SOURCE_DIR}/bin/default-loader-path.sh")
if(
is_linux

View File

@@ -292,14 +292,6 @@ if(xrpld)
)
target_sources(xrpld PRIVATE ${sources})
rpcspec_generate_instantiations(
OUT_VAR rpcspec_instantiations
VALUE_TYPE "::json::Value"
VIEW_HEADER "xrpld/rpc/detail/JsonObjectView.hpp"
HANDLERS book_changes ledger transaction_entry
)
target_sources(xrpld PRIVATE ${rpcspec_instantiations})
if(tests)
file(
GLOB_RECURSE sources
@@ -330,7 +322,6 @@ if(xrpld)
# antithesis_instrumentation.h, which is not exported as INTERFACE
target_include_directories(
xrpld
SYSTEM
PRIVATE ${CMAKE_SOURCE_DIR}/external/antithesis-sdk
)
endif()

View File

@@ -3,7 +3,7 @@
"requires": [
"zlib/1.3.2#1cb806da49011867778ffb6ac7190fcb%1782392402.122708",
"xxhash/0.8.3#681d36a0a6111fc56e5e45ea182c19cc%1782392402.420688",
"xrpl-rpc-spec/0.1.21#d536f87a2ae7d313452746cfa3ac4404%1790869005.122975",
"xrpl-rpc-spec/0.1.19#870b2d3abcfbbf13b61c2d3c69495060%1790348286.187549",
"sqlite3/3.53.0#324ada52333108388a9a6108bfa96734%1782392403.185447",
"soci/4.0.3#e726491a03468795453f7c83fc924a96%1782392402.679521",
"snappy/1.1.10#968fef506ff261592ec30c574d4a7809%1782307151.633168",
@@ -20,7 +20,7 @@
"libarchive/3.8.7#c446109bd1f1d8ba7936c94189bc50e6%1782392403.066892",
"jemalloc/5.3.1#1fc58d55316041f10fbc1e8a2eae632a%1776700028.228",
"gtest/1.17.0#5224b3b3ff3b4ce1133cbdd27d53ee7d%1782392402.791979",
"grpc/1.81.1#aaa93ab6cda2f2baa6a84490582c8adf%1791284951.826256",
"grpc/1.81.1#b87796a4269034856cbc1a2522db16eb%1788275071.530512",
"fast_float/8.2.10#f6f28d6bb22112078e7dbda611caf681%1785888854.601666",
"ed25519/2015.03#ae761bdc52730a843f0809bdf6c1b1f6%1782307148.15562",
"date/3.0.4#862e11e80030356b53c2c38599ceb32b%1782392402.538492",
@@ -34,15 +34,11 @@
"build_requires": [
"zlib/1.3.2#1cb806da49011867778ffb6ac7190fcb%1782392402.122708",
"strawberryperl/5.32.1.1#8d114504d172cfea8ea1662d09b6333e%1782395692.540639",
"re2/20251105#8579cfd0bda4daf0683f9e3898f964b4%1782392402.431897",
"protobuf/6.33.5#ff253ead763bd8d9904a52979cd21e81%1782392410.233933",
"openssl/3.6.3#f806de8933e3bf6f01016c6a888cee2e%1783945160.863288",
"nasm/2.16.01#31e26f2ee3c4346ecd347911bd126904%1782395690.33162",
"msys2/cci.latest#d22fe7b2808f5fd34d0a7923ace9c54f%1770657326.649",
"m4/1.4.19#1727f439cf74e83826ec96d0b4904eee%1784541921.659",
"grpc/1.81.1#aaa93ab6cda2f2baa6a84490582c8adf%1791284951.826256",
"cmake/4.3.3#840cf00ea09777e05c2050a50a82c722%1782392418.696091",
"c-ares/1.34.6#545240bb1c40e2cacd4362d6b8967650%1782392402.681654",
"b2/5.4.2#ffd6084a119587e70f11cd45d1a386e2%1782392402.624226",
"automake/1.16.5#b91b7c384c3deaa9d535be02da14d04f%1755524470.56",
"autoconf/2.71#51077f068e61700d65bb05541ea1e4b0%1731054366.86",

View File

@@ -3,13 +3,5 @@
core:non_interactive=True
core.download:parallel={{ os.cpu_count() }}
core.upload:parallel={{ os.cpu_count() }}
# Fall back to Conan Center's source backups when a recipe's upstream URL is down
# (e.g. the GNU FTP mirrors), see
# https://github.com/conan-io/conan-center-index/issues/28147#issuecomment-3183544772
# The backups are only tried once every upstream URL has used up its retries,
# so keep the retries low.
core.sources:download_urls=["origin", "https://c3i.jfrog.io/artifactory/conan-center-backup-sources"]
tools.files.download:retry=1
tools.files.download:retry=5
tools.files.download:retry_wait=10
# Fail fast on unreachable hosts (default connect timeout is 30s), keep the 60s read timeout.
core.net.http:timeout=(5, 60)

View File

@@ -29,9 +29,9 @@ os.version={{ min_macos_version }}
[conf]
{# The Boost recipe builds with b2, which doesn't use Conan's toolchain files. #}
{# Instead it hand-rolls the compiler for user-config.jam, #}
{# and its fallback probes a version-suffixed binary (e.g. `g++-<major>`) before plain `g++`. #}
{# Inside the Nix shell the wrapper only provides `g++`/`gcc` (no `-<major>` suffix), #}
{# so on a host that also has a system `g++-<major>` the probe escapes Nix #}
{# and its fallback probes a version-suffixed binary (e.g. `g++-15`) before plain `g++`. #}
{# Inside the Nix shell the wrapper only provides `g++`/`gcc` (no `-15` suffix), #}
{# so on a host that also has a system `g++-15` the probe escapes Nix #}
{# and picks the system compiler, which is mismatched with the Nix libraries #}
{# and breaks the build (e.g. Boost.Stacktrace link checks fail). #}
{# Pinning the executables here short-circuits that probe so Boost (and the rest of the toolchain) #}
@@ -49,31 +49,8 @@ tools.build:compiler_executables={'c':'{{ cc_exe }}','cpp':'{{ cxx_exe }}'}
user.package:cppstd_version=23
tools.info.package_id:confs+=["user.package:cppstd_version"]
{% if os == "Linux" and context == "build" %}
{# Build-context executables (protoc, grpc_cpp_plugin, build tools) run during the build #}
{# and would otherwise load libstdc++/libgcc from a Nix store path #}
{# that might change with a Nix toolchain update. #}
{# --as-needed drops the ones they link but don't use, #}
{# such as libatomic for grpc_cpp_plugin on arm64. #}
{% set static_runtime_flags = ["-static-libstdc++", "-static-libgcc", "-Wl,--as-needed"] %}
tools.build:exelinkflags+={{ static_runtime_flags }}
tools.info.package_id:confs+=["tools.build:exelinkflags"]
{% endif %}
{% if os == "Linux" and context == "build" %}
{# b2 links itself with its own script, which ignores exelinkflags #}
{# and only takes CXXFLAGS when use_cxx_env is set (see [buildenv] below). #}
[options]
b2/*:use_cxx_env=True
{% endif %}
[buildenv]
{# gRPC emits thousands of compiler warnings that we cannot act on. #}
{# CMake picks up CXXFLAGS, and unlike tools.build:cxxflags, #}
{# this is not part of the package ID, so binaries stay shareable. #}
grpc/*:CXXFLAGS=-w
{% if os == "Macos" %}
[buildenv]
{# os.version adds -mmacosx-version-min to compiler command lines, #}
{# but Boost.Context's b2 assembly (.S) rule ignores it, #}
{# so those objects keep the host SDK version and still warn at link time. #}
@@ -81,7 +58,3 @@ grpc/*:CXXFLAGS=-w
{# Scoped to boost/* since it is the only gap. #}
boost/*:MACOSX_DEPLOYMENT_TARGET={{ min_macos_version }}
{% endif %}
{% if os == "Linux" and context == "build" %}
b2/*:CXXFLAGS={{ static_runtime_flags | join(" ") }}
{% endif %}

View File

@@ -5,9 +5,6 @@ include(default)
{% if not sanitizers %}
{# Sanitizers not configured; no additional settings needed #}
{% elif context == "build" %}
{# Build-context packages are tools we run, not code we test, #}
{# so don't instrument them #}
{% else %}
{% if compiler == "msvc" %}

View File

@@ -40,7 +40,7 @@ class Xrpl(ConanFile):
"nudb/2.0.9",
"openssl/3.6.3",
"soci/4.0.3",
"xrpl-rpc-spec/0.1.21",
"xrpl-rpc-spec/0.1.19",
"zlib/1.3.2",
]
@@ -49,8 +49,7 @@ class Xrpl(ConanFile):
]
tool_requires = [
"grpc/<host_version>",
"protobuf/<host_version>",
"protobuf/6.33.5",
]
default_options = {
@@ -124,10 +123,6 @@ class Xrpl(ConanFile):
"xxhash/*:shared": False,
}
# default_options only reach the host context;
# give tool_requires (and their dependencies) the same dependency options.
default_build_options = {k: v for k, v in default_options.items() if "/" in k}
def set_version(self):
self.version = self.version or DEV_VERSION
@@ -154,7 +149,7 @@ class Xrpl(ConanFile):
self.requires("xxhash/0.8.3", transitive_headers=True)
exports_sources = (
"bin/nix/default-loader-path.sh",
"bin/default-loader-path.sh",
"CMakeLists.txt",
"cfg/*",
"cmake/*",

View File

@@ -80,12 +80,13 @@ function(add_xrpl_crate name)
# `cc` picks its runtime flag from `crt-static` alone, so it compiles a
# crate's C++ with `-MT`; Debug needs `-MTd` (to match cmake/XrplCompiler.cmake).
if(is_msvc)
corrosion_set_env_vars(${ARG_CRATE} "$<$<CONFIG:Debug>:CXXFLAGS=-MTd>")
corrosion_set_env_vars(
${ARG_CRATE}
"$<$<CONFIG:Debug>:CXXFLAGS=-MTd>"
)
endif()
corrosion_add_cxxbridge(
${name}_cxxbridge
CRATE ${ARG_CRATE}
FILES ${ARG_FILES}
corrosion_add_cxxbridge(${name}_cxxbridge CRATE ${ARG_CRATE} FILES
${ARG_FILES}
)
# Generated cxxbridge headers don't exist at configure time; CMake 3.28+
# validates INTERFACE_SOURCES on consuming targets. Clear it to skip the

View File

@@ -10,15 +10,14 @@ This document explains how to set one up.
support it — see [compiler support for C++23][cpp23-support].
The versions currently tested in CI are:
| Compiler | Version |
| ----------- | ------------------------------- |
| GCC | `gccVersion` in [packages.nix] |
| Clang | `llvmVersion` in [packages.nix] |
| Apple Clang | 21 |
| MSVC | Visual Studio 2026 |
| Compiler | Version |
| ----------- | ------------------ |
| GCC | 15.2 |
| Clang | 22 |
| Apple Clang | 21 |
| MSVC | Visual Studio 2026 |
LLVM tools (`clang-tidy` and `clang-format`)
come from the same LLVM release as Clang.
LLVM tools (`clang-tidy` and `clang-format`) are also pinned to version 22.
### Older compilers
@@ -157,4 +156,3 @@ version out of the box — run it via `run-clang-tidy`. No separate installation
is needed.
[cpp23-support]: https://en.cppreference.com/w/cpp/compiler_support/23
[packages.nix]: ../../nix/packages.nix

30
docs/build/nix.md vendored
View File

@@ -183,36 +183,24 @@ at link or run time.
> configuration CI covers, and no dependency binaries are published for it.
This is checked rather than assumed.
[`bin/nix/check-nix-store-refs.sh`](../../bin/nix/check-nix-store-refs.sh) takes one
file or directory and fails if a binary under it resolves a store path at run time.
[`bin/check-nix-store-refs.sh`](../../bin/check-nix-store-refs.sh) takes one file
or directory and fails if a binary under it resolves a store path at run time.
CI runs it over the build output and the Conan cache, and again in the upload job
before anything is published. You can run it yourself:
```bash
bin/nix/check-nix-store-refs.sh build
bin/nix/check-nix-store-refs.sh ~/.conan2-nix
bin/check-nix-store-refs.sh build
bin/check-nix-store-refs.sh ~/.conan2-nix
```
It works on Linux too, but asserts something narrower there: the toolchain always
writes the store into `PT_INTERP` and `RUNPATH`. That is fine for the pinned
glibc, whose path does not move, but not for the GCC runtime, which moves with
every GCC update. So [`conan/profiles/default`](../../conan/profiles/default)
links build-context packages, whose executables run during the build, with
`-static-libstdc++ -static-libgcc -Wl,--as-needed`, and
[`conan/profiles/sanitizers`](../../conan/profiles/sanitizers) does not
instrument them. CI checks that they load nothing from the store but glibc, from
the graph `conan install --format=json` writes:
writes the store into `PT_INTERP` and `RUNPATH`, and CI builds inside an image
whose store is fixed for its lifetime, so that is fine. Only the binaries
[`PatchNixBinary.cmake`](../../cmake/PatchNixBinary.cmake) retargets to the
system loader have to be clean, and those are what CI checks:
```bash
bin/nix/check-build-context-runtime.sh graph.json
```
Only the binaries [`PatchNixBinary.cmake`](../../cmake/PatchNixBinary.cmake)
retargets to the system loader have to be fully clean, and those are what CI
checks:
```bash
bin/nix/check-nix-store-refs.sh build/xrpld
bin/check-nix-store-refs.sh build/xrpld
```
### The libresolv stub

View File

@@ -178,11 +178,11 @@ A binary stops starting after a `nix flake update`, or after
dyld[57271]: Library not loaded: /nix/store/…-libresolv-93/lib/libresolv.9.dylib
```
[`bin/nix/check-nix-store-refs.sh`](../../bin/nix/check-nix-store-refs.sh) finds the
same thing without having to run anything, and names the file:
[`bin/check-nix-store-refs.sh`](../../bin/check-nix-store-refs.sh) finds the same
thing without having to run anything, and names the file:
```
$ bin/nix/check-nix-store-refs.sh ~/.conan2-nix
$ bin/check-nix-store-refs.sh ~/.conan2-nix
::error file=/Users/you/.conan2-nix/p/b/c-area24ded30c388c/p/bin/adig::references the Nix store at run time
/Users/you/.conan2-nix/p/b/c-area24ded30c388c/p/bin/adig
/nix/store/p4lp3xq4imd1qzqh08x8vcq2zfhi7rca-libresolv-93/lib/libresolv.9.dylib

View File

@@ -6,7 +6,7 @@ project(antithesis-sdk-cpp VERSION 0.4.4 LANGUAGES CXX)
add_library(antithesis-sdk-cpp INTERFACE antithesis_sdk.h)
# Note, both sections below created by xrpld project
target_include_directories(antithesis-sdk-cpp SYSTEM INTERFACE
target_include_directories(antithesis-sdk-cpp INTERFACE
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>
)

12
flake.lock generated
View File

@@ -2,11 +2,11 @@
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1791130267,
"narHash": "sha256-1sjwcQMcgAbmiip/VivBSiK8p2bTkQGOvCuO7vnwfx4=",
"lastModified": 1781173989,
"narHash": "sha256-fnzKKPvS+oieI/pTzotA5tkoM47EB1NpaBcgk4R97hE=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "9013764fcc0ea99fa16cf7aa7decf8a5c3889dd2",
"rev": "8c91a71d13451abc40eb9dae8910f972f979852f",
"type": "github"
},
"original": {
@@ -47,11 +47,11 @@
]
},
"locked": {
"lastModified": 1791189933,
"narHash": "sha256-Y8eN7ki0Urx4Ojf4w1xSpWg8uk/3lJqK+E0Oth77rH0=",
"lastModified": 1784611586,
"narHash": "sha256-OfqgY+0hp/zseZB7uyH0U8kIDPS4scZZCyAurEplvG0=",
"owner": "oxalica",
"repo": "rust-overlay",
"rev": "e60029353d0c48d216bc4b065168ccd4079c166f",
"rev": "14f58845249f3552a89b07772626b8d3c632fa86",
"type": "github"
},
"original": {

View File

@@ -86,12 +86,15 @@ public:
requires std::convertible_to<TT*, T*>
SharedIntrusive(SharedIntrusive<TT> const& rhs);
SharedIntrusive(SharedIntrusive&& rhs);
// noexcept so move_if_noexcept moves rather than copies, which NodePathStack's path_ vector
// relies on when it relocates. The body is a std::exchange on a raw pointer, which is what
// makes the noexcept honest.
SharedIntrusive(SharedIntrusive&& rhs) noexcept;
template <class TT>
requires std::convertible_to<TT*, T*>
SharedIntrusive(
SharedIntrusive<TT>&& rhs); // NOLINT(cppcoreguidelines-rvalue-reference-param-not-moved)
// NOLINTNEXTLINE(cppcoreguidelines-rvalue-reference-param-not-moved)
SharedIntrusive(SharedIntrusive<TT>&& rhs) noexcept;
SharedIntrusive&
operator=(SharedIntrusive const& rhs);
@@ -529,6 +532,22 @@ staticPointerCast(TT const& v)
return SharedPtr<T>(StaticCastTagSharedIntrusive{}, v);
}
/**
* Statically cast an intrusive pointer, moving out of it.
*
* Spelling the parameter as `SharedIntrusive<TT>&&` binds rvalues only and
* leaves lvalues to the `const&` overload above.
*
* @param v the pointer to cast, left empty afterwards.
* @return a pointer of the requested type to the same object.
*/
template <class T, class TT>
[[nodiscard]] SharedPtr<T>
staticPointerCast(SharedIntrusive<TT>&& v)
{
return SharedPtr<T>(StaticCastTagSharedIntrusive{}, std::move(v));
}
template <class T, class TT>
SharedPtr<T>
dynamicPointerCast(TT const& v)

View File

@@ -43,7 +43,7 @@ SharedIntrusive<T>::SharedIntrusive(SharedIntrusive<TT> const& rhs)
}
template <class T>
SharedIntrusive<T>::SharedIntrusive(SharedIntrusive&& rhs)
SharedIntrusive<T>::SharedIntrusive(SharedIntrusive&& rhs) noexcept
: ptr_{std::move(rhs).unsafeExchange(nullptr)}
{
}
@@ -51,7 +51,7 @@ SharedIntrusive<T>::SharedIntrusive(SharedIntrusive&& rhs)
template <class T>
template <class TT>
requires std::convertible_to<TT*, T*>
SharedIntrusive<T>::SharedIntrusive(SharedIntrusive<TT>&& rhs)
SharedIntrusive<T>::SharedIntrusive(SharedIntrusive<TT>&& rhs) noexcept
: ptr_{std::move(rhs).unsafeExchange(nullptr)}
{
}

View File

@@ -1,10 +1,15 @@
#pragma once
#include <xrpl/basics/TaggedCache.h>
#include <xrpl/basics/base_uint.h>
namespace xrpl {
using KeyCache = TaggedCache<UInt256, int, true>;
/**
* TaggedCache in key-only mode, holding no value per key.
*
* @tparam Key the key type to remember.
*/
template <class Key>
using KeyCache = TaggedCache<Key, int, true>;
} // namespace xrpl

View File

@@ -3,84 +3,9 @@
#include <algorithm>
#include <cassert>
#include <cstddef>
#include <cstdint>
#include <limits>
#include <optional>
namespace xrpl {
/**
* Add two signed 64-bit integers, returning std::nullopt when the exact
* mathematical sum is not representable in std::int64_t.
*/
[[nodiscard]] constexpr std::optional<std::int64_t>
checkedAdd(std::int64_t a, std::int64_t b) noexcept
{
using L = std::numeric_limits<std::int64_t>;
if ((b > 0 && a > L::max() - b) || (b < 0 && a < L::min() - b))
return std::nullopt;
return a + b;
}
/**
* Subtract two signed 64-bit integers, returning std::nullopt when the exact
* mathematical difference is not representable in std::int64_t.
*/
[[nodiscard]] constexpr std::optional<std::int64_t>
checkedSub(std::int64_t a, std::int64_t b) noexcept
{
using L = std::numeric_limits<std::int64_t>;
if ((b > 0 && a < L::min() + b) || (b < 0 && a > L::max() + b))
return std::nullopt;
return a - b;
}
static_assert(checkedAdd(0, 0) == 0);
static_assert(checkedAdd(1, -1) == 0);
static_assert(checkedAdd(-5, 2) == -3);
static_assert(!checkedAdd(std::numeric_limits<std::int64_t>::max(), 1).has_value());
static_assert(!checkedAdd(std::numeric_limits<std::int64_t>::min(), -1).has_value());
static_assert(
checkedAdd(std::numeric_limits<std::int64_t>::max() - 1, 1) ==
std::numeric_limits<std::int64_t>::max());
static_assert(
checkedAdd(
std::numeric_limits<std::int64_t>::min(),
std::numeric_limits<std::int64_t>::max()) == -1);
static_assert(
checkedAdd(
std::numeric_limits<std::int64_t>::max(),
std::numeric_limits<std::int64_t>::min()) == -1);
static_assert(
!checkedAdd(std::numeric_limits<std::int64_t>::max(), std::numeric_limits<std::int64_t>::max())
.has_value());
static_assert(
!checkedAdd(std::numeric_limits<std::int64_t>::min(), std::numeric_limits<std::int64_t>::min())
.has_value());
static_assert(checkedSub(0, 0) == 0);
static_assert(checkedSub(1, 1) == 0);
static_assert(checkedSub(-5, 2) == -7);
static_assert(checkedSub(-5, -2) == -3);
static_assert(!checkedSub(std::numeric_limits<std::int64_t>::min(), 1).has_value());
static_assert(!checkedSub(std::numeric_limits<std::int64_t>::max(), -1).has_value());
static_assert(
checkedSub(std::numeric_limits<std::int64_t>::min() + 1, 1) ==
std::numeric_limits<std::int64_t>::min());
static_assert(
checkedSub(-1, std::numeric_limits<std::int64_t>::max()) ==
std::numeric_limits<std::int64_t>::min());
static_assert(
!checkedSub(std::numeric_limits<std::int64_t>::max(), std::numeric_limits<std::int64_t>::min())
.has_value());
static_assert(
!checkedSub(std::numeric_limits<std::int64_t>::min(), std::numeric_limits<std::int64_t>::max())
.has_value());
/**
* Calculate one number divided by another number in percentage.
* The result is rounded up to the next integer, and capped in the range [0,100]

View File

@@ -36,7 +36,7 @@ template <typename T>
concept SomeChar = std::same_as<std::remove_cvref_t<T>, int8_t> ||
std::same_as<std::remove_cvref_t<T>, char> || std::same_as<std::remove_cvref_t<T>, uint8_t>;
inline constexpr std::array<std::optional<int>, 256> const kDigitLookupTable = [] {
inline constexpr std::array<std::optional<int>, 256> const kDigitLookupTable = []() {
std::array<std::optional<int>, 256> t{};
for (int i = 0; i < 10; ++i)

View File

@@ -783,7 +783,7 @@ TaggedCache<Key, T, IsKeyCache, SharedWeakUnionPointer, SharedPointerType, Hash,
std::atomic<int>& allRemovals,
std::scoped_lock<std::recursive_mutex> const&)
{
return std::thread([&, this] {
return std::thread([&, this]() {
int cacheRemovals = 0;
int mapRemovals = 0;
@@ -863,7 +863,7 @@ TaggedCache<Key, T, IsKeyCache, SharedWeakUnionPointer, SharedPointerType, Hash,
std::atomic<int>& allRemovals,
std::scoped_lock<std::recursive_mutex> const&)
{
return std::thread([&, this] {
return std::thread([&, this]() {
// NOLINTBEGIN https://github.com/XRPLF/rippled/issues/7056
int cacheRemovals = 0;
int mapRemovals = 0;

View File

@@ -17,7 +17,7 @@
namespace xrpl {
template <typename Key>
std::size_t
static std::size_t
extract(Key const& key)
{
return key;

View File

@@ -1604,7 +1604,7 @@ AgedOrderedContainer<IsMulti, IsMap, Key, T, Clock, Compare, Allocator>::erase(
beast::detail::AgedContainerIterator<IsConst, Iterator> pos)
requires(!IsBoostReverseIterator<Iterator>::value)
{
unlinkAndDeleteElement(&*(pos++).iterator());
unlinkAndDeleteElement(&*((pos++).iterator()));
return beast::detail::AgedContainerIterator<false, Iterator>(pos.iterator());
}
@@ -1617,7 +1617,7 @@ AgedOrderedContainer<IsMulti, IsMap, Key, T, Clock, Compare, Allocator>::erase(
requires(!IsBoostReverseIterator<Iterator>::value)
{
for (; first != last;)
unlinkAndDeleteElement(&*(first++).iterator());
unlinkAndDeleteElement(&*((first++).iterator()));
return beast::detail::AgedContainerIterator<false, Iterator>(first.iterator());
}

View File

@@ -2404,7 +2404,7 @@ beast::detail::AgedContainerIterator<false, Iterator>
AgedUnorderedContainer<IsMulti, IsMap, Key, T, Clock, Hash, KeyEqual, Allocator>::erase(
beast::detail::AgedContainerIterator<IsConst, Iterator> pos)
{
unlinkAndDeleteElement(&*(pos++).iterator());
unlinkAndDeleteElement(&*((pos++).iterator()));
return beast::detail::AgedContainerIterator<false, Iterator>(pos.iterator());
}
@@ -2424,7 +2424,7 @@ AgedUnorderedContainer<IsMulti, IsMap, Key, T, Clock, Hash, KeyEqual, Allocator>
beast::detail::AgedContainerIterator<IsConst, Iterator> last)
{
for (; first != last;)
unlinkAndDeleteElement(&*(first++).iterator());
unlinkAndDeleteElement(&*((first++).iterator()));
return beast::detail::AgedContainerIterator<false, Iterator>(first.iterator());
}

View File

@@ -390,7 +390,7 @@ void
hash_append(Hasher& h, boost::container::flat_set<Key, Compare, Alloc> const& v) noexcept
requires(IsContiguouslyHashable<Key, Hasher>::value)
{
h(&v.begin(), v.size() * sizeof(Key));
h(&(v.begin()), v.size() * sizeof(Key));
}
// tuple

View File

@@ -8,7 +8,7 @@ namespace beast::insight {
class HookImpl : public std::enable_shared_from_this<HookImpl>
{
public:
using HandlerType = std::function<void()>;
using HandlerType = std::function<void(void)>;
virtual ~HookImpl() = 0;
};

View File

@@ -61,7 +61,7 @@ isMulticast(Address const& addr)
inline bool
isPrivate(Address const& addr)
{
return addr.is_v4() ? isPrivate(addr.to_v4()) : isPrivate(addr.to_v6());
return (addr.is_v4()) ? isPrivate(addr.to_v4()) : isPrivate(addr.to_v6());
}
/**
@@ -70,7 +70,7 @@ isPrivate(Address const& addr)
inline bool
isPublic(Address const& addr)
{
return addr.is_v4() ? isPublic(addr.to_v4()) : isPublic(addr.to_v6());
return (addr.is_v4()) ? isPublic(addr.to_v4()) : isPublic(addr.to_v6());
}
} // namespace ip

View File

@@ -34,7 +34,7 @@ typeName()
name += " volatile";
if (std::is_lvalue_reference_v<T>)
{
name += '&';
name += "&";
}
else if (std::is_rvalue_reference_v<T>)
{

View File

@@ -20,7 +20,7 @@ namespace beast::unit_test {
namespace detail {
template <class String>
std::string
static std::string
makeReason(String const& reason, char const* file, int line)
{
std::string s(reason);

View File

@@ -47,8 +47,8 @@ public:
template <class F, class... Args>
explicit Thread(Suite& s, F&& f, Args&&... args) : s_(&s)
{
std::function<void()> b = [f = std::forward<F>(f),
... args = std::forward<Args>(args)] mutable {
std::function<void(void)> b = [f = std::forward<F>(f),
... args = std::forward<Args>(args)]() mutable {
std::invoke(f, args...);
};
t_ = std::thread(&Thread::run, this, std::move(b));
@@ -94,7 +94,7 @@ public:
private:
void
run(std::function<void()> f)
run(std::function<void(void)> f)
{
try
{

View File

@@ -54,7 +54,7 @@ JobQueue::Coro::post()
}
// sp keeps 'this' alive
if (jq_.addJob(type_, name_, [this, sp = shared_from_this()] { resume(); }))
if (jq_.addJob(type_, name_, [this, sp = shared_from_this()]() { resume(); }))
{
return true;
}
@@ -130,7 +130,7 @@ inline void
JobQueue::Coro::join()
{
std::unique_lock<std::mutex> lk(mutexRun_);
cv_.wait(lk, [this] { return !running_; });
cv_.wait(lk, [this]() { return !running_; });
}
} // namespace xrpl

View File

@@ -105,6 +105,16 @@ public:
std::vector<UInt256> const& amendments,
Family& family);
/**
* Create a ledger from a header whose maps are filled in afterwards.
*
* Both maps start Synching against the hashes the header carries, which are
* input rather than derived, so setImmutable() leaves them alone.
*
* @param info The header to build from.
* @param rules The rules in force.
* @param family The SHAMap family the maps belong to.
*/
Ledger(LedgerHeader const& info, Rules rules, Family& family);
/**
@@ -257,38 +267,79 @@ public:
header_.validated = true;
}
void
/**
* Mark this ledger as accepted and attempt to make it immutable.
*
* The close-time fields are recorded first, since the ledger hash covers
* them.
*
* @param closeTime The consensus-agreed close time.
* @param closeResolution The close time resolution.
* @param correctCloseTime Whether consensus agreed on the close time. If
* false, kSLcfNoConsensusTime is recorded in closeFlags instead.
* @return What setImmutable() returned.
*/
[[nodiscard]] bool
setAccepted(
NetClock::time_point closeTime,
NetClock::duration closeResolution,
bool correctCloseTime);
void
/**
* Mark this ledger as immutable, so it can no longer be modified.
*
* Only a map syncing against hashes from outside can be found unsound (see
* SHAMap::addKnownNode), so a caller whose ledger was built or loaded
* locally treats a false return as a broken internal invariant.
*
* @param rehash Whether to recompute the ledger hash from the header
* fields. The transaction and account hashes are recomputed from the
* maps too, but only the first time and only if the header did not
* supply them.
* @return false if either map is Invalid, leaving the immutable flag as it
* was and the header untouched. The ledger must then be discarded
* rather than retried, since one map may already be immutable.
*/
[[nodiscard]] bool
setImmutable(bool rehash = true);
/**
* @return Whether both maps have been settled, so the ledger can no longer
* change.
*/
bool
isImmutable() const
{
return immutable_;
}
/* Mark this ledger as "should be full".
/**
* @return Whether both maps can still be the maps the header names. See
* SHAMap::isValid().
*/
[[nodiscard]] bool
mapsValid() const
{
return txMap_.isValid() && stateMap_.isValid();
}
"Full" is metadata property of the ledger, it indicates
that the local server wants all the corresponding nodes
in durable storage.
This is marked `const` because it reflects metadata
and not data that is in common with other nodes on the
network.
*/
/**
* Mark this ledger as "should be full", indicating that the local server
* wants all the corresponding nodes in durable storage.
*
* Const because it reflects metadata, not data this ledger shares with
* other nodes on the network.
*/
void
setFull() const
{
txMap_.setFull();
// Sequence before flag, per map: setLedgerSeq() stores relaxed and setFull() stores
// release, and SHAMap::finishFetch() reads the sequence only after acquiring the flag, so
// only this order publishes it.
txMap_.setLedgerSeq(header_.seq);
stateMap_.setFull();
txMap_.setFull();
stateMap_.setLedgerSeq(header_.seq);
stateMap_.setFull();
}
void
@@ -418,8 +469,33 @@ private:
static std::pair<std::shared_ptr<STTx const>, std::shared_ptr<STObject const>>
deserializeTxPlusMeta(SHAMapItem const& item);
/**
* Make both maps immutable.
*
* Each map is settled on its own, so neither is left mid-sync because the
* other refused.
*
* @return Whether both maps are immutable.
*/
[[nodiscard]] bool
setMapsImmutable()
{
bool const txImmutable = txMap_.setImmutable();
bool const stateImmutable = stateMap_.setImmutable();
return txImmutable && stateImmutable;
}
bool immutable_;
/**
* Whether the header supplied the transaction and account hashes, so they
* must not be derived from the maps.
*
* True only for a ledger built from a header, which stays verified against
* the hash it was asked for. Fixed at construction, unlike immutable_.
*/
bool const mapHashesFromHeader_ = false;
// A SHAMap containing the transactions associated with this ledger.
SHAMap mutable txMap_;

View File

@@ -1,17 +1,11 @@
#pragma once
#include <xrpl/basics/base_uint.h>
#include <xrpl/basics/chrono.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/ledger/ApplyView.h>
#include <xrpl/ledger/ReadView.h>
#include <xrpl/ledger/entries/SLEBase.h>
#include <xrpl/protocol/Indexes.h>
#include <xrpl/protocol/LedgerFormats.h>
#include <xrpl/protocol/SField.h>
#include <map>
#include <set>
namespace xrpl {
@@ -31,52 +25,6 @@ public:
: Base(keylet::amendments(), view, j)
{
}
/**
* Returns the set of amendments this entry reports as enabled.
*
* @return the set of enabled amendments.
*/
[[nodiscard]] std::set<UInt256>
enabledAmendments() const
{
std::set<UInt256> amendments;
if (this->exists() && (*this)->isFieldPresent(sfAmendments))
{
auto const& v = (*this)->getFieldV256(sfAmendments);
amendments.insert_range(v);
}
return amendments;
}
/**
* Returns a map of amendments that have achieved majority, to the time
* majority was reached.
*
* @return a map of amendment to the time majority was reached.
*/
[[nodiscard]] std::map<UInt256, NetClock::time_point>
majorityAmendments() const
{
std::map<UInt256, NetClock::time_point> ret;
if (this->exists() && (*this)->isFieldPresent(sfMajorities))
{
using TimePoint = NetClock::time_point;
using Duration = TimePoint::duration;
auto const majorities = (*this)->getFieldArray(sfMajorities);
for (auto const& m : majorities)
{
ret[m.getFieldH256(sfAmendment)] = TimePoint(Duration(m.getFieldU32(sfCloseTime)));
}
}
return ret;
}
};
using AmendmentsEntryR = AmendmentsEntry<ReadView>;

View File

@@ -1,17 +1,11 @@
#pragma once
#include <xrpl/basics/base_uint.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/ledger/ApplyView.h>
#include <xrpl/ledger/ReadView.h>
#include <xrpl/ledger/entries/SLEBase.h>
#include <xrpl/protocol/Indexes.h>
#include <xrpl/protocol/LedgerFormats.h>
#include <xrpl/protocol/SField.h>
#include <xrpl/protocol/STVector256.h>
#include <cstddef>
#include <optional>
namespace xrpl {
@@ -31,31 +25,6 @@ public:
: Base(keylet::skip(), view, j)
{
}
/**
* Looks up a hash `diff` slots back from the most recent entry in the
* sfHashes vector (diff == 0 is the most recent entry).
*
* Callers decide which skip list entry to read and how to translate a
* target ledger sequence into `diff`; this only does the bounds check
* and vector indexing shared by both the recent (stride 1) and distant
* (stride 256) skip lists.
*
* @param diff how many slots back from the most recent hash to look up;
* 0 is the most recent hash.
* @return the hash at that slot, or std::nullopt if the entry does not
* exist or `diff` is out of range.
*/
[[nodiscard]] std::optional<UInt256>
hashAt(std::size_t diff) const
{
if (!this->exists())
return std::nullopt;
STVector256 const vec = (*this)->getFieldV256(sfHashes);
if (vec.size() > diff)
return vec[vec.size() - diff - 1];
return std::nullopt;
}
};
using LedgerHashesEntryR = LedgerHashesEntry<ReadView>;

View File

@@ -353,7 +353,7 @@ changeSpotPriceQuality(
}
if (auto const nTakerPaysPropose = (-b + root2(res)) / (2 * a); nTakerPaysPropose > 0)
{
auto const nTakerPays = [&] {
auto const nTakerPays = [&]() {
// The fee might make the AMM offer quality less than CLOB
// quality. Therefore, AMM offer has to satisfy this constraint:
// o / i >= q. Substituting o with swapAssetIn() gives: i <= O /
@@ -372,8 +372,8 @@ changeSpotPriceQuality(
auto const takerPays =
toAmount<TIn>(getAsset(pool.in), nTakerPays, Number::RoundingMode::Upward);
// should not fail
auto amounts = TAmounts<TIn, TOut>{takerPays, swapAssetIn(pool, takerPays, tfee)};
if (Quality{amounts} < quality &&
if (auto amounts = TAmounts<TIn, TOut>{takerPays, swapAssetIn(pool, takerPays, tfee)};
Quality{amounts} < quality &&
!withinRelativeDistance(Quality{amounts}, quality, Number(1, -7)))
{
JLOG(j.error()) << "changeSpotPriceQuality failed: " << to_string(pool.in) << " "
@@ -382,19 +382,21 @@ changeSpotPriceQuality(
<< " " << to_string(amounts.out);
Throw<std::runtime_error>("changeSpotPriceQuality failed");
}
JLOG(j.trace()) << "changeSpotPriceQuality succeeded: " << to_string(pool.in) << " "
<< to_string(pool.out) << " "
<< " " << quality << " " << tfee << " " << to_string(amounts.in) << " "
<< to_string(amounts.out);
return amounts;
else
{
JLOG(j.trace()) << "changeSpotPriceQuality succeeded: " << to_string(pool.in) << " "
<< to_string(pool.out) << " "
<< " " << quality << " " << tfee << " " << to_string(amounts.in)
<< " " << to_string(amounts.out);
return amounts;
}
}
JLOG(j.trace()) << "changeSpotPriceQuality calc failed: " << to_string(pool.in) << " "
<< to_string(pool.out) << " " << quality << " " << tfee;
return std::nullopt;
}
auto amounts = [&] {
auto amounts = [&]() {
bool const inIntegral = getAsset(pool.in).integral();
bool const outIntegral = getAsset(pool.out).integral();

View File

@@ -2,10 +2,8 @@
#include <xrpl/basics/Log.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/beast/utility/Zero.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/ledger/ApplyView.h>
#include <xrpl/ledger/ReadView.h>
#include <xrpl/ledger/helpers/AccountRootHelpers.h>
#include <xrpl/ledger/helpers/MPTokenHelpers.h>
#include <xrpl/ledger/helpers/RippleStateHelpers.h>
@@ -29,291 +27,6 @@
namespace xrpl {
/**
* Validate that @p account may lock @p amount of a token for later delivery
* to @p dest.
*
* The lock-side counterpart of escrowUnlockPreclaimHelper: every issuer
* control (locking opt-in, authorization, freeze/lock, transferability,
* spendable balance) that gates locking token value lives here, so any
* transactor that locks funds applies the same rules. The signature is
* view-based rather than PreclaimContext-based so it can also run from
* doApply.
*/
template <ValidIssueType T>
TER
escrowLockPreclaimHelper(
ReadView const& view,
AccountID const& account,
AccountID const& dest,
STAmount const& amount,
beast::Journal j);
template <>
inline TER
escrowLockPreclaimHelper<Issue>(
ReadView const& view,
AccountID const& account,
AccountID const& dest,
STAmount const& amount,
beast::Journal j)
{
auto const& issue = amount.get<Issue>();
auto const& issuer = amount.getIssuer();
// If the issuer is the same as the account, return tecNO_PERMISSION
if (issuer == account)
return tecNO_PERMISSION;
// If the lsfAllowTrustLineLocking is not enabled, return tecNO_PERMISSION
auto const sleIssuer = view.read(keylet::account(issuer));
if (!sleIssuer)
return tecNO_ISSUER;
if (!sleIssuer->isFlag(lsfAllowTrustLineLocking))
return tecNO_PERMISSION;
// If the account does not have a trustline to the issuer, return tecNO_LINE
auto const sleRippleState = view.read(keylet::trustLine(account, issuer, issue.currency));
if (!sleRippleState)
return tecNO_LINE;
STAmount const balance = (*sleRippleState)[sfBalance];
// If balance is positive, issuer must have higher address than account
if (balance > beast::kZero && issuer < account)
return tecNO_PERMISSION; // LCOV_EXCL_LINE
// If balance is negative, issuer must have lower address than account
if (balance < beast::kZero && issuer > account)
return tecNO_PERMISSION; // LCOV_EXCL_LINE
// If the issuer has requireAuth set, check if the account is authorized
if (auto const ter = requireAuth(view, issue, account); !isTesSuccess(ter))
return ter;
// If the issuer has requireAuth set, check if the destination is authorized
if (auto const ter = requireAuth(view, issue, dest); !isTesSuccess(ter))
return ter;
// If the issuer has frozen the account, return tecFROZEN
if (isFrozen(view, account, issue))
return tecFROZEN;
// If the issuer has frozen the destination, return tecFROZEN
if (isFrozen(view, dest, issue))
return tecFROZEN;
STAmount const spendableAmount =
accountHolds(view, account, issue.currency, issuer, FreezeHandling::IgnoreFreeze, j);
// If the balance is less than or equal to 0, return tecINSUFFICIENT_FUNDS
if (spendableAmount <= beast::kZero)
return tecINSUFFICIENT_FUNDS;
// If the spendable amount is less than the amount, return
// tecINSUFFICIENT_FUNDS
if (spendableAmount < amount)
return tecINSUFFICIENT_FUNDS;
// If the amount is not addable to the balance, return tecPRECISION_LOSS
if (!canAdd(spendableAmount, amount))
return tecPRECISION_LOSS;
return tesSUCCESS;
}
template <>
inline TER
escrowLockPreclaimHelper<MPTIssue>(
ReadView const& view,
AccountID const& account,
AccountID const& dest,
STAmount const& amount,
beast::Journal j)
{
AccountID const issuer = amount.getIssuer();
// If the issuer is the same as the account, return tecNO_PERMISSION
if (issuer == account)
return tecNO_PERMISSION;
// If the mpt does not exist, return tecOBJECT_NOT_FOUND
auto const issuanceKey = keylet::mptokenIssuance(amount.get<MPTIssue>().getMptID());
auto const sleIssuance = view.read(issuanceKey);
if (!sleIssuance)
return tecOBJECT_NOT_FOUND;
// If the lsfMPTCanEscrow is not enabled, return tecNO_PERMISSION
if (!sleIssuance->isFlag(lsfMPTCanEscrow))
return tecNO_PERMISSION;
// If the issuer is not the same as the issuer of the mpt, return
// tecNO_PERMISSION
if (sleIssuance->getAccountID(sfIssuer) != issuer)
return tecNO_PERMISSION; // LCOV_EXCL_LINE
// If the account does not have the mpt, return tecOBJECT_NOT_FOUND
if (!view.exists(keylet::mptoken(issuanceKey.key, account)))
return tecOBJECT_NOT_FOUND;
// If the issuer has requireAuth set, check if the account is
// authorized
auto const& mptIssue = amount.get<MPTIssue>();
if (auto const ter = requireAuth(view, mptIssue, account, AuthType::WeakAuth);
!isTesSuccess(ter))
return ter;
// If the issuer has requireAuth set, check if the destination is
// authorized
if (auto const ter = requireAuth(view, mptIssue, dest, AuthType::WeakAuth); !isTesSuccess(ter))
return ter;
// If the issuer has frozen the account, return tecLOCKED
if (isFrozen(view, account, *sleIssuance))
return tecLOCKED;
// If the issuer has frozen the destination, return tecLOCKED
if (isFrozen(view, dest, *sleIssuance))
return tecLOCKED;
// If the mpt cannot be transferred, return tecNO_AUTH
if (auto const ter = canTransfer(view, mptIssue, account, dest); !isTesSuccess(ter))
return ter;
STAmount const spendableAmount = accountHolds(
view,
account,
amount.get<MPTIssue>(),
FreezeHandling::IgnoreFreeze,
AuthHandling::IgnoreAuth,
j);
// If the balance is less than or equal to 0, return tecINSUFFICIENT_FUNDS
if (spendableAmount <= beast::kZero)
return tecINSUFFICIENT_FUNDS;
// If the spendable amount is less than the amount, return
// tecINSUFFICIENT_FUNDS
if (spendableAmount < amount)
return tecINSUFFICIENT_FUNDS;
return tesSUCCESS;
}
template <ValidIssueType T>
TER
escrowLockApplyHelper(
ApplyView& view,
AccountID const& issuer,
AccountID const& sender,
STAmount const& amount,
beast::Journal journal);
template <>
inline TER
escrowLockApplyHelper<Issue>(
ApplyView& view,
AccountID const& issuer,
AccountID const& sender,
STAmount const& amount,
beast::Journal journal)
{
// Defensive: Issuer cannot create an escrow
if (issuer == sender)
return tecINTERNAL; // LCOV_EXCL_LINE
auto const ter =
directSendNoFee(view, sender, issuer, amount, !amount.holds<MPTIssue>(), journal);
if (!isTesSuccess(ter))
return ter; // LCOV_EXCL_LINE
return tesSUCCESS;
}
template <>
inline TER
escrowLockApplyHelper<MPTIssue>(
ApplyView& view,
AccountID const& issuer,
AccountID const& sender,
STAmount const& amount,
beast::Journal journal)
{
// Defensive: Issuer cannot create an escrow
if (issuer == sender)
return tecINTERNAL; // LCOV_EXCL_LINE
auto const ter = lockEscrowMPT(view, sender, amount, journal);
if (!isTesSuccess(ter))
return ter; // LCOV_EXCL_LINE
return tesSUCCESS;
}
template <ValidIssueType T>
TER
escrowUnlockPreclaimHelper(
ReadView const& view,
AccountID const& account,
STAmount const& amount,
bool checkFreeze = true);
template <>
inline TER
escrowUnlockPreclaimHelper<Issue>(
ReadView const& view,
AccountID const& account,
STAmount const& amount,
bool checkFreeze)
{
AccountID const& issuer = amount.getIssuer();
// If the issuer is the same as the account, return tesSUCCESS
if (issuer == account)
return tesSUCCESS;
// If the issuer has requireAuth set, check if the destination is authorized
if (auto const ter = requireAuth(view, amount.get<Issue>(), account); !isTesSuccess(ter))
return ter;
// If the issuer has deep frozen the destination, return tecFROZEN
if (checkFreeze &&
isDeepFrozen(view, account, amount.get<Issue>().currency, amount.getIssuer()))
return tecFROZEN;
return tesSUCCESS;
}
template <>
inline TER
escrowUnlockPreclaimHelper<MPTIssue>(
ReadView const& view,
AccountID const& account,
STAmount const& amount,
bool checkFreeze)
{
AccountID const& issuer = amount.getIssuer();
// If the issuer is the same as the account, return tesSUCCESS
if (issuer == account)
return tesSUCCESS;
// If the mpt does not exist, return tecOBJECT_NOT_FOUND
auto const issuanceKey = keylet::mptokenIssuance(amount.get<MPTIssue>().getMptID());
auto const sleIssuance = view.read(issuanceKey);
if (!sleIssuance)
return tecOBJECT_NOT_FOUND;
// If the issuer has requireAuth set, check if the account is
// authorized
auto const& mptIssue = amount.get<MPTIssue>();
if (auto const ter = requireAuth(view, mptIssue, account, AuthType::WeakAuth);
!isTesSuccess(ter))
return ter;
// If the issuer has frozen the account, return tecLOCKED
if (checkFreeze && isFrozen(view, account, *sleIssuance))
return tecLOCKED;
return tesSUCCESS;
}
//------------------------------------------------------------------------------
template <ValidIssueType T>
TER
escrowUnlockApplyHelper(
@@ -342,6 +55,9 @@ escrowUnlockApplyHelper<Issue>(
bool createAsset,
beast::Journal journal)
{
auto const& issue = amount.get<Issue>();
Keylet const trustLineKey = keylet::trustLine(receiver, issue);
bool const recvLow = issuer > receiver;
bool const senderIssuer = issuer == sender;
bool const receiverIssuer = issuer == receiver;
@@ -351,10 +67,6 @@ escrowUnlockApplyHelper<Issue>(
if (receiverIssuer)
return tesSUCCESS;
auto const& issue = amount.get<Issue>();
Keylet const trustLineKey = keylet::trustLine(receiver, issue);
bool const recvLow = issuer > receiver;
if (!ctx.view.exists(trustLineKey) && createAsset)
{
// Can the account cover the trust line's reserve?

View File

@@ -103,7 +103,7 @@ public:
void
asyncHandshake(HandshakeType type, Callback cbFunc)
{
if ((type == SslSocket::client) || secure_)
if ((type == SslSocket::client) || (secure_))
{
// must be ssl
secure_ = true;

View File

@@ -67,7 +67,7 @@ class EncodedBlob
public:
explicit EncodedBlob(std::shared_ptr<NodeObject> const& obj)
: size_([&obj] {
: size_([&obj]() {
XRPL_ASSERT(obj, "xrpl::node_store::EncodedBlob::EncodedBlob : non-null input");
if (!obj)

View File

@@ -55,7 +55,7 @@ hash_append(Hasher& h, Book const& b)
using beast::hash_append;
hash_append(h, b.in, b.out);
if (b.domain)
hash_append(h, *b.domain);
hash_append(h, *(b.domain));
}
Book

View File

@@ -208,10 +208,7 @@ enum LedgerEntryType : std::uint16_t {
\
LEDGER_OBJECT(Sponsorship, \
LSF_FLAG(lsfSponsorshipRequireSignForFee, 0x00010000) \
LSF_FLAG(lsfSponsorshipRequireSignForReserve, 0x00020000)) \
\
LEDGER_OBJECT(LoanBroker, \
LSF_FLAG(lsfLoanBrokerPrivate, 0x00010000))
LSF_FLAG(lsfSponsorshipRequireSignForReserve, 0x00020000))
// clang-format on

View File

@@ -0,0 +1,28 @@
#pragma once
namespace xrpl {
/**
* @brief Enumeration of ledger shortcuts for specifying which ledger to use.
*
* These shortcuts provide a convenient way to reference commonly used ledgers
* without needing to specify their exact hash or sequence number.
*/
enum class LedgerShortcut {
/**
* The current working ledger (open, not yet closed)
*/
Current,
/**
* The most recently closed ledger (may not be validated)
*/
Closed,
/**
* The most recently validated ledger
*/
Validated
};
} // namespace xrpl

View File

@@ -457,7 +457,9 @@ private:
// The remove_cv and remove_reference are necessitated by the STBitString
// types. Their value() returns by const ref. We return those types
// by value.
template <typename T, typename V = std::remove_cvref_t<decltype(std::declval<T>().value())>>
template <
typename T,
typename V = std::remove_cv_t<std::remove_reference_t<decltype(std::declval<T>().value())>>>
V
getFieldByValue(SField const& field) const;

View File

@@ -188,7 +188,7 @@ private:
template <class LookupNodeID>
STValidation::STValidation(SerialIter& sit, LookupNodeID&& lookupNodeID, DeserializeOptions options)
: STObject(validationFormat(), sit, sfValidation, options.requireCanonicalOrder)
, signingPubKey_([this] {
, signingPubKey_([this]() {
auto const spk = getFieldVL(sfSigningPubKey);
if (publicKeyType(makeSlice(spk)) != KeyType::Secp256k1)

View File

@@ -252,7 +252,7 @@ public:
{
auto success = (offset + (Bits / 8)) <= data_.size();
if (success)
memcpy(data.begin(), &data_.front() + offset, (Bits / 8));
memcpy(data.begin(), &(data_.front()) + offset, (Bits / 8));
return success;
}

View File

@@ -449,7 +449,7 @@ public:
// Trait tells the requires-clause which types are allowed for construction.
template <typename T>
constexpr TERSubset(T rhs)
requires(Trait<std::remove_cvref_t<T>>::value)
requires(Trait<std::remove_cv_t<std::remove_reference_t<T>>>::value)
: code_(TERtoInt(rhs))
{
}

View File

@@ -226,10 +226,6 @@ inline constexpr FlagValue tfUniversalMask = ~tfUniversal;
TF_FLAG(tfLoanUnimpair, 0x00040000), \
MASK_ADJ(0)) \
\
TRANSACTION(LoanBrokerSet, \
TF_FLAG(tfLoanBrokerPrivate, 0x00010000), \
MASK_ADJ(0)) \
\
TRANSACTION(SponsorshipSet, \
TF_FLAG(tfSponsorshipSetRequireSignForFee, 0x00010000) \
TF_FLAG(tfSponsorshipClearRequireSignForFee, 0x00020000) \

View File

@@ -20,7 +20,6 @@ XRPL_FEATURE(SmartEscrow, Supported::No, VoteBehavior::DefaultN
XRPL_FEATURE(LendingProtocolV1_2, Supported::No, VoteBehavior::DefaultNo)
XRPL_FIX (Cleanup3_5_0, Supported::Yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(ConfidentialMPTKeyRotation, Supported::No, VoteBehavior::DefaultNo)
XRPL_FIX (BatchV1_2, Supported::Yes, VoteBehavior::DefaultYes)
XRPL_FIX (Cleanup3_4_0, Supported::Yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(Sponsor, Supported::Yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(BatchV1_1, Supported::Yes, VoteBehavior::DefaultNo)

View File

@@ -549,7 +549,6 @@ LEDGER_ENTRY(ltLOAN_BROKER, 0x0088, LoanBroker, loan_broker, ({
{sfCoverAvailable, SoeDefault},
{sfCoverRateMinimum, SoeDefault},
{sfCoverRateLiquidation, SoeDefault},
{sfDomainID, SoeOptional},
}))
/** A ledger object representing a loan between a Borrower and a Loan Broker

View File

@@ -900,7 +900,6 @@ TRANSACTION(ttLOAN_BROKER_SET, 74, LoanBrokerSet,
{sfDebtMaximum, SoeOptional},
{sfCoverRateMinimum, SoeOptional},
{sfCoverRateLiquidation, SoeOptional},
{sfDomainID, SoeOptional},
}))
/** This transaction deletes a Loan Broker */

View File

@@ -335,30 +335,6 @@ public:
{
return this->sle_->isFieldPresent(sfCoverRateLiquidation);
}
/**
* @brief Get sfDomainID (SoeOptional)
* @return The field value, or std::nullopt if not present.
*/
[[nodiscard]]
protocol_autogen::Optional<SF_UINT256::type::value_type>
getDomainID() const
{
if (hasDomainID())
return this->sle_->at(sfDomainID);
return std::nullopt;
}
/**
* @brief Check if sfDomainID is present.
* @return True if the field is present, false otherwise.
*/
[[nodiscard]]
bool
hasDomainID() const
{
return this->sle_->isFieldPresent(sfDomainID);
}
};
/**
@@ -602,17 +578,6 @@ public:
return *this;
}
/**
* @brief Set sfDomainID (SoeOptional)
* @return Reference to this builder for method chaining.
*/
LoanBrokerBuilder&
setDomainID(std::decay_t<typename SF_UINT256::type::value_type> const& value)
{
object_[sfDomainID] = value;
return *this;
}
/**
* @brief Build and return the completed LoanBroker wrapper.
* @param index The ledger entry index.

View File

@@ -213,32 +213,6 @@ public:
{
return this->tx_->isFieldPresent(sfCoverRateLiquidation);
}
/**
* @brief Get sfDomainID (SoeOptional)
* @return The field value, or std::nullopt if not present.
*/
[[nodiscard]]
protocol_autogen::Optional<SF_UINT256::type::value_type>
getDomainID() const
{
if (hasDomainID())
{
return this->tx_->at(sfDomainID);
}
return std::nullopt;
}
/**
* @brief Check if sfDomainID is present.
* @return True if the field is present, false otherwise.
*/
[[nodiscard]]
bool
hasDomainID() const
{
return this->tx_->isFieldPresent(sfDomainID);
}
};
/**
@@ -362,17 +336,6 @@ public:
return *this;
}
/**
* @brief Set sfDomainID (SoeOptional)
* @return Reference to this builder for method chaining.
*/
LoanBrokerSetBuilder&
setDomainID(std::decay_t<typename SF_UINT256::type::value_type> const& value)
{
object_[sfDomainID] = value;
return *this;
}
/**
* @brief Build and return the LoanBrokerSet wrapper.
* @param publicKey The public key for signing.

View File

@@ -178,7 +178,7 @@ public:
{
using namespace std::chrono_literals;
LockedSociSession session = perf::measureDurationAndLog(
[&] { return LockedSociSession(session_, lock_); }, "checkoutDb", 10ms, j_);
[&]() { return LockedSociSession(session_, lock_); }, "checkoutDb", 10ms, j_);
return session;
}

View File

@@ -9,6 +9,7 @@
#include <xrpl/protocol/AccountID.h>
#include <xrpl/protocol/ErrorCodes.h>
#include <xrpl/protocol/LedgerHeader.h>
#include <xrpl/protocol/LedgerShortcut.h>
#include <xrpl/protocol/Protocol.h>
#include <xrpl/protocol/TxMeta.h>
#include <xrpl/protocol/TxSearched.h>
@@ -104,6 +105,21 @@ public:
using TxnMetaLedgerType = std::tuple<Blob, Blob, std::uint32_t>;
using MetaTxsList = std::vector<TxnMetaLedgerType>;
using LedgerSequence = uint32_t;
using LedgerHash = UInt256;
using LedgerSpecifier = std::variant<LedgerRange, LedgerShortcut, LedgerSequence, LedgerHash>;
struct AccountTxArgs
{
AccountID account;
std::optional<LedgerSpecifier> ledger;
bool binary = false;
bool forward = false;
uint32_t limit = 0;
std::optional<AccountTxMarker> marker;
std::optional<DelegateFilter> delegate;
};
struct AccountTxResult
{
std::variant<AccountTxs, MetaTxsList> transactions;

View File

@@ -279,7 +279,7 @@ public:
virtual bool
unsubPeerStatus(std::uint64_t uListener) = 0;
virtual void
pubPeerStatus(std::function<json::Value()> const&) = 0;
pubPeerStatus(std::function<json::Value(void)> const&) = 0;
virtual bool
subConsensus(Ref ispListener) = 0;

View File

@@ -41,7 +41,7 @@ public:
}
bool
prepare(std::size_t bytes, std::function<void()>) override
prepare(std::size_t bytes, std::function<void(void)>) override
{
return true;
}

View File

@@ -47,7 +47,7 @@ public:
* empty vector.
*/
virtual std::pair<boost::tribool, std::vector<boost::asio::const_buffer>>
prepare(std::size_t bytes, std::function<void()> resume) = 0;
prepare(std::size_t bytes, std::function<void(void)> resume) = 0;
};
template <class Streambuf>
@@ -62,7 +62,7 @@ public:
}
std::pair<boost::tribool, std::vector<boost::asio::const_buffer>>
prepare(std::size_t bytes, std::function<void()>) override
prepare(std::size_t bytes, std::function<void(void)>) override
{
if (sb_.size() == 0)
return {true, {}};

View File

@@ -34,7 +34,7 @@ public:
* @return `true` if the writer is ready to provide more data.
*/
virtual bool
prepare(std::size_t bytes, std::function<void()> resume) = 0;
prepare(std::size_t bytes, std::function<void(void)> resume) = 0;
/**
* Returns a ConstBufferSequence representing the input sequence.

View File

@@ -342,10 +342,10 @@ BaseHTTPPeer<Handler, Impl>::doWriter(
bool keepAlive,
YieldContext doYield)
{
std::function<void()> resume;
std::function<void(void)> resume;
{
auto const p = impl().shared_from_this();
resume = std::function<void()>([this, p, writer, keepAlive] {
resume = std::function<void(void)>([this, p, writer, keepAlive]() {
util::spawn(strand_, [p, writer, keepAlive](YieldContext doYield) {
p->doWriter(writer, keepAlive, doYield);
});

View File

@@ -99,7 +99,7 @@ private:
port_.protocol.contains("wss2") || port_.protocol.contains("peer")};
bool plain_{
port_.protocol.contains("http") || port_.protocol.contains("ws") ||
port_.protocol.contains("ws2")};
(port_.protocol.contains("ws2"))};
static constexpr std::chrono::milliseconds kInitialAcceptDelay{50};
static constexpr std::chrono::milliseconds kMaxAcceptDelay{2000};
std::chrono::milliseconds acceptDelay_{kInitialAcceptDelay};

View File

@@ -60,7 +60,7 @@ private:
bool closed_ = false;
std::condition_variable cv_;
boost::container::flat_map<Work*, std::weak_ptr<Work>> map_;
std::function<void()> f_;
std::function<void(void)> f_;
public:
IOList() = default;
@@ -171,7 +171,7 @@ IOList::Work::destroy()
{
if (!ios_)
return;
std::function<void()> f;
std::function<void(void)> f;
{
std::scoped_lock const lock(ios_->m_);
ios_->map_.erase(this);

View File

@@ -3,9 +3,12 @@
#include <xrpl/basics/KeyCache.h>
#include <xrpl/basics/TaggedCache.h>
#include <xrpl/basics/base_uint.h>
#include <xrpl/basics/partitioned_unordered_map.h>
#include <xrpl/beast/hash/hash_append.h>
#include <xrpl/beast/insight/Collector.h>
#include <xrpl/beast/insight/NullCollector.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/shamap/SHAMapNodeID.h>
#include <atomic>
#include <chrono>
@@ -15,6 +18,112 @@
namespace xrpl {
/**
* Names a subtree whose descendants are all resident, by hash and by position.
*
* A node's hash covers its children but neither its depth nor its path, and one
* cache answers for every map built from a NodeFamily, so the position is what
* makes a hit answer for the position asked about and no other. The depth is
* kept beside the id because SHAMapNodeID masks its id to its own depth, so an
* all-zero id belongs to the root and to every all-zero prefix alike.
*
* Held as two fields rather than as a SHAMapNodeID, which is a CountedObject:
* one per entry would report the cache's population as live node IDs in the
* get_counts RPC, and would put an atomic increment on every temporary key a
* lookup builds.
*/
struct FullBelowKey
{
/**
* The hash of the subtree.
*/
UInt256 hash;
/**
* The masked id of the node the subtree was completed at.
*/
UInt256 position;
/**
* The depth of that node.
*/
std::uint32_t depth{0};
/**
* @param subtreeHash The hash of the subtree whose descendants are all
* resident.
* @param nodeID The node the subtree was completed at.
*/
FullBelowKey(UInt256 const& subtreeHash, SHAMapNodeID const& nodeID)
: hash(subtreeHash)
, position(nodeID.getNodeID())
, depth(static_cast<std::uint32_t>(nodeID.getDepth()))
{
}
/**
* The same key, named by the node above the position and the branch taken.
*
* Produces the fields the constructor above would for that child's own node
* ID, so the two forms name one entry.
*
* @param subtreeHash The hash of the subtree that is fully resident.
* @param parentID The node above the one the subtree was completed at.
* @param branch The branch of that node leading to it.
*/
FullBelowKey(UInt256 const& subtreeHash, SHAMapNodeID const& parentID, unsigned int branch)
: hash(subtreeHash)
, position(childNodeID(parentID.getNodeID(), parentID.getDepth(), branch))
, depth(static_cast<std::uint32_t>(parentID.getDepth() + 1))
{
}
[[nodiscard]] bool
operator==(FullBelowKey const& other) const = default;
};
// The per-entry cost of this cache is computed from these sizes, so pin the layout that arithmetic
// assumes: the three fields sit back to back with no padding.
static_assert(
sizeof(FullBelowKey) == (2 * sizeof(UInt256)) + sizeof(std::uint32_t),
"FullBelowKey must stay free of padding");
static_assert(
alignof(FullBelowKey) == alignof(std::uint32_t),
"FullBelowKey's lack of padding rests on its alignment");
/**
* Feed a key to a hasher, field by field.
*
* Written out rather than hashed in one call over the object's bytes, since
* beast::IsUniquelyRepresented is opt-in per type and has no case for a
* user-defined struct, whatever its layout.
*
* @param h The hasher to feed.
* @param key The key to hash.
*/
template <class Hasher>
void
hash_append(Hasher& h, FullBelowKey const& key) noexcept
{
using beast::hash_append;
hash_append(h, key.hash, key.position, key.depth);
}
/**
* Choose which partition of the cache's map a key lives in.
*
* Taken from the subtree hash, which is already uniformly distributed.
*
* @param key The key to place.
* @return The value PartitionedUnorderedMap reduces by its partition count.
*/
template <>
inline std::size_t
extract(FullBelowKey const& key)
{
return extract(key.hash);
}
namespace detail {
/**
@@ -24,12 +133,12 @@ namespace detail {
class BasicFullBelowCache
{
private:
using CacheType = KeyCache;
using CacheType = KeyCache<FullBelowKey>;
public:
static constexpr auto kDefaultCacheTargetSize = 0;
using key_type = UInt256;
using key_type = FullBelowKey;
using ClockType = CacheType::ClockType;
/**
@@ -86,13 +195,15 @@ public:
* Refresh the last access time of an item, if it exists.
* Thread safety:
* Safe to call from any thread.
* @param key The key to refresh.
* @return `true` If the key exists.
* @param hash The hash of the subtree to ask about.
* @param parentID The node above the position to ask about it at.
* @param branch The branch of that node leading to that position.
* @return `true` If that subtree is recorded as resident at that position.
*/
bool
touchIfExists(key_type const& key)
touchIfExists(UInt256 const& hash, SHAMapNodeID const& parentID, unsigned int branch)
{
return cache_.touchIfExists(key);
return cache_.touchIfExists(key_type{hash, parentID, branch});
}
/**
@@ -101,12 +212,13 @@ public:
* be refreshed.
* Thread safety:
* Safe to call from any thread.
* @param key The key to insert.
* @param hash The hash of the subtree whose descendants are all resident.
* @param nodeID The node that subtree was completed at.
*/
void
insert(key_type const& key)
insert(UInt256 const& hash, SHAMapNodeID const& nodeID)
{
cache_.insert(key);
cache_.insert(key_type{hash, nodeID});
}
/**

View File

@@ -278,11 +278,14 @@ This cache remembers which trie keys have all of their children resident in a
when creating the missing nodes list. Missing nodes are those nodes that a
`SHAMap` refers to but that are not stored in the local database.
As a depth-first walk of a `SHAMap` is performed, if an inner node answers true to
`isFullBelow()` then it is known that none of this node's children are missing
nodes, and thus that subtree does not need to be walked. These nodes are stored
in the FullBelowCache. Subsequent walks check the FullBelowCache first when
encountering a node, and ignore that subtree if found.
As a depth-first walk of a `SHAMap` completes an inner node, meaning none of the
nodes below it are missing, the node's hash and position are stored in the
FullBelowCache. Subsequent walks check the FullBelowCache when encountering a
child, and skip that subtree if found. The entry is keyed by position as well as
hash, since a hash commits to a node's contents rather than its place, and the
same node object is shared by hash between maps and positions. For the same
reason the inner node's own `isFullBelow()` flag is written and read for a
walk's root alone, where the position is fixed.
## `SHAMapTreeNode`

View File

@@ -2,6 +2,7 @@
#include <xrpl/basics/Blob.h>
#include <xrpl/basics/IntrusivePointer.h>
#include <xrpl/basics/Log.h>
#include <xrpl/basics/SHAMapHash.h>
#include <xrpl/basics/base_uint.h>
#include <xrpl/beast/utility/Journal.h>
@@ -17,6 +18,7 @@
#include <xrpl/shamap/SHAMapNodeID.h>
#include <xrpl/shamap/SHAMapTreeNode.h>
#include <atomic>
#include <condition_variable>
#include <cstddef>
#include <cstdint>
@@ -30,6 +32,7 @@
#include <set>
#include <stack>
#include <tuple>
#include <type_traits>
#include <utility>
#include <vector>
@@ -40,7 +43,7 @@ class SHAMapSyncFilter;
/**
* Describes the current state of a given SHAMap
*/
enum class SHAMapState {
enum class SHAMapState : std::uint8_t {
/**
* The map is in flux and objects can be added and removed.
*
@@ -71,20 +74,21 @@ enum class SHAMapState {
};
/**
* A SHAMap is both a radix tree with a fan-out of 16 and a Merkle tree.
* A SHAMap is both a trie with a fan-out of 16 and a Merkle tree.
*
* A radix tree is a tree with two properties:
* A trie holds a key in the position of its nodes: the path from the root down
* to a node spells out the prefix every key below it shares (the "prefix
* property"). A SHAMap spends one nibble of the 256-bit key per level, so an
* inner node has at most 16 children and a leaf sits at depth 64 at the
* deepest. A leaf also carries its own full key, so a reader can check that it
* was reached through the branches that key names.
*
* 1. The key for a node is represented by the node's position in the tree
* (the "prefix property").
* 2. A node with only one child is merged with that child
* (the "merge property")
* An insert creates an inner node at every nibble two keys share, and a delete
* collapses a chain that reduces to one leaf. Every edge spans exactly one
* nibble, so a path has one entry per level and 65 entries at the most, and a
* path's length names each node's depth. Traversal relies on that.
*
* These properties result in a significantly smaller memory footprint for
* a radix tree.
*
* A fan-out of 16 means that each node in the tree has at most 16
* children. See https://en.wikipedia.org/wiki/Radix_tree
* See https://en.wikipedia.org/wiki/Trie
*
* A Merkle tree is a tree where each non-leaf node is labelled with the hash
* of the combined labels of its children nodes.
@@ -120,21 +124,43 @@ private:
*/
std::uint32_t cowid_ = 1;
// ledgerSeq_, state_ and full_ are touched on the nodestore fetch path and on a
// getMissingNodes() walk, neither of which may block, so pin them lock-free.
static_assert(std::atomic<std::uint32_t>::is_always_lock_free);
static_assert(std::atomic<SHAMapState>::is_always_lock_free);
static_assert(std::atomic<bool>::is_always_lock_free);
/**
* The sequence of the ledger that this map references, if any.
*
* Written by Ledger::setFull() and by InboundLedger, and read on a
* nodestore reader thread. Relaxed both ways, since it is only a lookup
* hint for a store keyed by hash.
*/
std::uint32_t ledgerSeq_ = 0;
std::atomic<std::uint32_t> ledgerSeq_ = 0;
SHAMapTreeNodePtr root_;
mutable SHAMapState state_;
/**
* The map's state. Atomic because a getMissingNodes() walk writes it with
* the acquisition's lock released. Mutable because a const descend() can
* record the Invalid verdict.
*/
mutable std::atomic<SHAMapState> state_;
SHAMapType const type_;
bool backed_ = true; // Map is backed by the database
mutable bool full_ = false; // Map is believed complete in database
bool backed_ = true; // Map is backed by the database
/**
* Map is believed complete in database.
*
* Atomic because finishFetch() clears it on any nodestore reader thread,
* several of which can run at once.
*/
mutable std::atomic<bool> full_ = false;
public:
/**
* Number of children each non-leaf node has (the 'radix tree' part of the
* map)
* Number of children each non-leaf node has, which is the trie's fan-out
*/
static constexpr unsigned int kBranchFactor = SHAMapInnerNode::kBranchFactor;
@@ -143,6 +169,18 @@ public:
*/
static constexpr unsigned int kLeafDepth = 64;
/**
* Whether only a leaf may occupy a position at `depth`.
*
* @param depth the depth to judge.
* @return whether that depth is at or past kLeafDepth.
*/
[[nodiscard]] static constexpr bool
isLeafDepth(unsigned int depth)
{
return depth >= kLeafDepth;
}
using DeltaItem =
std::pair<boost::intrusive_ptr<SHAMapItem const>, boost::intrusive_ptr<SHAMapItem const>>;
using Delta = std::map<UInt256, DeltaItem>;
@@ -152,7 +190,14 @@ public:
SHAMap&
operator=(SHAMap const&) = delete;
// Take a snapshot of the given map:
/**
* Take a snapshot of the given map.
*
* @param other The map to snapshot. An Invalid source yields an Invalid
* snapshot.
* @param isMutable Whether the snapshot may be modified. Ignored when other
* is Invalid.
*/
SHAMap(SHAMap const& other, bool isMutable);
// build new map
@@ -190,8 +235,15 @@ public:
//--------------------------------------------------------------------------
// Returns a new map that's a snapshot of this one.
// Handles copy on write for mutable snapshots.
/**
* Return a new map that is a snapshot of this one.
*
* Handles copy on write for mutable snapshots. An invalid map yields an
* invalid snapshot.
*
* @param isMutable Whether the snapshot may be modified.
* @return The snapshot.
*/
std::shared_ptr<SHAMap>
snapShot(bool isMutable) const;
@@ -243,9 +295,11 @@ public:
/**
* Find the first item after the given item.
*
* @param id the identifier of the item.
*
* @note The item does not need to exist.
* @param id the identifier of the item, which need not exist in the map.
* @return an iterator at the first item with a greater key, or end() if the
* map holds no greater key.
* @throws SHAMapMissingNode if the map cannot be walked. end() is the
* separate answer that the map holds no greater key.
*/
ConstIterator
upperBound(UInt256 const& id) const;
@@ -253,9 +307,11 @@ public:
/**
* Find the object with the greatest object id smaller than the input id.
*
* @param id the identifier of the item.
*
* @note The item does not need to exist.
* @param id the identifier of the item, which need not exist in the map.
* @return an iterator at the last item with a smaller key, or end() if the
* map holds no smaller key.
* @throws SHAMapMissingNode if the map cannot be walked. end() is the
* separate answer that the map holds no smaller key.
*/
ConstIterator
lowerBound(UInt256 const& id) const;
@@ -296,9 +352,13 @@ public:
* concurrency, to discover nodes referenced in the
* SHAMap but not available locally.
*
* Only a leaf may occupy a position at or beyond kLeafDepth. A map that
* breaks that is marked Invalid and the traversal is abandoned, so callers
* ask isValid() to tell an empty result from a satisfied map.
*
* @param maxNodes The maximum number of found nodes to return
* @param filter The filter to use when retrieving nodes
* @param return The nodes known to be missing
* @return The nodes known to be missing, or empty if the map is Invalid
*/
std::vector<std::pair<SHAMapNodeID, UInt256>>
getMissingNodes(int maxNodes, SHAMapSyncFilter const* filter);
@@ -341,6 +401,9 @@ public:
* This function is used when receiving the root node of a SHAMap from a peer during ledger
* synchronization. The node must already have been deserialized.
*
* A root offered under a hash the map does not hold names another tree and
* is reported as invalid data. A root matching that hash is a duplicate.
*
* @param hash The expected hash of the root node.
* @param rootNode A deserialized root node to add.
* @param filter Optional sync filter to track received nodes.
@@ -360,14 +423,26 @@ public:
* is inserted at the position specified by nodeID. The node must already have been
* deserialized.
*
* A node that no valid tree can hold makes the map Invalid, which is
* terminal. An acquisition reaching that verdict gives up.
*
* @param nodeID The position in the tree where this node belongs.
* @param treeNode A deserialized tree node to add.
* @param filter Optional sync filter to track received nodes.
* @return Status indicating whether the node was useful, duplicate, or invalid.
*
* @note This function expects the treeNode to be a valid, deserialized SHAMapTreeNode. The
* caller is responsible for deserialization and basic validation before calling this
* function. This also means that the nodeID must be consistent with the node's content.
* @note The caller is responsible for deserialization. The position is
* checked here, in three steps that share two verdicts. A node
* the descent reaches at a position other than the one nodeID
* claims is refused with invalid(), and the map stays valid, since
* only the label was wrong. A leaf that hash-verified at the
* position nodeID claims, but whose own key does not lie under
* nodeID, condemns the map and returns mapInvalidated(). A node the
* descent itself condemns on the way, resolved from the local store
* or the filter rather than supplied by the caller, leaves the map
* Invalid and returns invalid(), since that verdict is not the
* caller's doing. Any map outside Synching, including one already
* condemned, returns duplicate().
*/
SHAMapAddNode
addKnownNode(
@@ -375,16 +450,57 @@ public:
SHAMapTreeNodePtr treeNode,
SHAMapSyncFilter const* filter);
// status functions
void
/**
* Mark this map as immutable, so it can no longer be modified.
*
* @return false if the map is Invalid and was left unchanged, true
* otherwise.
*/
[[nodiscard]] bool
setImmutable();
bool
/**
* @return Whether the map is being synced against a hash it was given, so
* its hash is fixed while nodes may still be added.
*/
[[nodiscard]] bool
isSynching() const;
/**
* @return Whether the map is settled, with its hash and nodes fixed.
*/
[[nodiscard]] bool
isImmutable() const;
/**
* Mark this map as syncing, fixing its hash while still allowing missing
* nodes to be added.
*
* The map must be freshly constructed. The body asserts that the map is not
* Invalid, so a caller that ignores the precondition stops a build with
* assertions enabled.
*/
void
setSynching();
/**
* Mark this map as no longer syncing, so it can be modified again.
*
* Moves only a Synching map. An Immutable map stays settled, a Modifying
* map has nothing to clear, and Invalid is terminal.
*/
void
clearSynching();
bool
/**
* Whether the map can still be the map it claims to be.
*
* A map that is merely missing nodes is valid, and stays valid until
* something proves the tree it is syncing against cannot exist.
*
* @return Whether the map has not been proven impossible.
*/
[[nodiscard]] bool
isValid() const;
// caution: otherMap must be accessed only by this function
@@ -421,109 +537,362 @@ public:
private:
/**
* A path from the root of the map down to some node, pairing each node with the ID naming its
* position.
* Whether placing `node` one level below `parentDepth` leaves it no room.
*
* The two halves of an entry must agree, and the only way to get that wrong is to compute an ID
* from the wrong branch. So this type does not accept an ID at all: every push takes the branch
* being descended and derives the ID itself, so a node and its ID cannot disagree. Reads are
* exposed through the same accessors a std::stack would offer.
* Only a leaf may sit at kLeafDepth, and only a parent above kLeafDepth has
* a nibble left to record the branch taken. Both bounds are needed, since a
* leaf one level above kLeafDepth is allowed. Position is judged separately,
* by the walk that keeps a path.
*
* The conjunct order is load bearing: the depth is tested first, so the
* virtual isInner() call runs only at the one level where the bound
* applies.
*
* @param parentDepth the depth of the node being descended from.
* @param node the node about to be placed one level below it.
* @return whether that placement is past the deepest level this kind of
* node may occupy.
*/
[[nodiscard]] static bool
pastLeafDepth(unsigned int parentDepth, SHAMapTreeNode const& node)
{
return isLeafDepth(parentDepth + 1u) && (node.isInner() || isLeafDepth(parentDepth));
}
/**
* A root-down path of nodes through the map.
*
* Entry `i` is the node at depth `i`, since every edge spans one nibble
* (see the class docstring). Only the node is stored. A caller that needs a
* position reads a depth and takes the nibbles from its own key.
*/
class NodePathStack
{
public:
/**
* @return whether the path holds no node at all.
*/
[[nodiscard]] bool
empty() const
{
return stack_.empty();
return path_.empty();
}
/**
* @return how many nodes the path holds, one more than the last node's
* depth.
*/
[[nodiscard]] std::size_t
size() const
{
return stack_.size();
return path_.size();
}
[[nodiscard]] std::pair<SHAMapTreeNodePtr, SHAMapNodeID> const&
/**
* The node at the end of the path.
*
* @return a reference into the path, which a later push may invalidate
* by reallocating, so a caller that pushes must ask again. An
* empty path yields a null node the caller can test.
*/
[[nodiscard]] SHAMapTreeNodePtr const&
top() const
{
XRPL_ASSERT(!stack_.empty(), "xrpl::SHAMap::NodePathStack::top : non-empty stack");
return stack_.top();
if (path_.empty())
{
// LCOV_EXCL_START
UNREACHABLE("xrpl::SHAMap::NodePathStack::top : empty stack");
static SHAMapTreeNodePtr const kEmpty;
return kEmpty;
// LCOV_EXCL_STOP
}
return path_.back();
}
/**
* The depth of the node at the end of the path.
*
* @return the depth, which is the entry's own index. Zero on an empty
* path.
*/
[[nodiscard]] unsigned int
topDepth() const
{
if (path_.empty())
{
// LCOV_EXCL_START
UNREACHABLE("xrpl::SHAMap::NodePathStack::topDepth : empty stack");
return 0;
// LCOV_EXCL_STOP
}
return static_cast<unsigned int>(path_.size() - 1);
}
/**
* Shorten the path by one node. An empty path is left as it is.
*/
void
pop()
{
XRPL_ASSERT(!stack_.empty(), "xrpl::SHAMap::NodePathStack::pop : non-empty stack");
stack_.pop();
if (path_.empty())
{
// LCOV_EXCL_START
UNREACHABLE("xrpl::SHAMap::NodePathStack::pop : empty stack");
return;
// LCOV_EXCL_STOP
}
path_.pop_back();
}
/**
* Discard the whole path.
*/
void
clear()
{
stack_ = {};
path_.clear();
pathKey_ = UInt256{};
}
/**
* Start a path at the root of the map, whose ID is the zero-depth ID by definition.
*/
void
pushRoot(SHAMapTreeNodePtr node)
{
XRPL_ASSERT(stack_.empty(), "xrpl::SHAMap::NodePathStack::pushRoot : empty stack");
stack_.emplace(std::move(node), SHAMapNodeID{});
}
/**
* Extend the path to the child of the current node reached by `branch`.
* Shorten the path by one node and hand that node to the caller.
*
* A node keeps the depth it was reached at, never a normalized kLeafDepth. Only a leaf may
* sit at kLeafDepth, since an inner node there would have no branch left to select.
*/
void
pushChild(SHAMapTreeNodePtr node, unsigned int branch)
{
XRPL_ASSERT(node, "xrpl::SHAMap::NodePathStack::pushChild : non-null node input");
XRPL_ASSERT(
!stack_.empty(), "xrpl::SHAMap::NodePathStack::pushChild : non-empty stack");
auto childID = stack_.top().second.getChildNodeID(branch);
XRPL_ASSERT_IF(
node->isInner(),
childID.getDepth() < kLeafDepth,
"xrpl::SHAMap::NodePathStack::pushChild : inner node above leaf depth");
XRPL_ASSERT_IF(
node->isLeaf(),
childID.isPrefixOf(leafKey(*node)),
"xrpl::SHAMap::NodePathStack::pushChild : leaf key below branch");
stack_.emplace(std::move(node), std::move(childID));
}
/**
* Extend the path to a node lying on the path to `target`.
* Moves the node out, transferring the existing reference.
*
* For nodes not reached by descending a known branch: the walk tracks only the key it is
* heading for, or the node is newly created. Either way `target` selects the branch.
* @return the node that was at the end of the path, or an empty
* pointer if there was none.
*/
void
pushNode(SHAMapTreeNodePtr node, UInt256 const& target)
[[nodiscard]] SHAMapTreeNodePtr
releaseNode()
{
if (stack_.empty())
if (path_.empty())
{
pushRoot(std::move(node));
// LCOV_EXCL_START
UNREACHABLE("xrpl::SHAMap::NodePathStack::releaseNode : empty stack");
return {};
// LCOV_EXCL_STOP
}
auto node = std::move(path_.back());
path_.pop_back();
return node;
}
/**
* Shorten the path by one node and hand it over as a `Node`.
*
* @tparam Node SHAMapInnerNode or SHAMapLeafNode.
* @return the node that was at the end of the path, or an empty pointer
* if the path was empty or that node is not a `Node`. The path
* is shortened either way.
*/
template <class Node>
[[nodiscard]] intr_ptr::SharedPtr<Node>
releaseNodeAs()
{
static_assert(
std::is_same_v<Node, SHAMapInnerNode> || std::is_same_v<Node, SHAMapLeafNode>,
"releaseNodeAs serves the two concrete node kinds");
auto node = releaseNode();
if (!node)
{
// The other half of releaseNode()'s empty-path contract, asserted there by the
// UNREACHABLE in its own LCOV_EXCL block.
return {}; // LCOV_EXCL_LINE
}
if constexpr (std::is_same_v<Node, SHAMapInnerNode>)
{
// A leaf sits only at the end of a path, which both callers asking for an inner
// node have released first. The other half of the contract their own UNREACHABLEs
// assert, in SHAMap::dirtyUp and SHAMap::delItem.
if (!node->isInner())
return {}; // LCOV_EXCL_LINE
}
else
{
pushChild(std::move(node), selectBranch(stack_.top().second, target));
if (!node->isLeaf())
return {};
}
return intr_ptr::staticPointerCast<Node>(std::move(node));
}
/**
* Start a path at the root of the map, which sits at depth zero by
* definition.
*
* @return false, leaving the path unchanged, if a path was already
* started.
*/
[[nodiscard]] bool
pushRoot(SHAMapTreeNodePtr node)
{
if (!path_.empty())
{
// LCOV_EXCL_START
UNREACHABLE("xrpl::SHAMap::NodePathStack::pushRoot : non-empty stack");
return false;
// LCOV_EXCL_STOP
}
// Reserved here rather than in the constructor, to keep SHAMap::end()
// allocation-free. A path holds at most kLeafDepth + 1 entries, so no push on a path
// started here reallocates. A copy is not given that room, so top()'s invalidation
// rule still holds.
path_.reserve(kLeafDepth + 1u);
path_.push_back(std::move(node));
return true;
}
/**
* Extend the path to the child of the node at its end reached by
* `branch`.
*
* Only a leaf may sit at kLeafDepth, since an inner node there would
* have no branch left to select.
*
* @param node the child to append.
* @param branch the branch of the current node that `node` was reached
* through, which serves to judge the node offered.
* @return false if there is no node to descend from or to push, no
* branch of that number, no room left below for the kind of
* node offered, or a leaf whose own key does not select
* `branch`. The path keeps its nodes.
*/
[[nodiscard]] bool
pushChild(SHAMapTreeNodePtr node, unsigned int branch)
{
if (path_.empty() || !node || branch >= kBranchFactor)
{
// LCOV_EXCL_START
UNREACHABLE("xrpl::SHAMap::NodePathStack::pushChild : no child to push");
return false;
// LCOV_EXCL_STOP
}
// A node must have room below it at the depth it is offered for. Reachable, since a
// node resolved by hash carries no position of its own.
auto const parentDepth = topDepth();
bool const tooDeep = pastLeafDepth(parentDepth, *node);
SOMETIMES(tooDeep, "xrpl::SHAMap::NodePathStack::pushChild : child past leaf depth");
if (tooDeep)
{
return false;
}
// A leaf's key must agree with every branch recorded above it, which the chain in
// `pathKey_` supplies. A path's length gives only the depths.
pathKey_ = childNodeID(pathKey_, parentDepth, branch);
// The test takes the depth and the two keys, so no SHAMapNodeID is built: it
// is a CountedObject, and one per push would report as a live node ID.
bool const misplaced =
node->isLeaf() && !samePositionAtDepth(parentDepth + 1u, pathKey_, leafKey(*node));
SOMETIMES(
misplaced, "xrpl::SHAMap::NodePathStack::pushChild : leaf key outside branch");
if (misplaced)
{
return false;
}
path_.push_back(std::move(node));
return true;
}
/**
* Extend the path by one node lying on the way to `target`, starting
* it if it is empty.
*
* For nodes not reached by descending a known branch: the walk tracks
* only the key it is heading for, or the node is newly created. Either
* way `target` names the branch.
*
* @param node the node to append.
* @param target the key the walk is heading for.
* @return whatever pushRoot or pushChild returned.
*/
[[nodiscard]] bool
pushNode(SHAMapTreeNodePtr node, UInt256 const& target)
{
if (path_.empty())
{
return pushRoot(std::move(node));
}
return pushChild(std::move(node), selectBranch(topDepth(), target));
}
private:
std::stack<std::pair<SHAMapTreeNodePtr, SHAMapNodeID>> stack_;
// path_[i] holds the node at depth i, by construction: pushRoot starts at depth 0 and
// pushChild only ever appends one level.
std::vector<SHAMapTreeNodePtr> path_;
// The branches descended, one nibble per level: nibble i is the branch taken from depth i.
// Nibbles below the path's end are stale after a pop, so every read masks at the path's
// own depth. childNodeID clears a nibble before writing it, so re-descending overwrites.
UInt256 pathKey_;
};
using DeltaRef =
std::pair<boost::intrusive_ptr<SHAMapItem const>, boost::intrusive_ptr<SHAMapItem const>>;
/**
* The sequence of the ledger this map references, read atomically.
*
* @return The sequence, or zero if the map references no ledger.
*/
[[nodiscard]] std::uint32_t
ledgerSeq() const;
/**
* The current state, read atomically.
*
* Orders state_ alone. The tree's nodes carry no ordering guarantees.
*
* @return The state as of the call, which a concurrent walk may already
* have moved past.
*/
[[nodiscard]] SHAMapState
state() const;
/**
* Record that the map is provably not the one it claims to be.
*
* Cannot fail, since Invalid outranks every other state. Const because a
* const descend() can reach this verdict while resolving a node.
*/
void
setInvalid() const;
/**
* Condemn the map, naming in the log the node whose position proves it.
*
* Const, because a read path can reach this verdict. See setInvalid().
*
* @param node the node that cannot sit where it was offered.
* @param hash the hash it was resolved under.
* @param position the place in the tree it was offered for.
*/
void
condemn(SHAMapTreeNode const& node, SHAMapHash const& hash, SHAMapNodeID const& position) const;
/**
* Move the map to a new state, atomically.
*
* With clearSynching(), the only writer of state_ past construction.
* Invalid is always stored, and every other transition is refused once
* the map is Invalid, which is what makes that verdict terminal.
* clearSynching() is narrower and moves only Synching.
*
* Const to let setInvalid() be const. state_ is mutable, which makes that
* legal.
*
* @param desired The state to move to.
* @return false if the map is Invalid and the requested state is not,
* leaving it unchanged. True otherwise.
*/
bool
trySetState(SHAMapState desired) const;
// tree node cache operations
SHAMapTreeNodePtr
cacheLookup(SHAMapHash const& hash) const;
@@ -550,9 +919,13 @@ private:
dirtyUp(NodePathStack& stack, UInt256 const& target, SHAMapTreeNodePtr terminal);
/**
* Walk towards the specified id, returning the node. Caller must check
* if the return is nullptr, and if not, if the node->peekItem()->key() ==
* id
* Walk towards the specified id, returning the node.
*
* @param id the key to walk towards, which need not be in the map.
* @param stack records the path walked, or nullptr to skip recording it.
* @return the leaf the walk ended on, or nullptr if it ended on an inner
* node or was refused. A returned leaf need not hold `id`, so
* callers compare its key themselves.
*/
SHAMapLeafNode*
walkTowardsKey(UInt256 const& id, NodePathStack* stack = nullptr) const;
@@ -564,10 +937,16 @@ private:
/**
* Unshare the node, allowing it to be modified
*
* @param node the node to unshare.
* @param depth the depth the node sits at, which says whether it is the
* root. A clone of the root has to be adopted as the new root. A
* clone of any other node is hooked up by the caller.
* @return the node, cloned if it was shared.
*/
template <class Node>
intr_ptr::SharedPtr<Node>
unshareNode(intr_ptr::SharedPtr<Node>, SHAMapNodeID const& nodeID);
[[nodiscard]] intr_ptr::SharedPtr<Node>
unshareNode(intr_ptr::SharedPtr<Node> node, unsigned int depth);
/**
* prepare a node to be modified before flushing
@@ -588,8 +967,13 @@ private:
/**
* Returns the first or last item at or below the node already on top of `stack`, extending
* `stack` with the path walked to reach it.
*
* @param stack the path to extend, whose last node the search starts from.
* @param direction whether to take the lowest or the highest branch at
* each level.
* @return the leaf found, or nullptr if no leaf lies below that node.
*/
SHAMapLeafNode*
[[nodiscard]] SHAMapLeafNode*
belowHelper(NodePathStack& stack, BelowDirection direction) const;
/**
@@ -603,7 +987,7 @@ private:
* @param direction First to search upwards from `id`, Last to search downwards.
* @return An iterator to the item found, or end() if no item lies on that side of `id`.
*/
ConstIterator
[[nodiscard]] ConstIterator
boundHelper(UInt256 const& id, BelowDirection direction) const;
// Simple descent
@@ -628,6 +1012,22 @@ private:
bool& pending,
DescendCallback&&) const;
/**
* Resolve the child of `parent` on `branch`, judging where it lands.
*
* Const, yet it records the Invalid verdict through setInvalid(). A new
* caller must therefore be a path that may reach that verdict. Today
* addKnownNode() is the only one.
*
* @param parent the inner node to descend from.
* @param parentID the position of `parent`.
* @param branch the branch of `parent` to resolve, which the caller has
* already bounded.
* @param filter an alternate source of nodes, or null for the store alone.
* @return the child and the child's position. A null child where the branch
* could not be resolved, or where what came back does not belong at
* that position, in which case the map is left Invalid.
*/
std::pair<SHAMapTreeNode*, SHAMapNodeID>
descend(
SHAMapInnerNode* parent,
@@ -728,10 +1128,29 @@ private:
};
// getMissingNodes helper functions
/**
* Examine the remaining branches of one inner node, recording or
* requesting what is missing.
*
* @param mn the walk's shared state, which collects the missing nodes.
* @param node the walk's current position, updated to the node to process
* next.
*/
void
gmnProcessNodes(MissingNodes&, MissingNodes::StackEntry& node);
static void
gmnProcessDeferredReads(MissingNodes&);
gmnProcessNodes(MissingNodes& mn, MissingNodes::StackEntry& node);
/**
* Wait for every read this pass posted, then hook up or record what each
* one resolved.
*
* Drains all of them even after judging the map, since an outstanding read
* holds a pointer to `mn`.
*
* @param mn the walk's shared state, holding the posted reads.
*/
void
gmnProcessDeferredReads(MissingNodes& mn);
// fetch from DB helper function
SHAMapTreeNodePtr
@@ -741,44 +1160,116 @@ private:
inline void
SHAMap::setFull()
{
full_ = true;
full_.store(true, std::memory_order_release);
}
inline void
SHAMap::setLedgerSeq(std::uint32_t lseq)
{
ledgerSeq_ = lseq;
ledgerSeq_.store(lseq, std::memory_order_relaxed);
}
inline void
inline std::uint32_t
SHAMap::ledgerSeq() const
{
return ledgerSeq_.load(std::memory_order_relaxed);
}
inline SHAMapState
SHAMap::state() const
{
return state_.load(std::memory_order_acquire);
}
inline bool
SHAMap::trySetState(SHAMapState desired) const
{
// Invalid is stored outright: that verdict outranks a concurrent transition.
if (desired == SHAMapState::Invalid)
{
state_.store(SHAMapState::Invalid, std::memory_order_release);
return true;
}
// A failed exchange both reports the state and refreshes expected, so no load is needed
// ahead of the loop.
auto expected = SHAMapState::Modifying;
while (expected != SHAMapState::Invalid)
{
if (state_.compare_exchange_weak(
expected, desired, std::memory_order_acq_rel, std::memory_order_acquire))
{
return true;
}
}
return false;
}
inline bool
SHAMap::setImmutable()
{
XRPL_ASSERT(state_ != SHAMapState::Invalid, "xrpl::SHAMap::setImmutable : state is valid");
state_ = SHAMapState::Immutable;
SOMETIMES(!isValid(), "xrpl::SHAMap::setImmutable : map is invalid");
return trySetState(SHAMapState::Immutable);
}
inline bool
SHAMap::isSynching() const
{
return state_ == SHAMapState::Synching;
return state() == SHAMapState::Synching;
}
inline bool
SHAMap::isImmutable() const
{
return state() == SHAMapState::Immutable;
}
inline void
SHAMap::setSynching()
{
state_ = SHAMapState::Synching;
// Guarded, so this is not a way out of Invalid.
if (!trySetState(SHAMapState::Synching))
{
// Only ever called on a freshly constructed map.
// LCOV_EXCL_START
UNREACHABLE("xrpl::SHAMap::setSynching : map is invalid");
// LCOV_EXCL_STOP
}
}
inline void
SHAMap::clearSynching()
{
state_ = SHAMapState::Modifying;
// Only Synching moves. A walk can end after the ledger settled, and then the map stays
// Immutable; a Modifying map has nothing to clear; an invalid map stays invalid. Only that
// last refusal is logged, since a concurrent walk is what produces it.
auto expected = SHAMapState::Synching;
if (state_.compare_exchange_strong(
expected, SHAMapState::Modifying, std::memory_order_acq_rel, std::memory_order_acquire))
{
return;
}
bool const invalid = expected == SHAMapState::Invalid;
SOMETIMES(invalid, "xrpl::SHAMap::clearSynching : map is invalid");
if (invalid)
{
JLOG(journal_.warn()) << "Refused to clear synching on an invalid map, root hash "
<< root_->getHash();
}
}
inline bool
SHAMap::isValid() const
{
return state_ != SHAMapState::Invalid;
return state() != SHAMapState::Invalid;
}
inline void
SHAMap::setInvalid() const
{
// Through trySetState() like every other transition, so state_ has one writer funnel.
trySetState(SHAMapState::Invalid);
}
inline void

View File

@@ -12,47 +12,148 @@ private:
int bad_;
int duplicate_;
// Whether a node in the batch proved the map impossible. A fact about the batch rather than
// about the map, so a caller can tell this batch's doing from a concurrent walk's verdict.
bool invalidatedMap_;
public:
SHAMapAddNode();
/**
* Record one node that was rejected.
*/
void
incInvalid();
/**
* Record one node that produced a good result.
*/
void
incUseful();
/**
* Record one node that was not needed: the map already held it, or the map
* or its acquisition was no longer taking nodes. isGood() counts it on the
* accepted side.
*/
void
incDuplicate();
void
reset();
/**
* @return How many nodes produced a good result.
*/
[[nodiscard]] int
getGood() const;
/**
* @return How many nodes this code rejected, which is not how many the
* batch was given: a batch that stops at its first bad node counts
* one, and a batch that carries on counts each.
*/
[[nodiscard]] int
getBad() const;
/**
* @return How many nodes were not needed, whether already held or offered
* to a map no longer taking nodes, tallied apart from the good and
* bad counts.
*/
[[nodiscard]] int
getDuplicate() const;
/**
* Whether nodes that produced a good result or were not needed outnumber
* the ones rejected.
*
* @return Whether the batch was good.
*/
[[nodiscard]] bool
isGood() const;
/**
* @return Whether at least one node in the batch was rejected.
*/
[[nodiscard]] bool
isInvalid() const;
/**
* @return Whether at least one node in the batch produced a good result.
*/
[[nodiscard]] bool
isUseful() const;
[[nodiscard]] std::string
get() const;
SHAMapAddNode&
operator+=(SHAMapAddNode const& n);
/**
* @return A verdict recording one node that was not needed.
*/
static SHAMapAddNode
duplicate();
/**
* @return A verdict recording one useful node.
*/
static SHAMapAddNode
useful();
/**
* @return A verdict recording one invalid node.
*/
static SHAMapAddNode
invalid();
/**
* @return A verdict recording one invalid node that proved the map
* impossible.
*/
static SHAMapAddNode
mapInvalidated();
/**
* Whether a node in the batch proved the map impossible.
*
* Reports what the batch did rather than what the map now holds, so a
* verdict another thread's walk reached is not read as this batch's. The
* two differ: SHAMap::setInvalid() is written with no lock held, by a
* getMissingNodes() walk its caller runs lock-free.
*
* @return Whether a node in the batch invalidated the map.
*/
[[nodiscard]] bool
invalidatedMap() const;
/**
* Clear every count back to zero.
*/
void
reset();
/**
* Render the tally as a log line.
*
* @return The tally, e.g. "good:2 bad:1 dupe:1", or "no nodes processed" if
* every count is zero.
*/
[[nodiscard]] std::string
get() const;
/**
* Add another verdict's counts into this one.
*
* @param n The verdict to add.
* @return This verdict, updated.
*/
SHAMapAddNode&
operator+=(SHAMapAddNode const& n);
private:
SHAMapAddNode(int good, int bad, int duplicate);
SHAMapAddNode(int good, int bad, int duplicate, bool invalidatedMap = false);
};
inline SHAMapAddNode::SHAMapAddNode() : good_(0), bad_(0), duplicate_(0)
inline SHAMapAddNode::SHAMapAddNode() : good_(0), bad_(0), duplicate_(0), invalidatedMap_(false)
{
}
inline SHAMapAddNode::SHAMapAddNode(int good, int bad, int duplicate)
: good_(good), bad_(bad), duplicate_(duplicate)
inline SHAMapAddNode::SHAMapAddNode(int good, int bad, int duplicate, bool invalidatedMap)
: good_(good), bad_(bad), duplicate_(duplicate), invalidatedMap_(invalidatedMap)
{
}
@@ -74,18 +175,30 @@ SHAMapAddNode::incDuplicate()
++duplicate_;
}
inline void
SHAMapAddNode::reset()
{
good_ = bad_ = duplicate_ = 0;
}
inline int
SHAMapAddNode::getGood() const
{
return good_;
}
inline int
SHAMapAddNode::getBad() const
{
return bad_;
}
inline int
SHAMapAddNode::getDuplicate() const
{
return duplicate_;
}
inline bool
SHAMapAddNode::isGood() const
{
return (good_ + duplicate_) > bad_;
}
inline bool
SHAMapAddNode::isInvalid() const
{
@@ -98,20 +211,10 @@ SHAMapAddNode::isUseful() const
return good_ > 0;
}
inline SHAMapAddNode&
SHAMapAddNode::operator+=(SHAMapAddNode const& n)
{
good_ += n.good_;
bad_ += n.bad_;
duplicate_ += n.duplicate_;
return *this;
}
inline bool
SHAMapAddNode::isGood() const
SHAMapAddNode::invalidatedMap() const
{
return (good_ + duplicate_) > bad_;
return invalidatedMap_;
}
inline SHAMapAddNode
@@ -132,6 +235,19 @@ SHAMapAddNode::invalid()
return SHAMapAddNode(0, 1, 0);
}
inline SHAMapAddNode
SHAMapAddNode::mapInvalidated()
{
return SHAMapAddNode(0, 1, 0, true);
}
inline void
SHAMapAddNode::reset()
{
good_ = bad_ = duplicate_ = 0;
invalidatedMap_ = false;
}
inline std::string
SHAMapAddNode::get() const
{
@@ -160,4 +276,17 @@ SHAMapAddNode::get() const
return ret;
}
inline SHAMapAddNode&
SHAMapAddNode::operator+=(SHAMapAddNode const& n)
{
good_ += n.good_;
bad_ += n.bad_;
duplicate_ += n.duplicate_;
// Or-ed, not summed: one node in the batch having proved the map impossible is the whole fact.
invalidatedMap_ = invalidatedMap_ || n.invalidatedMap_;
return *this;
}
} // namespace xrpl

View File

@@ -19,7 +19,8 @@ class SHAMapInnerNode final : public SHAMapTreeNode, public CountedObject<SHAMap
{
public:
/**
* Each inner node has 16 children (the 'radix tree' part of the map)
* Children per inner node: one branch per value of the key nibble that a
* node's depth selects.
*/
static constexpr unsigned int kBranchFactor = 16;
@@ -31,7 +32,25 @@ private:
*/
TaggedPointer hashesAndChildren_;
std::uint32_t fullBelowGen_ = 0;
// Pin that wrapping fullBelowGen_ in an atomic keeps the packed layout, and that the load
// isFullBelow() does once per node of every walk stays lock-free.
static_assert(std::atomic<std::uint32_t>::is_always_lock_free);
static_assert(sizeof(std::atomic<std::uint32_t>) == sizeof(std::uint32_t));
static_assert(alignof(std::atomic<std::uint32_t>) == alignof(std::uint32_t));
/**
* The generation in which a walk found every node below this one resident,
* with this node as the walk's root. The node object is shared by hash
* between maps and positions, and a hash commits to a node's contents
* rather than its place, so the flag is written and read for the root
* position only; below the root, the position-keyed FullBelowCache is the
* memo. Written from more than one thread, since canonicalization shares a
* node between maps and a walk can run with the acquisition lock released
* (see SHAMap::state_). Relaxed both ways, since a generation is only
* compared for equality and the children it vouches for are published
* through lock_.
*/
std::atomic<std::uint32_t> fullBelowGen_ = 0;
std::uint16_t isBranch_ = 0;
/**
@@ -148,10 +167,23 @@ public:
SHAMapTreeNodePtr
canonicalizeChild(unsigned int branch, SHAMapTreeNodePtr node);
// sync functions
/**
* Whether a walk rooted at this node found every node below it resident.
*
* Meaningful only for the node a walk starts at; see fullBelowGen_.
*
* @param generation The FullBelowCache generation the asking walk runs in.
* @return True if a walk rooted here completed in that generation.
*/
bool
isFullBelow(std::uint32_t generation) const;
/**
* Record that a walk rooted at this node found every node below it
* resident.
*
* @param gen The FullBelowCache generation the completing walk ran in.
*/
void
setFullBelowGen(std::uint32_t gen);
@@ -204,13 +236,13 @@ SHAMapInnerNode::getBranchCount() const
inline bool
SHAMapInnerNode::isFullBelow(std::uint32_t generation) const
{
return fullBelowGen_ == generation;
return fullBelowGen_.load(std::memory_order_relaxed) == generation;
}
inline void
SHAMapInnerNode::setFullBelowGen(std::uint32_t gen)
{
fullBelowGen_ = gen;
fullBelowGen_.store(gen, std::memory_order_relaxed);
}
} // namespace xrpl

View File

@@ -75,4 +75,41 @@ leafKey(SHAMapTreeNode const& node)
return safeDowncast<SHAMapLeafNode const&>(node).peekItem()->key();
}
/**
* Whether a node may occupy a position in a SHAMap.
*
* A leaf's own key names its position. An inner node carries no key, so every
* position agrees with it and a caller's own depth rules are what bound it.
*
* @param nodeID the position the node is claimed to occupy.
* @param node the node to judge.
* @return whether the node's own key agrees with that position.
*/
[[nodiscard]] inline bool
belongsAt(SHAMapNodeID const& nodeID, SHAMapTreeNode const& node)
{
return !node.isLeaf() || nodeID.isPrefixOf(leafKey(node));
}
/**
* Whether a node may occupy the child position a branch leads to.
*
* The overload above, asked about a child without building the child's id.
*
* @param parentID the position of the node above, whose depth must be below
* SHAMap::kLeafDepth.
* @param branch the branch of that node leading to the position judged.
* @param node the node to judge.
* @return whether the node's own key agrees with that position.
*/
[[nodiscard]] inline bool
belongsAt(SHAMapNodeID const& parentID, unsigned int branch, SHAMapTreeNode const& node)
{
if (!node.isLeaf())
return true;
auto const& key = leafKey(node);
return parentID.isPrefixOf(key) && selectBranch(parentID, key) == branch;
}
} // namespace xrpl

View File

@@ -123,6 +123,38 @@ operator<<(std::ostream& out, SHAMapNodeID const& node)
return out << to_string(node);
}
/**
* Returns the id of the child a parent's branch leads to
*
* A child's id is its parent's with the nibble at the parent's own depth set to
* the branch taken. Every other nibble is copied through, so a masked parent
* gives a masked child. The write counterpart of selectBranch.
*
* @param parentID the id of the parent. The nibble at parentDepth is
* overwritten rather than merged, so an id already carrying one
* there is accepted.
* @param parentDepth the depth of the parent, below SHAMap::kLeafDepth.
* @param branch the branch of the parent leading to the child.
* @return that child's id, whose depth is parentDepth + 1.
*/
[[nodiscard]] UInt256
childNodeID(UInt256 const& parentID, unsigned int parentDepth, unsigned int branch);
/**
* Whether two keys name the same position at a given depth
*
* Both sides are masked, so either may carry bits below `depth`.
* SHAMapNodeID::isPrefixOf is the stricter form.
*
* @param depth how many leading nibbles name the position, at most
* SHAMap::kLeafDepth. A greater depth is clamped to it.
* @param lhs one of the keys.
* @param rhs the other key.
* @return whether the two agree on those nibbles.
*/
[[nodiscard]] bool
samePositionAtDepth(unsigned int depth, UInt256 const& lhs, UInt256 const& rhs);
/**
* Return an object representing a serialized SHAMap Node ID
*
@@ -144,9 +176,26 @@ deserializeSHAMapNodeID(std::string_view s)
/** @} */
/**
* Returns the branch that would contain the given hash
* Returns the branch at the given depth that would contain the given hash
*
* @param depth the depth of the node whose branch to select.
* @param hash the key whose nibble at that depth names the branch.
* @return the branch containing the hash.
*/
[[nodiscard]] unsigned int
selectBranch(SHAMapNodeID const& id, UInt256 const& hash);
selectBranch(unsigned int depth, UInt256 const& hash);
/**
* Returns the branch that would contain the given hash
*
* @param id the node whose depth to read.
* @param hash the key whose nibble at that depth names the branch.
* @return the branch containing the hash.
*/
[[nodiscard]] inline unsigned int
selectBranch(SHAMapNodeID const& id, UInt256 const& hash)
{
return selectBranch(id.getDepth(), hash);
}
} // namespace xrpl

View File

@@ -30,6 +30,17 @@ public:
Blob&& nodeData,
SHAMapNodeType type) const = 0;
/**
* Fetch the node data for a hash, if this filter holds it.
*
* Postcondition: the data returned hashes to `nodeHash`. Callers adopt that
* hash without recomputing it and publish the node to the map's caches. An
* implementation checks the digest before answering, unless its own storage
* ties the data to the key.
*
* @param nodeHash the hash of the node wanted.
* @return the node's hash-prefixed wire data, or nothing.
*/
[[nodiscard]] virtual std::optional<Blob>
getNode(SHAMapHash const& nodeHash) const = 0;
};

View File

@@ -177,7 +177,18 @@ public:
virtual void
invariants(bool isRoot = false) const = 0;
static SHAMapTreeNodePtr
/**
* Parse a prefixed node, adopting `hash` without verifying it.
*
* Every caller must establish `hash` first. A node from the local store
* carries its own key, and a node from a sync filter is covered by
* SHAMapSyncFilter::getNode()'s postcondition.
*
* @param rawNode the serialized node, including its 4-byte hash prefix.
* @param hash the hash to adopt.
* @return the node.
*/
[[nodiscard]] static SHAMapTreeNodePtr
makeFromPrefix(Slice rawNode, SHAMapHash const& hash);
static SHAMapTreeNodePtr

View File

@@ -18,8 +18,6 @@
#include <xrpl/tx/invariants/SponsorshipInvariant.h>
#include <xrpl/tx/invariants/VaultInvariant.h>
#include <boost/multiprecision/cpp_int.hpp>
#include <cstdint>
#include <set>
#include <string>
@@ -141,7 +139,7 @@ public:
*/
class XRPNotCreated
{
boost::multiprecision::int128_t drops_ = 0;
std::int64_t drops_ = 0;
public:
void

View File

@@ -42,14 +42,6 @@ class ValidMPTIssuance
*/
bool referenceHoldingMutated_ = false;
/**
* Flags cleared on an existing MPTokenIssuance, except lsfMPTLocked,
* which tfMPTUnlock clears legitimately. Every other issuance flag is
* fixed at creation or set-once via MPTokenIssuanceSet, so any bit
* accumulated here is a bug. Enforced post-fixCleanup3_5_0.
*/
std::uint32_t issuanceFlagsCleared_ = 0;
/**
* MPTokens and RippleStates deleted during apply. finalize() checks each
* holder's AccountRoot to detect vault pseudo-account holdings deleted

View File

@@ -1,8 +1,6 @@
#pragma once
#include <xrpl/basics/MathUtilities.h>
#include <xrpl/basics/base_uint.h>
#include <xrpl/basics/contract.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/beast/utility/Zero.h>
#include <xrpl/protocol/AccountID.h>
@@ -25,7 +23,6 @@
#include <ostream>
#include <stdexcept>
#include <string>
#include <type_traits>
#include <utility>
#include <vector>
@@ -505,46 +502,6 @@ public:
};
/** @endcond */
/** @cond INTERNAL */
template <class T>
[[nodiscard]] std::optional<T>
checkedStepAddOpt(T const& lhs, T const& rhs)
{
if constexpr (std::is_same_v<T, XRPAmount>)
{
if (auto const r = checkedAdd(lhs.drops(), rhs.drops()))
return XRPAmount{*r};
return std::nullopt;
}
else if constexpr (std::is_same_v<T, MPTAmount>)
{
if (auto const r = checkedAdd(lhs.value(), rhs.value()))
return MPTAmount{*r};
return std::nullopt;
}
else if constexpr (std::is_same_v<T, IOUAmount>)
{
// IOUAmount is Number-backed and throws on overflow.
return lhs + rhs;
}
else
{
// A new amount type must decide explicitly how to add; do not fall back
// to an unchecked add.
static_assert(sizeof(T) == 0, "checkedStepAddOpt: unsupported amount type");
}
}
template <class T>
[[nodiscard]] T
checkedStepAdd(T const& lhs, T const& rhs)
{
if (auto const r = checkedStepAddOpt(lhs, rhs))
return *r;
Throw<FlowException>(tecPATH_DRY);
}
/** @endcond */
/** @cond INTERNAL */
// Check equal with tolerance
bool

View File

@@ -31,6 +31,7 @@
#include <cstdint>
#include <iterator>
#include <memory>
#include <numeric>
#include <optional>
#include <tuple>
#include <type_traits>
@@ -407,7 +408,7 @@ limitOut(
if (!qf || qf->isConst())
return remainingOut;
auto const out = [&] {
auto const out = [&]() {
auto const out = qf->outFromAvgQ(limitQuality);
if (!out)
return remainingOut;
@@ -645,21 +646,11 @@ flow(
boost::container::flat_multiset<TOutAmt> savedOuts;
savedOuts.reserve(maxTries);
// Returns std::nullopt if the aggregate overflows; callers treat that as a
// dry path.
auto sum = [](auto const& col) -> std::optional<std::decay_t<decltype(*col.begin())>> {
auto sum = [](auto const& col) {
using TResult = std::decay_t<decltype(*col.begin())>;
if (col.empty())
return TResult{beast::kZero};
TResult total = *col.begin();
for (auto it = col.begin() + 1; it != col.end(); ++it)
{
auto const next = checkedStepAddOpt(total, *it);
if (!next)
return std::nullopt;
total = *next;
}
return total;
return std::accumulate(col.begin() + 1, col.end(), *col.begin());
};
// These offers only need to be removed if the payment is not
@@ -679,7 +670,7 @@ flow(
ammContext.setMultiPath(activeStrands.size() > 1);
// Limit only if one strand and limitQuality
auto const limitRemainingOut = [&] {
auto const limitRemainingOut = [&]() {
if (activeStrands.size() == 1 && limitQuality)
{
if (auto const strand = activeStrands.get(0))
@@ -758,17 +749,9 @@ flow(
{
savedIns.insert(best->in);
savedOuts.insert(best->out);
auto const sumOut = sum(savedOuts);
if (!sumOut)
return {tecPATH_DRY, std::move(ofrsToRmOnFail)};
remainingOut = outReq - *sumOut;
remainingOut = outReq - sum(savedOuts);
if (sendMax)
{
auto const sumIn = sum(savedIns);
if (!sumIn)
return {tecPATH_DRY, std::move(ofrsToRmOnFail)};
remainingIn = *sendMax - *sumIn;
}
remainingIn = *sendMax - sum(savedIns);
if (flowDebugInfo)
{
@@ -803,12 +786,8 @@ flow(
break;
}
auto const actualOutOpt = sum(savedOuts);
auto const actualInOpt = sum(savedIns);
if (!actualOutOpt || !actualInOpt)
return {tecPATH_DRY, std::move(ofrsToRmOnFail)};
auto const actualOut = *actualOutOpt;
auto const actualIn = *actualInOpt;
auto const actualOut = sum(savedOuts);
auto const actualIn = sum(savedIns);
JLOG(j.trace()) << "Total flow: in: " << to_string(actualIn)
<< " out: " << to_string(actualOut);

Some files were not shown because too many files have changed in this diff Show More