giveSet() resets the map entry's acquire pointer unconditionally once a
set arrives, so any reply for that hash arriving after completion takes
gotData()'s ta == nullptr branch - charged outright, since
takeNodesLocked() is never reached to apply the late-reply allowance. The
tests for that allowance call acquire->takeNodes() directly, bypassing
gotData()/giveSet() entirely, so none of them exercise this: the real
production entry point for peer replies never reaches the allowance at
all.
Only reset the entry's acquire pointer when something other than the
acquisition itself supplied the set: a set arriving some other way still
cancels an acquisition genuinely in flight, but the acquisition completing
on its own is not that. Keeping it alive until newRound() sweeps the entry
lets a late reply for this hash still reach getAcquire() and, through it,
takeNodesLocked()'s allowance.
Addresses Copilot review feedback on PR #8093, split out into its own
branch: unlike the late-reply allowance bound in branch 14, this gap is
unchanged pre-existing behavior, not something this stack makes worse, so
there is no urgency tying it to that PR's release.
trigger() walks a ledger's state map with mtx_ released (the AS-node
getMissingNodes() call), so it can invalidate the map while a different
packet is still in flight in receiveNode(). SHAMap::addKnownNode() then
reports every node in that packet a duplicate rather than invalid, so
isSynching() alone cannot tell "this packet just finished the map" apart
from "some other packet already broke it" - the acquisition can end up
reporting itself satisfied on a map that is actually invalid, retrying a
hash no peer can ever complete instead of failing fast.
Check map.isValid() independently right after the locked receive, before
isSynching() can misread the verdict, and fail without charging the
packet that lost the race - it did nothing wrong.
Addresses Copilot review feedback on PR #8093, split out into its own
branch: this bug predates the stack and is not made worse by it, so there
is no urgency tying it to that PR's release.
A node that leaves a map invalid proves the hash being chased cannot
belong to any valid tree (see SHAMap::addKnownNode), so no peer can ever
complete it. TransactionAcquire::takeNodesLocked() and
InboundLedger::receiveNode() now fail the acquisition there instead of
retrying until the timeout chain runs out, discarding the whole batch: the
nodes hooked in ahead of the bad one belong to a tree that cannot exist.
stillNeed() and InboundTransactions::getSet() refuse to revive or refresh
such an acquisition, so a dead entry is swept rather than held open.
Charging happens under the same lock that reaches the verdict:
kFeeMalformedData for a node that invalidates the map, kFeeInvalidData for
data that is merely wrong.
A reply arriving after the set is settled is free once per peer the
acquisition actually asked - trigger() can send a targeted request to an
unsolicited sender directly, not only to peers addPeers() selected, so
requestedPeers_ tracks every peer sent a request either way. The allowance
is keyed by peer identity (lateReplyGranted_), not a shared count, so one
peer replaying its own already-accepted reply cannot exhaust the pass a
different, honest peer is still owed. Past the allowance, a reply is a
replay and costs kFeeUselessData.
InboundLedger::getLedger() now reports nothing once the acquisition has
failed, since a failed acquisition can still hold a partially built
ledger that must never be used. The guard sits at the one accessor rather
than relying on every caller to check isFailed() first; the pointer itself
is kept, since getJson() still reports on the partial maps of a failed
acquire.
TransactionAcquire::takeNodes() now accumulates one SHAMapAddNode across
the whole batch, so a packet ending on one bad node still counts the nodes
hooked in ahead of it, matching InboundLedger::receiveNode(). The body
moves to takeNodesLocked(); takeNodes() becomes a thin wrapper that
records progress once, on the single exit, since several inner exits stop
the batch early.
Progress turns on the batch being useful rather than merely good, so an
all-duplicate batch - an honest second responder to trigger()'s fan-out -
records none but still isn't charged.
TransactionAcquire::stillNeed() now restarts the retry timer whenever it
revives an acquisition, so a revived object resumes asking instead of
waiting for a peer to send data unprompted; expires_after() cancels any
pending wait, so this can't leave two timer chains running. It still
returns early when there's nothing to revive, so a running acquisition
keeps the wait it already has.
Tested by driving a real timeout chain to failure - cancel() alone never
arms a timer, so reviving from it wouldn't prove a timed-out chain
restarts - and confirming stillNeed() causes a request to go out again.
Addresses Copilot review feedback on PR #8090.
Ledger(LedgerHeader const&, Rules, Family&) now starts at immutable_
false: its maps are constructed Synching and filled in afterward by an
acquisition or a replay, so the ledger is only settled by setImmutable()
once both maps are sound. mapHashesFromHeader_ records that this
constructor's hashes are input rather than derived, so setImmutable()
leaves them alone - deriving them from the maps would relabel a map that
fell short of its target instead of refusing it.
isComplete() is read without mtx_, so the flag must not be published
before the ledger it describes is settled - a second thread could
otherwise take a still-mid-sync ledger, and a mutable ledger reaching
LedgerHistory::insert() or LedgerMaster::switchLCL() calls logicError().
done() now owns the publication: it settles the ledger and only then sets
complete_, while trigger()/receiveNode() just set have-flags and leave the
verdict to done().
A walk hands back a bare list of hashes, so an empty result doesn't
distinguish a satisfied map from one the walk abandoned.
InboundLedger::hasInvalidMap() now reports the difference.
Three places that read emptiness as "nothing left to fetch" ask it first:
tryDB(), since its two walks set haveState_/haveTransactions_
independently and one map can be abandoned while the other is merely
incomplete; trigger()'s aggressive-retry branch, since the
getNeededHashes() walk it just ran can reach the verdict itself; and
trigger()'s state-map walk, the one walk that runs with mtx_ released.
That last one is asked outside the guard that re-reads the flags after
re-locking, since the verdict is about the map rather than about the
round: another thread can report the ledger complete while the lock is
released, and the guard would then drop the verdict, leaving a ledger
reported complete whose map cannot be the one the header names. The claim
is withdrawn alongside the failure there for that reason.
A walk that reaches a position only a leaf may occupy now marks the map
Invalid and abandons the descent instead of continuing:
SHAMapNodeID::getChildNodeID() throws past kLeafDepth, uncaught, all the
way to std::terminate(). It's reachable without going through
addKnownNode() at all - InboundLedgers::gotStaleData() stores any
parseable node from an unsolicited liAS_NODE reply into the fetch pack by
its own hash with no relatedness check - making this a conditioned remote
denial of service, not just a single bad packet.
As in addKnownNode(), the depth check runs before the full-below cache
lookup, for the same cache-doesn't-cover-depth reason. Callers must
re-check isValid() before reading an empty result as nothing left to
fetch, which getMissingNodes()'s docstring now says.
SHAMap::setImmutable() now returns [[nodiscard]] bool and refuses a map
already proven impossible. Every state change goes through trySetState(),
the only writer of state_ past construction, so the order between the
states is stated once: its compare-exchange can't leave Invalid however it
interleaves with another thread's, and setInvalid() stores through the same
funnel rather than behind its back. Invalid is stored unconditionally
there, since only the map itself reaches that verdict and a walk that
reaches it has to win against a thread settling the map; refusing to
overwrite Immutable would leave a map proven impossible reporting itself
sound, which is what nothing downstream could recover from.
Ledger::setImmutable()/setAccepted() do the same one level up, checking
mapsValid() before touching anything and settling both maps independently
so neither is left mid-sync because the other refused. The map hashes are
read before the maps are settled, since getHash() can unshare a dirty
tree, but written to the header only once both maps have made it. A walk
that invalidates a map in between therefore leaves the header describing
what the ledger was built from rather than a map that has since been
abandoned.
Every call site now branches on the result. The two genesis paths and
buildLedgerImpl() call logicError(), since consensus can't tolerate an
invalid ledger; the load paths return early instead; InboundLedger and
TransactionAcquire withdraw complete_ alongside the failure, since for
them a refusal is an outcome a peer can produce. A test helper that cannot
reach BEAST_EXPECT throws instead, so a refusal cannot hand a broken
ledger to the assertions below.
SHAMap::addKnownNode() now reports invalid() for the two node shapes it
refuses to hook in: an inner node at kLeafDepth (a depth only a leaf may
occupy) and a node whose ID doesn't match where the descent stopped. Both
already read as bad data to callers, so neither counted as progress
before. No peer can satisfy a hash that reaches either shape, so retrying
is futile; the charge is a deterrent rather than a control, since the same
node can reach a map through a fetch pack or unsolicited object reply with
no peer to charge.
The depth check runs before the full-below cache lookup, since that cache
is keyed by hash (which doesn't cover depth) and shared across every map
in the family - checking depth first keeps the verdict independent of what
an unrelated map cached.
Background ledger acquisition reads and writes SHAMap::state_, ::full_,
::ledgerSeq_, and SHAMapInnerNode::fullBelowGen_ concurrently with the
thread driving it, so all four are now std::atomic. finishFetch() withdraws
full_ with an exchange behind a relaxed load, so exactly one reader thread
reports a gap; ledgerSeq_ stays relaxed both ways since it's only a
nodestore lookup hint.
Ledger::setFull() sets each map's sequence before its full flag, so the
release/exchange ordering makes the sequence visible to whichever thread's
exchange wins the gap report.
tryDB() can decide an acquisition can never succeed (a header hash/sequence
mismatch, or a zero account hash) without ever calling done(), so nothing
signals whatever is waiting, and logFailure() never records the hash in
recentFailures_ - the next round asks for the same doomed ledger again.
init() and trigger() now call done() on that path too, matching
checkLocal(), which already did.
DeepChain (src/tests/libxrpl/shamap/DeepChain.h) builds node chains for both
acquisition suites: fabricated chains that run to SHAMap::kLeafDepth, which
no valid tree can hold, and toLeaf() chains that complete an acquisition.
AcquireTestHelpers.h adds ChargeRecordingPeer, RequestCountingPeerSet
(deduping by tracked id like the real PeerSetImpl), packetFor(), waitFor(),
and tallyIs(), so both suites can drive an acquisition through its real
gotData() dispatch instead of reproducing it.
TransactionAcquire and InboundLedger drop final and take a defaulted
retryInterval, so tests can run a whole timeout chain in a fraction of a
second; nothing in production passes one.
AcquireTestHelpers.h is the first src/test file to include one from
src/tests, so levelization records a new test.app > tests.libxrpl edge in
ordering.txt. No loop is introduced: nothing under src/tests includes
src/test.
Addresses Copilot review feedback on PR #8081.
SHAMapAddNode gains getBad() and getDuplicate() beside getGood(), so a
verdict can be read as counts instead of just a log string. get()'s wording
is pinned by src/tests/libxrpl/shamap/SHAMapAddNode.cpp, the one place that
depends on it.
