Compare commits

...
Author SHA1 Message Date
Nicholas Dudfield 31044a8929 fix: replace detached threads in sleepFor(), fix tx-priority leak and rescan
- Replace std::thread::detach() in Coro::sleepFor() with
  boost::asio::steady_timer on the existing io_service thread pool.
  Previously each 50ms poll spawned a new detached thread (~600 per
  missing node over 30s timeout).

- Clear expired TX-priority ranges in InboundLedgers::sweep() for
  sequences at or below the current validated ledger, preventing
  unbounded growth of txPriorityRange_.

- Track lastCheckedSeq in submit_and_wait to only scan new validated
  ledgers each poll iteration, eliminating O(n*polls) rescan overhead.
2026-02-26 17:11:06 +07:00
Nicholas Dudfield ab4fb6501d chore: update levelization ordering 2026-02-26 15:33:27 +07:00
Nicholas Dudfield f629836482 chore: format RCLValidations.cpp and SubmitAndWait.cpp 2026-02-26 15:29:16 +07:00
Nicholas Dudfield 1f0e1e0ecf Merge remote-tracking branch 'origin/dev' into partial-sync-mode 2026-02-26 14:11:23 +07:00
tequ 8cfee6c8a3 Merge fixAMMClawbackRounding amendment into featureAMMClawback amendment 2026-02-25 19:07:45 +10:00
yinyiqian1 8673599d2b fixAMMClawbackRounding: adjust last holder's LPToken balance (#5513)
Due to rounding, the LPTokenBalance of the last LP might not match the LP's trustline balance. This was fixed for `AMMWithdraw` in `fixAMMv1_1` by adjusting the LPTokenBalance to be the same as the trustline balance. Since `AMMClawback` is also performing a withdrawal, we need to adjust LPTokenBalance as well in `AMMClawback.`

This change includes:
1. Refactored `verifyAndAdjustLPTokenBalance` function in `AMMUtils`, which both`AMMWithdraw` and `AMMClawback` call to adjust LPTokenBalance.
2. Added the unit test `testLastHolderLPTokenBalance` to test the scenario.
3. Modify the existing unit tests for `fixAMMClawbackRounding`.
2026-02-25 19:07:45 +10:00
tequ ec65e622aa Merge fixAMMv1_3 amendment into featureAMM amendment 2026-02-25 16:20:43 +10:00
Gregory Tsipenyuk 65837f49e1 fix: Add AMMv1_3 amendment (#5203)
* Add AMM bid/create/deposit/swap/withdraw/vote invariants:
  - Deposit, Withdrawal invariants: `sqrt(asset1Balance * asset2Balance) >= LPTokens`.
  - Bid: `sqrt(asset1Balance * asset2Balance) > LPTokens` and the pool balances don't change.
  - Create: `sqrt(asset1Balance * assetBalance2) == LPTokens`.
  - Swap: `asset1BalanceAfter * asset2BalanceAfter >= asset1BalanceBefore * asset2BalanceBefore`
     and `LPTokens` don't change.
  - Vote: `LPTokens` and pool balances don't change.
  - All AMM and swap transactions: amounts and tokens are greater than zero, except on withdrawal if all tokens
    are withdrawn.
* Add AMM deposit and withdraw rounding to ensure AMM invariant:
  - On deposit, tokens out are rounded downward and deposit amount is rounded upward.
  - On withdrawal, tokens in are rounded upward and withdrawal amount is rounded downward.
* Add Order Book Offer invariant to verify consumed amounts. Consumed amounts are less than the offer.
* Fix Bid validation. `AuthAccount` can't have duplicate accounts or the submitter account.
2026-02-25 16:20:43 +10:00
Nicholas Dudfield 44881c6255 merge: sync with origin/dev (rippled 2.4.0 repo restructure)
Merges 299 commits from origin/dev including the major rippled 2.4.0
repository restructure (src/ripple/ -> src/xrpld/ + include/xrpl/,
Builds/CMake/ -> cmake/, shards removal).

Resolved conflicts in 10 files, preserving all partial sync additions:
- Coro.ipp: kept our #include <thread> with new paths
- SHAMap.cpp: kept LocalValue.h, JobQueue.h, chrono includes
- NodeFamily.cpp: kept enhanced logging and addPriorityNode call
- InboundLedgers.cpp: kept RangeSet.h include with new paths
- NetworkOPs.cpp: updated beginConsensus to include clog param
- RPCHelpers.cpp: kept InboundLedgers.h, LocalValue.h includes
  and commented-out sync validation (updated for removed reporting())
- Handler.cpp: added submit_and_wait to new alphabetical handler list
- Handler.h: kept @@markers, updated to NO_CONDITION check
- SHAMapInnerNode.h: kept @@markers, added getBranchCount()
- ordering.txt: accepted upstream restructured dependency graph
2026-02-24 16:46:42 +07:00
RichardAH e5b21f026e Merge pull request #692 from Xahau/sync-2.4.0-rebased
Sync: Ripple(d) 2.4.0
2026-02-24 16:07:09 +10:00
Nicholas Dudfield 95bc06d4ea fix: remove debug log in onTimer() that causes use-after-free crash 2026-02-20 17:45:01 +07:00
Nicholas Dudfield 2ce3b3f1fd fix: make partial sync poll-wait opt-in via partialSyncWait flag
Previously finishFetch() entered the poll-wait loop for any coroutine
context, causing unit tests to spin for 30s on missing nodes with no
network to deliver them. Now requires explicit setPartialSyncWait(true)
from partial sync code paths (RPCHelpers, SubmitAndWait).
2026-02-20 16:54:06 +07:00
Nicholas Dudfield fe56844871 chore: update levelization ordering 2026-02-20 11:17:33 +07:00
Nicholas Dudfield c351266134 style: fix clang-format for Coro.ipp 2026-02-20 11:13:18 +07:00
Nicholas Dudfield ef01ac6977 fix: remove leftover conflict marker text in RippledCore.cmake 2026-02-20 11:12:02 +07:00
Nicholas Dudfield 11cce2eac8 Merge remote-tracking branch 'origin/dev' into partial-sync-mode
# Conflicts:
#	Builds/CMake/RippledCore.cmake
#	src/ripple/app/misc/impl/Manifest.cpp
2026-02-20 11:05:29 +07:00
Nicholas Dudfield b3c4e56c26 feat: check LedgerMaster in submit_and_wait for synced mode
When the node is synced and receiving transactions via gossip,
ledgers are built locally and won't be in InboundLedgers. Now
checks both:
- InboundLedgers (partial sync mode - ledgers from peers)
- LedgerMaster (synced mode - ledgers built from gossip)
2025-12-02 08:21:37 +07:00
Nicholas Dudfield 9ddf649e2a feat: add ledger range-based TX priority for faster tx detection
Simplify TX priority mechanism using RangeSet instead of per-TX hash
tracking. When submit_and_wait is called, it registers a ledger range
where TX nodes should be fetched before state nodes.

Key changes:
- Add prioritizeTxForLedgers(start, end) and isTxPrioritized(seq)
  to InboundLedgers using RangeSet<uint32_t>
- InboundLedger::trigger() checks range to decide TX-before-state order
- Remove complex per-TX hash tracking that couldn't help due to
  Merkle tree structure (need parent hashes to request children)
- Format CMake and source files
2025-12-01 15:31:15 +07:00
Nicholas Dudfield aeb2888fe9 feat: add submit_and_wait RPC for partial sync mode
Adds a new RPC handler that submits transactions and waits for
validated results, designed for nodes still syncing:

- Broadcasts raw tx to network without local state validation
- Indexes tx hashes from incoming txMap leaf nodes for fast lookup
- Polls for tx in partial ledgers, then waits for validation quorum
- Only returns when numTrustedForLedger >= quorum (truly validated)

Supporting changes:
- Add hasTx()/knownTxHashes_ to InboundLedger for tx tracking
- Add findTxLedger() to InboundLedgers to search across ledgers
- Add broadcastRawTransaction() to NetworkOPs for blind relay
- Add coroutine-local fetchTimeout to LocalValue.h
- SHAMap::finishFetch() now uses configurable timeout
2025-12-01 10:53:54 +07:00
Nicholas Dudfield 8263f39e3e feat: add Coro::sleepFor() for non-blocking poll-waits
Yields the coroutine and schedules resume after delay, freeing up
the job queue thread instead of blocking it with sleep_for().

Updated SHAMap::finishFetch() and RPCHelpers getLedger() to use
coro->sleepFor() for partial sync poll-wait loops.
2025-11-30 12:21:00 +07:00
Nicholas Dudfield dc5ec93207 feat: trigger early quorum calculation when VL is fetched
Call updateTrusted() immediately when all publisher lists become
available in applyListsAndBroadcast(), rather than waiting for
beginConsensus(). This allows validations to be trusted within
milliseconds of VL fetch instead of waiting 14+ seconds for
consensus to start.

Also adds debugging logs:
- PartialSync journal: untrusted validations during startup
- PartialSync journal: checkAccept quorum details
- ValidatorSite journal: VL fetch timing
2025-11-30 11:56:22 +07:00
Nicholas Dudfield 5d85d2df4b feat: add priority node fetching and network-observed ledger tracking
Partial sync mode improvements for faster RPC queries during sync:

- Track network-observed ledger from any validation (not just trusted)
  to allow queries before trusted validators are configured
- Add priority node fetching: queries can request specific nodes be
  fetched immediately via addPriorityNode/addPriorityHash
- Store state/transaction nodes directly to node store (not fetch pack)
  so partial sync queries find them immediately
- Add poll-wait loops in RPCHelpers for ledger header acquisition
- Replace postAndYield with sleep_for in SHAMap finishFetch
- Implement linear backoff for re-requests (50ms increments, max 2s)
2025-11-30 11:29:26 +07:00
Nicholas Dudfield c20c948183 feat: partial sync mode proof of concept
Allows RPC handlers to query ledgers that are still being acquired,
enabling faster node startup for read queries.

Key changes:
- Add coroutine detection via getCurrentCoroPtr() in LocalValue.h
- Add postAndYield() to Coro for safe poll-wait synchronization
- Modify SHAMap::finishFetch() to poll-wait for missing nodes when
  in coroutine context (30s timeout, re-requests missing nodes)
- Add getPartialLedger() to InboundLedgers for accessing incomplete
  ledgers that have headers
- Add getLastValidatedLedger() to LedgerMaster to get validated
  ledger hash even when not fully synced
- Update RPCHelpers getLedger() to fall back to partial ledgers
- Fix Manifest seq_++ bug for new manifest entries
- Add zero hash guard in NetworkOPs::checkLastClosedLedger()

Note: This is a proof of concept. Production use would require
fetch prioritization to make queries fast enough to be practical.
2025-11-30 09:45:32 +07:00
58 changed files with 3696 additions and 702 deletions

No files matched your search

+1
View File
@@ -203,5 +203,6 @@ xrpld.rpc > xrpl.protocol
xrpld.rpc > xrpl.resource
xrpld.rpc > xrpl.server
xrpld.shamap > xrpl.basics
xrpld.shamap > xrpld.core
xrpld.shamap > xrpld.nodestore
xrpld.shamap > xrpl.protocol
+62
View File
@@ -21,6 +21,7 @@
#define RIPPLE_BASICS_LOCALVALUE_H_INCLUDED
#include <boost/thread/tss.hpp>
#include <chrono>
#include <memory>
#include <unordered_map>
@@ -33,6 +34,16 @@ struct LocalValues
explicit LocalValues() = default;
bool onCoro = true;
void* coroPtr = nullptr; // Pointer to owning JobQueue::Coro (if any)
// When true, SHAMap::finishFetch() will poll-wait for missing nodes
// instead of returning empty. Only set by partial sync code paths.
bool partialSyncWait = false;
// Configurable timeout for SHAMap node fetching during partial sync.
// Zero means use the default (30s). RPC handlers can set this to
// customize poll-wait behavior.
std::chrono::milliseconds fetchTimeout{0};
struct BasicValue
{
@@ -127,6 +138,57 @@ LocalValue<T>::operator*()
.emplace(this, std::make_unique<detail::LocalValues::Value<T>>(t_))
.first->second->get());
}
// Returns pointer to current coroutine if running inside one, nullptr otherwise
inline void*
getCurrentCoroPtr()
{
auto lvs = detail::getLocalValues().get();
if (lvs && lvs->onCoro)
return lvs->coroPtr;
return nullptr;
}
// Check if partial sync wait is enabled for the current coroutine context.
inline bool
isPartialSyncWaitEnabled()
{
auto lvs = detail::getLocalValues().get();
if (lvs && lvs->onCoro)
return lvs->partialSyncWait;
return false;
}
// Enable/disable partial sync wait for the current coroutine context.
inline void
setPartialSyncWait(bool enabled)
{
auto lvs = detail::getLocalValues().get();
if (lvs && lvs->onCoro)
lvs->partialSyncWait = enabled;
}
// Get the configured fetch timeout for current coroutine context.
// Returns 0ms if not in a coroutine or no custom timeout set.
inline std::chrono::milliseconds
getCoroFetchTimeout()
{
auto lvs = detail::getLocalValues().get();
if (lvs && lvs->onCoro)
return lvs->fetchTimeout;
return std::chrono::milliseconds{0};
}
// Set the fetch timeout for the current coroutine context.
// Only works if called from within a coroutine.
inline void
setCoroFetchTimeout(std::chrono::milliseconds timeout)
{
auto lvs = detail::getLocalValues().get();
if (lvs && lvs->onCoro)
lvs->fetchTimeout = timeout;
}
} // namespace ripple
#endif
+2
View File
@@ -61,10 +61,12 @@ enum error_code_i {
rpcAMENDMENT_BLOCKED = 14,
// Networking
//@@start network-error-codes
rpcNO_CLOSED = 15,
rpcNO_CURRENT = 16,
rpcNO_NETWORK = 17,
rpcNOT_SYNCED = 18,
//@@end network-error-codes
// Ledger state
rpcACT_NOT_FOUND = 19,
+6
View File
@@ -97,6 +97,12 @@ public:
static IOUAmount
minPositiveAmount();
friend std::ostream&
operator<<(std::ostream& os, IOUAmount const& x)
{
return os << to_string(x);
}
};
inline IOUAmount::IOUAmount(beast::Zero)
+3
View File
@@ -28,6 +28,9 @@
namespace ripple {
bool
isFeatureEnabled(uint256 const& feature);
class DigestAwareReadView;
/** Rules controlling protocol behavior. */
+2
View File
@@ -89,9 +89,11 @@ constexpr static ErrorInfo unorderedErrorInfos[]{
{rpcNOT_SUPPORTED, "notSupported", "Operation not supported.", 501},
{rpcNO_CLOSED, "noClosed", "Closed ledger is unavailable.", 503},
{rpcNO_CURRENT, "noCurrent", "Current ledger is unavailable.", 503},
//@@start network-error-messages
{rpcNOT_SYNCED, "notSynced", "Not synced to the network.", 503},
{rpcNO_EVENTS, "noEvents", "Current transport does not support events.", 405},
{rpcNO_NETWORK, "noNetwork", "Not synced to the network.", 503},
//@@end network-error-messages
{rpcWRONG_NETWORK, "wrongNetwork", "Wrong network.", 503},
{rpcNO_PERMISSION, "noPermission", "You don't have permission for this command.", 401},
{rpcNO_PF_REQUEST, "noPathRequest", "No pathfinding request in progress.", 404},
+8
View File
@@ -153,4 +153,12 @@ Rules::operator!=(Rules const& other) const
{
return !(*this == other);
}
bool
isFeatureEnabled(uint256 const& feature)
{
auto const& rules = getCurrentTransactionRules();
return rules && rules->enabled(feature);
}
} // namespace ripple
File diff suppressed because it is too large. Load diff
+841 -241
View File
File diff suppressed because it is too large. Load diff
+28
View File
@@ -126,6 +126,34 @@ public:
return {};
}
virtual std::shared_ptr<Ledger const>
getPartialLedger(uint256 const& hash) override
{
return {};
}
virtual std::optional<uint256>
findTxLedger(uint256 const& txHash) override
{
return std::nullopt;
}
virtual void
addPriorityNode(std::uint32_t ledgerSeq, uint256 const& nodeHash) override
{
}
virtual void
prioritizeTxForLedgers(std::uint32_t start, std::uint32_t end) override
{
}
virtual bool
isTxPrioritized(std::uint32_t seq) const override
{
return false;
}
virtual bool
gotLedgerData(
LedgerHash const& ledgerHash,
+10 -1
View File
@@ -125,7 +125,6 @@ class AMM
STAmount const asset1_;
STAmount const asset2_;
uint256 const ammID_;
IOUAmount const initialLPTokens_;
bool log_;
bool doClose_;
// Predict next purchase price
@@ -138,6 +137,7 @@ class AMM
std::uint32_t const fee_;
AccountID const ammAccount_;
Issue const lptIssue_;
IOUAmount const initialLPTokens_;
public:
AMM(Env& env,
@@ -194,6 +194,12 @@ public:
Issue const& issue2,
std::optional<AccountID> const& account = std::nullopt) const;
std::tuple<STAmount, STAmount, STAmount>
balances(std::optional<AccountID> const& account = std::nullopt) const
{
return balances(asset1_.get<Issue>(), asset2_.get<Issue>(), account);
}
[[nodiscard]] bool
expectLPTokens(AccountID const& account, IOUAmount const& tokens) const;
@@ -428,6 +434,9 @@ private:
[[nodiscard]] bool
expectAuctionSlot(auto&& cb) const;
IOUAmount
initialTokens();
};
namespace amm {
+14
View File
@@ -33,6 +33,15 @@ class AMM;
enum class Fund { All, Acct, Gw, IOUOnly };
struct TestAMMArg
{
std::optional<std::pair<STAmount, STAmount>> pool = std::nullopt;
std::uint16_t tfee = 0;
std::optional<jtx::ter> ter = std::nullopt;
std::vector<FeatureBitset> features = {supported_amendments()};
bool noLog = false;
};
void
fund(
jtx::Env& env,
@@ -85,6 +94,11 @@ protected:
std::uint16_t tfee = 0,
std::optional<jtx::ter> const& ter = std::nullopt,
std::vector<FeatureBitset> const& features = {supported_amendments()});
void
testAMM(
std::function<void(jtx::AMM&, jtx::Env&)>&& cb,
TestAMMArg const& arg);
};
class AMMTest : public jtx::AMMTestBase
+6
View File
@@ -646,6 +646,12 @@ public:
void
disableFeature(uint256 const feature);
bool
enabled(uint256 feature) const
{
return current()->rules().enabled(feature);
}
private:
void
fund(bool setDefaultRipple, STAmount const& amount, Account const& account);
+6 -7
View File
@@ -18,8 +18,9 @@
//==============================================================================
#include <test/jtx/AMM.h>
#include <test/jtx/Env.h>
#include <xrpld/app/misc/AMMHelpers.h>
#include <xrpld/app/misc/AMMUtils.h>
#include <xrpld/rpc/detail/RPCHelpers.h>
#include <xrpl/protocol/AMMCore.h>
@@ -38,12 +39,10 @@ number(STAmount const& a)
return a;
}
static IOUAmount
initialTokens(STAmount const& asset1, STAmount const& asset2)
IOUAmount
AMM::initialTokens()
{
auto const product = number(asset1) * number(asset2);
return (IOUAmount)(product.mantissa() >= 0 ? root2(product)
: root2(-product));
return getLPTokensBalance();
}
AMM::AMM(
@@ -64,7 +63,6 @@ AMM::AMM(
, asset1_(asset1)
, asset2_(asset2)
, ammID_(keylet::amm(asset1_.issue(), asset2_.issue()).key)
, initialLPTokens_(initialTokens(asset1, asset2))
, log_(log)
, doClose_(close)
, lastPurchasePrice_(0)
@@ -77,6 +75,7 @@ AMM::AMM(
asset1_.issue().currency,
asset2_.issue().currency,
ammAccount_))
, initialLPTokens_(initialTokens())
{
}
+28 -6
View File
@@ -17,9 +17,9 @@
*/
//==============================================================================
#include <test/jtx/AMMTest.h>
#include <test/jtx/AMM.h>
#include <test/jtx/AMMTest.h>
#include <test/jtx/CaptureLogs.h>
#include <test/jtx/Env.h>
#include <test/jtx/pay.h>
#include <xrpld/rpc/RPCHandler.h>
@@ -104,15 +104,37 @@ AMMTestBase::testAMM(
std::uint16_t tfee,
std::optional<jtx::ter> const& ter,
std::vector<FeatureBitset> const& vfeatures)
{
testAMM(
std::move(cb),
TestAMMArg{
.pool = pool, .tfee = tfee, .ter = ter, .features = vfeatures});
}
void
AMMTestBase::testAMM(
std::function<void(jtx::AMM&, jtx::Env&)>&& cb,
TestAMMArg const& arg)
{
using namespace jtx;
for (auto const& features : vfeatures)
std::string logs;
for (auto const& features : arg.features)
{
Env env{*this, features};
// Env env{
// *this,
// features,
// arg.noLog ? std::make_unique<CaptureLogs>(&logs) : nullptr};
Env env(
*this,
envconfig(),
features,
nullptr,
beast::severities::kDisabled);
auto const [asset1, asset2] =
pool ? *pool : std::make_pair(XRP(10000), USD(10000));
arg.pool ? *arg.pool : std::make_pair(XRP(10000), USD(10000));
auto tofund = [&](STAmount const& a) -> STAmount {
if (a.native())
{
@@ -142,7 +164,7 @@ AMMTestBase::testAMM(
alice,
asset1,
asset2,
CreateArg{.log = false, .tfee = tfee, .err = ter});
CreateArg{.log = false, .tfee = arg.tfee, .err = arg.ter});
if (BEAST_EXPECT(
ammAlice.expectBalances(asset1, asset2, ammAlice.tokens())))
cb(ammAlice, env);
+92 -81
View File
@@ -203,98 +203,107 @@ public:
}
void
testVoteAndBid()
testVoteAndBid(FeatureBitset features)
{
testcase("Vote and Bid");
using namespace jtx;
testAMM([&](AMM& ammAlice, Env& env) {
BEAST_EXPECT(ammAlice.expectAmmRpcInfo(
XRP(10000), USD(10000), IOUAmount{10000000, 0}));
std::unordered_map<std::string, std::uint16_t> votes;
votes.insert({alice.human(), 0});
for (int i = 0; i < 7; ++i)
{
Account a(std::to_string(i));
votes.insert({a.human(), 50 * (i + 1)});
fund(env, gw, {a}, {USD(10000)}, Fund::Acct);
ammAlice.deposit(a, 10000000);
ammAlice.vote(a, 50 * (i + 1));
}
BEAST_EXPECT(ammAlice.expectTradingFee(175));
Account ed("ed");
Account bill("bill");
env.fund(XRP(1000), bob, ed, bill);
env(ammAlice.bid(
{.bidMin = 100, .authAccounts = {carol, bob, ed, bill}}));
BEAST_EXPECT(ammAlice.expectAmmRpcInfo(
XRP(80000),
USD(80000),
IOUAmount{79994400},
std::nullopt,
std::nullopt,
ammAlice.ammAccount()));
for (auto i = 0; i < 2; ++i)
{
std::unordered_set<std::string> authAccounts = {
carol.human(), bob.human(), ed.human(), bill.human()};
auto const ammInfo = i ? ammAlice.ammRpcInfo()
: ammAlice.ammRpcInfo(
std::nullopt,
std::nullopt,
std::nullopt,
std::nullopt,
ammAlice.ammAccount());
auto const& amm = ammInfo[jss::amm];
try
testAMM(
[&](AMM& ammAlice, Env& env) {
BEAST_EXPECT(ammAlice.expectAmmRpcInfo(
XRP(10000), USD(10000), IOUAmount{10000000, 0}));
std::unordered_map<std::string, std::uint16_t> votes;
votes.insert({alice.human(), 0});
for (int i = 0; i < 7; ++i)
{
// votes
auto const voteSlots = amm[jss::vote_slots];
auto votesCopy = votes;
for (std::uint8_t i = 0; i < 8; ++i)
Account a(std::to_string(i));
votes.insert({a.human(), 50 * (i + 1)});
fund(env, gw, {a}, {USD(10001)}, Fund::Acct);
ammAlice.deposit(a, 10000000);
ammAlice.vote(a, 50 * (i + 1));
}
BEAST_EXPECT(ammAlice.expectTradingFee(175));
Account ed("ed");
Account bill("bill");
env.fund(XRP(1000), bob, ed, bill);
env(ammAlice.bid(
{.bidMin = 100, .authAccounts = {carol, bob, ed, bill}}));
BEAST_EXPECT(ammAlice.expectAmmRpcInfo(
XRPAmount(80000000005),
STAmount{USD, UINT64_C(80'000'00000000005), -11},
IOUAmount{79994400},
std::nullopt,
std::nullopt,
ammAlice.ammAccount()));
for (auto i = 0; i < 2; ++i)
{
std::unordered_set<std::string> authAccounts = {
carol.human(), bob.human(), ed.human(), bill.human()};
auto const ammInfo = i ? ammAlice.ammRpcInfo()
: ammAlice.ammRpcInfo(
std::nullopt,
std::nullopt,
std::nullopt,
std::nullopt,
ammAlice.ammAccount());
auto const& amm = ammInfo[jss::amm];
try
{
if (!BEAST_EXPECT(
votes[voteSlots[i][jss::account].asString()] ==
voteSlots[i][jss::trading_fee].asUInt() &&
voteSlots[i][jss::vote_weight].asUInt() ==
12500))
// votes
auto const voteSlots = amm[jss::vote_slots];
auto votesCopy = votes;
for (std::uint8_t i = 0; i < 8; ++i)
{
if (!BEAST_EXPECT(
votes[voteSlots[i][jss::account]
.asString()] ==
voteSlots[i][jss::trading_fee]
.asUInt() &&
voteSlots[i][jss::vote_weight].asUInt() ==
12500))
return;
votes.erase(voteSlots[i][jss::account].asString());
}
if (!BEAST_EXPECT(votes.empty()))
return;
votes.erase(voteSlots[i][jss::account].asString());
}
if (!BEAST_EXPECT(votes.empty()))
return;
votes = votesCopy;
votes = votesCopy;
// bid
auto const auctionSlot = amm[jss::auction_slot];
for (std::uint8_t i = 0; i < 4; ++i)
{
if (!BEAST_EXPECT(authAccounts.contains(
// bid
auto const auctionSlot = amm[jss::auction_slot];
for (std::uint8_t i = 0; i < 4; ++i)
{
if (!BEAST_EXPECT(authAccounts.contains(
auctionSlot[jss::auth_accounts][i]
[jss::account]
.asString())))
return;
authAccounts.erase(
auctionSlot[jss::auth_accounts][i][jss::account]
.asString())))
.asString());
}
if (!BEAST_EXPECT(authAccounts.empty()))
return;
authAccounts.erase(
auctionSlot[jss::auth_accounts][i][jss::account]
.asString());
BEAST_EXPECT(
auctionSlot[jss::account].asString() ==
alice.human() &&
auctionSlot[jss::discounted_fee].asUInt() == 17 &&
auctionSlot[jss::price][jss::value].asString() ==
"5600" &&
auctionSlot[jss::price][jss::currency].asString() ==
to_string(ammAlice.lptIssue().currency) &&
auctionSlot[jss::price][jss::issuer].asString() ==
to_string(ammAlice.lptIssue().account));
}
catch (std::exception const& e)
{
fail(e.what(), __FILE__, __LINE__);
}
if (!BEAST_EXPECT(authAccounts.empty()))
return;
BEAST_EXPECT(
auctionSlot[jss::account].asString() == alice.human() &&
auctionSlot[jss::discounted_fee].asUInt() == 17 &&
auctionSlot[jss::price][jss::value].asString() ==
"5600" &&
auctionSlot[jss::price][jss::currency].asString() ==
to_string(ammAlice.lptIssue().currency) &&
auctionSlot[jss::price][jss::issuer].asString() ==
to_string(ammAlice.lptIssue().account));
}
catch (std::exception const& e)
{
fail(e.what(), __FILE__, __LINE__);
}
}
});
},
std::nullopt,
0,
std::nullopt,
{features});
}
void
@@ -337,9 +346,11 @@ public:
void
run() override
{
using namespace jtx;
auto const all = supported_amendments();
testErrors();
testSimpleRpc();
testVoteAndBid();
testVoteAndBid(all);
testFreeze();
testInvalidAmmField();
}
+4 -2
View File
@@ -100,8 +100,10 @@ public:
}
void
missingNodeAcquireBySeq(std::uint32_t refNum, uint256 const& nodeHash)
override
missingNodeAcquireBySeq(
std::uint32_t refNum,
uint256 const& nodeHash,
bool prioritize = false) override
{
Throw<std::runtime_error>("missing node");
}
@@ -192,8 +192,24 @@ handleNewValidation(
auto const outcome =
validations.add(calcNodeID(masterKey.value_or(signingKey)), val);
if (j.has_value())
{
JLOG(j->warn()) << "handleNewValidation: seq=" << seq
<< " hash=" << hash << " trusted=" << val->isTrusted()
<< " outcome="
<< (outcome == ValStatus::current ? "current"
: outcome == ValStatus::stale ? "stale"
: outcome == ValStatus::badSeq ? "badSeq"
: "other");
}
if (outcome == ValStatus::current)
{
// For partial sync: track the network-observed ledger from ANY
// validation (not just trusted). This allows queries before
// trusted validators are fully configured.
app.getLedgerMaster().setNetworkObservedLedger(hash, seq);
if (val->isTrusted())
{
// Was: app.getLedgerMaster().checkAccept(hash, seq);
@@ -213,6 +229,23 @@ handleNewValidation(
app.getLedgerMaster().checkAccept(hash, seq);
}
}
else
{
// Partial sync debug: only log untrusted validations during startup
// (before we have any validated ledger)
auto [lastHash, lastSeq] =
app.getLedgerMaster().getLastValidatedLedger();
if (lastSeq == 0)
{
auto jPartialSync = app.journal("PartialSync");
auto const quorum = app.validators().quorum();
auto const unlSize = app.validators().count();
JLOG(jPartialSync.debug())
<< "validation NOT trusted: seq=" << seq << " hash=" << hash
<< " unlSize=" << unlSize << " quorum=" << quorum
<< " (masterKey=" << (masterKey ? "found" : "none") << ")";
}
}
return;
}
+36
View File
@@ -80,6 +80,13 @@ public:
return mLedger;
}
/** Returns true if we have the ledger header (may still be incomplete). */
bool
hasHeader() const
{
return mHaveHeader;
}
std::uint32_t
getSeq() const
{
@@ -106,6 +113,26 @@ public:
void
runData();
/** Add a node hash to the priority queue for immediate fetching.
Used by partial sync mode to prioritize nodes needed by queries.
*/
void
addPriorityHash(uint256 const& hash);
/** Check if a transaction hash has been seen in this ledger's txMap.
Used by submit_and_wait to find transactions in partial ledgers.
*/
bool
hasTx(uint256 const& txHash) const;
/** Return the count of known transaction hashes (for debugging). */
std::size_t
knownTxCount() const
{
ScopedLockType sl(mtx_);
return knownTxHashes_.size();
}
void
touch()
{
@@ -175,9 +202,11 @@ private:
clock_type::time_point mLastAction;
std::shared_ptr<Ledger> mLedger;
//@@start state-tracking-members
bool mHaveHeader;
bool mHaveState;
bool mHaveTransactions;
//@@end state-tracking-members
bool mSignaled;
bool mByHash;
std::uint32_t mSeq;
@@ -185,6 +214,13 @@ private:
std::set<uint256> mRecentNodes;
// Priority nodes to fetch immediately (for partial sync queries)
std::set<uint256> priorityHashes_;
// Transaction hashes seen in incoming txMap leaf nodes (for
// submit_and_wait)
std::set<uint256> knownTxHashes_;
SHAMapAddNode mStats;
// Data we have received from peers
+40
View File
@@ -23,6 +23,7 @@
#include <xrpld/app/ledger/InboundLedger.h>
#include <xrpl/protocol/RippleLedgerHash.h>
#include <memory>
#include <optional>
namespace ripple {
@@ -56,6 +57,45 @@ public:
virtual std::shared_ptr<InboundLedger>
find(LedgerHash const& hash) = 0;
/** Get a partial ledger (has header but may be incomplete).
Used for partial sync mode - allows RPC queries against
ledgers that are still being acquired.
@return The ledger if header exists and not failed, nullptr otherwise.
*/
virtual std::shared_ptr<Ledger const>
getPartialLedger(uint256 const& hash) = 0;
/** Find which partial ledger contains a transaction.
Used by submit_and_wait to locate transactions as they appear
in incoming ledgers' txMaps.
@param txHash The transaction hash to search for
@return The ledger hash if found, nullopt otherwise
*/
virtual std::optional<uint256>
findTxLedger(uint256 const& txHash) = 0;
/** Add a priority node hash for immediate fetching.
Used by partial sync mode to prioritize specific nodes
needed by queries.
@param ledgerSeq The ledger sequence being acquired
@param nodeHash The specific node hash to prioritize
*/
virtual void
addPriorityNode(std::uint32_t ledgerSeq, uint256 const& nodeHash) = 0;
/** Add a ledger range where TX fetching should be prioritized.
Ledgers in this range will fetch TX nodes BEFORE state nodes.
Used by submit_and_wait to quickly detect transactions.
@param start First ledger sequence (inclusive)
@param end Last ledger sequence (inclusive)
*/
virtual void
prioritizeTxForLedgers(std::uint32_t start, std::uint32_t end) = 0;
/** Check if TX fetching should be prioritized for a ledger sequence. */
virtual bool
isTxPrioritized(std::uint32_t seq) const = 0;
// VFALCO TODO Remove the dependency on the Peer object.
//
virtual bool
+36
View File
@@ -280,6 +280,38 @@ public:
return !mValidLedger.empty();
}
//! Get the hash/seq of the last validated ledger (even if not resident).
std::pair<uint256, LedgerIndex>
getLastValidatedLedger()
{
std::lock_guard lock(m_mutex);
return mLastValidLedger;
}
//! For partial sync: set the network-observed ledger from any validation.
//! This allows queries before trusted validators are fully configured.
void
setNetworkObservedLedger(uint256 const& hash, LedgerIndex seq)
{
std::lock_guard lock(m_mutex);
if (seq > mNetworkObservedLedger.second)
{
JLOG(jPartialSync_.warn())
<< "network-observed ledger updated to seq=" << seq
<< " hash=" << hash;
mNetworkObservedLedger = std::make_pair(hash, seq);
}
}
//! Get the network-observed ledger (from any validations, not just
//! trusted).
std::pair<uint256, LedgerIndex>
getNetworkObservedLedger()
{
std::lock_guard lock(m_mutex);
return mNetworkObservedLedger;
}
// Returns the minimum ledger sequence in SQL database, if any.
std::optional<LedgerIndex>
minSqlSeq();
@@ -329,6 +361,7 @@ private:
Application& app_;
beast::Journal m_journal;
beast::Journal jPartialSync_;
std::recursive_mutex mutable m_mutex;
@@ -350,6 +383,9 @@ private:
// Fully validated ledger, whether or not we have the ledger resident.
std::pair<uint256, LedgerIndex> mLastValidLedger{uint256(), 0};
// Network-observed ledger from any validations (for partial sync).
std::pair<uint256, LedgerIndex> mNetworkObservedLedger{uint256(), 0};
LedgerHistory mLedgerHistory;
CanonicalTXSet mHeldTransactions{uint256()};
+95 -1
View File
@@ -41,6 +41,7 @@ namespace ripple {
using namespace std::chrono_literals;
//@@start tx-fetch-constants
enum {
// Number of peers to start with
peerCountStart = 5
@@ -69,6 +70,7 @@ enum {
,
reqNodes = 12
};
//@@end tx-fetch-constants
// millisecond for each ledger timeout
auto constexpr ledgerAcquireTimeout = 3000ms;
@@ -98,6 +100,8 @@ InboundLedger::InboundLedger(
, mPeerSet(std::move(peerSet))
{
JLOG(journal_.trace()) << "Acquiring ledger " << hash_;
JLOG(app_.journal("TxTrack").warn())
<< "NEW LEDGER seq=" << seq << " hash=" << hash;
touch();
}
@@ -157,6 +161,22 @@ InboundLedger::update(std::uint32_t seq)
touch();
}
void
InboundLedger::addPriorityHash(uint256 const& hash)
{
ScopedLockType sl(mtx_);
priorityHashes_.insert(hash);
JLOG(journal_.debug()) << "Added priority hash " << hash << " for ledger "
<< hash_;
}
bool
InboundLedger::hasTx(uint256 const& txHash) const
{
ScopedLockType sl(mtx_);
return knownTxHashes_.count(txHash) > 0;
}
bool
InboundLedger::checkLocal()
{
@@ -347,6 +367,7 @@ InboundLedger::tryDB(NodeStore::Database& srcDB)
}
}
//@@start completion-check
if (mHaveTransactions && mHaveState)
{
JLOG(journal_.debug()) << "Had everything locally";
@@ -356,6 +377,7 @@ InboundLedger::tryDB(NodeStore::Database& srcDB)
"ripple::InboundLedger::tryDB : valid ledger fees");
mLedger->setImmutable();
}
//@@end completion-check
}
/** Called with a lock by the PeerSet when the timer expires
@@ -520,6 +542,43 @@ InboundLedger::trigger(std::shared_ptr<Peer> const& peer, TriggerReason reason)
}
}
// Handle priority hashes immediately (for partial sync queries)
if (mHaveHeader && !priorityHashes_.empty())
{
JLOG(journal_.warn()) << "PRIORITY: trigger() sending "
<< priorityHashes_.size() << " priority requests";
protocol::TMGetObjectByHash tmBH;
tmBH.set_query(true);
tmBH.set_type(protocol::TMGetObjectByHash::otSTATE_NODE);
tmBH.set_ledgerhash(hash_.begin(), hash_.size());
for (auto const& h : priorityHashes_)
{
JLOG(journal_.warn()) << "PRIORITY: requesting node " << h;
protocol::TMIndexedObject* io = tmBH.add_objects();
io->set_hash(h.begin(), h.size());
if (mSeq != 0)
io->set_ledgerseq(mSeq);
}
// Send to all peers in our peer set
auto packet = std::make_shared<Message>(tmBH, protocol::mtGET_OBJECTS);
auto const& peerIds = mPeerSet->getPeerIds();
std::size_t sentCount = 0;
for (auto id : peerIds)
{
if (auto p = app_.overlay().findPeerByShortID(id))
{
p->send(packet);
++sentCount;
}
}
JLOG(journal_.warn()) << "PRIORITY: sent to " << sentCount << " peers";
priorityHashes_.clear();
}
protocol::TMGetLedger tmGL;
tmGL.set_ledgerhash(hash_.begin(), hash_.size());
@@ -613,7 +672,12 @@ InboundLedger::trigger(std::shared_ptr<Peer> const& peer, TriggerReason reason)
// Get the state data first because it's the most likely to be useful
// if we wind up abandoning this fetch.
if (mHaveHeader && !mHaveState && !failed_)
// When TX is prioritized for this ledger range, skip state until TX
// complete.
bool const txPrioritized =
mSeq != 0 && app_.getInboundLedgers().isTxPrioritized(mSeq);
if (mHaveHeader && !mHaveState && !failed_ &&
!(txPrioritized && !mHaveTransactions))
{
XRPL_ASSERT(
mLedger,
@@ -837,6 +901,9 @@ InboundLedger::takeHeader(std::string const& data)
mLedger->txMap().setLedgerSeq(mSeq);
mHaveHeader = true;
JLOG(app_.journal("TxTrack").warn())
<< "GOT HEADER seq=" << mSeq << " txHash=" << mLedger->info().txHash;
Serializer s(data.size() + 4);
s.add32(HashPrefix::ledgerMaster);
s.addRaw(data.data(), data.size());
@@ -906,6 +973,33 @@ InboundLedger::receiveNode(protocol::TMLedgerData& packet, SHAMapAddNode& san)
if (!nodeID)
throw std::runtime_error("data does not properly deserialize");
// For TX nodes, extract tx hash from leaf nodes for submit_and_wait
if (packet.type() == protocol::liTX_NODE)
{
auto const& data = node.nodedata();
// Leaf nodes have wire type as last byte
// Format: [tx+meta data...][32-byte tx hash][1-byte type]
if (data.size() >= 33)
{
uint8_t wireType =
static_cast<uint8_t>(data[data.size() - 1]);
// wireTypeTransactionWithMeta = 4
if (wireType == 4)
{
uint256 txHash;
std::memcpy(
txHash.data(), data.data() + data.size() - 33, 32);
auto [it, inserted] = knownTxHashes_.insert(txHash);
if (inserted)
{
JLOG(app_.journal("TxTrack").warn())
<< "GOT TX ledger=" << mSeq << " tx=" << txHash
<< " count=" << knownTxHashes_.size();
}
}
}
}
if (nodeID->isRoot())
{
san += map.addRootNode(rootHash, makeSlice(node.nodedata()), f);
+104 -2
View File
@@ -25,6 +25,7 @@
#include <xrpld/perflog/PerfLog.h>
#include <xrpl/basics/DecayingSample.h>
#include <xrpl/basics/Log.h>
#include <xrpl/basics/RangeSet.h>
#include <xrpl/basics/scope.h>
#include <xrpl/beast/container/aged_map.h>
#include <xrpl/beast/core/LexicalCast.h>
@@ -183,6 +184,89 @@ public:
return ret;
}
std::shared_ptr<Ledger const>
getPartialLedger(uint256 const& hash) override
{
auto inbound = find(hash);
if (inbound && inbound->hasHeader() && !inbound->isFailed())
return inbound->getLedger();
return nullptr;
}
std::optional<uint256>
findTxLedger(uint256 const& txHash) override
{
auto const swj = app_.journal("SubmitAndWait");
ScopedLockType sl(mLock);
JLOG(swj.debug()) << "findTxLedger tx=" << txHash << " searching "
<< mLedgers.size() << " inbound ledgers";
for (auto const& [hash, inbound] : mLedgers)
{
bool hasHdr = inbound->hasHeader();
bool failed = inbound->isFailed();
bool hasTx = hasHdr && !failed && inbound->hasTx(txHash);
JLOG(swj.trace())
<< "findTxLedger checking ledger seq=" << inbound->getSeq()
<< " hash=" << hash << " hasHeader=" << hasHdr
<< " failed=" << failed << " hasTx=" << hasTx;
if (hasTx)
{
JLOG(swj.warn()) << "findTxLedger FOUND tx=" << txHash
<< " in ledger seq=" << inbound->getSeq();
return hash;
}
}
JLOG(swj.debug()) << "findTxLedger tx=" << txHash << " NOT FOUND";
return std::nullopt;
}
void
addPriorityNode(std::uint32_t ledgerSeq, uint256 const& nodeHash) override
{
std::shared_ptr<InboundLedger> inbound;
{
ScopedLockType sl(mLock);
// Find inbound ledger by sequence (need to iterate)
for (auto const& [hash, ledger] : mLedgers)
{
if (ledger->getSeq() == ledgerSeq && !ledger->isFailed() &&
!ledger->isComplete())
{
inbound = ledger;
break;
}
}
}
if (inbound)
{
inbound->addPriorityHash(nodeHash);
JLOG(j_.warn()) << "PRIORITY: added node " << nodeHash
<< " for ledger seq " << ledgerSeq;
}
else
{
JLOG(j_.warn()) << "PRIORITY: no inbound ledger for seq "
<< ledgerSeq << " (node " << nodeHash << ")";
}
}
void
prioritizeTxForLedgers(std::uint32_t start, std::uint32_t end) override
{
std::lock_guard lock(txPriorityMutex_);
txPriorityRange_.insert(ClosedInterval<std::uint32_t>(start, end));
JLOG(j_.debug()) << "TX priority added for ledgers " << start << "-"
<< end;
}
bool
isTxPrioritized(std::uint32_t seq) const override
{
std::lock_guard lock(txPriorityMutex_);
return boost::icl::contains(txPriorityRange_, seq);
}
/*
This gets called when
"We got some data from an inbound ledger"
@@ -406,6 +490,11 @@ public:
}
else if ((la + std::chrono::minutes(1)) < start)
{
JLOG(app_.journal("SubmitAndWait").debug())
<< "sweep removing ledger seq=" << it->second->getSeq()
<< " complete=" << it->second->isComplete()
<< " failed=" << it->second->isFailed()
<< " knownTxCount=" << it->second->knownTxCount();
stuffToSweep.push_back(it->second);
// shouldn't cause the actual final delete
// since we are holding a reference in the vector.
@@ -420,13 +509,22 @@ public:
beast::expire(mRecentFailures, kReacquireInterval);
}
JLOG(j_.debug())
<< "Swept " << stuffToSweep.size() << " out of " << total
JLOG(app_.journal("SubmitAndWait").debug())
<< "sweep removed " << stuffToSweep.size() << " out of " << total
<< " inbound ledgers. Duration: "
<< std::chrono::duration_cast<std::chrono::milliseconds>(
m_clock.now() - start)
.count()
<< "ms";
// Clear expired TX-priority ranges (anything at or below validated)
{
std::lock_guard lock(txPriorityMutex_);
auto const validSeq = app_.getLedgerMaster().getValidLedgerIndex();
if (validSeq > 0 && !txPriorityRange_.empty())
txPriorityRange_.erase(
ClosedInterval<std::uint32_t>(0, validSeq));
}
}
void
@@ -463,6 +561,10 @@ private:
std::set<uint256> pendingAcquires_;
std::mutex acquiresMutex_;
// Ledger ranges where TX fetching should be prioritized
mutable std::mutex txPriorityMutex_;
RangeSet<std::uint32_t> txPriorityRange_;
};
//------------------------------------------------------------------------------
+20 -1
View File
@@ -107,6 +107,7 @@ LedgerMaster::LedgerMaster(
beast::Journal journal)
: app_(app)
, m_journal(journal)
, jPartialSync_(app.journal("PartialSync"))
, mLedgerHistory(collector, app)
, standalone_(app_.config().standalone())
, fetch_depth_(
@@ -916,11 +917,29 @@ LedgerMaster::checkAccept(uint256 const& hash, std::uint32_t seq)
auto validations = app_.validators().negativeUNLFilter(
app_.getValidations().getTrustedForLedger(hash, seq));
valCount = validations.size();
if (valCount >= app_.validators().quorum())
auto const quorum = app_.validators().quorum();
JLOG(jPartialSync_.warn())
<< "checkAccept: hash=" << hash << " seq=" << seq
<< " valCount=" << valCount << " quorum=" << quorum
<< " mLastValidLedger.seq=" << mLastValidLedger.second;
if (valCount >= quorum)
{
std::lock_guard ml(m_mutex);
if (seq > mLastValidLedger.second)
{
JLOG(jPartialSync_.warn())
<< "checkAccept: QUORUM REACHED - setting mLastValidLedger"
<< " seq=" << seq << " hash=" << hash;
mLastValidLedger = std::make_pair(hash, seq);
}
}
else
{
JLOG(jPartialSync_.debug())
<< "checkAccept: quorum not reached, need " << quorum
<< " have " << valCount;
}
if (seq == mValidLedgerSeq)
+1
View File
@@ -299,6 +299,7 @@ public:
logs_->journal("Collector")))
, m_jobQueue(std::make_unique<JobQueue>(
get_io_service(),
[](std::unique_ptr<Config> const& config) {
if (config->standalone() && !config->FORCE_MULTI_THREAD)
return 1;
+137 -7
View File
@@ -51,6 +51,8 @@ reduceOffer(auto const& amount)
} // namespace detail
enum class IsDeposit : bool { No = false, Yes = true };
/** Calculate LP Tokens given AMM pool reserves.
* @param asset1 AMM one side of the pool reserve
* @param asset2 AMM another side of the pool reserve
@@ -70,7 +72,7 @@ ammLPTokens(
* @return tokens
*/
STAmount
lpTokensIn(
lpTokensOut(
STAmount const& asset1Balance,
STAmount const& asset1Deposit,
STAmount const& lptAMMBalance,
@@ -99,7 +101,7 @@ ammAssetIn(
* @return tokens out amount
*/
STAmount
lpTokensOut(
lpTokensIn(
STAmount const& asset1Balance,
STAmount const& asset1Withdraw,
STAmount const& lptAMMBalance,
@@ -113,7 +115,7 @@ lpTokensOut(
* @return calculated asset amount
*/
STAmount
withdrawByTokens(
ammAssetOut(
STAmount const& assetBalance,
STAmount const& lptAMMBalance,
STAmount const& lpTokens,
@@ -517,13 +519,13 @@ square(Number const& n);
* withdraw to cancel out the precision loss.
* @param lptAMMBalance LPT AMM Balance
* @param lpTokens LP tokens to deposit or withdraw
* @param isDeposit true if deposit, false if withdraw
* @param isDeposit Yes if deposit, No if withdraw
*/
STAmount
adjustLPTokens(
STAmount const& lptAMMBalance,
STAmount const& lpTokens,
bool isDeposit);
IsDeposit isDeposit);
/** Calls adjustLPTokens() and adjusts deposit or withdraw amounts if
* the adjusted LP tokens are less than the provided LP tokens.
@@ -533,7 +535,7 @@ adjustLPTokens(
* @param lptAMMBalance LPT AMM Balance
* @param lpTokens LP tokens to deposit or withdraw
* @param tfee trading fee in basis points
* @param isDeposit true if deposit, false if withdraw
* @param isDeposit Yes if deposit, No if withdraw
* @return
*/
std::tuple<STAmount, std::optional<STAmount>, STAmount>
@@ -544,7 +546,7 @@ adjustAmountsByLPTokens(
STAmount const& lptAMMBalance,
STAmount const& lpTokens,
std::uint16_t tfee,
bool isDeposit);
IsDeposit isDeposit);
/** Positive solution for quadratic equation:
* x = (-b + sqrt(b**2 + 4*a*c))/(2*a)
@@ -552,6 +554,134 @@ adjustAmountsByLPTokens(
Number
solveQuadraticEq(Number const& a, Number const& b, Number const& c);
STAmount
multiply(STAmount const& amount, Number const& frac, Number::rounding_mode rm);
namespace detail {
inline Number::rounding_mode
getLPTokenRounding(IsDeposit isDeposit)
{
// Minimize on deposit, maximize on withdraw to ensure
// AMM invariant sqrt(poolAsset1 * poolAsset2) >= LPTokensBalance
return isDeposit == IsDeposit::Yes ? Number::downward : Number::upward;
}
inline Number::rounding_mode
getAssetRounding(IsDeposit isDeposit)
{
// Maximize on deposit, minimize on withdraw to ensure
// AMM invariant sqrt(poolAsset1 * poolAsset2) >= LPTokensBalance
return isDeposit == IsDeposit::Yes ? Number::upward : Number::downward;
}
} // namespace detail
/** Round AMM equal deposit/withdrawal amount. Deposit/withdrawal formulas
* calculate the amount as a fractional value of the pool balance. The rounding
* takes place on the last step of multiplying the balance by the fraction if
* AMMv1_3 is enabled.
*/
template <typename A>
STAmount
getRoundedAsset(
Rules const& rules,
STAmount const& balance,
A const& frac,
IsDeposit isDeposit)
{
auto const rm = detail::getAssetRounding(isDeposit);
return multiply(balance, frac, rm);
}
/** Round AMM single deposit/withdrawal amount.
* The lambda's are used to delay evaluation until the function
* is executed so that the calculation is not done twice. noRoundCb() is
* called if AMMv1_3 is disabled. Otherwise, the rounding is set and
* the amount is:
* isDeposit is Yes - the balance multiplied by productCb()
* isDeposit is No - the result of productCb(). The rounding is
* the same for all calculations in productCb()
*/
STAmount
getRoundedAsset(
Rules const& rules,
std::function<Number()>&& noRoundCb,
STAmount const& balance,
std::function<Number()>&& productCb,
IsDeposit isDeposit);
/** Round AMM deposit/withdrawal LPToken amount. Deposit/withdrawal formulas
* calculate the lptokens as a fractional value of the AMM total lptokens.
* The rounding takes place on the last step of multiplying the balance by
* the fraction if AMMv1_3 is enabled. The tokens are then
* adjusted to factor in the loss in precision (we only keep 16 significant
* digits) when adding the lptokens to the balance.
*/
STAmount
getRoundedLPTokens(
Rules const& rules,
STAmount const& balance,
Number const& frac,
IsDeposit isDeposit);
/** Round AMM single deposit/withdrawal LPToken amount.
* The lambda's are used to delay evaluation until the function is executed
* so that the calculations are not done twice.
* noRoundCb() is called if AMMv1_3 is disabled. Otherwise, the rounding is set
* and the lptokens are:
* if isDeposit is Yes - the result of productCb(). The rounding is
* the same for all calculations in productCb()
* if isDeposit is No - the balance multiplied by productCb()
* The lptokens are then adjusted to factor in the loss in precision
* (we only keep 16 significant digits) when adding the lptokens to the balance.
*/
STAmount
getRoundedLPTokens(
Rules const& rules,
std::function<Number()>&& noRoundCb,
STAmount const& lptAMMBalance,
std::function<Number()>&& productCb,
IsDeposit isDeposit);
/* Next two functions adjust asset in/out amount to factor in the adjusted
* lptokens. The lptokens are calculated from the asset in/out. The lptokens are
* then adjusted to factor in the loss in precision. The adjusted lptokens might
* be less than the initially calculated tokens. Therefore, the asset in/out
* must be adjusted. The rounding might result in the adjusted amount being
* greater than the original asset in/out amount. If this happens,
* then the original amount is reduced by the difference in the adjusted amount
* and the original amount. The actual tokens and the actual adjusted amount
* are then recalculated. The minimum of the original and the actual
* adjusted amount is returned.
*/
std::pair<STAmount, STAmount>
adjustAssetInByTokens(
Rules const& rules,
STAmount const& balance,
STAmount const& amount,
STAmount const& lptAMMBalance,
STAmount const& tokens,
std::uint16_t tfee);
std::pair<STAmount, STAmount>
adjustAssetOutByTokens(
Rules const& rules,
STAmount const& balance,
STAmount const& amount,
STAmount const& lptAMMBalance,
STAmount const& tokens,
std::uint16_t tfee);
/** Find a fraction of tokens after the tokens are adjusted. The fraction
* is used to adjust equal deposit/withdraw amount.
*/
Number
adjustFracByTokens(
Rules const& rules,
STAmount const& lptAMMBalance,
STAmount const& tokens,
Number const& frac);
} // namespace ripple
#endif // RIPPLE_APP_MISC_AMMHELPERS_H_INCLUDED
+11
View File
@@ -123,6 +123,17 @@ isOnlyLiquidityProvider(
Issue const& ammIssue,
AccountID const& lpAccount);
/** Due to rounding, the LPTokenBalance of the last LP might
* not match the LP's trustline balance. If it's within the tolerance,
* update LPTokenBalance to match the LP's trustline balance.
*/
Expected<bool, TER>
verifyAndAdjustLPTokenBalance(
Sandbox& sb,
STAmount const& lpTokens,
std::shared_ptr<SLE>& ammSle,
AccountID const& account);
} // namespace ripple
#endif // RIPPLE_APP_MISC_AMMUTILS_H_INLCUDED
+56
View File
@@ -225,6 +225,9 @@ public:
bool bLocal,
FailHard failType) override;
std::optional<uint256>
broadcastRawTransaction(Blob const& txBlob) override;
/**
* For transactions submitted directly by a client, apply batch of
* transactions and wait for this transaction to complete.
@@ -823,11 +826,13 @@ NetworkOPsImp::isNeedNetworkLedger()
return needNetworkLedger_;
}
//@@start is-full-check
inline bool
NetworkOPsImp::isFull()
{
return !needNetworkLedger_ && (mMode == OperatingMode::FULL);
}
//@@end is-full-check
std::string
NetworkOPsImp::getHostId(bool forAdmin)
@@ -1224,6 +1229,43 @@ NetworkOPsImp::processTransaction(
doTransactionAsync(transaction, bUnlimited, failType);
}
std::optional<uint256>
NetworkOPsImp::broadcastRawTransaction(Blob const& txBlob)
{
// Parse the transaction blob to get the hash
std::shared_ptr<STTx const> stx;
try
{
SerialIter sit(makeSlice(txBlob));
stx = std::make_shared<STTx const>(std::ref(sit));
}
catch (std::exception const& e)
{
JLOG(m_journal.warn())
<< "broadcastRawTransaction: Failed to parse tx blob: " << e.what();
return std::nullopt;
}
uint256 txHash = stx->getTransactionID();
// Broadcast to all peers without local validation
protocol::TMTransaction msg;
Serializer s;
stx->add(s);
msg.set_rawtransaction(s.data(), s.size());
msg.set_status(protocol::tsNEW); // tsNEW = origin node could not validate
msg.set_receivetimestamp(
app_.timeKeeper().now().time_since_epoch().count());
app_.overlay().foreach(
send_always(std::make_shared<Message>(msg, protocol::mtTRANSACTION)));
JLOG(m_journal.info()) << "broadcastRawTransaction: Broadcast tx "
<< txHash;
return txHash;
}
void
NetworkOPsImp::doTransactionAsync(
std::shared_ptr<Transaction> transaction,
@@ -1494,6 +1536,7 @@ NetworkOPsImp::apply(std::unique_lock<std::mutex>& batchLock)
bool const isEmitted =
hook::isEmittedTxn(*(e.transaction->getSTransaction()));
//@@start tx-relay
if (toSkip && !isEmitted)
{
protocol::TMTransaction tx;
@@ -1509,6 +1552,7 @@ NetworkOPsImp::apply(std::unique_lock<std::mutex>& batchLock)
app_.overlay().relay(e.transaction->getID(), tx, *toSkip);
e.transaction->setBroadcast();
}
//@@end tx-relay
}
if (validatedLedgerIndex)
@@ -1741,6 +1785,14 @@ NetworkOPsImp::checkLastClosedLedger(
if (!switchLedgers)
return false;
// Safety check: can't acquire a ledger with an invalid hash
if (!closedLedger.isNonZero())
{
JLOG(m_journal.warn())
<< "checkLastClosedLedger: closedLedger hash is zero, skipping";
return false;
}
auto consensus = m_ledgerMaster.getLedgerByHash(closedLedger);
if (!consensus)
@@ -1963,6 +2015,7 @@ NetworkOPsImp::endConsensus(std::unique_ptr<std::stringstream> const& clog)
// timing to make sure there shouldn't be a newer LCL. We need this
// information to do the next three tests.
//@@start mode-transitions
if (((mMode == OperatingMode::CONNECTED) ||
(mMode == OperatingMode::SYNCING)) &&
!ledgerChange)
@@ -1988,8 +2041,11 @@ NetworkOPsImp::endConsensus(std::unique_ptr<std::stringstream> const& clog)
setMode(OperatingMode::FULL);
}
}
//@@end mode-transitions
//@@start consensus-gate
beginConsensus(networkClosed, clog);
//@@end consensus-gate
}
void
+12
View File
@@ -32,6 +32,7 @@
#include <boost/asio.hpp>
#include <deque>
#include <memory>
#include <optional>
#include <tuple>
namespace ripple {
@@ -112,6 +113,17 @@ public:
bool bLocal,
FailHard failType) = 0;
/**
* Broadcast a raw transaction to peers without local validation.
* Used by submit_and_wait during partial sync mode when local state
* is not available for validation.
*
* @param txBlob The raw serialized transaction blob
* @return The transaction hash, or nullopt if parsing failed
*/
virtual std::optional<uint256>
broadcastRawTransaction(Blob const& txBlob) = 0;
//--------------------------------------------------------------------------
//
// Owner functions
+2
View File
@@ -31,6 +31,7 @@ namespace ripple {
not change them without verifying each use and ensuring that it is
not a breaking change.
*/
//@@start operating-mode-enum
enum class OperatingMode {
DISCONNECTED = 0, //!< not ready to process requests
CONNECTED = 1, //!< convinced we are talking to the network
@@ -38,6 +39,7 @@ enum class OperatingMode {
TRACKING = 3, //!< convinced we agree with the network
FULL = 4 //!< we have the ledger and can even validate
};
//@@end operating-mode-enum
class StateAccounting
{
+140 -53
View File
@@ -27,6 +27,9 @@ ammLPTokens(
STAmount const& asset2,
Issue const& lptIssue)
{
// AMM invariant: sqrt(asset1 * asset2) >= LPTokensBalance
auto const rounding = Number::downward;
NumberRoundModeGuard g(rounding);
auto const tokens = root2(asset1 * asset2);
return toSTAmount(lptIssue, tokens);
}
@@ -38,7 +41,7 @@ ammLPTokens(
* where f1 = 1 - tfee, f2 = (1 - tfee/2)/f1
*/
STAmount
lpTokensIn(
lpTokensOut(
STAmount const& asset1Balance,
STAmount const& asset1Deposit,
STAmount const& lptAMMBalance,
@@ -48,8 +51,10 @@ lpTokensIn(
auto const f2 = feeMultHalf(tfee) / f1;
Number const r = asset1Deposit / asset1Balance;
auto const c = root2(f2 * f2 + r / f1) - f2;
auto const t = lptAMMBalance * (r - c) / (1 + c);
return toSTAmount(lptAMMBalance.issue(), t);
// minimize tokens out
auto const frac = (r - c) / (1 + c);
return multiply(lptAMMBalance, frac, Number::downward);
}
/* Equation 4 solves equation 3 for b:
@@ -78,8 +83,10 @@ ammAssetIn(
auto const a = 1 / (t2 * t2);
auto const b = 2 * d / t2 - 1 / f1;
auto const c = d * d - f2 * f2;
return toSTAmount(
asset1Balance.issue(), asset1Balance * solveQuadraticEq(a, b, c));
// maximize deposit
auto const frac = solveQuadraticEq(a, b, c);
return multiply(asset1Balance, frac, Number::upward);
}
/* Equation 7:
@@ -87,7 +94,7 @@ ammAssetIn(
* where R = b/B, c = R*fee + 2 - fee
*/
STAmount
lpTokensOut(
lpTokensIn(
STAmount const& asset1Balance,
STAmount const& asset1Withdraw,
STAmount const& lptAMMBalance,
@@ -96,8 +103,10 @@ lpTokensOut(
Number const fr = asset1Withdraw / asset1Balance;
auto const f1 = getFee(tfee);
auto const c = fr * f1 + 2 - f1;
auto const t = lptAMMBalance * (c - root2(c * c - 4 * fr)) / 2;
return toSTAmount(lptAMMBalance.issue(), t);
// maximize tokens in
auto const frac = (c - root2(c * c - 4 * fr)) / 2;
return multiply(lptAMMBalance, frac, Number::upward);
}
/* Equation 8 solves equation 7 for b:
@@ -111,7 +120,7 @@ lpTokensOut(
* R = (t1**2 + t1*(f - 2)) / (t1*f - 1)
*/
STAmount
withdrawByTokens(
ammAssetOut(
STAmount const& assetBalance,
STAmount const& lptAMMBalance,
STAmount const& lpTokens,
@@ -119,8 +128,10 @@ withdrawByTokens(
{
auto const f = getFee(tfee);
Number const t1 = lpTokens / lptAMMBalance;
auto const b = assetBalance * (t1 * t1 - t1 * (2 - f)) / (t1 * f - 1);
return toSTAmount(assetBalance.issue(), b);
// minimize withdraw
auto const frac = (t1 * t1 - t1 * (2 - f)) / (t1 * f - 1);
return multiply(assetBalance, frac, Number::downward);
}
Number
@@ -133,12 +144,12 @@ STAmount
adjustLPTokens(
STAmount const& lptAMMBalance,
STAmount const& lpTokens,
bool isDeposit)
IsDeposit isDeposit)
{
// Force rounding downward to ensure adjusted tokens are less or equal
// to requested tokens.
saveNumberRoundMode rm(Number::setround(Number::rounding_mode::downward));
if (isDeposit)
if (isDeposit == IsDeposit::Yes)
return (lptAMMBalance + lpTokens) - lptAMMBalance;
return (lpTokens - lptAMMBalance) + lptAMMBalance;
}
@@ -151,47 +162,10 @@ adjustAmountsByLPTokens(
STAmount const& lptAMMBalance,
STAmount const& lpTokens,
std::uint16_t tfee,
bool isDeposit)
IsDeposit isDeposit)
{
auto const lpTokensActual =
adjustLPTokens(lptAMMBalance, lpTokens, isDeposit);
if (lpTokensActual == beast::zero)
{
auto const amount2Opt =
amount2 ? std::make_optional(STAmount{}) : std::nullopt;
return std::make_tuple(STAmount{}, amount2Opt, lpTokensActual);
}
if (lpTokensActual < lpTokens)
{
// Equal trade
if (amount2)
{
Number const fr = lpTokensActual / lpTokens;
auto const amountActual = toSTAmount(amount.issue(), fr * amount);
auto const amount2Actual =
toSTAmount(amount2->issue(), fr * *amount2);
return std::make_tuple(amountActual, amount2Actual, lpTokensActual);
}
// Single trade
auto const amountActual = [&]() {
if (isDeposit)
return ammAssetIn(
amountBalance, lptAMMBalance, lpTokensActual, tfee);
return withdrawByTokens(
amountBalance, lptAMMBalance, lpTokensActual, tfee);
}();
return std::make_tuple(amountActual, std::nullopt, lpTokensActual);
}
XRPL_ASSERT(
lpTokensActual == lpTokens,
"ripple::adjustAmountsByLPTokens : LP tokens match actual");
return {amount, amount2, lpTokensActual};
// AMMv1_3 amendment adjusts tokens and amounts in deposit/withdraw
return std::make_tuple(amount, amount2, lpTokens);
}
Number
@@ -215,4 +189,117 @@ solveQuadraticEqSmallest(Number const& a, Number const& b, Number const& c)
return (2 * c) / (-b + root2(d));
}
STAmount
multiply(STAmount const& amount, Number const& frac, Number::rounding_mode rm)
{
NumberRoundModeGuard g(rm);
auto const t = amount * frac;
return toSTAmount(amount.issue(), t, rm);
}
STAmount
getRoundedAsset(
Rules const& rules,
std::function<Number()>&& noRoundCb,
STAmount const& balance,
std::function<Number()>&& productCb,
IsDeposit isDeposit)
{
auto const rm = detail::getAssetRounding(isDeposit);
if (isDeposit == IsDeposit::Yes)
return multiply(balance, productCb(), rm);
NumberRoundModeGuard g(rm);
return toSTAmount(balance.issue(), productCb(), rm);
}
STAmount
getRoundedLPTokens(
Rules const& rules,
STAmount const& balance,
Number const& frac,
IsDeposit isDeposit)
{
auto const rm = detail::getLPTokenRounding(isDeposit);
auto const tokens = multiply(balance, frac, rm);
return adjustLPTokens(balance, tokens, isDeposit);
}
STAmount
getRoundedLPTokens(
Rules const& rules,
std::function<Number()>&& noRoundCb,
STAmount const& lptAMMBalance,
std::function<Number()>&& productCb,
IsDeposit isDeposit)
{
auto const tokens = [&] {
auto const rm = detail::getLPTokenRounding(isDeposit);
if (isDeposit == IsDeposit::Yes)
{
NumberRoundModeGuard g(rm);
return toSTAmount(lptAMMBalance.issue(), productCb(), rm);
}
return multiply(lptAMMBalance, productCb(), rm);
}();
return adjustLPTokens(lptAMMBalance, tokens, isDeposit);
}
std::pair<STAmount, STAmount>
adjustAssetInByTokens(
Rules const& rules,
STAmount const& balance,
STAmount const& amount,
STAmount const& lptAMMBalance,
STAmount const& tokens,
std::uint16_t tfee)
{
auto assetAdj = ammAssetIn(balance, lptAMMBalance, tokens, tfee);
auto tokensAdj = tokens;
// Rounding didn't work the right way.
// Try to adjust the original deposit amount by difference
// in adjust and original amount. Then adjust tokens and deposit amount.
if (assetAdj > amount)
{
auto const adjAmount = amount - (assetAdj - amount);
auto const t = lpTokensOut(balance, adjAmount, lptAMMBalance, tfee);
tokensAdj = adjustLPTokens(lptAMMBalance, t, IsDeposit::Yes);
assetAdj = ammAssetIn(balance, lptAMMBalance, tokensAdj, tfee);
}
return {tokensAdj, std::min(amount, assetAdj)};
}
std::pair<STAmount, STAmount>
adjustAssetOutByTokens(
Rules const& rules,
STAmount const& balance,
STAmount const& amount,
STAmount const& lptAMMBalance,
STAmount const& tokens,
std::uint16_t tfee)
{
auto assetAdj = ammAssetOut(balance, lptAMMBalance, tokens, tfee);
auto tokensAdj = tokens;
// Rounding didn't work the right way.
// Try to adjust the original deposit amount by difference
// in adjust and original amount. Then adjust tokens and deposit amount.
if (assetAdj > amount)
{
auto const adjAmount = amount - (assetAdj - amount);
auto const t = lpTokensIn(balance, adjAmount, lptAMMBalance, tfee);
tokensAdj = adjustLPTokens(lptAMMBalance, t, IsDeposit::No);
assetAdj = ammAssetOut(balance, lptAMMBalance, tokensAdj, tfee);
}
return {tokensAdj, std::min(amount, assetAdj)};
}
Number
adjustFracByTokens(
Rules const& rules,
STAmount const& lptAMMBalance,
STAmount const& tokens,
Number const& frac)
{
return tokens / lptAMMBalance;
}
} // namespace ripple
+30
View File
@@ -16,6 +16,8 @@
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <xrpld/app/misc/AMMHelpers.h>
#include <xrpld/app/misc/AMMUtils.h>
#include <xrpld/ledger/Sandbox.h>
#include <xrpl/basics/Log.h>
@@ -462,4 +464,32 @@ isOnlyLiquidityProvider(
return Unexpected<TER>(tecINTERNAL); // LCOV_EXCL_LINE
}
Expected<bool, TER>
verifyAndAdjustLPTokenBalance(
Sandbox& sb,
STAmount const& lpTokens,
std::shared_ptr<SLE>& ammSle,
AccountID const& account)
{
if (auto const res = isOnlyLiquidityProvider(sb, lpTokens.issue(), account);
!res)
return Unexpected<TER>(res.error());
else if (res.value())
{
if (withinRelativeDistance(
lpTokens,
ammSle->getFieldAmount(sfLPTokenBalance),
Number{1, -3}))
{
ammSle->setFieldAmount(sfLPTokenBalance, lpTokens);
sb.update(ammSle);
}
else
{
return Unexpected<TER>(tecAMM_INVALID_TOKENS);
}
}
return true;
}
} // namespace ripple
@@ -966,6 +966,16 @@ ValidatorList::applyListsAndBroadcast(
if (good)
{
networkOPs.clearUNLBlocked();
// For partial sync: trigger early quorum calculation so
// validations can be trusted before consensus starts
JLOG(j_.warn()) << "All publisher lists available, triggering "
"early updateTrusted for partial sync";
updateTrusted(
{}, // empty seenValidators - we just need quorum calculated
timeKeeper_.now(),
networkOPs,
overlay,
hashRouter);
}
}
bool broadcast = disposition <= ListDisposition::known_sequence;
@@ -166,6 +166,7 @@ ValidatorSite::load(
void
ValidatorSite::start()
{
JLOG(j_.warn()) << "ValidatorSite::start() called";
std::lock_guard l0{sites_mutex_};
std::lock_guard l1{state_mutex_};
if (timer_.expires_at() == clock_type::time_point{})
@@ -218,6 +219,11 @@ ValidatorSite::setTimer(
if (next != sites_.end())
{
pending_ = next->nextRefresh <= clock_type::now();
auto delay = std::chrono::duration_cast<std::chrono::milliseconds>(
next->nextRefresh - clock_type::now());
JLOG(j_.warn()) << "ValidatorSite::setTimer() pending=" << pending_
<< " delay=" << delay.count() << "ms"
<< " uri=" << next->startingResource->uri;
cv_.notify_all();
timer_.expires_at(next->nextRefresh);
auto idx = std::distance(sites_.begin(), next);
@@ -225,6 +231,10 @@ ValidatorSite::setTimer(
this->onTimer(idx, ec);
});
}
else
{
JLOG(j_.warn()) << "ValidatorSite::setTimer() no sites configured";
}
}
void
+18 -1
View File
@@ -79,6 +79,21 @@ AMMBid::preflight(PreflightContext const& ctx)
JLOG(ctx.j.debug()) << "AMM Bid: Invalid number of AuthAccounts.";
return temMALFORMED;
}
else
{
AccountID account = ctx.tx[sfAccount];
std::set<AccountID> unique;
for (auto const& obj : authAccounts)
{
auto authAccount = obj[sfAccount];
if (authAccount == account || unique.contains(authAccount))
{
JLOG(ctx.j.debug()) << "AMM Bid: Invalid auth.account.";
return temMALFORMED;
}
unique.insert(authAccount);
}
}
}
return preflight2(ctx);
@@ -233,7 +248,9 @@ applyBid(
auctionSlot.makeFieldAbsent(sfAuthAccounts);
// Burn the remaining bid amount
auto const saBurn = adjustLPTokens(
lptAMMBalance, toSTAmount(lptAMMBalance.issue(), burn), false);
lptAMMBalance,
toSTAmount(lptAMMBalance.issue(), burn),
IsDeposit::No);
if (saBurn >= lptAMMBalance)
{
// This error case should never occur.
+33 -6
View File
@@ -151,6 +151,17 @@ AMMClawback::applyGuts(Sandbox& sb)
if (!accountSle)
return tecINTERNAL; // LCOV_EXCL_LINE
// retrieve LP token balance inside the amendment gate to avoid
// inconsistent error behavior
auto const lpTokenBalance = ammLPHolds(sb, *ammSle, holder, j_);
if (lpTokenBalance == beast::zero)
return tecAMM_BALANCE;
if (auto const res =
verifyAndAdjustLPTokenBalance(sb, lpTokenBalance, ammSle, holder);
!res)
return res.error(); // LCOV_EXCL_LINE
auto const expected = ammHolds(
sb,
*ammSle,
@@ -248,10 +259,11 @@ AMMClawback::equalWithdrawMatchingOneAmount(
STAmount const& amount)
{
auto frac = Number{amount} / amountBalance;
auto const amount2Withdraw = amount2Balance * frac;
auto amount2Withdraw = amount2Balance * frac;
auto const lpTokensWithdraw =
toSTAmount(lptAMMBalance.issue(), lptAMMBalance * frac);
if (lpTokensWithdraw > holdLPtokens)
// if lptoken balance less than what the issuer intended to clawback,
// clawback all the tokens. Because we are doing a two-asset withdrawal,
@@ -272,18 +284,33 @@ AMMClawback::equalWithdrawMatchingOneAmount(
mPriorBalance,
ctx_.journal);
// Because we are doing a two-asset withdrawal,
// tfee is actually not used, so pass tfee as 0.
auto const& rules = sb.rules();
auto tokensAdj =
getRoundedLPTokens(rules, lptAMMBalance, frac, IsDeposit::No);
// LCOV_EXCL_START
if (tokensAdj == beast::zero)
return {tecAMM_INVALID_TOKENS, STAmount{}, STAmount{}, std::nullopt};
// LCOV_EXCL_STOP
frac = adjustFracByTokens(rules, lptAMMBalance, tokensAdj, frac);
auto amount2Rounded =
getRoundedAsset(rules, amount2Balance, frac, IsDeposit::No);
auto amountRounded =
getRoundedAsset(rules, amountBalance, frac, IsDeposit::No);
return AMMWithdraw::withdraw(
sb,
ammSle,
ammAccount,
holder,
amountBalance,
amount,
toSTAmount(amount2Balance.issue(), amount2Withdraw),
amountRounded,
amount2Rounded,
lptAMMBalance,
toSTAmount(lptAMMBalance.issue(), lptAMMBalance * frac),
tokensAdj,
0,
FreezeHandling::fhIGNORE_FREEZE,
WithdrawAll::No,
+106 -36
View File
@@ -545,7 +545,7 @@ AMMDeposit::deposit(
lptAMMBalance,
lpTokensDeposit,
tfee,
true);
IsDeposit::Yes);
if (lpTokensDepositActual <= beast::zero)
{
@@ -628,6 +628,15 @@ AMMDeposit::deposit(
return {tesSUCCESS, lptAMMBalance + lpTokensDepositActual};
}
static STAmount
adjustLPTokensOut(
Rules const& rules,
STAmount const& lptAMMBalance,
STAmount const& lpTokensDeposit)
{
return adjustLPTokens(lptAMMBalance, lpTokensDeposit, IsDeposit::Yes);
}
/** Proportional deposit of pools assets in exchange for the specified
* amount of LPTokens.
*/
@@ -645,16 +654,25 @@ AMMDeposit::equalDepositTokens(
{
try
{
auto const tokensAdj =
adjustLPTokensOut(view.rules(), lptAMMBalance, lpTokensDeposit);
if (tokensAdj == beast::zero)
return {tecAMM_INVALID_TOKENS, STAmount{}};
auto const frac =
divide(lpTokensDeposit, lptAMMBalance, lptAMMBalance.issue());
divide(tokensAdj, lptAMMBalance, lptAMMBalance.issue());
// amounts factor in the adjusted tokens
auto const amountDeposit =
getRoundedAsset(view.rules(), amountBalance, frac, IsDeposit::Yes);
auto const amount2Deposit =
getRoundedAsset(view.rules(), amount2Balance, frac, IsDeposit::Yes);
return deposit(
view,
ammAccount,
amountBalance,
multiply(amountBalance, frac, amountBalance.issue()),
multiply(amount2Balance, frac, amount2Balance.issue()),
amountDeposit,
amount2Deposit,
lptAMMBalance,
lpTokensDeposit,
tokensAdj,
depositMin,
deposit2Min,
std::nullopt,
@@ -711,37 +729,49 @@ AMMDeposit::equalDepositLimit(
std::uint16_t tfee)
{
auto frac = Number{amount} / amountBalance;
auto tokens = toSTAmount(lptAMMBalance.issue(), lptAMMBalance * frac);
if (tokens == beast::zero)
return {tecAMM_FAILED, STAmount{}};
auto const amount2Deposit = amount2Balance * frac;
auto tokensAdj =
getRoundedLPTokens(view.rules(), lptAMMBalance, frac, IsDeposit::Yes);
if (tokensAdj == beast::zero)
{
return {tecAMM_INVALID_TOKENS, STAmount{}};
}
// factor in the adjusted tokens
frac = adjustFracByTokens(view.rules(), lptAMMBalance, tokensAdj, frac);
auto const amount2Deposit =
getRoundedAsset(view.rules(), amount2Balance, frac, IsDeposit::Yes);
if (amount2Deposit <= amount2)
return deposit(
view,
ammAccount,
amountBalance,
amount,
toSTAmount(amount2Balance.issue(), amount2Deposit),
amount2Deposit,
lptAMMBalance,
tokens,
tokensAdj,
std::nullopt,
std::nullopt,
lpTokensDepositMin,
tfee);
frac = Number{amount2} / amount2Balance;
tokens = toSTAmount(lptAMMBalance.issue(), lptAMMBalance * frac);
if (tokens == beast::zero)
return {tecAMM_FAILED, STAmount{}};
auto const amountDeposit = amountBalance * frac;
tokensAdj =
getRoundedLPTokens(view.rules(), lptAMMBalance, frac, IsDeposit::Yes);
if (tokensAdj == beast::zero)
{
return {tecAMM_INVALID_TOKENS, STAmount{}}; // LCOV_EXCL_LINE
}
// factor in the adjusted tokens
frac = adjustFracByTokens(view.rules(), lptAMMBalance, tokensAdj, frac);
auto const amountDeposit =
getRoundedAsset(view.rules(), amountBalance, frac, IsDeposit::Yes);
if (amountDeposit <= amount)
return deposit(
view,
ammAccount,
amountBalance,
toSTAmount(amountBalance.issue(), amountDeposit),
amountDeposit,
amount2,
lptAMMBalance,
tokens,
tokensAdj,
std::nullopt,
std::nullopt,
lpTokensDepositMin,
@@ -767,17 +797,27 @@ AMMDeposit::singleDeposit(
std::optional<STAmount> const& lpTokensDepositMin,
std::uint16_t tfee)
{
auto const tokens = lpTokensIn(amountBalance, amount, lptAMMBalance, tfee);
auto const tokens = adjustLPTokensOut(
view.rules(),
lptAMMBalance,
lpTokensOut(amountBalance, amount, lptAMMBalance, tfee));
if (tokens == beast::zero)
return {tecAMM_FAILED, STAmount{}};
{
return {tecAMM_INVALID_TOKENS, STAmount{}};
}
// factor in the adjusted tokens
auto const [tokensAdj, amountDepositAdj] = adjustAssetInByTokens(
view.rules(), amountBalance, amount, lptAMMBalance, tokens, tfee);
if (tokensAdj == beast::zero)
return {tecAMM_INVALID_TOKENS, STAmount{}}; // LCOV_EXCL_LINE
return deposit(
view,
ammAccount,
amountBalance,
amount,
amountDepositAdj,
std::nullopt,
lptAMMBalance,
tokens,
tokensAdj,
std::nullopt,
std::nullopt,
lpTokensDepositMin,
@@ -801,8 +841,13 @@ AMMDeposit::singleDepositTokens(
STAmount const& lpTokensDeposit,
std::uint16_t tfee)
{
auto const tokensAdj =
adjustLPTokensOut(view.rules(), lptAMMBalance, lpTokensDeposit);
if (tokensAdj == beast::zero)
return {tecAMM_INVALID_TOKENS, STAmount{}};
// the adjusted tokens are factored in
auto const amountDeposit =
ammAssetIn(amountBalance, lptAMMBalance, lpTokensDeposit, tfee);
ammAssetIn(amountBalance, lptAMMBalance, tokensAdj, tfee);
if (amountDeposit > amount)
return {tecAMM_FAILED, STAmount{}};
return deposit(
@@ -812,7 +857,7 @@ AMMDeposit::singleDepositTokens(
amountDeposit,
std::nullopt,
lptAMMBalance,
lpTokensDeposit,
tokensAdj,
std::nullopt,
std::nullopt,
std::nullopt,
@@ -856,20 +901,29 @@ AMMDeposit::singleDepositEPrice(
{
if (amount != beast::zero)
{
auto const tokens =
lpTokensIn(amountBalance, amount, lptAMMBalance, tfee);
auto const tokens = adjustLPTokensOut(
view.rules(),
lptAMMBalance,
lpTokensOut(amountBalance, amount, lptAMMBalance, tfee));
if (tokens <= beast::zero)
return {tecAMM_FAILED, STAmount{}};
auto const ep = Number{amount} / tokens;
{
return {tecAMM_INVALID_TOKENS, STAmount{}};
}
// factor in the adjusted tokens
auto const [tokensAdj, amountDepositAdj] = adjustAssetInByTokens(
view.rules(), amountBalance, amount, lptAMMBalance, tokens, tfee);
if (tokensAdj == beast::zero)
return {tecAMM_INVALID_TOKENS, STAmount{}}; // LCOV_EXCL_LINE
auto const ep = Number{amountDepositAdj} / tokensAdj;
if (ep <= ePrice)
return deposit(
view,
ammAccount,
amountBalance,
amount,
amountDepositAdj,
std::nullopt,
lptAMMBalance,
tokens,
tokensAdj,
std::nullopt,
std::nullopt,
std::nullopt,
@@ -900,21 +954,37 @@ AMMDeposit::singleDepositEPrice(
auto const a1 = c * c;
auto const b1 = c * c * f2 * f2 + 2 * c - d * d;
auto const c1 = 2 * c * f2 * f2 + 1 - 2 * d * f2;
auto const amountDeposit = toSTAmount(
amountBalance.issue(),
f1 * amountBalance * solveQuadraticEq(a1, b1, c1));
auto amtNoRoundCb = [&] {
return f1 * amountBalance * solveQuadraticEq(a1, b1, c1);
};
auto amtProdCb = [&] { return f1 * solveQuadraticEq(a1, b1, c1); };
auto const amountDeposit = getRoundedAsset(
view.rules(), amtNoRoundCb, amountBalance, amtProdCb, IsDeposit::Yes);
if (amountDeposit <= beast::zero)
return {tecAMM_FAILED, STAmount{}};
auto const tokens =
toSTAmount(lptAMMBalance.issue(), amountDeposit / ePrice);
auto tokNoRoundCb = [&] { return amountDeposit / ePrice; };
auto tokProdCb = [&] { return amountDeposit / ePrice; };
auto const tokens = getRoundedLPTokens(
view.rules(), tokNoRoundCb, lptAMMBalance, tokProdCb, IsDeposit::Yes);
// factor in the adjusted tokens
auto const [tokensAdj, amountDepositAdj] = adjustAssetInByTokens(
view.rules(),
amountBalance,
amountDeposit,
lptAMMBalance,
tokens,
tfee);
if (tokensAdj == beast::zero)
return {tecAMM_INVALID_TOKENS, STAmount{}}; // LCOV_EXCL_LINE
return deposit(
view,
ammAccount,
amountBalance,
amountDeposit,
amountDepositAdj,
std::nullopt,
lptAMMBalance,
tokens,
tokensAdj,
std::nullopt,
std::nullopt,
std::nullopt,
+98 -48
View File
@@ -313,24 +313,9 @@ AMMWithdraw::applyGuts(Sandbox& sb)
// might not match the LP's trustline balance
if (auto const res =
isOnlyLiquidityProvider(sb, lpTokens.issue(), account_);
verifyAndAdjustLPTokenBalance(sb, lpTokens, ammSle, account_);
!res)
return {res.error(), false};
else if (res.value())
{
if (withinRelativeDistance(
lpTokens,
ammSle->getFieldAmount(sfLPTokenBalance),
Number{1, -3}))
{
ammSle->setFieldAmount(sfLPTokenBalance, lpTokens);
sb.update(ammSle);
}
else
{
return {tecAMM_INVALID_TOKENS, false};
}
}
auto const tfee = getTradingFee(ctx_.view(), *ammSle, account_);
@@ -523,7 +508,7 @@ AMMWithdraw::withdraw(
lpTokensAMMBalance,
lpTokensWithdraw,
tfee,
false);
IsDeposit::No);
return std::make_tuple(
amountWithdraw, amount2Withdraw, lpTokensWithdraw);
}();
@@ -682,6 +667,20 @@ AMMWithdraw::withdraw(
amount2WithdrawActual);
}
static STAmount
adjustLPTokensIn(
Rules const& rules,
STAmount const& lptAMMBalance,
STAmount const& lpTokensWithdraw,
WithdrawAll withdrawAll)
{
if (withdrawAll == WithdrawAll::Yes)
return lpTokensWithdraw;
return adjustLPTokens(lptAMMBalance, lpTokensWithdraw, IsDeposit::No);
}
/** Proportional withdrawal of pool assets for the amount of LPTokens.
*/
std::pair<TER, STAmount>
AMMWithdraw::equalWithdrawTokens(
Sandbox& view,
@@ -785,16 +784,22 @@ AMMWithdraw::equalWithdrawTokens(
journal);
}
auto const frac = divide(lpTokensWithdraw, lptAMMBalance, noIssue());
auto const withdrawAmount =
multiply(amountBalance, frac, amountBalance.issue());
auto const withdraw2Amount =
multiply(amount2Balance, frac, amount2Balance.issue());
auto const tokensAdj = adjustLPTokensIn(
view.rules(), lptAMMBalance, lpTokensWithdraw, withdrawAll);
if (tokensAdj == beast::zero)
return {
tecAMM_INVALID_TOKENS, STAmount{}, STAmount{}, std::nullopt};
// the adjusted tokens are factored in
auto const frac = divide(tokensAdj, lptAMMBalance, noIssue());
auto const amountWithdraw =
getRoundedAsset(view.rules(), amountBalance, frac, IsDeposit::No);
auto const amount2Withdraw =
getRoundedAsset(view.rules(), amount2Balance, frac, IsDeposit::No);
// LP is making equal withdrawal by tokens but the requested amount
// of LP tokens is likely too small and results in one-sided pool
// withdrawal due to round off. Fail so the user withdraws
// more tokens.
if (withdrawAmount == beast::zero || withdraw2Amount == beast::zero)
if (amountWithdraw == beast::zero || amount2Withdraw == beast::zero)
return {tecAMM_FAILED, STAmount{}, STAmount{}, STAmount{}};
return withdraw(
@@ -803,10 +808,10 @@ AMMWithdraw::equalWithdrawTokens(
ammAccount,
account,
amountBalance,
withdrawAmount,
withdraw2Amount,
amountWithdraw,
amount2Withdraw,
lptAMMBalance,
lpTokensWithdraw,
tokensAdj,
tfee,
freezeHanding,
withdrawAll,
@@ -861,7 +866,16 @@ AMMWithdraw::equalWithdrawLimit(
std::uint16_t tfee)
{
auto frac = Number{amount} / amountBalance;
auto const amount2Withdraw = amount2Balance * frac;
auto amount2Withdraw =
getRoundedAsset(view.rules(), amount2Balance, frac, IsDeposit::No);
auto tokensAdj =
getRoundedLPTokens(view.rules(), lptAMMBalance, frac, IsDeposit::No);
if (tokensAdj == beast::zero)
return {tecAMM_INVALID_TOKENS, STAmount{}};
// factor in the adjusted tokens
frac = adjustFracByTokens(view.rules(), lptAMMBalance, tokensAdj, frac);
amount2Withdraw =
getRoundedAsset(view.rules(), amount2Balance, frac, IsDeposit::No);
if (amount2Withdraw <= amount2)
{
return withdraw(
@@ -870,26 +884,34 @@ AMMWithdraw::equalWithdrawLimit(
ammAccount,
amountBalance,
amount,
toSTAmount(amount2.issue(), amount2Withdraw),
amount2Withdraw,
lptAMMBalance,
toSTAmount(lptAMMBalance.issue(), lptAMMBalance * frac),
tokensAdj,
tfee);
}
frac = Number{amount2} / amount2Balance;
auto const amountWithdraw = amountBalance * frac;
XRPL_ASSERT(
amountWithdraw <= amount,
"ripple::AMMWithdraw::equalWithdrawLimit : maximum amountWithdraw");
auto amountWithdraw =
getRoundedAsset(view.rules(), amountBalance, frac, IsDeposit::No);
tokensAdj =
getRoundedLPTokens(view.rules(), lptAMMBalance, frac, IsDeposit::No);
if (tokensAdj == beast::zero)
return {tecAMM_INVALID_TOKENS, STAmount{}}; // LCOV_EXCL_LINE
// factor in the adjusted tokens
frac = adjustFracByTokens(view.rules(), lptAMMBalance, tokensAdj, frac);
amountWithdraw =
getRoundedAsset(view.rules(), amountBalance, frac, IsDeposit::No);
if (amountWithdraw > amount)
return {tecAMM_FAILED, STAmount{}}; // LCOV_EXCL_LINE
return withdraw(
view,
ammSle,
ammAccount,
amountBalance,
toSTAmount(amount.issue(), amountWithdraw),
amountWithdraw,
amount2,
lptAMMBalance,
toSTAmount(lptAMMBalance.issue(), lptAMMBalance * frac),
tokensAdj,
tfee);
}
@@ -908,19 +930,29 @@ AMMWithdraw::singleWithdraw(
STAmount const& amount,
std::uint16_t tfee)
{
auto const tokens = lpTokensOut(amountBalance, amount, lptAMMBalance, tfee);
auto const tokens = adjustLPTokensIn(
view.rules(),
lptAMMBalance,
lpTokensIn(amountBalance, amount, lptAMMBalance, tfee),
isWithdrawAll(ctx_.tx));
if (tokens == beast::zero)
return {tecAMM_FAILED, STAmount{}};
{
return {tecAMM_INVALID_TOKENS, STAmount{}};
}
// factor in the adjusted tokens
auto const [tokensAdj, amountWithdrawAdj] = adjustAssetOutByTokens(
view.rules(), amountBalance, amount, lptAMMBalance, tokens, tfee);
if (tokensAdj == beast::zero)
return {tecAMM_INVALID_TOKENS, STAmount{}}; // LCOV_EXCL_LINE
return withdraw(
view,
ammSle,
ammAccount,
amountBalance,
amount,
amountWithdrawAdj,
std::nullopt,
lptAMMBalance,
tokens,
tokensAdj,
tfee);
}
@@ -945,8 +977,13 @@ AMMWithdraw::singleWithdrawTokens(
STAmount const& lpTokensWithdraw,
std::uint16_t tfee)
{
auto const tokensAdj = adjustLPTokensIn(
view.rules(), lptAMMBalance, lpTokensWithdraw, isWithdrawAll(ctx_.tx));
if (tokensAdj == beast::zero)
return {tecAMM_INVALID_TOKENS, STAmount{}};
// the adjusted tokens are factored in
auto const amountWithdraw =
withdrawByTokens(amountBalance, lptAMMBalance, lpTokensWithdraw, tfee);
ammAssetOut(amountBalance, lptAMMBalance, tokensAdj, tfee);
if (amount == beast::zero || amountWithdraw >= amount)
{
return withdraw(
@@ -957,7 +994,7 @@ AMMWithdraw::singleWithdrawTokens(
amountWithdraw,
std::nullopt,
lptAMMBalance,
lpTokensWithdraw,
tokensAdj,
tfee);
}
@@ -1006,11 +1043,24 @@ AMMWithdraw::singleWithdrawEPrice(
// t = T*(T + A*E*(f - 2))/(T*f - A*E)
Number const ae = amountBalance * ePrice;
auto const f = getFee(tfee);
auto const tokens = lptAMMBalance * (lptAMMBalance + ae * (f - 2)) /
(lptAMMBalance * f - ae);
if (tokens <= 0)
return {tecAMM_FAILED, STAmount{}};
auto const amountWithdraw = toSTAmount(amount.issue(), tokens / ePrice);
auto tokNoRoundCb = [&] {
return lptAMMBalance * (lptAMMBalance + ae * (f - 2)) /
(lptAMMBalance * f - ae);
};
auto tokProdCb = [&] {
return (lptAMMBalance + ae * (f - 2)) / (lptAMMBalance * f - ae);
};
auto const tokensAdj = getRoundedLPTokens(
view.rules(), tokNoRoundCb, lptAMMBalance, tokProdCb, IsDeposit::No);
if (tokensAdj <= beast::zero)
{
return {tecAMM_INVALID_TOKENS, STAmount{}};
}
auto amtNoRoundCb = [&] { return tokensAdj / ePrice; };
auto amtProdCb = [&] { return tokensAdj / ePrice; };
// the adjusted tokens are factored in
auto const amountWithdraw = getRoundedAsset(
view.rules(), amtNoRoundCb, amount, amtProdCb, IsDeposit::No);
if (amount == beast::zero || amountWithdraw >= amount)
{
return withdraw(
@@ -1021,7 +1071,7 @@ AMMWithdraw::singleWithdrawEPrice(
amountWithdraw,
std::nullopt,
lptAMMBalance,
toSTAmount(lptAMMBalance.issue(), tokens),
tokensAdj,
tfee);
}
+1 -1
View File
@@ -301,7 +301,7 @@ private:
std::uint16_t tfee);
/** Check from the flags if it's withdraw all */
WithdrawAll
static WithdrawAll
isWithdrawAll(STTx const& tx);
};
+308
View File
@@ -17,6 +17,9 @@
*/
//==============================================================================
#include <xrpld/app/misc/AMMHelpers.h>
#include <xrpld/app/misc/AMMUtils.h>
#include <xrpld/app/misc/CredentialHelpers.h>
#include <xrpld/app/tx/detail/InvariantCheck.h>
#include <xrpld/app/misc/CredentialHelpers.h>
@@ -1674,4 +1677,309 @@ ValidPermissionedDomain::finalize(
(sleStatus_[1] ? check(*sleStatus_[1], j) : true);
}
void
ValidAMM::visitEntry(
bool isDelete,
std::shared_ptr<SLE const> const& before,
std::shared_ptr<SLE const> const& after)
{
if (isDelete)
return;
if (after)
{
auto const type = after->getType();
// AMM object changed
if (type == ltAMM)
{
ammAccount_ = after->getAccountID(sfAccount);
lptAMMBalanceAfter_ = after->getFieldAmount(sfLPTokenBalance);
}
// AMM pool changed
else if (
(type == ltRIPPLE_STATE && after->getFlags() & lsfAMMNode) ||
(type == ltACCOUNT_ROOT && after->isFieldPresent(sfAMMID)))
{
ammPoolChanged_ = true;
}
}
if (before)
{
// AMM object changed
if (before->getType() == ltAMM)
{
lptAMMBalanceBefore_ = before->getFieldAmount(sfLPTokenBalance);
}
}
}
static bool
validBalances(
STAmount const& amount,
STAmount const& amount2,
STAmount const& lptAMMBalance,
ValidAMM::ZeroAllowed zeroAllowed)
{
bool const positive = amount > beast::zero && amount2 > beast::zero &&
lptAMMBalance > beast::zero;
if (zeroAllowed == ValidAMM::ZeroAllowed::Yes)
return positive ||
(amount == beast::zero && amount2 == beast::zero &&
lptAMMBalance == beast::zero);
return positive;
}
bool
ValidAMM::finalizeVote(bool enforce, beast::Journal const& j) const
{
if (lptAMMBalanceAfter_ != lptAMMBalanceBefore_ || ammPoolChanged_)
{
// LPTokens and the pool can not change on vote
// LCOV_EXCL_START
JLOG(j.error()) << "AMMVote invariant failed: "
<< lptAMMBalanceBefore_.value_or(STAmount{}) << " "
<< lptAMMBalanceAfter_.value_or(STAmount{}) << " "
<< ammPoolChanged_;
if (enforce)
return false;
// LCOV_EXCL_STOP
}
return true;
}
bool
ValidAMM::finalizeBid(bool enforce, beast::Journal const& j) const
{
if (ammPoolChanged_)
{
// The pool can not change on bid
// LCOV_EXCL_START
JLOG(j.error()) << "AMMBid invariant failed: pool changed";
if (enforce)
return false;
// LCOV_EXCL_STOP
}
// LPTokens are burnt, therefore there should be fewer LPTokens
else if (
lptAMMBalanceBefore_ && lptAMMBalanceAfter_ &&
(*lptAMMBalanceAfter_ > *lptAMMBalanceBefore_ ||
*lptAMMBalanceAfter_ <= beast::zero))
{
// LCOV_EXCL_START
JLOG(j.error()) << "AMMBid invariant failed: " << *lptAMMBalanceBefore_
<< " " << *lptAMMBalanceAfter_;
if (enforce)
return false;
// LCOV_EXCL_STOP
}
return true;
}
bool
ValidAMM::finalizeCreate(
STTx const& tx,
ReadView const& view,
bool enforce,
beast::Journal const& j) const
{
if (!ammAccount_)
{
// LCOV_EXCL_START
JLOG(j.error())
<< "AMMCreate invariant failed: AMM object is not created";
if (enforce)
return false;
// LCOV_EXCL_STOP
}
else
{
auto const [amount, amount2] = ammPoolHolds(
view,
*ammAccount_,
tx[sfAmount].get<Issue>(),
tx[sfAmount2].get<Issue>(),
fhIGNORE_FREEZE,
j);
// Create invariant:
// sqrt(amount * amount2) == LPTokens
// all balances are greater than zero
if (!validBalances(
amount, amount2, *lptAMMBalanceAfter_, ZeroAllowed::No) ||
ammLPTokens(amount, amount2, lptAMMBalanceAfter_->issue()) !=
*lptAMMBalanceAfter_)
{
JLOG(j.error()) << "AMMCreate invariant failed: " << amount << " "
<< amount2 << " " << *lptAMMBalanceAfter_;
if (enforce)
return false;
}
}
return true;
}
bool
ValidAMM::finalizeDelete(bool enforce, TER res, beast::Journal const& j) const
{
if (ammAccount_)
{
// LCOV_EXCL_START
std::string const msg = (res == tesSUCCESS)
? "AMM object is not deleted on tesSUCCESS"
: "AMM object is changed on tecINCOMPLETE";
JLOG(j.error()) << "AMMDelete invariant failed: " << msg;
if (enforce)
return false;
// LCOV_EXCL_STOP
}
return true;
}
bool
ValidAMM::finalizeDEX(bool enforce, beast::Journal const& j) const
{
if (ammAccount_)
{
// LCOV_EXCL_START
JLOG(j.error()) << "AMM swap invariant failed: AMM object changed";
if (enforce)
return false;
// LCOV_EXCL_STOP
}
return true;
}
bool
ValidAMM::generalInvariant(
ripple::STTx const& tx,
ripple::ReadView const& view,
ZeroAllowed zeroAllowed,
beast::Journal const& j) const
{
auto const [amount, amount2] = ammPoolHolds(
view,
*ammAccount_,
tx[sfAsset].get<Issue>(),
tx[sfAsset2].get<Issue>(),
fhIGNORE_FREEZE,
j);
// Deposit and Withdrawal invariant:
// sqrt(amount * amount2) >= LPTokens
// all balances are greater than zero
// unless on last withdrawal
auto const poolProductMean = root2(amount * amount2);
bool const nonNegativeBalances =
validBalances(amount, amount2, *lptAMMBalanceAfter_, zeroAllowed);
bool const strongInvariantCheck = poolProductMean >= *lptAMMBalanceAfter_;
// Allow for a small relative error if strongInvariantCheck fails
auto weakInvariantCheck = [&]() {
return *lptAMMBalanceAfter_ != beast::zero &&
withinRelativeDistance(
poolProductMean, Number{*lptAMMBalanceAfter_}, Number{1, -11});
};
if (!nonNegativeBalances ||
(!strongInvariantCheck && !weakInvariantCheck()))
{
JLOG(j.error()) << "AMM " << tx.getTxnType() << " invariant failed: "
<< tx.getHash(HashPrefix::transactionID) << " "
<< ammPoolChanged_ << " " << amount << " " << amount2
<< " " << poolProductMean << " "
<< lptAMMBalanceAfter_->getText() << " "
<< ((*lptAMMBalanceAfter_ == beast::zero)
? Number{1}
: ((*lptAMMBalanceAfter_ - poolProductMean) /
poolProductMean));
return false;
}
return true;
}
bool
ValidAMM::finalizeDeposit(
ripple::STTx const& tx,
ripple::ReadView const& view,
bool enforce,
beast::Journal const& j) const
{
if (!ammAccount_)
{
// LCOV_EXCL_START
JLOG(j.error()) << "AMMDeposit invariant failed: AMM object is deleted";
if (enforce)
return false;
// LCOV_EXCL_STOP
}
else if (!generalInvariant(tx, view, ZeroAllowed::No, j) && enforce)
return false;
return true;
}
bool
ValidAMM::finalizeWithdraw(
ripple::STTx const& tx,
ripple::ReadView const& view,
bool enforce,
beast::Journal const& j) const
{
if (!ammAccount_)
{
// Last Withdraw or Clawback deleted AMM
}
else if (!generalInvariant(tx, view, ZeroAllowed::Yes, j))
{
if (enforce)
return false;
}
return true;
}
bool
ValidAMM::finalize(
STTx const& tx,
TER const result,
XRPAmount const,
ReadView const& view,
beast::Journal const& j)
{
// Delete may return tecINCOMPLETE if there are too many
// trustlines to delete.
if (result != tesSUCCESS && result != tecINCOMPLETE)
return true;
bool const enforce = true; // view.rules().enabled(fixAMMv1_3);
switch (tx.getTxnType())
{
case ttAMM_CREATE:
return finalizeCreate(tx, view, enforce, j);
case ttAMM_DEPOSIT:
return finalizeDeposit(tx, view, enforce, j);
case ttAMM_CLAWBACK:
case ttAMM_WITHDRAW:
return finalizeWithdraw(tx, view, enforce, j);
case ttAMM_BID:
return finalizeBid(enforce, j);
case ttAMM_VOTE:
return finalizeVote(enforce, j);
case ttAMM_DELETE:
return finalizeDelete(enforce, result, j);
case ttCHECK_CASH:
case ttOFFER_CREATE:
case ttPAYMENT:
return finalizeDEX(enforce, j);
default:
break;
}
return true;
}
} // namespace ripple
+65 -1
View File
@@ -617,6 +617,69 @@ public:
beast::Journal const&);
};
class ValidAMM
{
std::optional<AccountID> ammAccount_;
std::optional<STAmount> lptAMMBalanceAfter_;
std::optional<STAmount> lptAMMBalanceBefore_;
bool ammPoolChanged_;
public:
enum class ZeroAllowed : bool { No = false, Yes = true };
ValidAMM() : ammPoolChanged_{false}
{
}
void
visitEntry(
bool,
std::shared_ptr<SLE const> const&,
std::shared_ptr<SLE const> const&);
bool
finalize(
STTx const&,
TER const,
XRPAmount const,
ReadView const&,
beast::Journal const&);
private:
bool
finalizeBid(bool enforce, beast::Journal const&) const;
bool
finalizeVote(bool enforce, beast::Journal const&) const;
bool
finalizeCreate(
STTx const&,
ReadView const&,
bool enforce,
beast::Journal const&) const;
bool
finalizeDelete(bool enforce, TER res, beast::Journal const&) const;
bool
finalizeDeposit(
STTx const&,
ReadView const&,
bool enforce,
beast::Journal const&) const;
// Includes clawback
bool
finalizeWithdraw(
STTx const&,
ReadView const&,
bool enforce,
beast::Journal const&) const;
bool
finalizeDEX(bool enforce, beast::Journal const&) const;
bool
generalInvariant(
STTx const&,
ReadView const&,
ZeroAllowed zeroAllowed,
beast::Journal const&) const;
};
// additional invariant checks can be declared above and then added to this
// tuple
using InvariantChecks = std::tuple<
@@ -636,7 +699,8 @@ using InvariantChecks = std::tuple<
NFTokenCountTracking,
ValidClawback,
ValidMPTIssuance,
ValidPermissionedDomain>;
ValidPermissionedDomain,
ValidAMM>;
/**
* @brief get a tuple of all invariant checks
+16 -1
View File
@@ -21,6 +21,8 @@
#define RIPPLE_APP_BOOK_OFFER_H_INCLUDED
#include <xrpld/ledger/View.h>
#include <xrpl/basics/Log.h>
#include <xrpl/basics/contract.h>
#include <xrpl/protocol/Quality.h>
#include <xrpl/protocol/Rules.h>
@@ -169,8 +171,21 @@ public:
* always returns true.
*/
bool
checkInvariant(TAmounts<TIn, TOut> const&, beast::Journal j) const
checkInvariant(TAmounts<TIn, TOut> const& consumed, beast::Journal j) const
{
if (consumed.in > m_amounts.in || consumed.out > m_amounts.out)
{
// LCOV_EXCL_START
JLOG(j.error())
<< "AMMOffer::checkInvariant failed: consumed "
<< to_string(consumed.in) << " " << to_string(consumed.out)
<< " amounts " << to_string(m_amounts.in) << " "
<< to_string(m_amounts.out);
return false;
// LCOV_EXCL_STOP
}
return true;
}
};
+66
View File
@@ -21,6 +21,8 @@
#define RIPPLE_CORE_COROINL_H_INCLUDED
#include <xrpl/basics/ByteUtilities.h>
#include <boost/asio/steady_timer.hpp>
#include <thread>
namespace ripple {
@@ -48,6 +50,7 @@ JobQueue::Coro::Coro(
},
boost::coroutines::attributes(megabytes(1)))
{
lvs_.coroPtr = this;
}
inline JobQueue::Coro::~Coro()
@@ -57,6 +60,7 @@ inline JobQueue::Coro::~Coro()
#endif
}
//@@start coro-yield
inline void
JobQueue::Coro::yield() const
{
@@ -66,6 +70,7 @@ JobQueue::Coro::yield() const
}
(*yield_)();
}
//@@end coro-yield
inline bool
JobQueue::Coro::post()
@@ -89,6 +94,7 @@ JobQueue::Coro::post()
return false;
}
//@@start coro-resume
inline void
JobQueue::Coro::resume()
{
@@ -113,6 +119,7 @@ JobQueue::Coro::resume()
running_ = false;
cv_.notify_all();
}
//@@end coro-resume
inline bool
JobQueue::Coro::runnable() const
@@ -148,6 +155,65 @@ JobQueue::Coro::join()
cv_.wait(lk, [this]() { return running_ == false; });
}
inline bool
JobQueue::Coro::postAndYield()
{
{
std::lock_guard lk(mutex_run_);
running_ = true;
}
// Flag starts false - will be set true right before yield()
yielding_.store(false, std::memory_order_release);
// Post a job that waits for yield to be ready, then resumes
if (!jq_.addJob(type_, name_, [this, sp = shared_from_this()]() {
// Spin-wait until yield() is about to happen
// yielding_ is set true immediately before (*yield_)() is called
while (!yielding_.load(std::memory_order_acquire))
std::this_thread::yield();
resume();
}))
{
std::lock_guard lk(mutex_run_);
running_ = false;
cv_.notify_all();
return false;
}
// Signal that we're about to yield, then yield
yielding_.store(true, std::memory_order_release);
yield();
// Clear flag after resuming
yielding_.store(false, std::memory_order_release);
return true;
}
inline bool
JobQueue::Coro::sleepFor(std::chrono::milliseconds delay)
{
{
std::lock_guard lk(mutex_run_);
running_ = true;
}
// Use an asio timer on the existing io_service thread pool
// instead of spawning a detached thread per sleep call
auto timer =
std::make_shared<boost::asio::steady_timer>(jq_.io_service_);
timer->expires_after(delay);
timer->async_wait(
[sp = shared_from_this(), timer](
boost::system::error_code const& ec) {
if (ec != boost::asio::error::operation_aborted)
sp->post();
});
yield();
return true;
}
} // namespace ripple
#endif
+21
View File
@@ -26,9 +26,11 @@
#include <xrpld/core/detail/Workers.h>
#include <xrpl/basics/LocalValue.h>
#include <xrpl/json/json_value.h>
#include <boost/asio/io_service.hpp>
#include <boost/coroutine/all.hpp>
#include <boost/range/begin.hpp> // workaround for boost 1.72 bug
#include <boost/range/end.hpp> // workaround for boost 1.72 bug
#include <atomic>
namespace ripple {
@@ -69,6 +71,7 @@ public:
std::condition_variable cv_;
boost::coroutines::asymmetric_coroutine<void>::pull_type coro_;
boost::coroutines::asymmetric_coroutine<void>::push_type* yield_;
std::atomic<bool> yielding_{false}; // For postAndYield synchronization
#ifndef NDEBUG
bool finished_ = false;
#endif
@@ -136,11 +139,28 @@ public:
/** Waits until coroutine returns from the user function. */
void
join();
/** Combined post and yield for poll-wait patterns.
Safely schedules resume before yielding, avoiding race conditions.
@return true if successfully posted and yielded, false if job queue
stopping.
*/
bool
postAndYield();
/** Sleep for a duration without blocking the job queue thread.
Yields the coroutine and schedules resume after the delay.
@param delay The duration to sleep.
@return true if successfully slept, false if job queue stopping.
*/
bool
sleepFor(std::chrono::milliseconds delay);
};
using JobFunction = std::function<void()>;
JobQueue(
boost::asio::io_service& io_service,
int threadCount,
beast::insight::Collector::ptr const& collector,
beast::Journal journal,
@@ -242,6 +262,7 @@ private:
using JobDataMap = std::map<JobType, JobTypeData>;
boost::asio::io_service& io_service_;
beast::Journal m_journal;
mutable std::mutex m_mutex;
std::uint64_t m_lastJob;
+3 -1
View File
@@ -25,12 +25,14 @@
namespace ripple {
JobQueue::JobQueue(
boost::asio::io_service& io_service,
int threadCount,
beast::insight::Collector::ptr const& collector,
beast::Journal journal,
Logs& logs,
perf::PerfLog& perfLog)
: m_journal(journal)
: io_service_(io_service)
, m_journal(journal)
, m_lastJob(0)
, m_invalidJobData(JobTypes::instance().getInvalid(), collector, logs)
, m_processCount(0)
+33 -4
View File
@@ -2493,6 +2493,12 @@ PeerImp::onMessage(std::shared_ptr<protocol::TMGetObjectByHash> const& m)
bool pLDo = true;
bool progress = false;
// For state/transaction node requests, store directly to db
// (not fetch pack) so partial sync queries can find them immediately
bool const directStore =
packet.type() == protocol::TMGetObjectByHash::otSTATE_NODE ||
packet.type() == protocol::TMGetObjectByHash::otTRANSACTION_NODE;
for (int i = 0; i < packet.objects_size(); ++i)
{
const protocol::TMIndexedObject& obj = packet.objects(i);
@@ -2525,10 +2531,33 @@ PeerImp::onMessage(std::shared_ptr<protocol::TMGetObjectByHash> const& m)
{
uint256 const hash{obj.hash()};
app_.getLedgerMaster().addFetchPack(
hash,
std::make_shared<Blob>(
obj.data().begin(), obj.data().end()));
if (directStore)
{
// Store directly to node store for immediate
// availability
auto const hotType =
(packet.type() ==
protocol::TMGetObjectByHash::otSTATE_NODE)
? hotACCOUNT_NODE
: hotTRANSACTION_NODE;
JLOG(p_journal_.warn())
<< "PRIORITY: received node " << hash << " for seq "
<< pLSeq << " storing to db";
app_.getNodeStore().store(
hotType,
Blob(obj.data().begin(), obj.data().end()),
hash,
pLSeq);
}
else
{
app_.getLedgerMaster().addFetchPack(
hash,
std::make_shared<Blob>(
obj.data().begin(), obj.data().end()));
}
}
}
}
+1
View File
@@ -174,6 +174,7 @@ Handler const handlerArray[]{
byRef(&doSubmitMultiSigned),
Role::USER,
NEEDS_CURRENT_LEDGER},
{"submit_and_wait", byRef(&doSubmitAndWait), Role::USER, NO_CONDITION},
{"transaction_entry", byRef(&doTransactionEntry), Role::USER, NO_CONDITION},
{"tx", byRef(&doTxJson), Role::USER, NEEDS_NETWORK_CONNECTION},
{"tx_history", byRef(&doTxHistory), Role::USER, NO_CONDITION, 1, 1},
+2
View File
@@ -93,6 +93,7 @@ conditionMet(Condition condition_required, T& context)
return rpcEXPIRED_VALIDATOR_LIST;
}
//@@start network-condition-check
if ((condition_required != NO_CONDITION) &&
(context.netOps.getOperatingMode() < OperatingMode::SYNCING))
{
@@ -103,6 +104,7 @@ conditionMet(Condition condition_required, T& context)
return rpcNO_NETWORK;
return rpcNOT_SYNCED;
}
//@@end network-condition-check
if (!context.app.config().standalone() &&
condition_required != NO_CONDITION)
+92 -7
View File
@@ -17,6 +17,7 @@
*/
//==============================================================================
#include <xrpld/app/ledger/InboundLedgers.h>
#include <xrpld/app/ledger/LedgerMaster.h>
#include <xrpld/app/ledger/LedgerToJson.h>
#include <xrpld/app/ledger/OpenLedger.h>
@@ -28,6 +29,7 @@
#include <xrpld/rpc/Context.h>
#include <xrpld/rpc/DeliveredAmount.h>
#include <xrpld/rpc/detail/RPCHelpers.h>
#include <xrpl/basics/LocalValue.h>
#include <xrpl/protocol/AccountID.h>
#include <xrpl/protocol/Feature.h>
#include <xrpl/protocol/RPCErr.h>
@@ -37,7 +39,6 @@
#include <boost/algorithm/string/case_conv.hpp>
#include <boost/algorithm/string/predicate.hpp>
#include <xrpl/resource/Fees.h>
#include <regex>
namespace ripple {
@@ -574,6 +575,11 @@ Status
getLedger(T& ledger, uint256 const& ledgerHash, Context& context)
{
ledger = context.ledgerMaster.getLedgerByHash(ledgerHash);
if (ledger == nullptr)
{
// Partial sync fallback: try to get incomplete ledger being acquired
ledger = context.app.getInboundLedgers().getPartialLedger(ledgerHash);
}
if (ledger == nullptr)
return {rpcLGR_NOT_FOUND, "ledgerNotFound"};
return Status::OK;
@@ -611,6 +617,14 @@ getLedger(T& ledger, uint32_t ledgerIndex, Context& context)
}
}
// Partial sync fallback: try to get incomplete ledger being acquired
if (ledger == nullptr)
{
auto hash = context.ledgerMaster.getHashBySeq(ledgerIndex);
if (hash.isNonZero())
ledger = context.app.getInboundLedgers().getPartialLedger(hash);
}
if (ledger == nullptr)
return {rpcLGR_NOT_FOUND, "ledgerNotFound"};
@@ -633,16 +647,87 @@ template <class T>
Status
getLedger(T& ledger, LedgerShortcut shortcut, Context& context)
{
if (isValidatedOld(context.ledgerMaster, context.app.config().standalone()))
{
if (context.apiVersion == 1)
return {rpcNO_NETWORK, "InsufficientNetworkMode"};
return {rpcNOT_SYNCED, "notSynced"};
}
//@@start sync-validation
// TODO: Re-enable for production. Disabled for partial sync testing.
// if (isValidatedOld(context.ledgerMaster,
// context.app.config().standalone()))
// {
// if (context.apiVersion == 1)
// return {rpcNO_NETWORK, "InsufficientNetworkMode"};
// return {rpcNOT_SYNCED, "notSynced"};
// }
//@@end sync-validation
if (shortcut == LedgerShortcut::VALIDATED)
{
ledger = context.ledgerMaster.getValidatedLedger();
// Partial sync fallback: try to get incomplete validated ledger
if (ledger == nullptr)
{
auto [hash, seq] = context.ledgerMaster.getLastValidatedLedger();
JLOG(context.j.warn())
<< "Partial sync: getValidatedLedger null, trying trusted hash="
<< hash << " seq=" << seq;
// If no trusted validations yet, try network-observed ledger
if (hash.isZero())
{
std::tie(hash, seq) =
context.ledgerMaster.getNetworkObservedLedger();
JLOG(context.j.warn())
<< "Partial sync: trying network-observed hash=" << hash
<< " seq=" << seq;
// Poll-wait for validations to arrive (up to ~10 seconds)
if (hash.isZero() && context.coro)
{
for (int i = 0; i < 100 && hash.isZero(); ++i)
{
context.coro->sleepFor(std::chrono::milliseconds(100));
std::tie(hash, seq) =
context.ledgerMaster.getNetworkObservedLedger();
}
if (hash.isNonZero())
{
JLOG(context.j.warn())
<< "Partial sync: got network-observed hash="
<< hash << " seq=" << seq;
}
}
}
if (hash.isNonZero())
{
setPartialSyncWait(true);
ledger = context.app.getInboundLedgers().getPartialLedger(hash);
// If no InboundLedger exists yet, trigger acquisition and wait
if (!ledger)
{
JLOG(context.j.warn())
<< "Partial sync: acquiring ledger " << hash;
context.app.getInboundLedgers().acquire(
hash, seq, InboundLedger::Reason::CONSENSUS);
// Poll-wait for the ledger header (up to ~10 seconds)
int i = 0;
for (; i < 100 && !ledger && context.coro; ++i)
{
context.coro->sleepFor(std::chrono::milliseconds(100));
ledger =
context.app.getInboundLedgers().getPartialLedger(
hash);
}
JLOG(context.j.warn())
<< "Partial sync: poll-wait completed after " << i
<< " iterations, ledger="
<< (ledger ? "found" : "null");
}
}
JLOG(context.j.warn()) << "Partial sync: getPartialLedger returned "
<< (ledger ? "ledger" : "null");
}
if (ledger == nullptr)
{
if (context.apiVersion == 1)
+2
View File
@@ -315,12 +315,14 @@ ServerHandler::onRequest(Session& session)
}
std::shared_ptr<Session> detachedSession = session.detach();
//@@start rpc-coro-usage
auto const postResult = m_jobQueue.postCoro(
jtCLIENT_RPC,
"RPC-Client",
[this, detachedSession](std::shared_ptr<JobQueue::Coro> coro) {
processSession(detachedSession, coro);
});
//@@end rpc-coro-usage
if (postResult == nullptr)
{
// The coroutine was rejected, probably because we're shutting down.
+2
View File
@@ -147,6 +147,8 @@ doSubmit(RPC::JsonContext&);
Json::Value
doSubmitMultiSigned(RPC::JsonContext&);
Json::Value
doSubmitAndWait(RPC::JsonContext&);
Json::Value
doSubscribe(RPC::JsonContext&);
Json::Value
doTransactionEntry(RPC::JsonContext&);
+335
View File
@@ -0,0 +1,335 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2024 XRPL Labs
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <xrpld/app/consensus/RCLValidations.h>
#include <xrpld/app/ledger/InboundLedgers.h>
#include <xrpld/app/ledger/LedgerMaster.h>
#include <xrpld/app/main/Application.h>
#include <xrpld/app/misc/NetworkOPs.h>
#include <xrpld/app/misc/ValidatorList.h>
#include <xrpld/rpc/Context.h>
#include <xrpld/rpc/DeliveredAmount.h>
#include <xrpld/rpc/detail/RPCHelpers.h>
#include <xrpl/basics/LocalValue.h>
#include <xrpl/basics/StringUtilities.h>
#include <xrpl/protocol/ErrorCodes.h>
#include <xrpl/protocol/RPCErr.h>
#include <xrpl/protocol/jss.h>
namespace ripple {
// Custom journal partition for submit_and_wait debugging
// Configure with [rpc_startup] { "command": "log_level", "partition":
// "SubmitAndWait", "severity": "debug" }
#define SWLOG(level) JLOG(context.app.journal("SubmitAndWait").level())
// {
// tx_blob: <hex-encoded signed transaction>
// timeout: <optional, max wait time in seconds, default 60>
// }
//
// Submit a transaction and wait for it to appear in a VALIDATED ledger.
// Designed for partial sync mode where the node may not have full state
// to validate locally - broadcasts raw transaction and monitors incoming
// ledgers for the result.
//
// The handler waits until:
// 1. Transaction is found in a ledger, AND
// 2. That ledger reaches validation quorum (enough trusted validators)
//
// Response:
// "validated": true - Transaction confirmed in validated ledger
// "error": "timeout" - Timeout waiting
// "error": "expired" - LastLedgerSequence exceeded
Json::Value
doSubmitAndWait(RPC::JsonContext& context)
{
Json::Value jvResult;
// Must have coroutine for polling
if (!context.coro)
{
return RPC::make_error(
rpcINTERNAL, "submit_and_wait requires coroutine context");
}
// Parse tx_blob
if (!context.params.isMember(jss::tx_blob))
{
return rpcError(rpcINVALID_PARAMS);
}
auto const txBlobHex = context.params[jss::tx_blob].asString();
auto const txBlob = strUnHex(txBlobHex);
if (!txBlob || txBlob->empty())
{
return rpcError(rpcINVALID_PARAMS);
}
// Parse the transaction to get hash and LastLedgerSequence
std::shared_ptr<STTx const> stx;
try
{
SerialIter sit(makeSlice(*txBlob));
stx = std::make_shared<STTx const>(std::ref(sit));
}
catch (std::exception& e)
{
jvResult[jss::error] = "invalidTransaction";
jvResult[jss::error_exception] = e.what();
return jvResult;
}
uint256 const txHash = stx->getTransactionID();
// Extract LastLedgerSequence if present
std::optional<std::uint32_t> lastLedgerSeq;
if (stx->isFieldPresent(sfLastLedgerSequence))
{
lastLedgerSeq = stx->getFieldU32(sfLastLedgerSequence);
}
// Parse timeout (default 60 seconds, max 120 seconds)
auto timeout = std::chrono::seconds(60);
if (context.params.isMember("timeout"))
{
auto const t = context.params["timeout"].asUInt();
if (t > 120)
{
return RPC::make_error(
rpcINVALID_PARAMS, "timeout must be <= 120 seconds");
}
timeout = std::chrono::seconds(t);
}
// Enable partial sync wait for SHAMap operations
setPartialSyncWait(true);
setCoroFetchTimeout(
std::chrono::duration_cast<std::chrono::milliseconds>(timeout / 2));
SWLOG(warn) << "starting for tx=" << txHash
<< " lastLedgerSeq=" << (lastLedgerSeq ? *lastLedgerSeq : 0)
<< " timeout=" << timeout.count() << "s";
// Poll for the transaction result
constexpr auto pollInterval = std::chrono::milliseconds(10);
auto const startTime = std::chrono::steady_clock::now();
// Broadcast IMMEDIATELY - don't wait for anything
SWLOG(warn) << "broadcasting tx=" << txHash;
auto broadcastResult = context.netOps.broadcastRawTransaction(*txBlob);
if (!broadcastResult)
{
SWLOG(warn) << "broadcast FAILED for tx=" << txHash;
jvResult[jss::error] = "broadcastFailed";
jvResult[jss::error_exception] =
"Failed to parse/broadcast transaction";
return jvResult;
}
SWLOG(warn) << "broadcast SUCCESS for tx=" << txHash;
// Prioritize TX fetching for ledgers in our window
// This makes TX nodes fetch before state nodes for faster detection
auto const startSeq = context.ledgerMaster.getValidLedgerIndex();
auto const endSeq = lastLedgerSeq.value_or(startSeq + 20);
context.app.getInboundLedgers().prioritizeTxForLedgers(startSeq, endSeq);
jvResult[jss::tx_hash] = to_string(txHash);
jvResult[jss::broadcast] = true;
// Track when we find the tx and in which ledger
std::optional<uint256> foundLedgerHash;
std::optional<std::uint32_t> foundLedgerSeq;
// Track last checked seq to avoid rescanning old ledgers
auto lastCheckedSeq = startSeq;
// Helper to check if a ledger is validated (has quorum)
auto isLedgerValidated = [&](uint256 const& ledgerHash) -> bool {
auto const quorum = context.app.validators().quorum();
if (quorum == 0)
return false; // No validators configured
auto const valCount =
context.app.getValidations().numTrustedForLedger(ledgerHash);
return valCount >= quorum;
};
// Helper to read tx result from a ledger
auto readTxResult = [&](std::shared_ptr<Ledger const> const& ledger,
std::string const& source) -> bool {
if (!ledger)
return false;
auto [sttx, stobj] = ledger->txRead(txHash);
if (!sttx || !stobj)
return false;
jvResult[jss::status] = "success";
jvResult[jss::validated] = true;
jvResult["found_via"] = source;
jvResult[jss::tx_json] = sttx->getJson(JsonOptions::none);
jvResult[jss::metadata] = stobj->getJson(JsonOptions::none);
jvResult[jss::ledger_hash] = to_string(ledger->info().hash);
jvResult[jss::ledger_index] = ledger->info().seq;
// Extract result code from metadata
if (stobj->isFieldPresent(sfTransactionResult))
{
auto const result =
TER::fromInt(stobj->getFieldU8(sfTransactionResult));
std::string token;
std::string human;
transResultInfo(result, token, human);
jvResult[jss::engine_result] = token;
jvResult[jss::engine_result_code] = TERtoInt(result);
jvResult[jss::engine_result_message] = human;
}
return true;
};
while (true)
{
auto const elapsed = std::chrono::steady_clock::now() - startTime;
if (elapsed >= timeout)
{
jvResult[jss::error] = "transactionTimeout";
jvResult[jss::error_message] =
"Transaction not validated within timeout period";
if (foundLedgerSeq)
{
jvResult["found_in_ledger"] = *foundLedgerSeq;
auto const valCount =
context.app.getValidations().numTrustedForLedger(
*foundLedgerHash);
auto const quorum = context.app.validators().quorum();
jvResult["validation_count"] =
static_cast<unsigned int>(valCount);
jvResult["quorum"] = static_cast<unsigned int>(quorum);
}
return jvResult;
}
// If we already found the tx, check if its ledger is now validated
if (foundLedgerHash)
{
if (isLedgerValidated(*foundLedgerHash))
{
// Ledger is validated! Try to read from InboundLedgers first
auto ledger = context.app.getInboundLedgers().getPartialLedger(
*foundLedgerHash);
if (ledger && readTxResult(ledger, "InboundLedgers"))
{
return jvResult;
}
// Try LedgerMaster (for when synced)
if (foundLedgerSeq)
{
ledger =
context.ledgerMaster.getLedgerBySeq(*foundLedgerSeq);
if (ledger && readTxResult(ledger, "LedgerMaster"))
{
return jvResult;
}
}
// Ledger validated but can't read yet - keep waiting
}
}
else
{
auto const currentValidatedSeq =
context.ledgerMaster.getValidLedgerIndex();
// Search InboundLedgers for the tx (partial sync mode)
auto const ledgerHash =
context.app.getInboundLedgers().findTxLedger(txHash);
if (ledgerHash)
{
auto const ledger =
context.app.getInboundLedgers().getPartialLedger(
*ledgerHash);
if (ledger)
{
foundLedgerHash = ledgerHash;
foundLedgerSeq = ledger->info().seq;
SWLOG(warn) << "FOUND tx in InboundLedgers seq="
<< ledger->info().seq;
if (isLedgerValidated(*ledgerHash))
{
if (readTxResult(ledger, "InboundLedgers"))
{
return jvResult;
}
}
}
}
// Search LedgerMaster for the tx (synced mode via gossip)
// Only check new ledgers since last iteration
if (!foundLedgerHash)
{
for (auto seq = lastCheckedSeq; seq <= currentValidatedSeq;
++seq)
{
auto ledger = context.ledgerMaster.getLedgerBySeq(seq);
if (ledger)
{
auto [sttx, stobj] = ledger->txRead(txHash);
if (sttx && stobj)
{
foundLedgerHash = ledger->info().hash;
foundLedgerSeq = seq;
SWLOG(warn)
<< "FOUND tx in LedgerMaster seq=" << seq;
// LedgerMaster ledgers are already validated
if (readTxResult(ledger, "LedgerMaster"))
{
return jvResult;
}
}
}
}
lastCheckedSeq = currentValidatedSeq + 1;
}
// Check LastLedgerSequence expiry
if (lastLedgerSeq && currentValidatedSeq > *lastLedgerSeq)
{
jvResult[jss::error] = "transactionExpired";
jvResult[jss::error_message] =
"LastLedgerSequence exceeded and transaction not found";
jvResult["last_ledger_sequence"] = *lastLedgerSeq;
jvResult["validated_ledger"] = currentValidatedSeq;
return jvResult;
}
}
// Sleep and continue polling
context.coro->sleepFor(pollInterval);
}
}
} // namespace ripple
+6 -1
View File
@@ -68,9 +68,14 @@ public:
*
* @param refNum Sequence of ledger to acquire.
* @param nodeHash Hash of missing node to report in throw.
* @param prioritize If true, prioritize fetching this specific node
* (used by partial sync mode for RPC queries).
*/
virtual void
missingNodeAcquireBySeq(std::uint32_t refNum, uint256 const& nodeHash) = 0;
missingNodeAcquireBySeq(
std::uint32_t refNum,
uint256 const& nodeHash,
bool prioritize = false) = 0;
/** Acquire ledger that has a missing node by ledger hash
*
+4 -1
View File
@@ -79,7 +79,10 @@ public:
reset() override;
void
missingNodeAcquireBySeq(std::uint32_t seq, uint256 const& hash) override;
missingNodeAcquireBySeq(
std::uint32_t seq,
uint256 const& hash,
bool prioritize = false) override;
void
missingNodeAcquireByHash(uint256 const& hash, std::uint32_t seq) override
+2
View File
@@ -213,6 +213,7 @@ SHAMapInnerNode::getBranchCount() const
return popcnt16(isBranch_);
}
//@@start full-below-methods
inline bool
SHAMapInnerNode::isFullBelow(std::uint32_t generation) const
{
@@ -224,6 +225,7 @@ SHAMapInnerNode::setFullBelowGen(std::uint32_t gen)
{
fullBelowGen_ = gen;
}
//@@end full-below-methods
} // namespace ripple
#endif
+4
View File
@@ -29,11 +29,13 @@
namespace ripple {
//@@start shamap-type-enum
enum class SHAMapType {
TRANSACTION = 1, // A tree of transactions
STATE = 2, // A tree of state nodes
FREE = 3, // A tree not part of a ledger
};
//@@end shamap-type-enum
inline std::string
to_string(SHAMapType t)
@@ -52,6 +54,7 @@ to_string(SHAMapType t)
}
}
//@@start shamap-missing-node-class
class SHAMapMissingNode : public std::runtime_error
{
public:
@@ -67,6 +70,7 @@ public:
{
}
};
//@@end shamap-missing-node-class
} // namespace ripple
+11 -3
View File
@@ -21,7 +21,6 @@
#include <xrpld/app/main/Application.h>
#include <xrpld/app/main/Tuning.h>
#include <xrpld/shamap/NodeFamily.h>
#include <sstream>
namespace ripple {
@@ -66,9 +65,18 @@ NodeFamily::reset()
}
void
NodeFamily::missingNodeAcquireBySeq(std::uint32_t seq, uint256 const& nodeHash)
NodeFamily::missingNodeAcquireBySeq(
std::uint32_t seq,
uint256 const& nodeHash,
bool prioritize)
{
JLOG(j_.error()) << "Missing node in " << seq;
JLOG(j_.error()) << "Missing node in " << seq << " hash=" << nodeHash
<< (prioritize ? " [PRIORITY]" : "");
// Add priority for the specific node hash needed by the query
if (prioritize && nodeHash.isNonZero())
app_.getInboundLedgers().addPriorityNode(seq, nodeHash);
std::unique_lock<std::mutex> lock(maxSeqMutex_);
if (maxSeq_ == 0)
{
+77
View File
@@ -17,13 +17,16 @@
*/
//==============================================================================
#include <xrpld/core/JobQueue.h>
#include <xrpld/shamap/SHAMap.h>
#include <xrpld/shamap/SHAMapAccountStateLeafNode.h>
#include <xrpld/shamap/SHAMapNodeID.h>
#include <xrpld/shamap/SHAMapSyncFilter.h>
#include <xrpld/shamap/SHAMapTxLeafNode.h>
#include <xrpld/shamap/SHAMapTxPlusMetaLeafNode.h>
#include <xrpl/basics/LocalValue.h>
#include <xrpl/basics/contract.h>
#include <chrono>
namespace ripple {
@@ -154,6 +157,7 @@ SHAMap::walkTowardsKey(uint256 const& id, SharedPtrNodeStack* stack) const
return static_cast<SHAMapLeafNode*>(inNode.get());
}
//@@start find-key
SHAMapLeafNode*
SHAMap::findKey(uint256 const& id) const
{
@@ -162,6 +166,7 @@ SHAMap::findKey(uint256 const& id) const
leaf = nullptr;
return leaf;
}
//@@end find-key
std::shared_ptr<SHAMapTreeNode>
SHAMap::fetchNodeFromDB(SHAMapHash const& hash) const
@@ -187,6 +192,70 @@ SHAMap::finishFetch(
full_ = false;
f_.missingNodeAcquireBySeq(ledgerSeq_, hash.as_uint256());
}
// If partial sync wait is enabled, poll-wait for the node
if (isPartialSyncWaitEnabled())
if (auto* coro =
static_cast<JobQueue::Coro*>(getCurrentCoroPtr()))
{
using namespace std::chrono;
constexpr auto pollInterval = 50ms;
constexpr auto defaultTimeout = 30s;
// Use coroutine-local timeout if set, otherwise default
auto coroTimeout = getCoroFetchTimeout();
auto timeout =
coroTimeout.count() > 0 ? coroTimeout : defaultTimeout;
auto const deadline = steady_clock::now() + timeout;
// Linear backoff for re-requests: 50ms, 100ms, 150ms... up
// to 2s
auto nextRequestDelay = 50ms;
constexpr auto maxRequestDelay = 2000ms;
constexpr auto backoffStep = 50ms;
auto nextRequestTime =
steady_clock::now() + nextRequestDelay;
JLOG(journal_.debug())
<< "finishFetch: waiting for node " << hash;
while (steady_clock::now() < deadline)
{
// Sleep for the poll interval (yields coroutine, frees
// job thread)
coro->sleepFor(pollInterval);
// Try to fetch from cache/db again
if (auto obj = f_.db().fetchNodeObject(
hash.as_uint256(), ledgerSeq_))
{
JLOG(journal_.debug())
<< "finishFetch: got node " << hash;
auto node = SHAMapTreeNode::makeFromPrefix(
makeSlice(obj->getData()), hash);
if (node)
canonicalize(hash, node);
return node;
}
// Re-request with priority using linear backoff
auto now = steady_clock::now();
if (now >= nextRequestTime)
{
f_.missingNodeAcquireBySeq(
ledgerSeq_,
hash.as_uint256(),
true /*prioritize*/);
// Increase delay for next request (linear backoff)
if (nextRequestDelay < maxRequestDelay)
nextRequestDelay += backoffStep;
nextRequestTime = now + nextRequestDelay;
}
}
JLOG(journal_.warn())
<< "finishFetch: timeout waiting for node " << hash;
}
return {};
}
@@ -268,6 +337,7 @@ SHAMap::fetchNodeNT(SHAMapHash const& hash, SHAMapSyncFilter* filter) const
}
*/
//@@start fetch-with-timeout
std::shared_ptr<SHAMapTreeNode>
SHAMap::fetchNodeNT(SHAMapHash const& hash, SHAMapSyncFilter* filter) const
{
@@ -309,6 +379,7 @@ SHAMap::fetchNodeNT(SHAMapHash const& hash, SHAMapSyncFilter* filter) const
return nullptr;
}
//@@end fetch-with-timeout
std::shared_ptr<SHAMapTreeNode>
SHAMap::fetchNodeNT(SHAMapHash const& hash) const
@@ -333,6 +404,7 @@ SHAMap::fetchNode(SHAMapHash const& hash) const
return node;
}
//@@start throw-on-missing
SHAMapTreeNode*
SHAMap::descendThrow(SHAMapInnerNode* parent, int branch) const
{
@@ -343,6 +415,7 @@ SHAMap::descendThrow(SHAMapInnerNode* parent, int branch) const
return ret;
}
//@@end throw-on-missing
std::shared_ptr<SHAMapTreeNode>
SHAMap::descendThrow(std::shared_ptr<SHAMapInnerNode> const& parent, int branch)
@@ -435,6 +508,7 @@ SHAMap::descend(
return std::make_pair(child, parentID.getChildNodeID(branch));
}
//@@start async-fetch
SHAMapTreeNode*
SHAMap::descendAsync(
SHAMapInnerNode* parent,
@@ -457,6 +531,7 @@ SHAMap::descendAsync(
if (filter)
ptr = checkFilter(hash, filter);
//@@start db-async-fetch
if (!ptr && backed_)
{
f_.db().asyncFetch(
@@ -470,6 +545,7 @@ SHAMap::descendAsync(
pending = true;
return nullptr;
}
//@@end db-async-fetch
}
if (ptr)
@@ -477,6 +553,7 @@ SHAMap::descendAsync(
return ptr.get();
}
//@@end async-fetch
template <class Node>
std::shared_ptr<Node>