Compare commits

...
Author SHA1 Message Date
tequ 86ac135c63 refactor: include transactor headers from transactions.macro
Partial cherry-pick of XRPLF/rippled#5590. Each TRANSACTION entry now
carries its transactor header behind `#if TRANSACTION_INCLUDE`, and
applySteps.cpp pulls them in by expanding the macro with
TRANSACTION_INCLUDE defined. Adding a transaction type no longer
requires touching the include list in applySteps.cpp.

ApplyContext.h (reached via Transactor.h) and XahauGenesis.h (unused
there) are dropped from applySteps.cpp.
2026-10-22 15:44:30 +09:00
Richard Holland 3dc41b2a59 ignore amendment cfg option (#900)
Use like so, on testnet:
[ignore_amendments]
HookOnV2
2026-10-17 09:19:39 +10:00
Richard Holland b9f2914c0e catalogue speed and memory fixes (#883) 2026-10-14 14:13:37 +10:00
tequ 023f55e283 fix: keep ledger_history in standalone mode for rwdb null node store (#893) 2026-10-14 12:30:51 +10:00
tequandRichard Holland d20e8ffb56 fix(hook): crash on the transaction after removing a HookName (#890)
* fix(hook): Read sfHookName via isFieldPresent instead of operator[]

SetHook removes a hook's name with makeFieldAbsent on an untemplated
sfHooks entry, which leaves an STI_NOTPRESENT placeholder for
sfHookName. STObject::at(~sfHookName) then fails the dynamic_cast,
dereferences the null template and asserts (null deref in release).

Every other optional field of an sfHooks entry is already read with
isFieldPresent/getField*, which treats the placeholder as absent.
Align the three sfHookName readers with that convention:

- Transactor::calculateHookChainFee
- Transactor::executeHookChain
- ClaimReward::preclaim (fix20260929 path)

hook::canHook now takes std::optional<Blob> so callers can pass the
copied value without a dangling Slice.

Add regression tests that remove a hook name with a HookName-only
SetHook update and then submit the next transaction (Invoke and
ClaimReward). Both crashed before this change.

* claude fixes

---------

Co-authored-by: Richard Holland <[email protected]>
2026-10-13 10:57:14 +10:00
Richard Holland c63ee8cc5f fix mkdir in release builder (#889) 2026-10-12 13:19:11 +10:00
Richard Holland 64a30b2a15 Various fixes (#888) 2026-10-12 12:48:34 +10:00
Richard Holland 0025355f43 UDP Superhighway (#877) 2026-10-12 10:54:31 +10:00
Richard Holland 7046616cdb FeatureRNG (#869) 2026-10-12 10:49:46 +10:00
117 changed files with 9108 additions and 534 deletions

No files matched your search

+1 -1
View File
@@ -26,7 +26,7 @@ Loop: xrpld.app xrpld.nodestore
xrpld.app > xrpld.nodestore
Loop: xrpld.app xrpld.overlay
xrpld.overlay ~= xrpld.app
xrpld.overlay == xrpld.app
Loop: xrpld.app xrpld.peerfinder
xrpld.app > xrpld.peerfinder
+2
View File
@@ -103,6 +103,7 @@ test.overlay > xrpld.overlay
test.overlay > xrpld.peerfinder
test.overlay > xrpld.shamap
test.overlay > xrpl.protocol
test.overlay > xrpl.resource
test.peerfinder > test.beast
test.peerfinder > test.unit_test
test.peerfinder > xrpl.basics
@@ -126,6 +127,7 @@ test.rpc > xrpl.basics
test.rpc > xrpld.app
test.rpc > xrpld.core
test.rpc > xrpld.net
test.rpc > xrpld.nodestore
test.rpc > xrpld.overlay
test.rpc > xrpld.rpc
test.rpc > xrpld.shamap
+1 -1
View File
@@ -97,7 +97,7 @@ else
# GH Action, runner
if [[ "$(git rev-parse --abbrev-ref HEAD)" == "release" ]]; then
echo "building on the release branch... placing it in builds/candidate"
mkdir /data/builds/candidate
mkdir -p /data/builds/candidate
cp /io/release-build/xahaud /data/builds/candidate/$(date +%Y).$(date +%-m).$(date +%-d)-$(git rev-parse --abbrev-ref HEAD)+$4
cp /io/release-build/release.info /data/builds/candidate/$(date +%Y).$(date +%-m).$(date +%-d)-$(git rev-parse --abbrev-ref HEAD)+$4.releaseinfo
else
+24
View File
@@ -564,6 +564,30 @@
#
#
#
# [udp_highway]
#
# 0 or 1.
#
# 0: Disable the Xahau UDP Superhighway (XUSH). [default]
# 1: Enable the Xahau UDP Superhighway (XUSH).
#
# The UDP Superhighway is a best-effort transport that runs alongside the
# peer protocol. When it is enabled the server also listens for UDP
# datagrams on its peer port number, and every transaction it relays to its
# peers is also sent, over UDP, to a small random set of other servers that
# have the highway enabled. It never replaces the peer protocol: anything
# sent over the highway is also relayed normally.
#
# To receive highway traffic, inbound UDP must be allowed to the peer port
# (in addition to inbound TCP).
#
# A server that keeps its address private (peer_private = 1, or any server
# configured as a validator) only exchanges highway traffic with the peers
# listed in [ips_fixed], and never advertises highway endpoints. Fixed peers
# are never advertised to other servers.
#
#
#
# [peers_max]
#
# The largest number of desired peer connections (incoming or outgoing).
-1
View File
@@ -151,4 +151,3 @@ D686F2538F410C9D0D856788E98E3579595DAF7B38D38887F81ECAC934B06040 HooksUpdate1
4B8466415FAB32FFA89D9DCBE166A42340115771DF611A7160F8D7439C87ECD8 fixNSDelete
EDB4EE4C524E16BDD91D9A529332DED08DCAAA51CC6DC897ACFA1A0ED131C5B6 fix240819
8063140E9260799D6716756B891CEC3E7006C4E4F277AB84670663A88F94B9C4 fixPageCap
88693F108C3CD8A967F3F4253A32DEF5E35F9406ACD2A11B88B11D90865763A9 fix240911
+1
View File
@@ -47,5 +47,6 @@
#define MEM_OVERLAP -43
#define TOO_MANY_STATE_MODIFICATIONS -44
#define TOO_MANY_NAMESPACES -45
#define TOO_LITTLE_ENTROPY -46
#define HOOK_ERROR_CODES
#endif //HOOK_ERROR_CODES
+6
View File
@@ -339,6 +339,12 @@ prepare(
uint32_t read_ptr,
uint32_t read_len);
extern int64_t
dice(uint32_t sides);
extern int64_t
util_random(uint32_t write_ptr, uint32_t write_len);
#ifdef __cplusplus
}
#endif
+6
View File
@@ -22,6 +22,8 @@
#define sfHookApiVersion ((1U << 16U) + 20U)
#define sfHookStateScale ((1U << 16U) + 21U)
#define sfLedgerFixType ((1U << 16U) + 22U)
#define sfEntropyCount ((1U << 16U) + 99U)
#define sfEntropyDenominator ((1U << 16U) + 100U)
#define sfNetworkID ((2U << 16U) + 1U)
#define sfFlags ((2U << 16U) + 2U)
#define sfSourceTag ((2U << 16U) + 3U)
@@ -153,6 +155,8 @@
#define sfEscrowID ((5U << 16U) + 35U)
#define sfURITokenID ((5U << 16U) + 36U)
#define sfDomainID ((5U << 16U) + 37U)
#define sfNextRandomDigest ((5U << 16U) + 89U)
#define sfRandomData ((5U << 16U) + 90U)
#define sfManifestID ((5U << 16U) + 91U)
#define sfHookOnOutgoing ((5U << 16U) + 93U)
#define sfHookOnIncoming ((5U << 16U) + 94U)
@@ -291,6 +295,7 @@
#define sfXChainCreateAccountAttestationCollectionElement ((14U << 16U) + 31U)
#define sfPriceData ((14U << 16U) + 32U)
#define sfCredential ((14U << 16U) + 33U)
#define sfRandomDigestEntry ((14U << 16U) + 88U)
#define sfManifest ((14U << 16U) + 90U)
#define sfAmountEntry ((14U << 16U) + 91U)
#define sfMintURIToken ((14U << 16U) + 92U)
@@ -323,6 +328,7 @@
#define sfAuthorizeCredentials ((15U << 16U) + 26U)
#define sfUnauthorizeCredentials ((15U << 16U) + 27U)
#define sfAcceptedCredentials ((15U << 16U) + 28U)
#define sfRandomDigests ((15U << 16U) + 91U)
#define sfAmounts ((15U << 16U) + 92U)
#define sfHookEmissions ((15U << 16U) + 93U)
#define sfImportVLKeys ((15U << 16U) + 94U)
+1
View File
@@ -61,6 +61,7 @@
#define ttNFTOKEN_MODIFY 70
#define ttPERMISSIONED_DOMAIN_SET 71
#define ttPERMISSIONED_DOMAIN_DELETE 72
#define ttENTROPY 89
#define ttMANIFEST_SET 91
#define ttCRON 92
#define ttCRON_SET 93
+5 -1
View File
@@ -15,6 +15,7 @@
#define uint256 std::string
#define featureHooksUpdate1 "1"
#define featureHooksUpdate2 "1"
#define featureRNG "1"
#define fix20250131 "1"
#define fixGuardDepth32 "1"
namespace hook_api {
@@ -385,7 +386,9 @@ enum class hook_return_code : int64_t {
MEM_OVERLAP = -43, // one or more specified buffers are the same memory
TOO_MANY_STATE_MODIFICATIONS = -44, // more than 256 modified state
// entires in the combined hook chains
TOO_MANY_NAMESPACES = -45
TOO_MANY_NAMESPACES = -45,
TOO_LITTLE_ENTROPY = -46 // featureRNG: no seed for this ledger, or too few
// validators contributed to it
};
enum class ExitType : uint8_t {
@@ -398,6 +401,7 @@ enum class ExitType : uint8_t {
const uint16_t max_state_modifications = 256;
const uint8_t max_slots = 255;
const uint8_t max_nonce = 255;
const uint16_t max_rng_draws = 256; // featureRNG, per Hook execution
const uint8_t max_emit = 255;
const uint8_t max_params = 16;
const double fee_base_multiplier = 1.1f;
+8
View File
@@ -298,4 +298,12 @@ HOOK_API_DEFINITION(
HOOK_API_DEFINITION(
int64_t, prepare, (uint32_t write_ptr, uint32_t write_len, uint32_t read_ptr, uint32_t read_len),
featureHooksUpdate2)
HOOK_API_DEFINITION(
int64_t, dice, (uint32_t sides),
featureRNG)
HOOK_API_DEFINITION(
int64_t, util_random, (uint32_t write_ptr, uint32_t write_len),
featureRNG)
// clang-format on
+4
View File
@@ -118,6 +118,10 @@ negativeUNL() noexcept;
Keylet const&
UNLReport() noexcept;
/** The (fixed) index of the featureRNG random number generator object. */
Keylet const&
random() noexcept;
/** The beginning of an order book */
struct book_t
{
+21
View File
@@ -152,6 +152,26 @@ public:
return static_cast<bool>(mHookEmissions);
}
/** featureRNG: the RNG state this transaction's first Hook draw saw.
Every value the transaction's Hooks drew is derived from it. */
void
setRandomData(uint256 const& randomData)
{
mRandomData = randomData;
}
bool
hasRandomData() const
{
return static_cast<bool>(mRandomData);
}
uint256 const&
getRandomData() const
{
return *mRandomData;
}
STAmount
getDeliveredAmount() const
{
@@ -176,6 +196,7 @@ private:
std::optional<STAmount> mDelivered;
std::optional<STArray> mHookExecutions;
std::optional<STArray> mHookEmissions;
std::optional<uint256> mRandomData;
STArray mNodes;
};
+3 -2
View File
@@ -34,10 +34,11 @@
// If you add an amendment here, then do not forget to increment `numFeatures`
// in include/xrpl/protocol/Feature.h.
XRPL_FEATURE(RNG, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(PWALoader, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FIX (20261005, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FEATURE(EscrowDestinationCancel, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FIX (20260929, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FIX (20260929, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FEATURE(HookOnV2_1, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(OnChainManifests, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FIX (HookMap, Supported::yes, VoteBehavior::DefaultYes)
@@ -89,7 +90,6 @@ XRPL_FIX (ReduceImport, Supported::yes, VoteBehavior::DefaultYe
XRPL_FEATURE(Touch, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(Remarks, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FIX (FloatDivide, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FIX (240911, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FIX (PageCap, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FIX (240819, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FIX (NSDelete, Supported::yes, VoteBehavior::DefaultNo)
@@ -165,6 +165,7 @@ XRPL_FEATURE(CryptoConditionsSuite, Supported::yes, VoteBehavior::Obsolete)
// pre-amendment code has been removed and the identifiers are deprecated.
// All known amendments and amendments that may appear in a validated
// ledger must be registered either here or above with the "active" amendments
XRPL_RETIRE(fix240911)
XRPL_RETIRE(fixXahauV2)
XRPL_RETIRE(fixXahauV1)
XRPL_RETIRE(MultiSign)
@@ -196,6 +196,26 @@ LEDGER_ENTRY_DUPLICATE(ltUNL_REPORT, 0x0052, UNLReport, unl_report, ({
{sfPreviousTxnLgrSeq, soeREQUIRED},
}))
/** featureRNG: the consensus random number generator.
A singleton. sfRandomData is seeded from validator reveals by the
ttENTROPY transactions applied at the start of each ledger, and then
iterated by every Hook API draw for the rest of that ledger.
sfRandomData and sfRandomDigests are never written to metadata, and
Hooks cannot load this object.
\sa keylet::random
*/
LEDGER_ENTRY(ltRANDOM, 0x526E, Random, random, ({
{sfLedgerSequence, soeREQUIRED},
{sfRandomData, soeREQUIRED},
{sfEntropyCount, soeREQUIRED},
{sfEntropyDenominator, soeREQUIRED},
{sfRandomDigests, soeREQUIRED},
{sfPreviousTxnID, soeREQUIRED},
{sfPreviousTxnLgrSeq, soeREQUIRED},
}))
/** A ledger object which contains a signer list for an account.
\sa keylet::signers
@@ -59,6 +59,8 @@ TYPED_SFIELD(sfHookExecutionIndex, UINT16, 19)
TYPED_SFIELD(sfHookApiVersion, UINT16, 20)
TYPED_SFIELD(sfHookStateScale, UINT16, 21)
TYPED_SFIELD(sfLedgerFixType, UINT16, 22)
TYPED_SFIELD(sfEntropyCount, UINT16, 99)
TYPED_SFIELD(sfEntropyDenominator, UINT16, 100)
// 32-bit integers (common)
TYPED_SFIELD(sfNetworkID, UINT32, 1)
@@ -211,6 +213,9 @@ TYPED_SFIELD(sfOfferID, UINT256, 34)
TYPED_SFIELD(sfEscrowID, UINT256, 35)
TYPED_SFIELD(sfURITokenID, UINT256, 36)
TYPED_SFIELD(sfDomainID, UINT256, 37)
TYPED_SFIELD(sfNextRandomDigest, UINT256, 89)
// RNG running state: never written to metadata (see featureRNG)
TYPED_SFIELD(sfRandomData, UINT256, 90, SField::sMD_Never)
TYPED_SFIELD(sfManifestID, UINT256, 91)
TYPED_SFIELD(sfHookOnOutgoing, UINT256, 93)
TYPED_SFIELD(sfHookOnIncoming, UINT256, 94)
@@ -385,6 +390,7 @@ UNTYPED_SFIELD(sfXChainClaimAttestationCollectionElement, OBJECT, 30)
UNTYPED_SFIELD(sfXChainCreateAccountAttestationCollectionElement, OBJECT, 31)
UNTYPED_SFIELD(sfPriceData, OBJECT, 32)
UNTYPED_SFIELD(sfCredential, OBJECT, 33)
UNTYPED_SFIELD(sfRandomDigestEntry, OBJECT, 88)
UNTYPED_SFIELD(sfManifest, OBJECT, 90)
UNTYPED_SFIELD(sfAmountEntry, OBJECT, 91)
UNTYPED_SFIELD(sfMintURIToken, OBJECT, 92)
@@ -425,6 +431,8 @@ UNTYPED_SFIELD(sfAuthAccounts, ARRAY, 25)
UNTYPED_SFIELD(sfAuthorizeCredentials, ARRAY, 26)
UNTYPED_SFIELD(sfUnauthorizeCredentials, ARRAY, 27)
UNTYPED_SFIELD(sfAcceptedCredentials, ARRAY, 28)
// RNG validator commitments: never written to metadata (see featureRNG)
UNTYPED_SFIELD(sfRandomDigests, ARRAY, 91, SField::sMD_Never)
UNTYPED_SFIELD(sfAmounts, ARRAY, 92)
UNTYPED_SFIELD(sfHookEmissions, ARRAY, 93)
UNTYPED_SFIELD(sfImportVLKeys, ARRAY, 94)
+183 -1
View File
@@ -24,11 +24,21 @@
/**
* TRANSACTION(tag, value, name, fields)
*
* To ease maintenance, you may replace any unneeded values with "..."
* e.g. #define TRANSACTION(tag, value, name, ...)
*
* You must define a transactor class in the `ripple` namespace named `name`,
* and include its header in `src/xrpld/app/tx/detail/applySteps.cpp`.
* and include its header alongside the TRANSACTOR definition using this
* format:
* #if TRANSACTION_INCLUDE
* # include <xrpld/app/tx/detail/HEADER.h>
* #endif
*/
/** This transaction type executes a payment. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/Payment.h>
#endif
TRANSACTION(ttPAYMENT, 0, Payment, ({
{sfDestination, soeREQUIRED},
{sfAmount, soeREQUIRED, soeMPTSupported},
@@ -41,6 +51,9 @@ TRANSACTION(ttPAYMENT, 0, Payment, ({
}))
/** This transaction type creates an escrow object. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/Escrow.h>
#endif
TRANSACTION(ttESCROW_CREATE, 1, EscrowCreate, ({
{sfDestination, soeREQUIRED},
{sfAmount, soeREQUIRED},
@@ -62,6 +75,9 @@ TRANSACTION(ttESCROW_FINISH, 2, EscrowFinish, ({
/** This transaction type adjusts various account settings. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/SetAccount.h>
#endif
TRANSACTION(ttACCOUNT_SET, 3, AccountSet, ({
{sfEmailHash, soeOPTIONAL},
{sfWalletLocator, soeOPTIONAL},
@@ -85,6 +101,9 @@ TRANSACTION(ttESCROW_CANCEL, 4, EscrowCancel, ({
}))
/** This transaction type sets or clears an account's "regular key". */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/SetRegularKey.h>
#endif
TRANSACTION(ttREGULAR_KEY_SET, 5, SetRegularKey, ({
{sfRegularKey, soeOPTIONAL},
}))
@@ -92,6 +111,9 @@ TRANSACTION(ttREGULAR_KEY_SET, 5, SetRegularKey, ({
// 6 deprecated
/** This transaction type creates an offer to trade one asset for another. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/CreateOffer.h>
#endif
TRANSACTION(ttOFFER_CREATE, 7, OfferCreate, ({
{sfTakerPays, soeREQUIRED},
{sfTakerGets, soeREQUIRED},
@@ -101,6 +123,9 @@ TRANSACTION(ttOFFER_CREATE, 7, OfferCreate, ({
}))
/** This transaction type cancels existing offers to trade one asset for another. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/CancelOffer.h>
#endif
TRANSACTION(ttOFFER_CANCEL, 8, OfferCancel, ({
{sfOfferSequence, soeOPTIONAL},
{sfOfferID, soeOPTIONAL}, // keylet as alternative to offerseq
@@ -109,6 +134,9 @@ TRANSACTION(ttOFFER_CANCEL, 8, OfferCancel, ({
// 9 deprecated
/** This transaction type creates a new set of tickets. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/CreateTicket.h>
#endif
TRANSACTION(ttTICKET_CREATE, 10, TicketCreate, ({
{sfTicketCount, soeREQUIRED},
}))
@@ -118,12 +146,18 @@ TRANSACTION(ttTICKET_CREATE, 10, TicketCreate, ({
/** This transaction type modifies the signer list associated with an account. */
// The SignerEntries are optional because a SignerList is deleted by
// setting the SignerQuorum to zero and omitting SignerEntries.
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/SetSignerList.h>
#endif
TRANSACTION(ttSIGNER_LIST_SET, 12, SignerListSet, ({
{sfSignerQuorum, soeREQUIRED},
{sfSignerEntries, soeOPTIONAL},
}))
/** This transaction type creates a new unidirectional XRP payment channel. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/PayChan.h>
#endif
TRANSACTION(ttPAYCHAN_CREATE, 13, PaymentChannelCreate, ({
{sfDestination, soeREQUIRED},
{sfAmount, soeREQUIRED},
@@ -151,6 +185,9 @@ TRANSACTION(ttPAYCHAN_CLAIM, 15, PaymentChannelClaim, ({
}))
/** This transaction type creates a new check. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/CreateCheck.h>
#endif
TRANSACTION(ttCHECK_CREATE, 16, CheckCreate, ({
{sfDestination, soeREQUIRED},
{sfSendMax, soeREQUIRED},
@@ -160,6 +197,9 @@ TRANSACTION(ttCHECK_CREATE, 16, CheckCreate, ({
}))
/** This transaction type cashes an existing check. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/CashCheck.h>
#endif
TRANSACTION(ttCHECK_CASH, 17, CheckCash, ({
{sfCheckID, soeREQUIRED},
{sfAmount, soeOPTIONAL},
@@ -167,11 +207,17 @@ TRANSACTION(ttCHECK_CASH, 17, CheckCash, ({
}))
/** This transaction type cancels an existing check. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/CancelCheck.h>
#endif
TRANSACTION(ttCHECK_CANCEL, 18, CheckCancel, ({
{sfCheckID, soeREQUIRED},
}))
/** This transaction type grants or revokes authorization to transfer funds. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/DepositPreauth.h>
#endif
TRANSACTION(ttDEPOSIT_PREAUTH, 19, DepositPreauth, ({
{sfAuthorize, soeOPTIONAL},
{sfUnauthorize, soeOPTIONAL},
@@ -180,6 +226,9 @@ TRANSACTION(ttDEPOSIT_PREAUTH, 19, DepositPreauth, ({
}))
/** This transaction type modifies a trustline between two accounts. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/SetTrust.h>
#endif
TRANSACTION(ttTRUST_SET, 20, TrustSet, ({
{sfLimitAmount, soeOPTIONAL},
{sfQualityIn, soeOPTIONAL},
@@ -187,6 +236,9 @@ TRANSACTION(ttTRUST_SET, 20, TrustSet, ({
}))
/** This transaction type deletes an existing account. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/DeleteAccount.h>
#endif
TRANSACTION(ttACCOUNT_DELETE, 21, AccountDelete, ({
{sfDestination, soeREQUIRED},
{sfDestinationTag, soeOPTIONAL},
@@ -194,11 +246,17 @@ TRANSACTION(ttACCOUNT_DELETE, 21, AccountDelete, ({
}))
/** This transaction type installs a hook. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/SetHook.h>
#endif
TRANSACTION(ttHOOK_SET, 22, SetHook, ({
{sfHooks, soeREQUIRED},
}))
/** This transaction mints a new NFT. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/NFTokenMint.h>
#endif
TRANSACTION(ttNFTOKEN_MINT, 25, NFTokenMint, ({
{sfNFTokenTaxon, soeREQUIRED},
{sfTransferFee, soeOPTIONAL},
@@ -210,12 +268,18 @@ TRANSACTION(ttNFTOKEN_MINT, 25, NFTokenMint, ({
}))
/** This transaction burns (i.e. destroys) an existing NFT. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/NFTokenBurn.h>
#endif
TRANSACTION(ttNFTOKEN_BURN, 26, NFTokenBurn, ({
{sfNFTokenID, soeREQUIRED},
{sfOwner, soeOPTIONAL},
}))
/** This transaction creates a new offer to buy or sell an NFT. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/NFTokenCreateOffer.h>
#endif
TRANSACTION(ttNFTOKEN_CREATE_OFFER, 27, NFTokenCreateOffer, ({
{sfNFTokenID, soeREQUIRED},
{sfAmount, soeREQUIRED},
@@ -225,11 +289,17 @@ TRANSACTION(ttNFTOKEN_CREATE_OFFER, 27, NFTokenCreateOffer, ({
}))
/** This transaction cancels an existing offer to buy or sell an existing NFT. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/NFTokenCancelOffer.h>
#endif
TRANSACTION(ttNFTOKEN_CANCEL_OFFER, 28, NFTokenCancelOffer, ({
{sfNFTokenOffers, soeREQUIRED},
}))
/** This transaction accepts an existing offer to buy or sell an existing NFT. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/NFTokenAcceptOffer.h>
#endif
TRANSACTION(ttNFTOKEN_ACCEPT_OFFER, 29, NFTokenAcceptOffer, ({
{sfNFTokenBuyOffer, soeOPTIONAL},
{sfNFTokenSellOffer, soeOPTIONAL},
@@ -237,12 +307,18 @@ TRANSACTION(ttNFTOKEN_ACCEPT_OFFER, 29, NFTokenAcceptOffer, ({
}))
/** This transaction claws back issued tokens. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/Clawback.h>
#endif
TRANSACTION(ttCLAWBACK, 30, Clawback, ({
{sfAmount, soeREQUIRED, soeMPTSupported},
{sfHolder, soeOPTIONAL},
}))
/** This transaction claws back tokens from an AMM pool. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/AMMClawback.h>
#endif
TRANSACTION(ttAMM_CLAWBACK, 31, AMMClawback, ({
{sfHolder, soeREQUIRED},
{sfAsset, soeREQUIRED},
@@ -251,6 +327,9 @@ TRANSACTION(ttAMM_CLAWBACK, 31, AMMClawback, ({
}))
/** This transaction type creates an AMM instance */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/AMMCreate.h>
#endif
TRANSACTION(ttAMM_CREATE, 35, AMMCreate, ({
{sfAmount, soeREQUIRED},
{sfAmount2, soeREQUIRED},
@@ -258,6 +337,9 @@ TRANSACTION(ttAMM_CREATE, 35, AMMCreate, ({
}))
/** This transaction type deposits into an AMM instance */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/AMMDeposit.h>
#endif
TRANSACTION(ttAMM_DEPOSIT, 36, AMMDeposit, ({
{sfAsset, soeREQUIRED},
{sfAsset2, soeREQUIRED},
@@ -269,6 +351,9 @@ TRANSACTION(ttAMM_DEPOSIT, 36, AMMDeposit, ({
}))
/** This transaction type withdraws from an AMM instance */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/AMMWithdraw.h>
#endif
TRANSACTION(ttAMM_WITHDRAW, 37, AMMWithdraw, ({
{sfAsset, soeREQUIRED},
{sfAsset2, soeREQUIRED},
@@ -279,6 +364,9 @@ TRANSACTION(ttAMM_WITHDRAW, 37, AMMWithdraw, ({
}))
/** This transaction type votes for the trading fee */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/AMMVote.h>
#endif
TRANSACTION(ttAMM_VOTE, 38, AMMVote, ({
{sfAsset, soeREQUIRED},
{sfAsset2, soeREQUIRED},
@@ -286,6 +374,9 @@ TRANSACTION(ttAMM_VOTE, 38, AMMVote, ({
}))
/** This transaction type bids for the auction slot */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/AMMBid.h>
#endif
TRANSACTION(ttAMM_BID, 39, AMMBid, ({
{sfAsset, soeREQUIRED},
{sfAsset2, soeREQUIRED},
@@ -295,12 +386,18 @@ TRANSACTION(ttAMM_BID, 39, AMMBid, ({
}))
/** This transaction type deletes AMM in the empty state */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/AMMDelete.h>
#endif
TRANSACTION(ttAMM_DELETE, 40, AMMDelete, ({
{sfAsset, soeREQUIRED},
{sfAsset2, soeREQUIRED},
}))
/** This transaction mints/burns/buys/sells a URI TOKEN */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/URIToken.h>
#endif
TRANSACTION(ttURITOKEN_MINT, 45, URITokenMint, ({
{sfURI, soeREQUIRED},
{sfDigest, soeOPTIONAL},
@@ -328,6 +425,9 @@ TRANSACTION(ttURITOKEN_CANCEL_SELL_OFFER, 49, URITokenCancelSellOffer, ({
}))
/** This transactions creates a crosschain sequence number */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/XChainBridge.h>
#endif
TRANSACTION(ttXCHAIN_CREATE_CLAIM_ID, 50, XChainCreateClaimID, ({
{sfXChainBridge, soeREQUIRED},
{sfSignatureReward, soeREQUIRED},
@@ -407,6 +507,9 @@ TRANSACTION(ttXCHAIN_CREATE_BRIDGE, 57, XChainCreateBridge, ({
}))
/** This transaction type creates or updates a DID */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/DID.h>
#endif
TRANSACTION(ttDID_SET, 58, DIDSet, ({
{sfDIDDocument, soeOPTIONAL},
{sfURI, soeOPTIONAL},
@@ -417,6 +520,9 @@ TRANSACTION(ttDID_SET, 58, DIDSet, ({
TRANSACTION(ttDID_DELETE, 59, DIDDelete, ({}))
/** This transaction type creates an Oracle instance */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/SetOracle.h>
#endif
TRANSACTION(ttORACLE_SET, 60, OracleSet, ({
{sfOracleDocumentID, soeREQUIRED},
{sfProvider, soeOPTIONAL},
@@ -427,17 +533,26 @@ TRANSACTION(ttORACLE_SET, 60, OracleSet, ({
}))
/** This transaction type deletes an Oracle instance */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/DeleteOracle.h>
#endif
TRANSACTION(ttORACLE_DELETE, 61, OracleDelete, ({
{sfOracleDocumentID, soeREQUIRED},
}))
/** This transaction type fixes a problem in the ledger state */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/LedgerStateFix.h>
#endif
TRANSACTION(ttLEDGER_STATE_FIX, 62, LedgerStateFix, ({
{sfLedgerFixType, soeREQUIRED},
{sfOwner, soeOPTIONAL},
}))
/** This transaction type creates a MPTokensIssuance instance */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/MPTokenIssuanceCreate.h>
#endif
TRANSACTION(ttMPTOKEN_ISSUANCE_CREATE, 63, MPTokenIssuanceCreate, ({
{sfAssetScale, soeOPTIONAL},
{sfTransferFee, soeOPTIONAL},
@@ -446,23 +561,35 @@ TRANSACTION(ttMPTOKEN_ISSUANCE_CREATE, 63, MPTokenIssuanceCreate, ({
}))
/** This transaction type destroys a MPTokensIssuance instance */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/MPTokenIssuanceDestroy.h>
#endif
TRANSACTION(ttMPTOKEN_ISSUANCE_DESTROY, 64, MPTokenIssuanceDestroy, ({
{sfMPTokenIssuanceID, soeREQUIRED},
}))
/** This transaction type sets flags on a MPTokensIssuance or MPToken instance */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/MPTokenIssuanceSet.h>
#endif
TRANSACTION(ttMPTOKEN_ISSUANCE_SET, 65, MPTokenIssuanceSet, ({
{sfMPTokenIssuanceID, soeREQUIRED},
{sfHolder, soeOPTIONAL},
}))
/** This transaction type authorizes a MPToken instance */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/MPTokenAuthorize.h>
#endif
TRANSACTION(ttMPTOKEN_AUTHORIZE, 66, MPTokenAuthorize, ({
{sfMPTokenIssuanceID, soeREQUIRED},
{sfHolder, soeOPTIONAL},
}))
/** This transaction type create an Credential instance */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/Credentials.h>
#endif
TRANSACTION(ttCREDENTIAL_CREATE, 67, CredentialCreate, ({
{sfSubject, soeREQUIRED},
{sfCredentialType, soeREQUIRED},
@@ -484,6 +611,9 @@ TRANSACTION(ttCREDENTIAL_DELETE, 69, CredentialDelete, ({
}))
/** This transaction type modify a NFToken */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/NFTokenModify.h>
#endif
TRANSACTION(ttNFTOKEN_MODIFY, 70, NFTokenModify, ({
{sfNFTokenID, soeREQUIRED},
{sfOwner, soeOPTIONAL},
@@ -491,29 +621,63 @@ TRANSACTION(ttNFTOKEN_MODIFY, 70, NFTokenModify, ({
}))
/** This transaction type creates or modifies a Permissioned Domain */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/PermissionedDomainSet.h>
#endif
TRANSACTION(ttPERMISSIONED_DOMAIN_SET, 71, PermissionedDomainSet, ({
{sfDomainID, soeOPTIONAL},
{sfAcceptedCredentials, soeREQUIRED},
}))
/** This transaction type deletes a Permissioned Domain */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/PermissionedDomainDelete.h>
#endif
TRANSACTION(ttPERMISSIONED_DOMAIN_DELETE, 72, PermissionedDomainDelete, ({
{sfDomainID, soeREQUIRED},
}))
/** featureRNG: a validator's entropy contribution for one ledger.
Each proposing UNLReport validator places exactly one of these in its own
initial position when it closes the ledger. It reveals the secret behind
the validator's commitment currently stored in the Random object (if any)
and commits to the secret it will reveal next time. It never enters the
open ledger, so a reveal only becomes public after the revealing
validator's transaction set is locked.
*/
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/Change.h>
#endif
TRANSACTION(ttENTROPY, 89, Entropy, ({
{sfLedgerSequence, soeREQUIRED},
{sfPublicKey, soeREQUIRED},
{sfNextRandomDigest, soeREQUIRED},
{sfRandomData, soeOPTIONAL},
}))
/* sfAccount is supplied by TxFormats::commonFields; listing it here as well
makes the SOTemplate reject the format at construction. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/SetManifest.h>
#endif
TRANSACTION(ttMANIFEST_SET, 91, SetManifest, ({
{sfManifest, soeREQUIRED},
}))
/* A pseudo-txn alarm signal for invoking a hook, emitted by validators after alarm set conditions are met */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/Cron.h>
#endif
TRANSACTION(ttCRON, 92, Cron, ({
{sfOwner, soeREQUIRED},
{sfLedgerSequence, soeREQUIRED},
}))
/* Sechedule an alarm for later */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/CronSet.h>
#endif
TRANSACTION(ttCRON_SET, 93, CronSet, ({
{sfDelaySeconds, soeOPTIONAL},
{sfRepeatCount, soeOPTIONAL},
@@ -521,6 +685,9 @@ TRANSACTION(ttCRON_SET, 93, CronSet, ({
}))
/* A note attaching transactor that allows the owner or issuer (on a object by object basis) to attach remarks */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/SetRemarks.h>
#endif
TRANSACTION(ttREMARKS_SET, 94, SetRemarks, ({
{sfObjectID, soeREQUIRED},
{sfRemarks, soeREQUIRED},
@@ -528,6 +695,9 @@ TRANSACTION(ttREMARKS_SET, 94, SetRemarks, ({
/* A payment transactor that delivers only the exact amounts specified, creating accounts and TLs as needed
* that the sender pays for. */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/Remit.h>
#endif
TRANSACTION(ttREMIT, 95, Remit, ({
{sfDestination, soeREQUIRED},
{sfAmounts, soeOPTIONAL},
@@ -542,12 +712,18 @@ TRANSACTION(ttREMIT, 95, Remit, ({
/** This transaction can only be used by the genesis account, which is controlled exclusively by
* rewards/governance hooks, to print new XRP to be delivered directly to an array of destinations,
* according to reward schedule */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/GenesisMint.h>
#endif
TRANSACTION(ttGENESIS_MINT, 96, GenesisMint, ({
{sfGenesisMints, soeREQUIRED},
}))
/** This transaction accepts a proof of burn from an external network as a basis
* for minting according to featureImport */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/Import.h>
#endif
TRANSACTION(ttIMPORT, 97, Import, ({
{sfBlob, soeREQUIRED},
{sfIssuer, soeOPTIONAL},
@@ -555,12 +731,18 @@ TRANSACTION(ttIMPORT, 97, Import, ({
/** This transaction resets accumulator/counters and claims a reward for holding an average balance
* from a specified hook */
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/ClaimReward.h>
#endif
TRANSACTION(ttCLAIM_REWARD, 98, ClaimReward, ({
{sfIssuer, soeOPTIONAL},
{sfClaimCurrency, soeOPTIONAL},
}))
/** This transaction invokes a hook, providing arbitrary data. Essentially as a 0 drop payment. **/
#if TRANSACTION_INCLUDE
# include <xrpld/app/tx/detail/Invoke.h>
#endif
TRANSACTION(ttINVOKE, 99, Invoke, ({
{sfBlob, soeOPTIONAL},
{sfDestination, soeOPTIONAL},
+2 -1
View File
@@ -171,6 +171,7 @@ JSS(amendment_blocked); // out: NetworkOPs
JSS(amm_account); // in: amm_info
JSS(amount); // out: AccountChannels, amm_info
JSS(amount2); // out: amm_info
JSS(anchored_through); // out: CatalogueLoad
JSS(api_version); // in: many, out: Version
JSS(api_version_low); // out: Version
JSS(applied); // out: SubmitTransaction
@@ -592,7 +593,6 @@ JSS(queue_data); // out: AccountInfo
JSS(queued); // out: SubmitTransaction
JSS(queued_duration_us);
JSS(quote_asset); // in: get_aggregate_price
JSS(random); // out: Random
JSS(raw_meta); // out: AcceptedLedgerTx
JSS(receive_currencies); // out: AccountCurrencies
JSS(reference_level); // out: TxQ
@@ -745,6 +745,7 @@ JSS(TRANSACTION_FLAGS_INDICES); // out: RPC server_definitions
JSS(type_hex); // out: STPathSet
JSS(unl); // out: UnlList
JSS(unlimited); // out: Connection.h
JSS(unpinned); // out: CatalogueUnpin
JSS(uptime); // out: GetCounts
JSS(uri); // out: ValidatorSites
JSS(url); // in/out: Subscribe, Unsubscribe
+10
View File
@@ -100,8 +100,18 @@ struct Port
return protocol.count("udp") > 0;
}
bool
has_peer() const
{
return protocol.count("peer") > 0;
}
// Maximum UDP packet size (default 64KB)
std::size_t udp_packet_size = 65536;
// If this is a peer port, also listen for UDP Superhighway (XUSH)
// datagrams on the same port number.
bool udp_highway = false;
};
std::ostream&
+10
View File
@@ -208,6 +208,16 @@ ServerImpl<Handler>::ports(std::vector<Port> const& ports)
sp->run();
}
// UDP Superhighway (XUSH) datagrams arrive on the peer port
// number. TCP and UDP port numbers are independent, so this
// needs no socket sharing with the TCP door.
if (internalPort.has_peer() && internalPort.udp_highway)
{
if (auto sp = ios_.emplace<UDPDoor<Handler>>(
handler_, io_context_, internalPort, j_))
sp->run();
}
}
}
return eps;
+10 -5
View File
@@ -78,10 +78,9 @@ public:
error_code ec;
// Create UDP endpoint from port configuration
auto const addr = port_.ip.to_v4();
boost::asio::ip::udp::endpoint udp_endpoint(addr, port_.port);
boost::asio::ip::udp::endpoint udp_endpoint(port_.ip, port_.port);
socket_.open(boost::asio::ip::udp::v4(), ec);
socket_.open(udp_endpoint.protocol(), ec);
if (ec)
{
JLOG(j_.error()) << "UDP socket open failed: " << ec.message();
@@ -94,17 +93,22 @@ public:
{
JLOG(j_.error())
<< "UDP set reuse_address failed: " << ec.message();
close();
return;
}
socket_.bind(udp_endpoint, ec);
if (ec)
{
JLOG(j_.error()) << "UDP socket bind failed: " << ec.message();
// Close the socket so that run() does not start receiving on an
// unbound socket, which would silently bind an ephemeral port.
JLOG(j_.error()) << "UDP socket bind to " << udp_endpoint
<< " failed: " << ec.message();
close();
return;
}
JLOG(j_.info()) << "UDP-RPC listening on " << udp_endpoint;
JLOG(j_.info()) << "UDP listening on " << udp_endpoint;
}
endpoint_type
@@ -169,6 +173,7 @@ private:
handler_.onUDPMessage(
std::string(recv_buffer_.data(), bytes_transferred),
tcp_endpoint,
port_,
[this, tcp_endpoint](std::string const& response) {
do_send(response, tcp_endpoint);
});
+8
View File
@@ -94,6 +94,7 @@ enum class LedgerNameSpace : std::uint16_t {
MANIFEST_DIR = LEDGER_NAMESPACE2(0x01, 'M'),
CREDENTIAL = LEDGER_NAMESPACE2(0x01, 'D'),
PERMISSIONED_DOMAIN = 'm',
RANDOM = 0x526E, // Rn
// No longer used or supported. Left here to reserve the space
// to avoid accidental reuse.
@@ -278,6 +279,13 @@ UNLReport() noexcept
return ret;
}
Keylet const&
random() noexcept
{
static Keylet const ret{ltRANDOM, indexHash(LedgerNameSpace::RANDOM)};
return ret;
}
Keylet
book_t::operator()(Book const& b) const
{
@@ -143,6 +143,14 @@ InnerObjectFormats::InnerObjectFormats()
{sfAccount, soeOPTIONAL},
});
add(sfRandomDigestEntry.jsonName,
sfRandomDigestEntry.getCode(),
{
{sfPublicKey, soeREQUIRED},
{sfNextRandomDigest, soeREQUIRED},
{sfLedgerSequence, soeREQUIRED},
});
add(sfAmountEntry.jsonName,
sfAmountEntry.getCode(),
{
+2 -1
View File
@@ -684,7 +684,8 @@ isPseudoTx(STObject const& tx)
auto tt = safe_cast<TxType>(*t);
return tt == ttAMENDMENT || tt == ttFEE || tt == ttUNL_MODIFY ||
tt == ttEMIT_FAILURE || tt == ttUNL_REPORT || tt == ttCRON;
tt == ttEMIT_FAILURE || tt == ttUNL_REPORT || tt == ttCRON ||
tt == ttENTROPY;
}
} // namespace ripple
+9
View File
@@ -49,6 +49,9 @@ TxMeta::TxMeta(
if (obj.isFieldPresent(sfHookEmissions))
setHookEmissions(obj.getFieldArray(sfHookEmissions));
if (obj.isFieldPresent(sfRandomData))
setRandomData(obj.getFieldH256(sfRandomData));
}
TxMeta::TxMeta(uint256 const& txid, std::uint32_t ledger, STObject const& obj)
@@ -75,6 +78,9 @@ TxMeta::TxMeta(uint256 const& txid, std::uint32_t ledger, STObject const& obj)
if (obj.isFieldPresent(sfHookEmissions))
setHookEmissions(obj.getFieldArray(sfHookEmissions));
if (obj.isFieldPresent(sfRandomData))
setRandomData(obj.getFieldH256(sfRandomData));
}
TxMeta::TxMeta(uint256 const& txid, std::uint32_t ledger, Blob const& vec)
@@ -245,6 +251,9 @@ TxMeta::getAsObject() const
if (hasHookEmissions())
metaData.setFieldArray(sfHookEmissions, getHookEmissions());
if (hasRandomData())
metaData.setFieldH256(sfRandomData, getRandomData());
return metaData;
}
+74
View File
@@ -1271,6 +1271,79 @@ public:
BEAST_EXPECT(table->needValidatedLedger(257));
}
void
testIgnoredUnsupported()
{
testcase("ignore_amendments");
using namespace std::chrono_literals;
weeks constexpr w(1);
test::jtx::Env env{*this, makeConfig()};
static std::vector<AmendmentTable::FeatureInfo> const supported =
makeDefaultYes(yes_);
// Ignore every unsupported_ amendment except the first one.
BEAST_EXPECT(unsupported_.size() > 1);
std::vector<std::string> const ignoredNames(
unsupported_.begin() + 1, unsupported_.end());
auto table = make_AmendmentTable(
env.app(),
w,
supported,
emptySection_,
emptySection_,
journal_,
makeSection(ignoredNames));
for (auto const& a : ignoredNames)
BEAST_EXPECT(table->isSupported(amendmentId(a)));
BEAST_EXPECT(!table->isSupported(amendmentId(unsupported_[0])));
majorityAmendments_t const majority;
// Enabling only ignored amendments must not block.
std::set<uint256> enabled;
for (auto const& a : ignoredNames)
enabled.insert(amendmentId(a));
table->doValidatedLedger(1, enabled, majority);
BEAST_EXPECT(!table->hasUnsupportedEnabled());
BEAST_EXPECT(table->unsupportedAmendments().empty());
// An ignored amendment must never be voted for.
auto const desired = table->getDesired();
for (auto const& a : ignoredNames)
BEAST_EXPECT(
std::find(desired.begin(), desired.end(), amendmentId(a)) ==
desired.end());
// A non-ignored unsupported amendment still blocks.
enabled.insert(amendmentId(unsupported_[0]));
table->doValidatedLedger(2, enabled, majority);
BEAST_EXPECT(table->hasUnsupportedEnabled());
// Bad entries are rejected.
Section bad("ignore_amendments");
bad.append("NotARealFeatureName");
try
{
make_AmendmentTable(
env.app(),
w,
supported,
emptySection_,
emptySection_,
journal_,
bad);
fail("expected bad [ignore_amendments] entry to throw");
}
catch (std::runtime_error const&)
{
pass();
}
}
void
testFeature(FeatureBitset const& feat)
{
@@ -1294,6 +1367,7 @@ public:
testBadConfig();
testEnableVeto();
testHasUnsupported();
testIgnoredUnsupported();
testFeature(all - fixMajorityCalc);
testFeature(all);
}
+19
View File
@@ -578,6 +578,25 @@ public:
ter(tecNO_TARGET));
env.close();
}
// HookName removed with a HookName-only update
// The sfHooks entry keeps an STI_NOTPRESENT placeholder for
// sfHookName, which preclaim must treat as "no name".
{
Json::Value jv;
jv[jss::Account] = issuer.human();
jv[jss::TransactionType] = jss::SetHook;
jv[jss::Flags] = 0;
jv[jss::Hooks] = Json::Value{Json::arrayValue};
jv[jss::Hooks][0U][jss::Hook][jss::HookName] = "";
env(jv, fee(XRP(1)));
env.close();
env(reward::claim(alice),
reward::issuer(issuer),
fee(XRP(1)),
ter(tesSUCCESS));
env.close();
}
// Vaild HookOn
{
auto hookObj = hso(jtx::genesis::AcceptHook, overrideFlag);
+258
View File
@@ -3043,6 +3043,91 @@ public:
BEAST_EXPECT(acct->value() == alice.id());
}
void
test_otxn_field_emit_failure(FeatureBitset features)
{
testcase("Test otxn_field on emit failure callback");
using namespace jtx;
using namespace hook_api;
auto const alice = Account{"alice"};
auto const bob = Account{"bob"};
// the cleanup pseudo carries sfLedgerSequence / sfTransactionHash,
// the original emitted txn carries sfDestination
STTx const pseudo = STTx(ttEMIT_FAILURE, [&](STObject& obj) {
obj.setFieldU32(sfLedgerSequence, 1);
obj.setFieldH256(sfTransactionHash, uint256{5});
});
STTx const emitted = STTx(ttPAYMENT, [&](STObject& obj) {
obj[sfAccount] = alice.id();
obj[sfDestination] = bob.id();
});
for (bool const withFix : {false, true})
{
Env env{
*this,
withFix ? features | fix20261005 : features - fix20261005};
OpenView ov{*env.current()};
ApplyContext applyCtx = createApplyContext(env, ov, pseudo);
StubHookContext stubCtx{.emitFailure = emitted};
auto hookCtx =
makeStubHookContext(applyCtx, alice.id(), alice.id(), stubCtx);
auto& api = hookCtx.api();
// present on both: read from the emitted txn either way
{
auto const result = api.otxn_field(sfAccount.getCode());
BEAST_EXPECT(result.has_value());
auto const* acct =
dynamic_cast<STAccount const*>(result.value());
BEAST_EXPECT(acct && acct->value() == alice.id());
}
// only on the emitted txn: before the fix presence was checked
// on the pseudo, so the field was reported missing
{
auto const result = api.otxn_field(sfDestination.getCode());
if (withFix)
{
BEAST_EXPECT(result.has_value());
auto const* dest =
dynamic_cast<STAccount const*>(result.value());
BEAST_EXPECT(dest && dest->value() == bob.id());
}
else
{
BEAST_EXPECT(
!result.has_value() && result.error() == DOESNT_EXIST);
}
}
// only on the pseudo: before the fix this throws out of
// getField on the emitted txn
if (withFix)
{
auto const result = api.otxn_field(sfLedgerSequence.getCode());
BEAST_EXPECT(
!result.has_value() && result.error() == DOESNT_EXIST);
}
else
{
bool threw = false;
try
{
(void)api.otxn_field(sfLedgerSequence.getCode());
}
catch (std::exception const&)
{
threw = true;
}
BEAST_EXPECT(threw);
}
}
}
void
test_otxn_id(FeatureBitset features)
{
@@ -3741,6 +3826,174 @@ public:
}
}
void
test_state_foreign_read_accounting(FeatureBitset features)
{
testcase("Test state_foreign read does not consume write resources");
using namespace jtx;
using namespace hook_api;
using namespace hook;
bool const hasFix = features[fix20261005];
bool const hasExtended = features[featureExtendedHookState];
Env env{*this, features};
auto const fees = env.current()->fees();
auto const alice = Account{"alice"};
auto const bob = Account{"bob"};
auto const carol = Account{"carol"};
env.fund(XRP(10000), alice);
// exactly one state reserve slot available
env.fund(fees.accountReserve(0) + fees.increment, bob);
// no state reserve slot available
env.fund(fees.accountReserve(0), carol);
env.close();
uint256 const readKey{1};
uint256 const writeKey{2};
uint256 const ns{3};
Bytes const existingData{0xAA, 0xBB};
Bytes writeData{0x01};
OpenView ov{*env.current()};
// Place an existing state entry (and its namespace directory) on
// ledger for each account, so state_foreign's cache-miss path reads it.
auto insertState = [&](AccountID const& acc) {
auto const dirKeylet = keylet::hookStateDir(acc, ns);
auto dir = std::make_shared<SLE>(dirKeylet);
dir->setFieldH256(sfRootIndex, dirKeylet.key);
dir->setFieldV256(
sfIndexes,
STVector256{std::vector<uint256>{
keylet::hookState(acc, readKey, ns).key}});
ov.rawInsert(dir);
auto hs =
std::make_shared<SLE>(keylet::hookState(acc, readKey, ns));
hs->setFieldVL(sfHookStateData, existingData);
hs->setFieldH256(sfHookStateKey, readKey);
hs->setFieldU64(sfOwnerNode, 0);
ov.rawInsert(hs);
};
insertState(alice.id());
insertState(bob.id());
insertState(carol.id());
STTx invokeTx = STTx(ttINVOKE, [&](STObject& obj) {});
ApplyContext applyCtx = createApplyContext(env, ov, invokeTx);
auto availableSlots = [&](AccountID const& acc) -> int64_t {
auto const sle = ov.read(keylet::account(acc));
int64_t avail = sle->getFieldAmount(sfBalance).xrp().drops() -
fees.accountReserve(sle->getFieldU32(sfOwnerCount)).drops();
return avail / fees.increment.drops();
};
{
// A read must not consume the shared modification limit.
HookStateMap stateMap;
auto hookCtx = makeStubHookContext(
applyCtx, alice.id(), alice.id(), {}, stateMap);
auto& api = hookCtx.api();
stateMap.modified_entry_count = max_state_modifications - 1;
auto const read = api.state_foreign(readKey, ns, alice.id());
BEAST_EXPECT(read.has_value() && read.value() == existingData);
int64_t const slots = availableSlots(alice.id());
BEAST_EXPECT(slots > 1);
BEAST_EXPECT(
stateMap.modified_entry_count ==
max_state_modifications - (hasFix ? 1 : 0));
BEAST_EXPECT(
std::get<0>(stateMap[alice.id()]) == slots - (hasFix ? 0 : 1));
auto const write =
api.state_foreign_set(writeKey, ns, alice.id(), writeData);
if (hasFix)
{
BEAST_EXPECT(write.has_value());
BEAST_EXPECT(
stateMap.modified_entry_count == max_state_modifications);
}
else
{
BEAST_EXPECT(!write.has_value());
BEAST_EXPECT(write.error() == TOO_MANY_STATE_MODIFICATIONS);
}
}
{
// A read must not consume the reserve budget.
HookStateMap stateMap;
auto hookCtx =
makeStubHookContext(applyCtx, bob.id(), bob.id(), {}, stateMap);
auto& api = hookCtx.api();
BEAST_EXPECT(availableSlots(bob.id()) == 1);
auto const read = api.state_foreign(readKey, ns, bob.id());
BEAST_EXPECT(read.has_value() && read.value() == existingData);
BEAST_EXPECT(std::get<0>(stateMap[bob.id()]) == (hasFix ? 1 : 0));
auto const write =
api.state_foreign_set(writeKey, ns, bob.id(), writeData);
if (hasFix)
{
BEAST_EXPECT(write.has_value());
BEAST_EXPECT(std::get<0>(stateMap[bob.id()]) == 0);
}
else
{
BEAST_EXPECT(!write.has_value());
BEAST_EXPECT(write.error() == RESERVE_INSUFFICIENT);
}
// Modifying the entry that was read needs no new reserve and
// is still charged as a modification.
auto const before = stateMap.modified_entry_count;
Bytes overwrite{0x02};
auto const update =
api.state_foreign_set(readKey, ns, bob.id(), overwrite);
BEAST_EXPECT(update.has_value());
BEAST_EXPECT(stateMap.modified_entry_count == before + 1);
}
{
// A read on an account with no spare reserve must succeed;
// previously ExtendedHookState's sanity check rejected it.
HookStateMap stateMap;
auto hookCtx = makeStubHookContext(
applyCtx, carol.id(), carol.id(), {}, stateMap);
auto& api = hookCtx.api();
BEAST_EXPECT(availableSlots(carol.id()) == 0);
auto const read = api.state_foreign(readKey, ns, carol.id());
if (hasExtended && !hasFix)
{
BEAST_EXPECT(!read.has_value());
BEAST_EXPECT(read.error() == INTERNAL_ERROR);
}
else
{
BEAST_EXPECT(read.has_value() && read.value() == existingData);
BEAST_EXPECT(stateMap.modified_entry_count == (hasFix ? 0 : 1));
}
// Writing a new key still fails on reserve either way.
auto const write =
api.state_foreign_set(writeKey, ns, carol.id(), writeData);
BEAST_EXPECT(!write.has_value());
if (!write.has_value())
BEAST_EXPECT(write.error() == RESERVE_INSUFFICIENT);
}
}
void
test_state_foreign_set_max(FeatureBitset features)
{
@@ -4958,6 +5211,7 @@ public:
test_fee_base(features);
test_otxn_field(features);
test_otxn_field_emit_failure(features);
test_ledger_keylet(features);
@@ -5012,6 +5266,10 @@ public:
test_state(features);
test_state_foreign(features);
test_state_foreign_read_accounting(features);
test_state_foreign_read_accounting(features - fix20261005);
test_state_foreign_read_accounting(
features - fix20261005 - featureExtendedHookState);
test_state_foreign_set(features - fixHookMap);
test_state_foreign_set(features);
test_state_foreign_set_max(features);
File diff suppressed because it is too large. Load diff
+146
View File
@@ -33,9 +33,11 @@
#include <xrpl/protocol/jss.h>
#include <boost/filesystem.hpp>
#include <chrono>
#include <exception>
#include <limits>
#include <string>
#include <thread>
namespace ripple {
namespace test {
@@ -93,6 +95,27 @@ class SHAMapStore_test : public beast::unit_test::suite
return cfg;
}
// Online delete with DatabasePinned over in-memory backends. Unlike
// pinnedPersistent, the hot store keeps nodes (type=rwdb is the null
// node store), so ledgers closed in the test can be fetched. There is
// no database_path, so every SHAMapStoreImp opens its own private
// state.db: a second instance built by a test does not see (or trip
// over) the rotation state of the Env's own.
static auto
pinnedInMemory(
std::unique_ptr<Config> cfg,
std::string const& nodePath,
std::string const& pinnedPath)
{
cfg = onlineDelete(std::move(cfg));
auto& section = cfg->section(ConfigSection::nodeDatabase());
section.set("type", "memory");
section.set("path", nodePath);
section.set("pinned_type", "memory");
section.set("pinned_path", pinnedPath);
return cfg;
}
bool
goodLedger(
jtx::Env& env,
@@ -1153,6 +1176,127 @@ public:
}
}
// catalogue_unpin of everything (or a rolled-back first load) leaves the
// pinned set empty. That must persist as "no pins", not as a string that
// fails to parse on the next start.
void
testPinnedRangeRestoreEmpty()
{
testcase("pinned range restore of an empty set");
using namespace jtx;
beast::temp_dir tempDir;
auto const tempPath = boost::filesystem::path(tempDir.path());
auto const nodePath = (tempPath / "node").string();
auto const pinnedPath = (tempPath / "pinned").string();
Env env(*this, envconfig(pinnedInMemory, nodePath, pinnedPath));
NodeStoreScheduler scheduler(env.app().getJobQueue());
auto const journal = env.app().journal("SHAMapStoreTest");
try
{
SHAMapStoreImp restoreCheck(env.app(), scheduler, journal);
RangeSet<std::uint32_t> ranges;
ranges.insert(range(100u, 200u));
restoreCheck.setPinnedRanges(ranges);
restoreCheck.setPinnedRanges(RangeSet<std::uint32_t>{});
restoreCheck.start();
restoreCheck.stop();
pass();
}
catch (std::exception const& e)
{
fail(
std::string("restart with no pinned ranges threw: ") +
e.what());
}
BEAST_EXPECT(
env.app().getLedgerMaster().getPinnedLedgersRangeSet().empty());
}
// A crash between a pinned-range checkpoint and the backend committing
// the last ledgers leaves the recorded interval's tail missing. Startup
// trims it back to the last ledger present instead of refusing to start.
void
testPinnedRangeRestoreTrimsMissingTail()
{
testcase("pinned range restore trims a missing tail");
using namespace jtx;
beast::temp_dir tempDir;
auto const tempPath = boost::filesystem::path(tempDir.path());
auto const nodePath = (tempPath / "node").string();
auto const pinnedPath = (tempPath / "pinned").string();
Env env(*this, envconfig(pinnedInMemory, nodePath, pinnedPath));
for (int i = 0; i < 4; ++i)
env.close();
auto& rdb = env.app().getRelationalDatabase();
auto const last = env.app().getLedgerMaster().getValidLedgerIndex();
auto const first = last - 2;
// Ledger saves may complete asynchronously; wait for the SQL rows.
for (int i = 0; i < 100 && rdb.getHashByIndex(last).isZero(); ++i)
std::this_thread::sleep_for(std::chrono::milliseconds(10));
if (!BEAST_EXPECT(
rdb.getHashByIndex(first).isNonZero() &&
rdb.getHashByIndex(last).isNonZero()))
return;
NodeStoreScheduler scheduler(env.app().getJobQueue());
auto const journal = env.app().journal("SHAMapStoreTest");
try
{
SHAMapStoreImp restoreCheck(env.app(), scheduler, journal);
// Recorded through last + 10, but only [first, last] exist.
RangeSet<std::uint32_t> recorded;
recorded.insert(range(first, last + 10));
restoreCheck.setPinnedRanges(recorded);
restoreCheck.start();
restoreCheck.stop();
}
catch (std::exception const& e)
{
fail(std::string("restart with a missing tail threw: ") + e.what());
return;
}
RangeSet<std::uint32_t> expected;
expected.insert(range(first, last));
BEAST_EXPECT(
env.app().getLedgerMaster().getPinnedLedgersRangeSet() == expected);
// A missing tail longer than the trim limit (65536 ledgers) is not
// an interrupted load: it means the wrong pinned store, so startup
// still refuses.
try
{
SHAMapStoreImp restoreCheck(env.app(), scheduler, journal);
RangeSet<std::uint32_t> recorded;
recorded.insert(range(first, last + 70000));
restoreCheck.setPinnedRanges(recorded);
restoreCheck.start();
restoreCheck.stop();
fail("Expected startup failure for an over-long missing tail");
}
catch (std::runtime_error const& e)
{
BEAST_EXPECT(
std::string(e.what()).find("Persisted pinned interval") !=
std::string::npos);
}
}
// Helper: compare two RangeSet<uint32_t> for set-equality. The
// test cares about which sequences are in the result, not the
// particular interval representation.
@@ -1302,6 +1446,8 @@ public:
testNullFactoryDropsWrites();
testRotate();
testPinnedRangeRestoreRequiresPinnedData();
testPinnedRangeRestoreEmpty();
testPinnedRangeRestoreTrimsMissingTail();
}
};
+9
View File
@@ -6146,6 +6146,15 @@ private:
pass();
}
void
testEntropyTSH(FeatureBitset features)
{
testcase("entropy tsh");
// pseudo transaction
pass();
}
// | otxn | tfBurnable | tsh | mint | burn | buy | sell | cancel
// | O | false | O | N/A | S | N/A | S | S
// | O | false | I | N/A | N | N/A | W | N/A
+325
View File
@@ -2134,6 +2134,118 @@ public:
}
}
{
/// Rename and remove the HookName with HookName-only updates
Env env{*this, features};
env.fund(XRP(10000), alice);
env.close();
// Create with hook name
auto jvh = hso(accept_wasm);
jvh[jss::HookName] = "41424344";
env(ripple::test::jtx::hook(alice, {{jvh}}, 0),
M("Create named hook"),
HSFEE);
env.close();
auto const hookNameOnlyUpdate = [&](std::string const& name) {
Json::Value jv;
jv[jss::Account] = alice.human();
jv[jss::TransactionType] = jss::SetHook;
jv[jss::Flags] = 0;
jv[jss::Hooks] = Json::Value{Json::arrayValue};
Json::Value iv;
iv[jss::HookName] = name;
jv[jss::Hooks][0U][jss::Hook] = iv;
return jv;
};
// Rename: update carrying only a non-empty sfHookName
env(hookNameOnlyUpdate("4142434445"),
M("Rename hook with a HookName-only update"),
HSFEE);
env.close();
{
auto const hooks = env.le(keylet::hook(alice));
BEAST_EXPECT(hooks && hooks->isFieldPresent(sfHooks));
auto const& hooksArray = hooks->getFieldArray(sfHooks);
BEAST_EXPECT(hooksArray.size() == 1);
BEAST_EXPECT(hooksArray[0].isFieldPresent(sfHookName));
BEAST_EXPECT(
strHex(hooksArray[0].getFieldVL(sfHookName)) ==
"4142434445");
}
// Remove: update carrying only an empty sfHookName
env(hookNameOnlyUpdate(""),
M("Remove hook name with a HookName-only update"),
HSFEE);
// In the same open ledger as the removal: the open view hands
// the next transaction the very sfHooks entry the SetHook just
// modified, so the fee calculation and the hook chain must
// tolerate the removed name. This is the path that crashed the
// node at submit, and crashed every validator replaying the
// pair while building the ledger.
{
auto jv = invoke::invoke(alice);
auto const expectedFee =
calculateBaseFee(*env.current(), *env.jt(jv).stx);
BEAST_EXPECT(expectedFee == drops(19));
env(jv,
M("Invoke in the same open ledger as the removal"),
HSFEE);
}
env.close();
{
auto const hookExecutions =
env.meta()->getFieldArray(sfHookExecutions);
BEAST_EXPECT(hookExecutions.size() == 1);
BEAST_EXPECT(
hookExecutions[0].getFieldH256(sfHookHash) == accept_hash);
}
{
auto const hooks = env.le(keylet::hook(alice));
BEAST_EXPECT(hooks && hooks->isFieldPresent(sfHooks));
auto const& hooksArray = hooks->getFieldArray(sfHooks);
BEAST_EXPECT(hooksArray.size() == 1);
BEAST_EXPECT(!hooksArray[0].isFieldPresent(sfHookName));
}
// The now non-named hook must still be charged for and
// executed by transactions in later ledgers, where the entry
// is re-read from its serialized form.
{
auto jv = invoke::invoke(alice);
auto const expectedFee =
calculateBaseFee(*env.current(), *env.jt(jv).stx);
BEAST_EXPECT(expectedFee == drops(19));
env(jv, M("Invoke after removing the hook name"), HSFEE);
env.close();
auto const hookExecutions =
env.meta()->getFieldArray(sfHookExecutions);
BEAST_EXPECT(hookExecutions.size() == 1);
BEAST_EXPECT(
hookExecutions[0].getFieldH256(sfHookHash) == accept_hash);
}
// Removing the name of a hook that has none must be a no-op,
// not a thrown FieldErr (-> tefEXCEPTION): makeFieldAbsent on
// the untemplated entry throws when the field is missing.
env(hookNameOnlyUpdate(""),
M("Remove hook name again (no-op)"),
HSFEE);
env.close();
{
auto const hooks = env.le(keylet::hook(alice));
BEAST_EXPECT(hooks && hooks->isFieldPresent(sfHooks));
auto const& hooksArray = hooks->getFieldArray(sfHooks);
BEAST_EXPECT(hooksArray.size() == 1);
BEAST_EXPECT(!hooksArray[0].isFieldPresent(sfHookName));
}
}
// Install named hook
auto jvh = hso(accept_wasm);
jvh[jss::HookName] = "41424344";
@@ -4675,6 +4787,218 @@ public:
}
}
void
test_emit_failure_callback(FeatureBitset features)
{
testcase("Test emit failure callback");
using namespace jtx;
// An emitted txn that is never applied expires into a ttEMIT_FAILURE
// cleanup pseudo, which removes its ltEMITTED_TXN entry and then
// runs the emitting Hook's cbak. Before fix20261005 the callback
// re-read that (already erased) entry and dereferenced null, taking
// the process down, so only the fixed behaviour can be run here.
Env env{*this, features | fix20261005};
auto const alice = Account{"alice"};
env.fund(XRP(10000), alice);
env.close();
TestHook hook_wasm = wasm[R"[test.hook](
#include <stdint.h>
extern int32_t _g (uint32_t id, uint32_t maxiter);
extern int64_t accept (uint32_t read_ptr, uint32_t read_len, int64_t error_code);
extern int64_t rollback (uint32_t read_ptr, uint32_t read_len, int64_t error_code);
extern int64_t emit (uint32_t write_ptr, uint32_t write_len, uint32_t read_ptr, uint32_t read_len);
extern int64_t hook_account (uint32_t write_ptr, uint32_t write_len);
extern int64_t etxn_reserve (uint32_t);
extern int64_t etxn_fee_base (uint32_t read_ptr, uint32_t read_len);
extern int64_t etxn_details (uint32_t write_ptr, uint32_t write_len);
extern int64_t ledger_seq (void);
extern int64_t otxn_id (uint32_t, uint32_t, uint32_t);
extern int64_t otxn_type (void);
extern int64_t otxn_field (uint32_t write_ptr, uint32_t write_len, uint32_t field_id);
extern int64_t state_set (uint32_t, uint32_t, uint32_t, uint32_t);
#define SBUF(x) (uint32_t)(x), sizeof(x)
#define ASSERT(x)\
if (!(x))\
rollback((uint32_t)#x, sizeof(#x), __LINE__);
#define DOESNT_EXIST (-5)
#define ttINVOKE 99
#define sfAccount ((8U << 16U) + 1U)
#define sfLedgerSequence ((2U << 16U) + 6U)
uint8_t key[32] = { [31] = 1U };
// an Invoke from the hook account, emitted with a callback
uint8_t txn[229] =
{
/* size, upto, field name */
/* 3, 0, tt = Invoke */ 0x12U, 0x00U, 0x63U,
/* 5, 3, flags */ 0x22U, 0x00U, 0x00U, 0x00U, 0x00U,
/* 5, 8, sequence */ 0x24U, 0x00U, 0x00U, 0x00U, 0x00U,
/* 6, 13, firstledgersequence */ 0x20U, 0x1AU, 0x00U, 0x00U, 0x00U, 0x00U,
/* 6, 19, lastledgersequence */ 0x20U, 0x1BU, 0x00U, 0x00U, 0x00U, 0x00U,
/* 9, 25, fee */ 0x68U, 0x40U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
/* 35, 34, signingpubkey */ 0x73U, 0x21U, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
/* 22, 69, account */ 0x81U, 0x14U, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
/* 138, 91, emit details */
/* 0, 229, */
};
#define FLS_OUT (txn + 15U)
#define LLS_OUT (txn + 21U)
#define FEE_OUT (txn + 26U)
#define ACCOUNT_OUT (txn + 71U)
#define EMIT_OUT (txn + 91U)
#define SET_UINT32(ptr, value)\
{\
uint32_t v = (value);\
(ptr)[0] = v >> 24U; (ptr)[1] = v >> 16U; (ptr)[2] = v >> 8U; (ptr)[3] = v;\
}
#define SET_NATIVE_AMOUNT(ptr, amount)\
{\
uint64_t a = (amount);\
(ptr)[0] = 0x40U + ((a >> 56U) & 0x3FU);\
(ptr)[1] = a >> 48U; (ptr)[2] = a >> 40U; (ptr)[3] = a >> 32U;\
(ptr)[4] = a >> 24U; (ptr)[5] = a >> 16U; (ptr)[6] = a >> 8U; (ptr)[7] = a;\
}
#define ACC_EQ(a, b)\
(*(uint64_t*)(a) == *(uint64_t*)(b) &&\
*(uint64_t*)((a) + 8U) == *(uint64_t*)((b) + 8U) &&\
*(uint32_t*)((a) + 16U) == *(uint32_t*)((b) + 16U))
int64_t cbak(uint32_t r)
{
_g(1,1);
// 1 = the emitted txn failed (ttEMIT_FAILURE)
ASSERT(r == 1);
// the otxn is the original emitted txn, not the pseudo
ASSERT(otxn_type() == ttINVOKE);
uint8_t me[20];
ASSERT(hook_account(SBUF(me)) == 20);
uint8_t acc[20];
ASSERT(otxn_field(SBUF(acc), sfAccount) == 20);
ASSERT(ACC_EQ(acc, me));
// carried by the pseudo but not by the emitted txn
ASSERT(otxn_field(0, 0, sfLedgerSequence) == DOESNT_EXIST);
// record the id of the emitted txn that failed
uint8_t id[32];
ASSERT(otxn_id(SBUF(id), 0) == 32);
ASSERT(state_set(SBUF(id), SBUF(key)) == 32);
return accept(0,0,0);
}
int64_t hook(uint32_t r)
{
_g(1,1);
ASSERT(etxn_reserve(1) == 1);
// FirstLedgerSequence is the current ledger, so by the time
// the emission directory is processed for the next ledger it
// has already passed: the txn is never applied and expires
// once LastLedgerSequence is behind the ledger
uint32_t seq = (uint32_t)ledger_seq();
SET_UINT32(FLS_OUT, seq);
SET_UINT32(LLS_OUT, seq + 1);
ASSERT(hook_account(ACCOUNT_OUT, 20) == 20);
ASSERT(etxn_details(EMIT_OUT, 138U) == 138);
int64_t fee = etxn_fee_base(SBUF(txn));
ASSERT(fee > 0);
SET_NATIVE_AMOUNT(FEE_OUT, fee);
uint8_t emithash[32];
ASSERT(emit(SBUF(emithash), SBUF(txn)) == 32);
return accept(0,0,0);
}
)[test.hook]"];
HASH_WASM(hook);
env(ripple::test::jtx::hook(alice, {{hso(hook_wasm, overrideFlag)}}, 0),
M("set emit failure callback"),
HSFEE);
env.close();
env(invoke::invoke(alice),
M("emit failure callback: emit"),
fee(XRP(1)),
ter(tesSUCCESS));
// the emitted txn, from the ltEMITTED_TXN this created
std::optional<uint256> emittedID;
{
auto const meta = env.meta(); // closes the ledger
BEAST_REQUIRE(meta);
for (auto const& node : meta->getFieldArray(sfAffectedNodes))
{
if (node.getFName() != sfCreatedNode ||
node.getFieldU16(sfLedgerEntryType) != ltEMITTED_TXN)
continue;
auto const& nf = const_cast<ripple::STObject&>(node)
.getField(sfNewFields)
.downcast<STObject>();
auto const& et = const_cast<ripple::STObject&>(nf)
.getField(sfEmittedTxn)
.downcast<STObject>();
Serializer s;
et.add(s);
SerialIter sit(s.slice());
emittedID = STTx(sit).getTransactionID();
}
}
BEAST_REQUIRE(emittedID);
auto const emittedKeylet = keylet::emittedTxn(*emittedID);
BEAST_REQUIRE(env.le(emittedKeylet));
// close until the cleanup pseudo has been applied
std::shared_ptr<STObject const> failureMeta;
for (int i = 0; i < 5 && !failureMeta; ++i)
{
env.close();
for (auto const& [tx, meta] : env.closed()->txs)
{
// the emitted txn itself never makes it into a ledger
BEAST_EXPECT(tx->getTransactionID() != *emittedID);
if (tx->getTxnType() == ttEMIT_FAILURE &&
tx->getFieldH256(sfTransactionHash) == *emittedID)
failureMeta = meta;
}
}
BEAST_REQUIRE(failureMeta);
// the cleanup removed the entry and its directory slot
BEAST_EXPECT(!env.le(emittedKeylet));
BEAST_EXPECT(!env.le(keylet::emittedDir()));
// the callback ran to accept() with the original emitted txn
BEAST_REQUIRE(failureMeta->isFieldPresent(sfHookExecutions));
auto const hookExecutions =
failureMeta->getFieldArray(sfHookExecutions);
BEAST_REQUIRE(hookExecutions.size() == 1);
BEAST_EXPECT(hookExecutions[0].getFieldH256(sfHookHash) == hook_hash);
BEAST_EXPECT(
hookExecutions[0].getAccountID(sfHookAccount) == alice.id());
BEAST_EXPECT(hookExecutions[0].getFieldU64(sfHookReturnCode) == 0);
BEAST_EXPECT(hookExecutions[0].getFieldU8(sfHookResult) == 3);
uint256 key;
key.data()[31] = 1;
auto const state =
env.le(keylet::hookState(alice.id(), key, uint256{beast::zero}));
BEAST_REQUIRE(state);
BEAST_EXPECT(
state->getFieldVL(sfHookStateData) ==
Blob(emittedID->begin(), emittedID->end()));
}
void
test_etxn_details(FeatureBitset features)
{
@@ -16192,6 +16516,7 @@ public:
testGuards(features);
test_emit(features); //
test_emit_failure_callback(features);
test_prepare(features);
// test_etxn_burden(features); // tested above
// test_etxn_generation(features); // tested above
+304
View File
@@ -35007,6 +35007,310 @@ std::map<std::string, std::vector<uint8_t>> wasm = {
0x65U, 0x64U, 0x2EU, 0x00U,
}},
{R"[test.hook](
#include <stdint.h>
extern int32_t _g (uint32_t id, uint32_t maxiter);
extern int64_t accept (uint32_t read_ptr, uint32_t read_len, int64_t error_code);
extern int64_t rollback (uint32_t read_ptr, uint32_t read_len, int64_t error_code);
extern int64_t emit (uint32_t write_ptr, uint32_t write_len, uint32_t read_ptr, uint32_t read_len);
extern int64_t hook_account (uint32_t write_ptr, uint32_t write_len);
extern int64_t etxn_reserve (uint32_t);
extern int64_t etxn_fee_base (uint32_t read_ptr, uint32_t read_len);
extern int64_t etxn_details (uint32_t write_ptr, uint32_t write_len);
extern int64_t ledger_seq (void);
extern int64_t otxn_id (uint32_t, uint32_t, uint32_t);
extern int64_t otxn_type (void);
extern int64_t otxn_field (uint32_t write_ptr, uint32_t write_len, uint32_t field_id);
extern int64_t state_set (uint32_t, uint32_t, uint32_t, uint32_t);
#define SBUF(x) (uint32_t)(x), sizeof(x)
#define ASSERT(x)\
if (!(x))\
rollback((uint32_t)#x, sizeof(#x), __LINE__);
#define DOESNT_EXIST (-5)
#define ttINVOKE 99
#define sfAccount ((8U << 16U) + 1U)
#define sfLedgerSequence ((2U << 16U) + 6U)
uint8_t key[32] = { [31] = 1U };
// an Invoke from the hook account, emitted with a callback
uint8_t txn[229] =
{
/* size, upto, field name */
/* 3, 0, tt = Invoke */ 0x12U, 0x00U, 0x63U,
/* 5, 3, flags */ 0x22U, 0x00U, 0x00U, 0x00U, 0x00U,
/* 5, 8, sequence */ 0x24U, 0x00U, 0x00U, 0x00U, 0x00U,
/* 6, 13, firstledgersequence */ 0x20U, 0x1AU, 0x00U, 0x00U, 0x00U, 0x00U,
/* 6, 19, lastledgersequence */ 0x20U, 0x1BU, 0x00U, 0x00U, 0x00U, 0x00U,
/* 9, 25, fee */ 0x68U, 0x40U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
/* 35, 34, signingpubkey */ 0x73U, 0x21U, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
/* 22, 69, account */ 0x81U, 0x14U, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
/* 138, 91, emit details */
/* 0, 229, */
};
#define FLS_OUT (txn + 15U)
#define LLS_OUT (txn + 21U)
#define FEE_OUT (txn + 26U)
#define ACCOUNT_OUT (txn + 71U)
#define EMIT_OUT (txn + 91U)
#define SET_UINT32(ptr, value)\
{\
uint32_t v = (value);\
(ptr)[0] = v >> 24U; (ptr)[1] = v >> 16U; (ptr)[2] = v >> 8U; (ptr)[3] = v;\
}
#define SET_NATIVE_AMOUNT(ptr, amount)\
{\
uint64_t a = (amount);\
(ptr)[0] = 0x40U + ((a >> 56U) & 0x3FU);\
(ptr)[1] = a >> 48U; (ptr)[2] = a >> 40U; (ptr)[3] = a >> 32U;\
(ptr)[4] = a >> 24U; (ptr)[5] = a >> 16U; (ptr)[6] = a >> 8U; (ptr)[7] = a;\
}
#define ACC_EQ(a, b)\
(*(uint64_t*)(a) == *(uint64_t*)(b) &&\
*(uint64_t*)((a) + 8U) == *(uint64_t*)((b) + 8U) &&\
*(uint32_t*)((a) + 16U) == *(uint32_t*)((b) + 16U))
int64_t cbak(uint32_t r)
{
_g(1,1);
// 1 = the emitted txn failed (ttEMIT_FAILURE)
ASSERT(r == 1);
// the otxn is the original emitted txn, not the pseudo
ASSERT(otxn_type() == ttINVOKE);
uint8_t me[20];
ASSERT(hook_account(SBUF(me)) == 20);
uint8_t acc[20];
ASSERT(otxn_field(SBUF(acc), sfAccount) == 20);
ASSERT(ACC_EQ(acc, me));
// carried by the pseudo but not by the emitted txn
ASSERT(otxn_field(0, 0, sfLedgerSequence) == DOESNT_EXIST);
// record the id of the emitted txn that failed
uint8_t id[32];
ASSERT(otxn_id(SBUF(id), 0) == 32);
ASSERT(state_set(SBUF(id), SBUF(key)) == 32);
return accept(0,0,0);
}
int64_t hook(uint32_t r)
{
_g(1,1);
ASSERT(etxn_reserve(1) == 1);
// FirstLedgerSequence is the current ledger, so by the time
// the emission directory is processed for the next ledger it
// has already passed: the txn is never applied and expires
// once LastLedgerSequence is behind the ledger
uint32_t seq = (uint32_t)ledger_seq();
SET_UINT32(FLS_OUT, seq);
SET_UINT32(LLS_OUT, seq + 1);
ASSERT(hook_account(ACCOUNT_OUT, 20) == 20);
ASSERT(etxn_details(EMIT_OUT, 138U) == 138);
int64_t fee = etxn_fee_base(SBUF(txn));
ASSERT(fee > 0);
SET_NATIVE_AMOUNT(FEE_OUT, fee);
uint8_t emithash[32];
ASSERT(emit(SBUF(emithash), SBUF(txn)) == 32);
return accept(0,0,0);
}
)[test.hook]",
{
0x00U, 0x61U, 0x73U, 0x6DU, 0x01U, 0x00U, 0x00U, 0x00U, 0x01U, 0x2CU,
0x07U, 0x60U, 0x02U, 0x7FU, 0x7FU, 0x01U, 0x7FU, 0x60U, 0x03U, 0x7FU,
0x7FU, 0x7EU, 0x01U, 0x7EU, 0x60U, 0x00U, 0x01U, 0x7EU, 0x60U, 0x02U,
0x7FU, 0x7FU, 0x01U, 0x7EU, 0x60U, 0x03U, 0x7FU, 0x7FU, 0x7FU, 0x01U,
0x7EU, 0x60U, 0x04U, 0x7FU, 0x7FU, 0x7FU, 0x7FU, 0x01U, 0x7EU, 0x60U,
0x01U, 0x7FU, 0x01U, 0x7EU, 0x02U, 0xCEU, 0x01U, 0x0DU, 0x03U, 0x65U,
0x6EU, 0x76U, 0x02U, 0x5FU, 0x67U, 0x00U, 0x00U, 0x03U, 0x65U, 0x6EU,
0x76U, 0x08U, 0x72U, 0x6FU, 0x6CU, 0x6CU, 0x62U, 0x61U, 0x63U, 0x6BU,
0x00U, 0x01U, 0x03U, 0x65U, 0x6EU, 0x76U, 0x09U, 0x6FU, 0x74U, 0x78U,
0x6EU, 0x5FU, 0x74U, 0x79U, 0x70U, 0x65U, 0x00U, 0x02U, 0x03U, 0x65U,
0x6EU, 0x76U, 0x0CU, 0x68U, 0x6FU, 0x6FU, 0x6BU, 0x5FU, 0x61U, 0x63U,
0x63U, 0x6FU, 0x75U, 0x6EU, 0x74U, 0x00U, 0x03U, 0x03U, 0x65U, 0x6EU,
0x76U, 0x0AU, 0x6FU, 0x74U, 0x78U, 0x6EU, 0x5FU, 0x66U, 0x69U, 0x65U,
0x6CU, 0x64U, 0x00U, 0x04U, 0x03U, 0x65U, 0x6EU, 0x76U, 0x07U, 0x6FU,
0x74U, 0x78U, 0x6EU, 0x5FU, 0x69U, 0x64U, 0x00U, 0x04U, 0x03U, 0x65U,
0x6EU, 0x76U, 0x09U, 0x73U, 0x74U, 0x61U, 0x74U, 0x65U, 0x5FU, 0x73U,
0x65U, 0x74U, 0x00U, 0x05U, 0x03U, 0x65U, 0x6EU, 0x76U, 0x06U, 0x61U,
0x63U, 0x63U, 0x65U, 0x70U, 0x74U, 0x00U, 0x01U, 0x03U, 0x65U, 0x6EU,
0x76U, 0x0CU, 0x65U, 0x74U, 0x78U, 0x6EU, 0x5FU, 0x72U, 0x65U, 0x73U,
0x65U, 0x72U, 0x76U, 0x65U, 0x00U, 0x06U, 0x03U, 0x65U, 0x6EU, 0x76U,
0x0AU, 0x6CU, 0x65U, 0x64U, 0x67U, 0x65U, 0x72U, 0x5FU, 0x73U, 0x65U,
0x71U, 0x00U, 0x02U, 0x03U, 0x65U, 0x6EU, 0x76U, 0x0CU, 0x65U, 0x74U,
0x78U, 0x6EU, 0x5FU, 0x64U, 0x65U, 0x74U, 0x61U, 0x69U, 0x6CU, 0x73U,
0x00U, 0x03U, 0x03U, 0x65U, 0x6EU, 0x76U, 0x0DU, 0x65U, 0x74U, 0x78U,
0x6EU, 0x5FU, 0x66U, 0x65U, 0x65U, 0x5FU, 0x62U, 0x61U, 0x73U, 0x65U,
0x00U, 0x03U, 0x03U, 0x65U, 0x6EU, 0x76U, 0x04U, 0x65U, 0x6DU, 0x69U,
0x74U, 0x00U, 0x05U, 0x03U, 0x03U, 0x02U, 0x06U, 0x06U, 0x05U, 0x03U,
0x01U, 0x00U, 0x02U, 0x06U, 0x08U, 0x01U, 0x7FU, 0x01U, 0x41U, 0x90U,
0x8DU, 0x04U, 0x0BU, 0x07U, 0x0FU, 0x02U, 0x04U, 0x63U, 0x62U, 0x61U,
0x6BU, 0x00U, 0x0DU, 0x04U, 0x68U, 0x6FU, 0x6FU, 0x6BU, 0x00U, 0x0EU,
0x0AU, 0x85U, 0x87U, 0x00U, 0x02U, 0x88U, 0x83U, 0x00U, 0x02U, 0x01U,
0x7FU, 0x01U, 0x7EU, 0x23U, 0x80U, 0x80U, 0x80U, 0x80U, 0x00U, 0x41U,
0xE0U, 0x00U, 0x6BU, 0x22U, 0x01U, 0x24U, 0x80U, 0x80U, 0x80U, 0x80U,
0x00U, 0x41U, 0x01U, 0x41U, 0x01U, 0x10U, 0x80U, 0x80U, 0x80U, 0x80U,
0x00U, 0x1AU, 0x02U, 0x40U, 0x20U, 0x00U, 0x41U, 0x01U, 0x46U, 0x0DU,
0x00U, 0x41U, 0xD5U, 0x89U, 0x80U, 0x80U, 0x00U, 0x41U, 0x07U, 0x42U,
0xC5U, 0x00U, 0x10U, 0x81U, 0x80U, 0x80U, 0x80U, 0x00U, 0x1AU, 0x0BU,
0x02U, 0x40U, 0x10U, 0x82U, 0x80U, 0x80U, 0x80U, 0x00U, 0x42U, 0xE3U,
0x00U, 0x51U, 0x0DU, 0x00U, 0x41U, 0xB3U, 0x88U, 0x80U, 0x80U, 0x00U,
0x41U, 0x18U, 0x42U, 0xC8U, 0x00U, 0x10U, 0x81U, 0x80U, 0x80U, 0x80U,
0x00U, 0x1AU, 0x0BU, 0x02U, 0x40U, 0x20U, 0x01U, 0x41U, 0xC0U, 0x00U,
0x6AU, 0x41U, 0x14U, 0x10U, 0x83U, 0x80U, 0x80U, 0x80U, 0x00U, 0x42U,
0x14U, 0x51U, 0x0DU, 0x00U, 0x41U, 0xBCU, 0x8AU, 0x80U, 0x80U, 0x00U,
0x41U, 0x1DU, 0x42U, 0xCBU, 0x00U, 0x10U, 0x81U, 0x80U, 0x80U, 0x80U,
0x00U, 0x1AU, 0x0BU, 0x02U, 0x40U, 0x20U, 0x01U, 0x41U, 0x20U, 0x6AU,
0x41U, 0x14U, 0x41U, 0x81U, 0x80U, 0x20U, 0x10U, 0x84U, 0x80U, 0x80U,
0x80U, 0x00U, 0x42U, 0x14U, 0x51U, 0x0DU, 0x00U, 0x41U, 0xF1U, 0x89U,
0x80U, 0x80U, 0x00U, 0x41U, 0x27U, 0x42U, 0xCDU, 0x00U, 0x10U, 0x81U,
0x80U, 0x80U, 0x80U, 0x00U, 0x1AU, 0x0BU, 0x02U, 0x40U, 0x02U, 0x40U,
0x20U, 0x01U, 0x29U, 0x03U, 0x20U, 0x20U, 0x01U, 0x29U, 0x03U, 0x40U,
0x52U, 0x0DU, 0x00U, 0x20U, 0x01U, 0x29U, 0x03U, 0x28U, 0x20U, 0x01U,
0x29U, 0x03U, 0x48U, 0x52U, 0x0DU, 0x00U, 0x20U, 0x01U, 0x28U, 0x02U,
0x30U, 0x20U, 0x01U, 0x28U, 0x02U, 0x50U, 0x46U, 0x0DU, 0x01U, 0x0BU,
0x41U, 0xE1U, 0x8AU, 0x80U, 0x80U, 0x00U, 0x41U, 0x10U, 0x42U, 0xCEU,
0x00U, 0x10U, 0x81U, 0x80U, 0x80U, 0x80U, 0x00U, 0x1AU, 0x0BU, 0x02U,
0x40U, 0x41U, 0x00U, 0x41U, 0x00U, 0x41U, 0x86U, 0x80U, 0x08U, 0x10U,
0x84U, 0x80U, 0x80U, 0x80U, 0x00U, 0x42U, 0x7BU, 0x51U, 0x0DU, 0x00U,
0x41U, 0x80U, 0x88U, 0x80U, 0x80U, 0x00U, 0x41U, 0x33U, 0x42U, 0xD1U,
0x00U, 0x10U, 0x81U, 0x80U, 0x80U, 0x80U, 0x00U, 0x1AU, 0x0BU, 0x02U,
0x40U, 0x20U, 0x01U, 0x41U, 0x20U, 0x41U, 0x00U, 0x10U, 0x85U, 0x80U,
0x80U, 0x80U, 0x00U, 0x42U, 0x20U, 0x51U, 0x0DU, 0x00U, 0x41U, 0xEFU,
0x88U, 0x80U, 0x80U, 0x00U, 0x41U, 0x1BU, 0x42U, 0xD5U, 0x00U, 0x10U,
0x81U, 0x80U, 0x80U, 0x80U, 0x00U, 0x1AU, 0x0BU, 0x02U, 0x40U, 0x20U,
0x01U, 0x41U, 0x20U, 0x41U, 0x80U, 0x8BU, 0x80U, 0x80U, 0x00U, 0x41U,
0x20U, 0x10U, 0x86U, 0x80U, 0x80U, 0x80U, 0x00U, 0x42U, 0x20U, 0x51U,
0x0DU, 0x00U, 0x41U, 0x8AU, 0x89U, 0x80U, 0x80U, 0x00U, 0x41U, 0x25U,
0x42U, 0xD6U, 0x00U, 0x10U, 0x81U, 0x80U, 0x80U, 0x80U, 0x00U, 0x1AU,
0x0BU, 0x41U, 0x00U, 0x41U, 0x00U, 0x42U, 0x00U, 0x10U, 0x87U, 0x80U,
0x80U, 0x80U, 0x00U, 0x21U, 0x02U, 0x20U, 0x01U, 0x41U, 0xE0U, 0x00U,
0x6AU, 0x24U, 0x80U, 0x80U, 0x80U, 0x80U, 0x00U, 0x20U, 0x02U, 0x0BU,
0xF6U, 0x83U, 0x00U, 0x03U, 0x01U, 0x7FU, 0x01U, 0x7EU, 0x01U, 0x7FU,
0x23U, 0x80U, 0x80U, 0x80U, 0x80U, 0x00U, 0x41U, 0x20U, 0x6BU, 0x22U,
0x01U, 0x24U, 0x80U, 0x80U, 0x80U, 0x80U, 0x00U, 0x41U, 0x01U, 0x41U,
0x01U, 0x10U, 0x80U, 0x80U, 0x80U, 0x80U, 0x00U, 0x1AU, 0x02U, 0x40U,
0x41U, 0x01U, 0x10U, 0x88U, 0x80U, 0x80U, 0x80U, 0x00U, 0x42U, 0x01U,
0x51U, 0x0DU, 0x00U, 0x41U, 0xDCU, 0x89U, 0x80U, 0x80U, 0x00U, 0x41U,
0x15U, 0x42U, 0xDFU, 0x00U, 0x10U, 0x81U, 0x80U, 0x80U, 0x80U, 0x00U,
0x1AU, 0x0BU, 0x41U, 0x00U, 0x10U, 0x89U, 0x80U, 0x80U, 0x80U, 0x00U,
0x22U, 0x02U, 0x3CU, 0x00U, 0xB2U, 0x8BU, 0x80U, 0x80U, 0x00U, 0x41U,
0x00U, 0x20U, 0x02U, 0x42U, 0x08U, 0x88U, 0x3CU, 0x00U, 0xB1U, 0x8BU,
0x80U, 0x80U, 0x00U, 0x41U, 0x00U, 0x20U, 0x02U, 0x42U, 0x10U, 0x88U,
0x3CU, 0x00U, 0xB0U, 0x8BU, 0x80U, 0x80U, 0x00U, 0x41U, 0x00U, 0x20U,
0x02U, 0x42U, 0x18U, 0x88U, 0x3CU, 0x00U, 0xAFU, 0x8BU, 0x80U, 0x80U,
0x00U, 0x41U, 0x00U, 0x20U, 0x02U, 0xA7U, 0x41U, 0x01U, 0x6AU, 0x22U,
0x03U, 0x3AU, 0x00U, 0xB8U, 0x8BU, 0x80U, 0x80U, 0x00U, 0x41U, 0x00U,
0x20U, 0x03U, 0x41U, 0x08U, 0x76U, 0x3AU, 0x00U, 0xB7U, 0x8BU, 0x80U,
0x80U, 0x00U, 0x41U, 0x00U, 0x20U, 0x03U, 0x41U, 0x10U, 0x76U, 0x3AU,
0x00U, 0xB6U, 0x8BU, 0x80U, 0x80U, 0x00U, 0x41U, 0x00U, 0x20U, 0x03U,
0x41U, 0x18U, 0x76U, 0x3AU, 0x00U, 0xB5U, 0x8BU, 0x80U, 0x80U, 0x00U,
0x02U, 0x40U, 0x41U, 0xE7U, 0x8BU, 0x80U, 0x80U, 0x00U, 0x41U, 0x14U,
0x10U, 0x83U, 0x80U, 0x80U, 0x80U, 0x00U, 0x42U, 0x14U, 0x51U, 0x0DU,
0x00U, 0x41U, 0x98U, 0x8AU, 0x80U, 0x80U, 0x00U, 0x41U, 0x24U, 0x42U,
0xE8U, 0x00U, 0x10U, 0x81U, 0x80U, 0x80U, 0x80U, 0x00U, 0x1AU, 0x0BU,
0x02U, 0x40U, 0x41U, 0xFBU, 0x8BU, 0x80U, 0x80U, 0x00U, 0x41U, 0x8AU,
0x01U, 0x10U, 0x8AU, 0x80U, 0x80U, 0x80U, 0x00U, 0x42U, 0x8AU, 0x01U,
0x51U, 0x0DU, 0x00U, 0x41U, 0xCBU, 0x88U, 0x80U, 0x80U, 0x00U, 0x41U,
0x24U, 0x42U, 0xE9U, 0x00U, 0x10U, 0x81U, 0x80U, 0x80U, 0x80U, 0x00U,
0x1AU, 0x0BU, 0x02U, 0x40U, 0x41U, 0xA0U, 0x8BU, 0x80U, 0x80U, 0x00U,
0x41U, 0xE5U, 0x01U, 0x10U, 0x8BU, 0x80U, 0x80U, 0x80U, 0x00U, 0x22U,
0x02U, 0x42U, 0x00U, 0x55U, 0x0DU, 0x00U, 0x41U, 0xD9U, 0x8AU, 0x80U,
0x80U, 0x00U, 0x41U, 0x08U, 0x42U, 0xEBU, 0x00U, 0x10U, 0x81U, 0x80U,
0x80U, 0x80U, 0x00U, 0x1AU, 0x0BU, 0x41U, 0x00U, 0x20U, 0x02U, 0x3CU,
0x00U, 0xC1U, 0x8BU, 0x80U, 0x80U, 0x00U, 0x41U, 0x00U, 0x20U, 0x02U,
0x42U, 0x08U, 0x88U, 0x3CU, 0x00U, 0xC0U, 0x8BU, 0x80U, 0x80U, 0x00U,
0x41U, 0x00U, 0x20U, 0x02U, 0x42U, 0x10U, 0x88U, 0x3CU, 0x00U, 0xBFU,
0x8BU, 0x80U, 0x80U, 0x00U, 0x41U, 0x00U, 0x20U, 0x02U, 0x42U, 0x18U,
0x88U, 0x3CU, 0x00U, 0xBEU, 0x8BU, 0x80U, 0x80U, 0x00U, 0x41U, 0x00U,
0x20U, 0x02U, 0x42U, 0x20U, 0x88U, 0x3CU, 0x00U, 0xBDU, 0x8BU, 0x80U,
0x80U, 0x00U, 0x41U, 0x00U, 0x20U, 0x02U, 0x42U, 0x28U, 0x88U, 0x3CU,
0x00U, 0xBCU, 0x8BU, 0x80U, 0x80U, 0x00U, 0x41U, 0x00U, 0x20U, 0x02U,
0x42U, 0x30U, 0x88U, 0x3CU, 0x00U, 0xBBU, 0x8BU, 0x80U, 0x80U, 0x00U,
0x41U, 0x00U, 0x20U, 0x02U, 0x42U, 0x38U, 0x88U, 0xA7U, 0x41U, 0x3FU,
0x71U, 0x41U, 0xC0U, 0x00U, 0x72U, 0x3AU, 0x00U, 0xBAU, 0x8BU, 0x80U,
0x80U, 0x00U, 0x02U, 0x40U, 0x20U, 0x01U, 0x41U, 0x20U, 0x41U, 0xA0U,
0x8BU, 0x80U, 0x80U, 0x00U, 0x41U, 0xE5U, 0x01U, 0x10U, 0x8CU, 0x80U,
0x80U, 0x80U, 0x00U, 0x42U, 0x20U, 0x51U, 0x0DU, 0x00U, 0x41U, 0xAFU,
0x89U, 0x80U, 0x80U, 0x00U, 0x41U, 0x26U, 0x42U, 0xEFU, 0x00U, 0x10U,
0x81U, 0x80U, 0x80U, 0x80U, 0x00U, 0x1AU, 0x0BU, 0x41U, 0x00U, 0x41U,
0x00U, 0x42U, 0x00U, 0x10U, 0x87U, 0x80U, 0x80U, 0x80U, 0x00U, 0x21U,
0x02U, 0x20U, 0x01U, 0x41U, 0x20U, 0x6AU, 0x24U, 0x80U, 0x80U, 0x80U,
0x80U, 0x00U, 0x20U, 0x02U, 0x0BU, 0x0BU, 0x85U, 0x05U, 0x02U, 0x00U,
0x41U, 0x80U, 0x08U, 0x0BU, 0xF1U, 0x02U, 0x6FU, 0x74U, 0x78U, 0x6EU,
0x5FU, 0x66U, 0x69U, 0x65U, 0x6CU, 0x64U, 0x28U, 0x30U, 0x2CU, 0x20U,
0x30U, 0x2CU, 0x20U, 0x73U, 0x66U, 0x4CU, 0x65U, 0x64U, 0x67U, 0x65U,
0x72U, 0x53U, 0x65U, 0x71U, 0x75U, 0x65U, 0x6EU, 0x63U, 0x65U, 0x29U,
0x20U, 0x3DU, 0x3DU, 0x20U, 0x44U, 0x4FU, 0x45U, 0x53U, 0x4EU, 0x54U,
0x5FU, 0x45U, 0x58U, 0x49U, 0x53U, 0x54U, 0x00U, 0x6FU, 0x74U, 0x78U,
0x6EU, 0x5FU, 0x74U, 0x79U, 0x70U, 0x65U, 0x28U, 0x29U, 0x20U, 0x3DU,
0x3DU, 0x20U, 0x74U, 0x74U, 0x49U, 0x4EU, 0x56U, 0x4FU, 0x4BU, 0x45U,
0x00U, 0x65U, 0x74U, 0x78U, 0x6EU, 0x5FU, 0x64U, 0x65U, 0x74U, 0x61U,
0x69U, 0x6CU, 0x73U, 0x28U, 0x45U, 0x4DU, 0x49U, 0x54U, 0x5FU, 0x4FU,
0x55U, 0x54U, 0x2CU, 0x20U, 0x31U, 0x33U, 0x38U, 0x55U, 0x29U, 0x20U,
0x3DU, 0x3DU, 0x20U, 0x31U, 0x33U, 0x38U, 0x00U, 0x6FU, 0x74U, 0x78U,
0x6EU, 0x5FU, 0x69U, 0x64U, 0x28U, 0x53U, 0x42U, 0x55U, 0x46U, 0x28U,
0x69U, 0x64U, 0x29U, 0x2CU, 0x20U, 0x30U, 0x29U, 0x20U, 0x3DU, 0x3DU,
0x20U, 0x33U, 0x32U, 0x00U, 0x73U, 0x74U, 0x61U, 0x74U, 0x65U, 0x5FU,
0x73U, 0x65U, 0x74U, 0x28U, 0x53U, 0x42U, 0x55U, 0x46U, 0x28U, 0x69U,
0x64U, 0x29U, 0x2CU, 0x20U, 0x53U, 0x42U, 0x55U, 0x46U, 0x28U, 0x6BU,
0x65U, 0x79U, 0x29U, 0x29U, 0x20U, 0x3DU, 0x3DU, 0x20U, 0x33U, 0x32U,
0x00U, 0x65U, 0x6DU, 0x69U, 0x74U, 0x28U, 0x53U, 0x42U, 0x55U, 0x46U,
0x28U, 0x65U, 0x6DU, 0x69U, 0x74U, 0x68U, 0x61U, 0x73U, 0x68U, 0x29U,
0x2CU, 0x20U, 0x53U, 0x42U, 0x55U, 0x46U, 0x28U, 0x74U, 0x78U, 0x6EU,
0x29U, 0x29U, 0x20U, 0x3DU, 0x3DU, 0x20U, 0x33U, 0x32U, 0x00U, 0x72U,
0x20U, 0x3DU, 0x3DU, 0x20U, 0x31U, 0x00U, 0x65U, 0x74U, 0x78U, 0x6EU,
0x5FU, 0x72U, 0x65U, 0x73U, 0x65U, 0x72U, 0x76U, 0x65U, 0x28U, 0x31U,
0x29U, 0x20U, 0x3DU, 0x3DU, 0x20U, 0x31U, 0x00U, 0x6FU, 0x74U, 0x78U,
0x6EU, 0x5FU, 0x66U, 0x69U, 0x65U, 0x6CU, 0x64U, 0x28U, 0x53U, 0x42U,
0x55U, 0x46U, 0x28U, 0x61U, 0x63U, 0x63U, 0x29U, 0x2CU, 0x20U, 0x73U,
0x66U, 0x41U, 0x63U, 0x63U, 0x6FU, 0x75U, 0x6EU, 0x74U, 0x29U, 0x20U,
0x3DU, 0x3DU, 0x20U, 0x32U, 0x30U, 0x00U, 0x68U, 0x6FU, 0x6FU, 0x6BU,
0x5FU, 0x61U, 0x63U, 0x63U, 0x6FU, 0x75U, 0x6EU, 0x74U, 0x28U, 0x41U,
0x43U, 0x43U, 0x4FU, 0x55U, 0x4EU, 0x54U, 0x5FU, 0x4FU, 0x55U, 0x54U,
0x2CU, 0x20U, 0x32U, 0x30U, 0x29U, 0x20U, 0x3DU, 0x3DU, 0x20U, 0x32U,
0x30U, 0x00U, 0x68U, 0x6FU, 0x6FU, 0x6BU, 0x5FU, 0x61U, 0x63U, 0x63U,
0x6FU, 0x75U, 0x6EU, 0x74U, 0x28U, 0x53U, 0x42U, 0x55U, 0x46U, 0x28U,
0x6DU, 0x65U, 0x29U, 0x29U, 0x20U, 0x3DU, 0x3DU, 0x20U, 0x32U, 0x30U,
0x00U, 0x66U, 0x65U, 0x65U, 0x20U, 0x3EU, 0x20U, 0x30U, 0x00U, 0x41U,
0x43U, 0x43U, 0x5FU, 0x45U, 0x51U, 0x28U, 0x61U, 0x63U, 0x63U, 0x2CU,
0x20U, 0x6DU, 0x65U, 0x29U, 0x00U, 0x00U, 0x41U, 0x80U, 0x0BU, 0x0BU,
0x85U, 0x02U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x01U, 0x12U, 0x00U, 0x63U, 0x22U, 0x00U, 0x00U,
0x00U, 0x00U, 0x24U, 0x00U, 0x00U, 0x00U, 0x00U, 0x20U, 0x1AU, 0x00U,
0x00U, 0x00U, 0x00U, 0x20U, 0x1BU, 0x00U, 0x00U, 0x00U, 0x00U, 0x68U,
0x40U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x73U, 0x21U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x81U, 0x14U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U, 0x00U,
0x00U, 0x00U, 0x00U,
}},
};
}
} // namespace ripple
+18 -2
View File
@@ -5055,7 +5055,15 @@ public:
run() override
{
using namespace test::jtx;
FeatureBitset const all{supported_amendments() - featureXahauGenesis};
// featureRNG is excluded: makeConfig() runs as a validator whenever
// voting settings are given, and a validator puts a ttENTROPY with a
// fresh random commitment into every ledger. That makes the
// transaction set hash, and so the canonical order in which
// different accounts' transactions apply when the ledger closes,
// differ from run to run, which breaks tests that assert balances
// depending on that order (e.g. unexpected balance change).
FeatureBitset const all{
supported_amendments() - featureXahauGenesis - featureRNG};
testQueueSeq(all);
testQueueTicket(all);
testTecResult(all);
@@ -5082,7 +5090,15 @@ public:
runMetaInfo()
{
using namespace test::jtx;
FeatureBitset const all{supported_amendments() - featureXahauGenesis};
// featureRNG is excluded: makeConfig() runs as a validator whenever
// voting settings are given, and a validator puts a ttENTROPY with a
// fresh random commitment into every ledger. That makes the
// transaction set hash, and so the canonical order in which
// different accounts' transactions apply when the ledger closes,
// differ from run to run, which breaks tests that assert balances
// depending on that order (e.g. unexpected balance change).
FeatureBitset const all{
supported_amendments() - featureXahauGenesis - featureRNG};
testAcctInQueueButEmpty(all);
testRPC(all);
testExpirationReplacement(all);
+193 -6
View File
@@ -28,6 +28,7 @@
#include <xrpl/protocol/STAccount.h>
#include <xrpl/protocol/digest.h>
#include <xrpl/protocol/jss.h>
#include <limits>
#include <string>
#include <vector>
@@ -4676,11 +4677,12 @@ struct XahauGenesis_test : public beast::unit_test::suite
// validate account fields
STAmount const postUser = preUser + netReward;
bool const has240819 = env.current()->rules().enabled(fix240819);
bool const has20261005 = env.current()->rules().enabled(fix20261005);
BEAST_EXPECT(expectAccountFields(
env,
user,
preLedger,
has240819 ? preLedger : preLedger + 1,
(has240819 && !has20261005) ? preLedger : preLedger + 1,
has240819 ? (preUser - feesXRP) : postUser,
preTime));
}
@@ -5567,8 +5569,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
env.close();
// calculate rewards
bool const hasFix = env.current()->rules().enabled(fix240819) &&
env.current()->rules().enabled(fix240911);
bool const hasFix = env.current()->rules().enabled(fix240819);
STAmount const netReward =
rewardUserAmount(*acctSle, preLedger, rateDrops);
BEAST_EXPECT(netReward == (hasFix ? XRP(3.329999) : XRP(3.329999)));
@@ -5659,8 +5660,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
env.close();
// calculate rewards
bool const hasFix = env.current()->rules().enabled(fix240819) &&
env.current()->rules().enabled(fix240911);
bool const hasFix = env.current()->rules().enabled(fix240819);
STAmount const netReward =
rewardUserAmount(*acctSle, preLedger, rateDrops);
BEAST_EXPECT(netReward == (hasFix ? XRP(3.479999) : XRP(6.663333)));
@@ -5678,9 +5678,196 @@ struct XahauGenesis_test : public beast::unit_test::suite
postUser == (hasFix ? XRP(2002.479999) : XRP(2005.663333)));
}
void
test20261005(FeatureBitset features)
{
using namespace jtx;
using namespace std::chrono_literals;
testcase("test fix20261005");
bool const has240819 = features[fix240819];
bool const has20261005 = features[fix20261005];
bool const nofix = !(has240819 && has20261005);
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
STAmount const feesXRP = XRP(1);
Account const user = Account("user");
env.fund(XRP(200), user);
env.close();
// setup governance
Account const alice = Account("alice");
Account const bob = Account("bob");
Account const carol = Account("carol");
Account const david = Account("david");
Account const edward = Account("edward");
env.fund(XRP(10000), alice, bob, carol, david, edward);
env.close();
std::vector<AccountID> initial_members_ids{
alice.id(), bob.id(), carol.id(), david.id(), edward.id()};
setupGov(env, initial_members_ids);
{
std::vector<uint8_t> vote_data{
0x00U, 0x80U, 0xC6U, 0xA4U, 0x7EU, 0x8DU, 0x03U, 0x55U};
updateTopic(
env, alice, bob, carol, david, edward, 'R', 'D', vote_data);
}
BEAST_EXPECT(hasUNLReport(env) == false);
env(claimReward(user, env.master), fee(feesXRP), ter(tesSUCCESS));
env.close();
env(noop(user),
fee(env.balance(user) - drops(500'000)),
ter(tesSUCCESS));
env.close();
BEAST_EXPECT(env.balance(user) == drops(500'000));
BEAST_EXPECT(env.balance(user) < XRP(1));
std::uint32_t const drainMarker =
env.le(keylet::account(user))->getFieldU32(sfRewardLgrLast);
for (int i = 0; i < 20; ++i)
env.close(10s);
BEAST_EXPECT(env.balance(user) < XRP(1));
env(pay(alice, user, XRP(1000)));
env.close();
std::uint32_t const fundMarker =
env.le(keylet::account(user))->getFieldU32(sfRewardLgrLast);
bool const markerFrozen = has240819 && !has20261005;
if (markerFrozen)
BEAST_EXPECT(fundMarker == drainMarker);
else
BEAST_EXPECT(fundMarker > drainMarker);
STAmount const preUser = env.balance(user);
env(claimReward(user, env.master), fee(feesXRP), ter(tesSUCCESS));
env.close();
env.close();
STAmount const postUser = env.balance(user);
if (nofix)
BEAST_EXPECT(postUser > preUser + XRP(1));
else
BEAST_EXPECT(postUser < preUser + XRP(1));
}
void
testRewardAccumulatorOverflow(FeatureBitset features)
{
using namespace jtx;
testcase("reward accumulator overflow");
bool const has20261005 = features[fix20261005];
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
STAmount const feesXRP = XRP(1);
Account const user = Account("user");
env.fund(XRP(10000), user);
env.close();
// setup governance
Account const alice = Account("alice");
Account const bob = Account("bob");
Account const carol = Account("carol");
Account const david = Account("david");
Account const edward = Account("edward");
env.fund(XRP(10000), alice, bob, carol, david, edward);
env.close();
std::vector<AccountID> initial_members_ids{
alice.id(), bob.id(), carol.id(), david.id(), edward.id()};
setupGov(env, initial_members_ids);
{
std::vector<uint8_t> vote_data{
0x00U, 0x80U, 0xC6U, 0xA4U, 0x7EU, 0x8DU, 0x03U, 0x55U};
updateTopic(
env, alice, bob, carol, david, edward, 'R', 'D', vote_data);
}
// opt in so the account carries the reward fields
env(claimReward(user, env.master), fee(feesXRP), ter(tesSUCCESS));
env.close();
env.close();
auto const preSle = env.le(keylet::account(user));
BEAST_REQUIRE(preSle);
BEAST_REQUIRE(preSle->isFieldPresent(sfRewardLgrLast));
BEAST_REQUIRE(preSle->isFieldPresent(sfRewardAccumulator));
std::uint32_t const preMarker = preSle->getFieldU32(sfRewardLgrLast);
BEAST_EXPECT(preMarker < env.current()->seq());
constexpr std::uint64_t maxAccum =
std::numeric_limits<std::uint64_t>::max();
// Push the accumulator to the brink in the open ledger so that any
// non-zero bal * lgrElapsed wraps around in finishApply.
env.app().openLedger().modify(
[&](OpenView& view, beast::Journal) -> bool {
auto const sle = view.read(keylet::account(user));
if (!sle)
return false;
auto replacement = std::make_shared<SLE>(*sle);
replacement->setFieldU64(sfRewardAccumulator, maxAccum);
view.rawReplace(replacement);
return true;
});
BEAST_EXPECT(
env.le(keylet::account(user))->getFieldU64(sfRewardAccumulator) ==
maxAccum);
// Any balance change on user runs the reward block in finishApply.
env(pay(alice, user, XRP(1)), ter(tesSUCCESS));
// Check the open ledger before close() rebuilds it without the
// injected accumulator.
auto const postSle = env.le(keylet::account(user));
BEAST_REQUIRE(postSle);
// the overflowing contribution is never added
BEAST_EXPECT(postSle->getFieldU64(sfRewardAccumulator) == maxAccum);
// with the fix the marker advances so the span can't be reused;
// without it the marker is left behind
if (has20261005)
BEAST_EXPECT(
postSle->getFieldU32(sfRewardLgrLast) == env.current()->seq());
else
BEAST_EXPECT(postSle->getFieldU32(sfRewardLgrLast) == preMarker);
env.close();
}
void
testRewardHookWithFeats(FeatureBitset features)
{
test20261005(features);
testRewardAccumulatorOverflow(features);
testLastCloseTime(features);
testInvalidRate0(features);
testInvalidRateGreater1(features);
@@ -5723,7 +5910,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
testGovernHookWithFeats(sa);
testRewardHookWithFeats(sa);
testRewardHookWithFeats(sa - fix240819);
testRewardHookWithFeats(sa - fix240819 - fix240911);
testRewardHookWithFeats(sa - fix20261005);
}
};
+62
View File
@@ -1052,6 +1052,67 @@ public:
BEAST_EXPECT(sim.synchronized());
}
void
testForcedInclusion()
{
using namespace csf;
using namespace std::chrono;
testcase("forced inclusion");
// Transaction n * 1000 + k belongs to peer n - 1. Each one is placed
// only in a single peer's open set and never relayed, so without
// forced inclusion it has one vote in five and stays out.
auto const owner = [](Tx const& tx) -> std::optional<PeerID> {
if (tx.id() < 1000)
return std::nullopt;
return PeerID{tx.id() / 1000 - 1};
};
auto const run = [&](bool withHook) {
ConsensusParms const parms{};
Sim sim;
PeerGroup peers = sim.createGroup(5);
peers.trustAndConnect(
peers, round<milliseconds>(0.2 * parms.ledgerGRANULARITY));
for (Peer* p : peers)
{
if (withHook)
p->txOwner = owner;
std::uint32_t const base =
(static_cast<std::uint32_t>(p->id) + 1) * 1000;
p->openTxs.insert(Tx{base + 1});
if (p->id == PeerID{4})
{
// more of its own than it may force
p->openTxs.insert(Tx{base + 2});
p->openTxs.insert(Tx{base + 3});
// one that names another owner, who does not hold it
p->openTxs.insert(Tx{1999});
}
}
sim.run(1);
BEAST_EXPECT(sim.synchronized());
return peers[0]->lastClosedLedger.txs();
};
{
auto const txs = run(false);
BEAST_EXPECT(txs.empty());
}
{
auto const txs = run(true);
// every owner's own transaction, and only the lowest of the
// several peer 4 put in its position
TxSetType const expected{
Tx{1001}, Tx{2001}, Tx{3001}, Tx{4001}, Tx{5001}};
BEAST_EXPECTS(txs == expected, to_string(txs));
}
}
void
run() override
{
@@ -1068,6 +1129,7 @@ public:
testHubNetwork();
testPreferredByBranch();
testPauseForLaggards();
testForcedInclusion();
}
};
+65
View File
@@ -454,6 +454,39 @@ port_wss_admin
}
}
void
testUdpHighway()
{
testcase("udp_highway");
// The UDP Superhighway is opt-in: off unless turned on.
{
Config c;
c.loadFromString("");
BEAST_EXPECT(!c.UDP_HIGHWAY);
}
std::vector<std::pair<std::string, bool>> const values{
{"0", false}, {"false", false}, {"1", true}, {"TRUE", true}};
for (auto const& [value, expected] : values)
{
Config c;
c.loadFromString("[udp_highway]\n" + value + "\n");
BEAST_EXPECT(c.UDP_HIGHWAY == expected);
}
try
{
Config c;
c.loadFromString("[udp_highway]\nsometimes\n");
fail();
}
catch (std::exception const&)
{
pass();
}
}
void
testNetworkID()
{
@@ -1089,6 +1122,36 @@ trustthesevalidators.gov
}
}
// Standalone forces ledger_history to zero, but [node_db] type=rwdb
// (the null node store) sizes its retained-ledger window from it and
// SHAMapStoreImp throws on zero. cfg/rippled.cfg ships exactly this
// configuration, and jtx never calls Config::setup, so pin it here.
void
testSetupStandaloneRwdb()
{
testcase("setup standalone rwdb keeps ledger_history");
for (std::string const type : {"rwdb", "NuDB"})
{
detail::RippledCfgGuard const cfg(
*this,
"testSetupStandaloneRwdb_" + type,
"",
"",
true,
"[node_db]\ntype=" + type + "\npath=main\n");
Config config;
config.setup(
cfg.configFile(),
/*bQuiet*/ true,
/* bSilent */ true,
/* bStandalone */ true);
BEAST_EXPECT(config.standalone());
BEAST_EXPECT(config.nullBackend() == (type == "rwdb"));
BEAST_EXPECT(config.LEDGER_HISTORY == (type == "rwdb" ? 256 : 0));
}
}
void
testPort()
{
@@ -1780,6 +1843,7 @@ r.ripple.com:51235
testValidatorsFile();
testSetup(false);
testSetup(true);
testSetupStandaloneRwdb();
testPort();
testZeroPort();
testWhitespace();
@@ -1792,6 +1856,7 @@ r.ripple.com:51235
testOverlay();
testNetworkID();
testEarlyManifestSync();
testUdpHighway();
}
};
+15
View File
@@ -33,6 +33,8 @@
#include <boost/container/flat_map.hpp>
#include <boost/container/flat_set.hpp>
#include <algorithm>
#include <functional>
#include <optional>
namespace ripple {
namespace test {
@@ -257,6 +259,11 @@ struct Peer
// Simulation parameters
ConsensusParms consensusParms;
//! Names the peer a transaction belongs to, for Consensus forced
//! inclusion (as featureRNG uses for entropy contributions). Unset, it
//! behaves exactly like an adaptor without the forcedTxOwner hook.
std::function<std::optional<PeerID>(Tx const&)> txOwner;
//! The collectors to report events to
CollectorRefs& collectors;
@@ -655,6 +662,14 @@ struct Peer
return consensusParms;
}
std::optional<PeerID>
forcedTxOwner(Tx const& tx, Ledger const&) const
{
if (!txOwner)
return std::nullopt;
return txOwner(tx);
}
// Not interested in tracking consensus mode changes for now
void
onModeChange(ConsensusMode, ConsensusMode)
+8
View File
@@ -66,6 +66,14 @@ public:
// Write the batch
storeBatch(*backend, batch);
// Durability on demand: RocksDB drains its write queue and
// syncs its WAL; in-memory backends have nothing to wait for;
// NuDB commits on its own thread and cannot promise it.
backend->store(batch.front());
bool const durable = backend->syncDurable();
BEAST_EXPECT(durable == (type != "nudb"));
backend->sync();
{
// Read it back in
Batch copy;
+542
View File
@@ -0,0 +1,542 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2026 XRPL Labs
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <test/jtx.h>
#include <xrpld/app/misc/HashRouter.h>
#include <xrpld/core/JobQueue.h>
#include <xrpld/overlay/detail/OverlayImpl.h>
#include <xrpld/overlay/detail/XUSH.h>
#include <xrpl/beast/net/IPAddressConversion.h>
#include <xrpl/protocol/Feature.h>
#include <xrpl/protocol/STTx.h>
#include <xrpl/resource/ResourceManager.h>
#include <xrpl/resource/detail/Tuning.h>
#include <boost/asio/ip/udp.hpp>
#include <algorithm>
#include <array>
#include <chrono>
#include <cstring>
#include <optional>
#include <set>
#include <string>
#include <thread>
#include <vector>
namespace ripple {
namespace test {
// Tests the UDP Superhighway (XUSH) handling in OverlayImpl: inbound datagram
// processing, outbound publishing and the XUSHPEER advertisement timer.
class OverlayXUSH_test : public beast::unit_test::suite
{
using udp = boost::asio::ip::udp;
static std::unique_ptr<Config>
highway(std::unique_ptr<Config> cfg)
{
cfg->UDP_HIGHWAY = true;
return cfg;
}
static beast::IP::Endpoint
ep(std::string const& s)
{
return beast::IP::Endpoint::from_string(s);
}
static std::set<beast::IP::Endpoint>
asSet(std::vector<beast::IP::Endpoint> const& v)
{
return {v.begin(), v.end()};
}
static Slice
slice(Buffer const& b)
{
return Slice(b.data(), b.size());
}
static std::string
str(Buffer const& b)
{
return std::string(reinterpret_cast<char const*>(b.data()), b.size());
}
static Buffer
serialize(STTx const& stx)
{
Serializer s;
stx.add(s);
return Buffer(s.data(), s.size());
}
// A pseudo-transaction, which servers never submit
static STTx
pseudoTx()
{
return STTx(ttAMENDMENT, [](auto& obj) {
obj.setAccountID(sfAccount, AccountID());
obj.setFieldH256(sfAmendment, uint256(2));
obj.setFieldU32(sfLedgerSequence, 5);
});
}
// A local UDP socket that highway datagrams can be sent to
struct Sink
{
boost::asio::io_context io;
udp::socket socket;
explicit Sink(boost::asio::ip::address const& addr)
: socket(io, udp::endpoint(addr, 0))
{
socket.non_blocking(true);
}
beast::IP::Endpoint
endpoint() const
{
auto const local = socket.local_endpoint();
return beast::IP::Endpoint(local.address(), local.port());
}
void
sendTo(Buffer const& datagram, std::uint16_t port)
{
socket.send_to(
boost::asio::buffer(datagram.data(), datagram.size()),
udp::endpoint(socket.local_endpoint().address(), port));
}
std::optional<std::string>
receive(std::chrono::milliseconds timeout = std::chrono::seconds(5))
{
auto const deadline = std::chrono::steady_clock::now() + timeout;
std::array<char, 4096> buf;
while (std::chrono::steady_clock::now() < deadline)
{
udp::endpoint from;
boost::system::error_code ec;
auto const n =
socket.receive_from(boost::asio::buffer(buf), from, 0, ec);
if (!ec)
return std::string(buf.data(), n);
if (ec != boost::asio::error::would_block &&
ec != boost::asio::error::try_again)
return std::nullopt;
std::this_thread::sleep_for(std::chrono::milliseconds(5));
}
return std::nullopt;
}
// Receives datagrams until one of the given type arrives
std::optional<std::string>
receive(xush::MessageType type)
{
while (auto d = receive())
{
if (xush::classify(makeSlice(*d)) == type)
return d;
}
return std::nullopt;
}
};
struct Fixture
{
jtx::Env env;
OverlayImpl& overlay;
PeerFinder::Manager& peerFinder;
Resource::Manager& resources;
Fixture(beast::unit_test::suite& suite, bool enabled)
: env(suite, enabled ? jtx::envconfig(highway) : jtx::envconfig())
, overlay(dynamic_cast<OverlayImpl&>(env.app().overlay()))
, peerFinder(overlay.peerFinder())
, resources(env.app().getResourceManager())
{
}
// Delivers a datagram as if it had been received from `remote`
void
deliver(Buffer const& datagram, beast::IP::Endpoint const& remote)
{
overlay.processXUSH(
str(datagram), beast::IP::to_asio_endpoint(remote));
}
void
deliver(
std::vector<Buffer> const& datagrams,
beast::IP::Endpoint const& remote)
{
for (auto const& d : datagrams)
deliver(d, remote);
}
// Tracks the resource balance charged to `remote`
Resource::Consumer
probe(beast::IP::Endpoint const& remote)
{
return resources.newInboundEndpoint(remote);
}
std::set<beast::IP::Endpoint>
targets()
{
return asSet(peerFinder.highway_targets(1000));
}
// Whether `probe` was charged at least `fee` since `before`. Balances
// are normalized by the decay window, so a charge of `fee` raises the
// balance by at least fee / decayWindowSeconds.
static bool
charged(Resource::Consumer& probe, int before, int fee)
{
return probe.balance() - before >=
fee / Resource::decayWindowSeconds;
}
bool
isBad(uint256 const& txid)
{
env.app().getJobQueue().rendezvous();
return env.app().getHashRouter().getFlags(txid) & SF_BAD;
}
};
void
testDisabled()
{
testcase("disabled");
using namespace jtx;
Fixture f(*this, false);
auto const remote = ep("203.0.113.1:51235");
// Nothing is learned or sent while the highway is disabled
f.deliver(xush::encodePeers({ep("8.8.8.8:51235")}), remote);
BEAST_EXPECT(f.targets().empty());
Sink sink(boost::asio::ip::make_address("127.0.0.1"));
f.peerFinder.addFixedPeer("sink", {sink.endpoint()});
BEAST_EXPECT(f.targets().empty());
Account const alice("alice");
f.env.fund(XRP(10000), alice);
f.env.close();
Buffer const blob(64);
f.overlay.publishTxXUSH(slice(blob), uint256(1));
BEAST_EXPECT(!sink.receive(std::chrono::milliseconds(200)));
}
void
testPeers()
{
testcase("XUSHPEER");
Fixture f(*this, true);
auto const remote = ep("203.0.113.2:51235");
auto probe = f.probe(remote);
BEAST_EXPECT(f.targets().empty());
f.deliver(
xush::encodePeers({ep("8.8.8.8:51235"), ep("1.1.1.1:51235")}),
remote);
BEAST_EXPECT(
f.targets() == asSet({ep("8.8.8.8:51235"), ep("1.1.1.1:51235")}));
auto const balance = probe.balance();
// Unknown datagram types are ignored and cost almost nothing
f.deliver(Buffer("XUSHPING", 8), remote);
BEAST_EXPECT(!Fixture::charged(probe, balance, 100));
// A malformed advertisement is charged for
Buffer malformed(xush::tagSize + 2);
std::memcpy(malformed.data(), "XUSHPEER\x05\x00", malformed.size());
f.deliver(malformed, remote);
BEAST_EXPECT(Fixture::charged(probe, balance, 200));
BEAST_EXPECT(f.targets().size() == 2);
}
void
testTransaction()
{
testcase("XUSHTXNF transaction");
using namespace jtx;
Fixture f(*this, true);
auto& env = f.env;
auto const remote = ep("203.0.113.3:51235");
Account const alice("alice");
Account const bob("bob");
env.fund(XRP(10000), alice, bob);
env.close();
auto const jt = env.jt(pay(alice, bob, XRP(100)));
auto const txid = jt.stx->getTransactionID();
// Fragments arrive out of order and are reassembled
auto datagrams = xush::encodeTxn(slice(serialize(*jt.stx)), txid, 40);
BEAST_EXPECT(datagrams.size() > 2);
std::reverse(datagrams.begin(), datagrams.end());
f.deliver(datagrams, remote);
env.app().getJobQueue().rendezvous();
BEAST_EXPECT(!f.isBad(txid));
env.close();
BEAST_EXPECT(env.balance(bob) == XRP(10100));
// Delivering it again is harmless
f.deliver(datagrams, remote);
env.app().getJobQueue().rendezvous();
env.close();
BEAST_EXPECT(env.balance(bob) == XRP(10100));
}
void
testBadTransactions()
{
testcase("XUSHTXNF bad transactions");
using namespace jtx;
Fixture f(*this, true);
auto& env = f.env;
auto const remote = ep("203.0.113.4:51235");
auto probe = f.probe(remote);
Account const alice("alice");
Account const bob("bob");
env.fund(XRP(10000), alice, bob);
env.close();
auto const good = env.jt(pay(alice, bob, XRP(100)));
auto const blob = serialize(*good.stx);
auto const txid = good.stx->getTransactionID();
// The ID in the header must match the transaction
auto balance = probe.balance();
f.deliver(xush::encodeTxn(slice(blob), uint256(1)), remote);
BEAST_EXPECT(Fixture::charged(probe, balance, 400));
BEAST_EXPECT(!f.isBad(txid));
// Garbage that doesn't parse as a transaction
balance = probe.balance();
f.deliver(
xush::encodeTxn(makeSlice(std::string("garbage")), uint256(2)),
remote);
BEAST_EXPECT(Fixture::charged(probe, balance, 400));
// Fragments of one transaction that disagree with each other
balance = probe.balance();
f.deliver(xush::encodeTxn(slice(blob), uint256(3), 40).front(), remote);
f.deliver(xush::encodeTxn(slice(blob), uint256(3), 60).front(), remote);
BEAST_EXPECT(Fixture::charged(probe, balance, 400));
// A malformed fragment
balance = probe.balance();
f.deliver(Buffer("XUSHTXNF", 8), remote);
BEAST_EXPECT(Fixture::charged(probe, balance, 200));
// Pseudo-transactions are never accepted
balance = probe.balance();
auto const pseudo = pseudoTx();
f.deliver(
xush::encodeTxn(
slice(serialize(pseudo)), pseudo.getTransactionID()),
remote);
BEAST_EXPECT(Fixture::charged(probe, balance, 2000));
// An expired transaction is remembered as bad
auto const expired =
env.jt(pay(alice, bob, XRP(100)), last_ledger_seq(1));
auto const expiredId = expired.stx->getTransactionID();
balance = probe.balance();
f.deliver(
xush::encodeTxn(slice(serialize(*expired.stx)), expiredId), remote);
BEAST_EXPECT(f.isBad(expiredId));
BEAST_EXPECT(Fixture::charged(probe, balance, 150));
// A badly signed transaction is remembered as bad
auto corrupted = good.stx->getJson(JsonOptions::none);
corrupted[jss::TxnSignature] = std::string(128, '0');
STTx const badSig(parse(corrupted));
auto const badSigId = badSig.getTransactionID();
balance = probe.balance();
f.deliver(xush::encodeTxn(slice(serialize(badSig)), badSigId), remote);
BEAST_EXPECT(f.isBad(badSigId));
BEAST_EXPECT(Fixture::charged(probe, balance, 2000));
// A known bad transaction is charged for again
balance = probe.balance();
f.deliver(xush::encodeTxn(slice(serialize(badSig)), badSigId), remote);
BEAST_EXPECT(Fixture::charged(probe, balance, 150));
env.close();
BEAST_EXPECT(env.balance(bob) == XRP(10000));
}
void
testDisconnect()
{
testcase("resource limit");
Fixture f(*this, true);
auto const remote = ep("203.0.113.5:51235");
auto probe = f.probe(remote);
// A sender that keeps misbehaving is ignored
Buffer const blob(64);
for (int i = 0; i < 10000 && probe.balance() < Resource::dropThreshold;
++i)
f.deliver(xush::encodeTxn(slice(blob), uint256(1)), remote);
BEAST_EXPECT(probe.balance() >= Resource::dropThreshold);
f.deliver(xush::encodePeers({ep("8.8.8.8:51235")}), remote);
BEAST_EXPECT(f.targets().empty());
// Other senders are not affected
f.deliver(
xush::encodePeers({ep("8.8.8.8:51235")}), ep("203.0.113.6:51235"));
BEAST_EXPECT(f.targets() == asSet({ep("8.8.8.8:51235")}));
}
void
testPublish()
{
testcase("publish");
using namespace jtx;
Fixture f(*this, true);
auto& env = f.env;
Account const alice("alice");
Account const bob("bob");
env.fund(XRP(10000), alice, bob);
env.close();
Sink sink4(boost::asio::ip::make_address("127.0.0.1"));
f.peerFinder.addFixedPeer("sink4", {sink4.endpoint()});
std::optional<Sink> sink6;
try
{
sink6.emplace(boost::asio::ip::make_address("::1"));
f.peerFinder.addFixedPeer("sink6", {sink6->endpoint()});
}
catch (std::exception const&)
{
log << "IPv6 loopback unavailable, skipping" << std::endl;
}
// A transaction relayed by NetworkOPs is published on the highway
env(pay(alice, bob, XRP(100)));
auto const stx = env.tx();
auto const blob = serialize(*stx);
auto check = [&](Sink& sink) {
auto const d = sink.receive(xush::MessageType::txn);
if (!BEAST_EXPECT(d))
return;
auto const fragment = xush::decodeTxnFragment(makeSlice(*d));
if (!BEAST_EXPECT(fragment))
return;
BEAST_EXPECT(fragment->txid == stx->getTransactionID());
BEAST_EXPECT(fragment->count == 1);
BEAST_EXPECT(fragment->payload == slice(blob));
};
check(sink4);
if (sink6)
check(*sink6);
// Large transactions are fragmented
Buffer big(5000);
for (std::size_t i = 0; i < big.size(); ++i)
big.data()[i] = static_cast<std::uint8_t>(i);
f.overlay.publishTxXUSH(slice(big), uint256(7));
xush::Reassembler reassembler;
std::optional<Buffer> rebuilt;
while (!rebuilt)
{
auto const d = sink4.receive(xush::MessageType::txn);
if (!BEAST_EXPECT(d))
break;
auto const fragment = xush::decodeTxnFragment(makeSlice(*d));
if (!BEAST_EXPECT(fragment && fragment->txid == uint256(7)))
break;
auto result = reassembler.add(
sink4.endpoint(),
*fragment,
xush::Reassembler::clock_type::now());
if (result.status == xush::Reassembler::Status::complete)
rebuilt = std::move(result.txn);
}
BEAST_EXPECT(rebuilt && *rebuilt == big);
// Transactions too large for the highway are not sent
Buffer const huge(xush::maxTxnSize + 1);
f.overlay.publishTxXUSH(slice(huge), uint256(8));
f.overlay.publishTxXUSH(slice(big), uint256(9));
auto const d = sink4.receive(xush::MessageType::txn);
if (BEAST_EXPECT(d))
{
auto const fragment = xush::decodeTxnFragment(makeSlice(*d));
BEAST_EXPECT(fragment && fragment->txid == uint256(9));
}
}
void
testSendPeers()
{
testcase("advertise");
Fixture f(*this, true);
Sink sink(boost::asio::ip::make_address("127.0.0.1"));
f.peerFinder.addFixedPeer("sink", {sink.endpoint()});
// Learned endpoints are advertised to targets once per second, but
// fixed peers are never advertised.
f.peerFinder.add_highway_peers({ep("8.8.8.8:51235")});
auto const d = sink.receive(xush::MessageType::peers);
if (!BEAST_EXPECT(d))
return;
auto const endpoints = xush::decodePeers(makeSlice(*d));
BEAST_EXPECT(
endpoints && asSet(*endpoints) == asSet({ep("8.8.8.8:51235")}));
}
public:
void
run() override
{
testDisabled();
testPeers();
testTransaction();
testBadTransactions();
testDisconnect();
testPublish();
testSendPeers();
}
};
BEAST_DEFINE_TESTSUITE(OverlayXUSH, overlay, ripple);
} // namespace test
} // namespace ripple
+313
View File
@@ -0,0 +1,313 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2026 XRPL Labs
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <xrpld/overlay/Message.h>
#include <xrpld/overlay/detail/ProtocolMessage.h>
#include <xrpl/beast/unit_test.h>
#include <xrpl/protocol/messages.h>
#include <boost/asio/buffer.hpp>
#include <boost/beast/core/multi_buffer.hpp>
#include <boost/system/error_code.hpp>
#include <cstdint>
#include <memory>
#include <string>
#include <vector>
namespace ripple {
namespace test {
/** Receive-side framing tests for invokeProtocolMessage.
PeerImp::onReadMessage hands its entire read buffer to
invokeProtocolMessage, which must decode exactly one frame and report
exactly that frame's size as consumed. A single read routinely contains
several frames, so the payload parser must never look past the payload
size declared in the frame header.
*/
class ProtocolMessage_test : public beast::unit_test::suite
{
using Compressed = compression::Compressed;
/** Minimal stand-in for the PeerImp handler interface. */
struct Handler
{
bool compression = false;
std::vector<std::shared_ptr<::google::protobuf::Message>> received;
std::vector<std::uint16_t> types;
bool
compressionEnabled() const
{
return compression;
}
void
onMessageBegin(
std::uint16_t,
std::shared_ptr<::google::protobuf::Message> const&,
std::size_t,
std::size_t,
bool)
{
}
template <class T>
void
onMessage(std::shared_ptr<T> const& m)
{
received.push_back(m);
}
void
onMessageEnd(
std::uint16_t type,
std::shared_ptr<::google::protobuf::Message> const&)
{
types.push_back(type);
}
void
onMessageUnknown(std::uint16_t type)
{
types.push_back(type);
}
};
/** Loads wire bytes into a multi_buffer as one or two committed
segments, split at `split` (no split if split is 0 or past the end).
*/
static void
load(
boost::beast::multi_buffer& rb,
std::vector<std::uint8_t> const& wire,
std::size_t split = 0)
{
auto commit = [&rb](std::uint8_t const* p, std::size_t n) {
if (n != 0)
rb.commit(boost::asio::buffer_copy(
rb.prepare(n), boost::asio::buffer(p, n)));
};
if (split == 0 || split >= wire.size())
{
commit(wire.data(), wire.size());
return;
}
commit(wire.data(), split);
commit(wire.data() + split, wire.size() - split);
}
/** Mirrors the consume loop in PeerImp::onReadMessage. */
static boost::system::error_code
drain(boost::beast::multi_buffer& rb, Handler& handler)
{
std::size_t hint = 0;
while (rb.size() > 0)
{
auto const [consumed, ec] =
invokeProtocolMessage(rb.data(), handler, hint);
if (ec)
return ec;
if (consumed == 0)
break;
rb.consume(consumed);
}
return {};
}
static std::vector<std::uint8_t>
frame(
::google::protobuf::Message const& m,
protocol::MessageType type,
Compressed compressed = Compressed::Off)
{
Message msg(m, type);
return msg.getBuffer(compressed);
}
static protocol::TMPing
makePing(protocol::TMPing::pingType type, std::uint32_t seq)
{
protocol::TMPing ping;
ping.set_type(type);
ping.set_seq(seq);
return ping;
}
template <class T>
std::shared_ptr<T>
receivedAs(Handler const& h, std::size_t i)
{
if (!BEAST_EXPECT(i < h.received.size()))
return {};
auto p = std::dynamic_pointer_cast<T>(h.received[i]);
BEAST_EXPECT(p);
return p;
}
void
testCoalescedUncompressed()
{
testcase("Coalesced uncompressed frames");
auto const first = makePing(protocol::TMPing::ptPING, 7);
auto second = makePing(protocol::TMPing::ptPONG, 99);
second.set_nettime(5);
auto wire = frame(first, protocol::mtPING);
auto const firstSize = wire.size();
auto const tail = frame(second, protocol::mtPING);
wire.insert(wire.end(), tail.begin(), tail.end());
// Both frames delivered by a single read, with every possible
// segmentation of the read buffer (0 means one contiguous segment).
for (std::size_t split = 0; split < wire.size(); ++split)
{
boost::beast::multi_buffer rb;
load(rb, wire, split);
Handler h;
auto const ec = drain(rb, h);
BEAST_EXPECTS(
!ec, "split " + std::to_string(split) + ": " + ec.message());
BEAST_EXPECT(rb.size() == 0);
if (!BEAST_EXPECT(h.received.size() == 2))
continue;
auto const a = receivedAs<protocol::TMPing>(h, 0);
auto const b = receivedAs<protocol::TMPing>(h, 1);
if (!a || !b)
continue;
BEAST_EXPECT(a->type() == protocol::TMPing::ptPING);
BEAST_EXPECT(a->seq() == 7);
BEAST_EXPECT(!a->has_nettime());
BEAST_EXPECT(b->type() == protocol::TMPing::ptPONG);
BEAST_EXPECT(b->seq() == 99);
BEAST_EXPECT(b->nettime() == 5);
}
// The first frame on its own, followed by a partial second frame,
// is delivered alone and the remainder is left for the next read.
{
boost::beast::multi_buffer rb;
std::vector<std::uint8_t> partial(
wire.begin(), wire.begin() + firstSize + 3);
load(rb, partial);
Handler h;
BEAST_EXPECT(!drain(rb, h));
BEAST_EXPECT(h.received.size() == 1);
BEAST_EXPECT(rb.size() == 3);
}
}
void
testUncompressedThenCompressed()
{
testcase("Uncompressed frame followed by compressed frame");
auto const ping = makePing(protocol::TMPing::ptPING, 1234);
protocol::TMTransaction tx;
tx.set_rawtransaction(std::string(4096, 'x'));
tx.set_status(protocol::tsNEW);
auto wire = frame(ping, protocol::mtPING);
auto const txFrame = frame(tx, protocol::mtTRANSACTION, Compressed::On);
// The second frame must really be compressed for this test to mean
// anything: its header is what the first parse used to run into.
if (!BEAST_EXPECT(!txFrame.empty() && (txFrame[0] & 0x80)))
return;
wire.insert(wire.end(), txFrame.begin(), txFrame.end());
for (std::size_t split = 0; split < wire.size(); split += 7)
{
boost::beast::multi_buffer rb;
load(rb, wire, split);
Handler h;
h.compression = true;
auto const ec = drain(rb, h);
BEAST_EXPECTS(
!ec, "split " + std::to_string(split) + ": " + ec.message());
BEAST_EXPECT(rb.size() == 0);
if (!BEAST_EXPECT(h.received.size() == 2))
continue;
auto const p = receivedAs<protocol::TMPing>(h, 0);
auto const t = receivedAs<protocol::TMTransaction>(h, 1);
if (!p || !t)
continue;
BEAST_EXPECT(p->type() == protocol::TMPing::ptPING);
BEAST_EXPECT(p->seq() == 1234);
BEAST_EXPECT(t->rawtransaction() == tx.rawtransaction());
BEAST_EXPECT(t->status() == protocol::tsNEW);
}
}
void
testDeclaredSizeIsAuthoritative()
{
testcase("Declared payload size bounds the parse");
auto const ping = makePing(protocol::TMPing::ptPING, 7);
auto wire = frame(ping, protocol::mtPING);
// Shrink the declared payload size by one byte without removing the
// byte from the stream. The declared payload alone is a truncated
// TMPing, so the frame must be rejected rather than completed from
// bytes that lie beyond it.
std::uint32_t declared = (std::uint32_t(wire[0]) << 24) |
(std::uint32_t(wire[1]) << 16) | (std::uint32_t(wire[2]) << 8) |
std::uint32_t(wire[3]);
if (!BEAST_EXPECT(declared > 0))
return;
--declared;
wire[0] = static_cast<std::uint8_t>(declared >> 24);
wire[1] = static_cast<std::uint8_t>(declared >> 16);
wire[2] = static_cast<std::uint8_t>(declared >> 8);
wire[3] = static_cast<std::uint8_t>(declared);
boost::beast::multi_buffer rb;
load(rb, wire);
Handler h;
auto const ec = drain(rb, h);
BEAST_EXPECT(ec == boost::system::errc::bad_message);
BEAST_EXPECT(h.received.empty());
}
public:
void
run() override
{
testCoalescedUncompressed();
testUncompressedThenCompressed();
testDeclaredSizeIsAuthoritative();
}
};
BEAST_DEFINE_TESTSUITE(ProtocolMessage, overlay, ripple);
} // namespace test
} // namespace ripple
+466
View File
@@ -0,0 +1,466 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2026 XRPL Labs
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <xrpld/overlay/detail/XUSH.h>
#include <xrpl/beast/unit_test.h>
#include <algorithm>
#include <chrono>
#include <cstring>
#include <set>
#include <string>
#include <vector>
namespace ripple {
namespace test {
class XUSH_test : public beast::unit_test::suite
{
using Status = xush::Reassembler::Status;
static beast::IP::Endpoint
ep(std::string const& s)
{
return beast::IP::Endpoint::from_string(s);
}
static Slice
slice(Buffer const& b)
{
return Slice(b.data(), b.size());
}
static Buffer
makeTxn(std::size_t size)
{
Buffer b(size);
for (std::size_t i = 0; i < size; ++i)
b.data()[i] = static_cast<std::uint8_t>(i * 31 + 7);
return b;
}
static void
putU32(std::uint8_t* p, std::uint32_t v)
{
p[0] = static_cast<std::uint8_t>(v >> 24);
p[1] = static_cast<std::uint8_t>(v >> 16);
p[2] = static_cast<std::uint8_t>(v >> 8);
p[3] = static_cast<std::uint8_t>(v);
}
// A XUSHTXNF datagram with an arbitrary (possibly invalid) header
static Buffer
rawFragment(
uint256 const& txid,
std::uint32_t totalSize,
std::uint32_t count,
std::uint32_t index,
std::size_t payloadSize)
{
Buffer b(xush::txnHeaderSize + payloadSize);
std::memset(b.data(), 0xAB, b.size());
std::memcpy(b.data(), "XUSHTXNF", xush::tagSize);
std::memcpy(b.data() + xush::tagSize, txid.data(), txid.size());
putU32(b.data() + 40, totalSize);
putU32(b.data() + 44, count);
putU32(b.data() + 48, index);
return b;
}
// A XUSHPEER datagram with arbitrary (possibly invalid) counts
static Buffer
rawPeers(std::uint8_t n4, std::uint8_t n6, std::size_t bodySize)
{
Buffer b(xush::tagSize + 2 + bodySize);
std::memset(b.data(), 0, b.size());
std::memcpy(b.data(), "XUSHPEER", xush::tagSize);
b.data()[8] = n4;
b.data()[9] = n6;
return b;
}
void
testClassify()
{
testcase("classify");
using namespace xush;
auto type = [](std::string const& s) { return classify(makeSlice(s)); };
BEAST_EXPECT(type("XUSHPEER") == MessageType::peers);
BEAST_EXPECT(type("XUSHPEER and more") == MessageType::peers);
BEAST_EXPECT(type("XUSHTXNF") == MessageType::txn);
BEAST_EXPECT(type("XUSHPING") == MessageType::unknown);
BEAST_EXPECT(type("XUSHPEE") == MessageType::unknown);
BEAST_EXPECT(type("") == MessageType::unknown);
BEAST_EXPECT(type("{\"command\":\"ping\"}") == MessageType::unknown);
}
void
testPeers()
{
testcase("XUSHPEER");
using namespace xush;
std::vector<beast::IP::Endpoint> const endpoints{
ep("1.2.3.4:51235"), ep("[2001:db8::1]:21337"), ep("5.6.7.8:1")};
auto const datagram = encodePeers(endpoints);
BEAST_EXPECT(datagram.size() == tagSize + 2 + 2 * 8 + 20);
BEAST_EXPECT(classify(slice(datagram)) == MessageType::peers);
if (auto const decoded = decodePeers(slice(datagram));
BEAST_EXPECT(decoded))
{
BEAST_EXPECT(
std::set<beast::IP::Endpoint>(
decoded->begin(), decoded->end()) ==
std::set<beast::IP::Endpoint>(
endpoints.begin(), endpoints.end()));
}
// An empty advertisement is valid
if (auto const decoded = decodePeers(slice(encodePeers({})));
BEAST_EXPECT(decoded))
BEAST_EXPECT(decoded->empty());
// At most maxAdvertisedPeers are advertised, and the result fits in
// a single datagram even if they are all IPv6
{
std::vector<beast::IP::Endpoint> many;
for (std::uint16_t i = 1; i <= maxAdvertisedPeers + 10; ++i)
many.emplace_back(
boost::asio::ip::make_address("2001:db8::1"), i);
auto const big = encodePeers(many);
BEAST_EXPECT(big.size() <= maxDatagramSize);
auto const decoded = decodePeers(slice(big));
BEAST_EXPECT(decoded && decoded->size() == maxAdvertisedPeers);
}
// Truncated or padded
BEAST_EXPECT(!decodePeers(Slice(datagram.data(), datagram.size() - 1)));
{
Buffer padded(datagram.size() + 1);
std::memcpy(padded.data(), datagram.data(), datagram.size());
BEAST_EXPECT(!decodePeers(slice(padded)));
}
BEAST_EXPECT(!decodePeers(Slice(datagram.data(), tagSize + 1)));
// Too many endpoints, even with a consistent size
BEAST_EXPECT(!decodePeers(slice(rawPeers(
maxAdvertisedPeers + 1, 0, (maxAdvertisedPeers + 1) * 8))));
BEAST_EXPECT(decodePeers(
slice(rawPeers(maxAdvertisedPeers, 0, maxAdvertisedPeers * 8))));
// A port that doesn't fit in 16 bits
{
auto bad = rawPeers(1, 0, 8);
putU32(bad.data() + tagSize + 2 + 4, 0x10000);
BEAST_EXPECT(!decodePeers(slice(bad)));
putU32(bad.data() + tagSize + 2 + 4, 0xFFFF);
BEAST_EXPECT(decodePeers(slice(bad)));
}
// Wrong tag
BEAST_EXPECT(!decodePeers(slice(rawFragment(uint256{1}, 1, 1, 0, 1))));
}
void
testTxnEncoding()
{
testcase("XUSHTXNF encoding");
using namespace xush;
uint256 const txid{42};
// A typical transaction fits in a single datagram
{
auto const txn = makeTxn(200);
auto const datagrams = encodeTxn(slice(txn), txid);
BEAST_EXPECT(datagrams.size() == 1);
BEAST_EXPECT(datagrams[0].size() == txnHeaderSize + 200);
auto const fragment = decodeTxnFragment(slice(datagrams[0]));
if (BEAST_EXPECT(fragment))
{
BEAST_EXPECT(fragment->txid == txid);
BEAST_EXPECT(fragment->totalSize == 200);
BEAST_EXPECT(fragment->count == 1);
BEAST_EXPECT(fragment->index == 0);
BEAST_EXPECT(fragment->payload == slice(txn));
}
}
// Larger transactions are split, and no datagram is too large
for (auto const size :
{maxFragmentPayload,
maxFragmentPayload + 1,
3 * maxFragmentPayload,
3 * maxFragmentPayload + 17})
{
auto const txn = makeTxn(size);
auto const datagrams = encodeTxn(slice(txn), txid);
auto const count =
(size + maxFragmentPayload - 1) / maxFragmentPayload;
BEAST_EXPECT(datagrams.size() == count);
std::size_t total = 0;
for (std::size_t i = 0; i < datagrams.size(); ++i)
{
BEAST_EXPECT(datagrams[i].size() <= maxDatagramSize);
auto const fragment = decodeTxnFragment(slice(datagrams[i]));
if (!BEAST_EXPECT(fragment))
continue;
BEAST_EXPECT(fragment->index == i);
BEAST_EXPECT(fragment->count == count);
BEAST_EXPECT(fragment->totalSize == size);
BEAST_EXPECT(
fragment->payload ==
Slice(
txn.data() + i * maxFragmentPayload,
fragment->payload.size()));
total += fragment->payload.size();
}
BEAST_EXPECT(total == size);
}
// Limits
BEAST_EXPECT(encodeTxn(Slice{}, txid).empty());
BEAST_EXPECT(encodeTxn(slice(Buffer(maxTxnSize + 1)), txid).empty());
{
auto const datagrams = encodeTxn(slice(Buffer(maxTxnSize)), txid);
BEAST_EXPECT(!datagrams.empty());
BEAST_EXPECT(datagrams.size() <= maxFragments);
}
}
void
testTxnDecoding()
{
testcase("XUSHTXNF decoding");
using namespace xush;
uint256 const id{7};
auto decode = [](Buffer const& b) {
return decodeTxnFragment(slice(b)).has_value();
};
BEAST_EXPECT(decode(rawFragment(id, 10, 2, 1, 5)));
BEAST_EXPECT(decode(rawFragment(id, 10, 1, 0, 10)));
BEAST_EXPECT(decode(rawFragment(id, maxTxnSize, maxFragments, 0, 1)));
// No payload
BEAST_EXPECT(!decode(rawFragment(id, 10, 1, 0, 0)));
// Bad total size
BEAST_EXPECT(!decode(rawFragment(id, 0, 1, 0, 1)));
BEAST_EXPECT(!decode(rawFragment(id, maxTxnSize + 1, 2, 0, 1)));
// Bad fragment count
BEAST_EXPECT(!decode(rawFragment(id, 10, 0, 0, 1)));
BEAST_EXPECT(!decode(rawFragment(id, 10, 11, 0, 1)));
BEAST_EXPECT(
!decode(rawFragment(id, maxTxnSize, maxFragments + 1, 0, 1)));
// Bad index
BEAST_EXPECT(!decode(rawFragment(id, 10, 2, 2, 1)));
// Payload larger than the transaction
BEAST_EXPECT(!decode(rawFragment(id, 10, 2, 0, 11)));
// Wrong tag
BEAST_EXPECT(!decodeTxnFragment(slice(encodePeers({}))));
}
void
testReassembly()
{
testcase("reassembly");
using namespace xush;
using namespace std::chrono_literals;
auto const now = Reassembler::clock_type::now();
auto const alice = ep("1.2.3.4:5000");
auto const bob = ep("5.6.7.8:5000");
auto const carol = ep("9.9.9.9:5000");
uint256 const id1{1};
uint256 const id2{2};
auto const txn1 = makeTxn(2 * maxFragmentPayload + 100);
auto const txn2 = makeTxn(2 * maxFragmentPayload + 200);
auto const d1 = encodeTxn(slice(txn1), id1);
auto const d2 = encodeTxn(slice(txn2), id2);
BEAST_EXPECT(d1.size() == 3 && d2.size() == 3);
auto f1 = [&](std::size_t i) {
return *decodeTxnFragment(slice(d1[i]));
};
auto f2 = [&](std::size_t i) {
return *decodeTxnFragment(slice(d2[i]));
};
// Out of order, with a repeated fragment
{
Reassembler r;
BEAST_EXPECT(r.add(alice, f1(2), now).status == Status::pending);
BEAST_EXPECT(r.add(alice, f1(0), now).status == Status::pending);
BEAST_EXPECT(r.add(alice, f1(0), now).status == Status::pending);
BEAST_EXPECT(r.pending() == 1);
auto const result = r.add(alice, f1(1), now);
BEAST_EXPECT(result.status == Status::complete);
BEAST_EXPECT(result.txn == txn1);
BEAST_EXPECT(r.pending() == 0);
BEAST_EXPECT(r.bytes() == 0);
}
// A transaction in a single datagram needs no state
{
Reassembler r;
auto const small = makeTxn(100);
auto const d = encodeTxn(slice(small), id2);
auto const result =
r.add(alice, *decodeTxnFragment(slice(d[0])), now);
BEAST_EXPECT(result.status == Status::complete);
BEAST_EXPECT(result.txn == small);
BEAST_EXPECT(r.pending() == 0);
// ... but must be consistent
auto bad = *decodeTxnFragment(slice(d[0]));
bad.totalSize += 1;
BEAST_EXPECT(r.add(alice, bad, now).status == Status::invalid);
}
// Senders are kept apart
{
Reassembler r;
BEAST_EXPECT(r.add(alice, f1(0), now).status == Status::pending);
BEAST_EXPECT(r.add(alice, f1(1), now).status == Status::pending);
BEAST_EXPECT(r.add(bob, f1(2), now).status == Status::pending);
BEAST_EXPECT(r.pending() == 2);
BEAST_EXPECT(r.add(alice, f1(2), now).status == Status::complete);
BEAST_EXPECT(r.pending() == 1);
}
// A header that disagrees with earlier fragments
{
Reassembler r;
BEAST_EXPECT(r.add(alice, f1(0), now).status == Status::pending);
auto bad = f1(1);
bad.count += 1;
BEAST_EXPECT(r.add(alice, bad, now).status == Status::invalid);
BEAST_EXPECT(r.pending() == 0);
BEAST_EXPECT(r.bytes() == 0);
}
// More bytes than the advertised total size
{
Reassembler r;
auto first = f1(0);
first.count = 2;
first.totalSize = maxFragmentPayload + 10;
auto second = f1(1);
second.count = 2;
second.totalSize = first.totalSize;
BEAST_EXPECT(r.add(alice, first, now).status == Status::pending);
BEAST_EXPECT(r.add(alice, second, now).status == Status::invalid);
BEAST_EXPECT(r.pending() == 0);
}
// Every fragment present but fewer bytes than the total size
{
Reassembler r;
auto first = f1(0);
first.count = 2;
first.totalSize = 2 * maxFragmentPayload + 5;
auto second = f1(1);
second.count = 2;
second.totalSize = first.totalSize;
BEAST_EXPECT(r.add(alice, first, now).status == Status::pending);
BEAST_EXPECT(r.add(alice, second, now).status == Status::invalid);
BEAST_EXPECT(r.pending() == 0);
}
// Partial transactions time out
{
Reassembler r(
Reassembler::defaultMaxPending,
Reassembler::defaultMaxBytes,
30s);
BEAST_EXPECT(r.add(alice, f1(0), now).status == Status::pending);
BEAST_EXPECT(r.add(alice, f1(1), now).status == Status::pending);
// Not yet
BEAST_EXPECT(
r.add(bob, f2(0), now + 30s).status == Status::pending);
BEAST_EXPECT(r.pending() == 2);
// Now: the first transaction has to start over
BEAST_EXPECT(
r.add(bob, f2(1), now + 31s).status == Status::pending);
BEAST_EXPECT(r.pending() == 1);
BEAST_EXPECT(
r.add(alice, f1(2), now + 31s).status == Status::pending);
BEAST_EXPECT(r.pending() == 2);
}
// The number of partial transactions is limited
{
Reassembler r(2, Reassembler::defaultMaxBytes, 30s);
BEAST_EXPECT(r.add(alice, f1(0), now).status == Status::pending);
BEAST_EXPECT(r.add(bob, f1(0), now).status == Status::pending);
BEAST_EXPECT(r.add(carol, f1(0), now).status == Status::overloaded);
BEAST_EXPECT(r.pending() == 2);
// Fragments of transactions already being rebuilt are accepted
BEAST_EXPECT(r.add(alice, f1(1), now).status == Status::pending);
BEAST_EXPECT(r.add(alice, f1(2), now).status == Status::complete);
BEAST_EXPECT(r.add(carol, f1(0), now).status == Status::pending);
}
// The number of bytes buffered is limited
{
Reassembler r(
Reassembler::defaultMaxPending,
maxFragmentPayload + Reassembler::fragmentOverhead,
30s);
BEAST_EXPECT(r.add(alice, f1(0), now).status == Status::pending);
BEAST_EXPECT(r.add(alice, f1(1), now).status == Status::overloaded);
BEAST_EXPECT(r.add(bob, f1(0), now).status == Status::overloaded);
BEAST_EXPECT(r.pending() == 1);
BEAST_EXPECT(
r.bytes() ==
maxFragmentPayload + Reassembler::fragmentOverhead);
}
}
public:
void
run() override
{
testClassify();
testPeers();
testTxnEncoding();
testTxnDecoding();
testReassembly();
}
};
BEAST_DEFINE_TESTSUITE(XUSH, overlay, ripple);
} // namespace test
} // namespace ripple
+276
View File
@@ -0,0 +1,276 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2026 XRPL Labs
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <test/unit_test/SuiteJournal.h>
#include <xrpld/peerfinder/PeerfinderManager.h>
#include <xrpld/peerfinder/detail/Tuning.h>
#include <xrpld/peerfinder/make_Manager.h>
#include <xrpl/basics/BasicConfig.h>
#include <xrpl/basics/chrono.h>
#include <xrpl/beast/insight/NullCollector.h>
#include <xrpl/beast/net/IPAddressConversion.h>
#include <xrpl/beast/unit_test.h>
#include <boost/asio/io_context.hpp>
#include <memory>
#include <set>
#include <string>
#include <vector>
namespace ripple {
namespace PeerFinder {
// Tests the UDP Superhighway (XUSH) endpoint set kept by PeerFinder::Manager
class HighwayPeers_test : public beast::unit_test::suite
{
test::SuiteJournal journal_{"HighwayPeers_test", *this};
struct Fixture
{
boost::asio::io_context io;
TestStopwatch clock;
BasicConfig basicConfig;
std::unique_ptr<Manager> manager;
Fixture(beast::Journal journal, bool enabled, bool peerPrivate)
: manager(make_Manager(
io,
clock,
journal,
basicConfig,
beast::insight::NullCollector::New(),
false))
{
Config config;
config.udpHighway = enabled;
config.peerPrivate = peerPrivate;
manager->setConfig(config);
}
~Fixture()
{
manager->stop();
}
};
static beast::IP::Endpoint
ep(std::string const& s)
{
return beast::IP::Endpoint::from_string(s);
}
// A distinct public IPv4 endpoint
static beast::IP::Endpoint
publicEp(std::uint32_t i)
{
return beast::IP::Endpoint(
boost::asio::ip::address_v4(0x2C000000 + i), 51235);
}
static std::set<beast::IP::Endpoint>
asSet(std::vector<beast::IP::Endpoint> const& v)
{
return {v.begin(), v.end()};
}
void
testDisabled()
{
testcase("disabled");
Fixture f(journal_, false, false);
f.manager->addFixedPeer("fixed", {ep("10.0.0.1:51235")});
f.manager->add_highway_peers({ep("8.8.8.8:51235")});
f.manager->onRedirects(
beast::IP::to_asio_endpoint(ep("8.8.4.4:51235")),
{beast::IP::to_asio_endpoint(ep("1.1.1.1:51235"))});
BEAST_EXPECT(f.manager->highway_targets(100).empty());
BEAST_EXPECT(f.manager->highway_adverts(100).empty());
}
void
testLearning()
{
testcase("learning");
Fixture f(journal_, true, false);
// Only valid public endpoints are learned from other servers
f.manager->add_highway_peers(
{ep("8.8.8.8:51235"),
ep("10.0.0.1:51235"),
ep("127.0.0.1:51235"),
ep("9.9.9.9:0"),
ep("0.0.0.0:51235")});
BEAST_EXPECT(
asSet(f.manager->highway_targets(100)) ==
std::set<beast::IP::Endpoint>{ep("8.8.8.8:51235")});
// Fixed peers are trusted, even with a private address, but they
// are never advertised
f.manager->addFixedPeer("fixed", {ep("10.0.0.2:51235")});
BEAST_EXPECT(
asSet(f.manager->highway_targets(100)) ==
(std::set<beast::IP::Endpoint>{
ep("8.8.8.8:51235"), ep("10.0.0.2:51235")}));
BEAST_EXPECT(
asSet(f.manager->highway_adverts(100)) ==
std::set<beast::IP::Endpoint>{ep("8.8.8.8:51235")});
// A learned endpoint that is later configured as fixed stops being
// advertised
f.manager->addFixedPeer("fixed2", {ep("8.8.8.8:51235")});
BEAST_EXPECT(f.manager->highway_adverts(100).empty());
BEAST_EXPECT(f.manager->highway_targets(100).size() == 2);
// Redirects are learned too
f.manager->onRedirects(
beast::IP::to_asio_endpoint(ep("8.8.4.4:51235")),
{beast::IP::to_asio_endpoint(ep("1.1.1.1:51235"))});
BEAST_EXPECT(f.manager->highway_targets(100).size() == 3);
}
void
testSampling()
{
testcase("sampling");
Fixture f(journal_, true, false);
std::vector<beast::IP::Endpoint> endpoints;
for (std::uint32_t i = 0; i < 100; ++i)
endpoints.push_back(publicEp(i));
f.manager->add_highway_peers(endpoints);
for (int i = 0; i < 10; ++i)
{
auto const targets = f.manager->highway_targets(16);
BEAST_EXPECT(targets.size() == 16);
BEAST_EXPECT(asSet(targets).size() == 16);
}
BEAST_EXPECT(
asSet(f.manager->highway_targets(1000)) == asSet(endpoints));
BEAST_EXPECT(f.manager->highway_targets(0).empty());
}
void
testPrivate()
{
testcase("private");
Fixture f(journal_, true, true);
f.manager->addFixedPeer("fixed", {ep("10.0.0.2:51235")});
f.manager->add_highway_peers({ep("8.8.8.8:51235")});
// A private server only talks to its fixed peers, and advertises
// nothing
BEAST_EXPECT(
asSet(f.manager->highway_targets(100)) ==
std::set<beast::IP::Endpoint>{ep("10.0.0.2:51235")});
BEAST_EXPECT(f.manager->highway_adverts(100).empty());
}
void
testExpiry()
{
testcase("expiry");
using namespace std::chrono_literals;
Fixture f(journal_, true, false);
auto const ttl = Tuning::highwayPeerSecondsToLive;
auto const advertised = ep("8.8.8.8:51235");
auto const redirected = ep("1.1.1.1:51235");
auto const fixed = ep("10.0.0.2:51235");
f.manager->addFixedPeer("fixed", {fixed});
f.manager->add_highway_peers({advertised});
f.manager->add_highway_peers({redirected});
// Advertisements don't extend the life of an entry, but
// authenticated sources (here, a redirect) do
f.clock.advance(ttl / 2);
f.manager->add_highway_peers({advertised, redirected});
f.manager->onRedirects(
beast::IP::to_asio_endpoint(ep("8.8.4.4:51235")),
{beast::IP::to_asio_endpoint(redirected)});
f.clock.advance(ttl / 2);
f.manager->once_per_second();
BEAST_EXPECT(f.manager->highway_targets(100).size() == 3);
f.clock.advance(1s);
f.manager->once_per_second();
BEAST_EXPECT(
asSet(f.manager->highway_targets(100)) ==
(std::set<beast::IP::Endpoint>{redirected, fixed}));
f.clock.advance(ttl / 2);
f.manager->once_per_second();
BEAST_EXPECT(
asSet(f.manager->highway_targets(100)) ==
std::set<beast::IP::Endpoint>{fixed});
// An expired entry can be learned again
f.manager->add_highway_peers({advertised});
BEAST_EXPECT(f.manager->highway_targets(100).size() == 2);
}
void
testLimit()
{
testcase("limit");
Fixture f(journal_, true, false);
auto const max = Tuning::highwayPeersMax;
std::vector<beast::IP::Endpoint> endpoints;
for (std::uint32_t i = 0; i < max + 100; ++i)
endpoints.push_back(publicEp(i));
f.manager->add_highway_peers(endpoints);
BEAST_EXPECT(f.manager->highway_targets(max + 100).size() == max);
// Fixed peers don't count towards the limit
f.manager->addFixedPeer("fixed", {ep("10.0.0.2:51235")});
BEAST_EXPECT(f.manager->highway_targets(max + 100).size() == max + 1);
// Disabling the highway forgets everything
Config config;
config.udpHighway = false;
f.manager->setConfig(config);
BEAST_EXPECT(f.manager->highway_targets(max + 100).empty());
}
public:
void
run() override
{
testDisabled();
testLearning();
testSampling();
testPrivate();
testExpiry();
testLimit();
}
};
BEAST_DEFINE_TESTSUITE(HighwayPeers, peerfinder, ripple);
} // namespace PeerFinder
} // namespace ripple
+640 -1
View File
@@ -18,16 +18,22 @@
//==============================================================================
#include <test/jtx.h>
#include <xrpld/app/ledger/Ledger.h>
#include <xrpld/app/ledger/LedgerMaster.h>
#include <xrpld/core/ConfigSections.h>
#include <xrpld/nodestore/detail/DatabasePinnedImp.h>
#include <xrpl/beast/unit_test.h>
#include <xrpl/beast/utility/temp_dir.h>
#include <xrpl/protocol/jss.h>
#include <boost/filesystem.hpp>
#include <chrono>
#include <cstddef>
#include <cstdlib>
#include <cstring>
#include <fstream>
#include <grpc/impl/codegen/compression_types.h>
#include <iterator>
#include <optional>
#include <thread>
namespace ripple {
@@ -102,14 +108,28 @@ struct CatalogueEnvOpts
};
// envconfig with pinned_type set so catalogue_load passes the config guard.
// Uses rwdb for in-process speed (allowed in standalone).
//
// envconfig() selects [node_db] type=rwdb, which is the tree-only null node
// store: SHAMapStore never builds DatabasePinned on it, so catalogue_load
// would refuse to run. Use the in-memory "memory" store for the hot side
// instead. It is shared by path within the process, so each config gets its
// own path (SHAMapStore creates the directory; nothing is written to it).
// The pinned side uses rwdb, an in-memory persistent backend.
inline std::unique_ptr<Config>
catalogueEnvconfig(CatalogueEnvOpts const& opts = {})
{
auto cfg = test::jtx::envconfig();
auto& nodeDb = cfg->section(ConfigSection::nodeDatabase());
if (opts.withPinnedType)
{
nodeDb.set("type", "memory");
nodeDb.set(
"path",
(boost::filesystem::temp_directory_path() /
boost::filesystem::unique_path("catalogue-test-%%%%-%%%%-%%%%"))
.string());
nodeDb.set("pinned_type", "rwdb");
}
nodeDb.set("online_delete", opts.onlineDelete);
return cfg;
}
@@ -1067,6 +1087,621 @@ class Catalogue_test : public beast::unit_test::suite
}
public:
// ---- Editing uncompressed catalogues -----------------------------------
//
// Ledger record: seq, hash, txHash, accountHash, parentHash, drops,
// closeFlags, closeTimeResolution, closeTime, parentCloseTime; then the
// state map and the tx map, each a run of leaf records ending in
// tnTERMINAL. A leaf record is type (1), key (32), and, unless it is a
// tnREMOVE, size (4) and data.
static constexpr std::size_t catLedgerHeaderSize =
4 + 32 * 4 + 8 + sizeof(int) + 4 + 8 + 8;
struct CatLedgerRecord
{
std::size_t header; // first byte of the ledger header
std::size_t state; // first byte of the state map
std::size_t tx; // first byte of the tx map
std::size_t end; // one past the tx map's tnTERMINAL
};
// Position just past the map that starts at `pos`.
static std::size_t
skipCatalogueMap(std::vector<std::uint8_t> const& bytes, std::size_t pos)
{
while (pos < bytes.size())
{
auto const type = bytes[pos++];
if (type == 255) // tnTERMINAL
return pos;
pos += 32;
if (type == 254) // tnREMOVE
continue;
if (pos + 4 > bytes.size())
break;
std::uint32_t size = 0;
std::memcpy(&size, &bytes[pos], 4);
pos += 4 + size;
}
return bytes.size();
}
// Position of the size field of the first leaf (not a removal) in the
// map that starts at `pos`, if the map has one.
static std::optional<std::size_t>
firstLeafSizeField(std::vector<std::uint8_t> const& bytes, std::size_t pos)
{
while (pos < bytes.size())
{
auto const type = bytes[pos];
if (type == 255)
return std::nullopt;
if (type != 254)
return pos + 1 + 32;
pos += 1 + 32;
}
return std::nullopt;
}
static std::vector<CatLedgerRecord>
indexCatalogue(std::vector<std::uint8_t> const& bytes)
{
std::vector<CatLedgerRecord> records;
std::size_t pos = sizeof(TestCATLHeader);
while (pos + catLedgerHeaderSize <= bytes.size())
{
CatLedgerRecord r;
r.header = pos;
r.state = pos + catLedgerHeaderSize;
r.tx = skipCatalogueMap(bytes, r.state);
r.end = skipCatalogueMap(bytes, r.tx);
records.push_back(r);
pos = r.end;
}
return records;
}
static std::vector<std::uint8_t>
readCatalogue(std::string const& path)
{
std::ifstream in(path, std::ios::binary);
return {
std::istreambuf_iterator<char>(in),
std::istreambuf_iterator<char>()};
}
static void
writeCatalogue(std::string const& path, std::vector<std::uint8_t> const& b)
{
std::ofstream out(path, std::ios::binary | std::ios::trunc);
out.write(reinterpret_cast<char const*>(b.data()), b.size());
}
static Json::Value
loadCatalogue(test::jtx::Env& env, std::string const& path, bool ignoreHash)
{
Json::Value params{Json::objectValue};
params[jss::input_file] = path;
if (ignoreHash)
params[jss::ignore_hash] = true;
return env.client().invoke("catalogue_load", params)[jss::result];
}
static bool
errorContains(Json::Value const& result, std::string const& what)
{
return result[jss::status] == "error" &&
result[jss::error_message].asString().find(what) !=
std::string::npos;
}
void
testCatalogueLoadFailures(FeatureBitset features)
{
testcase("catalogue_load: failures part way through a file");
using namespace test::jtx;
using beast::severities::kDisabled;
boost::filesystem::path tempDir =
boost::filesystem::temp_directory_path() /
boost::filesystem::unique_path();
boost::filesystem::create_directories(tempDir);
auto const goodPath = (tempDir / "good.catl").string();
// An uncompressed catalogue of ledgers 3-6, so records can be edited.
{
Env env{*this, catalogueEnvconfig(), features, nullptr, kDisabled};
prepareLedgerData(env, 3);
Json::Value params{Json::objectValue};
params[jss::min_ledger] = 3;
params[jss::max_ledger] = 6;
params[jss::output_file] = goodPath;
params[jss::compression_level] = 0;
auto const result =
env.client().invoke("catalogue_create", params)[jss::result];
BEAST_EXPECT(result[jss::status] == jss::success);
}
auto const good = readCatalogue(goodPath);
auto const records = indexCatalogue(good);
if (!BEAST_EXPECT(
records.size() == 4 && records.back().end == good.size()))
return;
// Write `bytes` to a new file, load it into a fresh server, and hand
// the result and the resulting pinned set to `check`.
int fileNo = 0;
auto loadEdited = [&](std::vector<std::uint8_t> const& bytes,
bool ignoreHash,
auto&& check) {
auto const path =
(tempDir / ("edited" + std::to_string(++fileNo) + ".catl"))
.string();
writeCatalogue(path, bytes);
Env env{*this, catalogueEnvconfig(), features, nullptr, kDisabled};
auto const result = loadCatalogue(env, path, ignoreHash);
check(
result, env.app().getLedgerMaster().getPinnedLedgersRangeSet());
};
auto const hashField = offsetof(TestCATLHeader, hash);
auto const maxLedgerField = offsetof(TestCATLHeader, max_ledger);
// Offsets within a ledger header record.
constexpr std::size_t accountHashAt = 4 + 32 * 2;
constexpr std::size_t dropsAt = 4 + 32 * 4;
// The stored checksum is wrong but every ledger is intact: all four
// load, then the whole-file check fails and the run is rolled back.
{
auto bytes = good;
bytes[hashField] ^= 0xff;
loadEdited(
bytes, false, [&](auto const& result, auto const& pinned) {
BEAST_EXPECT(errorContains(
result, "catalogue hash verification failed"));
BEAST_EXPECT(pinned.empty());
});
}
// A ledger part way through is corrupt. The load stops there, the
// rest of the file is still hashed, the checksum fails, and the
// ledgers loaded before it are rolled back too.
{
auto bytes = good;
bytes[records[2].header + dropsAt] ^= 0x01;
loadEdited(
bytes, false, [&](auto const& result, auto const& pinned) {
BEAST_EXPECT(errorContains(
result, "catalogue hash verification failed"));
BEAST_EXPECT(pinned.empty());
});
}
// A checksum failure only rolls back what this run added: ledgers
// that were already pinned stay pinned.
{
auto bytes = good;
bytes[hashField] ^= 0xff;
auto const badPath = (tempDir / "badsum.catl").string();
writeCatalogue(badPath, bytes);
Env env{*this, catalogueEnvconfig(), features, nullptr, kDisabled};
BEAST_EXPECT(
loadCatalogue(env, goodPath, false)[jss::status] ==
jss::success);
BEAST_EXPECT(errorContains(
loadCatalogue(env, badPath, false),
"catalogue hash verification failed"));
auto& lm = env.app().getLedgerMaster();
BEAST_EXPECT(to_string(lm.getPinnedLedgersRangeSet()) == "3-6");
// The pinned store: an empty batch is a no-op, an in-memory
// backend is durable on demand, and a fetch made for
// online_delete's copy pass (duplicate == true) finds nodes
// that exist only in the pinned store.
auto* const pinnedDb = dynamic_cast<NodeStore::DatabasePinnedImp*>(
&env.app().getNodeStore());
if (BEAST_EXPECT(pinnedDb))
{
pinnedDb->storePinnedBatch({});
BEAST_EXPECT(pinnedDb->syncPinned());
auto const stateRoot =
uint256::fromVoid(&good[records[0].header + accountHashAt]);
BEAST_EXPECT(env.app().getNodeStore().fetchNodeObject(
stateRoot, 0, NodeStore::FetchType::synchronous, true));
}
}
// Out of sequence: ledger 4's record claims to be ledger 9.
{
auto bytes = good;
std::uint32_t const nine = 9;
std::memcpy(&bytes[records[1].header], &nine, 4);
loadEdited(
bytes, true, [&](auto const& result, auto const& pinned) {
BEAST_EXPECT(errorContains(result, "out of sequence"));
BEAST_EXPECT(boost::icl::contains(pinned, 3u));
BEAST_EXPECT(!boost::icl::contains(pinned, 4u));
});
}
// The header promises a ledger 7 the file does not contain. Ledgers
// 3-6 load (hash checks are off) before the file runs out.
{
auto bytes = good;
std::uint32_t const seven = 7;
std::memcpy(&bytes[maxLedgerField], &seven, 4);
loadEdited(
bytes, true, [&](auto const& result, auto const& pinned) {
BEAST_EXPECT(errorContains(
result, "Unexpected end of catalogue file"));
BEAST_EXPECT(to_string(pinned) == "3-6");
});
}
// An absurd leaf size in each kind of map is rejected while parsing.
auto withHugeLeaf = [&](std::size_t sizeField) {
auto bytes = good;
std::uint32_t const huge = 0xFFFFFFFF;
std::memcpy(&bytes[sizeField], &huge, 4);
return bytes;
};
// ... in the base ledger's state map.
if (auto const field = firstLeafSizeField(good, records[0].state);
BEAST_EXPECT(field))
{
loadEdited(
withHugeLeaf(*field),
true,
[&](auto const& result, auto const& pinned) {
BEAST_EXPECT(errorContains(
result, "Failed to load base ledger state"));
BEAST_EXPECT(pinned.empty());
});
}
// ... in a later ledger's state delta.
{
std::optional<std::size_t> index;
std::optional<std::size_t> field;
for (std::size_t i = 1; i < records.size() && !field; ++i)
if ((field = firstLeafSizeField(good, records[i].state)))
index = i;
if (BEAST_EXPECT(field))
{
std::uint32_t const seq = 3 + *index;
loadEdited(
withHugeLeaf(*field),
true,
[&](auto const& result, auto const& pinned) {
BEAST_EXPECT(errorContains(
result, "Failed to apply ledger delta"));
BEAST_EXPECT(boost::icl::contains(pinned, seq - 1));
BEAST_EXPECT(!boost::icl::contains(pinned, seq));
});
}
}
// ... in a ledger's transaction map.
{
std::optional<std::size_t> index;
std::optional<std::size_t> field;
for (std::size_t i = 0; i < records.size() && !field; ++i)
if ((field = firstLeafSizeField(good, records[i].tx)))
index = i;
if (BEAST_EXPECT(field))
{
std::uint32_t const seq = 3 + *index;
loadEdited(
withHugeLeaf(*field),
true,
[&](auto const& result, auto const& pinned) {
BEAST_EXPECT(errorContains(
result, "Failed to apply ledger delta"));
BEAST_EXPECT(!boost::icl::contains(pinned, seq));
});
}
}
boost::filesystem::remove_all(tempDir);
}
void
testCataloguePinnedStoreRequired(FeatureBitset features)
{
testcase("catalogue_load / catalogue_unpin: pinned store required");
using namespace test::jtx;
using beast::severities::kDisabled;
// pinned_type is set, but [node_db] type=rwdb is the null node
// store, on which SHAMapStore never builds the pinned store.
{
auto cfg = test::jtx::envconfig();
auto& nodeDb = cfg->section(ConfigSection::nodeDatabase());
nodeDb.set("pinned_type", "rwdb");
nodeDb.set("online_delete", "256");
Env env{*this, std::move(cfg), features, nullptr, kDisabled};
Json::Value params{Json::objectValue};
params[jss::input_file] = "/nonexistent.catl";
auto const result =
env.client().invoke("catalogue_load", params)[jss::result];
BEAST_EXPECT(
errorContains(result, "requires the pinned node store"));
}
// catalogue_unpin needs pinned_type configured at all.
{
Env env{
*this,
catalogueEnvconfig({.withPinnedType = false}),
features,
nullptr,
kDisabled};
Json::Value params{Json::objectValue};
params[jss::ledger_index_min] = 1;
params[jss::ledger_index_max] = 2;
auto const result =
env.client().invoke("catalogue_unpin", params)[jss::result];
BEAST_EXPECT(errorContains(result, "pinned_type"));
}
}
void
testCatalogueChainLinkage(FeatureBitset features)
{
testcase("catalogue_load: Ledgers must chain");
using namespace test::jtx;
Env env{
*this,
catalogueEnvconfig(),
features,
nullptr,
beast::severities::kDisabled,
};
prepareLedgerData(env, 3);
boost::filesystem::path tempDir =
boost::filesystem::temp_directory_path() /
boost::filesystem::unique_path();
boost::filesystem::create_directories(tempDir);
auto cataloguePath = (tempDir / "test.catl").string();
// Uncompressed, so ledger records can be located and edited.
{
Json::Value params{Json::objectValue};
params[jss::min_ledger] = 3;
params[jss::max_ledger] = 6;
params[jss::output_file] = cataloguePath;
params[jss::compression_level] = 0;
auto const result =
env.client().invoke("catalogue_create", params)[jss::result];
BEAST_EXPECT(result[jss::status] == jss::success);
}
std::vector<std::uint8_t> bytes;
{
std::ifstream in(cataloguePath, std::ios::binary);
bytes.assign(
std::istreambuf_iterator<char>(in),
std::istreambuf_iterator<char>());
}
// Ledger record: seq, hash, txHash, accountHash, parentHash, drops,
// closeFlags, closeTimeResolution, closeTime, parentCloseTime; then
// the state map and tx map as leaf records ending in tnTERMINAL.
constexpr std::size_t ledgerHeaderSize =
4 + 32 * 4 + 8 + sizeof(int) + 4 + 8 + 8;
auto skipMap = [&](std::size_t pos) -> std::size_t {
while (pos < bytes.size())
{
auto const type = bytes[pos++];
if (type == 255) // tnTERMINAL
return pos;
pos += 32;
if (type == 254) // tnREMOVE
continue;
if (pos + 4 > bytes.size())
break;
std::uint32_t size = 0;
std::memcpy(&size, &bytes[pos], 4);
pos += 4 + size;
}
return bytes.size();
};
// Skip the base ledger (3) to reach ledger 4.
std::size_t pos = sizeof(TestCATLHeader) + ledgerHeaderSize;
pos = skipMap(skipMap(pos));
if (!BEAST_EXPECT(pos + ledgerHeaderSize <= bytes.size()))
return;
LedgerInfo info;
std::size_t p = pos;
auto take = [&](void* dst, std::size_t n) {
std::memcpy(dst, &bytes[p], n);
p += n;
};
std::uint64_t drops = 0, closeTime = 0, parentCloseTime = 0;
std::uint32_t resolution = 0;
take(&info.seq, 4);
std::size_t const hashOffset = p;
take(info.hash.data(), 32);
take(info.txHash.data(), 32);
take(info.accountHash.data(), 32);
std::size_t const parentOffset = p;
take(info.parentHash.data(), 32);
take(&drops, 8);
take(&info.closeFlags, sizeof(info.closeFlags));
take(&resolution, 4);
take(&closeTime, 8);
take(&parentCloseTime, 8);
info.drops = drops;
info.closeTime = NetClock::time_point{NetClock::duration{closeTime}};
info.parentCloseTime =
NetClock::time_point{NetClock::duration{parentCloseTime}};
info.closeTimeResolution = NetClock::duration{resolution};
BEAST_EXPECT(info.seq == 4);
// Sanity: the record parses to a ledger whose hash checks out.
BEAST_EXPECT(calculateLedgerHash(info) == info.hash);
// Re-parent ledger 4 and fix up its hash, so the file stays
// internally consistent ledger by ledger; only the chain check can
// catch it.
info.parentHash = ~info.parentHash;
info.hash = calculateLedgerHash(info);
std::memcpy(&bytes[parentOffset], info.parentHash.data(), 32);
std::memcpy(&bytes[hashOffset], info.hash.data(), 32);
{
std::ofstream out(
cataloguePath, std::ios::binary | std::ios::trunc);
out.write(
reinterpret_cast<char const*>(bytes.data()), bytes.size());
}
Env loadEnv{
*this,
catalogueEnvconfig(),
features,
nullptr,
beast::severities::kDisabled,
};
Json::Value params{Json::objectValue};
params[jss::input_file] = cataloguePath;
params[jss::ignore_hash] = true; // the file was edited
auto const result =
loadEnv.client().invoke("catalogue_load", params)[jss::result];
BEAST_EXPECT(result[jss::status] == "error");
BEAST_EXPECT(
result[jss::error_message].asString().find(
"does not chain to its predecessor") != std::string::npos);
// The base ledger verified and stays pinned; nothing after it does.
auto const pinned =
loadEnv.app().getLedgerMaster().getPinnedLedgersRangeSet();
BEAST_EXPECT(boost::icl::contains(pinned, 3u));
BEAST_EXPECT(!boost::icl::contains(pinned, 4u));
boost::filesystem::remove_all(tempDir);
}
void
testCatalogueUnpin(FeatureBitset features)
{
testcase("catalogue_unpin");
using namespace test::jtx;
Env env{
*this,
catalogueEnvconfig(),
features,
nullptr,
beast::severities::kDisabled,
};
prepareLedgerData(env, 5);
boost::filesystem::path tempDir =
boost::filesystem::temp_directory_path() /
boost::filesystem::unique_path();
boost::filesystem::create_directories(tempDir);
auto cataloguePath = (tempDir / "test.catl").string();
{
Json::Value params{Json::objectValue};
params[jss::min_ledger] = 3;
params[jss::max_ledger] = 8;
params[jss::output_file] = cataloguePath;
auto const result =
env.client().invoke("catalogue_create", params)[jss::result];
BEAST_EXPECT(result[jss::status] == jss::success);
}
{
Json::Value params{Json::objectValue};
params[jss::input_file] = cataloguePath;
auto const result =
env.client().invoke("catalogue_load", params)[jss::result];
BEAST_EXPECT(result[jss::status] == jss::success);
}
auto& lm = env.app().getLedgerMaster();
BEAST_EXPECT(to_string(lm.getPinnedLedgersRangeSet()) == "3-8");
auto unpin = [&](std::optional<Json::Value> min,
std::optional<Json::Value> max) {
Json::Value params{Json::objectValue};
if (min)
params[jss::ledger_index_min] = *min;
if (max)
params[jss::ledger_index_max] = *max;
return env.client().invoke("catalogue_unpin", params)[jss::result];
};
// Bad parameters
BEAST_EXPECT(unpin({}, {})[jss::status] == "error");
BEAST_EXPECT(unpin(5, {})[jss::status] == "error");
BEAST_EXPECT(unpin(6, 5)[jss::status] == "error");
BEAST_EXPECT(unpin(-1, 5)[jss::status] == "error");
BEAST_EXPECT(unpin("x", 5)[jss::status] == "error");
BEAST_EXPECT(to_string(lm.getPinnedLedgersRangeSet()) == "3-8");
// Cut a hole
{
auto const result = unpin(5, 6);
BEAST_EXPECT(result[jss::status] == jss::success);
BEAST_EXPECT(result[jss::unpinned] == "5-6");
BEAST_EXPECT(result[jss::complete_ledgers_pinned] == "3-4,7-8");
BEAST_EXPECT(lm.isPinned(4) && lm.isPinned(7));
BEAST_EXPECT(!lm.isPinned(5) && !lm.isPinned(6));
}
// A range only partly pinned unpins just the pinned part
{
auto const result = unpin(1, 4);
BEAST_EXPECT(result[jss::status] == jss::success);
BEAST_EXPECT(result[jss::unpinned] == "3-4");
BEAST_EXPECT(result[jss::complete_ledgers_pinned] == "7-8");
}
// Nothing pinned in range: success, nothing unpinned. (Unsigned
// JSON values are accepted as well as non-negative signed ones.)
{
auto const result = unpin(Json::UInt(5), Json::UInt(6));
BEAST_EXPECT(result[jss::status] == jss::success);
BEAST_EXPECT(result[jss::unpinned] == "empty");
BEAST_EXPECT(result[jss::complete_ledgers_pinned] == "7-8");
}
// Unpinning the rest leaves nothing pinned. (state.db then records
// an empty set, which must not stop the next start; see
// SHAMapStore_test.)
{
auto const result = unpin(1, 100);
BEAST_EXPECT(result[jss::status] == jss::success);
BEAST_EXPECT(result[jss::unpinned] == "7-8");
BEAST_EXPECT(result[jss::complete_ledgers_pinned] == "empty");
BEAST_EXPECT(lm.getPinnedLedgersRangeSet().empty());
}
// Re-loading the file pins it again
{
Json::Value params{Json::objectValue};
params[jss::input_file] = cataloguePath;
auto const result =
env.client().invoke("catalogue_load", params)[jss::result];
BEAST_EXPECT(result[jss::status] == jss::success);
BEAST_EXPECT(to_string(lm.getPinnedLedgersRangeSet()) == "3-8");
}
boost::filesystem::remove_all(tempDir);
}
void
run() override
{
@@ -1081,6 +1716,10 @@ public:
testCatalogueFileSize(all);
testCatalogueCompression(all);
testCatalogueStatus(all);
testCatalogueChainLinkage(all);
testCatalogueUnpin(all);
testCatalogueLoadFailures(all);
testCataloguePinnedStoreRequired(all);
}
};
+50
View File
@@ -1934,6 +1934,56 @@ static RPCCallTestData const rpcCallTestArray[] = {
RPCCallTestData::bad_cast,
R"()"},
// catalogue_unpin
// ------------------------------------------------------------
{"catalogue_unpin: minimal.",
__LINE__,
{"catalogue_unpin", "5000001", "6000000"},
RPCCallTestData::no_exception,
R"({
"method" : "catalogue_unpin",
"params" : [
{
"api_version" : %API_VER%,
"ledger_index_max" : 6000000,
"ledger_index_min" : 5000001
}
]
})"},
{"catalogue_unpin: too few arguments.",
__LINE__,
{"catalogue_unpin", "5000001"},
RPCCallTestData::no_exception,
R"({
"method" : "catalogue_unpin",
"params" : [
{
"error" : "badSyntax",
"error_code" : 1,
"error_message" : "Syntax error."
}
]
})"},
{"catalogue_unpin: too many arguments.",
__LINE__,
{"catalogue_unpin", "1", "2", "3"},
RPCCallTestData::no_exception,
R"({
"method" : "catalogue_unpin",
"params" : [
{
"error" : "badSyntax",
"error_code" : 1,
"error_message" : "Syntax error."
}
]
})"},
{"catalogue_unpin: non-integer bound.",
__LINE__,
{"catalogue_unpin", "first", "6000000"},
RPCCallTestData::bad_cast,
R"()"},
// channel_authorize
// -----------------------------------------------------------
{"channel_authorize: minimal.",
+95
View File
@@ -36,6 +36,8 @@
#include <boost/utility/in_place_factory.hpp>
#include <chrono>
#include <condition_variable>
#include <mutex>
#include <optional>
#include <stdexcept>
#include <thread>
@@ -158,6 +160,7 @@ public:
onUDPMessage(
std::string const& message,
boost::asio::ip::tcp::endpoint const& remoteEndpoint,
Port const& port,
std::function<void(std::string const&)> sendResponse)
{
}
@@ -371,6 +374,7 @@ public:
onUDPMessage(
std::string const& message,
boost::asio::ip::tcp::endpoint const& remoteEndpoint,
Port const& port,
std::function<void(std::string const&)> sendResponse)
{
}
@@ -404,6 +408,96 @@ public:
pass();
}
// Records the datagrams delivered to onUDPMessage
struct UDPHandler : TestHandler
{
std::mutex mutex;
std::condition_variable cv;
std::vector<std::pair<std::string, bool>> received;
void
onUDPMessage(
std::string const& message,
boost::asio::ip::tcp::endpoint const& remoteEndpoint,
Port const& port,
std::function<void(std::string const&)> sendResponse)
{
std::lock_guard lock(mutex);
received.emplace_back(message, port.has_peer());
cv.notify_all();
}
bool
wait(std::size_t count, std::chrono::milliseconds timeout)
{
std::unique_lock lock(mutex);
return cv.wait_for(
lock, timeout, [&] { return received.size() >= count; });
}
};
// A peer port with udp_highway set also accepts UDP datagrams on the
// same port number; one without it does not.
void
udpHighwayTests()
{
testcase("UDP Superhighway door");
using namespace std::chrono_literals;
using udp = boost::asio::ip::udp;
TestSink sink{*this};
TestThread thread;
sink.threshold(beast::severities::Severity::kAll);
beast::Journal journal{sink};
auto const addr = boost::asio::ip::make_address(getEnvLocalhostAddr());
auto run = [&](bool highway) {
UDPHandler handler;
auto s = make_Server(handler, thread.get_io_context(), journal);
std::vector<Port> ports(1);
ports.back().name = "peer";
ports.back().ip = addr;
ports.back().port = 0;
ports.back().protocol.insert("peer");
ports.back().udp_highway = highway;
auto const eps = s->ports(ports);
if (!BEAST_EXPECT(eps.size() == 1))
return;
// The TCP door picked the port number; the UDP door shares it.
auto const port = eps.begin()->second.port();
BEAST_EXPECT(port != 0);
boost::asio::io_context io;
udp::socket client(io, udp::endpoint(addr, 0));
std::string const message = "XUSHTEST";
client.send_to(
boost::asio::buffer(message), udp::endpoint(addr, port));
if (highway)
{
if (BEAST_EXPECT(handler.wait(1, 5s)))
{
std::lock_guard lock(handler.mutex);
BEAST_EXPECT(handler.received.front().first == message);
BEAST_EXPECT(handler.received.front().second);
}
}
else
{
BEAST_EXPECT(!handler.wait(1, 200ms));
}
s = nullptr;
};
run(true);
run(false);
}
void
testBadConfig()
{
@@ -547,6 +641,7 @@ public:
{
basicTests();
stressTest();
udpHighwayTests();
testBadConfig();
}
};
+79
View File
@@ -29,7 +29,9 @@
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/protocol/Serializer.h>
#include <algorithm>
#include <optional>
#include <vector>
namespace ripple {
namespace tests {
@@ -131,6 +133,7 @@ public:
testCanonicalizeMergesInnerChildren(journal);
testCanonicalizeSharesDuplicateContent(journal);
testGetLinkedNode(journal);
testUpsertAndFlushToSink(journal);
}
void
@@ -284,6 +287,82 @@ public:
BEAST_EXPECT(liveLinked > 0);
}
void
testUpsertAndFlushToSink(beast::Journal const& journal)
{
testcase("upsertGiveItem and flushDirty to a sink");
tests::TestNodeFamily f(journal);
SHAMap a{SHAMapType::FREE, f}; // built with upsertGiveItem
SHAMap b{SHAMapType::FREE, f}; // reference: addItem/updateGiveItem
// Distinct first nibbles: a root with three leaf children.
constexpr uint256 h1(
"092891fe4ef6cee585fdc6fda0e09eb4d386363158ec3321b8123e5a772c6ca7");
constexpr uint256 h2(
"436ccbac3347baa1f1e53baeef1f43334da88f1f6d70d963b833afd6dfa289fe");
constexpr uint256 h3(
"b92891fe4ef6cee585fdc6fda1e09eb4d386363158ec3321b8123e5a772c6ca8");
auto const nm = SHAMapNodeType::tnTRANSACTION_NM;
// Absent keys are added.
BEAST_EXPECT(a.upsertGiveItem(nm, make_shamapitem(h1, IntToVUC(1))));
BEAST_EXPECT(a.upsertGiveItem(nm, make_shamapitem(h2, IntToVUC(2))));
BEAST_EXPECT(a.upsertGiveItem(nm, make_shamapitem(h3, IntToVUC(3))));
// A present key has its data replaced.
BEAST_EXPECT(a.upsertGiveItem(nm, make_shamapitem(h2, IntToVUC(20))));
// A change of leaf type is refused and leaves the item untouched.
BEAST_EXPECT(!a.upsertGiveItem(
SHAMapNodeType::tnTRANSACTION_MD,
make_shamapitem(h1, IntToVUC(10))));
BEAST_EXPECT(b.addItem(nm, make_shamapitem(h1, IntToVUC(1))));
BEAST_EXPECT(b.addItem(nm, make_shamapitem(h2, IntToVUC(2))));
BEAST_EXPECT(b.addItem(nm, make_shamapitem(h3, IntToVUC(3))));
BEAST_EXPECT(b.updateGiveItem(nm, make_shamapitem(h2, IntToVUC(20))));
{
auto const v20 = IntToVUC(20);
auto const& item = a.peekItem(h2);
BEAST_EXPECT(
item && item->slice() == Slice(v20.data(), v20.size()));
auto const v1 = IntToVUC(1);
auto const& item1 = a.peekItem(h1);
BEAST_EXPECT(
item1 && item1->slice() == Slice(v1.data(), v1.size()));
}
// Flush to a sink. Do not call getHash() on `a` first: it unshares
// the map and the flush would then have nothing to write.
auto const cacheBefore = f.getTreeNodeCache()->size();
std::vector<uint256> written;
bool sinkArgsOk = true;
SHAMap::FlushSink const sink =
[&](NodeObjectType t, Blob&& data, uint256 const& hash) {
sinkArgsOk =
sinkArgsOk && t == hotTRANSACTION_NODE && !data.empty();
written.push_back(hash);
};
a.flushDirty(hotTRANSACTION_NODE, sink);
BEAST_EXPECT(sinkArgsOk);
BEAST_EXPECT(written.size() == 4); // root + three leaves
// Unlike flushDirty(t), nothing was canonicalized into the cache.
BEAST_EXPECT(f.getTreeNodeCache()->size() == cacheBefore);
// The upserted map matches the reference, and its root was written.
BEAST_EXPECT(a.getHash() == b.getHash());
BEAST_EXPECT(
std::find(
written.begin(), written.end(), a.getHash().as_uint256()) !=
written.end());
// Everything is now shared: a second flush writes nothing.
written.clear();
a.flushDirty(hotTRANSACTION_NODE, sink);
BEAST_EXPECT(written.empty());
}
void
testGetLinkedNode(beast::Journal const& journal)
{
+124 -3
View File
@@ -41,8 +41,11 @@
#include <xrpl/basics/random.h>
#include <xrpl/beast/core/LexicalCast.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/beast/utility/rngfill.h>
#include <xrpl/crypto/csprng.h>
#include <xrpl/protocol/BuildInfo.h>
#include <xrpl/protocol/Feature.h>
#include <xrpl/protocol/Indexes.h>
#include <xrpl/protocol/digest.h>
#include <algorithm>
@@ -348,6 +351,10 @@ RCLConsensus::Adaptor::onClose(
make_shamapitem(tx.first->getTransactionID(), s.slice()));
}
// featureRNG: our entropy contribution, if we add one. It is not tracked
// by the censorship detector: it is only valid for this ledger.
std::optional<uint256> entropyTxID;
// Keep enough validation history for the next flag ledger's score table
// even when we don't vote this round (not proposing / wrong LCL).
// Otherwise toKeep_ goes stale, validations older than
@@ -389,6 +396,26 @@ RCLConsensus::Adaptor::onClose(
app_.getValidations(),
initialSet);
}
// featureRNG: our entropy contribution goes into our own initial
// position, never the open ledger, so it only becomes public once our
// transaction set for this ledger is fixed.
if (prevLedger->rules().enabled(featureRNG))
{
if (auto const entropyTx = makeEntropyTx(prevLedger))
{
Serializer s;
entropyTx->add(s);
entropyTxID = entropyTx->getTransactionID();
if (!initialSet->addItem(
SHAMapNodeType::tnTRANSACTION_NM,
make_shamapitem(*entropyTxID, s.slice())))
{
entropyTxID.reset();
JLOG(j_.warn()) << "RNG: failed to add entropy txn";
}
}
}
}
// Now we need an immutable snapshot
@@ -400,9 +427,10 @@ RCLConsensus::Adaptor::onClose(
RCLCensorshipDetector<TxID, LedgerIndex>::TxIDSeqVec proposed;
initialSet->visitLeaves(
[&proposed,
seq](boost::intrusive_ptr<SHAMapItem const> const& item) {
proposed.emplace_back(item->key(), seq);
[&proposed, &entropyTxID, seq](
boost::intrusive_ptr<SHAMapItem const> const& item) {
if (item->key() != entropyTxID)
proposed.emplace_back(item->key(), seq);
});
censorshipDetector_.propose(std::move(proposed));
@@ -422,6 +450,99 @@ RCLConsensus::Adaptor::onClose(
validatorKeys_.nodeID}};
}
std::optional<STTx>
RCLConsensus::Adaptor::makeEntropyTx(
std::shared_ptr<Ledger const> const& prevLedger)
{
if (!validatorKeys_.keys)
return std::nullopt;
auto const& masterKey = validatorKeys_.keys->masterPublicKey;
// The commitment the ledger currently holds for us, if any.
std::optional<uint256> commitment;
if (auto const sle = prevLedger->read(keylet::random()))
{
for (auto const& entry : sle->getFieldArray(sfRandomDigests))
{
if (makeSlice(entry.getFieldVL(sfPublicKey)) == masterKey.slice())
{
commitment = entry.getFieldH256(sfNextRandomDigest);
break;
}
}
}
// We only ever hold the secret from our last contribution, which has
// never been published. Reveal it iff the ledger holds its commitment.
// Otherwise (first contribution, restart, or our last contribution was
// not included) we only commit this time, and our previous secret,
// whatever became of it, is never used again.
std::optional<uint256> reveal;
if (commitment && rngSecret_ && rngSecret_->first == *commitment)
reveal = rngSecret_->second;
uint256 secret;
beast::rngfill(secret.begin(), secret.size(), crypto_prng());
uint256 const nextCommitment = sha512Half(secret);
rngSecret_.emplace(nextCommitment, secret);
auto const seq = prevLedger->info().seq + 1;
JLOG(j_.debug()) << "RNG: entropy txn for ledger " << seq
<< (reveal ? " (reveal and commit)" : " (commit only)");
return STTx(ttENTROPY, [&](auto& obj) {
obj.setFieldU32(sfLedgerSequence, seq);
obj.setFieldVL(sfPublicKey, masterKey.slice());
obj.setFieldH256(sfNextRandomDigest, nextCommitment);
if (reveal)
obj.setFieldH256(sfRandomData, *reveal);
});
}
std::optional<NodeID>
RCLConsensus::Adaptor::forcedTxOwner(
RCLCxTx const& tx,
RCLCxLedger const& prevLedger) const
{
if (!prevLedger.ledger_ || !prevLedger.ledger_->rules().enabled(featureRNG))
return std::nullopt;
try
{
SerialIter sit(tx.tx_->slice());
STTx const stx(sit);
if (stx.getTxnType() != ttENTROPY ||
stx.getFieldU32(sfLedgerSequence) != prevLedger.seq() + 1)
return std::nullopt;
auto const pk = stx.getFieldVL(sfPublicKey);
if (!publicKeyType(makeSlice(pk)))
return std::nullopt;
// Only the exact bytes makeEntropyTx would build, so an owner cannot
// force extra fields (memos and the like) in with its contribution.
STTx const canonical(ttENTROPY, [&](auto& obj) {
obj.setFieldU32(sfLedgerSequence, prevLedger.seq() + 1);
obj.setFieldVL(sfPublicKey, pk);
obj.setFieldH256(
sfNextRandomDigest, stx.getFieldH256(sfNextRandomDigest));
if (stx.isFieldPresent(sfRandomData))
obj.setFieldH256(sfRandomData, stx.getFieldH256(sfRandomData));
});
if (canonical.getTransactionID() != tx.id())
return std::nullopt;
return calcNodeID(PublicKey(makeSlice(pk)));
}
catch (std::exception const&)
{
return std::nullopt;
}
}
void
RCLConsensus::Adaptor::onForceAccept(
Result const& result,
+23
View File
@@ -34,10 +34,12 @@
#include <xrpl/basics/Log.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/protocol/RippleLedgerHash.h>
#include <xrpl/protocol/STTx.h>
#include <xrpl/protocol/STValidation.h>
#include <atomic>
#include <memory>
#include <mutex>
#include <optional>
#include <set>
#include <sstream>
#include <string>
@@ -91,6 +93,11 @@ class RCLConsensus
RCLCensorshipDetector<TxID, LedgerIndex> censorshipDetector_;
NegativeUNLVote nUnlVote_;
// featureRNG: the secret behind the commitment in our most recent
// entropy contribution, as {commitment, secret}. Only touched by
// onClose, which runs under the consensus lock.
std::optional<std::pair<uint256, uint256>> rngSecret_;
public:
using Ledger_t = RCLCxLedger;
using NodeID_t = NodeID;
@@ -182,7 +189,23 @@ class RCLConsensus
return parms_;
}
/** featureRNG: if tx is a validator's entropy contribution for the
ledger after prevLedger, the NodeID of that validator.
Consensus votes for such a transaction while it is in its owner's
own position during the opening window of the establish phase.
*/
std::optional<NodeID>
forcedTxOwner(RCLCxTx const& tx, RCLCxLedger const& prevLedger) const;
private:
/** featureRNG: build our entropy contribution for the ledger after
prevLedger, revealing our previous secret if the ledger holds its
commitment, and committing to a fresh one.
*/
std::optional<STTx>
makeEntropyTx(std::shared_ptr<Ledger const> const& prevLedger);
//---------------------------------------------------------------------
// The following members implement the generic Consensus requirements
// and are marked private to indicate ONLY Consensus<Adaptor> will call
+14
View File
@@ -491,6 +491,16 @@ public:
Expected<uint256, HookReturnCode>
ledger_nonce() const;
/// RNG APIs (featureRNG)
// A uniform integer in [0, sides).
Expected<uint64_t, HookReturnCode>
dice(uint32_t sides) const;
// len (1 to 512) random bytes.
Expected<Bytes, HookReturnCode>
util_random(uint32_t len) const;
Expected<Keylet, HookReturnCode>
ledger_keylet(Keylet const& klLo, Keylet const& klHi) const;
@@ -630,6 +640,10 @@ private:
Rules const& rules,
int recursion_depth = 0) // used internally
const;
// featureRNG: advance the ledger's RNG state and return one output block.
Expected<uint256, HookReturnCode>
rng_next() const;
};
} // namespace hook
+7 -2
View File
@@ -86,7 +86,7 @@ bool
canHook(
STTx const& tx,
ripple::uint256 hookOn,
std::optional<ripple::Slice> hookName);
std::optional<ripple::Blob> hookName);
bool
canEmit(ripple::TxType txType, ripple::uint256 hookCanEmit);
@@ -129,7 +129,11 @@ apply(
uint32_t wasmParam,
uint8_t hookChainPosition,
// result of apply() if this is weak exec
std::shared_ptr<STObject const> const& provisionalMeta);
std::shared_ptr<STObject const> const& provisionalMeta,
// fix20261005: the original emitted transaction for a ttEMIT_FAILURE
// callback, captured before the cleanup pseudo removed its
// ltEMITTED_TXN entry. Ignored before the fix.
std::optional<ripple::STObject> const& emitFailureTxn = std::nullopt);
struct HookContext;
@@ -207,6 +211,7 @@ struct HookContext
uint16_t emit_nonce_counter{
0}; // incremented whenever nonce is called to ensure unique nonces
uint16_t ledger_nonce_counter{0};
uint16_t rng_draws{0}; // featureRNG: dice/random calls this execution
int64_t expected_etxn_count{-1}; // make this a 64bit int so the uint32
// from the hookapi cant overflow it
std::map<ripple::uint256, bool> nonce_used{};
+191 -7
View File
@@ -4,7 +4,10 @@
#include <xrpld/app/ledger/OpenLedger.h>
#include <xrpld/app/ledger/TransactionMaster.h>
#include <xrpld/app/tx/detail/Import.h>
#include <xrpld/ledger/ApplyViewImpl.h>
#include <xrpl/protocol/STParsedJSON.h>
#include <algorithm>
#include <limits>
namespace hook {
@@ -1581,6 +1584,18 @@ HookAPI::otxn_field(uint32_t field_id) const
if (fieldType == sfInvalid)
return Unexpected(INVALID_FIELD);
// fix20261005: on an emit failure callback the field is read from the
// original emitted transaction, so check presence there rather than on
// the ttEMIT_FAILURE pseudo (which carries sfLedgerSequence and
// sfTransactionHash the emitted txn lacks, and lacks most of its fields).
if (hookCtx.emitFailure &&
hookCtx.applyCtx.view().rules().enabled(fix20261005))
{
if (!hookCtx.emitFailure->isFieldPresent(fieldType))
return Unexpected(DOESNT_EXIST);
return &hookCtx.emitFailure->getField(fieldType);
}
if (!hookCtx.applyCtx.tx.isFieldPresent(fieldType))
return Unexpected(DOESNT_EXIST);
@@ -1895,6 +1910,147 @@ HookAPI::ledger_nonce() const
return hash;
}
// featureRNG
//
// The ledger's RNG state lives in the Random object. The ttENTROPY
// transactions applied at the start of the ledger seed it from validator
// reveals; after that every draw from any Hook replaces it with
// H(step, state). So the state iterates between API calls, between Hook
// executions and between transactions, including transactions that end in a
// tec: ApplyContext::discard keeps the advance, so a rollback cannot preserve
// an RNG state for whatever executes next.
//
// No Hook can read the state, so none can work out what a later draw will
// return, even one that ran earlier in the same transaction: slot_set and
// ledger_keylet refuse the Random object, slot_set refuses ttENTROPY
// transactions, the object is left out of provisional metadata (meta_slot),
// and its state and commitment fields are never written to any metadata.
//
// Each draw returns H(output, state, txid, hook account, hook hash, chain
// position, strong, callback, wasm param). Those extra inputs keep outputs
// distinct for different executions even if they somehow saw the same state.
//
// The state the transaction's first draw saw is written to the transaction's
// final metadata (sfRandomData), never its provisional metadata.
Expected<uint256, HookReturnCode>
HookAPI::rng_next() const
{
static constexpr std::uint32_t rngOutputTag = 0x524E474F; // 'RNGO'
static constexpr std::uint32_t rngStepTag = 0x524E4749; // 'RNGI'
auto& applyCtx = hookCtx.applyCtx;
auto& view = applyCtx.view();
if (!view.rules().enabled(featureRNG))
return Unexpected(TOO_LITTLE_ENTROPY);
if (hookCtx.rng_draws >= hook_api::max_rng_draws)
return Unexpected(TOO_MANY_NONCES);
auto sle = view.peek(keylet::random());
if (!sle)
return Unexpected(TOO_LITTLE_ENTROPY);
// A closed ledger may only draw on its own seed. Open ledger application
// is a preview, so it may run on the last closed ledger's state; the
// final result is decided when the ledger closes.
auto const seq = view.seq();
auto const seededFor = sle->getFieldU32(sfLedgerSequence);
if (seededFor != seq && !(view.open() && seededFor + 1 == seq))
return Unexpected(TOO_LITTLE_ENTROPY);
// At least 80% of the UNLReport must have contributed (at least one
// validator if there is no UNLReport). Both fields are 16 bits on the
// ledger; they are widened here so the scaled comparison cannot overflow.
std::uint32_t const count = sle->getFieldU16(sfEntropyCount);
std::uint32_t const denominator = sle->getFieldU16(sfEntropyDenominator);
if (count == 0 || count * 5 < denominator * 4)
return Unexpected(TOO_LITTLE_ENTROPY);
uint256 const state = sle->getFieldH256(sfRandomData);
if (auto* avi = dynamic_cast<ApplyViewImpl*>(&view))
avi->setRandomData(state);
auto const& hr = hookCtx.result;
uint256 const out = ripple::sha512Half(
rngOutputTag,
state,
applyCtx.tx.getTransactionID(),
hr.account,
hr.hookHash,
static_cast<std::uint8_t>(hr.hookChainPosition),
static_cast<std::uint8_t>(hr.isStrong ? 1 : 0),
static_cast<std::uint8_t>(hr.isCallback ? 1 : 0),
hr.wasmParam);
sle->setFieldH256(sfRandomData, ripple::sha512Half(rngStepTag, state));
view.update(sle);
++hookCtx.rng_draws;
return out;
}
Expected<uint64_t, HookReturnCode>
HookAPI::dice(uint32_t sides) const
{
if (sides == 0)
return Unexpected(INVALID_ARGUMENT);
auto block = rng_next();
if (!block)
return Unexpected(block.error());
// Rejection sampling over big-endian 64-bit words, so every side is
// exactly equally likely. A word is rejected with probability below
// 2^-32, so the bound on blocks is never reached in practice; it only
// keeps the loop finite.
std::uint64_t const max = std::numeric_limits<std::uint64_t>::max();
std::uint64_t const excess = (max % sides + 1) % sides; // 2^64 mod sides
std::uint64_t word = 0;
uint256 b = *block;
for (int i = 0; i < 16; ++i)
{
for (std::size_t w = 0; w < 4; ++w)
{
word = 0;
for (std::size_t k = 0; k < 8; ++k)
word = (word << 8) | b.data()[w * 8 + k];
if (word <= max - excess)
return word % sides;
}
b = ripple::sha512Half(b);
}
return word % sides;
}
Expected<Bytes, HookReturnCode>
HookAPI::util_random(uint32_t len) const
{
if (len == 0)
return Unexpected(TOO_SMALL);
if (len > 512)
return Unexpected(TOO_BIG);
auto block = rng_next();
if (!block)
return Unexpected(block.error());
Bytes out;
out.reserve(len);
uint256 b = *block;
while (true)
{
auto const n = std::min<std::size_t>(32, len - out.size());
out.insert(out.end(), b.data(), b.data() + n);
if (out.size() >= len)
break;
b = ripple::sha512Half(b);
}
return out;
}
Expected<Keylet, HookReturnCode>
HookAPI::ledger_keylet(Keylet const& klLo, Keylet const& klHi) const
{
@@ -1905,6 +2061,10 @@ HookAPI::ledger_keylet(Keylet const& klLo, Keylet const& klHi) const
std::optional<ripple::uint256> found =
hookCtx.applyCtx.view().succ(klLo.key, klHi.key.next());
// featureRNG: the RNG object is invisible to Hooks, even by key.
if (found && *found == keylet::random().key)
found = hookCtx.applyCtx.view().succ(*found, klHi.key.next());
if (!found)
return Unexpected(DOESNT_EXIST);
@@ -2149,6 +2309,12 @@ HookAPI::slot_set(Bytes const& data, uint32_t slot_no) const
if (kl->key == beast::zero)
return Unexpected(DOESNT_EXIST);
// featureRNG: the RNG state must never be readable by a Hook, or an
// earlier Hook could compute what later draws will return. Matched on
// the key because a Hook-supplied keylet type can be ltANY.
if (kl->key == keylet::random().key)
return Unexpected(DOESNT_EXIST);
auto const sle = hookCtx.applyCtx.view().read(*kl);
if (!sle)
return Unexpected(DOESNT_EXIST);
@@ -2172,6 +2338,11 @@ HookAPI::slot_set(Bytes const& data, uint32_t slot_no) const
slot_value = p->first->getSTransaction();
else
return Unexpected(DOESNT_EXIST);
// featureRNG: validator reveals are never readable by Hooks.
if (*slot_value &&
(*slot_value)->getFieldU16(sfTransactionType) == ttENTROPY)
return Unexpected(DOESNT_EXIST);
}
else
return Unexpected(INVALID_ARGUMENT);
@@ -2828,11 +2999,19 @@ HookAPI::set_state_cache(
if (availableForReserves < hookStateScale && modified)
return Unexpected(RESERVE_INSUFFICIENT);
// Prior to fix20261005 the first-account branch charged write
// resources (a state modification, a reserve slot and, if needed, a
// namespace slot) even when the entry was only being read into the
// cache (modified == false). Every other branch only charges for
// writes. With the fix, reads into the cache are free; a later write
// of the same key is charged by the existing-key branch below.
bool const chargeWrite = modified || !view.rules().enabled(fix20261005);
int64_t namespaceCount = accSLE->isFieldPresent(sfHookNamespaces)
? accSLE->getFieldV256(sfHookNamespaces).size()
: 0;
if (createNamespace)
if (createNamespace && chargeWrite)
{
// overflow should never ever happen but check anyway
if (namespaceCount + 1 < namespaceCount)
@@ -2842,15 +3021,20 @@ HookAPI::set_state_cache(
return Unexpected(TOO_MANY_NAMESPACES);
}
stateMap.modified_entry_count++;
if (chargeWrite)
{
stateMap.modified_entry_count++;
// sanity check
if (view.rules().enabled(featureExtendedHookState) &&
availableForReserves < hookStateScale)
return Unexpected(INTERNAL_ERROR);
// sanity check
if (view.rules().enabled(featureExtendedHookState) &&
availableForReserves < hookStateScale)
return Unexpected(INTERNAL_ERROR);
availableForReserves -= hookStateScale;
}
stateMap[acc] = {
availableForReserves - hookStateScale,
availableForReserves,
namespaceCount,
hookStateScale,
{{ns, {{key, {modified, data}}}}}};
+70 -10
View File
@@ -558,7 +558,8 @@ getTransactionalStakeHolders(STTx const& tx, ReadView const& rv)
case ttFEE:
case ttUNL_MODIFY:
case ttEMIT_FAILURE:
case ttUNL_REPORT: {
case ttUNL_REPORT:
case ttENTROPY: {
break;
}
default: {
@@ -813,7 +814,7 @@ bool
hook::canHook(
STTx const& tx,
ripple::uint256 hookOn,
std::optional<ripple::Slice> hookName)
std::optional<ripple::Blob> hookName)
{
if (!canHookTT(tx.getTxnType(), hookOn))
return false;
@@ -827,7 +828,7 @@ hook::canHook(
// transaction, so we can't hook without the hook name
return false;
return tx[sfHookName] == *hookName;
return tx.getFieldVL(sfHookName) == *hookName;
}
ripple::uint256
@@ -1041,8 +1042,29 @@ hook::apply(
bool isStrong,
uint32_t wasmParam,
uint8_t hookChainPosition,
std::shared_ptr<STObject const> const& provisionalMeta)
std::shared_ptr<STObject const> const& provisionalMeta,
std::optional<ripple::STObject> const& emitFailureTxn)
{
std::optional<ripple::STObject> emitFailure;
if (isCallback && (wasmParam & 1))
{
if (applyCtx.view().rules().enabled(fix20261005))
{
// fix20261005: by the time the callback runs the ttEMIT_FAILURE
// pseudo has already erased the ltEMITTED_TXN entry from this
// view, so it can't be read back here. The caller captures the
// original emitted transaction before base application.
emitFailure = emitFailureTxn;
}
else
{
// pre-fix: reads the entry the pseudo has already erased
emitFailure = (*(applyCtx.view().peek(keylet::emittedTxn(
applyCtx.tx.getFieldH256(sfTransactionHash)))))
.downcast<STObject>();
}
}
HookContext hookCtx = {
.applyCtx = applyCtx,
// we will return this context object (RVO / move constructed)
@@ -1074,12 +1096,7 @@ hook::apply(
.foreignStateSetDisabled = false,
.provisionalMeta = provisionalMeta,
.foreignStateGrantCache = {}},
.emitFailure = isCallback && wasmParam & 1
? std::optional<ripple::STObject>(
(*(applyCtx.view().peek(keylet::emittedTxn(
applyCtx.tx.getFieldH256(sfTransactionHash)))))
.downcast<STObject>())
: std::optional<ripple::STObject>(),
.emitFailure = std::move(emitFailure),
.module = nullptr,
.api_ = nullptr};
@@ -2866,6 +2883,49 @@ DEFINE_HOOK_FUNCTION(
HOOK_TEARDOWN();
}
// featureRNG: a uniform integer in [0, sides)
DEFINE_HOOK_FUNCTION(int64_t, dice, uint32_t sides)
{
HOOK_SETUP();
auto const result = api.dice(sides);
if (!result)
return result.error();
return result.value();
HOOK_TEARDOWN();
}
// featureRNG: write_len (1 to 512) random bytes
DEFINE_HOOK_FUNCTION(
int64_t,
util_random,
uint32_t write_ptr,
uint32_t write_len)
{
HOOK_SETUP();
if (NOT_IN_BOUNDS(write_ptr, write_len, memory_length))
return OUT_OF_BOUNDS;
auto const result = api.util_random(write_len);
if (!result)
return result.error();
auto const& bytes = result.value();
WRITE_WASM_MEMORY_AND_RETURN(
write_ptr,
write_len,
bytes.data(),
bytes.size(),
memory,
memory_length);
HOOK_TEARDOWN();
}
DEFINE_HOOK_FUNCTION(
int64_t,
ledger_keylet,
+3 -2
View File
@@ -25,7 +25,8 @@ namespace ripple {
AcceptedLedger::AcceptedLedger(
std::shared_ptr<ReadView const> const& ledger,
Application& app)
Application& app,
bool withJson)
: mLedger(ledger)
{
transactions_.reserve(256);
@@ -33,7 +34,7 @@ AcceptedLedger::AcceptedLedger(
auto insertAll = [&](auto const& txns) {
for (auto const& item : txns)
transactions_.emplace_back(std::make_unique<AcceptedLedgerTx>(
ledger, item.first, item.second));
ledger, item.first, item.second, withJson));
};
transactions_.reserve(256);
+7 -1
View File
@@ -44,9 +44,15 @@ namespace ripple {
class AcceptedLedger : public CountedObject<AcceptedLedger>
{
public:
/** @param withJson build each transaction's publishable JSON. Pass
false only for ledgers that will never be published from this
object (e.g. pinned catalogue history being saved to SQL);
getJson() on such transactions returns a null value.
*/
AcceptedLedger(
std::shared_ptr<ReadView const> const& ledger,
Application& app);
Application& app,
bool withJson = true);
std::shared_ptr<ReadView const> const&
getLedger() const
+7 -1
View File
@@ -28,7 +28,8 @@ namespace ripple {
AcceptedLedgerTx::AcceptedLedgerTx(
std::shared_ptr<ReadView const> const& ledger,
std::shared_ptr<STTx const> const& txn,
std::shared_ptr<STObject const> const& met)
std::shared_ptr<STObject const> const& met,
bool withJson)
: mTxn(txn)
, mMeta(txn->getTransactionID(), ledger->seq(), *met)
, mAffected(mMeta.getAffectedAccounts())
@@ -41,6 +42,11 @@ AcceptedLedgerTx::AcceptedLedgerTx(
met->add(s);
mRawMeta = std::move(s.modData());
// The JSON (full tx + meta + hex raw meta per transaction) is only
// needed for publishing. Bulk SQL saves of pinned history skip it.
if (!withJson)
return;
mJson = Json::objectValue;
mJson[jss::transaction] = mTxn->getJson(JsonOptions::none);
+3 -1
View File
@@ -46,7 +46,8 @@ public:
AcceptedLedgerTx(
std::shared_ptr<ReadView const> const& ledger,
std::shared_ptr<STTx const> const&,
std::shared_ptr<STObject const> const&);
std::shared_ptr<STObject const> const&,
bool withJson = true);
std::shared_ptr<STTx const> const&
getTxn() const
@@ -88,6 +89,7 @@ public:
std::string
getEscMeta() const;
/** Null if constructed with withJson == false. */
Json::Value const&
getJson() const
{
+13
View File
@@ -228,6 +228,19 @@ public:
isPinned(std::uint32_t seq);
void
unpinLedger(std::uint32_t seq);
/** Remove `ranges` from the pinned set.
@param dropFromComplete of the ledgers actually unpinned, those to
also remove from complete_ledgers now (used to roll back a
load). Otherwise unpinned ledgers stay complete until online
delete releases them.
@return the ledgers that were pinned and no longer are.
*/
RangeSet<std::uint32_t>
unpinLedgers(
RangeSet<std::uint32_t> const& ranges,
RangeSet<std::uint32_t> const& dropFromComplete = {});
bool
isValidated(ReadView const& ledger);
bool
@@ -106,6 +106,41 @@ applyTransactions(
bool certainRetry = true;
std::size_t count = 0;
// featureRNG: validator entropy contributions are applied before anything
// else, so this ledger's seed is final before any Hook can draw from it.
// Pseudo-transactions otherwise sort to an arbitrary position. Each is
// attempted exactly once; one that fails is never retried later, where it
// could change the seed after Hooks had drawn from it.
if (view.rules().enabled(featureRNG))
{
for (auto it = txns.begin(); it != txns.end();)
{
if (it->second->getTxnType() != ttENTROPY)
{
++it;
continue;
}
auto const txid = it->first.getTXID();
try
{
if (applyTransaction(
app, view, *it->second, false, tapNONE, j) ==
ApplyTransactionResult::Success)
++count;
else
failed.insert(txid);
}
catch (std::exception const& ex)
{
JLOG(j.warn())
<< "Transaction " << txid << " throws: " << ex.what();
failed.insert(txid);
}
it = txns.erase(it);
}
}
// Attempt to apply all of the retriable transactions
for (int pass = 0; pass < LEDGER_TOTAL_PASSES; ++pass)
{
+16 -1
View File
@@ -501,7 +501,9 @@ LedgerMaster::storeLedger(std::shared_ptr<Ledger const> ledger, bool pin)
std::lock_guard sl(mPinnedLock);
mPinnedLedgers.insert(range(seq, seq));
}
JLOG(m_journal.info()) << "Pinned ledger : " << seq;
// catalogue_load pins every ledger it imports; at info this would
// emit one line per ledger for the whole file.
JLOG(m_journal.debug()) << "Pinned ledger : " << seq;
}
return true;
}
@@ -586,6 +588,19 @@ LedgerMaster::unpinLedger(std::uint32_t seq)
mPinnedLedgers.erase(range(seq, seq));
}
RangeSet<std::uint32_t>
LedgerMaster::unpinLedgers(
RangeSet<std::uint32_t> const& ranges,
RangeSet<std::uint32_t> const& dropFromComplete)
{
std::scoped_lock lock(mCompleteLock, mPinnedLock);
RangeSet<std::uint32_t> removed = mPinnedLedgers & ranges;
mPinnedLedgers -= removed;
if (!dropFromComplete.empty())
mCompleteLedgers -= (removed & dropFromComplete);
return removed;
}
bool
LedgerMaster::isValidated(ReadView const& ledger)
{
+17 -1
View File
@@ -1260,13 +1260,29 @@ ApplicationImp::setup(boost::program_options::variables_map const& cmdline)
Section const& upVoted = config_->section(SECTION_AMENDMENTS);
Section const& ignored = config_->section(SECTION_IGNORE_AMENDMENTS);
// [ignore_amendments] exists to keep long-lived test networks syncable
// after an amendment they activated is removed from the codebase. It
// must never be usable on a production network.
if (!ignored.lines().empty() &&
(config_->NETWORK_ID == 0 || config_->NETWORK_ID == 21337))
{
JLOG(m_journal.fatal()) << "[" SECTION_IGNORE_AMENDMENTS
"] is not permitted on "
"network_id "
<< config_->NETWORK_ID;
return false;
}
m_amendmentTable = make_AmendmentTable(
*this,
config().AMENDMENT_MAJORITY_TIME,
supported,
upVoted,
downVoted,
logs_->journal("Amendments"));
logs_->journal("Amendments"),
ignored);
}
Pathfinder::initPathTable();
+2 -1
View File
@@ -211,7 +211,8 @@ make_AmendmentTable(
std::vector<AmendmentTable::FeatureInfo> const& supported,
Section const& enabled,
Section const& vetoed,
beast::Journal journal);
beast::Journal journal,
Section const& ignored = Section{});
} // namespace ripple
+6
View File
@@ -1725,6 +1725,12 @@ NetworkOPsImp::apply(std::unique_lock<std::mutex>& batchLock)
tx.set_deferred(e.result == terQUEUED);
// FIXME: This should be when we received it
app_.overlay().relay(e.transaction->getID(), tx, *toSkip);
// Also fan it out over the UDP Superhighway, if enabled.
// shouldRelay() above ensures we do this at most once.
app_.overlay().publishTxXUSH(
s.slice(), e.transaction->getID());
e.transaction->setBroadcast();
}
}
+51 -9
View File
@@ -381,7 +381,9 @@ void
SHAMapStoreImp::loadPinnedRanges()
{
auto rangesStr = state_db_.getPinnedRanges();
if (rangesStr.empty())
// "empty" is what to_string() renders for an empty set; accept it in
// case it was stored before setPinnedRanges() learned to write "".
if (rangesStr.empty() || rangesStr == "empty")
return;
auto const& nscfg = app_.config().section(ConfigSection::nodeDatabase());
@@ -411,22 +413,62 @@ SHAMapStoreImp::loadPinnedRanges()
static_cast<bool>(app_.getNodeStore().fetchNodeObject(hash, seq));
};
//
// catalogue_load records ranges at checkpoints, after handing the pinned
// backend every node of the recorded ledgers, but not every backend can
// make those writes durable on demand (NuDB commits on its own thread).
// A crash can therefore leave the tail of an interval recorded but not
// on disk. Each ledger's nodes reach the backend before its header, and
// ledgers in sequence, so what survives is a prefix: trim the upper bound
// back to the last ledger whose header is present instead of refusing to
// start. A missing lower bound still means the configured pinned store
// is not the one these ranges were loaded into.
constexpr std::uint32_t maxTailTrim = 65536;
RangeSet<std::uint32_t> verified;
for (auto const& interval : persistedRanges)
{
if (!hasLedgerData(interval.lower()) ||
!hasLedgerData(interval.upper()))
{
Throw<std::runtime_error>(
"Persisted pinned interval " +
std::to_string(interval.lower()) + "-" +
auto const lower = interval.lower();
auto upper = interval.upper();
auto const notPresent = [&]() {
return std::runtime_error(
"Persisted pinned interval " + std::to_string(lower) + "-" +
std::to_string(interval.upper()) +
" is not present in the currently configured pinned store");
};
if (!hasLedgerData(lower))
Throw<std::runtime_error>(notPresent());
std::uint32_t trimmed = 0;
while (upper > lower && !hasLedgerData(upper))
{
if (++trimmed > maxTailTrim)
Throw<std::runtime_error>(notPresent());
--upper;
}
if (trimmed)
{
JLOG(journal_.warn())
<< "Pinned interval " << lower << "-" << interval.upper()
<< " was recorded but its last " << trimmed
<< " ledger(s) are not in the pinned store (likely an "
"interrupted catalogue_load); trimmed to "
<< lower << "-" << upper
<< ". Re-run catalogue_load to restore them.";
}
verified.insert(range(lower, upper));
}
if (verified != persistedRanges)
state_db_.setPinnedRanges(to_string(verified));
JLOG(journal_.info()) << "Loaded pinned ranges from database: "
<< rangesStr;
app_.getLedgerMaster().setPinnedLedgersRangeSet(persistedRanges);
<< to_string(verified);
app_.getLedgerMaster().setPinnedLedgersRangeSet(verified);
}
void
+5 -3
View File
@@ -184,9 +184,11 @@ public:
void
setPinnedRanges(RangeSet<std::uint32_t> const& ranges) override
{
// Save pinned ranges to database for persistence
std::string rangesStr = to_string(ranges);
state_db_.setPinnedRanges(rangesStr);
// Save pinned ranges to database for persistence. to_string()
// renders an empty set as "empty", which from_string() rejects, so
// store "" instead: loadPinnedRanges() treats that as nothing pinned.
state_db_.setPinnedRanges(
ranges.empty() ? std::string{} : to_string(ranges));
}
void
+78 -6
View File
@@ -302,6 +302,15 @@ struct AmendmentState
/** Indicates an amendment that this server has code support for. */
bool supported = false;
/** Indicates an amendment listed in [ignore_amendments]. The server has
no code for it, but must not become amendment blocked if it is (or
becomes) enabled on the ledger. Such an amendment is reported as
`supported` internally so every blocking check (LedgerMaster,
NetworkOPs, Change) passes, but is never voted for and is shown as
unsupported + ignored over RPC.
*/
bool ignored = false;
/** The name of this amendment, possibly empty. */
std::string name;
@@ -476,7 +485,8 @@ public:
std::vector<FeatureInfo> const& supported,
Section const& enabled,
Section const& vetoed,
beast::Journal journal);
beast::Journal journal,
Section const& ignored);
uint256
find(std::string const& name) const override;
@@ -543,7 +553,8 @@ AmendmentTableImpl::AmendmentTableImpl(
std::vector<FeatureInfo> const& supported,
Section const& enabled,
Section const& vetoed,
beast::Journal journal)
beast::Journal journal,
Section const& ignored)
: lastUpdateSeq_(0)
, majorityTime_(majorityTime)
, unsupportedEnabled_(false)
@@ -586,6 +597,59 @@ AmendmentTableImpl::AmendmentTableImpl(
<< " voted by default if not enabled on the ledger.";
}
// Parse ignored amendments from config. Each line is either a registered
// feature name (e.g. "HookOnV2") or "<64 hex amendment ID> [name]". These
// are deliberately NOT persisted to wallet.db: removing the line from the
// config must restore normal amendment-blocking behaviour.
for (auto const& line : ignored.lines())
{
std::string const trimmed = boost::algorithm::trim_copy(line);
if (trimmed.empty())
continue;
std::vector<std::string> parts;
boost::algorithm::split(
parts,
trimmed,
boost::algorithm::is_space(),
boost::algorithm::token_compress_on);
uint256 id;
std::string name;
if (id.parseHex(parts[0]))
name = parts.size() > 1 ? parts[1] : std::string{};
else if (auto const f = getRegisteredFeature(parts[0]))
{
id = *f;
name = parts[0];
}
else
Throw<std::runtime_error>(
"Invalid entry '" + line + "' in [" + ignored.name() +
"]: expected a registered feature name or amendment ID");
AmendmentState& s = add(id, lock);
if (s.supported)
{
JLOG(j_.warn()) << "[" << ignored.name() << "] lists amendment "
<< id << " (" << s.name
<< ") which this server supports; entry has no "
"effect.";
continue;
}
if (!name.empty())
s.name = name;
s.supported = true;
s.ignored = true;
s.vote = AmendmentVote::obsolete;
JLOG(j_.warn()) << "Amendment " << id << " (" << s.name
<< ") is unsupported but listed in [" << ignored.name()
<< "]: this server will NOT become amendment blocked "
"if it is enabled on the ledger.";
}
hash_set<uint256> detect_conflict;
// Parse enabled amendments from config
for (std::pair<uint256, std::string> const& a : parseSection(enabled))
@@ -775,7 +839,12 @@ AmendmentTableImpl::enable(uint256 const& amendment)
s.enabled = true;
if (!s.supported)
if (s.ignored)
{
JLOG(j_.warn()) << "Ignored unsupported amendment " << amendment << " ("
<< s.name << ") is enabled on the ledger.";
}
else if (!s.supported)
{
JLOG(j_.error()) << "Unsupported amendment " << amendment
<< " activated.";
@@ -1044,7 +1113,9 @@ AmendmentTableImpl::injectJson(
if (!fs.name.empty())
v[jss::name] = fs.name;
v[jss::supported] = fs.supported;
v[jss::supported] = fs.supported && !fs.ignored;
if (fs.ignored)
v["ignored"] = true;
if (!fs.enabled && isAdmin)
{
if (fs.vote == AmendmentVote::obsolete)
@@ -1113,10 +1184,11 @@ make_AmendmentTable(
std::vector<AmendmentTable::FeatureInfo> const& supported,
Section const& enabled,
Section const& vetoed,
beast::Journal journal)
beast::Journal journal,
Section const& ignored)
{
return std::make_unique<AmendmentTableImpl>(
app, majorityTime, supported, enabled, vetoed, journal);
app, majorityTime, supported, enabled, vetoed, journal, ignored);
}
} // namespace ripple
+10 -9
View File
@@ -106,17 +106,18 @@ TxQ::FeeMetrics::update(
TxQ::Setup const& setup)
{
std::vector<FeeLevel64> feeLevels;
auto const txBegin = view.txs.begin();
auto const txEnd = view.txs.end();
auto const size = std::distance(txBegin, txEnd);
feeLevels.reserve(size);
std::for_each(txBegin, txEnd, [&](auto const& tx) {
for (auto const& tx : view.txs)
{
// featureRNG: every proposing validator puts a zero-fee ttENTROPY
// into every ledger. These are consensus plumbing, not demand for
// ledger space, so they must not raise the expected transaction count
// or drag down the median fee level used for escalation.
if (tx.first->getTxnType() == ttENTROPY)
continue;
feeLevels.push_back(getFeeLevelPaid(view, *tx.first));
});
}
std::sort(feeLevels.begin(), feeLevels.end());
XRPL_ASSERT(
size == feeLevels.size(),
"ripple::TxQ::FeeMetrics::update : fee levels size");
std::size_t const size = feeLevels.size();
JLOG((timeLeap ? j_.warn() : j_.debug()))
<< "Ledger " << view.info().seq << " has " << size << " transactions. "
+28 -20
View File
@@ -258,42 +258,50 @@ saveValidatedLedger(
ledger->info().txHash == ledger->txMap().getHash().as_uint256(),
"ripple::detail::saveValidatedLedger : transaction hash match");
bool const pinned = app.getLedgerMaster().isPinned(seq);
// Save the ledger header in the hashed object store
{
Serializer s(128);
s.add32(HashPrefix::ledgerMaster);
addRaw(ledger->info(), s);
// Use pinnedLEDGER type for pinned ledgers, hotLEDGER for others
auto ledgerType =
app.getLedgerMaster().isPinned(seq) ? pinnedLEDGER : hotLEDGER;
auto ledgerType = pinned ? pinnedLEDGER : hotLEDGER;
app.getNodeStore().store(
ledgerType, std::move(s.modData()), ledger->info().hash, seq);
}
// Pinned (catalogue) ledgers are saved in bulk: they are never cached
// or published from here, so their AcceptedLedger skips the per-tx JSON
// and is not built at all when there are no tx tables to fill.
std::shared_ptr<AcceptedLedger> aLedger;
try
if (!pinned || app.config().useTxTables())
{
aLedger = app.getAcceptedLedgerCache().fetch(ledger->info().hash);
if (!aLedger)
try
{
aLedger = std::make_shared<AcceptedLedger>(ledger, app);
// Only cache if the ledger is NOT in the pinned range
if (!app.getLedgerMaster().isPinned(ledger->info().seq))
aLedger = app.getAcceptedLedgerCache().fetch(ledger->info().hash);
if (!aLedger)
{
app.getAcceptedLedgerCache().canonicalize_replace_client(
ledger->info().hash, aLedger);
aLedger = std::make_shared<AcceptedLedger>(
ledger, app, /*withJson=*/!pinned);
// Only cache if the ledger is NOT in the pinned range
if (!pinned)
{
app.getAcceptedLedgerCache().canonicalize_replace_client(
ledger->info().hash, aLedger);
}
}
}
}
catch (std::exception const&)
{
JLOG(j.warn()) << "An accepted ledger was missing nodes";
app.getLedgerMaster().failedSave(seq, ledger->info().hash);
// Clients can now trust the database for information about this
// ledger sequence.
app.pendingSaves().finishWork(seq);
return false;
catch (std::exception const&)
{
JLOG(j.warn()) << "An accepted ledger was missing nodes";
app.getLedgerMaster().failedSave(seq, ledger->info().hash);
// Clients can now trust the database for information about this
// ledger sequence.
app.pendingSaves().finishWork(seq);
return false;
}
}
{
+24
View File
@@ -50,7 +50,31 @@ ApplyContext::ApplyContext(
void
ApplyContext::discard()
{
// featureRNG: Hook draws are never undone. If this transaction drew from
// the RNG, its advanced state survives the discard (as does the record of
// what its Hooks were given), so a rollback cannot preserve an RNG state
// for whatever executes next.
std::optional<uint256> const randomData = view_->getRandomData();
std::optional<uint256> rngState;
if (randomData)
{
if (auto const sle = view_->read(keylet::random()))
rngState = sle->getFieldH256(sfRandomData);
}
view_.emplace(&base_, flags_);
if (randomData)
view_->setRandomData(*randomData);
if (rngState)
{
if (auto sle = view_->peek(keylet::random()))
{
sle->setFieldH256(sfRandomData, *rngState);
view_->update(sle);
}
}
}
std::optional<TxMeta>
+170
View File
@@ -34,6 +34,8 @@
#include <xrpl/protocol/Feature.h>
#include <xrpl/protocol/Indexes.h>
#include <xrpl/protocol/TxFlags.h>
#include <xrpl/protocol/digest.h>
#include <algorithm>
#include <string_view>
namespace ripple {
@@ -98,6 +100,21 @@ Change::preflight(PreflightContext const& ctx)
}
}
if (ctx.tx.getTxnType() == ttENTROPY)
{
if (!ctx.rules.enabled(featureRNG))
{
JLOG(ctx.j.warn()) << "Change: RNG is not enabled.";
return temDISABLED;
}
if (!publicKeyType(makeSlice(ctx.tx.getFieldVL(sfPublicKey))))
{
JLOG(ctx.j.warn()) << "Change: Entropy has an invalid PublicKey.";
return temMALFORMED;
}
}
return tesSUCCESS;
}
@@ -157,6 +174,29 @@ Change::preclaim(PreclaimContext const& ctx)
case ttUNL_MODIFY:
case ttEMIT_FAILURE:
return tesSUCCESS;
case ttENTROPY: {
if (ctx.tx.getFieldU32(sfLedgerSequence) != ctx.view.seq())
{
JLOG(ctx.j.warn()) << "Change: Entropy for wrong ledger.";
return tefFAILURE;
}
// Only validators on the UNLReport may contribute, when there is
// one. Without a UNLReport (new networks, standalone, unit tests)
// inclusion by consensus is the only gate.
auto const unlRep = ctx.view.read(keylet::UNLReport());
if (!unlRep || !unlRep->isFieldPresent(sfActiveValidators))
return tesSUCCESS;
auto const pk = ctx.tx.getFieldVL(sfPublicKey);
for (auto const& av : unlRep->getFieldArray(sfActiveValidators))
if (av.getFieldVL(sfPublicKey) == pk)
return tesSUCCESS;
JLOG(ctx.j.warn()) << "Change: Entropy from a validator that is "
"not on the UNLReport.";
return tefFAILURE;
}
case ttUNL_REPORT: {
if (!ctx.tx.isFieldPresent(sfImportVLKey) ||
ctx.app.config().IMPORT_VL_KEYS.empty())
@@ -211,12 +251,142 @@ Change::doApply()
return applyEmitFailure();
case ttUNL_REPORT:
return applyUNLReport();
case ttENTROPY:
return applyEntropy();
default:
UNREACHABLE("ripple::Change::doApply : invalid transaction type");
return tefFAILURE;
}
}
// featureRNG
//
// The Random object holds one commitment per validator: the hash of the
// secret that validator will reveal next. A ttENTROPY reveals the secret for
// the stored commitment (if it has one) and replaces the commitment with a new
// one. A reveal that matches is mixed into sfRandomData and counted. Anything
// else (no commitment yet, a lost secret, a mismatch) just re-registers, which
// is no different from the validator having withheld.
//
// The first ttENTROPY of each ledger restarts sfRandomData. All of them are
// applied before any other transaction (see BuildLedger), so by the time Hooks
// run, sfRandomData is the seed for this ledger. The Hook API then iterates it
// on every draw.
TER
Change::applyEntropy()
{
// Commitments not refreshed within this many ledgers are forgotten.
static constexpr std::uint32_t maxCommitmentAge = 8;
// domain separators for the two hashes computed here
static constexpr std::uint32_t rngSeedTag = 0x524E4753; // 'RNGS'
static constexpr std::uint32_t rngAccumulateTag = 0x524E4741; // 'RNGA'
auto const seq = view().seq();
auto const& tx = ctx_.tx;
auto const pk = tx.getFieldVL(sfPublicKey);
auto sle = view().peek(keylet::random());
bool const created = !sle;
// find this validator's commitment, if it has one
std::optional<STObject> commitment;
if (!created)
{
for (auto const& entry : sle->getFieldArray(sfRandomDigests))
{
if (entry.getFieldVL(sfPublicKey) == pk)
{
commitment = entry;
break;
}
}
}
// one contribution per validator per ledger
if (commitment && commitment->getFieldU32(sfLedgerSequence) == seq)
{
JLOG(j_.warn()) << "Change: second Entropy from one validator in "
<< "ledger " << seq;
return tefFAILURE;
}
if (created)
sle = std::make_shared<SLE>(keylet::random());
// the first ttENTROPY applied in this ledger restarts the accumulator
if (created || sle->getFieldU32(sfLedgerSequence) != seq)
{
std::size_t denominator = 0;
if (auto const unlRep = view().read(keylet::UNLReport());
unlRep && unlRep->isFieldPresent(sfActiveValidators))
denominator = unlRep->getFieldArray(sfActiveValidators).size();
sle->setFieldU32(sfLedgerSequence, seq);
sle->setFieldU16(sfEntropyCount, 0);
sle->setFieldU16(
sfEntropyDenominator,
static_cast<std::uint16_t>(
std::min<std::size_t>(denominator, 0xFFFFU)));
sle->setFieldH256(
sfRandomData,
sha512Half(rngSeedTag, seq, view().info().parentHash));
}
// mix in the reveal if it matches the validator's commitment
if (commitment && tx.isFieldPresent(sfRandomData))
{
auto const reveal = tx.getFieldH256(sfRandomData);
if (sha512Half(reveal) == commitment->getFieldH256(sfNextRandomDigest))
{
sle->setFieldH256(
sfRandomData,
sha512Half(
rngAccumulateTag,
sle->getFieldH256(sfRandomData),
makeSlice(pk),
reveal));
sle->setFieldU16(
sfEntropyCount, sle->getFieldU16(sfEntropyCount) + 1);
}
else
{
JLOG(j_.warn()) << "Change: Entropy reveal does not match the "
<< "validator's commitment, ignoring it.";
}
}
// store the new commitment, dropping stale ones
STArray digests(sfRandomDigests);
for (auto const& entry : sle->getFieldArray(sfRandomDigests))
{
if (entry.getFieldVL(sfPublicKey) == pk)
continue;
if (entry.getFieldU32(sfLedgerSequence) + maxCommitmentAge < seq)
continue;
digests.push_back(entry);
}
STObject entry(sfRandomDigestEntry);
entry.setFieldVL(sfPublicKey, pk);
entry.setFieldH256(sfNextRandomDigest, tx.getFieldH256(sfNextRandomDigest));
entry.setFieldU32(sfLedgerSequence, seq);
digests.push_back(std::move(entry));
digests.sort([](STObject const& a, STObject const& b) {
return a.getFieldVL(sfPublicKey) < b.getFieldVL(sfPublicKey);
});
sle->setFieldArray(sfRandomDigests, digests);
if (created)
view().insert(sle);
else
view().update(sle);
return tesSUCCESS;
}
TER
Change::applyUNLReport()
{
+4
View File
@@ -76,6 +76,9 @@ private:
TER
applyUNLReport();
TER
applyEntropy();
};
using EnableAmendment = Change;
@@ -83,6 +86,7 @@ using SetFee = Change;
using UNLModify = Change;
using EmitFailure = Change;
using UNLReport = Change;
using Entropy = Change;
} // namespace ripple
+3 -2
View File
@@ -158,8 +158,9 @@ ClaimReward::preclaim(PreclaimContext const& ctx)
auto const& hookOn =
hook::getHookOn(hook, sleDef, sfHookOnIncoming);
auto const hookName = ctx.view.rules().enabled(fix20260929)
? hook[~sfHookName]
auto const hookName = ctx.view.rules().enabled(fix20260929) &&
hook.isFieldPresent(sfHookName)
? std::optional<Blob>(hook.getFieldVL(sfHookName))
: std::nullopt;
if (hook::canHook(ctx.tx, hookOn, hookName))
{
@@ -604,6 +604,7 @@ LedgerEntryTypesMatch::visitEntry(
case ltCRON:
case ltIMPORT_VLSEQ:
case ltUNL_REPORT:
case ltRANDOM:
case ltAMM:
case ltBRIDGE:
case ltXCHAIN_OWNED_CLAIM_ID:
+9 -1
View File
@@ -1740,7 +1740,15 @@ SetHook::setHook()
{
if (newHookName->size() == 0)
{
newHook.makeFieldAbsent(sfHookName);
// Erase the field outright rather than
// makeFieldAbsent: newHook carries no template, so
// makeFieldAbsent leaves an STI_NOTPRESENT
// placeholder in the in-memory entry that every
// reader must then special-case, and it throws
// (-> tefEXCEPTION) when the hook has no name to
// remove. delField matches the serialized form of
// the entry and is a no-op on a nameless hook.
newHook.delField(sfHookName);
}
else
{
+77 -12
View File
@@ -287,7 +287,9 @@ Transactor::calculateHookChainFee(
// LCOV_EXCL_STOP
}
auto const hookName = hookObj[~sfHookName];
std::optional<Blob> const hookName = hookObj.isFieldPresent(sfHookName)
? std::optional<Blob>(hookObj.getFieldVL(sfHookName))
: std::nullopt;
uint32_t flags = 0;
if (hookObj.isFieldPresent(sfFlags))
@@ -1409,7 +1411,9 @@ Transactor::executeHookChain(
// LCOV_EXCL_STOP
}
auto const hookName = hookObj[~sfHookName];
std::optional<Blob> const hookName = hookObj.isFieldPresent(sfHookName)
? std::optional<Blob>(hookObj.getFieldVL(sfHookName))
: std::nullopt;
// check if the hook can fire
uint256 hookOn = hook::getHookOn(
@@ -1541,6 +1545,21 @@ Transactor::doHookCallback(
if (!emitDetails.isFieldPresent(sfEmitCallback))
return;
bool const isEmitFailure = ctx_.tx.getTxnType() == ttEMIT_FAILURE;
// fix20261005: an emit failure callback is only meaningful with the
// original emitted transaction. If it wasn't in the ledger when this
// pseudo was applied there is nothing to report on, so skip rather than
// read the (already removed) entry.
if (isEmitFailure && view().rules().enabled(fix20261005) &&
!emitFailureTxn_)
{
JLOG(j_.warn()) << "HookError["
<< ctx_.tx.getFieldH256(sfTransactionHash)
<< "]: emitted txn missing on emit failure callback";
return;
}
AccountID const& callbackAccountID =
emitDetails.getAccountID(sfEmitCallback);
uint256 const& callbackHookHash = emitDetails.getFieldH256(sfEmitHookHash);
@@ -1624,9 +1643,10 @@ Transactor::doHookCallback(
true,
true,
false,
ctx_.tx.getTxnType() == ttEMIT_FAILURE ? 1UL : 0UL,
isEmitFailure ? 1UL : 0UL,
hook_no - 1,
provisionalMeta);
provisionalMeta,
emitFailureTxn_);
executedHookCount_++;
@@ -2089,6 +2109,24 @@ Transactor::operator()()
}
}
// fix20261005: a ttEMIT_FAILURE erases the expired ltEMITTED_TXN entry in
// doApply, but the Hook callback that runs afterwards (finishApply ->
// doHookCallback) needs the original emitted transaction. Capture it
// before base application mutates the view.
if (hooksEnabled && isTesSuccess(result) &&
ctx_.tx.getTxnType() == ttEMIT_FAILURE &&
view().rules().enabled(fix20261005))
{
if (auto const sle = view().read(
keylet::emittedTxn(ctx_.tx.getFieldH256(sfTransactionHash)));
sle && sle->isFieldPresent(sfEmittedTxn))
{
if (auto const inner = dynamic_cast<STObject const*>(
sle->peekAtPField(sfEmittedTxn)))
emitFailureTxn_.emplace(*inner);
}
}
// fall through allows normal apply
if (isTesSuccess(result))
result = apply();
@@ -2119,6 +2157,17 @@ Transactor::finishApply(
if (ctx_.size() > oversizeMetaDataCap)
result = tecOVERSIZE;
// featureRNG: a transaction whose Hooks drew from the RNG is never
// retried on a tec. Under tapRETRY (the first closed-ledger pass, and
// open-ledger retries) a tec would otherwise drop the view without going
// through reset(), so the RNG state would not be rolled forward: a Hook
// that rolled back (tecHOOK_REJECTED) would leave the state it drew from
// in place for the next transaction in the pass, and would draw again on
// the final pass against a different state. Draws only exist when the
// amendment is enabled, so this needs no separate gate.
bool const rngDrawn =
dynamic_cast<ApplyViewImpl&>(ctx_.view()).getRandomData().has_value();
if (isTecClaim(result) && (view().flags() & tapFAIL_HARD))
{
// If the tapFAIL_HARD flag is set, a tec result
@@ -2129,7 +2178,8 @@ Transactor::finishApply(
else if (
(result == tecOVERSIZE) || (result == tecKILLED) ||
(result == tecINCOMPLETE) || (result == tecEXPIRED) ||
(isTecClaimHardFail(result, view().flags())))
(isTecClaimHardFail(result, view().flags())) ||
(rngDrawn && isTecClaim(result)))
{
JLOG(j_.trace()) << "reapplying because of " << transToken(result);
@@ -2260,7 +2310,6 @@ Transactor::finishApply(
uint32_t lgrCur = view().seq();
bool const has240819 = view().rules().enabled(fix240819);
bool const has240911 = view().rules().enabled(fix240911);
bool const hasIOURewardClaim =
view().rules().enabled(featureIOURewardClaim);
auto const& sfRewardFields =
@@ -2380,10 +2429,12 @@ Transactor::finishApply(
if (!finalFields.isFieldPresent(sfBalance))
continue;
bool const has20261005 = view().rules().enabled(fix20261005);
uint64_t bal =
finalFields.getFieldAmount(sfBalance).xrp().drops() / 1'000'000;
if (bal == 0)
if (!has20261005 && bal == 0)
continue;
uint32_t lgrLast = sle->getFieldU32(sfRewardLgrLast);
@@ -2391,19 +2442,33 @@ Transactor::finishApply(
uint32_t lgrElapsed = lgrCur - lgrLast;
// overflow safety
if (!has240911 &&
(lgrElapsed > lgrCur || lgrElapsed > lgrLast ||
lgrElapsed == 0))
continue;
if (has240911 && (lgrElapsed > lgrCur || lgrElapsed == 0))
if (lgrElapsed > lgrCur || lgrElapsed == 0)
{
if (has20261005)
{
sle->setFieldU32(sfRewardLgrLast, lgrCur);
view().update(sle);
}
continue;
}
uint64_t accum = sle->getFieldU64(sfRewardAccumulator);
// When bal == 0 there is no accumulator contribution to add, but
// the marker must still advance (handled below) so the stale span
// cannot be reused. accumNew == accum in that case.
uint64_t accumNew = accum + bal * ((uint64_t)lgrElapsed);
// check for overflow
if (accumNew < accum)
{
if (has20261005)
{
sle->setFieldU32(sfRewardLgrLast, lgrCur);
view().update(sle);
}
continue;
}
sle->setFieldU64(sfRewardAccumulator, accumNew);
sle->setFieldU32(sfRewardLgrLast, lgrCur);
+5
View File
@@ -246,6 +246,11 @@ protected:
touched_; // fix20261005: accounts touched (featureTouch) in the
// current view. Never populated before the fix, emptied
// whenever the view is discarded.
std::optional<STObject>
emitFailureTxn_; // fix20261005: for ttEMIT_FAILURE, the original
// emitted transaction, captured before doApply
// erases its ltEMITTED_TXN entry. Consumed by the
// callback in doHookCallback.
// The post-apply pipeline of operator(): tec handling (reset),
// invariants, balance rewards, weak hooks and commit.
+15 -55
View File
@@ -18,61 +18,21 @@
//==============================================================================
#include <xrpld/app/tx/applySteps.h>
#include <xrpld/app/tx/detail/AMMBid.h>
#include <xrpld/app/tx/detail/AMMClawback.h>
#include <xrpld/app/tx/detail/AMMCreate.h>
#include <xrpld/app/tx/detail/AMMDelete.h>
#include <xrpld/app/tx/detail/AMMDeposit.h>
#include <xrpld/app/tx/detail/AMMVote.h>
#include <xrpld/app/tx/detail/AMMWithdraw.h>
#include <xrpld/app/tx/detail/ApplyContext.h>
#include <xrpld/app/tx/detail/CancelCheck.h>
#include <xrpld/app/tx/detail/CancelOffer.h>
#include <xrpld/app/tx/detail/CashCheck.h>
#include <xrpld/app/tx/detail/Change.h>
#include <xrpld/app/tx/detail/ClaimReward.h>
#include <xrpld/app/tx/detail/Clawback.h>
#include <xrpld/app/tx/detail/CreateCheck.h>
#include <xrpld/app/tx/detail/CreateOffer.h>
#include <xrpld/app/tx/detail/CreateTicket.h>
#include <xrpld/app/tx/detail/Credentials.h>
#include <xrpld/app/tx/detail/Cron.h>
#include <xrpld/app/tx/detail/CronSet.h>
#include <xrpld/app/tx/detail/DID.h>
#include <xrpld/app/tx/detail/DeleteAccount.h>
#include <xrpld/app/tx/detail/DeleteOracle.h>
#include <xrpld/app/tx/detail/DepositPreauth.h>
#include <xrpld/app/tx/detail/Escrow.h>
#include <xrpld/app/tx/detail/GenesisMint.h>
#include <xrpld/app/tx/detail/Import.h>
#include <xrpld/app/tx/detail/Invoke.h>
#include <xrpld/app/tx/detail/LedgerStateFix.h>
#include <xrpld/app/tx/detail/MPTokenAuthorize.h>
#include <xrpld/app/tx/detail/MPTokenIssuanceCreate.h>
#include <xrpld/app/tx/detail/MPTokenIssuanceDestroy.h>
#include <xrpld/app/tx/detail/MPTokenIssuanceSet.h>
#include <xrpld/app/tx/detail/NFTokenAcceptOffer.h>
#include <xrpld/app/tx/detail/NFTokenBurn.h>
#include <xrpld/app/tx/detail/NFTokenCancelOffer.h>
#include <xrpld/app/tx/detail/NFTokenCreateOffer.h>
#include <xrpld/app/tx/detail/NFTokenMint.h>
#include <xrpld/app/tx/detail/NFTokenModify.h>
#include <xrpld/app/tx/detail/PayChan.h>
#include <xrpld/app/tx/detail/Payment.h>
#include <xrpld/app/tx/detail/PermissionedDomainDelete.h>
#include <xrpld/app/tx/detail/PermissionedDomainSet.h>
#include <xrpld/app/tx/detail/Remit.h>
#include <xrpld/app/tx/detail/SetAccount.h>
#include <xrpld/app/tx/detail/SetHook.h>
#include <xrpld/app/tx/detail/SetManifest.h>
#include <xrpld/app/tx/detail/SetOracle.h>
#include <xrpld/app/tx/detail/SetRegularKey.h>
#include <xrpld/app/tx/detail/SetRemarks.h>
#include <xrpld/app/tx/detail/SetSignerList.h>
#include <xrpld/app/tx/detail/SetTrust.h>
#include <xrpld/app/tx/detail/URIToken.h>
#include <xrpld/app/tx/detail/XChainBridge.h>
#include <xrpld/app/tx/detail/XahauGenesis.h>
#pragma push_macro("TRANSACTION")
#undef TRANSACTION
// Do nothing
#define TRANSACTION(...)
#define TRANSACTION_INCLUDE 1
#include <xrpl/protocol/detail/transactions.macro>
#undef TRANSACTION
#pragma pop_macro("TRANSACTION")
// DO NOT INCLUDE TRANSACTOR HEADER FILES HERE.
// See the instructions at the top of transactions.macro instead.
#include <xrpl/protocol/TxFormats.h>
#include <stdexcept>
+176
View File
@@ -32,7 +32,9 @@
#include <boost/logic/tribool.hpp>
#include <chrono>
#include <concepts>
#include <deque>
#include <map>
#include <optional>
#include <sstream>
@@ -532,6 +534,50 @@ private:
void
updateOurPositions(std::unique_ptr<std::stringstream> const& clog);
/** Whether a disputed transaction must be in our position regardless of
its vote count.
Optional adaptor hook: an adaptor that provides
`std::optional<NodeID_t> forcedTxOwner(Tx_t const&, Ledger_t const&)`
can name a node that a transaction belongs to (featureRNG uses this
for each validator's entropy contribution). During the opening
window of the establish phase such a transaction is voted yes iff its
owner's own position contains it, but at most one per owner per round
(see forcedTxs_). Afterwards it is voted on like any other disputed
transaction, so an owner publishing late cannot force churn. Adaptors
without the hook are unaffected.
*/
bool
forcedInclusion(typename Result::Dispute_t const& dispute) const;
/** The owner the adaptor names for a disputed transaction while the
forced-inclusion window is open. Always unseated for an adaptor
without the forcedTxOwner hook.
*/
std::optional<NodeID_t>
forcedOwner(typename Result::Dispute_t const& dispute) const;
/** Add any forced-inclusion transactions to our position. */
void
mergeForcedTxs(std::unique_ptr<std::stringstream> const& clog);
/** The one transaction each owner may force into our position this
round, chosen by mergeForcedTxs the first time the owner's position
offers one. An owner whose position carries several cannot make us
carry them all. Cleared when we close.
*/
std::map<NodeID_t, typename Tx_t::ID> forcedTxs_;
/** forcedTxOwner results for this round, keyed by transaction ID.
The adaptor's answer depends only on the transaction and the previous
ledger, both fixed for the round, so it is computed once per disputed
transaction rather than on every tick (forcedOwner is reached several
times per dispute per tick, and the RCL adaptor deserialises the
transaction to answer). Cleared when we close.
*/
mutable std::map<typename Tx_t::ID, std::optional<NodeID_t>> forcedOwners_;
bool
haveConsensus(std::unique_ptr<std::stringstream> const& clog);
@@ -1368,6 +1414,8 @@ Consensus<Adaptor>::phaseEstablish(
<< "avMIN_CONSENSUS_TIME: " << parms.avMIN_CONSENSUS_TIME.count()
<< "ms. ";
mergeForcedTxs(clog);
// Give everyone a chance to take an initial position
if (result_->roundTime.read() < parms.ledgerMIN_CONSENSUS)
{
@@ -1418,6 +1466,8 @@ Consensus<Adaptor>::closeLedger(std::unique_ptr<std::stringstream> const& clog)
phase_ = ConsensusPhase::establish;
JLOG(j_.debug()) << "transitioned to ConsensusPhase::establish";
rawCloseTimes_.self = now_;
forcedTxs_.clear();
forcedOwners_.clear();
result_.emplace(adaptor_.onClose(previousLedger_, now_, mode_.get()));
result_->roundTime.reset(clock_.now());
@@ -1513,6 +1563,10 @@ Consensus<Adaptor>::updateOurPositions(
std::optional<typename TxSet_t::MutableTxSet> mutableSet;
for (auto& [txId, dispute] : result_->disputes)
{
// Already merged into our position by mergeForcedTxs.
if (forcedInclusion(dispute))
continue;
// Because the threshold for inclusion increases,
// time can change our position on a dispute
if (dispute.updateVote(
@@ -1659,6 +1713,128 @@ Consensus<Adaptor>::updateOurPositions(
}
}
template <class Adaptor>
std::optional<typename Consensus<Adaptor>::NodeID_t>
Consensus<Adaptor>::forcedOwner(typename Result::Dispute_t const& dispute) const
{
if constexpr (requires(Adaptor& a, Tx_t const& tx, Ledger_t const& l) {
{
a.forcedTxOwner(tx, l)
} -> std::convertible_to<std::optional<NodeID_t>>;
})
{
if (!result_)
return std::nullopt;
ConsensusParms const& parms = adaptor_.parms();
if (result_->roundTime.read() >=
parms.ledgerMIN_CONSENSUS + parms.ledgerGRANULARITY)
return std::nullopt;
auto const& id = dispute.tx().id();
auto it = forcedOwners_.find(id);
if (it == forcedOwners_.end())
it = forcedOwners_
.emplace(
id,
adaptor_.forcedTxOwner(dispute.tx(), previousLedger_))
.first;
return it->second;
}
else
{
return std::nullopt;
}
}
template <class Adaptor>
bool
Consensus<Adaptor>::forcedInclusion(
typename Result::Dispute_t const& dispute) const
{
std::optional<NodeID_t> const owner = forcedOwner(dispute);
if (!owner)
return false;
// our own contribution: keep it while the window is open
if (*owner == result_->position.nodeID())
return result_->txns.exists(dispute.tx().id());
// otherwise only the one transaction chosen for this owner this round
auto const it = forcedTxs_.find(*owner);
if (it == forcedTxs_.end() || it->second != dispute.tx().id())
return false;
return dispute.getVote(*owner).value_or(false);
}
template <class Adaptor>
void
Consensus<Adaptor>::mergeForcedTxs(
std::unique_ptr<std::stringstream> const& clog)
{
XRPL_ASSERT(result_, "ripple::Consensus::mergeForcedTxs : result is set");
// Choose the one transaction each owner may force this round: the lowest
// ID its position offers the first time it offers any. Nodes that see the
// same position make the same choice.
NodeID_t const& self = result_->position.nodeID();
std::map<NodeID_t, typename Tx_t::ID> chosen;
for (auto const& [txId, dispute] : result_->disputes)
{
std::optional<NodeID_t> const owner = forcedOwner(dispute);
if (!owner || *owner == self || forcedTxs_.count(*owner) != 0 ||
!dispute.getVote(*owner).value_or(false))
continue;
auto const [it, inserted] = chosen.emplace(*owner, txId);
if (!inserted && txId < it->second)
it->second = txId;
}
forcedTxs_.merge(chosen);
std::optional<typename TxSet_t::MutableTxSet> mutableSet;
for (auto& [txId, dispute] : result_->disputes)
{
if (dispute.getOurVote() || !forcedInclusion(dispute))
continue;
dispute.setOurVote(true);
if (!mutableSet)
mutableSet.emplace(result_->txns);
mutableSet->insert(dispute.tx());
}
if (!mutableSet)
return;
TxSet_t newSet{std::move(*mutableSet)};
auto const newID = newSet.id();
result_->txns = std::move(newSet);
JLOG(j_.info()) << "Position change: forced inclusion, tx " << newID;
CLOG(clog) << "Position change: forced inclusion, tx " << newID << ". ";
result_->position.changePosition(
newID, result_->position.closeTime(), now_);
if (acquired_.emplace(newID, result_->txns).second)
{
if (!result_->position.isBowOut())
adaptor_.share(result_->txns);
for (auto const& [nodeId, peerPos] : currPeerPositions_)
{
if (peerPos.proposal().position() == newID)
updateDisputes(nodeId, result_->txns);
}
}
if (!result_->position.isBowOut() &&
(mode_.get() == ConsensusMode::proposing))
adaptor_.propose(result_->position);
}
template <class Adaptor>
bool
Consensus<Adaptor>::haveConsensus(
+14
View File
@@ -29,6 +29,7 @@
#include <xrpl/protocol/UintTypes.h>
#include <boost/container/flat_map.hpp>
#include <memory>
#include <optional>
namespace ripple {
@@ -113,6 +114,19 @@ public:
void
unVote(NodeID_t const& peer);
/** A peer's vote, if we have one.
@param peer Identifier of peer.
*/
std::optional<bool>
getVote(NodeID_t const& peer) const
{
auto const it = votes_.find(peer);
if (it == votes_.end())
return std::nullopt;
return it->second;
}
/** Update our vote given progression of consensus.
Updates our vote on this disputed transaction based on our peers' votes
+5
View File
@@ -197,6 +197,11 @@ public:
// True to ask peers not to relay current IP.
bool PEER_PRIVATE = false;
// True to enable the Xahau UDP Superhighway (XUSH): a best-effort UDP
// transport on the peer port used to fan transactions out to other
// highway-enabled servers in addition to the normal peer relay.
bool UDP_HIGHWAY = false;
// peers_max is a legacy configuration, which is going to be replaced
// with individual inbound peers peers_in_max and outbound peers
// peers_out_max configuration. for now we support both the legacy and
+4
View File
@@ -72,6 +72,7 @@ struct ConfigSection
#define SECTION_PATH_SEARCH_FAST "path_search_fast"
#define SECTION_PATH_SEARCH_MAX "path_search_max"
#define SECTION_PEER_PRIVATE "peer_private"
#define SECTION_UDP_HIGHWAY "udp_highway"
#define SECTION_PEERS_MAX "peers_max"
#define SECTION_PEERS_IN_MAX "peers_in_max"
#define SECTION_PEERS_OUT_MAX "peers_out_max"
@@ -101,6 +102,9 @@ struct ConfigSection
#define SECTION_VALIDATORS "validators"
#define SECTION_VALIDATOR_TOKEN "validator_token"
#define SECTION_VETO_AMENDMENTS "veto_amendments"
// Testnet-only: amendments enabled on-ledger that this node should not treat
// as unsupported (i.e. should not become amendment blocked over).
#define SECTION_IGNORE_AMENDMENTS "ignore_amendments"
#define SECTION_WORKERS "workers"
} // namespace ripple
+8 -1
View File
@@ -405,7 +405,11 @@ Config::setup(
HTTPClient::initializeSSLContext(*this, j_);
if (RUN_STANDALONE)
// Standalone never fetches history, so ledger_history is normally
// meaningless there. The rwdb null node store (nullBackend) uses it
// to size the in-memory retained-ledger window, and SHAMapStoreImp
// rejects zero for it, so keep the configured value in that case.
if (RUN_STANDALONE && !nullBackend())
LEDGER_HISTORY = 0;
std::string ledgerTxDbType;
@@ -533,6 +537,9 @@ Config::loadFromString(std::string const& fileContents)
if (getSingleSection(secConfig, SECTION_PEER_PRIVATE, strTemp, j_))
PEER_PRIVATE = beast::lexicalCastThrow<bool>(strTemp);
if (getSingleSection(secConfig, SECTION_UDP_HIGHWAY, strTemp, j_))
UDP_HIGHWAY = beast::lexicalCastThrow<bool>(strTemp);
if (getSingleSection(secConfig, SECTION_PEERS_MAX, strTemp, j_))
{
PEERS_MAX = beast::lexicalCastThrow<std::size_t>(strTemp);
+16
View File
@@ -128,6 +128,21 @@ public:
return hookExecution_.size();
}
/** featureRNG: record the RNG state seen by this transaction's first Hook
draw. Later calls are ignored. Written only to the final metadata. */
void
setRandomData(uint256 const& randomData)
{
if (!randomData_)
randomData_ = randomData;
}
std::optional<uint256> const&
getRandomData() const
{
return randomData_;
}
/** Get the number of modified entries
*/
std::size_t
@@ -148,6 +163,7 @@ private:
std::optional<STAmount> deliver_;
std::vector<STObject> hookExecution_;
std::vector<STObject> hookEmission_;
std::optional<uint256> randomData_;
};
} // namespace ripple
@@ -157,6 +157,12 @@ ApplyStateTable::generateTxMeta(
std::uint16_t nodeType = curNode
? curNode->getFieldU16(sfLedgerEntryType)
: origNode->getFieldU16(sfLedgerEntryType);
// featureRNG: the RNG object is never in provisional metadata, which
// weak Hooks can read through meta_slot.
if (isProvisional && nodeType == ltRANDOM)
continue;
meta.setAffectedNode(item.first, *type, nodeType);
if (type == &sfDeletedNode)
{
@@ -302,6 +308,7 @@ ApplyStateTable::apply(
std::optional<STAmount> const& deliver,
std::vector<STObject> const& hookExecution,
std::vector<STObject> const& hookEmission,
std::optional<uint256> const& randomData,
bool isDryRun,
beast::Journal j)
{
@@ -316,6 +323,11 @@ ApplyStateTable::apply(
auto [meta, newMod] =
generateTxMeta(to, tx, deliver, hookExecution, hookEmission, j);
// featureRNG: added only here, never to provisional metadata, so a
// weak Hook reading meta_slot cannot learn the RNG state.
if (randomData)
meta.setRandomData(*randomData);
if (!isDryRun)
{
// add any new modified nodes to the modification set
@@ -97,6 +97,7 @@ public:
std::optional<STAmount> const& deliver,
std::vector<STObject> const& hookExecution,
std::vector<STObject> const& hookEmission,
std::optional<uint256> const& randomData,
bool isDryRun,
beast::Journal j);
+9 -1
View File
@@ -37,7 +37,15 @@ ApplyViewImpl::apply(
beast::Journal j)
{
return items_.apply(
to, tx, ter, deliver_, hookExecution_, hookEmission_, isDryRun, j);
to,
tx,
ter,
deliver_,
hookExecution_,
hookEmission_,
randomData_,
isDryRun,
j);
}
TxMeta
+17
View File
@@ -851,6 +851,22 @@ private:
return jvRequest;
}
// catalogue_unpin <ledger_index_min> <ledger_index_max>
Json::Value
parseCatalogueUnpin(Json::Value const& jvParams)
{
Json::Value jvRequest(Json::objectValue);
if (jvParams.size() >= 2)
{
// asUInt() throws on non-numeric input rather than yielding 0
jvRequest[jss::ledger_index_min] = jvParams[0u].asUInt();
jvRequest[jss::ledger_index_max] = jvParams[1u].asUInt();
}
return jvRequest;
}
// catalogue_status - no parameters required
Json::Value
parseCatalogueStatus(Json::Value const& jvParams)
@@ -1470,6 +1486,7 @@ public:
{"catalogue_create", &RPCParser::parseCatalogueCreate, 3, 4},
{"catalogue_load", &RPCParser::parseCatalogueLoad, 1, 2},
{"catalogue_status", &RPCParser::parseCatalogueStatus, 0, 0},
{"catalogue_unpin", &RPCParser::parseCatalogueUnpin, 2, 2},
{"channel_authorize", &RPCParser::parseChannelAuthorize, 3, 4},
{"channel_verify", &RPCParser::parseChannelVerify, 4, 4},
{"connect", &RPCParser::parseConnect, 1, 2},
+16
View File
@@ -121,6 +121,22 @@ public:
virtual void
sync() = 0;
/** Make every object previously passed to store() or storeBatch()
durable before returning.
@return true if, on return, those writes are as durable as this
backend ever makes them (trivially so for in-memory
backends); false if the backend cannot do this on demand and
they become durable later on its own schedule (e.g. NuDB's
commit thread).
*/
virtual bool
syncDurable()
{
sync();
return false;
}
/** Visit every object in the database
This is usually called during import.
@note This routine will not be called concurrently with itself
@@ -186,6 +186,13 @@ public:
{
}
bool
syncDurable() override
{
// Nothing here outlives the process, so there is nothing to wait for.
return true;
}
void
for_each(std::function<void(std::shared_ptr<NodeObject>)> f) override
{
@@ -291,6 +291,9 @@ public:
void
sync() override
{
// NuDB commits on its context thread (about once a second) and
// offers no public barrier while that thread runs, so this cannot
// force durability; syncDurable() therefore reports false.
}
void
@@ -81,6 +81,13 @@ public:
{
}
bool
syncDurable() override
{
// Nothing here outlives the process, so there is nothing to wait for.
return true;
}
void
for_each(std::function<void(std::shared_ptr<NodeObject>)> f) override
{
@@ -164,6 +164,13 @@ public:
{
}
bool
syncDurable() override
{
// Nothing here outlives the process, so there is nothing to wait for.
return true;
}
void
for_each(std::function<void(std::shared_ptr<NodeObject>)> f) override
{
@@ -383,6 +383,33 @@ public:
void
sync() override
{
(void)syncDurable();
}
bool
syncDurable() override
{
// Drain anything queued through store() into RocksDB, then flush
// and fsync the WAL so every write so far survives a crash.
m_batch.waitForWriting();
if (!m_db)
return false;
auto const ret = m_db->FlushWAL(/*sync=*/true);
if (ret.ok())
return true;
// Some user-supplied options (e.g. allow_mmap_writes) make WAL sync
// unavailable. Report "not durable on demand" rather than failing:
// callers then fall back to the non-durable checkpoint path.
if (ret.IsNotSupported())
{
JLOG(m_journal.warn())
<< "RocksDB cannot sync its WAL on demand: " << ret.ToString();
return false;
}
Throw<std::runtime_error>("sync failed: " + ret.ToString());
}
void
+5 -2
View File
@@ -74,13 +74,16 @@ public:
int
getWriteLoad();
/** Block until every object passed to store() so far has been handed
to the callback's writeBatch(). */
void
waitForWriting();
private:
void
performScheduledTask() override;
void
writeBatch();
void
waitForWriting();
private:
using LockType = std::recursive_mutex;
@@ -91,6 +91,36 @@ DatabasePinnedImp::store(
}
}
void
DatabasePinnedImp::storePinnedBatch(Batch const& batch)
{
if (batch.empty())
return;
std::uint64_t bytes = 0;
for (auto const& obj : batch)
{
XRPL_ASSERT(
obj && !isPinnedType(obj->getType()),
"ripple::NodeStore::DatabasePinnedImp::storePinnedBatch : "
"serializable type");
bytes += obj->getData().size();
}
// The persistent backends allowed for pinned_type (NuDB, RocksDB, and
// rwdb/memory in standalone) are safe to write from several threads,
// so this may overlap a BatchWriter flush of single store() calls.
persistent_->storeBatch(batch);
pinnedStoreCount_ += batch.size();
storeStats(batch.size(), bytes);
}
bool
DatabasePinnedImp::syncPinned()
{
return persistent_->syncDurable();
}
std::shared_ptr<NodeObject>
DatabasePinnedImp::fetchNodeObject(
uint256 const& hash,
@@ -112,7 +142,11 @@ DatabasePinnedImp::fetchNodeObject(
{
// Try persistent backend first (pinned data)
if (auto obj = tryPersistent(hash, fetchReport))
{
if (duplicate)
copyToRotating(*obj, hash, ledgerSeq);
return obj;
}
// Fallback: try rotating backends
return rotating_.fetchNodeObject(
@@ -130,7 +164,30 @@ DatabasePinnedImp::fetchNodeObject(
return obj;
// Fallback: try persistent backend (pinned data)
return tryPersistent(hash, fetchReport);
auto obj = tryPersistent(hash, fetchReport);
// `duplicate` is set by online_delete's copy of the current validated
// state into the new writable backend. A hot ledger can share nodes
// that exist only in persistent storage (e.g. after catalogue_load
// switched the LCL to its last ledger); copy those too, so the hot
// store becomes self-contained after one rotation and pinned history
// can be unpinned and its store discarded without breaking live state.
// The pinned-first path above does the same, so the copy happens
// whichever backend is tried first.
if (obj && duplicate)
copyToRotating(*obj, hash, ledgerSeq);
return obj;
}
void
DatabasePinnedImp::copyToRotating(
NodeObject const& obj,
uint256 const& hash,
std::uint32_t ledgerSeq)
{
Blob data(obj.getData());
rotating_.store(obj.getType(), std::move(data), hash, ledgerSeq);
}
void
@@ -262,8 +319,8 @@ DatabasePinnedImp::tryPersistent(uint256 const& hash, FetchReport& fetchReport)
if (status == ok && nodeObject)
{
fetchReport.wasFound = true;
// Note: We do NOT copy pinned data to rotating storage even if
// duplicate=true. Pinned data stays in persistent storage.
// Copying into rotating storage for duplicate=true is handled by
// the caller (see fetchNodeObject's default path).
}
return nodeObject;
}
@@ -121,6 +121,20 @@ public:
std::uint32_t ledgerSeq) override;
void
sync() override;
/** Write node objects of pinned history straight to the persistent
backend in one batch. Objects must carry hot (serializable) types.
Used by catalogue_load, which builds whole batches per ledger.
*/
void
storePinnedBatch(Batch const& batch);
/** Make everything written to the persistent backend durable.
@return false if the backend cannot guarantee that on demand.
*/
bool
syncPinned();
bool
storeLedger(std::shared_ptr<Ledger const> const& srcLedger) override;
void
@@ -141,6 +155,14 @@ private:
bool
likelyPinned(std::uint32_t ledgerSeq) const;
/** Copy an object found only in persistent storage into the writable
rotating backend (the `duplicate` path of fetchNodeObject). */
void
copyToRotating(
NodeObject const& obj,
uint256 const& hash,
std::uint32_t ledgerSeq);
std::shared_ptr<NodeObject>
tryPersistent(uint256 const& hash, FetchReport& fetchReport);
+18
View File
@@ -235,6 +235,24 @@ public:
*/
virtual Json::Value
txMetrics() const = 0;
/** Process a datagram received on the UDP Superhighway (XUSH).
Datagrams arrive on the peer port. They are unauthenticated and are
accounted for against the sender's IP address.
*/
virtual void
processXUSH(
std::string const& message,
boost::asio::ip::tcp::endpoint const& remoteEndpoint) = 0;
/** Send a serialized transaction to UDP Superhighway (XUSH) peers.
Does nothing unless the highway is enabled. Callers must relay each
transaction at most once (see HashRouter::shouldRelay).
*/
virtual void
publishTxXUSH(Slice const& tx, uint256 const& txid) = 0;
};
} // namespace ripple
+389
View File
@@ -21,10 +21,12 @@
#include <xrpld/app/ledger/OpenLedger.h>
#include <xrpld/app/misc/HashRouter.h>
#include <xrpld/app/misc/NetworkOPs.h>
#include <xrpld/app/misc/Transaction.h>
#include <xrpld/app/misc/ValidatorList.h>
#include <xrpld/app/misc/ValidatorSite.h>
#include <xrpld/app/rdb/RelationalDatabase.h>
#include <xrpld/app/rdb/Wallet.h>
#include <xrpld/app/tx/apply.h>
#include <xrpld/overlay/Cluster.h>
#include <xrpld/overlay/detail/ConnectAttempt.h>
#include <xrpld/overlay/detail/PeerImp.h>
@@ -36,8 +38,10 @@
#include <xrpl/basics/make_SSLContext.h>
#include <xrpl/basics/random.h>
#include <xrpl/beast/core/LexicalCast.h>
#include <xrpl/beast/net/IPAddressConversion.h>
#include <xrpl/protocol/Feature.h>
#include <xrpl/protocol/STTx.h>
#include <xrpl/resource/Fees.h>
#include <xrpl/server/SimpleWriter.h>
#include <xrpld/core/ConfigSections.h>
@@ -107,6 +111,7 @@ OverlayImpl::Timer::on_timer(error_code ec)
overlay_.m_peerFinder->once_per_second();
overlay_.sendEndpoints();
overlay_.sendXUSHPeers();
overlay_.autoConnect();
if (overlay_.app_.config().TX_REDUCE_RELAY_ENABLE)
overlay_.sendTxQueue();
@@ -150,6 +155,8 @@ OverlayImpl::OverlayImpl(
, next_id_(1)
, timer_count_(0)
, slots_(app.logs(), *this)
, xushSocket4_(io_context_)
, xushSocket6_(io_context_)
, m_stats(
std::bind(&OverlayImpl::collect_metrics, this),
collector,
@@ -495,6 +502,21 @@ OverlayImpl::start()
m_peerFinder->setConfig(config);
m_peerFinder->start();
if (app_.config().UDP_HIGHWAY)
{
openXUSH();
// A private server only exchanges highway traffic with its fixed
// peers, so without any the highway is enabled but idle.
if (config.peerPrivate && app_.config().IPS_FIXED.empty())
{
JLOG(journal_.warn())
<< "XUSH: [udp_highway] is enabled, but this server is "
"private (peer_private or validator) and has no "
"[ips_fixed] peers, so no highway traffic will be sent";
}
}
auto addIps = [this](std::vector<std::string> ips, bool fixed) {
beast::Journal const& j = app_.journal("Overlay");
for (auto& ip : ips)
@@ -567,6 +589,7 @@ OverlayImpl::stop()
cond_.wait(lock, [this] { return list_.empty(); });
}
m_peerFinder->stop();
closeXUSH();
}
//------------------------------------------------------------------------------
@@ -1492,6 +1515,372 @@ OverlayImpl::deleteIdlePeers()
slots_.deleteIdlePeers();
}
//------------------------------------------------------------------------------
//
// XUSH (Xahau UDP Superhighway)
//
//------------------------------------------------------------------------------
namespace {
// Called from the JobQueue. Performs the expensive checks and then hands the
// transaction to NetworkOPs which, if it applies, relays it over both the peer
// protocol and the highway.
void
checkXUSHTransaction(
Application& app,
beast::Journal j,
std::shared_ptr<STTx const> const& stx,
Resource::Consumer usage)
{
auto const txID = stx->getTransactionID();
try
{
if (stx->isFieldPresent(sfLastLedgerSequence) &&
(stx->getFieldU32(sfLastLedgerSequence) <
app.getLedgerMaster().getValidLedgerIndex()))
{
app.getHashRouter().setFlags(txID, SF_BAD);
usage.charge(Resource::feeUselessData, "XUSH expired tx");
return;
}
// A manifest-authorized SetManifest's Fee follows the voted base fee,
// so a miss may be a fee setting this server has not reached yet.
// Every ingress path runs this before checkValidity(). Drop it, but
// do not mark it bad: the same bytes can become canonical here.
if (auto const [fee, feeReason] = checkManifestIngressFee(app, *stx);
fee == ManifestIngressFee::Refused)
{
JLOG(j.trace()) << "XUSH: dropping manifest-authorized tx " << txID
<< ": " << feeReason;
usage.charge(Resource::feeUselessData, "XUSH manifest fee miss");
return;
}
if (auto const [validity, reason] = checkValidity(
app.getHashRouter(),
*stx,
app.getLedgerMaster().getValidatedRules(),
app.config());
validity != Validity::Valid)
{
JLOG(j.trace())
<< "XUSH: transaction " << txID << " failed checks: " << reason;
app.getHashRouter().setFlags(txID, SF_BAD);
usage.charge(Resource::feeInvalidSignature, "XUSH tx signature");
return;
}
std::string why;
auto tx = std::make_shared<Transaction>(stx, why, app);
if (tx->getStatus() == INVALID)
{
JLOG(j.trace())
<< "XUSH: transaction " << txID << " is invalid: " << why;
app.getHashRouter().setFlags(txID, SF_BAD);
usage.charge(Resource::feeInvalidSignature, "XUSH tx (impossible)");
return;
}
app.getOPs().processTransaction(
tx, false, false, NetworkOPs::FailHard::no);
}
catch (std::exception const& ex)
{
JLOG(j.warn()) << "XUSH: exception checking transaction " << txID
<< ": " << ex.what();
app.getHashRouter().setFlags(txID, SF_BAD);
usage.charge(Resource::feeInvalidData, "XUSH tx exception");
}
}
} // namespace
void
OverlayImpl::processXUSH(
std::string const& message,
boost::asio::ip::tcp::endpoint const& remoteEndpoint)
{
if (!app_.config().UDP_HIGHWAY)
return;
auto const remote = beast::IPAddressConversion::from_asio(remoteEndpoint);
// Datagrams are unauthenticated, so they are accounted for against the
// sender's IP address exactly like an inbound peer connection.
auto usage = m_resourceManager.newInboundEndpoint(remote);
if (usage.disconnect(journal_))
return;
usage.charge(Resource::feeTrivialPeer, "XUSH datagram");
Slice const datagram(message.data(), message.size());
switch (xush::classify(datagram))
{
case xush::MessageType::peers:
onXUSHPeers(datagram, remote, usage);
break;
case xush::MessageType::txn:
onXUSHTxnFragment(datagram, remote, usage);
break;
case xush::MessageType::unknown:
// Ignored, so that new datagram types can be added later.
JLOG(journal_.trace()) << "XUSH: unknown datagram from " << remote;
break;
}
}
void
OverlayImpl::onXUSHPeers(
Slice datagram,
beast::IP::Endpoint const& remote,
Resource::Consumer& usage)
{
auto const endpoints = xush::decodePeers(datagram);
if (!endpoints)
{
usage.charge(Resource::feeMalformedRequest, "XUSHPEER malformed");
return;
}
JLOG(journal_.trace()) << "XUSHPEER from " << remote << " with "
<< endpoints->size() << " endpoints";
m_peerFinder->add_highway_peers(*endpoints);
}
void
OverlayImpl::onXUSHTxnFragment(
Slice datagram,
beast::IP::Endpoint const& remote,
Resource::Consumer& usage)
{
auto const fragment = xush::decodeTxnFragment(datagram);
if (!fragment)
{
usage.charge(Resource::feeMalformedRequest, "XUSHTXNF malformed");
return;
}
auto result = [&] {
std::lock_guard lock(xushMutex_);
return xushReassembler_.add(
remote, *fragment, xush::Reassembler::clock_type::now());
}();
switch (result.status)
{
case xush::Reassembler::Status::pending:
break;
case xush::Reassembler::Status::overloaded:
JLOG(journal_.debug())
<< "XUSHTXNF: reassembly limit reached, dropped fragment from "
<< remote;
break;
case xush::Reassembler::Status::invalid:
usage.charge(Resource::feeInvalidData, "XUSHTXNF inconsistent");
break;
case xush::Reassembler::Status::complete:
onXUSHTransaction(result.txn, fragment->txid, usage);
break;
}
}
void
OverlayImpl::onXUSHTransaction(
Buffer const& blob,
uint256 const& txid,
Resource::Consumer& usage)
{
// If we've never been in sync, there's nothing we can do with it
if (app_.getOPs().isNeedNetworkLedger())
return;
std::shared_ptr<STTx const> stx;
try
{
SerialIter sit(Slice(blob.data(), blob.size()));
stx = std::make_shared<STTx const>(sit);
}
catch (std::exception const& ex)
{
JLOG(journal_.debug()) << "XUSH: malformed transaction: " << ex.what();
usage.charge(Resource::feeInvalidData, "XUSH malformed tx");
return;
}
auto const txID = stx->getTransactionID();
// The ID in the header can't be used for anything (in particular, not to
// suppress a transaction) until it's known to match the content.
if (txID != txid)
{
usage.charge(Resource::feeInvalidData, "XUSH tx ID mismatch");
return;
}
// Emitted transactions are never relayed, and pseudo-transactions are
// never submitted by servers.
if (stx->isFieldPresent(sfEmitDetails) || isPseudoTx(*stx))
{
usage.charge(Resource::feeHeavyBurdenPeer, "XUSH emitted/pseudo tx");
return;
}
int flags;
if (!app_.getHashRouter().shouldProcess(
txID, 0, flags, std::chrono::seconds(10)))
{
// We have seen this transaction recently
if (flags & SF_BAD)
usage.charge(Resource::feeUselessData, "XUSH known bad tx");
return;
}
if (app_.getLedgerMaster().getValidatedLedgerAge() >
std::chrono::minutes(4))
{
JLOG(journal_.trace()) << "XUSH: no new transactions until synced";
return;
}
if (app_.getJobQueue().getJobCount(jtTRANSACTION) >
app_.config().MAX_TRANSACTIONS)
{
incJqTransOverflow();
JLOG(journal_.info()) << "XUSH: transaction queue is full";
return;
}
app_.getJobQueue().addJob(
jtTRANSACTION,
"XUSH->checkTransaction",
[&app = app_, j = journal_, stx, usage]() {
checkXUSHTransaction(app, j, stx, usage);
});
}
void
OverlayImpl::publishTxXUSH(Slice const& tx, uint256 const& txid)
{
if (!app_.config().UDP_HIGHWAY)
return;
auto const targets = m_peerFinder->highway_targets(xush::txnFanout);
if (targets.empty())
return;
auto const datagrams = xush::encodeTxn(tx, txid);
if (datagrams.empty())
{
JLOG(journal_.debug())
<< "XUSH: transaction " << txid << " is too large for the highway";
return;
}
sendXUSH(datagrams, targets);
}
void
OverlayImpl::sendXUSHPeers()
{
if (!app_.config().UDP_HIGHWAY)
return;
auto const adverts =
m_peerFinder->highway_adverts(xush::maxAdvertisedPeers);
if (adverts.empty())
return;
auto const targets = m_peerFinder->highway_targets(xush::advertFanout);
if (targets.empty())
return;
sendXUSH({xush::encodePeers(adverts)}, targets);
}
void
OverlayImpl::sendXUSH(
std::vector<Buffer> const& datagrams,
std::vector<beast::IP::Endpoint> const& targets)
{
std::lock_guard lock(xushSendMutex_);
for (auto const& target : targets)
{
auto& socket = target.address().is_v4() ? xushSocket4_ : xushSocket6_;
if (!socket.is_open())
continue;
boost::asio::ip::udp::endpoint const to(
target.address(), target.port());
for (auto const& datagram : datagrams)
{
boost::system::error_code ec;
socket.send_to(
boost::asio::buffer(datagram.data(), datagram.size()),
to,
0,
ec);
// The socket never blocks: if it can't take a datagram now, the
// rest are dropped too. The peer protocol still relays them.
if (ec)
{
JLOG(journal_.trace()) << "XUSH: send to " << target
<< " failed: " << ec.message();
break;
}
}
}
}
void
OverlayImpl::openXUSH()
{
using boost::asio::ip::udp;
std::lock_guard lock(xushSendMutex_);
auto open = [this](udp::socket& socket, udp const& protocol) {
boost::system::error_code ec;
socket.open(protocol, ec);
if (!ec && protocol == udp::v6())
socket.set_option(boost::asio::ip::v6_only(true), ec);
if (!ec)
socket.non_blocking(true, ec);
if (ec)
{
JLOG(journal_.warn()) << "XUSH: unable to open "
<< (protocol == udp::v4() ? "IPv4" : "IPv6")
<< " socket: " << ec.message();
boost::system::error_code ignored;
socket.close(ignored);
}
};
open(xushSocket4_, udp::v4());
open(xushSocket6_, udp::v6());
}
void
OverlayImpl::closeXUSH()
{
std::lock_guard lock(xushSendMutex_);
boost::system::error_code ignored;
xushSocket4_.close(ignored);
xushSocket6_.close(ignored);
}
//------------------------------------------------------------------------------
Overlay::Setup
+62
View File
@@ -28,6 +28,7 @@
#include <xrpld/overlay/detail/Handshake.h>
#include <xrpld/overlay/detail/TrafficCount.h>
#include <xrpld/overlay/detail/TxMetrics.h>
#include <xrpld/overlay/detail/XUSH.h>
#include <xrpld/peerfinder/PeerfinderManager.h>
#include <xrpld/rpc/ServerHandler.h>
#include <xrpl/basics/Resolver.h>
@@ -40,6 +41,7 @@
#include <boost/algorithm/string/predicate.hpp>
#include <boost/asio/basic_waitable_timer.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/ip/udp.hpp>
#include <boost/asio/ssl/context.hpp>
#include <boost/asio/strand.hpp>
#include <boost/container/flat_map.hpp>
@@ -135,6 +137,15 @@ private:
// Protects the message and the sequence list of manifests
std::mutex manifestLock_;
// XUSH (Xahau UDP Superhighway) transaction reassembly
std::mutex xushMutex_;
xush::Reassembler xushReassembler_;
// XUSH (Xahau UDP Superhighway) sockets used for sending
std::mutex xushSendMutex_;
boost::asio::ip::udp::socket xushSocket4_;
boost::asio::ip::udp::socket xushSocket6_;
//--------------------------------------------------------------------------
public:
@@ -442,6 +453,14 @@ public:
txMetrics_.addMetrics(args...);
}
void
processXUSH(
std::string const& message,
boost::asio::ip::tcp::endpoint const& remoteEndpoint) override;
void
publishTxXUSH(Slice const& tx, uint256 const& txid) override;
private:
void
squelch(
@@ -617,6 +636,49 @@ private:
}
m_stats.peerDisconnects = getPeerDisconnect();
}
//--------------------------------------------------------------------------
//
// XUSH (Xahau UDP Superhighway)
//
//--------------------------------------------------------------------------
/** Open the sockets used to send highway datagrams. */
void
openXUSH();
/** Close the sockets used to send highway datagrams. */
void
closeXUSH();
/** Advertise some highway endpoints to some highway peers. */
void
sendXUSHPeers();
/** Send every datagram to every target. Sends never block: datagrams
that can't be sent immediately are dropped. */
void
sendXUSH(
std::vector<Buffer> const& datagrams,
std::vector<beast::IP::Endpoint> const& targets);
void
onXUSHPeers(
Slice datagram,
beast::IP::Endpoint const& remote,
Resource::Consumer& usage);
void
onXUSHTxnFragment(
Slice datagram,
beast::IP::Endpoint const& remote,
Resource::Consumer& usage);
void
onXUSHTransaction(
Buffer const& blob,
uint256 const& txid,
Resource::Consumer& usage);
};
} // namespace ripple
+18 -2
View File
@@ -30,6 +30,7 @@
#include <boost/asio/buffers_iterator.hpp>
#include <boost/system/error_code.hpp>
#include <cstdint>
#include <limits>
#include <memory>
#include <optional>
#include <type_traits>
@@ -285,8 +286,23 @@ parseMessageContent(MessageHeader const& header, Buffers const& buffers)
if (payloadSize == 0 || !m->ParseFromArray(payload.data(), payloadSize))
return {};
}
else if (!m->ParseFromZeroCopyStream(&stream))
return {};
else
{
// The buffer sequence handed to us is the peer's whole read buffer,
// which may hold further frames after this one (several frames are
// routinely coalesced into a single read). Parse exactly the declared
// payload: an unbounded parse would run on into the next frame's
// header and payload, either rejecting a valid message or merging
// foreign bytes into it, and would disagree with the
// total_wire_size that the caller consumes.
//
// payload_wire_size is at most 26 bits wide for uncompressed frames
// and was already checked against maximiumMessageSize, so it fits.
static_assert(maximiumMessageSize <= std::numeric_limits<int>::max());
if (!m->ParseFromBoundedZeroCopyStream(
&stream, static_cast<int>(header.payload_wire_size)))
return {};
}
return m;
}
Loaded 100 of 117 files, more files were not shown because too many files have changed in this diff. Show more