Compare commits

..

30 Commits

Author SHA1 Message Date
Denis Angell
5fe5ef727c [fold] fix build error 2024-11-19 12:27:41 +01:00
RichardAH
d8515b5afe Merge branch 'dev' into sync-rippled 2024-11-19 14:58:51 +10:00
RichardAH
f6d789464e Merge branch 'dev' into sync-rippled 2024-11-12 08:57:53 +10:00
Denis Angell
c34ca594a4 Merge branch 'dev' into sync-rippled 2024-10-31 17:09:44 +01:00
Denis Angell
e47d6891cc [fold] fix bad merge
- add back filter for ripple state on account_channels
- add back network id test (env auto adds network id in xahau)
2024-10-31 13:18:40 +01:00
Denis Angell
bfe1463c37 [fold] bad merge 2024-10-31 11:20:11 +01:00
Richard Holland
8d04a1a434 clang 2024-10-25 12:59:46 +11:00
RichardAH
d688644727 Merge branch 'dev' into sync-rippled 2024-10-25 11:34:19 +10:00
Denis Angell
f0b6e57408 Merge branch 'dev' into sync-rippled 2024-10-16 10:21:16 +02:00
RichardAH
43ae851238 Merge branch 'dev' into sync-rippled 2024-03-25 08:46:34 +11:00
Denis Angell
eefc0f1150 Revert "Fix typo (#4508)"
This reverts commit 2956f14de8.
2024-03-18 12:22:46 +01:00
Denis Angell
87097576f4 Revert "Fix the fix for std::result_of (#4496)"
This reverts commit cee8409d60.
2024-03-18 12:22:13 +01:00
Chenna Keshava B S
0c73050e6f fix: remove redundant moves (#4565)
- Resolve gcc compiler warning:
      AccountObjects.cpp:182:47: warning: redundant move in initialization [-Wredundant-move]
  - The std::move() operation on trivially copyable types may generate a
    compile warning in newer versions of gcc.
- Remove extraneous header (unused imports) from a unit test file.
2024-03-18 12:16:20 +01:00
Denis Angell
ae1c00e339 fix node size estimation (#4536)
Fix a bug in the `NODE_SIZE` auto-detection feature in `Config.cpp`.
Specifically, this patch corrects the calculation for the total amount
of RAM available, which was previously returned in bytes, but is now
being returned in units of the system's memory unit. Additionally, the
patch adjusts the node size based on the number of available hardware
threads of execution.
2024-03-18 12:16:10 +01:00
Scott Schurr
44bc7f6109 Trivial: add comments for NFToken-related invariants (#4558) 2024-03-18 12:16:00 +01:00
Scott Determan
be7bb83a05 Add missing includes for gcc 13.1: (#4555)
gcc 13.1 failed to compile due to missing headers. This patch adds the
needed headers.
2024-03-18 12:15:49 +01:00
Scott Determan
289c1ebc68 Fix unaligned load and stores: (#4528) (#4531)
Misaligned load and store operations are supported by both Intel and ARM
CPUs. However, in C++, these operations are undefined behavior (UB).
Substituting these operations with a `memcpy` fixes this UB. The
compiled assembly code is equivalent to the original, so there is no
performance penalty to using memcpy.

For context: The unaligned load and store operations fixed here were
originally introduced in the slab allocator (#4218).
2024-03-18 12:15:34 +01:00
Ed Hennis
997b487bbb Move faulty assert (#4533)
This assert was put in the wrong place, but it only triggers if shards
are configured. This change moves the assert to the right place and
updates it to ensure correctness.

The assert could be hit after the server downloads some shards. It may
be necessary to restart after the shards are downloaded.

Note that asserts are normally checked only in debug builds, so release
packages should not be affected.

Introduced in: #4319 (66627b26cf)
2024-03-18 12:09:31 +01:00
Scott Determan
2157440cda Ensure that switchover vars are initialized before use: (#4527)
Global variables in different TUs are initialized in an undefined order.
At least one global variable was accessing a global switchover variable.
This caused the switchover variable to be accessed in an uninitialized
state.

Since the switchover is always explicitly set before transaction
processing, this bug can not effect transaction processing, but could
effect unit tests (and potentially the value of some global variables).
Note: at the time of this patch the offending bug is not yet in
production.
2024-03-18 12:08:56 +01:00
Shawn Xie
3f76ff5afe Add nftoken_id, nftoken_ids, offer_id fields for NFTokens (#4447)
Three new fields are added to the `Tx` responses for NFTs:

1. `nftoken_id`: This field is included in the `Tx` responses for
   `NFTokenMint` and `NFTokenAcceptOffer`. This field indicates the
   `NFTokenID` for the `NFToken` that was modified on the ledger by the
   transaction.
2. `nftoken_ids`: This array is included in the `Tx` response for
   `NFTokenCancelOffer`. This field provides a list of all the
   `NFTokenID`s for the `NFToken`s that were modified on the ledger by
   the transaction.
3. `offer_id`: This field is included in the `Tx` response for
   `NFTokenCreateOffer` transactions and shows the OfferID of the
   `NFTokenOffer` created.

The fields make it easier to track specific tokens and offers. The
implementation includes code (by @ledhed2222) from the Clio project to
extract NFTokenIDs from mint transactions.
2024-03-18 12:08:41 +01:00
drlongle
c923970607 fix!: Prevent API from accepting seed or public key for account (#4404)
The API would allow seeds (and public keys) to be used in place of
accounts at several locations in the API. For example, when calling
account_info, you could pass `"account": "foo"`. The string "foo" is
treated like a seed, so the method returns `actNotFound` (instead of
`actMalformed`, as most developers would expect). In the early days,
this was a convenience to make testing easier. However, it allows for
poor security practices, so it is no longer a good idea. Allowing a
secret or passphrase is now considered a bug. Previously, it was
controlled by the `strict` option on some methods. With this commit,
since the API does not interpret `account` as `seed`, the option
`strict` is no longer needed and is removed.

Removing this behavior from the API is a [breaking
change](https://xrpl.org/request-formatting.html#breaking-changes). One
could argue that it shouldn't be done without bumping the API version;
however, in this instance, there is no evidence that anyone is using the
API in the "legacy" way. Furthermore, it is a potential security hole,
as it allows users to send secrets to places where they are not needed,
where they could end up in logs, error messages, etc. There's no reason
to take such a risk with a seed/secret, since only the public address is
needed.

Resolves: #3329, #3330, #4337

BREAKING CHANGE: Remove non-strict account parsing (#3330)
2024-03-18 12:07:50 +01:00
solmsted
2956f14de8 Fix typo (#4508) 2024-03-18 12:05:50 +01:00
John Freeman
e2f61ce86c Fix errors for Clang 16: (#4501)
Address issues related to the removal of `std::{u,bi}nary_function` in
C++17 and some warnings with Clang 16. Some warnings appeared with the
upgrade to Apple clang version 14.0.3 (clang-1403.0.22.14.1).

- `std::{u,bi}nary_function` were removed in C++17. They were empty
  classes with a few associated types. We already have conditional code
  to define the types. Just make it unconditional.
- libc++ checks a cast in an unevaluated context to see if a type
  inherits from a binary function class in the standard library, e.g.
  `std::equal_to`, and this causes an error when the type privately
  inherits from such a class. Change these instances to public
  inheritance.
- We don't need a middle-man for the empty base optimization. Prefer to
  inherit directly from an empty class than from
  `beast::detail::empty_base_optimization`.
- Clang warns when all the uses of a variable are removed by conditional
  compilation of assertions. Add a `[[maybe_unused]]` annotation to
  suppress it.
- As a drive-by clean-up, remove commented code.

See related work in #4486.
2024-03-18 12:04:41 +01:00
Mark Travis
1fafd1059d Use quorum specified via command line: (#4489)
If `--quorum` setting is present on the command line, use the specified
value as the minimum quorum. This allows for the use of a potentially
fork-unsafe quorum, but it is sometimes necessary for small and test
networks.

Fix #4488.

---------

Co-authored-by: RichardAH <richard.holland@starstone.co.nz>
2024-03-18 12:04:09 +01:00
John Freeman
cee8409d60 Fix the fix for std::result_of (#4496)
Newer compilers, such as Apple Clang 15.0, have removed `std::result_of`
as part of C++20. The build instructions provided a fix for this (by
adding a preprocessor definition), but the fix was broken.

This fixes the fix by:
* Adding the `conf` prefix for tool configurations (which had been
  forgotten).
* Passing `extra_b2_flags` to `boost` package to fix its build.
  * Define `BOOST_ASIO_HAS_STD_INVOKE_RESULT` in order to build boost
    1.77 with a newer compiler.
2024-03-18 12:01:25 +01:00
RichardAH
f23c32cc00 Prevent replay attacks with NetworkID field: (#4370)
Add a `NetworkID` field to help prevent replay attacks on and from
side-chains.

The new field must be used when the server is using a network id > 1024.

To preserve legacy behavior, all chains with a network ID less than 1025
retain the existing behavior. This includes Mainnet, Testnet, Devnet,
and hooks-testnet. If `sfNetworkID` is present in any transaction
submitted to any of the nodes on one of these chains, then
`telNETWORK_ID_MAKES_TX_NON_CANONICAL` is returned.

Since chains with a network ID less than 1025, including Mainnet, retain
the existing behavior, there is no need for an amendment.

The `NetworkID` helps to prevent replay attacks because users specify a
`NetworkID` field in every transaction for that chain.

This change introduces a new UINT32 field, `sfNetworkID` ("NetworkID").
There are also three new local error codes for transaction results:

- `telNETWORK_ID_MAKES_TX_NON_CANONICAL`
- `telREQUIRES_NETWORK_ID`
- `telWRONG_NETWORK`

To learn about the other transaction result codes, see:
https://xrpl.org/transaction-results.html

Local error codes were chosen because a transaction is not necessarily
malformed if it is submitted to a node running on the incorrect chain.
This is a local error specific to that node and could be corrected by
switching to a different node or by changing the `network_id` on that
node. See:
https://xrpl.org/connect-your-rippled-to-the-xrp-test-net.html

In addition to using `NetworkID`, it is still generally recommended to
use different accounts and keys on side-chains. However, people will
undoubtedly use the same keys on multiple chains; for example, this is
common practice on other blockchain networks. There are also some
legitimate use cases for this.

A `app.NetworkID` test suite has been added, and `core.Config` was
updated to include some network_id tests.
2024-03-18 12:00:37 +01:00
Nik Bougalis
1b835b7c05 Avoid using std::shared_ptr when not necessary: (#4218)
The `Ledger` class contains two `SHAMap` instances: the state and
transaction maps. Previously, the maps were dynamically allocated using
`std::make_shared` despite the fact that they did not require lifetime
management separate from the lifetime of the `Ledger` instance to which
they belong.

The two `SHAMap` instances are now regular member variables. Some smart
pointers and dynamic memory allocation was avoided by using stack-based
alternatives.

Commit 3 of 3 in #4218.
2024-03-18 11:57:30 +01:00
Nik Bougalis
9ec1e527a3 Optimize SHAMapItem and leverage new slab allocator: (#4218)
The `SHAMapItem` class contains a variable-sized buffer that
holds the serialized data associated with a particular item
inside a `SHAMap`.

Prior to this commit, the buffer for the serialized data was
allocated separately. Coupled with the fact that most instances
of `SHAMapItem` were wrapped around a `std::shared_ptr` meant
that an instantiation might result in up to three separate
memory allocations.

This commit switches away from `std::shared_ptr` for `SHAMapItem`
and uses `boost::intrusive_ptr` instead, allowing the reference
count for an instance to live inside the instance itself. Coupled
with using a slab-based allocator to optimize memory allocation
for the most commonly sized buffers, the net result is significant
memory savings. In testing, the reduction in memory usage hovers
between 400MB and 650MB. Other scenarios might result in larger
savings.

In performance testing with NFTs, this commit reduces memory size by
about 15% sustained over long duration.

Commit 2 of 3 in #4218.
2024-03-18 11:57:15 +01:00
Nik Bougalis
469eb2b8ac Introduce support for a slabbed allocator: (#4218)
When instantiating a large amount of fixed-sized objects on the heap
the overhead that dynamic memory allocation APIs impose will quickly
become significant.

In some cases, allocating a large amount of memory at once and using
a slabbing allocator to carve the large block into fixed-sized units
that are used to service requests for memory out will help to reduce
memory fragmentation significantly and, potentially, improve overall
performance.

This commit introduces a new `SlabAllocator<>` class that exposes an
API that is _similar_ to the C++ concept of an `Allocator` but it is
not meant to be a general-purpose allocator.

It should not be used unless profiling and analysis of specific memory
allocation patterns indicates that the additional complexity introduced
will improve the performance of the system overall, and subsequent
profiling proves it.

A helper class, `SlabAllocatorSet<>` simplifies handling of variably
sized objects that benefit from slab allocations.

This commit incorporates improvements suggested by Greg Popovitch
(@greg7mdp).

Commit 1 of 3 in #4218.
2024-03-18 11:57:00 +01:00
ledhed2222
0dc262dd9c Add jss fields used by Clio nft_info: (#4320)
Add Clio-specific JSS constants to ensure a common vocabulary of
keywords in Clio and this project. By providing visibility of the full
API keyword namespace, it reduces the likelihood of developers
introducing minor variations on names used by Clio, or unknowingly
claiming a keyword that Clio has already claimed. This change moves this
project slightly away from having only the code necessary for running
the core server, but it is a step toward the goal of keeping this
server's and Clio's APIs similar. The added JSS constants are annotated
to indicate their relevance to Clio.

Clio can be found here: https://github.com/XRPLF/clio

Signed-off-by: ledhed2222 <ledhed2222@users.noreply.github.com>
2024-03-18 11:56:20 +01:00
12 changed files with 37 additions and 552 deletions

View File

@@ -729,7 +729,6 @@ if (tests)
src/test/app/LedgerLoad_test.cpp
src/test/app/LedgerMaster_test.cpp
src/test/app/LedgerReplay_test.cpp
src/test/app/LedgerStress_test.cpp
src/test/app/LoadFeeTrack_test.cpp
src/test/app/Manifest_test.cpp
src/test/app/MultiSign_test.cpp

View File

@@ -322,9 +322,6 @@ public:
void
transactionBatch();
void
forceTransactionBatch();
/**
* Attempt to apply transactions and post-process based on the results.
*
@@ -1139,7 +1136,6 @@ NetworkOPsImp::strOperatingMode(OperatingMode const mode, bool const admin)
void
NetworkOPsImp::submitTransaction(std::shared_ptr<STTx const> const& iTrans)
{
// Launch async task and return immediately
if (isNeedNetworkLedger())
{
// Nothing we can do if we've never been in sync
@@ -1151,9 +1147,9 @@ NetworkOPsImp::submitTransaction(std::shared_ptr<STTx const> const& iTrans)
// Enforce Network bar for emitted txn
if (view->rules().enabled(featureHooks) && hook::isEmittedTxn(*iTrans))
{
// RH NOTE: Warning removed here due to ConsesusSet using this
// function which continually triggers this bar. Doesn't seem
// dangerous, just annoying.
// RH NOTE: Warning removed here due to ConsesusSet using this function
// which continually triggers this bar. Doesn't seem dangerous, just
// annoying.
// JLOG(m_journal.warn())
// << "Submitted transaction invalid: EmitDetails present.";
@@ -1168,9 +1164,9 @@ NetworkOPsImp::submitTransaction(std::shared_ptr<STTx const> const& iTrans)
if ((flags & SF_BAD) != 0)
{
// RH NOTE: Warning removed here due to ConsesusSet using this
// function which continually triggers this bar. Doesn't seem
// dangerous, just annoying.
// RH NOTE: Warning removed here due to ConsesusSet using this function
// which continually triggers this bar. Doesn't seem dangerous, just
// annoying.
// JLOG(m_journal.warn()) << "Submitted transaction cached bad";
return;
@@ -1368,17 +1364,6 @@ NetworkOPsImp::doTransactionSync(
} while (transaction->getApplying());
}
void
NetworkOPsImp::forceTransactionBatch()
{
std::unique_lock<std::mutex> lock(mMutex);
mDispatchState = DispatchState::scheduled;
while (mTransactions.size())
{
apply(lock);
}
}
void
NetworkOPsImp::transactionBatch()
{
@@ -1413,6 +1398,7 @@ NetworkOPsImp::apply(std::unique_lock<std::mutex>& batchLock)
std::unique_lock ledgerLock{
m_ledgerMaster.peekMutex(), std::defer_lock};
std::lock(masterLock, ledgerLock);
app_.openLedger().modify([&](OpenView& view, beast::Journal j) {
for (TransactionStatus& e : transactions)
{

View File

@@ -137,10 +137,7 @@ public:
std::shared_ptr<Transaction>& transaction,
bool bUnlimited,
bool bLocal,
FailHard failType = FailHard::no) = 0;
virtual void
forceTransactionBatch() = 0;
FailHard failType) = 0;
//--------------------------------------------------------------------------
//

View File

@@ -24,7 +24,6 @@
#include <ripple/app/misc/TxQ.h>
#include <ripple/app/tx/apply.h>
#include <ripple/basics/mulDiv.h>
#include <ripple/protocol/AccountID.h>
#include <ripple/protocol/Feature.h>
#include <ripple/protocol/jss.h>
#include <ripple/protocol/st.h>
@@ -1898,15 +1897,7 @@ TxQ::tryDirectApply(
// transaction straight into the ledger.
FeeLevel64 const feeLevelPaid = getFeeLevelPaid(view, *tx);
static auto const genesisAccountId = calcAccountID(
generateKeyPair(KeyType::secp256k1, generateSeed("masterpassphrase"))
.first);
// RH NOTE: exempting the genesis account from fee escalation is useful for
// stress testing it also shouldn't require an amendment because it will be
// fought out in consensus.
if (feeLevelPaid >= requiredFeeLevel ||
(*tx)[sfAccount] == genesisAccountId)
if (feeLevelPaid >= requiredFeeLevel)
{
// Attempt to apply the transaction directly.
auto const transactionID = tx->getTransactionID();

View File

@@ -458,13 +458,6 @@ Change::activateXahauGenesis()
bool const isTest =
(ctx_.tx.getFlags() & tfTestSuite) && ctx_.app.config().standalone();
// RH NOTE: we'll only configure xahau governance structure on networks that
// begin with 2133... so production xahau: 21337 and its testnet 21338
// with 21330-21336 and 21339 also valid and reserved for dev nets etc.
// all other Network IDs will be conventionally configured.
if ((ctx_.app.config().NETWORK_ID / 10) != 2133 && !isTest)
return;
auto [ng_entries, l1_entries, l2_entries, gov_params] =
normalizeXahauGenesis(
isTest ? TestNonGovernanceDistribution : NonGovernanceDistribution,

View File

@@ -38,7 +38,6 @@
#include <memory>
#include <optional>
#include <unordered_set>
#include <iterator>
namespace ripple {
@@ -322,12 +321,6 @@ public:
// The range of transactions
txs_type txs;
std::size_t
txCount() const
{
return std::distance(txs.begin(), txs.end());
}
};
//------------------------------------------------------------------------------

View File

@@ -1,320 +0,0 @@
#include <ripple/app/ledger/LedgerMaster.h>
#include <ripple/app/misc/TxQ.h>
#include <ripple/basics/chrono.h>
#include <ripple/protocol/AccountID.h>
#include <ripple/protocol/Feature.h>
#include <ripple/protocol/jss.h>
#include <algorithm>
#include <chrono>
#include <map>
#include <mutex>
#include <test/jtx.h>
#include <test/jtx/Env.h>
#include <thread>
#include <vector>
namespace ripple {
namespace test {
using namespace jtx;
class LedgerStress_test : public beast::unit_test::suite
{
private:
static constexpr std::size_t TXN_PER_LEDGER = 50000;
static constexpr std::size_t MAX_TXN_PER_ACCOUNT = 5; // Increased from 1
static constexpr std::chrono::seconds MAX_CLOSE_TIME{15};
static constexpr std::size_t REQUIRED_ACCOUNTS =
(TXN_PER_LEDGER + MAX_TXN_PER_ACCOUNT - 1) / MAX_TXN_PER_ACCOUNT;
// Get number of hardware threads and use half
const std::size_t NUM_THREADS =
std::max(std::thread::hardware_concurrency() / 2, 1u);
struct LedgerMetrics
{
std::chrono::milliseconds submitTime{0};
std::chrono::milliseconds closeTime{0};
std::size_t txCount{0};
std::size_t successfulTxCount{
0}; // Added to track successful transactions
std::size_t failedTxCount{0}; // Added to track failed transactions
XRPAmount baseFee{0};
void
log(beast::Journal const& journal) const
{
std::cout << "Metrics - Submit time: " << submitTime.count()
<< "ms, "
<< "Close time: " << closeTime.count() << "ms, "
<< "Transaction count: " << txCount << ", "
<< "Successful: " << successfulTxCount << ", "
<< "Failed: " << failedTxCount << ", "
<< "Base fee: " << baseFee;
}
};
// Thread-safe console output
std::mutex consoleMutex;
std::atomic<std::size_t> totalSuccessfulTxns{0};
template <typename T>
void
threadSafeLog(T const& message)
{
std::lock_guard<std::mutex> lock(consoleMutex);
std::cout << message << std::endl;
}
XRPAmount
getEscalatedFee(jtx::Env& env) const
{
auto const metrics = env.app().getTxQ().getMetrics(*env.current());
auto const baseFee = env.current()->fees().base;
auto const feeLevel =
mulDiv(metrics.medFeeLevel, baseFee, metrics.referenceFeeLevel)
.second;
auto const escalatedFee = XRPAmount{feeLevel};
return XRPAmount{escalatedFee.drops() + (escalatedFee.drops())};
}
std::vector<jtx::Account>
createAccounts(jtx::Env& env, std::size_t count)
{
std::vector<jtx::Account> accounts;
accounts.reserve(count);
for (std::size_t i = 0; i < count; ++i)
{
std::string name = "account" + std::to_string(i);
auto account = jtx::Account(name);
accounts.push_back(account);
env.fund(false, XRP(100000), account);
if (i % 2500 == 0 && i != 0)
threadSafeLog("Accounts created: " + std::to_string(i));
}
env.close();
return accounts;
}
// Structure to hold work assignment for each thread
struct ThreadWork
{
std::size_t startAccountIdx;
std::size_t endAccountIdx;
std::size_t numTxnsToSubmit;
std::size_t successfulTxns{0};
std::size_t failedTxns{0};
};
void
submitBatchThread(
jtx::Env& env,
std::vector<jtx::Account> const& accounts,
ThreadWork& work) // Changed to non-const reference to update metrics
{
auto const escalatedFee = getEscalatedFee(env);
std::size_t txnsSubmitted = 0;
// Track sequence numbers for all accounts in this thread's range
std::map<AccountID, std::uint32_t> seqNumbers;
for (std::size_t i = work.startAccountIdx; i < work.endAccountIdx; ++i)
{
seqNumbers[accounts[i].id()] = env.seq(accounts[i]);
}
// Pre-calculate recipient indices for better distribution
std::vector<std::size_t> recipientIndices;
recipientIndices.reserve(accounts.size() - 1);
for (std::size_t i = 0; i < accounts.size(); ++i)
{
if (i < work.startAccountIdx || i >= work.endAccountIdx)
{
recipientIndices.push_back(i);
}
}
std::size_t recipientIdx = 0;
for (std::size_t i = work.startAccountIdx;
i < work.endAccountIdx && txnsSubmitted < work.numTxnsToSubmit;
++i)
{
auto const& sender = accounts[i];
// Calculate how many txns to submit from this account
std::size_t txnsRemaining = work.numTxnsToSubmit - txnsSubmitted;
std::size_t txnsForAccount =
std::min(MAX_TXN_PER_ACCOUNT, txnsRemaining);
// Submit transactions
for (std::size_t tx = 0; tx < txnsForAccount; ++tx)
{
// Select next recipient using round-robin
auto const& recipient =
accounts[recipientIndices[recipientIdx]];
recipientIdx = (recipientIdx + 1) % recipientIndices.size();
try
{
env.inject(
pay(sender, recipient, XRP(1)),
fee(escalatedFee),
seq(seqNumbers[sender.id()]));
++work.successfulTxns;
seqNumbers[sender.id()]++;
}
catch (std::exception const& e)
{
++work.failedTxns;
threadSafeLog(
"Exception submitting transaction: " +
std::string(e.what()));
}
++txnsSubmitted;
}
}
}
void
runStressTest(std::size_t numLedgers)
{
testcase(
"Multithreaded stress test: " + std::to_string(TXN_PER_LEDGER) +
" txns/ledger for " + std::to_string(numLedgers) +
" ledgers using " + std::to_string(NUM_THREADS) + " threads");
Env env{*this, envconfig(many_workers)};
env.app().config().MAX_TRANSACTIONS = TXN_PER_LEDGER;
auto const journal = env.app().journal("LedgerStressTest");
// Get actual hardware thread count
std::size_t hardwareThreads =
static_cast<std::size_t>(std::thread::hardware_concurrency());
if (hardwareThreads == 0)
hardwareThreads = 4; // Fallback
const std::size_t THREAD_COUNT = std::min(NUM_THREADS, hardwareThreads);
threadSafeLog(
"Using " + std::to_string(THREAD_COUNT) + " hardware threads");
threadSafeLog(
"Creating " + std::to_string(REQUIRED_ACCOUNTS) + " accounts");
auto accounts = createAccounts(env, REQUIRED_ACCOUNTS);
std::vector<LedgerMetrics> metrics;
metrics.reserve(numLedgers);
for (std::size_t ledger = 0; ledger < numLedgers; ++ledger)
{
threadSafeLog("Starting ledger " + std::to_string(ledger));
LedgerMetrics ledgerMetrics;
auto submitStart = std::chrono::steady_clock::now();
ledgerMetrics.baseFee = env.current()->fees().base;
// Calculate even distribution of work
std::vector<ThreadWork> threadAssignments;
threadAssignments.reserve(THREAD_COUNT);
std::size_t baseWorkload = TXN_PER_LEDGER / THREAD_COUNT;
std::size_t remainder = TXN_PER_LEDGER % THREAD_COUNT;
std::size_t accountsPerThread = accounts.size() / THREAD_COUNT;
std::size_t totalAccountsAssigned = 0;
for (std::size_t t = 0; t < THREAD_COUNT; ++t)
{
ThreadWork work;
work.startAccountIdx = totalAccountsAssigned;
work.endAccountIdx = (t == THREAD_COUNT - 1)
? accounts.size()
: work.startAccountIdx + accountsPerThread;
work.numTxnsToSubmit = baseWorkload + (t < remainder ? 1 : 0);
totalAccountsAssigned = work.endAccountIdx;
threadAssignments.push_back(work);
}
// Launch threads with work assignments
std::vector<std::thread> threads;
threads.reserve(THREAD_COUNT);
for (std::size_t t = 0; t < THREAD_COUNT; ++t)
{
threads.emplace_back(
[&env, &accounts, &work = threadAssignments[t], this]() {
submitBatchThread(env, accounts, work);
});
}
// Wait for all threads
for (auto& thread : threads)
{
if (thread.joinable())
thread.join();
}
// Aggregate metrics from all threads
ledgerMetrics.successfulTxCount = 0;
ledgerMetrics.failedTxCount = 0;
for (auto const& work : threadAssignments)
{
ledgerMetrics.successfulTxCount += work.successfulTxns;
ledgerMetrics.failedTxCount += work.failedTxns;
}
ledgerMetrics.submitTime =
std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now() - submitStart);
auto closeStart = std::chrono::steady_clock::now();
env.close();
ledgerMetrics.closeTime =
std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now() - closeStart);
auto const closed = env.closed();
ledgerMetrics.txCount = closed->txCount();
ledgerMetrics.log(journal);
metrics.push_back(ledgerMetrics);
auto const totalTime =
ledgerMetrics.submitTime + ledgerMetrics.closeTime;
// Updated expectations
BEAST_EXPECT(
ledgerMetrics.txCount >=
ledgerMetrics.successfulTxCount * 0.8); // Allow 20% variance
BEAST_EXPECT(
ledgerMetrics.closeTime <=
std::chrono::duration_cast<std::chrono::milliseconds>(
MAX_CLOSE_TIME));
threadSafeLog(
"\nCompleted ledger " + std::to_string(ledger) + " in " +
std::to_string(totalTime.count()) + "ms" + " with " +
std::to_string(ledgerMetrics.successfulTxCount) +
" successful transactions using " +
std::to_string(THREAD_COUNT) + " threads");
}
}
public:
void
run() override
{
runStressTest(5);
}
};
BEAST_DEFINE_TESTSUITE(LedgerStress, app, ripple);
} // namespace test
} // namespace ripple

View File

@@ -19,7 +19,6 @@
#include <ripple/app/misc/HashRouter.h>
#include <ripple/app/tx/apply.h>
#include <ripple/app/tx/impl/XahauGenesis.h>
#include <ripple/core/Config.h>
#include <ripple/json/json_reader.h>
#include <ripple/protocol/Feature.h>
#include <ripple/protocol/Indexes.h>
@@ -28,7 +27,6 @@
#include <ripple/protocol/jss.h>
#include <string>
#include <test/jtx.h>
#include <test/jtx/envconfig.h>
#include <vector>
#define BEAST_REQUIRE(x) \
@@ -61,18 +59,7 @@ maybe_to_string(T val, std::enable_if_t<!std::is_integral_v<T>, int> = 0)
using namespace XahauGenesis;
namespace ripple {
inline std::unique_ptr<Config>
makeNetworkConfig(uint32_t networkID)
{
using namespace test::jtx;
return envconfig([&](std::unique_ptr<Config> cfg) {
cfg->NETWORK_ID = networkID;
return cfg;
});
}
namespace test {
/*
Accounts used in this test suite:
alice: AE123A8556F3CF91154711376AFB0F894F832B3D,
@@ -138,8 +125,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
bool burnedViaTest =
false, // means the calling test already burned some of the genesis
bool skipTests = false,
bool const testFlag = false,
bool const badNetID = false)
bool const testFlag = false)
{
using namespace jtx;
@@ -197,20 +183,6 @@ struct XahauGenesis_test : public beast::unit_test::suite
if (skipTests)
return;
if (badNetID)
{
BEAST_EXPECT(
100000000000000000ULL ==
env.app().getLedgerMaster().getClosedLedger()->info().drops);
auto genesisAccRoot = env.le(keylet::account(genesisAccID));
BEAST_REQUIRE(!!genesisAccRoot);
BEAST_EXPECT(
genesisAccRoot->getFieldAmount(sfBalance) ==
XRPAmount(100000000000000000ULL));
return;
}
// sum the initial distribution balances, these should equal total coins
// in the closed ledger
std::vector<std::pair<std::string, XRPAmount>> const& l1membership =
@@ -470,59 +442,17 @@ struct XahauGenesis_test : public beast::unit_test::suite
{
testcase("Test activation");
using namespace jtx;
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
activate(__LINE__, env, false, false, false);
}
void
testBadNetworkIDActivation(FeatureBitset features)
{
testcase("Test Bad Network ID activation");
using namespace jtx;
std::vector<int> badNetIDs{
0,
1,
2,
10,
100,
1000,
10000,
20000,
21000,
21328,
21329,
21340,
21341,
65535};
for (int netid : badNetIDs)
{
Env env{
*this,
makeNetworkConfig(netid),
features - featureXahauGenesis};
activate(__LINE__, env, false, false, false, true);
}
for (int netid = 21330; netid <= 21339; ++netid)
{
Env env{
*this,
makeNetworkConfig(netid),
features - featureXahauGenesis};
activate(__LINE__, env, false, false, false, false);
}
}
void
testWithSignerList(FeatureBitset features)
{
using namespace jtx;
testcase("Test signerlist");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
Account const alice{"alice", KeyType::ed25519};
env.fund(XRP(1000), alice);
@@ -538,8 +468,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
{
using namespace jtx;
testcase("Test regkey");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
env.memoize(env.master);
Account const alice("alice");
@@ -738,11 +667,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
{
using namespace jtx;
testcase("Test governance membership voting L1");
Env env{
*this,
makeNetworkConfig(21337),
features - featureXahauGenesis,
nullptr};
Env env{*this, envconfig(), features - featureXahauGenesis, nullptr};
auto const alice = Account("alice");
auto const bob = Account("bob");
@@ -2186,8 +2111,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace jtx;
testcase("Test governance membership voting L2");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
auto const alice = Account("alice");
auto const bob = Account("bob");
@@ -3784,7 +3708,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace std::chrono_literals;
testcase("test last close time");
Env env{*this, makeNetworkConfig(21337), features};
Env env{*this, envconfig(), features};
validateTime(lastClose(env), 0);
// last close = 0
@@ -3814,8 +3738,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace jtx;
testcase("test claim reward rate is == 0");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
STAmount const feesXRP = XRP(1);
@@ -3860,8 +3783,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace jtx;
testcase("test claim reward rate is > 1");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
STAmount const feesXRP = XRP(1);
@@ -3906,8 +3828,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace jtx;
testcase("test claim reward delay is == 0");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
STAmount const feesXRP = XRP(1);
@@ -3952,8 +3873,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace jtx;
testcase("test claim reward delay is < 0");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
STAmount const feesXRP = XRP(1);
@@ -3998,8 +3918,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace jtx;
testcase("test claim reward before time");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
STAmount const feesXRP = XRP(1);
@@ -4049,8 +3968,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace std::chrono_literals;
testcase("test claim reward valid without unl report");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
bool const has240819 = env.current()->rules().enabled(fix240819);
double const rateDrops = 0.00333333333 * 1'000'000;
@@ -4197,8 +4115,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace std::chrono_literals;
testcase("test claim reward valid with unl report");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
double const rateDrops = 0.00333333333 * 1'000'000;
STAmount const feesXRP = XRP(1);
@@ -4333,7 +4250,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
{
FeatureBitset _features = features - featureXahauGenesis;
auto const amend = withXahauV1 ? _features : _features - fixXahauV1;
Env env{*this, makeNetworkConfig(21337), amend};
Env env{*this, envconfig(), amend};
double const rateDrops = 0.00333333333 * 1'000'000;
STAmount const feesXRP = XRP(1);
@@ -4470,8 +4387,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace std::chrono_literals;
testcase("test claim reward optin optout");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
bool const has240819 = env.current()->rules().enabled(fix240819);
double const rateDrops = 0.00333333333 * 1'000'000;
@@ -4583,8 +4499,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace std::chrono_literals;
testcase("test claim reward bal == 1");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
double const rateDrops = 0.00333333333 * 1'000'000;
STAmount const feesXRP = XRP(1);
@@ -4672,8 +4587,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace std::chrono_literals;
testcase("test claim reward elapsed_since_last == 1");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
double const rateDrops = 0.00333333333 * 1'000'000;
STAmount const feesXRP = XRP(1);
@@ -4754,8 +4668,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace std::chrono_literals;
testcase("test claim reward elapsed_since_last == 0");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
STAmount const feesXRP = XRP(1);
@@ -5016,8 +4929,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace std::chrono_literals;
testcase("test compound interest over 12 claims");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
double const rateDrops = 0.00333333333 * 1'000'000;
STAmount const feesXRP = XRP(1);
@@ -5115,8 +5027,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace std::chrono_literals;
testcase("test deposit");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
double const rateDrops = 0.00333333333 * 1'000'000;
STAmount const feesXRP = XRP(1);
@@ -5206,8 +5117,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace std::chrono_literals;
testcase("test deposit withdraw");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
double const rateDrops = 0.00333333333 * 1'000'000;
STAmount const feesXRP = XRP(1);
@@ -5299,8 +5209,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace std::chrono_literals;
testcase("test deposit late");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
double const rateDrops = 0.00333333333 * 1'000'000;
STAmount const feesXRP = XRP(1);
@@ -5390,8 +5299,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace std::chrono_literals;
testcase("test deposit late withdraw");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
double const rateDrops = 0.00333333333 * 1'000'000;
STAmount const feesXRP = XRP(1);
@@ -5484,8 +5392,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace std::chrono_literals;
testcase("test no claim");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
double const rateDrops = 0.00333333333 * 1'000'000;
STAmount const feesXRP = XRP(1);
@@ -5573,8 +5480,7 @@ struct XahauGenesis_test : public beast::unit_test::suite
using namespace std::chrono_literals;
testcase("test no claim late");
Env env{
*this, makeNetworkConfig(21337), features - featureXahauGenesis};
Env env{*this, envconfig(), features - featureXahauGenesis};
double const rateDrops = 0.00333333333 * 1'000'000;
STAmount const feesXRP = XRP(1);
@@ -5688,7 +5594,6 @@ struct XahauGenesis_test : public beast::unit_test::suite
testGovernHookWithFeats(FeatureBitset features)
{
testPlainActivation(features);
testBadNetworkIDActivation(features);
testWithSignerList(features);
testWithRegularKey(features);
testGovernanceL1(features);

View File

@@ -148,12 +148,6 @@ public:
operator=(Env const&) = delete;
Env(Env const&) = delete;
Application*
getApp()
{
return bundle_.app;
}
/**
* @brief Create Env using suite, Config pointer, and explicit features.
*
@@ -514,9 +508,6 @@ public:
virtual void
submit(JTx const& jt);
virtual void
inject_jtx(JTx const& jt);
/** Use the submit RPC command with a provided JTx object.
This calls postconditions.
*/
@@ -538,13 +529,6 @@ public:
submit(jt(std::forward<JsonValue>(jv), fN...));
}
template <class JsonValue, class... FN>
void
inject(JsonValue&& jv, FN const&... fN)
{
inject_jtx(jt(std::forward<JsonValue>(jv), fN...));
}
template <class JsonValue, class... FN>
void
operator()(JsonValue&& jv, FN const&... fN)
@@ -607,6 +591,7 @@ public:
void
disableFeature(uint256 const feature);
private:
void
fund(bool setDefaultRipple, STAmount const& amount, Account const& account);

View File

@@ -86,9 +86,6 @@ std::unique_ptr<Config> no_admin(std::unique_ptr<Config>);
std::unique_ptr<Config>
no_admin_networkid(std::unique_ptr<Config> cfg);
std::unique_ptr<Config>
many_workers(std::unique_ptr<Config> cfg);
std::unique_ptr<Config> secure_gateway(std::unique_ptr<Config>);
std::unique_ptr<Config> admin_localnet(std::unique_ptr<Config>);

View File

@@ -49,8 +49,6 @@
#include <test/jtx/sig.h>
#include <test/jtx/trust.h>
#include <test/jtx/utility.h>
#include <ripple/app/misc/NetworkOPs.h>
#include <ripple/app/misc/Transaction.h>
namespace ripple {
namespace test {
@@ -126,17 +124,13 @@ Env::close(
{
// Round up to next distinguishable value
using namespace std::chrono_literals;
auto& netOPs = app().getOPs();
netOPs.forceTransactionBatch();
bool res = true;
closeTime += closed()->info().closeTimeResolution - 1s;
timeKeeper().set(closeTime);
// Go through the rpc interface unless we need to simulate
// a specific consensus delay.
if (consensusDelay)
netOPs.acceptLedger(consensusDelay);
app().getOPs().acceptLedger(consensusDelay);
else
{
auto resp = rpc("ledger_accept");
@@ -290,33 +284,6 @@ Env::parseResult(Json::Value const& jr)
return std::make_pair(ter, isTesSuccess(ter) || isTecClaim(ter));
}
void
Env::inject_jtx(JTx const& jt)
{
Application& app = *(getApp());
auto& netOPs = app.getOPs();
if (jt.stx)
{
std::string reason;
// make a copy
//STTx* newData = new STTx(*jt.stx);
//auto stx = std::shared_ptr<STTx const>(newData);
auto id = jt.stx->getTransactionID();
auto tx = std::make_shared<Transaction>(jt.stx, reason, app);
/*
static int counter = 0;
counter++;
if (counter % 2500 == 0)
std::cout << "inject_jtx [" << counter++ << "] id=" << id << "\n";
*/
netOPs.processTransaction(tx, true, false);
}
return postconditions(jt, ter_, true);
}
void
Env::submit(JTx const& jt)
{

View File

@@ -76,14 +76,6 @@ setupConfigForUnitTests(Config& cfg)
namespace jtx {
std::unique_ptr<Config>
many_workers(std::unique_ptr<Config> cfg)
{
cfg->WORKERS = 128;
return cfg;
}
std::unique_ptr<Config>
no_admin(std::unique_ptr<Config> cfg)
{