* Add jss fields used by Clio `nft_info`: (#4320)
Add Clio-specific JSS constants to ensure a common vocabulary of
keywords in Clio and this project. By providing visibility of the full
API keyword namespace, it reduces the likelihood of developers
introducing minor variations on names used by Clio, or unknowingly
claiming a keyword that Clio has already claimed. This change moves this
project slightly away from having only the code necessary for running
the core server, but it is a step toward the goal of keeping this
server's and Clio's APIs similar. The added JSS constants are annotated
to indicate their relevance to Clio.
Clio can be found here: https://github.com/XRPLF/clio
Signed-off-by: ledhed2222 <ledhed2222@users.noreply.github.com>
* Introduce support for a slabbed allocator: (#4218)
When instantiating a large amount of fixed-sized objects on the heap
the overhead that dynamic memory allocation APIs impose will quickly
become significant.
In some cases, allocating a large amount of memory at once and using
a slabbing allocator to carve the large block into fixed-sized units
that are used to service requests for memory out will help to reduce
memory fragmentation significantly and, potentially, improve overall
performance.
This commit introduces a new `SlabAllocator<>` class that exposes an
API that is _similar_ to the C++ concept of an `Allocator` but it is
not meant to be a general-purpose allocator.
It should not be used unless profiling and analysis of specific memory
allocation patterns indicates that the additional complexity introduced
will improve the performance of the system overall, and subsequent
profiling proves it.
A helper class, `SlabAllocatorSet<>` simplifies handling of variably
sized objects that benefit from slab allocations.
This commit incorporates improvements suggested by Greg Popovitch
(@greg7mdp).
Commit 1 of 3 in #4218.
* Optimize `SHAMapItem` and leverage new slab allocator: (#4218)
The `SHAMapItem` class contains a variable-sized buffer that
holds the serialized data associated with a particular item
inside a `SHAMap`.
Prior to this commit, the buffer for the serialized data was
allocated separately. Coupled with the fact that most instances
of `SHAMapItem` were wrapped around a `std::shared_ptr` meant
that an instantiation might result in up to three separate
memory allocations.
This commit switches away from `std::shared_ptr` for `SHAMapItem`
and uses `boost::intrusive_ptr` instead, allowing the reference
count for an instance to live inside the instance itself. Coupled
with using a slab-based allocator to optimize memory allocation
for the most commonly sized buffers, the net result is significant
memory savings. In testing, the reduction in memory usage hovers
between 400MB and 650MB. Other scenarios might result in larger
savings.
In performance testing with NFTs, this commit reduces memory size by
about 15% sustained over long duration.
Commit 2 of 3 in #4218.
* Avoid using std::shared_ptr when not necessary: (#4218)
The `Ledger` class contains two `SHAMap` instances: the state and
transaction maps. Previously, the maps were dynamically allocated using
`std::make_shared` despite the fact that they did not require lifetime
management separate from the lifetime of the `Ledger` instance to which
they belong.
The two `SHAMap` instances are now regular member variables. Some smart
pointers and dynamic memory allocation was avoided by using stack-based
alternatives.
Commit 3 of 3 in #4218.
* Prevent replay attacks with NetworkID field: (#4370)
Add a `NetworkID` field to help prevent replay attacks on and from
side-chains.
The new field must be used when the server is using a network id > 1024.
To preserve legacy behavior, all chains with a network ID less than 1025
retain the existing behavior. This includes Mainnet, Testnet, Devnet,
and hooks-testnet. If `sfNetworkID` is present in any transaction
submitted to any of the nodes on one of these chains, then
`telNETWORK_ID_MAKES_TX_NON_CANONICAL` is returned.
Since chains with a network ID less than 1025, including Mainnet, retain
the existing behavior, there is no need for an amendment.
The `NetworkID` helps to prevent replay attacks because users specify a
`NetworkID` field in every transaction for that chain.
This change introduces a new UINT32 field, `sfNetworkID` ("NetworkID").
There are also three new local error codes for transaction results:
- `telNETWORK_ID_MAKES_TX_NON_CANONICAL`
- `telREQUIRES_NETWORK_ID`
- `telWRONG_NETWORK`
To learn about the other transaction result codes, see:
https://xrpl.org/transaction-results.html
Local error codes were chosen because a transaction is not necessarily
malformed if it is submitted to a node running on the incorrect chain.
This is a local error specific to that node and could be corrected by
switching to a different node or by changing the `network_id` on that
node. See:
https://xrpl.org/connect-your-rippled-to-the-xrp-test-net.html
In addition to using `NetworkID`, it is still generally recommended to
use different accounts and keys on side-chains. However, people will
undoubtedly use the same keys on multiple chains; for example, this is
common practice on other blockchain networks. There are also some
legitimate use cases for this.
A `app.NetworkID` test suite has been added, and `core.Config` was
updated to include some network_id tests.
* Fix the fix for std::result_of (#4496)
Newer compilers, such as Apple Clang 15.0, have removed `std::result_of`
as part of C++20. The build instructions provided a fix for this (by
adding a preprocessor definition), but the fix was broken.
This fixes the fix by:
* Adding the `conf` prefix for tool configurations (which had been
forgotten).
* Passing `extra_b2_flags` to `boost` package to fix its build.
* Define `BOOST_ASIO_HAS_STD_INVOKE_RESULT` in order to build boost
1.77 with a newer compiler.
* Use quorum specified via command line: (#4489)
If `--quorum` setting is present on the command line, use the specified
value as the minimum quorum. This allows for the use of a potentially
fork-unsafe quorum, but it is sometimes necessary for small and test
networks.
Fix#4488.
---------
Co-authored-by: RichardAH <richard.holland@starstone.co.nz>
* Fix errors for Clang 16: (#4501)
Address issues related to the removal of `std::{u,bi}nary_function` in
C++17 and some warnings with Clang 16. Some warnings appeared with the
upgrade to Apple clang version 14.0.3 (clang-1403.0.22.14.1).
- `std::{u,bi}nary_function` were removed in C++17. They were empty
classes with a few associated types. We already have conditional code
to define the types. Just make it unconditional.
- libc++ checks a cast in an unevaluated context to see if a type
inherits from a binary function class in the standard library, e.g.
`std::equal_to`, and this causes an error when the type privately
inherits from such a class. Change these instances to public
inheritance.
- We don't need a middle-man for the empty base optimization. Prefer to
inherit directly from an empty class than from
`beast::detail::empty_base_optimization`.
- Clang warns when all the uses of a variable are removed by conditional
compilation of assertions. Add a `[[maybe_unused]]` annotation to
suppress it.
- As a drive-by clean-up, remove commented code.
See related work in #4486.
* Fix typo (#4508)
* fix!: Prevent API from accepting seed or public key for account (#4404)
The API would allow seeds (and public keys) to be used in place of
accounts at several locations in the API. For example, when calling
account_info, you could pass `"account": "foo"`. The string "foo" is
treated like a seed, so the method returns `actNotFound` (instead of
`actMalformed`, as most developers would expect). In the early days,
this was a convenience to make testing easier. However, it allows for
poor security practices, so it is no longer a good idea. Allowing a
secret or passphrase is now considered a bug. Previously, it was
controlled by the `strict` option on some methods. With this commit,
since the API does not interpret `account` as `seed`, the option
`strict` is no longer needed and is removed.
Removing this behavior from the API is a [breaking
change](https://xrpl.org/request-formatting.html#breaking-changes). One
could argue that it shouldn't be done without bumping the API version;
however, in this instance, there is no evidence that anyone is using the
API in the "legacy" way. Furthermore, it is a potential security hole,
as it allows users to send secrets to places where they are not needed,
where they could end up in logs, error messages, etc. There's no reason
to take such a risk with a seed/secret, since only the public address is
needed.
Resolves: #3329, #3330, #4337
BREAKING CHANGE: Remove non-strict account parsing (#3330)
* Add nftoken_id, nftoken_ids, offer_id fields for NFTokens (#4447)
Three new fields are added to the `Tx` responses for NFTs:
1. `nftoken_id`: This field is included in the `Tx` responses for
`NFTokenMint` and `NFTokenAcceptOffer`. This field indicates the
`NFTokenID` for the `NFToken` that was modified on the ledger by the
transaction.
2. `nftoken_ids`: This array is included in the `Tx` response for
`NFTokenCancelOffer`. This field provides a list of all the
`NFTokenID`s for the `NFToken`s that were modified on the ledger by
the transaction.
3. `offer_id`: This field is included in the `Tx` response for
`NFTokenCreateOffer` transactions and shows the OfferID of the
`NFTokenOffer` created.
The fields make it easier to track specific tokens and offers. The
implementation includes code (by @ledhed2222) from the Clio project to
extract NFTokenIDs from mint transactions.
* Ensure that switchover vars are initialized before use: (#4527)
Global variables in different TUs are initialized in an undefined order.
At least one global variable was accessing a global switchover variable.
This caused the switchover variable to be accessed in an uninitialized
state.
Since the switchover is always explicitly set before transaction
processing, this bug can not effect transaction processing, but could
effect unit tests (and potentially the value of some global variables).
Note: at the time of this patch the offending bug is not yet in
production.
* Move faulty assert (#4533)
This assert was put in the wrong place, but it only triggers if shards
are configured. This change moves the assert to the right place and
updates it to ensure correctness.
The assert could be hit after the server downloads some shards. It may
be necessary to restart after the shards are downloaded.
Note that asserts are normally checked only in debug builds, so release
packages should not be affected.
Introduced in: #4319 (66627b26cf)
* Fix unaligned load and stores: (#4528) (#4531)
Misaligned load and store operations are supported by both Intel and ARM
CPUs. However, in C++, these operations are undefined behavior (UB).
Substituting these operations with a `memcpy` fixes this UB. The
compiled assembly code is equivalent to the original, so there is no
performance penalty to using memcpy.
For context: The unaligned load and store operations fixed here were
originally introduced in the slab allocator (#4218).
* Add missing includes for gcc 13.1: (#4555)
gcc 13.1 failed to compile due to missing headers. This patch adds the
needed headers.
* Trivial: add comments for NFToken-related invariants (#4558)
* fix node size estimation (#4536)
Fix a bug in the `NODE_SIZE` auto-detection feature in `Config.cpp`.
Specifically, this patch corrects the calculation for the total amount
of RAM available, which was previously returned in bytes, but is now
being returned in units of the system's memory unit. Additionally, the
patch adjusts the node size based on the number of available hardware
threads of execution.
* fix: remove redundant moves (#4565)
- Resolve gcc compiler warning:
AccountObjects.cpp:182:47: warning: redundant move in initialization [-Wredundant-move]
- The std::move() operation on trivially copyable types may generate a
compile warning in newer versions of gcc.
- Remove extraneous header (unused imports) from a unit test file.
* Revert "Fix the fix for std::result_of (#4496)"
This reverts commit cee8409d60.
* Revert "Fix typo (#4508)"
This reverts commit 2956f14de8.
* clang
* [fold] bad merge
* [fold] fix bad merge
- add back filter for ripple state on account_channels
- add back network id test (env auto adds network id in xahau)
* [fold] fix build error
---------
Signed-off-by: ledhed2222 <ledhed2222@users.noreply.github.com>
Co-authored-by: ledhed2222 <ledhed2222@users.noreply.github.com>
Co-authored-by: Nik Bougalis <nikb@bougalis.net>
Co-authored-by: RichardAH <richard.holland@starstone.co.nz>
Co-authored-by: John Freeman <jfreeman08@gmail.com>
Co-authored-by: Mark Travis <mtrippled@users.noreply.github.com>
Co-authored-by: solmsted <steven.olm@gmail.com>
Co-authored-by: drlongle <drlongle@gmail.com>
Co-authored-by: Shawn Xie <35279399+shawnxie999@users.noreply.github.com>
Co-authored-by: Scott Determan <scott.determan@yahoo.com>
Co-authored-by: Ed Hennis <ed@ripple.com>
Co-authored-by: Scott Schurr <scott@ripple.com>
Co-authored-by: Chenna Keshava B S <21219765+ckeshava@users.noreply.github.com>
* fix ns delete owner count
* add a new success code and refactor success checks, limit ns delete operations to 256 entries per txn
---------
Co-authored-by: Richard Holland <richard.holland@starstone.co.nz>
Before this change any non-zero Sequence field was handled as
a non-ticketed transaction, even if a TicketSequence was
present. We learned that this could lead to user confusion.
So the rules are tightened up.
Now if any transaction contains both a non-zero Sequence
field and a TicketSequence field then that transaction
returns a temSEQ_AND_TICKET error code.
The (deprecated) "sign" and "submit" RPC commands are tuned
up so they auto-insert a Sequence field of zero if they see
a TicketSequence in the transaction.
No amendment is needed because this change is going into
the first release that supports the TicketBatch amendment.
The previous error description was focused on keys that are too long,
but this error can occur if the key is too short or does not contain
the correct prefix.
Tickets are a mechanism to allow for the "out-of-order" execution of
transactions on the XRP Ledger.
This commit, if merged, reworks the existing support for tickets and
introduces support for 'ticket batching', completing the feature set
needed for tickets.
The code is gated under the newly-introduced `TicketBatch` amendment
and the `Tickets` amendment, which is not presently active on the
network, is being removed.
The specification for this change can be found at:
https://github.com/xrp-community/standards-drafts/issues/16
The tecUNFUNDED code is actively used when attempting to create payment
channels; the messages incorrectly list it as deprecated.
Meanwhile, the tecUNFUNDED_ADD code actually is an unused legacy code,
dating back to when there was a WalletAdd transactor. The terLAST and
terFUNDS_SPENT codes are also unused legacy codes.
Engine result messages are not part of the binary format and are
documented as subject to change without notice, so this should not
require an amendment nor a new API version.
Align error code table for human readability.
The XRP Ledger utilizes an account model. Unlike systems based on a UTXO
model, XRP Ledger accounts are first-class objects. This design choice
allows the XRP Ledger to offer rich functionality, including the ability
to own objects (offers, escrows, checks, signer lists) as well as other
advanced features, such as key rotation and configurable multi-signing
without needing to change a destination address.
The trade-off is that accounts must be stored on ledger. The XRP Ledger
applies reserve requirements, in XRP, to protect the shared global ledger
from growing excessively large as the result of spam or malicious usage.
Prior to this commit, accounts had been permanent objects; once created,
they could never be deleted.
This commit introduces a new amendment "DeletableAccounts" which, if
enabled, will allow account objects to be deleted by executing the new
"AccountDelete" transaction. Any funds remaining in the account will
be transferred to an account specified in the deletion transaction.
The amendment changes the mechanics of account creation; previously
a new account would have an initial sequence number of 1. Accounts
created after the amendment will have an initial sequence number that
is equal to the ledger in which the account was created.
Accounts can only be deleted if they are not associated with any
obligations (like RippleStates, Escrows, or PayChannels) and if the
current ledger sequence number exceeds the account's sequence number
by at least 256 so that, if recreated, the account can be protected
from transaction replay.
The XRP Ledger allows an account to authorize a secondary key pair,
called a regular key pair, to sign future transactions, while keeping
the master key pair offline.
The regular key pair can be changed as often as desired, without
requiring other changes on the account.
If merged, this commit corrects a minor technical flaw which would
allow an account holder to specify the master key as the account's
new regular key.
The change is controlled by the `fixMasterKeyAsRegularKey` amendment
which, if enabled, will:
1. Prevent specifying an account's master key as the account's
regular key.
2. Prevent the "Disable Master Key" flag from incorrectly affecting
regular keys.
As described in #2314, when an offer executed with `Fill or Kill`
semantics, the server would return `tesSUCCESS` even if the order
couldn't be filled and was aborted. This would require additional
processing of metadata by users to determine the effects of the
transaction.
This commit introduces the `fix1578` amendment which, if enabled,
will cause the server to return the new `tecKILLED` error code
instead of `tesSUCCESS` for `Fill or Kill` orders that could not
be filled.
Additionally, the `fix1578` amendment will prevent the setting of
the `No Ripple` flag on trust lines with negative balance; trying
to set the flag on such a trust line will fail with the new error
code `tecNEGATIVE_BALANCE`.
The lsfDepositAuth flag limits the AccountIDs that can deposit into
the account that has the flag set. The original design only
allowed deposits to complete if the account with the flag set also
signed the transaction that caused the deposit.
The DepositPreauth ledger type allows an account with the
lsfDepositAuth flag set to preauthorize additional accounts.
This preauthorization allows them to sign deposits as well. An
account can add DepositPreauth objects to the ledger (and remove
them as well) using the DepositPreauth transaction.
The six different ranges of TER codes are broken up into six
different enumerations. A template class allows subsets of
these enumerations to be aggregated. This technique allows
verification at compile time that no TEC codes are returned
before the signature is checked.
Conversion between TER instance and integer is provided by
named functions. This makes accidental conversion almost
impossible and makes type abuse easier to spot in the code
base.
Introduce a new ledger type: ltCHECK
Introduce three new transactions that operate on checks:
- "CheckCreate" which adds the check entry to the ledger. The
check is a promise from the source of the check that the
destination of the check may cash the check and receive up to
the SendMax specified on the check. The check may have an
expiration, after which the check may no longer be cashed.
- "CheckCash" is a request by the destination of the check to
transfer a requested amount of funds, up to the check's SendMax,
from the source to the destination. The destination may receive
less than the SendMax due to transfer fees.
When cashing a check, the destination specifies the smallest
amount of funds that will be acceptable. If the transfer
completes and delivers the requested amount, then the check is
considered cashed and removed from the ledger. If enough funds
cannot be delivered, then the transaction fails and the check
remains in the ledger.
Attempting to cash the check after its expiration will fail.
- "CheckCancel" removes the check from the ledger without
transferring funds. Either the check's source or destination
can cancel the check at any time. After a check has expired,
any account can cancel the check.
Facilities related to checks are on the "Checks" amendment.
* If the transaction can't be queued, recover to the open ledger once,
and drop it on the next attempt.
* New result codes for transactions that can not queue.
* Add minimum queue size.
* Remove the obsolete and incorrect SF_RETRY flag.
* fix#2215
Sets a maximum TransferRate value of 100%. Squashes any
requested TransferRate over the limit to the max value.
This change requires an amendment ("fix1201") in rippled.
Adds test functionality for enabling an amendment mid-test.
Additionally, adds tests utilizing varying transfer rates both
with and without the amendment enabled.
Add new functionality to enforce one or more sanity checks (invariants)
on transactions. Add tests for each new invariant check. Allow
for easily adding additional invariant checks in the future.
Also Resolves
-------------
- RIPD-1426
- RIPD-1427
- RIPD-1428
- RIPD-1429
- RIPD-1430
- RIPD-1431
- RIPD-1432
Release Notes
-------------
Creates a new ammendment named "EnforceInvariants" which must be
enabled in order for these new checks to run on each transaction.
Add an amendment to allow gateways to set a "tick size"
for assets they issue. There are no changes unless the
amendment is enabled (since the tick size option cannot
be set).
With the amendment enabled:
AccountSet transactions may set a "TickSize" parameter.
Legal values are 0 and 3-15 inclusive. Zero removes the
setting. 3-15 allow that many decimal digits of precision
in the pricing of offers for assets issued by this account.
For asset pairs with XRP, the tick size imposed, if any,
is the tick size of the issuer of the non-XRP asset. For
asset pairs without XRP, the tick size imposed, if any,
is the smaller of the two issuer's configured tick sizes.
The tick size is imposed by rounding the offer quality
down to nearest tick and recomputing the non-critical
side of the offer. For a buy, the amount offered is
rounded down. For a sell, the amount charged is rounded up.
Gateways must enable a TickSize on their account for this
feature to benefit them.
The primary expected benefit is the elimination of bots
fighting over the tip of the order book. This means:
- Quicker price discovery as outpricing someone by a
microscopic amount is made impossible. Currently
bots can spend hours outbidding each other with no
significant price movement.
- A reduction in offer creation and cancellation spam.
- More offers left on the books as priority means
something when you can't outbid by a microscopic amount.
