Commit Graph

262 Commits

Author SHA1 Message Date
Scott Schurr
0839a202c9 Reduce console noise coming from unit tests:
A few unit tests have historically generated a lot of noise
to the console from log writes.  This noise was not useful
and made it harder to locate actual test failures.

By changing the log level of these tests from
- severities::kError to
- severities::kDisabled
it was possible to remove that noise coming from the logs.
2022-07-17 22:17:24 -07:00
Scott Schurr
80bda7cc48 Introduce fixNFTokenDirV1 amendment:
o Fixes an off-by-one when determining which NFTokenPage an
  NFToken belongs on.
o Improves handling of packed sets of 32 NFTs with
  identical low 96-bits.
o Fixes marker handling by the account_nfts RPC command.
o Tightens constraints of NFTokenPage invariant checks.

Adds unit tests to exercise the fixed cases as well as tests
for previously untested functionality.
2022-05-10 13:34:12 -07:00
Devon White
dac080f1c8 Advance ripple.app.rdb 2022-05-10 13:34:12 -07:00
Richard Holland
01c37fed69 Introduce the ExpandedSignerList amendment:
The amendment increases the maximum sign of an account's signer
list from 8 to 32.

Like all new features, the associated amendment is configured with
a default vote of "no" and server operators will have to vote for
it explicitly if they believe it is useful.
2022-05-10 13:34:12 -07:00
Ed Hennis
04bd5878f1 Track total trustlines and avoid duplications 2022-05-10 13:34:12 -07:00
Nik Bougalis
70779f6850 Introduce NFT support (XLS020) 2022-04-06 13:29:48 -07:00
Edward Hennis
df60e46750 Improve deterministic transaction sorting in TxQ:
* Txs with the same fee level will sort by TxID XORed with the parent
  ledger hash.
* The TxQ is re-sorted after every ledger.
* Attempt to future-proof the TxQ tie breaking test
2022-03-23 23:28:04 -07:00
Nik Bougalis
417cfc2fb0 Adjust pathfinding configuration defaults:
The pathfinding engine built into the code has several configurable
parameters to adjust the depth of the paths indexed and explored.

These parameters can dramatically impact the performance and memory
consumption of a server; higher values can result in resource usage
increasing exponentially.

These default values were decided early and somewhat arbitrarily at
a time when the network and the size of the network state were much
smaller.

This commit adjusts the default values to reduce the depth of paths
to more reasonable levels; unless explicitly overriden, the changes
mean that pathfinding operations will return fewer, shallower paths
than previous versions of the software.
2022-01-12 18:54:03 -08:00
natenichols
e28989638d Enforce account RPC limits by account objects traversed 2021-12-16 11:57:08 -08:00
Edward Hennis
e3acb61d57 Increase TxQ default minimum and target sizes 2021-12-14 17:47:48 -08:00
Edward Hennis
b1c9b134dc Make transaction queue order deterministic:
* Sort by fee level (which is the current behavior) then by transaction
  ID (hash).
* Edge case when the account at the end of the queue submits a higher
  paying transaction to walk backwards and compare against the cheapest
  transaction from a different account.
* Use std::if_any to simplify the JobQueue::isOverloaded loop.
2021-12-14 17:47:48 -08:00
Edward Hennis
ae9930b87d Consensus transaction recovery/deferral completely ignores the TxQ 2021-12-14 17:47:38 -08:00
Nik Bougalis
eb6b79bed7 Adjust default number of threads for JobQueue:
The existing calculation would limit the maximum number of threads
that would be created by default to at most 6; this may have been
reasonable a few years ago, but given both the load on the network
as of today and the increase in the number of CPU cores, the value
should be revisited.

This commit, if merged, changes the default calculation for nodes
that are configured as `large` or `huge` to allow for up to twelve
threads.
2021-11-17 20:52:11 -08:00
Edward Hennis
4a9bd7ed6d Refactor Feature name management and creation:
* Only require adding the new feature names in one place. (Also need to
  increment a counter, but a check on startup will catch that.)
* Allows rippled to have the code to support a given amendment, but
  not vote for it by default. This allows the amendment to be enabled in
  a future version without necessarily amendment blocking these older
  versions.
* The default vote is carried with the amendment name in the list of
  supported amendments.
* The amendment table is constructed with the amendment and default
  vote.
2021-10-06 12:24:19 -07:00
Gregory Tsipenyuk
ea145d12c7 Improve transaction relaying logic:
The existing logic involves every server sending every transaction
that it receives to all its peers (except the one that it received
a transaction from).

This commit instead uses a randomized algorithm, where a node will
randomly select peers to relay a given transaction to, caching the
list of transaction hashes that are not relayed and forwading them
to peers once every second. Peers can then determine whether there
are transactions that they have not seen and can request them from
the node which has them.

It is expected that this feature will further reduce the bandwidth
needed to operate a server.
2021-09-13 15:13:15 -07:00
Edward Hennis
9202197354 Fix nested locks in ValidatorSite:
* Modify relevant unit tests to recreate the problematic scenario.
2021-07-27 11:35:50 -07:00
Scott Schurr
bf75094224 Introduce CheckCashMakesTrustLine amendment:
With this amendment, the CheckCash transaction creates a TrustLine
if needed.  The change is modeled after offer crossing.  And,
similar to offer crossing, cashing a check allows an account to
exceed its trust line limit.
2021-07-27 11:35:50 -07:00
Scott Schurr
8d59c7dd40 Redistribute unit test priorities based on current measurements 2021-07-27 11:35:50 -07:00
Scott Schurr
10e4608ce0 Improve unit test coverage of AccountDelete transactor 2021-06-03 10:58:24 -07:00
Scott Schurr
ff3d2e7c29 Use jtx::offer_cancel more places:
Replace explicit construction of OfferCancel transactions in unit
tests with calls to jtx::offer_cancel().
2021-06-03 10:58:24 -07:00
Scott Schurr
2a7c573dec Fix clang warnings about copies vs references:
A recent version of clang notes a number of places in range
for loops where the code base was making unnecessary copies
or using const lvalue references to extend lifetimes.  This
fixes the places that clang identified.
2021-06-03 10:58:24 -07:00
Scott Schurr
22cc9a254a Maintenance of Offer and PayStrand unit tests 2021-06-03 10:58:24 -07:00
Peng Wang
2eb1c6a396 Enable testing beta RPC API version with config 2021-06-02 13:37:40 -07:00
Miguel Portilla
80c2302fd3 Extend peer shard info 2021-06-01 15:37:15 -07:00
John Freeman
a2a37a928a Redesign stoppable object pattern 2021-06-01 15:36:28 -07:00
manojsdoshi
c10c0be11b Merge master (1.7.2) into develop (1.8.0-b2) 2021-05-26 14:05:55 -07:00
seelabs
1bb99e5d3c Rm some offers where the quality is reduced:
Substantial reductions in an offer's effective quality from its
initial quality may clog offer books.
2021-05-07 15:02:23 -07:00
cdy20
6d82fb83a0 Relational DB interface 2021-04-01 10:38:22 -07:00
Howard Hinnant
9932a19139 Reduce coupling to date.h by calling C++17 chrono functions 2021-03-17 15:02:15 -07:00
Scott Schurr
3b33318dc8 Prefer std::optional over boost:optional:
Some of the boost::optionals must remain for now.  Both
boost::beast and SOCI have interfaces that require
boost::optional.
2021-03-11 14:35:31 -08:00
Nik Bougalis
b4699c3b46 Improve the Byzantine validator detector:
This commit expands the detection capabilities of the Byzantine
validation detector. Prior to this commit, only validators that
were on a server's UNL were monitored. Now, all the validations
that a server receives are passed through the detector.
2021-03-11 14:35:31 -08:00
Nik Bougalis
27d978b891 Simplify SHAMapItem construction:
The existing class offered several constructors which were mostly
unnecessary. This commit eliminates all existing constructors and
introduces a single new one, taking a `Slice`.

The internal buffer is switched from `std::vector` to `Buffer` to
save a minimum of 8 bytes (plus the buffer slack that is inherent
in `std::vector`) per SHAMapItem instance.
2021-03-11 14:35:30 -08:00
Scott Schurr
a2e1a7a84d TicketSequence with non-zero Sequence is an error:
Before this change any non-zero Sequence field was handled as
a non-ticketed transaction, even if a TicketSequence was
present.  We learned that this could lead to user confusion.
So the rules are tightened up.

Now if any transaction contains both a non-zero Sequence
field and a TicketSequence field then that transaction
returns a temSEQ_AND_TICKET error code.

The (deprecated) "sign" and "submit" RPC commands are tuned
up so they auto-insert a Sequence field of zero if they see
a TicketSequence in the transaction.

No amendment is needed because this change is going into
the first release that supports the TicketBatch amendment.
2021-02-12 13:33:26 -08:00
seelabs
36be4856fd Fix path finding for XRP dst and sendmax:
* Fix bug where incorrect max amount was set for XRP
* Fix bug where incorrect source currencies were set when XRP was the dst and a
  sendmax amount was set
2021-01-25 18:49:49 -08:00
seelabs
fe129e8e4f Optimize payment path exploration in flow:
* Use theoretical quality to order the strands
* Do not use strands below the user specified quality limit
* Stop exploring strands (at the current quality iteration) once any strand is non-dry
2021-01-25 18:49:49 -08:00
Peng Wang
7e97bfce10 Implement ledger forward replay 2021-01-25 18:49:49 -08:00
Scott Schurr
44fe0e1fc4 Add support for the 'TicketBatch' amendment:
Support for 'out-of-sequence' transaction execution was introduced
in commit 7724cca384.

The changes in that commit were gated under a feature but there was
no corresponding amendment introduced that would allow the network
to vote on this amendment.

This commit introduces 'TicketBatch' amendment as the amendment
that is associated with the tickets feature. If the amendment is
enabled, it will activate support for tickets.

This commit also removes several workarounds that are no longer
needed in unit tests.
2021-01-08 14:43:06 -05:00
Edward Hennis
4b9d3ca7de Support UNLs with future effective dates:
* Creates a version 2 of the UNL file format allowing publishers to
  pre-publish the next UNL while the current one is still valid.
* Version 1 of the UNL file format is still valid and backward
  compatible.
* Also causes rippled to lock down if it has no valid UNLs, similar to
  being amendment blocked, except reversible.
* Resolves #3548
* Resolves #3470
2021-01-08 12:35:08 -05:00
Howard Hinnant
28ed2b9e69 Persist API-configured voting settings:
Prior to this commit, the amendments that a server would vote in support
of or against could be configured both via the configuration file and
via the command line "feature" command. Changes made in the configuration
file would only be loaded once at server startup and changes made via the
command line take effect immediately but are not persisted across
restarts.

This commit deprecates management of amendments via the configuration
file and stores the relevant information in the `wallet.db` database
file.

1. On startup, the new code parses the configuration file.
2. If the `[veto_amendments]` or `[amendments]` sections are present,
   we check if the `FeatureVotes` table is present in `wallet.db`.
3. If it is not, we create the `FeatureVotes` table and transfer the
   settings from the config file.
4. Proceed normally but only reference the `FeatureVotes` table instead
   of the config file.
5. Warns if the voting table already exists in `wallet.db` and there
   exists voting sections in the config file. The config file is ignored
   in this case.

This change addresses & closes #3366
2020-12-17 05:56:17 -08:00
Nik Bougalis
1bb294afbc Refactor and improve the SHAMap code:
This commit combines a number of cleanups, targeting both the
code structure and the code logic. Large changes include:

 - Using more strongly-typed classes for SHAMap nodes, instead of relying
   on runtime-time detection of class types. This change saves 16 bytes
   of memory per node.
 - Improving the interface of SHAMap::addGiveItem and SHAMap::addItem to
   avoid the need for passing two bool arguments.
 - Documenting the "copy-on-write" semantics that SHAMap uses to
   efficiently track changes in individual nodes.
 - Removing unused code and simplifying several APIs.
 - Improving function naming.
2020-12-04 12:45:13 -08:00
Nik Bougalis
f072469409 Simplify & modernize code:
- Simplify and consolidate code for parsing hex input.
- Replace beast::endian::order with boost::endian::order.
- Simplify CountedObject code.
- Remove pre-C++17 workarounds in favor of C++17 based solutions.
- Improve `base_uint` and simplify its hex-parsing interface by
  consolidating the `SexHex` and `SetHexExact` methods into one
  API: `parseHex` which forces callers to verify the result of
  the operation; as a result some public-facing API endpoints
  may now return errors when passed values that were previously
  accepted.
- Remove the simple fallback implementations of SHA2 and RIPEMD
  introduced to reduce our dependency on OpenSSL. The code is
  slow and rarely, if ever, exercised and we rely on OpenSSL
  functionality for Boost.ASIO as well.
2020-12-04 12:45:12 -08:00
Scott Schurr
17abca1caa Use macros to instantiate most SField instances:
There have been cases in the past where SFields have been defined
in such a way that they did not follow our conventions.  In
particular, the string representation of an SField should match
the in-code name of the SField.

This change leverages the preprocessor to encourage SFields to
be properly constructed.

The suffixes of SField types are changed to be the same as
the suffixes of corresponding SerializedTypeIDs.  This allows
The preprocessor to match types using simple name pasting.

Since the string representation of the SField is part of our
stable API, the name of sfPayChannel was changed to sfChannel.
This change allows sfChannel to follow our conventions while
making no changes to our external API.
2020-12-04 12:44:19 -08:00
Edward Hennis
a3f2196d4e Test cancelling offers that are still in the TxQ 2020-11-18 13:25:27 -08:00
Edward Hennis
068db1f48b Generalize unit test that depends on number of amendments 2020-10-13 11:31:54 -07:00
Nathan Nichols
795de3a75a Load transaction metadata from SQLite:
When processing the `tx` command, we will now load both the transaction
and its metadata directly from SQLite.

Previously the `tx` RPC call was querying SQLite for the transaction
and then separately querying the key-value store for the metadata.
2020-09-17 15:01:37 -07:00
Nathan Nichols
b1d47c65d4 Add public_key to the "vl" method response:
The "/vl" HTTP endpoint can be used to request a particular
UNL from a rippled instance.

This commit, if merged, includes the public key of the requested
list in the response.

This commit fixes #3392
2020-09-01 16:39:00 -07:00
Gregory Tsipenyuk
707868be33 Implement sticky DNS resolution for validator list retrieval:
When attempting to load a validator list from a configured
site, attempt to reuse the last IP that was successfully
used if that IP is still present in the DNS response.

Otherwise, randomly select an IP address from the list of
IPs provided by the DNS system.

This commit fixes #3494.
2020-09-01 10:37:19 -07:00
Gregory Tsipenyuk
9b9f34f881 Optimize relaying of validation and proposal messages:
With few exceptions, servers will typically receive multiple copies
of any given message from its directly connected peers. For servers
with several peers this can impact the processing latency and force
it to do redundant work. Proposal and validation messages are often
relayed with extremely high redundancy.

This commit, if merged, introduces experimental code that attempts
to optimize the relaying of proposals and validations by allowing
servers to instruct their peers to "squelch" delivery of selected
proposals and validations. Servers making squelching decisions by
a process that evaluates the fitness and performance of a given
server and randomly selecting a subset of the best candidates.

The experimental code is presently disabled and must be explicitly
enabled by server operators that wish to test it.
2020-09-01 09:07:32 -07:00
Scott Schurr
7724cca384 Implement enhanced Ticket support:
Tickets are a mechanism to allow for the "out-of-order" execution of
transactions on the XRP Ledger.

This commit, if merged, reworks the existing support for tickets and
introduces support for 'ticket batching', completing the feature set
needed for tickets.

The code is gated under the newly-introduced `TicketBatch` amendment
and the `Tickets` amendment, which is not presently active on the
network, is being removed.

The specification for this change can be found at:
https://github.com/xrp-community/standards-drafts/issues/16
2020-09-01 08:58:57 -07:00
Peng Wang
12c0e8148b Improve naming of fields associated with the NegativeUNL 2020-08-06 10:05:43 -07:00