Commit Graph

857 Commits

Author SHA1 Message Date
seelabs
92d35e54c7 Switch from C++17 to C++20 2022-08-25 08:38:28 -07:00
Nik Bougalis
59326bbbc5 Introduce the NonFungibleTokensV1_1 amendment:
The XLS-20 implementation contained two bugs that would require the
introduction of amendments. This complicates the adoption of XLS-20
by requiring a staggered amendment activation, first of the two fix
amendments, followed by the `NonFungibleTokensV1` amendment.

After consideration, the consensus among node operators is that the
process should be simplified by the introduction of a new amendment
that, if enabled, would behaves as if the `NonFungibleTokensV1` and
the two fix amendments (`fixNFTokenDirV1` and `fixNFTokenNegOffer`)
were activated at once.

This commit implements this proposal; it does not introduce any new
functionality or additional features, above and beyond that offered
by the existing amendments.
2022-07-17 22:17:33 -07:00
Scott Schurr
8266d9d598 Correct a technical flaw with NFT offers:
The existing code would, incorrectly, allow negative amounts in offers
for non-fungible tokens. Such offers would be handled very differently
depending on the context: a direct offer would fail with an error code
indicating an internal processing error, whereas brokered offers would
improperly succeed.

This commit introduces the `fixNFTokenNegOffer` amendment that detects
such offers during creation and returns an appropriate error code.

The commit also extends the existing code to allow for buy offers that
contain a `Destination` field, so that a specific broker can be set in
the offer.
2022-07-17 22:17:24 -07:00
Scott Schurr
0839a202c9 Reduce console noise coming from unit tests:
A few unit tests have historically generated a lot of noise
to the console from log writes.  This noise was not useful
and made it harder to locate actual test failures.

By changing the log level of these tests from
- severities::kError to
- severities::kDisabled
it was possible to remove that noise coming from the logs.
2022-07-17 22:17:24 -07:00
Chenna Keshava B S
d632f9f6c8 Properly handle incorrect port numbers in parseURL (fixes #4200) 2022-07-11 09:56:39 -07:00
Scott Schurr
80bda7cc48 Introduce fixNFTokenDirV1 amendment:
o Fixes an off-by-one when determining which NFTokenPage an
  NFToken belongs on.
o Improves handling of packed sets of 32 NFTs with
  identical low 96-bits.
o Fixes marker handling by the account_nfts RPC command.
o Tightens constraints of NFTokenPage invariant checks.

Adds unit tests to exercise the fixed cases as well as tests
for previously untested functionality.
2022-05-10 13:34:12 -07:00
Devon White
dac080f1c8 Advance ripple.app.rdb 2022-05-10 13:34:12 -07:00
Devon White
767dd4ff3f Address failures in DatabaseDownloader unittest 2022-05-10 13:34:12 -07:00
Richard Holland
01c37fed69 Introduce the ExpandedSignerList amendment:
The amendment increases the maximum sign of an account's signer
list from 8 to 32.

Like all new features, the associated amendment is configured with
a default vote of "no" and server operators will have to vote for
it explicitly if they believe it is useful.
2022-05-10 13:34:12 -07:00
Ed Hennis
04bd5878f1 Track total trustlines and avoid duplications 2022-05-10 13:34:12 -07:00
Ed Hennis
e836375d99 Don't load trust lines that can't participate in path finding
* "A path is considered invalid if and only if it enters and exits an
  address node through trust lines where No Ripple has been enabled for
  that address." (https://xrpl.org/rippling.html#specifics)
* When loading trust lines for an account "Alice" which was reached
  via a trust line that has the No Ripple flag set on Alice's side, do
  not use or cache any of Alice's trust lines which have the No Ripple
  flag set on Alice's side. For typical "end-user" accounts, this will
  return no trust lines.
2022-05-10 13:34:12 -07:00
Nik Bougalis
70779f6850 Introduce NFT support (XLS020) 2022-04-06 13:29:48 -07:00
Nik Bougalis
c7e6803956 Simplify LedgerMaster:
- Eliminate `tune` member function and allow `LedgerHistory`
  to fully initialize itself.
2022-03-30 15:16:27 -07:00
Nik Bougalis
48803a48af Improve ledger-fetching logic:
When fetching ledgers, the existing code would isolate the peer
that sent the most useful responses and issue follow up queries
only to that peer.

This commit increases the query aggressiveness, and changes the
mechanism used to select which peers to issue follow-up queries
to so as to more evenly spread the load along those peers which
provided useful responses.
2022-03-29 16:16:34 -07:00
Edward Hennis
df60e46750 Improve deterministic transaction sorting in TxQ:
* Txs with the same fee level will sort by TxID XORed with the parent
  ledger hash.
* The TxQ is re-sorted after every ledger.
* Attempt to future-proof the TxQ tie breaking test
2022-03-23 23:28:04 -07:00
Edward Hennis
e7e672c3f8 Incremental improvements to path finding memory usage:
* Abort background path finding when closed or disconnected
* Exit pathfinding job thread if there are no requests left
* Don't bother creating the path find job if there are no requests
* Refactor to remove circular dependency between InfoSub and PathRequest
2022-03-23 23:28:04 -07:00
Richard Holland
a07a729e3d Reserve field codes for Hooks:
In order to preserve the Hooks ABI, it is important that field
values used for hooks be stable going forward.

This commit reserves the required codes so that they will not
be repurposed before Hooks can be proposed for inclusion in
the codebase.
2022-03-23 10:48:39 -07:00
Michael Legleux
a01cadbfd5 Move Beast & fix #include paths 2022-03-04 14:16:59 -08:00
CJ Cobb
11ca9a946c Add successor information to clio ETL messages
* Allow clio to ask for object successors and predecessors from rippled
* Add lower_bound and last_below to SHAMap
2022-03-04 13:38:48 -08:00
Mark Travis
8f82b62e0d Use CIDR notation for admin and secure_gateway 2022-03-01 11:27:50 -08:00
John Freeman
c2a08a1f26 Simplify the Job Queue:
This is a refactor aimed at cleaning up and simplifying the existing
job queue.

As of now, all jobs are cancelled at the same time and in the same
way, so this commit removes the per-job cancellation token. If the
need for such support is demonstrated, support can be re-added.

* Revise documentation for ClosureCounter and Workers.
* Simplify code, removing unnecessary function arguments and
  deduplicating expressions
* Restructure job handlers to no longer need to pass a job's
  handle to the job.
2022-03-01 11:25:03 -08:00
Michael Legleux
95ecf296ad Install the protocol buffer deps during smoketests 2022-02-07 12:38:40 -08:00
Nik Bougalis
289bc0afd9 Improve handling of endpoints during peer discovery 2022-02-07 10:33:02 -08:00
Nik Bougalis
417cfc2fb0 Adjust pathfinding configuration defaults:
The pathfinding engine built into the code has several configurable
parameters to adjust the depth of the paths indexed and explored.

These parameters can dramatically impact the performance and memory
consumption of a server; higher values can result in resource usage
increasing exponentially.

These default values were decided early and somewhat arbitrarily at
a time when the network and the size of the network state were much
smaller.

This commit adjusts the default values to reduce the depth of paths
to more reasonable levels; unless explicitly overriden, the changes
mean that pathfinding operations will return fewer, shallower paths
than previous versions of the software.
2022-01-12 18:54:03 -08:00
Mark Travis
7c12f01358 Parallel ledger loader & I/O performance improvements:
- Only duplicate records from archive to writable during online_delete.
- Log duration of nodestore reads.
- Include nodestore counters in perf_log output.
- Remove gratuitous nodestore activity counting.
- Report initial sync duration in server_info and perfLog.
- Report state_accounting in perfLog.
- Make state_accounting durations more accurate.
- Parallel ledger loader.
- Config parameter to load ledgers on start.
2022-01-10 15:29:21 -08:00
natenichols
e28989638d Enforce account RPC limits by account objects traversed 2021-12-16 11:57:08 -08:00
Mark Travis
db720a59e4 Log resource limit disconnections. 2021-12-15 11:26:13 -08:00
Edward Hennis
e3acb61d57 Increase TxQ default minimum and target sizes 2021-12-14 17:47:48 -08:00
Edward Hennis
b1c9b134dc Make transaction queue order deterministic:
* Sort by fee level (which is the current behavior) then by transaction
  ID (hash).
* Edge case when the account at the end of the queue submits a higher
  paying transaction to walk backwards and compare against the cheapest
  transaction from a different account.
* Use std::if_any to simplify the JobQueue::isOverloaded loop.
2021-12-14 17:47:48 -08:00
Edward Hennis
ae9930b87d Consensus transaction recovery/deferral completely ignores the TxQ 2021-12-14 17:47:38 -08:00
Scott Schurr
ad805eb95b Improve handling of the X-Forwarded-For and Forwarded HTTP headers 2021-12-14 17:47:04 -08:00
Edward Hennis
5050b366d9 Return correct error code for node_to_shard command errors 2021-11-22 18:40:32 -08:00
Mark Travis
19018e8959 Introduce partitioned unordered maps:
This commit implements partitioned unordered maps and makes it possible
to traverse such a map in parallel, allowing for more efficient use of
CPU resources.

The `CachedSLEs`, `TaggedCache`, and `KeyCache` classes make use of the
new functionality, which should improve performance.
2021-11-18 14:38:35 -05:00
Nik Bougalis
eb6b79bed7 Adjust default number of threads for JobQueue:
The existing calculation would limit the maximum number of threads
that would be created by default to at most 6; this may have been
reasonable a few years ago, but given both the load on the network
as of today and the increase in the number of CPU cores, the value
should be revisited.

This commit, if merged, changes the default calculation for nodes
that are configured as `large` or `huge` to allow for up to twelve
threads.
2021-11-17 20:52:11 -08:00
Peng Wang
e2a42184b9 Implement tx stream with history 2021-10-21 12:05:48 -07:00
Devon White
00a4c3a478 Implement node-to-shard RPC control 2021-10-19 16:27:30 -07:00
seelabs
b6ed50eb03 Remove unused variable:
Clang 13 warns about `nbytes` being unused. This patch removes this unused variable.
2021-10-06 13:17:10 -07:00
Edward Hennis
4a9bd7ed6d Refactor Feature name management and creation:
* Only require adding the new feature names in one place. (Also need to
  increment a counter, but a check on startup will catch that.)
* Allows rippled to have the code to support a given amendment, but
  not vote for it by default. This allows the amendment to be enabled in
  a future version without necessarily amendment blocking these older
  versions.
* The default vote is carried with the amendment name in the list of
  supported amendments.
* The amendment table is constructed with the amendment and default
  vote.
2021-10-06 12:24:19 -07:00
Scott Schurr
c50d166c23 Add ripple::Expected which simulates std::expected:
Also integrates use of ripple::Expected into the code base.
2021-09-13 15:13:15 -07:00
seelabs
54e5d5fc35 Add missing include 2021-09-13 15:13:15 -07:00
CJ Cobb
3fb60a89a3 Validate data size before calling uint256::fromVoid 2021-09-13 15:13:15 -07:00
Gregory Tsipenyuk
ea145d12c7 Improve transaction relaying logic:
The existing logic involves every server sending every transaction
that it receives to all its peers (except the one that it received
a transaction from).

This commit instead uses a randomized algorithm, where a node will
randomly select peers to relay a given transaction to, caching the
list of transaction hashes that are not relayed and forwading them
to peers once every second. Peers can then determine whether there
are transactions that they have not seen and can request them from
the node which has them.

It is expected that this feature will further reduce the bandwidth
needed to operate a server.
2021-09-13 15:13:15 -07:00
Nik Bougalis
af5f28cbf8 Cleanup LedgerEntryType & TxType:
This commit removes the `ltINVALID` pseudo-type identifier from
`LedgerEntryType` and the `ttINVALID` pseudo-type identifier from
`TxType` and includes several small additional improvements that
help to simplify the code base.

It also improves the documentation `LedgerEntryType` and `TxType`,
which was all over the place, and highlights some important caveats
associated with making changes to the ledger and transaction type
identifiers.

The commit also adds a safety check to the `KnownFormats<>` class,
that will catch the the accidental reuse of format identifiers.
Ideally, this should be done at compile time but C++ does not (yet?)
allow for the sort of introspection that would enable this.
2021-09-13 15:13:14 -07:00
Nik Bougalis
234b754038 Eliminate code duplication and improve documentation:
The legacy functions `cdirFirst` and `dirFirst` were mostly
identical; the differences were only type-related. The same
situation existed with `cdirNext` and `dirNext`.

This commit removes the duplicated code by introducing new
template functions that abstract away the differences that
are present between each pair of functions.

This commit also improves the naming of function arguments,
helping to elucidate their purpose & use and to make the
code self-documenting.
2021-09-13 15:13:13 -07:00
Nik Bougalis
c231adf324 Minor cleanups in offer processing code 2021-09-09 11:26:06 -07:00
Fraser Waters
10d73655bc Move signer_list in JSON response for api_version 2:
See https://github.com/ripple/xrpl-dev-portal/issues/938 for context.
2021-07-27 11:35:50 -07:00
Edward Hennis
9202197354 Fix nested locks in ValidatorSite:
* Modify relevant unit tests to recreate the problematic scenario.
2021-07-27 11:35:50 -07:00
seelabs
cd27b5f2bd Some code cleanups tagged by static analysis 2021-07-27 11:35:50 -07:00
Nik Bougalis
3752234161 Report additional fields in validation stream:
The HardenedValidations amendment introduces additional fields
in validations:

- `sfValidatedHash`, if present, is the hash the of last ledger that
  the validator considers to be fully validated.
- `sfCookie`, if present, is a 64-bit cookie (the default
  implementation selects it randomly at startup but other
  implementations are possible), which can be used to improve the
  detection and classification of duplicate validations.
- `sfServerVersion`, if present, reports the version of the software
  that the validator is running. By surfacing this information,
  server operators gain additional insight about variety of software
  on the network.

If merged, this commit fixes #3797 by adding the fields to the
`validations` stream as shown below:

- `sfValidateHash` as `validated_hash`: a 256-bit hex string;
- `sfCookie` as `cookie`: a 64-bit integer as a string; and
- `sfServerVersion` as `server_version`: a 64-bit integer as
  a string.
2021-07-27 11:35:50 -07:00
Scott Schurr
bf75094224 Introduce CheckCashMakesTrustLine amendment:
With this amendment, the CheckCash transaction creates a TrustLine
if needed.  The change is modeled after offer crossing.  And,
similar to offer crossing, cashing a check allows an account to
exceed its trust line limit.
2021-07-27 11:35:50 -07:00