13015 Commits

Author SHA1 Message Date
Nik Bougalis
433feade5d Automatically determine the node size:
The `[node_size]` configuration parameter is used to tune various
parameters based on the hardware that the code is running on. The
parameter can take five distinct values: `tiny`, `small`, `medium`,
`large` and `huge`.

The default value in the code is `tiny` but the default configuration
file sets the value to `medium`. This commit attempts to detect the
amount of RAM on the system and adjusts the node size default value
based on the amount of RAM and the number of hardware execution
threads on the system.

The decision matrix currently used is:

|         |   1  | 2 or 3 |   ≥ 4  |
|:-------:|:----:|:------:|:------:|
|  > ~8GB | tiny |   tiny |   tiny |
| > ~12GB | tiny |  small |  small |
| > ~16GB | tiny |  small | medium |
| > ~24GB | tiny |  small |  large |
| > ~32GB | tiny |  small |   huge |

Some systems exclude memory reserved by the the hardware, the kernel
or the underlying hypervisor so the automatic detection code may end
up determining the node_size to be one less than "appropriate" given
the above table.

The detection algorithm is simplistic and does not take into account
other relevant factors. Therefore, for production-quality servers it
is recommended that server operators examine the system holistically
and determine what the appropriate size is instead of relying on the
automatic detection code.

To aid server operators, the node size will now be reported in the
`server_info` API as `node_size` when the command is invoked in
'admin' mode.
2021-06-03 10:58:24 -07:00
Scott Schurr
10e4608ce0 Improve unit test coverage of AccountDelete transactor 2021-06-03 10:58:24 -07:00
Scott Schurr
ff3d2e7c29 Use jtx::offer_cancel more places:
Replace explicit construction of OfferCancel transactions in unit
tests with calls to jtx::offer_cancel().
2021-06-03 10:58:24 -07:00
Scott Schurr
7822a28c87 Replace dirAdd() calls with dirAppend() or dirInsert(): 2021-06-03 10:58:24 -07:00
Scott Schurr
dcba79be48 Only warn on non-pseudo transactions without accounts:
Closes #3153
2021-06-03 10:58:24 -07:00
Scott Schurr
2a7c573dec Fix clang warnings about copies vs references:
A recent version of clang notes a number of places in range
for loops where the code base was making unnecessary copies
or using const lvalue references to extend lifetimes.  This
fixes the places that clang identified.
2021-06-03 10:58:24 -07:00
Scott Schurr
22cc9a254a Maintenance of Offer and PayStrand unit tests 2021-06-03 10:58:24 -07:00
Scott Schurr
09ae9168ca Remove some "Env::close" noise from unit test output 2021-06-03 10:58:24 -07:00
CJ Cobb
9eb9b8f631 Fix reporting mode build issue
* Add isCaughtUp() as a member function of RelationalDBInterfacePostgres
2021-06-03 10:58:24 -07:00
CJ Cobb
6298daba1a Support dns in gRPC connection for reporting ETL 2021-06-03 10:58:24 -07:00
Peng Wang
2eb1c6a396 Enable testing beta RPC API version with config 2021-06-02 13:37:40 -07:00
Peng Wang
7717056cf2 Add higher bound to validations to keep 2021-06-02 13:37:40 -07:00
seelabs
9fd5cd303d Do not use __func__ inside lambdas 2021-06-02 13:37:40 -07:00
seelabs
04ff6249d5 Support boost 1.76:
* remove unused `std::hash` specialization
* Use `std::chrono::floor` instead of `floor`
2021-06-02 13:37:40 -07:00
seelabs
fa9ecae2d6 Update doc specifying min gcc version:
* resolves https://github.com/ripple/rippled/issues/3782
* gcc 8 is required for the charconv include file
2021-06-02 13:37:40 -07:00
Devon White
62d2b76fa8 Advance shard store unittests 2021-06-02 13:37:30 -07:00
Devon White
d95aab1139 Add lookup DBs to the Relational DB Interface
* Create SQLite database for mapping transaction IDs to shard indexes
* Create SQLite database for mapping ledger hashes to shard indexes
* Create additional test cases for the shard database
2021-06-01 15:37:15 -07:00
Miguel Portilla
80c2302fd3 Extend peer shard info 2021-06-01 15:37:15 -07:00
Devon White
38f954fd46 Improve NodeStore to ShardStore imports
* Run the import process in a background thread
* Prevent online_delete from removing ledgers pending import
2021-06-01 15:37:15 -07:00
John Freeman
14b2f27c3e Remove dead code 2021-06-01 15:37:04 -07:00
John Freeman
a2a37a928a Redesign stoppable object pattern 2021-06-01 15:36:28 -07:00
manojsdoshi
c10c0be11b Merge master (1.7.2) into develop (1.8.0-b2) 2021-05-26 14:05:55 -07:00
manojsdoshi
34ee4ca0cb Set version to 1.7.2 1.7.2 2021-05-19 12:22:03 -07:00
Edward Hennis
30fd45890b Create comparator wrapper classes for MSVC 2019 build issues:
* The std::less and std::equal_to comparators have [[nodiscard]], which
  conflicts with boost::bimap validations.
2021-05-17 10:43:56 -07:00
Edward Hennis
36fe1966c3 Update version of vcpkg for Windows builds 2021-05-17 10:43:56 -07:00
seelabs
1bb99e5d3c Rm some offers where the quality is reduced:
Substantial reductions in an offer's effective quality from its
initial quality may clog offer books.
2021-05-07 15:02:23 -07:00
CJ Cobb
430802c1cf Add load_factor to server_info in reporting mode
* load_factor was missing from server_info when the server was running in
  reporting mode. Now, the reporting mode server calls server_info on the p2p
  node, and propagates the load_factor back to the client.
2021-05-07 14:58:29 -07:00
Nik Bougalis
9106a06579 Set version to 1.7.1 2021-04-01 13:40:41 -07:00
Nik Bougalis
79e69da364 Adjust OpenSSL defaults and mitigate CVE-2021-3499:
In order to effectively mitigate CVE-2021-3499 even when compiling
against versions of OpenSSL prior to 1.1.1k, this commit:

1) requires use of TLS 1.2 or later. Note that both TLS 1.0 and
   TLS 1.1 have been officially deprecated for over a year.
2) disables renegotiation support for TLS 1.2 connections.

Lastly, this commit also changes the default list of ciphers that
the server offers, limiting it only to ciphers that are part of
TLS 1.2.
2021-04-01 13:40:40 -07:00
CJ Cobb
73116297aa Properly encode results from the tx RPC command:
The `tx` command supports output in both "text" and "binary" modes,
controlled by the binary flag. For more details on the command and
the possible arguments, please see: https://xrpl.org/tx.html.

The existing handler would incorrectly deal with metadata when in
binary mode. This commit corrects this issue, ensuring that the
metadata is properly encoded, depending on the mode.
2021-04-01 13:21:02 -07:00
CJ Cobb
8579eb0c19 Maintain compatibility for forwarded RPC responses:
Typically, an RPC response contains a `result` field, which
contains details about the operation performed. For ease of
parsing, forwarded responses must look like a non-forwarded
response.

In some instances the response was incorrectly composed, so
that the actual `result` object would be encapsulated by an
outer `result` object, breaking existing code.

This commit, addresses this issue and correctly "folds" the
`result` field, ensuring a consistent schema for responses.
2021-04-01 10:40:05 -07:00
Mark Travis
9c8caddc5a Support HTTP health check in reporting mode. 2021-04-01 10:40:05 -07:00
manojsdoshi
2913847925 Set version to 1.8.0-b2 2021-04-01 10:39:22 -07:00
Crypto Brad Garlinghouse
cf8438fe1d Remove obsolete URLs and references to Ripple 2021-04-01 10:38:47 -07:00
cdy20
6d82fb83a0 Relational DB interface 2021-04-01 10:38:22 -07:00
CJ Cobb
207e1730e9 Serialize metadata as blob in tx RPC when binary flag is true 2021-04-01 10:37:46 -07:00
J. Scott Branson
f0424fe7dd Document parameters in example configuration file 2021-04-01 10:37:38 -07:00
Alloy Networks
2e456a835d Remove legacy protocol version (1.2) 2021-04-01 10:37:17 -07:00
Edward Hennis
ab9039e77d Create comparator wrapper classes for MSVC 2019 build issues:
* The std::less and std::equal_to comparators have [[nodiscard]], which
  conflicts with boost::bimap validations.
2021-03-18 14:12:41 -07:00
Howard Hinnant
9932a19139 Reduce coupling to date.h by calling C++17 chrono functions 2021-03-17 15:02:15 -07:00
Howard Hinnant
64e4a89470 Clarify the safety of NetClock::time_point arithmetic:
* NetClock::rep is uint32_t and can be error-prone when
  used with subtraction.
* Fixes #3656
2021-03-17 15:01:52 -07:00
manojsdoshi
9d89d4c188 Set version to 1.8.0-b1 2021-03-11 16:21:51 -08:00
Edward Hennis
b2bf2b6e6b Add instructions for resolving validation failures:
* Add instructions to the workflow at the point where the failure would
  occur, which is where someone experiencing a failure is most likely to
  look. To keep things simple, the instructions are always printed. The
  assumption is that if the job succeeds, nobody is likely to look
  anyway.
* Provides the diff of the failure as an artifact, so the user can apply
  it directly to their repo.
* Also update the levelization/README.md to clarify the levels a little
  bit.
2021-03-11 14:35:31 -08:00
Scott Schurr
3b33318dc8 Prefer std::optional over boost:optional:
Some of the boost::optionals must remain for now.  Both
boost::beast and SOCI have interfaces that require
boost::optional.
2021-03-11 14:35:31 -08:00
Scott Schurr
85307b29d0 Add constexpr constructor for base_uint 2021-03-11 14:35:31 -08:00
John Northrup
95426efb8a Update to use Ripple Artifactory for docker cache
This updates the build process to use the local Artifactory server as a docker image cache to avoid being rate limited by docker hub during the build process.
2021-03-11 14:35:31 -08:00
CJ Cobb
3e2b568ef9 Add list of dependencies for building reporting mode 2021-03-11 14:35:31 -08:00
Nik Bougalis
a06525649d Use libsecp256k1 instead of OpenSSL for key derivation:
While most of the code associated with secp256k1 operations had
been migrated to libsecp256k1, the deterministic key derivation
code was still using calls to OpenSSL.

If merged, this commit replaces the OpenSSL-based routines with
new libsecp256k1-based implementations. No functional change is
expected and the change should be transparent.

This commit also removes several support classes and utility
functions that wrapped or adapted various OpenSSL types that
are no longer needed.

A tip of the hat to the original author of this truly superb
library, Dr. Pieter Wuille, and to all other contributors.
2021-03-11 14:35:31 -08:00
Nik Bougalis
b4699c3b46 Improve the Byzantine validator detector:
This commit expands the detection capabilities of the Byzantine
validation detector. Prior to this commit, only validators that
were on a server's UNL were monitored. Now, all the validations
that a server receives are passed through the detector.
2021-03-11 14:35:31 -08:00
Nik Bougalis
27d978b891 Simplify SHAMapItem construction:
The existing class offered several constructors which were mostly
unnecessary. This commit eliminates all existing constructors and
introduces a single new one, taking a `Slice`.

The internal buffer is switched from `std::vector` to `Buffer` to
save a minimum of 8 bytes (plus the buffer slack that is inherent
in `std::vector`) per SHAMapItem instance.
2021-03-11 14:35:30 -08:00