Commit Graph

165 Commits

Author SHA1 Message Date
Scott Schurr
ff3d2e7c29 Use jtx::offer_cancel more places:
Replace explicit construction of OfferCancel transactions in unit
tests with calls to jtx::offer_cancel().
2021-06-03 10:58:24 -07:00
Scott Schurr
2a7c573dec Fix clang warnings about copies vs references:
A recent version of clang notes a number of places in range
for loops where the code base was making unnecessary copies
or using const lvalue references to extend lifetimes.  This
fixes the places that clang identified.
2021-06-03 10:58:24 -07:00
Scott Schurr
09ae9168ca Remove some "Env::close" noise from unit test output 2021-06-03 10:58:24 -07:00
Peng Wang
2eb1c6a396 Enable testing beta RPC API version with config 2021-06-02 13:37:40 -07:00
Miguel Portilla
80c2302fd3 Extend peer shard info 2021-06-01 15:37:15 -07:00
Devon White
38f954fd46 Improve NodeStore to ShardStore imports
* Run the import process in a background thread
* Prevent online_delete from removing ledgers pending import
2021-06-01 15:37:15 -07:00
John Freeman
a2a37a928a Redesign stoppable object pattern 2021-06-01 15:36:28 -07:00
cdy20
6d82fb83a0 Relational DB interface 2021-04-01 10:38:22 -07:00
CJ Cobb
207e1730e9 Serialize metadata as blob in tx RPC when binary flag is true 2021-04-01 10:37:46 -07:00
Scott Schurr
3b33318dc8 Prefer std::optional over boost:optional:
Some of the boost::optionals must remain for now.  Both
boost::beast and SOCI have interfaces that require
boost::optional.
2021-03-11 14:35:31 -08:00
Scott Schurr
85307b29d0 Add constexpr constructor for base_uint 2021-03-11 14:35:31 -08:00
CJ Cobb
27543170d0 Add Reporting Mode
* Add a new operating mode to rippled called reporting mode
* Add ETL mechanism for a reporting node to extract data from a p2p node
* Add new gRPC methods to faciliate ETL
* Use Postgres in place of SQLite in reporting mode
* Add Cassandra as a nodestore option
* Update logic of RPC handlers when running in reporting mode
* Add ability to forward RPCs to a p2p node
2021-01-20 11:30:03 -08:00
Nik Bougalis
70bacb349e Improve manifest relaying and domain reporting:
- The changes to manifest relaying introduced with commit f74b469e68
  will cause newly accepted manifests to be sent back to the peer from
  which they were received. This no longer happens: a newly accepted
  manifest is never sent back to the peer we received it from.
- When encountering a manifest without a domain set, the `manifest` and
  `validator_info` commands would include an empty string as the domain
  associated with the manifest. This no longer happens: if a domain is
  not present, the `domain` field will not be.
2021-01-20 10:53:24 -08:00
Scott Schurr
44fe0e1fc4 Add support for the 'TicketBatch' amendment:
Support for 'out-of-sequence' transaction execution was introduced
in commit 7724cca384.

The changes in that commit were gated under a feature but there was
no corresponding amendment introduced that would allow the network
to vote on this amendment.

This commit introduces 'TicketBatch' amendment as the amendment
that is associated with the tickets feature. If the amendment is
enabled, it will activate support for tickets.

This commit also removes several workarounds that are no longer
needed in unit tests.
2021-01-08 14:43:06 -05:00
Edward Hennis
4b9d3ca7de Support UNLs with future effective dates:
* Creates a version 2 of the UNL file format allowing publishers to
  pre-publish the next UNL while the current one is still valid.
* Version 1 of the UNL file format is still valid and backward
  compatible.
* Also causes rippled to lock down if it has no valid UNLs, similar to
  being amendment blocked, except reversible.
* Resolves #3548
* Resolves #3470
2021-01-08 12:35:08 -05:00
JoelKatz
6dd3d825c8 Adjust time resolution of genesis ledger:
Due to some quirky emergent behavior, the server can't really begin
synching until twice the default close time resolution of the genesis
ledger, which is 30 seconds, has passed. In effect, this causes a one
minute delay.

This commit adjusts the default close time resolution down to the
minimum allowed resoluion of 10 seconds, so the corresponding delay
is reduced by 67% down to 20 seconds. This should be enough time to
ensure the server has reasonable connectivity without unduly delaying
initial synch times.
2020-12-18 15:23:26 -08:00
Nik Bougalis
f072469409 Simplify & modernize code:
- Simplify and consolidate code for parsing hex input.
- Replace beast::endian::order with boost::endian::order.
- Simplify CountedObject code.
- Remove pre-C++17 workarounds in favor of C++17 based solutions.
- Improve `base_uint` and simplify its hex-parsing interface by
  consolidating the `SexHex` and `SetHexExact` methods into one
  API: `parseHex` which forces callers to verify the result of
  the operation; as a result some public-facing API endpoints
  may now return errors when passed values that were previously
  accepted.
- Remove the simple fallback implementations of SHA2 and RIPEMD
  introduced to reduce our dependency on OpenSSL. The code is
  slow and rarely, if ever, exercised and we rely on OpenSSL
  functionality for Boost.ASIO as well.
2020-12-04 12:45:12 -08:00
Scott Schurr
17abca1caa Use macros to instantiate most SField instances:
There have been cases in the past where SFields have been defined
in such a way that they did not follow our conventions.  In
particular, the string representation of an SField should match
the in-code name of the SField.

This change leverages the preprocessor to encourage SFields to
be properly constructed.

The suffixes of SField types are changed to be the same as
the suffixes of corresponding SerializedTypeIDs.  This allows
The preprocessor to match types using simple name pasting.

Since the string representation of the SField is part of our
stable API, the name of sfPayChannel was changed to sfChannel.
This change allows sfChannel to follow our conventions while
making no changes to our external API.
2020-12-04 12:44:19 -08:00
Devon White
cf5ca9a5cf Download queued shards only if there is space for them all 2020-11-19 10:57:13 -08:00
Nathan Nichols
8116b569c7 Refactor AccountLinesRPC no_ripple_default logic 2020-09-17 15:01:37 -07:00
Crypto Brad Garlinghouse
831e03ad2a Update Base58 codec and remove Bitcoin support:
Use C++17 constant expressions to calculate the inverse
alphabet map at compile time instead of at runtime.

Remove support for encoding & decoding tokens using the
Bitcoin alphabet.
2020-09-01 16:39:00 -07:00
Devon White
6c268a3e9c Allow multiple paths for shard storage:
* Distinguish between recent and historical shards
* Allow multiple storage paths for historical shards
* Add documentation for this feature
* Add unit tests
2020-09-01 16:39:00 -07:00
Gregory Tsipenyuk
801b1580f5 Improve handling of RPC ledger_index argument:
Some RPC commands return `ledger_index` as a quoted numeric
string. This change allows the returned value to be directly
copied and used for follow-on RPC commands.

This commit fixes #3533
2020-09-01 16:39:00 -07:00
Scott Schurr
7724cca384 Implement enhanced Ticket support:
Tickets are a mechanism to allow for the "out-of-order" execution of
transactions on the XRP Ledger.

This commit, if merged, reworks the existing support for tickets and
introduces support for 'ticket batching', completing the feature set
needed for tickets.

The code is gated under the newly-introduced `TicketBatch` amendment
and the `Tickets` amendment, which is not presently active on the
network, is being removed.

The specification for this change can be found at:
https://github.com/xrp-community/standards-drafts/issues/16
2020-09-01 08:58:57 -07:00
Peng Wang
12c0e8148b Improve naming of fields associated with the NegativeUNL 2020-08-06 10:05:43 -07:00
seelabs
d317060ae4 Fix a race condition with TrustedPublisherServer:
There was a race condition in `on_accept` where the object's destructor
could run while `on_accept` was called.

This patch ensures that if `on_accept` is called then the object remains
valid for the duration of the call.
2020-08-06 09:56:04 -07:00
Peng Wang
706ca874b0 Implement negative UNL functionality:
This change can help improve the liveness of the network during periods of network
instability, by allowing the network to track which validators are presently not online
and to disregard them for the purposes of quorum calculations.
2020-06-30 09:15:37 -07:00
Gregory Tsipenyuk
df29e98ea5 Improve amendment processing and activation logic:
* The amendment ballot counting code contained a minor technical
  flaw, caused by the use of integer arithmetic and rounding
  semantics, that could allow amendments to reach majority with
  slightly less than 80% support. This commit introduces an
  amendment which, if enabled, will ensure that activation
  requires at least 80% support.
* This commit also introduces a configuration option to adjust
  the amendment activation hysteresis. This option is useful on
  test networks, but should not be used on the main network as
  is a network-wide consensus parameter that should not be
  changed on a per-server basis; doing so can result in a
  hard-fork.

Fixes #3396
2020-06-25 19:46:43 -07:00
Howard Hinnant
1067086f71 Consolidate "Not Synced" error messages:
Work on a version 2 of the XRP Network API has begun. The new
API returns:

* `notSynced` in place of `noClosed`, `noCurrent`, and `noNetwork`;
* `invalidParams` in place of `lgrIdxInvalid`.

The new version 2 API cannot be selected yet, as it remains a work
in progress.

Fixes #3269
2020-06-25 19:45:59 -07:00
p2peer
93bf77bdec Unit tests for database shards 2020-05-29 17:54:19 -07:00
Howard Hinnant
2d23e7bd18 Use std::size in place of std::extent
* Fixes RIPD-269
2020-05-29 14:03:08 -07:00
CJ Cobb
16f79d160a Add delivered amount to GetAccountTransactionHistory responses 2020-05-26 18:38:51 -07:00
Devon White
ac766ec0eb Introduce ShardArchiveHandler improvements:
* Improve documentation
* Make the ShardArchiveHandler rather than the DatabaseShardImp perform
  LastLedgerHash verification for downloaded shards
* Remove ShardArchiveHandler's singleton implementation and make it an
  Application member
* Have the Application invoke ShardArchiveHandler initialization
  instead of clients
* Add RecoveryHandler as a ShardArchiveHandler derived class
* Improve commenting
2020-05-26 18:37:01 -07:00
Scott Schurr
894d3463ce Extend unit testing of account_tx with deleted account:
The unit test now verifies that if an account is not present in the
starting account_tx ledger, account_tx still iterates down and finds
the transaction that deletes the account (and earlier transactions).
2020-05-05 16:05:23 -07:00
John Freeman
5b5226d518 Cleanup the 'PeerSet' hierarchy:
This commit introduces no functional changes but cleans up the
code and shrinks the surface area by removing dead and unused
code, leveraging std:: alternatives to hand-rolled code and
improving comments and documentation.
2020-05-05 16:05:23 -07:00
Nik Bougalis
74f9edef07 Prefer keylets instead of naked hashes:
Entries in the ledger are located using 256-bit locators. The locators
are calculated using a wide range of parameters specific to the entry
whose locator we are calculating (e.g. an account's locator is derived
from the account's address, whereas the locator for an offer is derived
from the account and the offer sequence.)

Keylets enhance type safety during lookup and make the code more robust,
so this commit removes most of the earlier code, which used naked
uint256 values.
2020-05-05 16:05:22 -07:00
Nik Bougalis
381606aba2 Harden validations:
This commit introduces the "HardenedValidations" amendment which,
if enabled, allows validators to include additional information in
their validations that can increase the robustness of consensus.

Specifically, the commit introduces a new optional field that can
be set in validation messages can be used to attest to the hash of
the latest ledger that a validator considers to be fully validated.

Additionally, the commit leverages the previously introduced "cookie"
field to improve the robustness of the network by making it possible
for servers to automatically detect accidental misconfiguration which
results in two or more validators using the same validation key.
2020-05-01 12:55:11 -07:00
Pretty Printer
50760c6935 Format first-party source according to .clang-format 2020-04-23 10:02:04 -07:00
Scott Schurr
0d83223445 Remove conditionals for fix1201 enabled 14Nov2017 2020-04-09 11:42:34 -07:00
Scott Schurr
a6246b0baa Remove conditionals for fix1373 enabled 07Jul2017 2020-04-09 11:42:34 -07:00
Scott Schurr
6bb9dd22e0 Remove conditionals for featureCryptoConditions enabled 03Jan2017 2020-04-09 09:31:06 -07:00
seelabs
ade5eb71cf Fix unneeded copies in range some range for loops:
clang 10 warns about an unneeded copy for these range for
loops (range-loop-construct warnings)
2020-04-06 17:22:59 -07:00
Devon White
905a97e0aa Make ShardArchiveHandler downloads more resilient:
* Make ShardArchiveHandler a singleton.
* Add state database for ShardArchiveHandler.
* Use temporary database for SSLHTTPDownloader downloads.
* Make ShardArchiveHandler a Stoppable class.
* Automatically resume interrupted downloads at server start.
2020-04-06 17:22:47 -07:00
Miguel Portilla
cc452dfa9b Improve shard concurrency:
* Reduce lock scope on all public functions
* Use TaskQueue to process shard finalization in separate thread
* Store shard last ledger hash and other info in backend
* Use temp SQLite DB versus control file when acquiring
* Remove boost serialization from cmake files
2020-04-06 17:22:23 -07:00
CJ Cobb
e7ce3909d2 gRPC support for account_tx and tx
- Add support for all transaction types and ledger object types to gRPC
  implementation of tx and account_tx.

- Create common handlers for tx and account_tx.

- Remove mutex and abort() from gRPC server. JobQueue is stopped before
  gRPC server, with all coroutines executed to completion, so no need for
  synchronization.
2020-02-25 19:21:55 -08:00
Edward Hennis
5ff23f8f31 Warn operators about upcoming unknown amendments:
* When an unknown amendment reaches majority, log an error-level
  message, and return a `warnings` array on all successful
  admin-level RPC calls to `server_info` and `server_state` with
  a message describing the problem, and the expected deadline.
* In addition to the `amendment_blocked` flag returned by
  `server_info` and `server_state`, return a warning with a more
  verbose description when the server is amendment blocked.
* Check on every flag ledger to see if the amendment(s) lose majority.
  Logs again if they don't, resumes normal operations if they did.

The intention is to give operators earlier warning that their
instances are in danger of being amendment blocked, which will
hopefully motivate them to update ahead of time.
2020-02-12 10:19:23 -08:00
Scott Schurr
2831ed0538 Add unittests for peer_reservations command line interface 2020-02-11 19:14:24 -08:00
Mo Morsi
e24dd2f954 Fix broken tests (missing api_version parameter) 2020-02-11 19:14:24 -08:00
Mo Morsi
60f0f5224d Accept 'strict' param in certain CLI options
account_info, owner_info, account_currencies
2020-02-10 23:36:31 -08:00
Mo Morsi
3578acaf0b Add new validator_info and manifest rpc methods
Returns local validator details and specified manifest information
respectively. Folded and rebased on latest develop
2020-02-10 23:36:31 -08:00