Commit Graph

12256 Commits

Author SHA1 Message Date
Yusuf Sahin HAMZA
27484f78a9 Use libarchive version 3.4.3 2020-05-29 17:54:19 -07:00
p2peer
93bf77bdec Unit tests for database shards 2020-05-29 17:54:19 -07:00
Miguel Portilla
728651b5d5 Use LZ4_decompress_safe 2020-05-29 17:54:19 -07:00
Miguel Portilla
fb74eefa9e Update LZ4 library to 1.9.2 2020-05-29 17:54:19 -07:00
manojsdoshi
853c964194 Fix a build issue caused by the latest Docker version:
* The latest docker version is not supported by
   artifactory causing the package build to fail.
   Setting the docker version to 19.03.8 to fix the build.
2020-05-29 17:54:19 -07:00
Gregory Tsipenyuk
645c06764b Update the default port for [ips] and [ips_fixed]:
If a port number is not specified in the [ips] or [ips_fixed]
blocks, automatically add the new default peer port which was
registered with IANA: 2459. Also use 2459 if no port is specified
with manually using the `connect` command; previously it was
using 6561, which could have resulted in spurious failures.
This commit, if merged, fixes #2861.
2020-05-29 17:52:49 -07:00
Howard Hinnant
2d23e7bd18 Use std::size in place of std::extent
* Fixes RIPD-269
2020-05-29 14:03:08 -07:00
Howard Hinnant
0290d0b82c Create health_check rpc
* Gives a summary of the health of the node:
  Healthy, Warning, or Critical

* Last validated ledger age:
  <7s is Healthy,
  7s to 20s is Warning
  > 20s is Critcal

* If amendment blocked, Critical

* Number of peers:
  > 7 is Healthy
  1 to 7 is Warning
  0 is Critical

* server state:
  One of full, validating or proposing is Healthy
  One of syncing, tracking or connected is Warning
  All other states are Critical

* load factor:
  <= 100 is Healthy
  101 to 999 is Warning
  >= 1000 is Critical

* If not Healthy, info field contains data that is considered not
  Healthy.

Fixes: #2809
2020-05-29 14:02:08 -07:00
manojsdoshi
3d86b49dae Set version to 1.6.0-b6 2020-05-26 18:48:13 -07:00
John Freeman
0b9e935806 Find date::date in CMake package configuration file 2020-05-26 18:41:30 -07:00
Mark Travis
17412d17e2 Fix historical ledger acquisition when using online deletion:
Commit e257a22 introduced changes in the logic used to acquire historical
ledgers. The logic could cause historical ledgers to be acquired only since
the last online deletion interval instead of the configured value to allow
deletion.
2020-05-26 18:39:48 -07:00
seelabs
99f5193699 Disable formatting operator&:
Clang 10.0.0 and 10.0.1 disagree on formatting operator&
Disable until we upgrade to clang 11.
2020-05-26 18:39:36 -07:00
seelabs
328e42ad42 Minor cleanups:
* Make sure variables are always initialized
* Use lround instead of adding .5 and casting
* Remove some unneeded vars
* Check for null before calling strcmp
* Remove redundant if conditions
* Remove make_TxQ factory function
2020-05-26 18:39:24 -07:00
seelabs
6d28f2a8d9 Cleanup code using move semantics 2020-05-26 18:39:16 -07:00
seelabs
421417ab07 Fixes for gcc 10 2020-05-26 18:39:06 -07:00
Scott Schurr
68494a308e AmendmentTable improvements 2020-05-26 18:38:58 -07:00
CJ Cobb
16f79d160a Add delivered amount to GetAccountTransactionHistory responses 2020-05-26 18:38:51 -07:00
John Freeman
8f984042f4 Try to fix usage of pkg-config for static linking 2020-05-26 18:37:52 -07:00
Edward Hennis
eb1a699c5a Fix typo in error message 2020-05-26 18:37:45 -07:00
Devon White
ac766ec0eb Introduce ShardArchiveHandler improvements:
* Improve documentation
* Make the ShardArchiveHandler rather than the DatabaseShardImp perform
  LastLedgerHash verification for downloaded shards
* Remove ShardArchiveHandler's singleton implementation and make it an
  Application member
* Have the Application invoke ShardArchiveHandler initialization
  instead of clients
* Add RecoveryHandler as a ShardArchiveHandler derived class
* Improve commenting
2020-05-26 18:37:01 -07:00
Devon White
21340a1c1e Validate LastLedgerHash for downloaded shards:
* Add documentation for shard validation
* Retrieve last ledger hash for imported shards
* Verify the last ledger hash in Shard::finalize
* Limit last ledger hash retrieval attempts for imported shards
* Use a common function for removing failed shards
* Add new ShardInfo::State for imported shards
2020-05-26 18:36:54 -07:00
Nik Bougalis
c594f3b0cf Reduce visibility of retired amendment identifiers:
Identifiers for retired amendments should not generally be used
in the codebase.

This commit reduces their visibility down to one translation
unit and marks them as unused and deprecated to prevent
accidental reuse.
2020-05-26 18:36:15 -07:00
Nik Bougalis
268e28a278 Tune relaying of untrusted proposals & validations:
In deciding whether to relay a proposal or validation, a server would
consider whether it was issued by a validator on that server's UNL.

While both trusted proposals and validations were always relayed,
the code prioritized relaying of untrusted proposals over untrusted
validations. While not technically incorrect, validations are
generally more "valuable" because they are required during the
consensus process, whereas proposals are not, strictly, required.

The commit introduces two new configuration options, allowing server
operators to fine-tune the relaying behavior:

The `[relay_proposals]` option controls the relaying behavior for
proposals received by this server. It has two settings: "trusted"
and "all" and the default is "trusted".

The `[relay_validations]` options controls the relaying behavior for
validations received by this server. It has two settings: "trusted"
and "all" and the default is "all".

This change does not require an amendment as it does not affect
transaction processing.
2020-05-26 18:36:06 -07:00
Nik Bougalis
ca664b17d3 Improve handling of sfLedgerSequence field:
The sfLedgerSequence field is designated as optional in the object
template but it is effectively required and validations which do not
include it were, correctly, rejected.

This commit migrates the check outside of the peer code and into the
constructor used for validations being deserialized for the network.

Furthermore, the code will generate an error if a validation that is
generated by a server does not include the field.
2020-05-26 18:35:58 -07:00
Nik Bougalis
3936110c8d Use boost::circular_buffer:
The existing code used std::deque along with a size check to constrain the
size of a buffer and, effectively, "hand rolled" a circular buffer. This
change simply migrates directly to boost::circular_buffer.
2020-05-26 18:35:49 -07:00
Nik Bougalis
9f91870b1c Adjust frequency of mtENDPOINTS messages 2020-05-26 18:35:39 -07:00
Nik Bougalis
97712107b7 Set version to 1.6.0-b5 2020-05-05 16:05:24 -07:00
Carl Hua
d9fa148688 Revert "Add PR automation for project boards"
This reverts commit cd78ce3118.
2020-05-05 16:05:23 -07:00
Miguel Portilla
d88a7c73b4 Improve online delete backend locking 2020-05-05 16:05:23 -07:00
Scott Schurr
894d3463ce Extend unit testing of account_tx with deleted account:
The unit test now verifies that if an account is not present in the
starting account_tx ledger, account_tx still iterates down and finds
the transaction that deletes the account (and earlier transactions).
2020-05-05 16:05:23 -07:00
John Freeman
5b5226d518 Cleanup the 'PeerSet' hierarchy:
This commit introduces no functional changes but cleans up the
code and shrinks the surface area by removing dead and unused
code, leveraging std:: alternatives to hand-rolled code and
improving comments and documentation.
2020-05-05 16:05:23 -07:00
Alloy Networks
d025f3fb28 Add descriptive comments to 'getRippledInfo.sh':
The script, when invoked by a server operator can collect information
useful for debugging, while attempting to redact potentially sensitive
data.

It contained no explanation or other exposition to allow people who
look at the file but aren't familiar with shell scripts to understand
its purpose.
2020-05-05 16:05:23 -07:00
Markus Alvila
11be30dd40 Correct typo in comment 2020-05-05 16:05:23 -07:00
Elliot Lee
4fad421c8a Correct typos in SECURITY.md 2020-05-05 16:05:23 -07:00
seelabs
57b3543e7b Support boost 1.73 2020-05-05 16:05:23 -07:00
Kirill Fomichev
a00543b6bc Fix docs about misconfigured neighbor port 2020-05-05 16:05:23 -07:00
Kirill Fomichev
dbd25f0e32 Remove excessive redirect call on PeerManager 2020-05-05 16:05:23 -07:00
Nik Bougalis
62a3f33d72 Remove the built-in "sustain" watchdog:
The built-in watchdog is simplistic and can, sometimes, cause problems
especially on systems that have the ability to automatically start and
monitor processes.

This commit removes the sustain system entirely, changes the handling
of the SIGTERM signal to properly terminate the process and improves
the error message reported to the user when the command line used to
start `rippled` is incorrect and malformed.
2020-05-05 16:05:23 -07:00
Nik Bougalis
74f9edef07 Prefer keylets instead of naked hashes:
Entries in the ledger are located using 256-bit locators. The locators
are calculated using a wide range of parameters specific to the entry
whose locator we are calculating (e.g. an account's locator is derived
from the account's address, whereas the locator for an offer is derived
from the account and the offer sequence.)

Keylets enhance type safety during lookup and make the code more robust,
so this commit removes most of the earlier code, which used naked
uint256 values.
2020-05-05 16:05:22 -07:00
Nik Bougalis
dbee3f01b7 Clean up and modernize code:
This commit removes obsolete comments, dead or no longer useful
code, and workarounds for several issues that were present in older
compilers that we no longer support.

Specifically:

- It improves the transaction metadata handling class, simplifying
  its use and making it less error-prone.
- It reduces the footprint of the Serializer class by consolidating
  code and leveraging templates.
- It cleanups the ST* class hierarchy, removing dead code, improving
  and consolidating code to reduce complexity and code duplication.
- It shores up the handling of currency codes and the conversation
  between 160-bit currency codes and their string representation.
- It migrates beast::secure_erase to the ripple namespace and uses
  a call to OpenSSL_cleanse instead of the custom implementation.
2020-05-05 16:05:22 -07:00
Nik Bougalis
6c72d5cf7e Improve loading of validator tokens (RIPD-1687):
A deliberately malformed token can cause the server to crash during
startup. This is not remotely exploitable and would require someone
with access to the configuration file of the server to make changes
and then restart the server.

Acknowledgements:
Guido Vranken for responsibly disclosing this issue.

Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers to
responsibly disclose any issues they may find.

Ripple is generously sponsoring a bug bounty program for the
rippled project. For more information please visit:

    https://ripple.com/bug-bounty
2020-05-01 12:55:12 -07:00
Nik Bougalis
2827de4d63 Report the server version in published validations:
Currently there is no mechanism for a validator to report the
version of the software it is currently running. Such reports
can be useful for those who are developing network monitoring
dashboards and server operators in general.

This commit, if merged, defines an encoding scheme to encode
a version string into a 64-bit unsigned integer and adds an
additional optional field to validations.

This commit piggybacks on "HardenedValidations" amendment to
determine whether version information should be propagated
or not.

The general encoding scheme is:

XXXXXXXX-XXXXXXXX-YYYYYYYY-YYYYYYYY-YYYYYYYY-YYYYYYYY-YYYYYYYY-YYYYYYYY

X: 16 bits identifying the particular implementation
Y: 48 bits of data specific to the implementation

The rippled-specific format (implementation ID is: 0x18 0x3B) is:

00011000-00111011-MMMMMMMM-mmmmmmmm-pppppppp-TTNNNNNN-00000000-00000000

    M: 8-bit major version (0-255)
    m: 8-bit minor version (0-255)
    p: 8-bit patch version (0-255)
    T: 11 if neither an RC nor a beta
       10 if an RC
       01 if a beta
    N: 6-bit rc/beta number (1-63)
2020-05-01 12:55:12 -07:00
Nik Bougalis
381606aba2 Harden validations:
This commit introduces the "HardenedValidations" amendment which,
if enabled, allows validators to include additional information in
their validations that can increase the robustness of consensus.

Specifically, the commit introduces a new optional field that can
be set in validation messages can be used to attest to the hash of
the latest ledger that a validator considers to be fully validated.

Additionally, the commit leverages the previously introduced "cookie"
field to improve the robustness of the network by making it possible
for servers to automatically detect accidental misconfiguration which
results in two or more validators using the same validation key.
2020-05-01 12:55:11 -07:00
Nik Bougalis
567e42e071 Deprecate 'Time to Live' fields 2020-04-30 20:05:09 -07:00
manojsdoshi
2bf3b194fa Set version to 1.6.0-b4 2020-04-27 16:17:15 -07:00
manojsdoshi
444ea56182 Adding support to build rippled packages for ubuntu 20.04 2020-04-27 16:16:56 -07:00
CJ Cobb
d797589164 Add README for gRPC protobuf folder 2020-04-23 18:11:12 -07:00
Edward Hennis
1577c775b3 Close download socket before result is passed to the callback:
* Specifically relevent to the processing of UNL downloads via
  ValidatorSite.
2020-04-23 18:10:21 -07:00
Edward Hennis
3bf0b724a3 Adjust timeouts in Validator Site tests:
* Prevents spurious failures due to DNS resolution.
2020-04-23 18:10:12 -07:00
Miguel Portilla
bd8dbb87b6 Update to RocksBD 6.7.3 2020-04-23 18:09:45 -07:00