12581 Commits

Author SHA1 Message Date
manojsdoshi
7b048b423e Set version to 1.6.0-rc1 2020-07-08 13:40:03 -07:00
manojsdoshi
a459206848 Set version to 1.6.0-b9 2020-07-08 13:38:27 -07:00
Howard Hinnant
3a3b0b4c14 Modify health check API
* Fixes #3486
* load factor computation normalized by load_base.
* last validated ledger age set to -1 while syncing.
* Return status changed:
*    healthy  -> ok
*    warning  -> service_unavailable
*    critical -> internal_server_error
2020-07-08 13:35:07 -07:00
manojsdoshi
de0c527387 Set version to 1.6.0-b8 2020-06-30 12:40:51 -07:00
manojsdoshi
b54453d1a7 Fix a build issue caused by pip3 being unavailable 2020-06-30 09:15:37 -07:00
Peng Wang
706ca874b0 Implement negative UNL functionality:
This change can help improve the liveness of the network during periods of network
instability, by allowing the network to track which validators are presently not online
and to disregard them for the purposes of quorum calculations.
2020-06-30 09:15:37 -07:00
Peng Wang
51bd4626b1 Implement version upgrade warning:
If the 'HardenedValidations' amendment is enabled, this commit will
track the version of the software that validators embed in their
validations.

If a server notices that at least 60% of the validators on its UNL
are running a newer version than it is running, it will periodically
print an informational message, reminding the operator to check for
update.
2020-06-30 09:15:37 -07:00
mDuo13
cf6f40ea8f Deprecate unused/obsolete error codes:
The tecUNFUNDED code is actively used when attempting to create payment
channels; the messages incorrectly list it as deprecated.

Meanwhile, the tecUNFUNDED_ADD code actually is an unused legacy code,
dating back to when there was a WalletAdd transactor. The terLAST and
terFUNDS_SPENT codes are also unused legacy codes.

Engine result messages are not part of the binary format and are
documented as subject to change without notice, so this should not
require an amendment nor a new API version.

Align error code table for human readability.
2020-06-30 08:52:18 -07:00
seelabs
d3798f6290 Remove CryptoConditionsSuite stub amendment:
The amendment was partially complete, included no functional code
and, even if activated, it would result in no changes to transaction
proessing. Despite this, removing the amendment is the prudent course
of action and avoids the possibility of an accidental activation.

If additional cryptoconditions are implemented, they will be each
assigned a new, unique amendment code.
2020-06-30 08:52:18 -07:00
Rome Reginelli
4e33a1abf7 crawl_shards comment fix
Corrects the public_key parameter name in the comment.
See https://github.com/ripple/xrpl-dev-portal/pull/854 for context.
2020-06-30 08:52:18 -07:00
Miguel Portilla
86e8f2e232 Add Shard Family 2020-06-30 08:52:18 -07:00
Miguel Portilla
91e857874f Improve LedgerMaster shard acquisition 2020-06-30 08:52:17 -07:00
Miguel Portilla
8f50fd051e Use NuDB version 2.0.3 2020-06-30 08:52:17 -07:00
p2peer
94e8e94750 Add support for deterministic database shards (#2688):
This commit, if merged, adds support to allow multiple indepedent nodes to
produce a binary identical shard for a given range of ledgers. The advantage
is that servers can use content-addressable storage, and can more efficiently
retrieve shards by downloading from multiple peers at once and then verifying
the integrity of a shard by cross-checking its checksum with the checksum
other servers report.
2020-06-30 08:52:17 -07:00
Edward Hennis
54ece72b62 Update cmake so that rippled can build as a submodule 2020-06-25 19:47:20 -07:00
Edward Hennis
4702c8b591 Improve online_delete configuration and DB tuning:
* Document delete_batch, back_off_milliseconds, age_threshold_seconds.
* Convert those time values to chrono types.
* Fix bug that ignored age_threshold_seconds.
* Add a "recovery buffer" to the config that gives the node a chance to
  recover before aborting online delete.
* Add begin/end log messages around the SQL queries.
* Add a new configuration section: [sqlite] to allow tuning the sqlite
  database operations. Ignored on full/large history servers.
* Update documentation of [node_db] and [sqlite] in the
  rippled-example.cfg file.

Resolves #3321
2020-06-25 19:46:43 -07:00
Mark Travis
00702f28c2 Improve reporting of ledger age in server_info 2020-06-25 19:46:43 -07:00
Gregory Tsipenyuk
df29e98ea5 Improve amendment processing and activation logic:
* The amendment ballot counting code contained a minor technical
  flaw, caused by the use of integer arithmetic and rounding
  semantics, that could allow amendments to reach majority with
  slightly less than 80% support. This commit introduces an
  amendment which, if enabled, will ensure that activation
  requires at least 80% support.
* This commit also introduces a configuration option to adjust
  the amendment activation hysteresis. This option is useful on
  test networks, but should not be used on the main network as
  is a network-wide consensus parameter that should not be
  changed on a per-server basis; doing so can result in a
  hard-fork.

Fixes #3396
2020-06-25 19:46:43 -07:00
Nik Bougalis
fe9922d654 Improve compression support:
* Optimize parsing of compressed message headers
* Enforce protocol-defined message size maxima
* Update comments
2020-06-25 19:46:43 -07:00
Nik Bougalis
362a017eee Cleanup SHAMap and simplify interfaces:
* Improve error reporting (more readable exception messages)
* Reduce function complexity (split oversized function to smaller pieces)
* Reduce code duplication
* Reduce buffer copying
2020-06-25 19:46:43 -07:00
Nik Bougalis
e8f3525226 Improve Slice:
Slice should, eventually, be replaced by std::string_view<std::uint8_t>
so begin adding some helpful functions to align its interface.
2020-06-25 19:46:43 -07:00
Howard Hinnant
1067086f71 Consolidate "Not Synced" error messages:
Work on a version 2 of the XRP Network API has begun. The new
API returns:

* `notSynced` in place of `noClosed`, `noCurrent`, and `noNetwork`;
* `invalidParams` in place of `lgrIdxInvalid`.

The new version 2 API cannot be selected yet, as it remains a work
in progress.

Fixes #3269
2020-06-25 19:45:59 -07:00
Howard Hinnant
0214d83aa5 Improve handling of empty buffer in varint parsing (RIPD-1683) 2020-06-25 19:45:59 -07:00
CJ Cobb
d69a902876 Add comments to protobuf files 2020-06-25 19:09:19 -07:00
manojsdoshi
f43aeda49c Set version to 1.6.0-b7 2020-05-29 17:55:01 -07:00
Yusuf Sahin HAMZA
27484f78a9 Use libarchive version 3.4.3 2020-05-29 17:54:19 -07:00
p2peer
93bf77bdec Unit tests for database shards 2020-05-29 17:54:19 -07:00
Miguel Portilla
728651b5d5 Use LZ4_decompress_safe 2020-05-29 17:54:19 -07:00
Miguel Portilla
fb74eefa9e Update LZ4 library to 1.9.2 2020-05-29 17:54:19 -07:00
manojsdoshi
853c964194 Fix a build issue caused by the latest Docker version:
* The latest docker version is not supported by
   artifactory causing the package build to fail.
   Setting the docker version to 19.03.8 to fix the build.
2020-05-29 17:54:19 -07:00
Gregory Tsipenyuk
645c06764b Update the default port for [ips] and [ips_fixed]:
If a port number is not specified in the [ips] or [ips_fixed]
blocks, automatically add the new default peer port which was
registered with IANA: 2459. Also use 2459 if no port is specified
with manually using the `connect` command; previously it was
using 6561, which could have resulted in spurious failures.
This commit, if merged, fixes #2861.
2020-05-29 17:52:49 -07:00
Howard Hinnant
2d23e7bd18 Use std::size in place of std::extent
* Fixes RIPD-269
2020-05-29 14:03:08 -07:00
Howard Hinnant
0290d0b82c Create health_check rpc
* Gives a summary of the health of the node:
  Healthy, Warning, or Critical

* Last validated ledger age:
  <7s is Healthy,
  7s to 20s is Warning
  > 20s is Critcal

* If amendment blocked, Critical

* Number of peers:
  > 7 is Healthy
  1 to 7 is Warning
  0 is Critical

* server state:
  One of full, validating or proposing is Healthy
  One of syncing, tracking or connected is Warning
  All other states are Critical

* load factor:
  <= 100 is Healthy
  101 to 999 is Warning
  >= 1000 is Critical

* If not Healthy, info field contains data that is considered not
  Healthy.

Fixes: #2809
2020-05-29 14:02:08 -07:00
manojsdoshi
3d86b49dae Set version to 1.6.0-b6 2020-05-26 18:48:13 -07:00
John Freeman
0b9e935806 Find date::date in CMake package configuration file 2020-05-26 18:41:30 -07:00
Mark Travis
17412d17e2 Fix historical ledger acquisition when using online deletion:
Commit e257a22 introduced changes in the logic used to acquire historical
ledgers. The logic could cause historical ledgers to be acquired only since
the last online deletion interval instead of the configured value to allow
deletion.
2020-05-26 18:39:48 -07:00
seelabs
99f5193699 Disable formatting operator&:
Clang 10.0.0 and 10.0.1 disagree on formatting operator&
Disable until we upgrade to clang 11.
2020-05-26 18:39:36 -07:00
seelabs
328e42ad42 Minor cleanups:
* Make sure variables are always initialized
* Use lround instead of adding .5 and casting
* Remove some unneeded vars
* Check for null before calling strcmp
* Remove redundant if conditions
* Remove make_TxQ factory function
2020-05-26 18:39:24 -07:00
seelabs
6d28f2a8d9 Cleanup code using move semantics 2020-05-26 18:39:16 -07:00
seelabs
421417ab07 Fixes for gcc 10 2020-05-26 18:39:06 -07:00
Scott Schurr
68494a308e AmendmentTable improvements 2020-05-26 18:38:58 -07:00
CJ Cobb
16f79d160a Add delivered amount to GetAccountTransactionHistory responses 2020-05-26 18:38:51 -07:00
John Freeman
8f984042f4 Try to fix usage of pkg-config for static linking 2020-05-26 18:37:52 -07:00
Edward Hennis
eb1a699c5a Fix typo in error message 2020-05-26 18:37:45 -07:00
Devon White
ac766ec0eb Introduce ShardArchiveHandler improvements:
* Improve documentation
* Make the ShardArchiveHandler rather than the DatabaseShardImp perform
  LastLedgerHash verification for downloaded shards
* Remove ShardArchiveHandler's singleton implementation and make it an
  Application member
* Have the Application invoke ShardArchiveHandler initialization
  instead of clients
* Add RecoveryHandler as a ShardArchiveHandler derived class
* Improve commenting
2020-05-26 18:37:01 -07:00
Devon White
21340a1c1e Validate LastLedgerHash for downloaded shards:
* Add documentation for shard validation
* Retrieve last ledger hash for imported shards
* Verify the last ledger hash in Shard::finalize
* Limit last ledger hash retrieval attempts for imported shards
* Use a common function for removing failed shards
* Add new ShardInfo::State for imported shards
2020-05-26 18:36:54 -07:00
Nik Bougalis
c594f3b0cf Reduce visibility of retired amendment identifiers:
Identifiers for retired amendments should not generally be used
in the codebase.

This commit reduces their visibility down to one translation
unit and marks them as unused and deprecated to prevent
accidental reuse.
2020-05-26 18:36:15 -07:00
Nik Bougalis
268e28a278 Tune relaying of untrusted proposals & validations:
In deciding whether to relay a proposal or validation, a server would
consider whether it was issued by a validator on that server's UNL.

While both trusted proposals and validations were always relayed,
the code prioritized relaying of untrusted proposals over untrusted
validations. While not technically incorrect, validations are
generally more "valuable" because they are required during the
consensus process, whereas proposals are not, strictly, required.

The commit introduces two new configuration options, allowing server
operators to fine-tune the relaying behavior:

The `[relay_proposals]` option controls the relaying behavior for
proposals received by this server. It has two settings: "trusted"
and "all" and the default is "trusted".

The `[relay_validations]` options controls the relaying behavior for
validations received by this server. It has two settings: "trusted"
and "all" and the default is "all".

This change does not require an amendment as it does not affect
transaction processing.
2020-05-26 18:36:06 -07:00
Nik Bougalis
ca664b17d3 Improve handling of sfLedgerSequence field:
The sfLedgerSequence field is designated as optional in the object
template but it is effectively required and validations which do not
include it were, correctly, rejected.

This commit migrates the check outside of the peer code and into the
constructor used for validations being deserialized for the network.

Furthermore, the code will generate an error if a validation that is
generated by a server does not include the field.
2020-05-26 18:35:58 -07:00
Nik Bougalis
3936110c8d Use boost::circular_buffer:
The existing code used std::deque along with a size check to constrain the
size of a buffer and, effectively, "hand rolled" a circular buffer. This
change simply migrates directly to boost::circular_buffer.
2020-05-26 18:35:49 -07:00