Commit 4dc08f8202 introduced support for
deterministic shards, which makes the sharding functionality provided
by rippled more useful.
After merging, several opportunities for further improvements to the
deterministic sharding implementation were identified and a significant
increase int memory usage during shard finalization was detected.
Because of these issues, the commit is being reverted and the feature is
being rolled back. It will be reintroduced in a future release.
There was a race condition in `on_accept` where the object's destructor
could run while `on_accept` was called.
This patch ensures that if `on_accept` is called then the object remains
valid for the duration of the call.
This change can help improve the liveness of the network during periods of network
instability, by allowing the network to track which validators are presently not online
and to disregard them for the purposes of quorum calculations.
If the 'HardenedValidations' amendment is enabled, this commit will
track the version of the software that validators embed in their
validations.
If a server notices that at least 60% of the validators on its UNL
are running a newer version than it is running, it will periodically
print an informational message, reminding the operator to check for
update.
This commit, if merged, adds support to allow multiple indepedent nodes to
produce a binary identical shard for a given range of ledgers. The advantage
is that servers can use content-addressable storage, and can more efficiently
retrieve shards by downloading from multiple peers at once and then verifying
the integrity of a shard by cross-checking its checksum with the checksum
other servers report.
* Document delete_batch, back_off_milliseconds, age_threshold_seconds.
* Convert those time values to chrono types.
* Fix bug that ignored age_threshold_seconds.
* Add a "recovery buffer" to the config that gives the node a chance to
recover before aborting online delete.
* Add begin/end log messages around the SQL queries.
* Add a new configuration section: [sqlite] to allow tuning the sqlite
database operations. Ignored on full/large history servers.
* Update documentation of [node_db] and [sqlite] in the
rippled-example.cfg file.
Resolves#3321
* The amendment ballot counting code contained a minor technical
flaw, caused by the use of integer arithmetic and rounding
semantics, that could allow amendments to reach majority with
slightly less than 80% support. This commit introduces an
amendment which, if enabled, will ensure that activation
requires at least 80% support.
* This commit also introduces a configuration option to adjust
the amendment activation hysteresis. This option is useful on
test networks, but should not be used on the main network as
is a network-wide consensus parameter that should not be
changed on a per-server basis; doing so can result in a
hard-fork.
Fixes#3396
Work on a version 2 of the XRP Network API has begun. The new
API returns:
* `notSynced` in place of `noClosed`, `noCurrent`, and `noNetwork`;
* `invalidParams` in place of `lgrIdxInvalid`.
The new version 2 API cannot be selected yet, as it remains a work
in progress.
Fixes#3269
* Make sure variables are always initialized
* Use lround instead of adding .5 and casting
* Remove some unneeded vars
* Check for null before calling strcmp
* Remove redundant if conditions
* Remove make_TxQ factory function
* Improve documentation
* Make the ShardArchiveHandler rather than the DatabaseShardImp perform
LastLedgerHash verification for downloaded shards
* Remove ShardArchiveHandler's singleton implementation and make it an
Application member
* Have the Application invoke ShardArchiveHandler initialization
instead of clients
* Add RecoveryHandler as a ShardArchiveHandler derived class
* Improve commenting
In deciding whether to relay a proposal or validation, a server would
consider whether it was issued by a validator on that server's UNL.
While both trusted proposals and validations were always relayed,
the code prioritized relaying of untrusted proposals over untrusted
validations. While not technically incorrect, validations are
generally more "valuable" because they are required during the
consensus process, whereas proposals are not, strictly, required.
The commit introduces two new configuration options, allowing server
operators to fine-tune the relaying behavior:
The `[relay_proposals]` option controls the relaying behavior for
proposals received by this server. It has two settings: "trusted"
and "all" and the default is "trusted".
The `[relay_validations]` options controls the relaying behavior for
validations received by this server. It has two settings: "trusted"
and "all" and the default is "all".
This change does not require an amendment as it does not affect
transaction processing.
The sfLedgerSequence field is designated as optional in the object
template but it is effectively required and validations which do not
include it were, correctly, rejected.
This commit migrates the check outside of the peer code and into the
constructor used for validations being deserialized for the network.
Furthermore, the code will generate an error if a validation that is
generated by a server does not include the field.
The unit test now verifies that if an account is not present in the
starting account_tx ledger, account_tx still iterates down and finds
the transaction that deletes the account (and earlier transactions).
This commit introduces no functional changes but cleans up the
code and shrinks the surface area by removing dead and unused
code, leveraging std:: alternatives to hand-rolled code and
improving comments and documentation.
Entries in the ledger are located using 256-bit locators. The locators
are calculated using a wide range of parameters specific to the entry
whose locator we are calculating (e.g. an account's locator is derived
from the account's address, whereas the locator for an offer is derived
from the account and the offer sequence.)
Keylets enhance type safety during lookup and make the code more robust,
so this commit removes most of the earlier code, which used naked
uint256 values.
This commit removes obsolete comments, dead or no longer useful
code, and workarounds for several issues that were present in older
compilers that we no longer support.
Specifically:
- It improves the transaction metadata handling class, simplifying
its use and making it less error-prone.
- It reduces the footprint of the Serializer class by consolidating
code and leveraging templates.
- It cleanups the ST* class hierarchy, removing dead code, improving
and consolidating code to reduce complexity and code duplication.
- It shores up the handling of currency codes and the conversation
between 160-bit currency codes and their string representation.
- It migrates beast::secure_erase to the ripple namespace and uses
a call to OpenSSL_cleanse instead of the custom implementation.
A deliberately malformed token can cause the server to crash during
startup. This is not remotely exploitable and would require someone
with access to the configuration file of the server to make changes
and then restart the server.
Acknowledgements:
Guido Vranken for responsibly disclosing this issue.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers to
responsibly disclose any issues they may find.
Ripple is generously sponsoring a bug bounty program for the
rippled project. For more information please visit:
https://ripple.com/bug-bounty
This commit introduces the "HardenedValidations" amendment which,
if enabled, allows validators to include additional information in
their validations that can increase the robustness of consensus.
Specifically, the commit introduces a new optional field that can
be set in validation messages can be used to attest to the hash of
the latest ledger that a validator considers to be fully validated.
Additionally, the commit leverages the previously introduced "cookie"
field to improve the robustness of the network by making it possible
for servers to automatically detect accidental misconfiguration which
results in two or more validators using the same validation key.
* canonicalize_replace_cache
* canonicalize_replace_client
Now it is clear at the call site that if there are
duplicate copies of the data between the cache and
the caller, which copy gets replaced.
Additionally data parameter is now const-correct.
If it is not going to be replaced (canonicalize_replace_cache),
then the shared_ptr to the client data is const.
* Peers negotiate compression via HTTP Header "X-Offer-Compression: lz4"
* Messages greater than 70 bytes and protocol type messages MANIFESTS,
ENDPOINTS, TRANSACTION, GET_LEDGER, LEDGER_DATA, GET_OBJECT,
and VALIDATORLIST are compressed
* If the compressed message is larger than the uncompressed message
then the uncompressed message is sent
* Compression flag and the compression algorithm type are included
in the message header
* Only LZ4 block compression is currently supported
The payment engine restricts payment paths so two steps do not input the
same Currency/Issuer or output the same Currency/Issuer. This check was
skipped when the path started or ended with XRP. An example of a path
that was incorrectly accepted was: XRP -> //USD -> //XRP -> EUR
This patch enables the path loop check for paths that start or end with
XRP.
* Make ShardArchiveHandler a singleton.
* Add state database for ShardArchiveHandler.
* Use temporary database for SSLHTTPDownloader downloads.
* Make ShardArchiveHandler a Stoppable class.
* Automatically resume interrupted downloads at server start.
* Reduce lock scope on all public functions
* Use TaskQueue to process shard finalization in separate thread
* Store shard last ledger hash and other info in backend
* Use temp SQLite DB versus control file when acquiring
* Remove boost serialization from cmake files
When computing rates for offers, an STAmount's value can be out of can be out of
range (before canonicalizing). There was an assert that could incorrectly fire
in some cases. This patch removes that assert.
