The existing logic involves every server sending every transaction
that it receives to all its peers (except the one that it received
a transaction from).
This commit instead uses a randomized algorithm, where a node will
randomly select peers to relay a given transaction to, caching the
list of transaction hashes that are not relayed and forwading them
to peers once every second. Peers can then determine whether there
are transactions that they have not seen and can request them from
the node which has them.
It is expected that this feature will further reduce the bandwidth
needed to operate a server.
- Add validation/proposal reduce-relay feature negotiation to
the handshake
- Make squelch duration proportional to a number of peers that
can be squelched
- Refactor makeRequest()/makeResponse() to facilitate handshake
unit-testing
- Fix compression enable flag for inbound peer
- Fix compression algorithm parsing in the header parser
- Fix squelch duration in onMessage(TMSquelch)
This commit fixes 3624, fixes 3639 and fixes 3641
* Creates a version 2 of the UNL file format allowing publishers to
pre-publish the next UNL while the current one is still valid.
* Version 1 of the UNL file format is still valid and backward
compatible.
* Also causes rippled to lock down if it has no valid UNLs, similar to
being amendment blocked, except reversible.
* Resolves#3548
* Resolves#3470
- Simplify and consolidate code for parsing hex input.
- Replace beast::endian::order with boost::endian::order.
- Simplify CountedObject code.
- Remove pre-C++17 workarounds in favor of C++17 based solutions.
- Improve `base_uint` and simplify its hex-parsing interface by
consolidating the `SexHex` and `SetHexExact` methods into one
API: `parseHex` which forces callers to verify the result of
the operation; as a result some public-facing API endpoints
may now return errors when passed values that were previously
accepted.
- Remove the simple fallback implementations of SHA2 and RIPEMD
introduced to reduce our dependency on OpenSSL. The code is
slow and rarely, if ever, exercised and we rely on OpenSSL
functionality for Boost.ASIO as well.
The existing SHAMapNodeID object has both a valid and an invalid state
and requirs callers to verify the state of an instance prior to using
it. A simple set of changes removes that restriction and ensures that
all instances are valid, making the code more robust.
This change also:
1. Introduces a new function to construct a SHAMapNodeID from a
serialized blob; and
2. Reduces the amount of constructors the class exposes.
Support for IPv6 messages was added with commit 08382d866b
and version 1.1.0. No peer presently connected to the network in a useful capacity fails
to understand v2 messages.
This commit removes the code that generates and processes v1 messages and deletes legacy
messages from the protocol buffer definition file.
With few exceptions, servers will typically receive multiple copies
of any given message from its directly connected peers. For servers
with several peers this can impact the processing latency and force
it to do redundant work. Proposal and validation messages are often
relayed with extremely high redundancy.
This commit, if merged, introduces experimental code that attempts
to optimize the relaying of proposals and validations by allowing
servers to instruct their peers to "squelch" delivery of selected
proposals and validations. Servers making squelching decisions by
a process that evaluates the fitness and performance of a given
server and randomly selecting a subset of the best candidates.
The experimental code is presently disabled and must be explicitly
enabled by server operators that wish to test it.
* Peers negotiate compression via HTTP Header "X-Offer-Compression: lz4"
* Messages greater than 70 bytes and protocol type messages MANIFESTS,
ENDPOINTS, TRANSACTION, GET_LEDGER, LEDGER_DATA, GET_OBJECT,
and VALIDATORLIST are compressed
* If the compressed message is larger than the uncompressed message
then the uncompressed message is sent
* Compression flag and the compression algorithm type are included
in the message header
* Only LZ4 block compression is currently supported
This commit restructures the HTTP based protocol negotiation that `rippled`
executes and introduces support for negotiation of compression for peer
links which, if implemented, should result in significant bandwidth savings
for some server roles.
This commit also introduces the new `[network_id]` configuration option
that administrators can use to specify which network the server is part of
and intends to join. This makes it possible for servers from different
networks to drop the link early.
The changeset also improves the log messages generated when negotiation
of a peer link upgrade fails. In the past, no useful information would
be logged, making it more difficult for admins to troubleshoot errors.
This commit also fixes RIPD-237 and RIPD-451
This patch removes calls to several deprecated asio functions.
* `io_service::post` becomes `post` (free function)
* `io_service::work` becomes `executor_work_guard`
* `io_service::wrap` becomes `bind_executor`
* `get_io_context` becomes `get_executor` or `get_executor().context()`
This patch was tested with boost 1.69 and 1.70. The functions
`ripple::get_lowest_layer` and `beast::create_waitable_timer` are required to
handle a breaking difference between these versions. When rippled no longer
needs to support pre 1.70 boost versions, both of these functions may be
removed, and the waitable timer injections may also be removed.
Fixes: RIPD-1574
Alias beast address classes to the asio equivalents. Adjust users of
address classes accordingly. Fix resolver class so that it can support
ipv6 addresses. Make unit tests use ipv6 localhost network. Extend
endpoint peer message to support string endpoint
representations while also supporting the existing fields (both are
optional/repeated types). Expand test for Livecache and Endpoint.
Workaround some false positive ipaddr tests on windows (asio bug?)
Replaced usage of address::from_string(deprecated) with free function
make_address. Identified a remaining use of v4 address type and
replaced with the more appropriate IPEndpoint type (rpc_ip cmdline
option). Add CLI flag for using ipv4 with unit tests.
Release Notes
-------------
The optional rpc_port command line flag is deprecated. The rpc_ip
parameter now works as documented and accepts ip and port combined.
* RIPD-1617, RIPD-1619, RIPD-1621:
Verify serialized public keys more strictly before
using them.
* RIPD-1618:
* Simplify the base58 decoder logic.
* Reduce the complexity of the base58 encoder and
eliminate a potential out-of-bounds memory access.
* Improve type safety by using an `enum class` to
enforce strict type checking for token types.
* RIPD-1616:
Avoid calling `memcpy` with a null pointer even if the
size is specified as zero, since it results in undefined
behavior.
Acknowledgements:
Ripple thanks Guido Vranken for responsibly disclosing these
issues.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers
to responsibly disclose any issues that they may find. For
more on Ripple's Bug Bounty program, please visit:
https://ripple.com/bug-bounty
All uses of beast::Thread were previously removed from the code
base, so beast::Thread is removed. One piece of beast::Thread
needed to be preserved: the ability to set the current thread's
name. So there's now a beast::CurrentThreadName that allows the
current thread's name to be set and returned.
Thread naming is also cleaned up a bit. ThreadName.h and .cpp
are removed since beast::CurrentThreadName does a better job.
ThreadEntry is also removed, but its terminateHandler() is
preserved in TerminateHandler.cpp. The revised terminateHandler()
uses beast::CurrentThreadName to recover the name of the running
thread.
Finally, the NO_LOG_UNHANDLED_EXCEPTIONS #define is removed since
it was discovered that the MacOS debugger preserves the stack
of the original throw even if the terminateHandler() rethrows.
Instead of specifying a static list of trusted validators in the config
or validators file, the configuration can now include trusted validator
list publisher keys.
The trusted validator list and quorum are now reset each consensus
round using the latest validator lists and the list of recent
validations seen. The minimum validation quorum is now only
configurable via the command line.
The existing configuration includes 512 and 1024 bit DH
parameters and supports ciphers such as RC4 and 3DES and
hash algorithms like SHA-1 which are no longer considered
secure.
Going forward, use only 2048-bit DH parameters and define
a new default set of modern ciphers to use:
HIGH:!aNULL:!MD5:!DSS:!SHA1:!3DES:!RC4:!EXPORT:!DSS
Additionally, allow administrators who wish to have different
settings to configure custom global and per-port ciphers suites
in the configuration file using the `ssl_ciphers` directive.
