Validator lists from configured remote sites are fetched at a regular
interval. Fetched lists are expected to be in JSON format and contain the
following fields:
* "manifest": Base64-encoded serialization of a manifest containing the
validator publisher's master and signing public keys.
* "blob": Base64-encoded JSON string containing a "sequence",
"expiration" and "validators" field. "expiration" contains the Ripple
timestamp (seconds since January 1st, 2000 (00:00 UTC)) for when the
list expires. "validators" contains an array of objects with a
"validation_public_key" field.
* "signature": Hex-encoded signature of the blob using the publisher's
signing key.
* "version": 1
* "refreshInterval" (optional)
Instead of specifying a static list of trusted validators in the config
or validators file, the configuration can now include trusted validator
list publisher keys.
The trusted validator list and quorum are now reset each consensus
round using the latest validator lists and the list of recent
validations seen. The minimum validation quorum is now only
configurable via the command line.
The existing configuration includes 512 and 1024 bit DH
parameters and supports ciphers such as RC4 and 3DES and
hash algorithms like SHA-1 which are no longer considered
secure.
Going forward, use only 2048-bit DH parameters and define
a new default set of modern ciphers to use:
HIGH:!aNULL:!MD5:!DSS:!SHA1:!3DES:!RC4:!EXPORT:!DSS
Additionally, allow administrators who wish to have different
settings to configure custom global and per-port ciphers suites
in the configuration file using the `ssl_ciphers` directive.
* Load specified [validators_file] relative to config dir
* Add default [validators_file] to rippled-example.cfg
* Remove [validators] and [validation_quorum] from rippled-example.cfg
* Add [validation_quorum] to validators-example.txt
* Allow validators.txt to be a symlink
* Throw for invalid [validators_file] instead of logging
* Trust own master public key from configured manifest
* Do not load untrusted manifests from database
Trusted validators are loaded from [validators] and [validator_keys]
sections from both rippled.cfg and validators.txt
Quorum is loaded from [validation_quorum] section in validators.txt
only if it is not configured in rippled.cfg
* Tweak account XRP balance and sequence if needed before preclaim.
* Limit total fees in flight to minimum reserve / account balance.
* LastLedgerSequence must be at least 2 more than the current ledger to be queued.
* Limit 10 transactions per account in the queue at a time.
* Limit queuing multiple transactions after transactions that affect authentication.
* Zero base fee transactions are treated as having a fixed fee level of 256000 instead of infinite.
* Full queue: new txn can only kick out a tx if the fee is higher than that account's average fee.
* Queued tx retry limit prevents indefinitely stuck txns.
* Return escalation factors in server_info and _state when escalated.
* Update documentation.
* Update experimental config to only include the % increase.
* Convert TxQ metric magic numbers to experimental config.
The new code removes the ability to specify domain names
in the [validators] configuration block, and no longer
supports the [validators_site] option.
More details on the supported configurations are available
under doc/rippled-example.cfg.
This is designed for use by proxies in front of rippled. Configured IPs
can forward identifying user data in HTTP headers, including
user name and origin IP. If the user name exists, then resource limits
are lifted for that session. However, administrative commands are still
reserved only for administrative sessions.
Multiple servers behind NAT might share a single public IP, making it
difficult for them to connect to the Ripple network since multiple
incoming connections from the same non-private IP are currently not
allowed.
RippleD now automatically allows between 2 and 5 incoming connections,
from the same public IP based on the total number of peers that it is
configured to accept.
Administrators can manually change the limit by adding an "ip_limit"
key value pair in the [overlay] stanza of the configuration file and
specifying a positive non-zero number. For example:
[overlay]
ip_limit=3
The previous "one connection per IP" strategy can be emulated by
setting "ip_limit" to 1.
The implementation imposes both soft and hard upper limits and will
adjust the value so that a single IP cannot consume all inbound slots.
The first few transactions are added to the open ledger at
the base fee (ie. 10 drops). Once enough transactions are
added, the required fee will jump dramatically. If additional
transactions are added, the fee will grow exponentially.
Transactions that don't have a high enough fee to be applied to
the ledger are added to the queue in order from highest fee to
lowest. Whenever a new ledger is accepted as validated, transactions
are first applied from the queue to the open ledger in fee order
until either all transactions are applied or the fee again jumps
too high for the remaining transactions.
Current implementation is restricted to one transaction in the
queue per account. Some groundwork has been laid to expand in
the future.
Note that this fee logic escalates independently of the load-based
fee logic (ie. LoadFeeTrack). Submitted transactions must meet
the load fee to be considered for the queue, and must meet both
fees to be put into open ledger.
* Add fields for local and remote IP addresses in hello.
* Add configuration for known local public IP address
* Set fields appropriately
* Check the fields
* Disallow self connection by key
* Deprecate rpc_admin_allow section from configuration file
* New port-specific setting 'admin':
* Comma-separated list of IP addresses that are allowed administrative
privileges (subject to username & password authentication if configured)
* 127.0.0.1 is no longer a default admin IP.
* 0.0.0.0 may be specified to indicate "any IP" but cannot be combined
with other IP addresses.
Inbound and outbound peer connections always use HTTP handshakes to
negotiate connections, instead of the deprecated TMHello protocol
message.
rippled versions 0.27.0 and later support both optional HTTP handshakes
and legacy TMHello messages, so always using HTTP handshakes should not
cause disruption. However, versions before 0.27.0 will no longer be
able to participate in the overlay network - support for handshaking
via the TMHello message is removed.
* A legacy value is a config section with a single-line.
* These values may be read from the BasicConfig interface so
the deprecated Config class does not need to be exposed to
clients.
* Made Config class more testable.
This adds support for a cgi /crawl request, issued over HTTPS to the configured
peer protocol port. The response to the request is a JSON object containing
the node public key, type, and IP address of each directly connected neighbor.
The IP address is suppressed unless the neighbor has requested its address
to be revealed by adding "Crawl: public" to its HTTP headers. This field is
currently set by the peer_private option in the rippled.cfg file.
The LevelDB and HyperLevelDB are removed from the backend choices. Neither
were recommended for production environments. As RocksDB is not available
on Windows platforms yet, the recommended backend choice for Windows is NuDB.
The NuDB database backend is a high performance key/value store presented
as an alternative to RocksDB on Mac and Linux deployments, and the preferred
backend option for Windows deployments. The LevelDB backend is deprecated for
all platforms.
This includes these changes:
* Add Backend::verify API for doing consistency checks
* Add Database::close so caller can catch exceptions
* Improved Timing test for NodeStore creates a simulated workload
Makes rippled configurable to support deletion of all data in its key-value
store (nodestore) and ledger and transaction SQLite databases based on
validated ledger sequence numbers. All records from a specified ledger
and forward shall remain available in the key-value store and SQLite, and
all data prior to that specific ledger may be deleted.
Additionally, the administrator may require that an RPC command be
executed to enable deletion. This is to align data deletion with local
policy.
This changes the behavior and configuration specification of the listening
ports that rippled uses to accept incoming connections for the supported
protocols: peer (Peer Protocol), http (JSON-RPC over HTTP), https (JSON-RPC)
over HTTPS, ws (Websockets Clients), and wss (Secure Websockets Clients).
Each listening port is now capable of handshaking in multiple protocols
specified in the configuration file (subject to some restrictions). Each
port can be configured to provide its own SSL certificate, or to use a
self-signed certificate. Ports can be configured to share settings, this
allows multiple ports to use the same certificate or values. The list of
ports is dynamic, administrators can open as few or as many ports as they
like. Authentication settings such as user/password or admin user/admin
password (for administrative commands on RPC or Websockets interfaces) can
also be specified per-port.
As the configuration file has changed significantly, administrators will
need to update their ripple.cfg files and carefully review the documentation
and new settings.
Changes:
* rippled-example.cfg updated with documentation and new example settings:
All obsolete websocket, rpc, and peer configuration sections have been
removed, the documentation updated, and a new documented set of example
settings added.
* HTTP::Writer abstraction for sending HTTP server requests and responses
* HTTP::Handler handler improvements to support Universal Port
* HTTP::Handler handler supports legacy Peer protocol handshakes
* HTTP::Port uses shared_ptr<boost::asio::ssl::context>
* HTTP::PeerImp and Overlay use ssl_bundle to support Universal Port
* New JsonWriter to stream message and body through HTTP server
* ServerHandler refactored to support Universal Port and legacy peers
* ServerHandler Setup struct updated for Universal Port
* Refactor some PeerFinder members
* WSDoor and Websocket code stores and uses the HTTP::Port configuration
* Websocket autotls class receives the current secure/plain SSL setting
* Remove PeerDoor and obsolete Overlay peer accept code
* Remove obsolete RPCDoor and synchronous RPC handling code
* Remove other obsolete classes, types, and files
* Command line tool uses ServerHandler Setup for port and authorization info
* Fix handling of admin_user, admin_password in administrative commands
* Fix adminRole to check credentials for Universal Port
* Updated Overlay README.md
* Overlay sends IP:port redirects on HTTP Upgrade peer connection requests:
Incoming peers who handshake using the HTTP Upgrade mechanism don't get
a slot, and always get HTTP Status 503 redirect containing a JSON
content-body with a set of alternate IP and port addresses to try, learned
from PeerFinder. A future commit related to the Hub and Spoke feature will
change the response to grant the peer a slot when there are peer slots
available.
* HTTP responses to outgoing Peer connect requests parse redirect IP:ports:
When the [overlay] configuration section (which is experimental) has
http_handshake = 1, HTTP redirect responses will have the JSON content-body
parsed to obtain the redirect IP:port addresses.
* Use a single io_service for HTTP::Server and Overlay:
This is necessary to allow HTTP::Server to pass sockets to and from Overlay
and eventually Websockets. Unfortunately Websockets is not so easily changed
to use an externally provided io_service. This will be addressed in a future
commit, and is one step necessary ease the restriction on ports configured
to offer Websocket protocols in the .cfg file.
These changes prepare Overlay for the Universal Port and Hub and Spoke
features.
* Add [overlay configuration section:
The [overlay] section uses the new BasicConfig interface that
supports key-value pairs in the section. Some exposition is added to the
example cfg file. The new settings for overlay are related to the Hub and
Spoke feature which is currently in development. Production servers should
not set these configuration options, they are clearly marked experimental
in the example cfg file.
Other changes:
* Use _MSC_VER to detect Visual Studio
* Use ssl_bundle in Overlay::Peer
* Use shared_ptr to SSL context in Overlay:
* Removed undocumented PEER_SSL_CIPHER_LIST configuration setting
* Add Section::name: The Section object now stores its name for better diagnostic messages.
This configuration section uses the new BasicConfig interface that supports
key-value pairs in the section. Some exposition is added to the example cfg
file. The new settings for overlay are related to the Hub and Spoke feature
which is currently in development. Production servers should not set
these configuration options, they are clearly marked experimental in the
example cfg file.
Conflicts:
src/ripple/overlay/impl/OverlayImpl.cpp
src/ripple/overlay/impl/OverlayImpl.h
src/ripple/overlay/impl/PeerImp.cpp
src/ripple/overlay/impl/PeerImp.h
* Set default node_db to RocksDB from HyperlevelDB, to match gateway
appliances and Ripple Labs server configurations.
* Set some rpc_startup options to reduce logging and save disk space.
* More sensible documentation regarding WebSocket SSL directives
Signed-off-by: The Doctor <drwho@virtadpt.net>
