* New src/ripple/crypto and src/ripple/protocol directories
* Merged src/ripple/common into src/ripple/basics
* Move resource/api files up a level
* Add headers for "include what you use"
* Normalized include guards
* Renamed to JsonFields.h
* Remove obsolete files
* Remove net.h unity header
* Remove resource.h unity header
* Removed some deprecated unity includes
This changes the behavior and configuration specification of the listening
ports that rippled uses to accept incoming connections for the supported
protocols: peer (Peer Protocol), http (JSON-RPC over HTTP), https (JSON-RPC)
over HTTPS, ws (Websockets Clients), and wss (Secure Websockets Clients).
Each listening port is now capable of handshaking in multiple protocols
specified in the configuration file (subject to some restrictions). Each
port can be configured to provide its own SSL certificate, or to use a
self-signed certificate. Ports can be configured to share settings, this
allows multiple ports to use the same certificate or values. The list of
ports is dynamic, administrators can open as few or as many ports as they
like. Authentication settings such as user/password or admin user/admin
password (for administrative commands on RPC or Websockets interfaces) can
also be specified per-port.
As the configuration file has changed significantly, administrators will
need to update their ripple.cfg files and carefully review the documentation
and new settings.
Changes:
* rippled-example.cfg updated with documentation and new example settings:
All obsolete websocket, rpc, and peer configuration sections have been
removed, the documentation updated, and a new documented set of example
settings added.
* HTTP::Writer abstraction for sending HTTP server requests and responses
* HTTP::Handler handler improvements to support Universal Port
* HTTP::Handler handler supports legacy Peer protocol handshakes
* HTTP::Port uses shared_ptr<boost::asio::ssl::context>
* HTTP::PeerImp and Overlay use ssl_bundle to support Universal Port
* New JsonWriter to stream message and body through HTTP server
* ServerHandler refactored to support Universal Port and legacy peers
* ServerHandler Setup struct updated for Universal Port
* Refactor some PeerFinder members
* WSDoor and Websocket code stores and uses the HTTP::Port configuration
* Websocket autotls class receives the current secure/plain SSL setting
* Remove PeerDoor and obsolete Overlay peer accept code
* Remove obsolete RPCDoor and synchronous RPC handling code
* Remove other obsolete classes, types, and files
* Command line tool uses ServerHandler Setup for port and authorization info
* Fix handling of admin_user, admin_password in administrative commands
* Fix adminRole to check credentials for Universal Port
* Updated Overlay README.md
* Overlay sends IP:port redirects on HTTP Upgrade peer connection requests:
Incoming peers who handshake using the HTTP Upgrade mechanism don't get
a slot, and always get HTTP Status 503 redirect containing a JSON
content-body with a set of alternate IP and port addresses to try, learned
from PeerFinder. A future commit related to the Hub and Spoke feature will
change the response to grant the peer a slot when there are peer slots
available.
* HTTP responses to outgoing Peer connect requests parse redirect IP:ports:
When the [overlay] configuration section (which is experimental) has
http_handshake = 1, HTTP redirect responses will have the JSON content-body
parsed to obtain the redirect IP:port addresses.
* Use a single io_service for HTTP::Server and Overlay:
This is necessary to allow HTTP::Server to pass sockets to and from Overlay
and eventually Websockets. Unfortunately Websockets is not so easily changed
to use an externally provided io_service. This will be addressed in a future
commit, and is one step necessary ease the restriction on ports configured
to offer Websocket protocols in the .cfg file.
The stop sequence for Overlay had a race condition where autoconnect could
be called after close_all, resulting in a hang on exit. This resolves the
problem by putting the close and timer operations on a strand:
* Rename some Overlay members
* Put close on strand and tidy up members
* Use completion handler instead of coroutine for timer
* Use App io_service in PeerFinder
This new factory is intended for benchmarking against the existing RocksDBFactory and has the following differences.
* Does not use BatchWriter
* Disables WAL for writes to memtable
* Uses a hash index in blocks
* Uses RocksDB OptimizeFor… functions
See Benchmarks.md for further discussion of some of the issues raised by investigation of RocksDB performance.
The Stoppable interface aids in the enforcement of invariants needed to
successful start and stop a multi-threaded application composed of classes
that depend on each other in complex ways.
* Test written to confirm the current behavior.
* Comments updated to reflect the current behavior.
* Public API reduced to what is currently in use.
* Protected data members made private.
* volatile bool members changed to std::atomic<bool>.
* std::atomic<int> members changed to std::atomic<bool>.
* Name storage uses std::string
These changes are necessary to support the Universal port feature. Synopsis:
* Persist HTTP peer io_service::work lifetime:
This simplification eliminates any potential for bugs caused by incorrect
lifetime management of the io_service::work object.
* Restructure Door to prevent data races, and handle clean exit:
The Server, Door, Door::detector, and Peer objects work together to
correctly implement graceful stop and destructors that block until
all child objects have been destroyed.
Cleanups:
* De-pimpl HTTP::Server
* Rename ServerImpl data members
* Tidy up HTTP::Port interface
This is class whose interface is identical to the boost::asio::basic_streambuf,
and uses an implementation that stores the data in multiple discontiguous
linear buffers, expanding and shrinking as needed.
Beast includes a lot of code for encapsulating cross-platform differences
which are not used or needed by rippled. Additionally, a lot of that code
implements functionality that is available from the standard library.
This moves away from custom implementations of features that the standard
library provides and reduces the number of platform-specific interfaces
andfeatures that Beast makes available.
Highlights include:
* Use std:: instead of beast implementations when possible
* Reduce the use of beast::String in public interfaces
* Remove Windows-specific COM and Registry code
* Reduce the public interface of beast::File
* Reduce the public interface of beast::SystemStats
* Remove unused sysctl/getsysinfo functions
* Remove beast::Logger
This is a cleanup to the structure of the sources.
* Rename to ServerHandler
* Move private implementation declaration to separate header
* De-inline function definitions in the class declaration.
Many classes required to support type-erasure of handlers and boost::asio
types are now obsolete, so these classes and files are removed:
HTTPClientType, FixedInputBuffer, PeerRole, socket_wrapper,
client_session, basic_url, abstract_socket, buffer_sequence, memory_buffer,
enable_wait_for_async, shared_handler, wrap_handler, streambuf,
ContentBodyBuffer, SSLContext, completion-handler based handshake detectors.
These structural changes are made:
* Some missing includes added to headers
* asio module directory flattened
* Removed MultiSocket. Code that previously used the MultiSocket now uses
a combination of boost::asio coroutines and CRTP.
* Sitefiles headers rolled up and directory flattened.
* Disabled Sitefiles use of deprecated HTTPClient.
* Validators headers tidied up.
* Disabled Validators use of deprecated HTTPClient.
The MultiSocket is obsolete technology which is superceded by a more
straightforward, template based implementation that is compatible with
boost::asio::coroutines. This removes support for the unused PROXY handshake
feature. After this change a large number of classes and source files may be
removed.
When JSON-RPC and Websocket responses are calculated, the result is stored
in intermediate Json::Value objects and later composed in a single linear
memory buffer before being sent to the socket. These classes support a
new model for building responses that supports incremental construction
of JSON replies in constant time and removes the requirement that all
data returned be located in continuguous memory.
* New JsonWriter incrementally writes JSON with O(1) granularity and memory.
* Array, Object are RAII wrappers for the O(1) JsonWriter.
This class was used to allow stream style operator<< to write to the
HTTP::Session. This is being superceded by a more robust object-based model
that supports coroutines.
This works around the limitation that 1.56 boost::asio::ssl::stream objects
do not support r-value move or construction. It is required when the stream
does not own the socket.
The MultiSocket class implements a socket that handshakes in multiple
protocols including SSL and PROXY. Unfortunately the way it type-erases the
handlers and buffers is incompatible with boost::asio coroutines. To pave the
way for coroutines this is part of a larger set of changes that roll back the
usage of MultiSocket to older code, and some custom implementations that use
templates. The custom implementations are more simple since they use
coroutines. Removing MultiSocket will make many other classes and source files
unused, a big win for trimming down the codebase size.
Previously, the PeerFinder manager constructed with a Callback object
provided by the owner which was used to perform operations like connecting,
disconnecting, and sending messages. This made it difficult to change the
overlay code because a single call into the PeerFinder could cause both
OverlayImpl and PeerImp to be re-entered one or more times, sometimes while
holding a recursive mutex. This change eliminates the callback by changing
PeerFinder functions to return values indicating the action the caller should
take.
As a result of this change the PeerFinder no longer needs its own dedicated
thread. OverlayImpl is changed to call into PeerFinder on a timer to perform
periodic activities. Furthermore the Checker class used to perform connectivity
checks has been refactored. It no longer uses an abstract base class, in order
to not type-erase the handler passed to async_connect (ensuring compatibility
with coroutines). To allow unit tests that don't need a network, the Logic
class is now templated on the Checker type. Currently the Manager provides its
own io_service. However, this can easily be changed so that the io_service is
provided upon construction.
Summary
* Remove unused SiteFiles dependency injection
* Remove Callback and update signatures for public APIs
* Remove obsolete functions
* Move timer to overlay
* Steps toward a shared io_service
* Templated, simplified Checker
* Tidy up Checker declaration
The remoteAddress is the address as seen on the socket, which for
incoming connections has a random port chosen by the remote implementation
that is different from the port number used to accept connections by the
remote listening socket. The checkedAddress is the remote address as seen
on the socket, combined with the port advertised in the TMEndpoints message.
This fixes the reporting and metadata associated with addresses tested
for connectivity.
The README has been updated to reflect that uptime is no longer part of
the metadata associated with IP addresses saved for bootstrapping.
* Fix bug with more than one complete request in a read buffer
* Use stackful coroutines for simplified control flow
* Door refactored to detect handshakes
* Remove dependency on MultiSocket
* Remove dependency on handshake detect logic framework
* Remove unused functions
* Remove unused constructor
* Use delegating constructors
* Mark some observers deprecated
* Clean up declaration parameter names
* Add checked and unchecked constructors
* De-inline unnecessary inlined functions
* Reorder and regroup members into sections
* Move globals from the unity file to the .cpp
* Change some member functions to be free functions
* Put implementation in one .cpp and the test in another .cpp
Remove unused STAmount constructor and delegate two others No change in functionality.
