New classes are introduced to represent HTTP messages and their
associated bodies. The parser interface is reworked to use CRTP,
error codes, and trait checks.
New classes:
* basic_headers
Models field/value pairs in a HTTP message.
* message
Models a HTTP message, body behavior defined by template argument.
Parsed message carries metadata generated during parsing.
* parser
Produces parsed messages.
* empty_body, string_body, basic_streambuf_body
Classes used to represent content bodies in various ways.
New functions:
* read, async_read, write, async_write
Read and write HTTP messages on a socket.
New concepts:
* Body: Represents the HTTP Content-Body.
* Field: A HTTP header field.
* FieldSequence: A forward sequence of fields.
* Reader: Parses a Body from a stream of bytes.
* Writer: Serializes a Body to buffers.
basic_parser changes:
* add write methods which throw exceptions instead
* error_code passed via parameter instead of return value
* fold private member calls into existing callbacks
* basic_parser uses CRTP instead of virtual members
* add documentation on Derived requirements for CRTP
impl/http-parser changes:
* joyent renamed to nodejs to reflect upstream changes
Replace Journal public data members with member function accessors
in order to make Journal lighter weight. The change makes a
Journal cheaper to pass by value.
Also add missing stream checks (e.g., calls to JLOG) to avoid
text processing that ultimately will not be stored in the log.
The RippleAddress class was used to represent a number of fundamentally
different types: account public keys, account secret keys, node public
keys, node secret keys, seeds and generators.
The class is replaced by the following types:
* PublicKey for account and node public keys
* SecretKey for account and node private keys
* Generator for generating secp256k1 accounts
* Seed for account, node and generator seeds
* Add fields for local and remote IP addresses in hello.
* Add configuration for known local public IP address
* Set fields appropriately
* Check the fields
* Disallow self connection by key
These routines replace existing code to compute SHA512-Half hashes.
The new code accumulates serialized data into a hashing context
instead of allocating a buffer, for improved performance.
Inbound and outbound peer connections always use HTTP handshakes to
negotiate connections, instead of the deprecated TMHello protocol
message.
rippled versions 0.27.0 and later support both optional HTTP handshakes
and legacy TMHello messages, so always using HTTP handshakes should not
cause disruption. However, versions before 0.27.0 will no longer be
able to participate in the overlay network - support for handshaking
via the TMHello message is removed.
* Remove unused members
* SerialIter holds only a pointer and offset now
* Use free functions for some Serializer members
* Use SerialIter in some places instead of Serializer
An alternative to the unity build, the classic build compiles each
translation unit individually. This adds more modules to the classic build:
* Remove unity header app.h
* Add missing includes as needed
* Remove obsolete NodeStore backend code
* Add app/, core/, crypto/, json/, net/, overlay/, peerfinder/ to classic build
All of the logic for establishing an outbound peer connection including
the initial HTTP handshake exchange is moved into a separate class. This
allows PeerImp to have a strong invariant: All PeerImp objects that exist
represent active peer connections that have already gone through the
handshake process.
This introduces a considerable change in the way that peers handshake. Instead
of sending the TMHello protocol message, the peer making the connection (client
role) sends an HTTP Upgrade request along with some special headers. The peer
acting in the server role sends an HTTP response completing the upgrade and
transition to RTXP (Ripple Transaction Protocol, a.k.a. peer protocol). If the
server has no available slots, then it sends a 503 Service Unavailable HTTP
response with a JSON content-body containing IP addresses of other servers to
try. The information that was previously contained in the TMHello message is
now communicated in the HTTP request and HTTP response including the secure
cookie to prevent man in the middle attacks. This information is documented
in the overlay README.md file.
To prevent disruption on the network, the handshake feature is rolled out in
two parts. This is part 1, where new servents acting in the client role will
send the old style TMHello handshake, and new servents acting in the server
role can automatically detect and accept both the old style TMHello handshake,
or the HTTP request accordingly. This detection happens in the Server module,
which supports the universal port. An experimental .cfg setting allows clients
to instead send HTTP handshakes when establishing peer connections. When this
code has reached a significant fraction of the network, these clients will be
able to establish a connection to the Ripple network using HTTP handshakes.
These changes clean up the handling of the socket for peers. It fixes a long
standing bug in the graceful close sequence, where remaining data such as the
IP addresses of other servers to try, did not get sent. Redundant state
variables for the peer are removed and the treatment of completion handlers is
streamlined. The treatment of SSL short reads and secure shutdown is also fixed.
Logging for the peers in the overlay module are divided into two partitions:
"Peer" and "Protocol". The Peer partition records activity taking place on the
socket while the Protocol partition informs about RTXP specific actions such as
transaction relay, fetch packs, and consensus rounds. The severity on the log
partitions may be adjusted independently to diagnose problems. Every log
message for peers is prefixed with a small, unique integer id in brackets,
to accurately associate log messages with peers.
HTTP handshaking is the first step in implementing the Hub and Spoke feature,
which transforms the network from a homogeneous network where all peers are
the same, into a structured network where peers with above average capabilities
in their ability to process ledgers and transactions self-assemble to form a
backbone of high powered machines which in turn serve a much larger number of
'leaves' with lower capacities with a goal to improve the number of
transactions that may be retired over time.
