HTTP Handshaking for Peers on Universal Port (RIPD-446):

This introduces a considerable change in the way that peers handshake. Instead
of sending the TMHello protocol message, the peer making the connection (client
role) sends an HTTP Upgrade request along with some special headers. The peer
acting in the server role sends an HTTP response completing the upgrade and
transition to RTXP (Ripple Transaction Protocol, a.k.a. peer protocol). If the
server has no available slots, then it sends a 503 Service Unavailable HTTP
response with a JSON content-body containing IP addresses of other servers to
try. The information that was previously contained in the TMHello message is
now communicated in the HTTP request and HTTP response including the secure
cookie to prevent man in the middle attacks. This information is documented
in the overlay README.md file.

To prevent disruption on the network, the handshake feature is rolled out in
two parts. This is part 1, where new servents acting in the client role will
send the old style TMHello handshake, and new servents acting in the server
role can automatically detect and accept both the old style TMHello handshake,
or the HTTP request accordingly. This detection happens in the Server module,
which supports the universal port. An experimental .cfg setting allows clients
to instead send HTTP handshakes when establishing peer connections. When this
code has reached a significant fraction of the network, these clients will be
able to establish a connection to the Ripple network using HTTP handshakes.

These changes clean up the handling of the socket for peers. It fixes a long
standing bug in the graceful close sequence, where remaining data such as the
IP addresses of other servers to try, did not get sent. Redundant state
variables for the peer are removed and the treatment of completion handlers is
streamlined. The treatment of SSL short reads and secure shutdown is also fixed.

Logging for the peers in the overlay module are divided into two partitions:
"Peer" and "Protocol". The Peer partition records activity taking place on the
socket while the Protocol partition informs about RTXP specific actions such as
transaction relay, fetch packs, and consensus rounds. The severity on the log
partitions may be adjusted independently to diagnose problems. Every log
message for peers is prefixed with a small, unique integer id in brackets,
to accurately associate log messages with peers.

HTTP handshaking is the first step in implementing the Hub and Spoke feature,
which transforms the network from a homogeneous network where all peers are
the same, into a structured network where peers with above average capabilities
in their ability to process ledgers and transactions self-assemble to form a
backbone of high powered machines which in turn serve a much larger number of
'leaves' with lower capacities with a goal to improve the number of
transactions that may be retired over time.

Builds/VisualStudio2013/RippleD.vcxproj

@@ -2506,7 +2506,10 @@
     </ClCompile>
     <ClInclude Include="..\..\src\ripple\overlay\impl\PeerImp.h">
     </ClInclude>
-    <ClInclude Include="..\..\src\ripple\overlay\impl\peer_protocol_detector.h">
+    <ClCompile Include="..\..\src\ripple\overlay\impl\TMHello.cpp">
+      <ExcludedFromBuild>True</ExcludedFromBuild>
+    </ClCompile>
+    <ClInclude Include="..\..\src\ripple\overlay\impl\TMHello.h">
     </ClInclude>
     <ClInclude Include="..\..\src\ripple\overlay\impl\Tuning.h">
     </ClInclude>
@@ -2525,6 +2528,9 @@
     <ClCompile Include="..\..\src\ripple\overlay\tests\short_read.test.cpp">
       <ExcludedFromBuild>True</ExcludedFromBuild>
     </ClCompile>
+    <ClCompile Include="..\..\src\ripple\overlay\tests\TMHello.test.cpp">
+      <ExcludedFromBuild>True</ExcludedFromBuild>
+    </ClCompile>
     <ClCompile Include="..\..\src\ripple\peerfinder\impl\Bootcache.cpp">
       <ExcludedFromBuild>True</ExcludedFromBuild>
     </ClCompile>

Builds/VisualStudio2013/RippleD.vcxproj.filters

@@ -3531,7 +3531,10 @@
     <ClInclude Include="..\..\src\ripple\overlay\impl\PeerImp.h">
       <Filter>ripple\overlay\impl</Filter>
     </ClInclude>
-    <ClInclude Include="..\..\src\ripple\overlay\impl\peer_protocol_detector.h">
+    <ClCompile Include="..\..\src\ripple\overlay\impl\TMHello.cpp">
+      <Filter>ripple\overlay\impl</Filter>
+    </ClCompile>
+    <ClInclude Include="..\..\src\ripple\overlay\impl\TMHello.h">
       <Filter>ripple\overlay\impl</Filter>
     </ClInclude>
     <ClInclude Include="..\..\src\ripple\overlay\impl\Tuning.h">
@@ -3558,6 +3561,9 @@
     <ClCompile Include="..\..\src\ripple\overlay\tests\short_read.test.cpp">
       <Filter>ripple\overlay\tests</Filter>
     </ClCompile>
+    <ClCompile Include="..\..\src\ripple\overlay\tests\TMHello.test.cpp">
+      <Filter>ripple\overlay\tests</Filter>
+    </ClCompile>
     <ClCompile Include="..\..\src\ripple\peerfinder\impl\Bootcache.cpp">
       <Filter>ripple\peerfinder\impl</Filter>
     </ClCompile>

src/ripple/app/consensus/LedgerConsensus.cpp

@@ -1963,7 +1963,7 @@ private:
 
     #if 0
     // FIXME: We can't do delayed relay because we don't have the signature
-                std::set<Peer::ShortId> peers
+                std::set<Peer::id_t> peers
 
                 if (relay && getApp().getHashRouter ().swapSet (proposal.getSuppress (), set, SF_RELAYED))
                 {

src/ripple/app/misc/IHashRouter.h

@@ -44,7 +44,7 @@ namespace ripple {
 class IHashRouter
 {
 public:
-    // The type here *MUST* match the type of Peer::ShortId
+    // The type here *MUST* match the type of Peer::id_t
     typedef std::uint32_t PeerShortID;
 
     // VFALCO NOTE this preferred alternative to default parameters makes

src/ripple/app/misc/NetworkOPs.cpp

@@ -1031,7 +1031,7 @@ Transaction::pointer NetworkOPsImp::processTransactionCb (
 
         if (didApply || ((mMode != omFULL) && !bFailHard && bLocal))
         {
-            std::set<Peer::ShortId> peers;
+            std::set<Peer::id_t> peers;
 
             if (getApp().getHashRouter ().swapSet (
                     trans->getID (), peers, SF_RELAYED))
@@ -1593,7 +1593,7 @@ void NetworkOPsImp::processTrustedProposal (
 
         if (relay)
         {
-            std::set<Peer::ShortId> peers;
+            std::set<Peer::id_t> peers;
             if (getApp().getHashRouter ().swapSet (
                 proposal->getSuppressionID (), peers, SF_RELAYED))
             {

src/ripple/app/peers/PeerSet.cpp

@@ -57,7 +57,7 @@ bool PeerSet::peerHas (Peer::ptr const& ptr)
 {
     ScopedLockType sl (mLock);
 
-    if (!mPeers.insert (std::make_pair (ptr->getShortId (), 0)).second)
+    if (!mPeers.insert (std::make_pair (ptr->id (), 0)).second)
         return false;
 
     newPeer (ptr);
@@ -67,7 +67,7 @@ bool PeerSet::peerHas (Peer::ptr const& ptr)
 void PeerSet::badPeer (Peer::ptr const& ptr)
 {
     ScopedLockType sl (mLock);
-    mPeers.erase (ptr->getShortId ());
+    mPeers.erase (ptr->id ());
 }
 
 void PeerSet::setTimer ()

src/ripple/app/peers/PeerSet.h

@@ -154,7 +154,7 @@ protected:
     boost::asio::deadline_timer             mTimer;
 
     // VFALCO TODO Verify that these are used in the way that the names suggest.
-    typedef Peer::ShortId PeerIdentifier;
+    typedef Peer::id_t PeerIdentifier;
     typedef int ReceivedChunkCount;
     typedef hash_map <PeerIdentifier, ReceivedChunkCount> PeerSetMap;
 

src/ripple/overlay/Overlay.h

@@ -118,7 +118,7 @@ public:
     /** Returns the peer with the matching short id, or null. */
     virtual
     Peer::ptr
-    findPeerByShortID (Peer::ShortId const& id) = 0;
+    findPeerByShortID (Peer::id_t const& id) = 0;
 
     /** Visit every active peer and return a value
         The functor must:

src/ripple/overlay/Peer.h

@@ -40,31 +40,54 @@ class Peer
 public:
     typedef std::shared_ptr <Peer> ptr;
 
-    /** Uniquely identifies a particular connection of a peer. */
+    /** Uniquely identifies a peer.
-    typedef std::uint32_t ShortId;
+        This can be stored in tables to find the peer later. Callers
+        can discover if the peer is no longer connected and make
+        adjustments as needed.
+    */
+    using id_t = std::uint32_t;
 
     //
     // Network
     //
 
-    virtual void send (Message::pointer const& m) = 0;
+    virtual
-    virtual beast::IP::Endpoint getRemoteAddress() const = 0;
+    void
+    send (Message::pointer const& m) = 0;
+
+    virtual
+    beast::IP::Endpoint
+    getRemoteAddress() const = 0;
 
     /** Adjust this peer's load balance based on the type of load imposed. */
-    virtual void charge (Resource::Charge const& fee) = 0;
+    virtual
+    void
+    charge (Resource::Charge const& fee) = 0;
 
     //
     // Identity
     //
 
-    virtual ShortId getShortId () const = 0;
+    virtual
-    virtual RippleAddress const& getNodePublic () const = 0;
+    id_t
-    virtual Json::Value json () = 0;
+    id() const = 0;
+
+    virtual
+    RippleAddress const&
+    getNodePublic() const = 0;
+
+    virtual
+    Json::Value json() = 0;
+
     // VFALCO TODO Replace both with
     //             boost::optional<std::string> const& cluster_id();
-    //
+    virtual
-    virtual bool isInCluster () const = 0;
+    bool
-    virtual std::string const& getClusterNodeName() const = 0;
+    isInCluster() const = 0;
+
+    virtual
+    std::string const&
+    getClusterNodeName() const = 0;
 
     //
     // Ledger

src/ripple/overlay/README.md

@@ -63,12 +63,11 @@ custom fields to communicate protocol specific information:
 
 ```
 GET / HTTP/1.1
-User-Agent: Ripple-0.26.0
+User-Agent: rippled-0.27.0
 Local-Address: 192.168.0.101:8421
-Upgrade: Ripple/1.2, Ripple/1.3
+Upgrade: RTXP/1.2, RTXP/1.3
 Connection: Upgrade
 Connect-As: Leaf, Peer
-Content-Length: 0
 Accept-Encoding: identity, zlib, snappy
 Protocol-Public-Key: aBRoQibi2jpDofohooFuzZi9nEzKw9Zdfc4ExVNmuXHaJpSPh8uJ
 Protocol-Session-Cookie: 71ED064155FFADFA38782C5E0158CB26
@@ -79,9 +78,9 @@ HTTP response indicating the connection status:
 
 ```
 HTTP/1.1 101 Switching Protocols
-Server: Ripple-0.26.5
+Server: rippled-0.27.0
 Remote-Address: 63.104.209.13
-Upgrade: Ripple/1.2
+Upgrade: RTXP/1.2
 Connection: Upgrade
 Connect-As: Leaf
 Transfer-Encoding: snappy
@@ -96,7 +95,7 @@ servers that may have open slots:
 
 ```
 HTTP/1.1 503 Service Unavailable
-Server: Ripple-0.26.5
+Server: rippled-0.27.0
 Remote-Address: 63.104.209.13
 Content-Length: 253
 Content-Type: application/json
@@ -138,7 +137,7 @@ Content-Type: application/json
 * `Upgrade`
 
     This field must be present and for requests consist of a comma delimited
-    list of at least one element where each element is of the form "Ripple/"
+    list of at least one element where each element is of the form "RTXP/"
     followed by the dotted major and minor protocol version number. For
     responses the value must be a single element matching one of the elements
     provided in the corresponding request field. If the server does not

src/ripple/overlay/impl/OverlayImpl.cpp

@@ -22,8 +22,11 @@
 #include <ripple/server/JsonWriter.h>
 #include <ripple/overlay/impl/OverlayImpl.h>
 #include <ripple/overlay/impl/PeerImp.h>
+#include <ripple/overlay/impl/TMHello.h>
 #include <ripple/peerfinder/make_Manager.h>
 #include <beast/ByteOrder.h>
+#include <beast/http/rfc2616.h>
+#include <beast/utility/ci_char_traits.h>
 
 namespace ripple {
 
@@ -69,7 +72,7 @@ OverlayImpl::Timer::Timer (OverlayImpl& overlay)
 }
 
 void
-OverlayImpl::Timer::close()
+OverlayImpl::Timer::stop()
 {
     error_code ec;
     timer_.cancel(ec);
@@ -128,14 +131,14 @@ OverlayImpl::OverlayImpl (
         pathToDbFileOrDirectory, get_seconds_clock(),
             deprecatedLogs().journal("PeerFinder")))
     , m_resolver (resolver)
-    , m_nextShortId (0)
+    , next_id_(1)
 {
     beast::PropertyStream::Source::add (m_peerFinder.get());
 }
 
 OverlayImpl::~OverlayImpl ()
 {
-    close();
+    stop();
 
     // Block until dependent objects have been destroyed.
     // This is just to catch improper use of the Stoppable API.
@@ -149,8 +152,7 @@ OverlayImpl::~OverlayImpl ()
 void
 OverlayImpl::onLegacyPeerHello (
     std::unique_ptr<beast::asio::ssl_bundle>&& ssl_bundle,
-        boost::asio::const_buffer buffer,
+        boost::asio::const_buffer buffer, endpoint_type remote_endpoint)
-            boost::asio::ip::tcp::endpoint remote_address)
 {
     error_code ec;
     auto const local_endpoint (ssl_bundle->socket.local_endpoint(ec));
@@ -159,51 +161,130 @@ OverlayImpl::onLegacyPeerHello (
 
     auto const slot = m_peerFinder->new_inbound_slot (
         beast::IPAddressConversion::from_asio(local_endpoint),
-            beast::IPAddressConversion::from_asio(remote_address));
+            beast::IPAddressConversion::from_asio(remote_endpoint));
 
-    if (slot != nullptr)
+    if (slot == nullptr)
-        return addpeer (std::make_shared<PeerImp>(std::move(ssl_bundle),
+        // self connect, close
-            boost::asio::const_buffers_1(buffer),
+        return;
-                beast::IPAddressConversion::from_asio(remote_address),
+
-                    *this, m_resourceManager, *m_peerFinder, slot));
+    auto const peer = std::make_shared<PeerImp>(std::move(ssl_bundle),
+        boost::asio::const_buffers_1(buffer), remote_endpoint, *this,
+            m_resourceManager, *m_peerFinder, slot, next_id_++);
+    {
+        // As we are not on the strand, run() must be called
+        // while holding the lock, otherwise new I/O can be
+        // queued after a call to stop().
+        std::lock_guard <decltype(mutex_)> lock (mutex_);
+        add(peer);    
+        peer->run();
+    }
 }
 
 Handoff
 OverlayImpl::onHandoff (std::unique_ptr <beast::asio::ssl_bundle>&& ssl_bundle,
     beast::http::message&& request,
-        boost::asio::ip::tcp::endpoint remote_address)
+        endpoint_type remote_endpoint)
 {
     Handoff handoff;
     if (! isPeerUpgrade(request))
         return handoff;
 
+    handoff.moved = true;
+
+    if (journal_.trace) journal_.trace <<
+        "Peer connection upgrade from " << remote_endpoint;
+
     error_code ec;
     auto const local_endpoint (ssl_bundle->socket.local_endpoint(ec));
     if (ec)
     {
-        // log?
+        if (journal_.trace) journal_.trace <<
-        // Since we don't call std::move the socket will be closed.
+            "Peer " << remote_endpoint << " failed: " << ec.message();
+        return handoff;
+    }
+
+    auto consumer = m_resourceManager.newInboundEndpoint(
+        beast::IPAddressConversion::from_asio(remote_endpoint));
+    if (consumer.disconnect())
+        return handoff;
+
+    auto const slot = m_peerFinder->new_inbound_slot (
+        beast::IPAddressConversion::from_asio(local_endpoint),
+            beast::IPAddressConversion::from_asio(remote_endpoint));
+
+    if (slot == nullptr)
+    {
+        // self-connect, close
         handoff.moved = false;
         return handoff;
     }
 
     // TODO Validate HTTP request
 
-    auto const slot = m_peerFinder->new_inbound_slot (
-        beast::IPAddressConversion::from_asio(local_endpoint),
-            beast::IPAddressConversion::from_asio(remote_address));
-
-    if (slot == nullptr)
     {
-        // self connect
+        auto const types = beast::rfc2616::split_commas(
+            request.headers["Connect-As"]);
+        if (std::find_if(types.begin(), types.end(),
+                [](std::string const& s)
+                {
+                    return beast::ci_equal(s,
+                        std::string("peer"));
+                }) == types.end())
+        {
+            handoff.moved = false;
+            handoff.response = makeRedirectResponse(slot, request,
+                remote_endpoint.address());
+            handoff.keep_alive = request.keep_alive();
+            return handoff;
+        }
+    }
+
+    handoff.moved = true;
+    bool success = true;
+    protocol::TMHello hello;
+    std::tie(hello, success) = parseHello (request, journal_);
+    if(! success)
+        return handoff;
+    uint256 sharedValue;
+    std::tie(sharedValue, success) = makeSharedValue(
+        ssl_bundle->stream.native_handle(), journal_);
+    if(! success)
+        return handoff;
+    RippleAddress publicKey;
+    std::tie(publicKey, success) = verifyHello (hello,
+        sharedValue, journal_, getApp());
+    if(! success)
+        return handoff;
+
+    std::string name;
+    bool clusterNode = getApp().getUNL().nodeInCluster(
+        publicKey, name);
+    auto const result = m_peerFinder->activate (slot,
+        RipplePublicKey(publicKey), clusterNode);
+
+    if (result != PeerFinder::Result::success)
+    {
+        if (journal_.trace) journal_.trace <<
+            "Peer " << remote_endpoint << " redirected, slots full";
         handoff.moved = false;
+        handoff.response = makeRedirectResponse(slot, request,
+            remote_endpoint.address());
+        handoff.keep_alive = request.keep_alive();
         return handoff;
     }
 
-    // For now, always redirect
+    auto const peer = std::make_shared<PeerImp>(std::move(ssl_bundle),
-    // Full, give them some addresses
+        std::move(request), hello, remote_endpoint, publicKey, consumer,
-    handoff.response = makeRedirectResponse(slot, request);
+            slot,  *this, m_resourceManager, *m_peerFinder, next_id_++);
-    handoff.keep_alive = request.keep_alive();
+    {
+        // As we are not on the strand, run() must be called
+        // while holding the lock, otherwise new I/O can be
+        // queued after a call to stop().
+        std::lock_guard <decltype(mutex_)> lock (mutex_);
+        add(peer);
+        peer->run();
+    }
+    handoff.moved = true;
     return handoff;
 }
 
@@ -214,14 +295,18 @@ OverlayImpl::isPeerUpgrade(beast::http::message const& request)
 {
     if (! request.upgrade())
         return false;
-    if (request.headers["Upgrade"] != "Ripple/1.2")
+    auto const versions = parse_ProtocolVersions(
+        request.headers["Upgrade"]);
+    if (versions.size() == 0)
+        return false;
+    if (! request.request() && request.status() != 101)
         return false;
     return true;
 }
 
 std::shared_ptr<HTTP::Writer>
 OverlayImpl::makeRedirectResponse (PeerFinder::Slot::ptr const& slot,
-    beast::http::message const& request)
+    beast::http::message const& request, address_type remote_address)
 {
     Json::Value json(Json::objectValue);
     {
@@ -235,6 +320,7 @@ OverlayImpl::makeRedirectResponse (PeerFinder::Slot::ptr const& slot,
     m.request(false);
     m.status(503);
     m.reason("Service Unavailable");
+    m.headers.append("Remote-Address", remote_address.to_string());
     m.version(request.version());
     if (request.version() == std::make_pair(1, 0))
     {
@@ -251,20 +337,20 @@ OverlayImpl::connect (beast::IP::Endpoint const& remote_endpoint)
 {
     assert(work_);
 
-    PeerFinder::Slot::ptr const slot (
+    PeerFinder::Slot::ptr const slot =
-        m_peerFinder->new_outbound_slot (remote_endpoint));
+        m_peerFinder->new_outbound_slot (remote_endpoint);
-
     if (slot == nullptr)
         return;
-
+    auto const peer = std::make_shared <PeerImp> (remote_endpoint,
-    addpeer (std::make_shared <PeerImp> (remote_endpoint, io_service_, *this,
+        io_service_, *this, m_resourceManager, *m_peerFinder, slot,
-        m_resourceManager, *m_peerFinder, slot, setup_.context));
+            setup_.context, next_id_++);
-}
+    {
-
+        // We're on the strand but lets make this code
-Peer::ShortId
+        // the same as the others to avoid confusion.
-OverlayImpl::next_id()
+        std::lock_guard <decltype(mutex_)> lock (mutex_);
-{
+        add(peer);
-    return ++m_nextShortId;
+        peer->run();
+    }
 }
 
 //--------------------------------------------------------------------------
@@ -273,8 +359,8 @@ void
 OverlayImpl::remove (PeerFinder::Slot::ptr const& slot)
 {
     std::lock_guard <decltype(mutex_)> lock (mutex_);
-    PeersBySlot::iterator const iter (m_peers.find (slot));
+    auto const iter = m_peers.find (slot);
-    assert (iter != m_peers.end ());
+    assert(iter != m_peers.end ());
     m_peers.erase (iter);
 }
 
@@ -284,6 +370,14 @@ OverlayImpl::remove (PeerFinder::Slot::ptr const& slot)
 //
 //--------------------------------------------------------------------------
 
+// Caller must hold the mutex
+void
+OverlayImpl::checkStopped ()
+{
+    if (isStopping() && areChildrenStopped () && list_.empty())
+        stopped();
+}
+
 void
 OverlayImpl::onPrepare()
 {
@@ -321,17 +415,14 @@ OverlayImpl::onPrepare()
     if (!bootstrapIps.empty ())
     {
         m_resolver.resolve (bootstrapIps,
-            [this](
+            [this](std::string const& name,
-                std::string const& name,
                 std::vector <beast::IP::Endpoint> const& addresses)
             {
                 std::vector <std::string> ips;
-
+                ips.reserve(addresses.size());
                 for (auto const& addr : addresses)
                     ips.push_back (to_string (addr));
-
                 std::string const base ("config: ");
-
                 if (!ips.empty ())
                     m_peerFinder->addFallbackStrings (base + name, ips);
             });
@@ -364,7 +455,7 @@ OverlayImpl::onStart ()
 void
 OverlayImpl::onStop ()
 {
-    strand_.dispatch(std::bind(&OverlayImpl::close, this));
+    strand_.dispatch(std::bind(&OverlayImpl::stop, this));
 }
 
 void
@@ -392,7 +483,7 @@ OverlayImpl::onWrite (beast::PropertyStream::Map& stream)
     are known.
 */
 void
-OverlayImpl::activate (Peer::ptr const& peer)
+OverlayImpl::activate (std::shared_ptr<PeerImp> const& peer)
 {
     std::lock_guard <decltype(mutex_)> lock (mutex_);
 
@@ -400,41 +491,35 @@ OverlayImpl::activate (Peer::ptr const& peer)
     {
         auto const result (m_shortIdMap.emplace (
             std::piecewise_construct,
-                std::make_tuple (peer->getShortId()),
+                std::make_tuple (peer->id()),
                     std::make_tuple (peer)));
         assert(result.second);
         (void) result.second;
     }
 
     {
-        auto const result (m_publicKeyMap.emplace (
+        auto const result (m_publicKeyMap.emplace(
-            std::piecewise_construct,
+            peer->getNodePublic(), peer));
-                std::make_tuple (peer->getNodePublic()),
-                    std::make_tuple (peer)));
         assert(result.second);
         (void) result.second;
     }
 
     journal_.debug <<
         "activated " << peer->getRemoteAddress() <<
-        " (" << peer->getShortId() <<
+        " (" << peer->id() <<
         ":" << RipplePublicKey(peer->getNodePublic()) << ")";
 
     // We just accepted this peer so we have non-zero active peers
     assert(size() != 0);
 }
 
-/** A peer is being disconnected
-    This is called during the disconnection of a known, activated peer. It
-    will not be called for outbound peer connections that don't succeed or
-    for connections of peers that are dropped prior to being activated.
-*/
 void
-OverlayImpl::onPeerDisconnect (Peer::ptr const& peer)
+OverlayImpl::onPeerDeactivate (Peer::id_t id,
+    RippleAddress const& publicKey)
 {
     std::lock_guard <decltype(mutex_)> lock (mutex_);
-    m_shortIdMap.erase (peer->getShortId ());
+    m_shortIdMap.erase(id);
-    m_publicKeyMap.erase (peer->getNodePublic ());
+    m_publicKeyMap.erase(publicKey);
 }
 
 /** The number of active peers on the network
@@ -456,36 +541,45 @@ OverlayImpl::json ()
 }
 
 Overlay::PeerSequence
-OverlayImpl::getActivePeers ()
+OverlayImpl::getActivePeers()
 {
     Overlay::PeerSequence ret;
-
     std::lock_guard <decltype(mutex_)> lock (mutex_);
-
     ret.reserve (m_publicKeyMap.size ());
-
     for (auto const& e : m_publicKeyMap)
     {
-        assert (e.second);
+        auto const sp = e.second.lock();
-        ret.push_back (e.second);
+        if (sp)
+            ret.push_back(sp);
     }
 
     return ret;
 }
 
 Peer::ptr
-OverlayImpl::findPeerByShortID (Peer::ShortId const& id)
+OverlayImpl::findPeerByShortID (Peer::id_t const& id)
 {
     std::lock_guard <decltype(mutex_)> lock (mutex_);
-    PeerByShortId::iterator const iter (
+    auto const iter = m_shortIdMap.find (id);
-        m_shortIdMap.find (id));
     if (iter != m_shortIdMap.end ())
-        return iter->second;
+        return iter->second.lock();
     return Peer::ptr();
 }
 
 //------------------------------------------------------------------------------
 
+void
+OverlayImpl::add (std::shared_ptr<PeerImp> const& peer)
+{
+    {
+        auto const result =
+            m_peers.emplace (peer->slot(), peer);
+        assert (result.second);
+        (void) result.second;
+    }
+    list_.emplace(peer.get(), peer);
+}
+
 void
 OverlayImpl::remove (Child& child)
 {
@@ -495,16 +589,8 @@ OverlayImpl::remove (Child& child)
         checkStopped();
 }
 
-// Caller must hold the mutex
 void
-OverlayImpl::checkStopped ()
+OverlayImpl::stop()
-{
-    if (isStopping() && areChildrenStopped () && list_.empty())
-        stopped();
-}
-
-void
-OverlayImpl::close()
 {
     std::lock_guard<decltype(mutex_)> lock(mutex_);
     if (work_)
@@ -515,28 +601,11 @@ OverlayImpl::close()
             auto const child = _.second.lock();
             // Happens when the child is about to be destroyed
             if (child != nullptr)
-                child->close();
+                child->stop();
         }
     }
 }
 
-void
-OverlayImpl::addpeer (std::shared_ptr<PeerImp> const& peer)
-{
-    std::lock_guard <decltype(mutex_)> lock (mutex_);
-    {
-        std::pair <PeersBySlot::iterator, bool> const result (
-            m_peers.emplace (peer->slot(), peer));
-        assert (result.second);
-        (void) result.second;
-    }
-    list_.emplace(peer.get(), peer);
-
-    // This has to happen while holding the lock,
-    // otherwise the socket might not be canceled during a stop.
-    peer->start();
-}
-
 void
 OverlayImpl::autoConnect()
 {
@@ -551,16 +620,15 @@ OverlayImpl::sendEndpoints()
     auto const result = m_peerFinder->buildEndpointsForPeers();
     for (auto const& e : result)
     {
-        // VFALCO TODO Make sure this doesn't race with closing the peer
+        std::shared_ptr<PeerImp> peer;
-        PeerImp::ptr peer;
         {
             std::lock_guard <decltype(mutex_)> lock (mutex_);
-            PeersBySlot::iterator const iter = m_peers.find (e.first);
+            auto const iter = m_peers.find (e.first);
             if (iter != m_peers.end())
                 peer = iter->second.lock();
         }
         if (peer)
-            peer->send_endpoints (e.second.begin(), e.second.end());
+            peer->sendEndpoints (e.second.begin(), e.second.end());
     }
 }
 

src/ripple/overlay/impl/OverlayImpl.h

@@ -59,22 +59,17 @@ public:
         virtual ~Child();
 
     public:
-        virtual void close() = 0;
+        virtual void stop() = 0;
     };
 
 private:
     using clock_type = std::chrono::steady_clock;
     using socket_type = boost::asio::ip::tcp::socket;
+    using address_type = boost::asio::ip::address;
+    using endpoint_type = boost::asio::ip::tcp::endpoint;
     using error_code = boost::system::error_code;
     using yield_context = boost::asio::yield_context;
 
-    using PeersBySlot = hash_map <PeerFinder::Slot::ptr,
-        std::weak_ptr <PeerImp>>;
-
-    using PeerByPublicKey = hash_map<RippleAddress, Peer::ptr>;
-
-    using PeerByShortId = hash_map<Peer::ShortId, Peer::ptr>;
-
     struct Timer
         : Child
         , std::enable_shared_from_this<Timer>
@@ -85,7 +80,7 @@ private:
         Timer (OverlayImpl& overlay);
 
         void
-        close() override;
+        stop() override;
 
         void
         run();
@@ -112,20 +107,16 @@ private:
 
     std::unique_ptr <PeerFinder::Manager> m_peerFinder;
 
-    /** Associates slots to peers. */
+    hash_map <PeerFinder::Slot::ptr,
-    PeersBySlot m_peers;
+        std::weak_ptr <PeerImp>> m_peers;
 
-    /** Tracks peers by their public key */
+    hash_map<RippleAddress, std::weak_ptr<PeerImp>> m_publicKeyMap;
-    PeerByPublicKey m_publicKeyMap;
 
-    /** Tracks peers by their session ID */
+    hash_map<Peer::id_t, std::weak_ptr<PeerImp>> m_shortIdMap;
-    PeerByShortId m_shortIdMap;
 
-    /** The resolver we use for peer hostnames */
     Resolver& m_resolver;
 
-    /** Monotically increasing identifiers for peers */
+    std::atomic <Peer::id_t> next_id_;
-    std::atomic <Peer::ShortId> m_nextShortId;
 
     //--------------------------------------------------------------------------
 
@@ -135,7 +126,10 @@ public:
             beast::File const& pathToDbFileOrDirectory,
                 Resolver& resolver, boost::asio::io_service& io_service);
 
-    ~OverlayImpl ();
+    ~OverlayImpl();
+
+    OverlayImpl (OverlayImpl const&) = delete;
+    OverlayImpl& operator= (OverlayImpl const&) = delete;
 
     Setup const&
     setup() const
@@ -152,21 +146,18 @@ public:
     void
     onLegacyPeerHello (std::unique_ptr<beast::asio::ssl_bundle>&& ssl_bundle,
         boost::asio::const_buffer buffer,
-            boost::asio::ip::tcp::endpoint remote_address) override;
+            endpoint_type remote_endpoint) override;
 
     Handoff
     onHandoff (std::unique_ptr <beast::asio::ssl_bundle>&& bundle,
         beast::http::message&& request,
-            boost::asio::ip::tcp::endpoint remote_address) override;
+            endpoint_type remote_endpoint) override;
 
     PeerSequence
     getActivePeers() override;
 
     Peer::ptr
-    findPeerByShortID (Peer::ShortId const& id) override;
+    findPeerByShortID (Peer::id_t const& id) override;
-
-    Peer::ShortId
-    next_id();
 
     void
     remove (PeerFinder::Slot::ptr const& slot);
@@ -177,26 +168,20 @@ public:
         are known.
     */
     void
-    activate (Peer::ptr const& peer);
+    activate (std::shared_ptr<PeerImp> const& peer);
 
-    /** A peer is being disconnected
+    /** Called when an active peer is destroyed. */
-        This is called during the disconnection of a known, activated peer. It
-        will not be called for outbound peer connections that don't succeed or
-        for connections of peers that are dropped prior to being activated.
-    */
     void
-    onPeerDisconnect (Peer::ptr const& peer);
+    onPeerDeactivate (Peer::id_t id, RippleAddress const& publicKey);
-
-private:
-    OverlayImpl (OverlayImpl const&) = delete;
-    OverlayImpl& operator= (OverlayImpl const&) = delete;
 
+    static
     bool
     isPeerUpgrade (beast::http::message const& request);
 
+private:
     std::shared_ptr<HTTP::Writer>
     makeRedirectResponse (PeerFinder::Slot::ptr const& slot,
-        beast::http::message const& request);
+        beast::http::message const& request, address_type remote_address);
 
     void
     connect (beast::IP::Endpoint const& remote_endpoint) override;
@@ -213,6 +198,9 @@ private:
     // Stoppable
     //
 
+    void
+    checkStopped();
+
     void
     onPrepare() override;
 
@@ -234,17 +222,14 @@ private:
 
     //--------------------------------------------------------------------------
 
+    void
+    add (std::shared_ptr<PeerImp> const& peer);
+
     void
     remove (Child& child);
 
     void
-    checkStopped ();
+    stop();
-
-    void
-    close();
-
-    void
-    addpeer (std::shared_ptr<PeerImp> const& peer);
 
     void
     autoConnect();

src/ripple/overlay/impl/PeerImp.h

@@ -25,7 +25,6 @@
 #include <ripple/overlay/impl/message_name.h>
 #include <ripple/overlay/impl/message_stream.h>
 #include <ripple/overlay/impl/OverlayImpl.h>
-#include <ripple/overlay/impl/peer_protocol_detector.h>
 #include <ripple/app/misc/ProofOfWork.h>
 #include <ripple/app/misc/ProofOfWorkFactory.h>
 #include <ripple/core/Config.h>
@@ -41,6 +40,7 @@
 #include <beast/http/parser.h>
 #include <beast/utility/WrappedSink.h>
 #include <cstdint>
+#include <queue>
 
 namespace ripple {
 
@@ -54,117 +54,62 @@ class PeerImp
     , private abstract_protocol_handler
 {
 public:
+    /** Type of connection.
+        The affects how messages are routed.
+    */
+    enum class Type
+    {
+        legacy,
+        leaf,
+        peer
+    };
+
     /** Current state */
-    enum State
+    enum class State
     {
         /** A connection is being established (outbound) */
-         stateConnecting
+        connecting
 
         /** Connection has been successfully established */
-        ,stateConnected
+        ,connected
 
         /** Handshake has been received from this peer */
-        ,stateHandshaked
+        ,handshaked
 
         /** Running the Ripple protocol actively */
-        ,stateActive
+        ,active
-
-        /** Gracefully closing */
-        ,stateGracefulClose
     };
 
     typedef std::shared_ptr <PeerImp> ptr;
 
 private:
-    /** Wraps a Journal::Sink to prefix it's output. */
-    class WrappedSink : public beast::Journal::Sink
-    {
-    private:
-        std::string prefix_;
-        beast::Journal::Sink& sink_;
-
-    public:
-        explicit
-        WrappedSink (beast::Journal::Sink& sink)
-            : sink_ (sink)
-        {
-        }
-
-        explicit
-        WrappedSink (beast::Journal const& journal)
-            : sink_ (journal.sink())
-        {
-        }
-
-        void prefix (std::string const& s)
-        {
-            prefix_ = s;
-        }
-
-        bool
-        active (beast::Journal::Severity level) const override
-        {
-            return sink_.active (level);
-        }
-
-        bool
-        console () const override
-        {
-            return sink_.console ();
-        }
-
-        void console (bool output) override
-        {
-            sink_.console (output);
-        }
-
-        beast::Journal::Severity
-        severity() const
-        {
-            return sink_.severity();
-        }
-
-        void severity (beast::Journal::Severity level)
-        {
-            sink_.severity (level);
-        }
-
-        void write (beast::Journal::Severity level, std::string const& text)
-        {
-            using beast::Journal;
-            sink_.write (level, prefix_ + text);
-        }
-    };
-
     using clock_type = std::chrono::steady_clock;
     using error_code= boost::system::error_code ;
     using yield_context = boost::asio::yield_context;
     using socket_type = boost::asio::ip::tcp::socket;
     using stream_type = boost::asio::ssl::stream <socket_type&>;
+    using address_type = boost::asio::ip::address;
+    using endpoint_type = boost::asio::ip::tcp::endpoint;
 
     // Time alloted for a peer to send a HELLO message (DEPRECATED)
     static const boost::posix_time::seconds nodeVerifySeconds;
 
-    // The clock drift we allow a remote peer to have
-    static const std::uint32_t clockToleranceDeltaSeconds = 20;
-
     // The length of the smallest valid finished message
     static const size_t sslMinimumFinishedLength = 12;
 
-    WrappedSink sink_;
+    id_t const id_;
-    WrappedSink p_sink_;
+    beast::WrappedSink sink_;
+    beast::WrappedSink p_sink_;
     beast::Journal journal_;
     beast::Journal p_journal_;
     std::unique_ptr<beast::asio::ssl_bundle> ssl_bundle_;
     socket_type& socket_;
     stream_type& stream_;
     boost::asio::io_service::strand strand_;
-    boost::asio::deadline_timer timer_;
+    boost::asio::basic_waitable_timer<
+        std::chrono::steady_clock> timer_;
 
-    // A unique identifier (up to a restart of rippled) for this particular
+    Type type_ = Type::legacy;
-    // peer instance. A peer that disconnects will, upon reconnection, get a
-    // new ID.
-    ShortId shortId_ = 0;
 
     // Updated at each stage of the connection process to reflect
     // the current conditions as closely as possible.
@@ -188,10 +133,7 @@ private:
 
     std::string name_;
 
-    // Both sides of the peer calculate this value and verify that it matches
+    uint256 sharedValue_;
-    // to detect/prevent man-in-the-middle attacks.
-    //
-    uint256 secureCookie_;
 
     // The indices of the smallest and largest ledgers this peer has available
     //
@@ -205,23 +147,20 @@ private:
     std::list<uint256> recentTxSets_;
     mutable std::mutex recentLock_;
 
-    std::list <Message::pointer> send_queue_;
-    Message::pointer send_packet_;
     protocol::TMStatusChange last_status_;
     protocol::TMHello hello_;
 
     Resource::Consumer usage_;
-
-    // The slot assigned to us by PeerFinder
     PeerFinder::Slot::ptr slot_;
 
     beast::asio::streambuf read_buffer_;
-    boost::optional <beast::http::message> http_message_;
+    beast::http::message http_message_;
     boost::optional <beast::http::parser> http_parser_;
     beast::http::body http_body_;
     message_stream message_stream_;
-
     beast::asio::streambuf write_buffer_;
+    std::queue<Message::pointer> send_queue_;
+    bool gracefulClose_ = false;
 
     std::unique_ptr <LoadEvent> load_event_;
 
@@ -234,22 +173,24 @@ public:
     /** Create an incoming legacy peer from an established ssl connection. */
     template <class ConstBufferSequence>
     PeerImp (std::unique_ptr<beast::asio::ssl_bundle>&& ssl_bundle,
-        ConstBufferSequence const& buffer, beast::IP::Endpoint remoteAddress,
+        ConstBufferSequence const& buffer, endpoint_type remote_endpoint,
             OverlayImpl& overlay, Resource::Manager& resourceManager,
                 PeerFinder::Manager& peerFinder,
-                    PeerFinder::Slot::ptr const& slot);
+                    PeerFinder::Slot::ptr const& slot, id_t id);
 
     /** Create an active incoming peer from an established ssl connection. */
     PeerImp (std::unique_ptr<beast::asio::ssl_bundle>&& ssl_bundle,
-        beast::http::message&& request, beast::IP::Endpoint remoteAddress,
+        beast::http::message&& request, protocol::TMHello const& hello,
-            OverlayImpl& overlay, Resource::Manager& resourceManager,
+            endpoint_type remote_endpoint, RippleAddress const& publicKey,
-                PeerFinder::Manager& peerFinder, PeerFinder::Slot::ptr const& slot);
+                Resource::Consumer consumer, PeerFinder::Slot::ptr const& slot,
+                    OverlayImpl& overlay, Resource::Manager& resourceManager,
+                        PeerFinder::Manager& peerFinder, id_t id);
 
     /** Create an outgoing peer. */
     PeerImp (beast::IP::Endpoint remoteAddress, boost::asio::io_service& io_service,
         OverlayImpl& overlay, Resource::Manager& resourceManager,
             PeerFinder::Manager& peerFinder, PeerFinder::Slot::ptr const& slot,
-                std::shared_ptr<boost::asio::ssl::context> const& context);
+                std::shared_ptr<boost::asio::ssl::context> const& context, id_t id);
 
     virtual
     ~PeerImp ();
@@ -262,14 +203,11 @@ public:
 
     // Work-around for calling shared_from_this in constructors
     void
-    start();
+    run();
 
+    // Called when Overlay gets a stop request.
     void
-    getLedger (protocol::TMGetLedger& packet);
+    stop() override;
-
-    // Cancel all I/O and close the socket
-    void
-    close() override;
 
     //
     // Network
@@ -283,10 +221,13 @@ public:
         typename std::iterator_traits<FwdIt>::value_type,
             PeerFinder::Endpoint>::value>>
     void
-    send_endpoints (FwdIt first, FwdIt last);
+    sendEndpoints (FwdIt first, FwdIt last);
 
     beast::IP::Endpoint
-    getRemoteAddress() const override;
+    getRemoteAddress() const override
+    {
+        return remote_address_;
+    }
 
     void
     charge (Resource::Charge const& fee) override;
@@ -295,27 +236,42 @@ public:
     // Identity
     //
 
-    Peer::ShortId
+    Peer::id_t
-    getShortId () const override;
+    id() const override
+    {
+        return id_;
+    }
 
     RippleAddress const&
-    getNodePublic () const override;
+    getNodePublic () const override
+    {
+        return publicKey_;
+    }
 
     Json::Value
     json() override;
 
     bool
-    isInCluster () const override;
+    isInCluster () const override
+    {
+        return clusterNode_;
+    }
 
     std::string const&
-    getClusterNodeName() const override;
+    getClusterNodeName() const override
+    {
+        return name_;
+    }
 
     //
     // Ledger
     //
 
     uint256 const&
-    getClosedLedgerHash () const override;
+    getClosedLedgerHash () const override
+    {
+        return closedLedgerHash_;
+    }
 
     bool
     hasLedger (uint256 const& hash, std::uint32_t seq) const override;
@@ -337,77 +293,95 @@ public:
 
 private:
     void
-    setPrefix();
+    close();
-
-    //
-    // client role
-    //
 
     void
-    do_connect();
+    fail(std::string const& reason);
 
     void
-    on_connect (error_code ec);
+    fail(std::string const& name, error_code ec);
 
+    void
+    gracefulClose();
+
+    void
+    setTimer();
+
+    void
+    cancelTimer();
+
+    static
+    std::string
+    makePrefix(id_t id);
+
+    static
     beast::http::message
-    make_request();
+    makeRequest (boost::asio::ip::address const& remote_address);
+
+    // Called when the timer wait completes
+    void
+    onTimer (boost::system::error_code const& ec);
+
+    // Called when SSL shutdown completes
+    void
+    onShutdown (error_code ec);
+
+    //
+    // outbound completion path
+    //
 
     void
-    on_connect_ssl (error_code ec);
+    doConnect();
 
     void
-    on_write_http_request (error_code ec, std::size_t bytes_transferred);
+    onConnect (error_code ec);
+
+    void
+    onHandshake (error_code ec);
+
+    void
+    onWriteRequest (error_code ec, std::size_t bytes_transferred);
+
+    void
+    onReadResponse (error_code ec, std::size_t bytes_transferred);
 
     template <class Streambuf>
     void
     processResponse (beast::http::message const& m, Streambuf const& body);
 
-    void
-    on_read_http_response (error_code ec, std::size_t bytes_transferred);
-
     //
-    // server role
+    // inbound completion path
     //
 
     void
-    do_accept();
+    doAccept();
 
     void
-    on_accept_ssl (error_code ec);
+    doLegacyAccept();
-
-    void
-    on_read_http_detect (error_code ec, std::size_t bytes_transferred);
-
-    void
-    on_read_http_request (error_code ec, std::size_t bytes_transferred);
 
+    static
     beast::http::message
-    make_response (beast::http::message const& req);
+    makeResponse (beast::http::message const& req,
+        uint256 const& sharedValue);
 
     void
-    on_write_http_response (error_code ec, std::size_t bytes_transferred);
+    onWriteResponse (error_code ec, std::size_t bytes_transferred);
 
     //
-    // protocol
+    // protocol message loop
     //
 
+    // Starts the protocol message loop
     void
-    do_protocol_start();
+    doProtocolStart (bool legacy);
 
+    // Called when protocol message bytes are received
     void
-    on_read_protocol (error_code ec, std::size_t bytes_transferred);
+    onReadMessage (error_code ec, std::size_t bytes_transferred);
 
+    // Called when protocol messages bytes are sent
     void
-    on_write_protocol (error_code ec, std::size_t bytes_transferred);
+    onWriteMessage (error_code ec, std::size_t bytes_transferred);
-
-    void
-    handleShutdown (boost::system::error_code const& ec);
-
-    void
-    handleWrite (boost::system::error_code const& ec, size_t bytes);
-
-    void
-    handleVerifyTimer (boost::system::error_code const& ec);
 
     //--------------------------------------------------------------------------
     //
@@ -466,48 +440,8 @@ private:
 
     //--------------------------------------------------------------------------
 
-    // DEPRECATED Close the socket
     void
-    detach (const char* rsn);
+    sendGetPeers();
-
-    void
-    sendGetPeers ();
-
-    static
-    void
-    charge (std::weak_ptr <PeerImp>& peer, Resource::Charge const& fee);
-
-    void
-    sendForce (const Message::pointer& packet);
-
-    /** Hashes the latest finished message from an SSL stream
-        @param sslSession the session to get the message from.
-        @param hash       the buffer into which the hash of the retrieved
-                          message will be saved. The buffer MUST be at least
-                          64 bytes long.
-        @param getMessage a pointer to the function to call to retrieve the
-                          finished message. This be either:
-                          `SSL_get_finished` or
-                          `SSL_get_peer_finished`.
-        @return `true` if successful, `false` otherwise.
-    */
-    bool
-    hashLatestFinishedMessage (const SSL *sslSession, unsigned char *hash,
-        size_t (*getFinishedMessage)(const SSL *, void *buf, size_t));
-
-    /** Generates a secure cookie to protect against man-in-the-middle attacks
-        This function should never fail under normal circumstances and regular
-        server operation.
-        A failure prevents the cookie value from being calculated which is an
-        important component of connection security. If this function fails, a
-        secure connection cannot be established and the link MUST be dropped.
-        @return `true` if the cookie was generated, `false` otherwise.
-        @note failure is an exceptional situation - it should never happen and
-              will almost always indicate an active man-in-the-middle attack is
-              taking place.
-    */
-    bool
-    calculateSessionCookie ();
 
     /** Perform a secure handshake with the peer at the other end.
         If this function returns false then we cannot guarantee that there
@@ -530,36 +464,25 @@ private:
     void
     doProofOfWork (Job&, std::weak_ptr <PeerImp> peer, ProofOfWork::pointer pow);
 
-    static
-    void checkTransaction (Job&, int flags, STTx::pointer stx,
-        std::weak_ptr<PeerImp> peer);
-
-    // Called from our JobQueue
-    static
     void
-    checkPropose (Job& job, Overlay* pPeers,
+    checkTransaction (Job&, int flags, STTx::pointer stx);
-        std::shared_ptr<protocol::TMProposeSet> packet,
-            LedgerProposal::pointer proposal, uint256 consensusLCL,
-                RippleAddress nodePublic, std::weak_ptr<PeerImp> peer,
-                    bool fromCluster, beast::Journal journal);
 
-    static
     void
-    checkValidation (Job&, Overlay* pPeers, STValidation::pointer val,
+    checkPropose (Job& job,
-        bool isTrusted, bool isCluster,
+        std::shared_ptr<protocol::TMProposeSet> const& packet,
-            std::shared_ptr<protocol::TMValidation> packet,
+            LedgerProposal::pointer proposal, uint256 consensusLCL);
-                std::weak_ptr<PeerImp> peer, beast::Journal journal);
 
-    static
     void
-    sGetLedger (std::weak_ptr<PeerImp> wPeer,
+    checkValidation (Job&, STValidation::pointer val,
-        std::shared_ptr <protocol::TMGetLedger> packet);
+        bool isTrusted, std::shared_ptr<protocol::TMValidation> const& packet);
 
-    /** Called when we receive tx set data. */
-    static
     void
-    peerTXData (Job&, std::weak_ptr <PeerImp> wPeer, uint256 const& hash,
+    getLedger (std::shared_ptr<protocol::TMGetLedger> const&packet);
-        std::shared_ptr <protocol::TMLedgerData> pPacket,
+
+    // Called when we receive tx set data.
+    void
+    peerTXData (Job&, uint256 const& hash,
+        std::shared_ptr <protocol::TMLedgerData> const& pPacket,
             beast::Journal journal);
 };
 
@@ -567,13 +490,14 @@ private:
 
 template <class ConstBufferSequence>
 PeerImp::PeerImp (std::unique_ptr<beast::asio::ssl_bundle>&& ssl_bundle,
-    ConstBufferSequence const& buffer, beast::IP::Endpoint remoteAddress,
+    ConstBufferSequence const& buffer, endpoint_type remote_endpoint,
         OverlayImpl& overlay, Resource::Manager& resourceManager,
             PeerFinder::Manager& peerFinder,
-                PeerFinder::Slot::ptr const& slot)
+                PeerFinder::Slot::ptr const& slot, id_t id)
     : Child (overlay)
-    , sink_(deprecatedLogs().journal("Peer"))
+    , id_(id)
-    , p_sink_(deprecatedLogs().journal("Protocol"))
+    , sink_(deprecatedLogs().journal("Peer"), makePrefix(id))
+    , p_sink_(deprecatedLogs().journal("Protocol"), makePrefix(id))
     , journal_ (sink_)
     , p_journal_(p_sink_)
     , ssl_bundle_(std::move(ssl_bundle))
@@ -581,23 +505,23 @@ PeerImp::PeerImp (std::unique_ptr<beast::asio::ssl_bundle>&& ssl_bundle,
     , stream_ (ssl_bundle_->stream)
     , strand_ (socket_.get_io_service())
     , timer_ (socket_.get_io_service())
-    , remote_address_ (remoteAddress)
+    , remote_address_ (
+        beast::IPAddressConversion::from_asio(remote_endpoint))
     , resourceManager_ (resourceManager)
     , peerFinder_ (peerFinder)
     , overlay_ (overlay)
     , m_inbound (true)
-    , state_ (stateConnected)
+    , state_ (State::connected)
     , slot_ (slot)
     , message_stream_(*this)
 {
-    setPrefix();
     read_buffer_.commit(boost::asio::buffer_copy(read_buffer_.prepare(
         boost::asio::buffer_size(buffer)), buffer));
 }
 
 template <class FwdIt, class>
 void
-PeerImp::send_endpoints (FwdIt first, FwdIt last)
+PeerImp::sendEndpoints (FwdIt first, FwdIt last)
 {
     protocol::TMEndpoints tm;
     for (;first != last; ++first)

src/ripple/overlay/impl/TMHello.cpp

@@ -0,0 +1,339 @@
+//------------------------------------------------------------------------------
+/*
+    This file is part of rippled: https://github.com/ripple/rippled
+    Copyright (c) 2012, 2013 Ripple Labs Inc.
+
+    Permission to use, copy, modify, and/or distribute this software for any
+    purpose  with  or without fee is hereby granted, provided that the above
+    copyright notice and this permission notice appear in all copies.
+
+    THE  SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+    WITH  REGARD  TO  THIS  SOFTWARE  INCLUDING  ALL  IMPLIED  WARRANTIES  OF
+    MERCHANTABILITY  AND  FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+    ANY  SPECIAL ,  DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+    WHATSOEVER  RESULTING  FROM  LOSS  OF USE, DATA OR PROFITS, WHETHER IN AN
+    ACTION  OF  CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+    OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+*/
+//==============================================================================
+
+#include <ripple/app/main/Application.h>
+#include <ripple/protocol/BuildInfo.h>
+#include <ripple/overlay/impl/TMHello.h>
+#include <beast/crypto/base64.h>
+#include <beast/http/rfc2616.h>
+#include <beast/module/core/text/LexicalCast.h>
+#include <boost/regex.hpp>
+#include <algorithm>
+
+// VFALCO Shouldn't we have to include the OpenSSL
+// headers or something for SSL_get_finished?
+
+namespace ripple {
+
+/** Hashes the latest finished message from an SSL stream
+    @param sslSession the session to get the message from.
+    @param hash       the buffer into which the hash of the retrieved
+                        message will be saved. The buffer MUST be at least
+                        64 bytes long.
+    @param getMessage a pointer to the function to call to retrieve the
+                        finished message. This be either:
+                        `SSL_get_finished` or
+                        `SSL_get_peer_finished`.
+    @return `true` if successful, `false` otherwise.
+*/
+static
+bool
+hashLastMessage (SSL const* ssl, unsigned char* hash,
+    size_t (*get)(const SSL *, void *buf, size_t))
+{
+    enum
+    {
+        sslMinimumFinishedLength = 12
+    };
+    unsigned char buf[1024];
+    std::memset(hash, 0, 64);
+    size_t len = get (ssl, buf, sizeof (buf));
+    if(len < sslMinimumFinishedLength)
+        return false;
+    SHA512 (buf, len, hash);
+    return true;
+}
+
+std::pair<uint256, bool>
+makeSharedValue (SSL* ssl, beast::Journal journal)
+{
+    std::pair<uint256, bool> result = { {}, false };
+
+    unsigned char sha1[64];
+    unsigned char sha2[64];
+
+    if (!hashLastMessage(ssl, sha1, SSL_get_finished))
+    {
+        journal.error << "Cookie generation: local setup not complete";
+        return result;
+    }
+
+    if (!hashLastMessage(ssl, sha2, SSL_get_peer_finished))
+    {
+        journal.error << "Cookie generation: peer setup not complete";
+        return result;
+    }
+
+    // If both messages hash to the same value (i.e. match) something is
+    // wrong. This would cause the resulting cookie to be 0.
+    if (memcmp (sha1, sha2, sizeof (sha1)) == 0)
+    {
+        journal.error << "Cookie generation: identical finished messages";
+        return result;
+    }
+
+    for (size_t i = 0; i < sizeof (sha1); ++i)
+        sha1[i] ^= sha2[i];
+
+    // Finally, derive the actual cookie for the values that
+    // we have calculated.
+    result.first = Serializer::getSHA512Half (sha1, sizeof(sha1));
+    result.second = true;
+    return result;
+}
+
+protocol::TMHello
+buildHello (uint256 const& sharedValue, Application& app)
+{
+    protocol::TMHello h;
+
+    Blob vchSig;
+    app.getLocalCredentials ().getNodePrivate ().signNodePrivate (
+        sharedValue, vchSig);
+
+    h.set_protoversion (to_packed (BuildInfo::getCurrentProtocol()));
+    h.set_protoversionmin (to_packed (BuildInfo::getMinimumProtocol()));
+    h.set_fullversion (BuildInfo::getFullVersionString ());
+    h.set_nettime (app.getOPs ().getNetworkTimeNC ());
+    h.set_nodepublic (app.getLocalCredentials ().getNodePublic (
+        ).humanNodePublic ());
+    h.set_nodeproof (&vchSig[0], vchSig.size ());
+    // h.set_ipv4port (portNumber); // ignored now
+    h.set_testnet (false);
+
+    // We always advertise ourselves as private in the HELLO message. This
+    // suppresses the old peer advertising code and allows PeerFinder to
+    // take over the functionality.
+    h.set_nodeprivate (true);
+
+    auto const closedLedger = app.getLedgerMaster().getClosedLedger();
+
+    if (closedLedger && closedLedger->isClosed ())
+    {
+        uint256 hash = closedLedger->getHash ();
+        h.set_ledgerclosed (hash.begin (), hash.size ());
+        hash = closedLedger->getParentHash ();
+        h.set_ledgerprevious (hash.begin (), hash.size ());
+    }
+
+    return h;
+}
+
+void
+appendHello (beast::http::message& m,
+    protocol::TMHello const& hello)
+{
+    auto& h = m.headers;
+
+    //h.append ("Protocol-Versions",...
+
+    h.append ("Public-Key", hello.nodepublic());
+
+    h.append ("Session-Signature", beast::base64_encode (
+        hello.nodeproof()));
+
+    if (hello.has_nettime())
+        h.append ("Network-Time", std::to_string (hello.nettime()));
+
+    if (hello.has_ledgerindex())
+        h.append ("Ledger", std::to_string (hello.ledgerindex()));
+
+    if (hello.has_ledgerclosed())
+        h.append ("Closed-Ledger", beast::base64_encode (
+            hello.ledgerclosed()));
+
+    if (hello.has_ledgerprevious())
+        h.append ("Previous-Ledger", beast::base64_encode (
+            hello.ledgerprevious()));
+}
+
+std::vector<ProtocolVersion>
+parse_ProtocolVersions (std::string const& s)
+{
+    static boost::regex const re (
+        "^"                  // start of line
+        "RTXP/"              // the string "RTXP/"
+        "([1-9][0-9]*)"      // a number (non-zero and with no leading zeroes)
+        "\\."                // a period
+        "(0|[1-9][0-9]*)"    // a number (no leading zeroes unless exactly zero)
+        "$"                  // The end of the string
+        , boost::regex_constants::optimize);
+
+    auto const list = beast::rfc2616::split_commas (s);
+    std::vector<ProtocolVersion> result;
+    for (auto const& s : list)
+    {
+        boost::smatch m;
+        if (! boost::regex_match (s, m, re))
+            continue;
+        int major;
+        int minor;
+        if (! beast::lexicalCastChecked (
+                major, std::string (m[1])))
+            continue;
+        if (! beast::lexicalCastChecked (
+                minor, std::string (m[2])))
+            continue;
+        result.push_back (std::make_pair (major, minor));
+    }
+    std::sort(result.begin(), result.end());
+    result.erase(std::unique (result.begin(), result.end()), result.end());
+    return result;
+}
+
+std::pair<protocol::TMHello, bool>
+parseHello (beast::http::message const& m, beast::Journal journal)
+{
+    auto const& h = m.headers;
+    std::pair<protocol::TMHello, bool> result = { {}, false };
+    protocol::TMHello& hello = result.first;
+
+    // protocol version in TMHello is obsolete,
+    // it is supplanted by the values in the headers.
+
+    {
+        // Required
+        auto const iter = h.find ("Public-Key");
+        if (iter == h.end())
+            return result;
+        RippleAddress addr;
+        addr.setNodePublic (iter->second);
+        if (! addr.isValid())
+            return result;
+        hello.set_nodepublic (iter->second);
+    }
+
+    {
+        // Required
+        auto const iter = h.find ("Session-Signature");
+        if (iter == h.end())
+            return result;
+        // TODO Security Review
+        hello.set_nodeproof (beast::base64_decode (iter->second));
+    }
+
+    {
+        auto const iter = h.find (m.request() ?
+            "User-Agent" : "Server");
+        if (iter != h.end())
+            hello.set_fullversion (iter->second);
+    }
+
+    {
+        auto const iter = h.find ("Network-Time");
+        if (iter != h.end())
+        {
+            std::uint64_t nettime;
+            if (! beast::lexicalCastChecked(nettime, iter->second))
+                return result;
+            hello.set_nettime (nettime);
+        }
+    }
+
+    {
+        auto const iter = h.find ("Ledger");
+        if (iter != h.end())
+        {
+            LedgerIndex ledgerIndex;
+            if (! beast::lexicalCastChecked(ledgerIndex, iter->second))
+                return result;
+            hello.set_ledgerindex (ledgerIndex);
+        }
+    }
+
+    {
+        auto const iter = h.find ("Closed-Ledger");
+        if (iter != h.end())
+            hello.set_ledgerclosed (beast::base64_decode (iter->second));
+    }
+
+    {
+        auto const iter = h.find ("Previous-Ledger");
+        if (iter != h.end())
+            hello.set_ledgerprevious (beast::base64_decode (iter->second));
+    }
+
+    result.second = true;
+    return result;
+}
+
+std::pair<RippleAddress, bool>
+verifyHello (protocol::TMHello const& h, uint256 const& sharedValue,
+    beast::Journal journal, Application& app)
+{
+    std::pair<RippleAddress, bool> result = { {}, false };
+    std::uint32_t const ourTime = app.getOPs().getNetworkTimeNC();
+    std::uint32_t const minTime = ourTime - clockToleranceDeltaSeconds;
+    std::uint32_t const maxTime = ourTime + clockToleranceDeltaSeconds;
+
+#ifdef BEAST_DEBUG
+    if (h.has_nettime ())
+    {
+        std::int64_t to = ourTime;
+        to -= h.nettime ();
+        journal.debug <<
+            "Connect: time offset " << to;
+    }
+#endif
+
+    auto const protocol = BuildInfo::make_protocol(h.protoversion());
+
+    if (h.has_nettime () &&
+        ((h.nettime () < minTime) || (h.nettime () > maxTime)))
+    {
+        if (h.nettime () > maxTime)
+        {
+            journal.info <<
+                "Clock for is off by +" << h.nettime() - ourTime;
+        }
+        else if (h.nettime () < minTime)
+        {
+            journal.info <<
+                "Clock is off by -" << ourTime - h.nettime();
+        }
+    }
+    else if (h.protoversionmin () > to_packed (
+        BuildInfo::getCurrentProtocol()))
+    {
+        journal.info <<
+            "Hello: Disconnect: Protocol mismatch [" <<
+            "Peer expects " << to_string (protocol) <<
+            " and we run " << to_string (BuildInfo::getCurrentProtocol()) << "]";
+    }
+    else if (! result.first.setNodePublic (h.nodepublic()))
+    {
+        journal.info <<
+            "Hello: Disconnect: Bad node public key.";
+    }
+    else if (! result.first.verifyNodePublic (
+        sharedValue, h.nodeproof (), ECDSA::not_strict))
+    {
+        // Unable to verify they have private key for claimed public key.
+        journal.info <<
+            "Hello: Disconnect: Failed to verify session.";
+    }
+    else
+    {
+        // Successful connection.
+        result.second = true;
+    }
+    return result;
+}
+
+}

src/ripple/overlay/impl/TMHello.h

@@ -0,0 +1,80 @@
+//------------------------------------------------------------------------------
+/*
+    This file is part of rippled: https://github.com/ripple/rippled
+    Copyright (c) 2012, 2013 Ripple Labs Inc.
+
+    Permission to use, copy, modify, and/or distribute this software for any
+    purpose  with  or without fee is hereby granted, provided that the above
+    copyright notice and this permission notice appear in all copies.
+
+    THE  SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+    WITH  REGARD  TO  THIS  SOFTWARE  INCLUDING  ALL  IMPLIED  WARRANTIES  OF
+    MERCHANTABILITY  AND  FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+    ANY  SPECIAL ,  DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+    WHATSOEVER  RESULTING  FROM  LOSS  OF USE, DATA OR PROFITS, WHETHER IN AN
+    ACTION  OF  CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+    OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+*/
+//==============================================================================
+
+#ifndef RIPPLE_OVERLAY_TMHELLO_H_INCLUDED
+#define RIPPLE_OVERLAY_TMHELLO_H_INCLUDED
+
+#include "ripple.pb.h"
+#include <ripple/protocol/BuildInfo.h>
+#include <ripple/protocol/UintTypes.h>
+#include <beast/http/message.h>
+#include <beast/utility/Journal.h>
+#include <utility>
+
+namespace ripple {
+
+enum
+{
+    // The clock drift we allow a remote peer to have
+    clockToleranceDeltaSeconds = 20
+};
+
+/** Computes a shared value based on the SSL connection state.
+    When there is no man in the middle, both sides will compute the same
+    value. In the presence of an attacker, the computed values will be
+    different.
+    If the shared value generation fails, the link MUST be dropped.
+    @return A pair. Second will be false if shared value generation failed.
+*/
+std::pair<uint256, bool>
+makeSharedValue (SSL* ssl, beast::Journal journal);
+
+/** Build a TMHello protocol message. */
+protocol::TMHello
+buildHello (uint256 const& sharedValue, Application& app);
+
+/** Insert HTTP headers based on the TMHello protocol message. */
+void
+appendHello (beast::http::message& m, protocol::TMHello const& hello);
+
+/** Parse HTTP headers into TMHello protocol message.
+    @return A pair. Second will be false if the parsing failed.
+*/
+std::pair<protocol::TMHello, bool>
+parseHello (beast::http::message const& m, beast::Journal journal);
+
+/** Validate and store the public key in the TMHello.
+    This includes signature verification on the shared value.
+    @return A pair. Second will be false if the check failed.
+*/
+std::pair<RippleAddress, bool>
+verifyHello (protocol::TMHello const& h, uint256 const& sharedValue,
+    beast::Journal journal, Application& app);
+
+/** Parse a set of protocol versions.
+    The returned list contains no duplicates and is sorted ascending.
+    Any strings that are not parseable as RTXP protocol strings are
+    excluded from the result set.
+*/
+std::vector<ProtocolVersion>
+parse_ProtocolVersions (std::string const& s);
+
+}
+
+#endif

src/ripple/overlay/impl/peer_protocol_detector.h

@@ -1,73 +0,0 @@
-//------------------------------------------------------------------------------
-/*
-    This file is part of rippled: https://github.com/ripple/rippled
-    Copyright (c) 2012, 2013 Ripple Labs Inc.
-
-    Permission to use, copy, modify, and/or distribute this software for any
-    purpose  with  or without fee is hereby granted, provided that the above
-    copyright notice and this permission notice appear in all copies.
-
-    THE  SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
-    WITH  REGARD  TO  THIS  SOFTWARE  INCLUDING  ALL  IMPLIED  WARRANTIES  OF
-    MERCHANTABILITY  AND  FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
-    ANY  SPECIAL ,  DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
-    WHATSOEVER  RESULTING  FROM  LOSS  OF USE, DATA OR PROFITS, WHETHER IN AN
-    ACTION  OF  CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
-    OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-*/
-//==============================================================================
-
-#ifndef RIPPLE_OVERLAY_PEER_PROTOCOL_DETECTOR_H_INCLUDED
-#define RIPPLE_OVERLAY_PEER_PROTOCOL_DETECTOR_H_INCLUDED
-
-#include <boost/asio/buffer.hpp>
-#include <boost/logic/tribool.hpp>
-#include <cstdint>
-
-namespace ripple {
-
-/** Detects the peer protocol handshake. */
-class peer_protocol_detector
-{
-public:
-    /** Returns `true` if the buffers contain the required protocol messages.
-        The peer protcol requires the 'hello' message as the first item on
-        the stream. We check the 6-byte message header to determine if the
-        hello is present.
-        @return `false` if the buffers cannot possibly contain the message, or
-            `boost::indeterminate` if more data is needed.
-    */
-    template <class ConstBufferSequence>
-    boost::tribool
-    operator() (ConstBufferSequence const& buffers);
-};
-
-template <class ConstBufferSequence>
-boost::tribool
-peer_protocol_detector::operator() (
-    ConstBufferSequence const& buffers)
-{
-    std::array <std::uint8_t, 6> data;
-    auto const n (boost::asio::buffer_copy (
-        boost::asio::buffer(data), buffers));
-    /*
-    Protocol messages are framed by a 6 byte header which includes
-    a big-endian 4-byte length followed by a big-endian 2-byte type.
-    The type for 'hello' is 1.
-    */
-    if (n>=1 && data[0] != 0)
-        return false;
-    if (n>=2 && data[1] != 0)
-        return false;
-    if (n>=5 && data[4] != 0)
-        return false;
-    if (n>=6 && data[5] != 1)
-        return false;
-    if (n>=6)
-        return true;
-    return boost::indeterminate;
-}
-
-} // ripple
-
-#endif

src/ripple/overlay/predicates.h

@@ -154,15 +154,15 @@ struct peer_in_cluster
 /** Select all peers that are in the specified set */
 struct peer_in_set
 {
-    std::set <Peer::ShortId> const& peerSet;
+    std::set <Peer::id_t> const& peerSet;
 
-    peer_in_set (std::set<Peer::ShortId> const& peers)
+    peer_in_set (std::set<Peer::id_t> const& peers)
         : peerSet (peers)
     { }
 
     bool operator() (Peer::ptr const& peer) const
     {
-        if (peerSet.count (peer->getShortId ()) == 0)
+        if (peerSet.count (peer->id ()) == 0)
             return false;
 
         return true;

src/ripple/overlay/tests/TMHello.test.cpp

@@ -0,0 +1,70 @@
+//------------------------------------------------------------------------------
+/*
+    This file is part of rippled: https://github.com/ripple/rippled
+    Copyright 2014 Ripple Labs Inc.
+
+    Permission to use, copy, modify, and/or distribute this software for any
+    purpose  with  or without fee is hereby granted, provided that the above
+    copyright notice and this permission notice appear in all copies.
+
+    THE  SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+    WITH  REGARD  TO  THIS  SOFTWARE  INCLUDING  ALL  IMPLIED  WARRANTIES  OF
+    MERCHANTABILITY  AND  FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+    ANY  SPECIAL ,  DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+    WHATSOEVER  RESULTING  FROM  LOSS  OF USE, DATA OR PROFITS, WHETHER IN AN
+    ACTION  OF  CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+    OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+*/
+//==============================================================================
+
+#include <ripple/overlay/impl/TMHello.h>
+#include <beast/unit_test/suite.h>
+
+namespace ripple {
+
+class TMHello_test : public beast::unit_test::suite
+{
+private:
+    template <class FwdIt>
+    static
+    std::string
+    join (FwdIt first, FwdIt last, char c = ',')
+    {
+        std::string result;
+        if (first == last)
+            return result;
+        result = to_string(*first++);
+        while(first != last)
+            result += "," + to_string(*first++);
+        return result;
+    }
+
+    void
+    check(std::string const& s, std::string const& answer)
+    {
+        auto const result = parse_ProtocolVersions(s);
+        expect(join(result.begin(), result.end()) == answer);
+    }
+
+public:
+    void
+    test_protocolVersions()
+    {
+        check("", "");
+        check("RTXP/1.0", "1.0");
+        check("RTXP/1.0, Websocket/1.0", "1.0");
+        check("RTXP/1.0, RTXP/1.0", "1.0");
+        check("RTXP/1.0, RTXP/1.1", "1.0,1.1");
+        check("RTXP/1.1, RTXP/1.0", "1.0,1.1");
+    }
+
+    void
+    run()
+    {
+        test_protocolVersions();
+    }
+};
+
+BEAST_DEFINE_TESTSUITE(TMHello,overlay,ripple);
+
+}

src/ripple/overlay/tests/short_read.test.cpp

@@ -1,7 +1,7 @@
 //------------------------------------------------------------------------------
 /*
-    This file is part of Beast: https://github.com/vinniefalco/Beast
+    This file is part of rippled: https://github.com/ripple/rippled
-    Copyright 2013, Vinnie Falco <vinnie.falco@gmail.com>
+    Copyright 2014 Ripple Labs Inc.
 
     Permission to use, copy, modify, and/or distribute this software for any
     purpose  with  or without fee is hereby granted, provided that the above
@@ -33,6 +33,17 @@
 #include <utility>
 
 namespace ripple {
+/*
+
+Findings from the test:
+
+If the remote host calls async_shutdown then the local host's
+async_read will complete with eof.
+
+If both hosts call async_shutdown then the calls to async_shutdown
+will complete with eof.
+
+*/
 
 class short_read_test : public beast::unit_test::suite
 {
@@ -290,19 +301,26 @@ private:
             {
                 if (ec)
                     return fail("handshake", ec);
+#if 1
                 boost::asio::async_read_until(stream_, buf_, "\n", strand_.wrap(
                     std::bind(&Connection::on_read, shared_from_this(),
                         beast::asio::placeholders::error,
                             beast::asio::placeholders::bytes_transferred)));
+#else
+                close();
+#endif
             }
 
             void
             on_read(error_code ec, std::size_t bytes_transferred)
             {
                 if (ec == boost::asio::error::eof)
+                {
+                    server_.test_.log << "[server] read: EOF";
                     return stream_.async_shutdown(strand_.wrap(std::bind(
                         &Connection::on_shutdown, shared_from_this(),
                             beast::asio::placeholders::error)));
+                }
                 if (ec)
                     return fail("read", ec);
 
@@ -463,7 +481,7 @@ private:
                 buf_.consume(bytes_transferred);
                 if (ec)
                     return fail("write", ec);
-#if 0
+#if 1
                 boost::asio::async_read_until(stream_, buf_, "\n", strand_.wrap(
                     std::bind(&Connection::on_read, shared_from_this(),
                         beast::asio::placeholders::error,

src/ripple/peerfinder/Manager.h

@@ -204,7 +204,7 @@ public:
     */
     virtual
     bool
-    connected (Slot::ptr const& slot,
+    onConnected (Slot::ptr const& slot,
         beast::IP::Endpoint const& local_endpoint) = 0;
 
     /** Request an active slot type. */

src/ripple/peerfinder/impl/Logic.h

@@ -360,7 +360,7 @@ public:
     }
 
     bool
-    connected (SlotImp::ptr const& slot,
+    onConnected (SlotImp::ptr const& slot,
         beast::IP::Endpoint const& local_endpoint)
     {
         if (m_journal.trace) m_journal.trace << beast::leftw (18) <<

src/ripple/peerfinder/impl/Manager.cpp

@@ -159,11 +159,11 @@ public:
     //--------------------------------------------------------------------------
 
     bool
-    connected (Slot::ptr const& slot,
+    onConnected (Slot::ptr const& slot,
         beast::IP::Endpoint const& local_endpoint) override
     {
         SlotImp::ptr impl (std::dynamic_pointer_cast <SlotImp> (slot));
-        return m_logic.connected (impl, local_endpoint);
+        return m_logic.onConnected (impl, local_endpoint);
     }
 
     Result

src/ripple/protocol/BuildInfo.h

@@ -25,51 +25,48 @@
 
 namespace ripple {
 
+/** Describes a Ripple/RTXP protocol version. */
+using ProtocolVersion = std::pair<std::uint16_t, std::uint16_t>;
+
 /** Versioning information for this build. */
-namespace BuildInfo
+// VFALCO The namespace is deprecated
-{
+namespace BuildInfo {
-    /** Server version.
 
-        Follows the Semantic Versioning Specification:
+/** Server version.
+    Follows the Semantic Versioning Specification:
+    http://semver.org/
+*/
+std::string const&
+getVersionString();
 
-        http://semver.org/
+/** Full server version string.
-    */
+    This includes the name of the server. It is used in the peer
-    std::string const& getVersionString ();
+    protocol hello message and also the headers of some HTTP replies.
+*/
+std::string const&
+getFullVersionString();
 
-    /** Full server version string.
+/** Construct a protocol version from a packed 32-bit protocol identifier */
+ProtocolVersion
+make_protocol (std::uint32_t version);
 
-        This includes the name of the server. It is used in the peer
+/** The protocol version we speak and prefer. */
-        protocol hello message and also the headers of some HTTP replies.
+ProtocolVersion const&
-    */
+getCurrentProtocol();
-    std::string const& getFullVersionString ();
 
-    //--------------------------------------------------------------------------
+/** The oldest protocol version we will accept. */
+ProtocolVersion const& getMinimumProtocol ();
 
-    /** The wire protocol version.
+char const*
+getRawVersionString();
 
-        The version consists of two unsigned 16 bit integers representing
+} // BuildInfo (DEPRECATED)
-        major and minor version numbers. All values are permissible.
-    */
-    using Protocol = std::pair <std::uint16_t const, std::uint16_t const>;
-
-    /** Construct a protocol version from a packed 32-bit protocol identifier */
-    Protocol
-    make_protocol (std::uint32_t version);
-
-    /** The protocol version we speak and prefer. */
-    Protocol const& getCurrentProtocol ();
-
-    /** The oldest protocol version we will accept. */
-    Protocol const& getMinimumProtocol ();
-
-    char const* getRawVersionString ();
-};
 
 std::string
-to_string (BuildInfo::Protocol const& p);
+to_string (ProtocolVersion const& p);
 
 std::uint32_t
-to_packed (BuildInfo::Protocol const& p);
+to_packed (ProtocolVersion const& p);
 
 } // ripple
 

src/ripple/protocol/impl/BuildInfo.cpp

@@ -49,9 +49,10 @@ char const* getRawVersionString ()
     return rawText;
 }
 
-Protocol const& getCurrentProtocol ()
+ProtocolVersion const&
+getCurrentProtocol ()
 {
-    static Protocol currentProtocol (
+    static ProtocolVersion currentProtocol (
     //--------------------------------------------------------------------------
     //
     // The protocol version we speak and prefer (edit this if necessary)
@@ -65,9 +66,10 @@ Protocol const& getCurrentProtocol ()
     return currentProtocol;
 }
 
-Protocol const& getMinimumProtocol ()
+ProtocolVersion const&
+getMinimumProtocol ()
 {
-    static Protocol minimumProtocol (
+    static ProtocolVersion minimumProtocol (
 
     //--------------------------------------------------------------------------
     //
@@ -88,7 +90,8 @@ Protocol const& getMinimumProtocol ()
 //
 //------------------------------------------------------------------------------
 
-std::string const& getVersionString ()
+std::string const&
+getVersionString ()
 {
     struct SanityChecker
     {
@@ -129,10 +132,10 @@ std::string const& getFullVersionString ()
     return value.fullVersionString;
 }
 
-Protocol
+ProtocolVersion
 make_protocol (std::uint32_t version)
 {
-    return Protocol (
+    return ProtocolVersion(
         static_cast<std::uint16_t> ((version >> 16) & 0xffff),
         static_cast<std::uint16_t> (version & 0xffff));
 }
@@ -140,13 +143,13 @@ make_protocol (std::uint32_t version)
 }
 
 std::string
-to_string (BuildInfo::Protocol const& p)
+to_string (ProtocolVersion const& p)
 {
     return std::to_string (p.first) + "." + std::to_string (p.second);
 }
 
 std::uint32_t
-to_packed (BuildInfo::Protocol const& p)
+to_packed (ProtocolVersion const& p)
 {
     return (static_cast<std::uint32_t> (p.first) << 16) + p.second;
 }
@@ -166,10 +169,10 @@ public:
     }
 
 
-    BuildInfo::Protocol
+    ProtocolVersion
     from_version (std::uint16_t major, std::uint16_t minor)
     {
-        return BuildInfo::Protocol (major, minor);
+        return ProtocolVersion (major, minor);
     }
 
     void testValues ()

src/ripple/server/ServerHandler.h

@@ -86,6 +86,11 @@ public:
     virtual
     Setup const&
     setup() const = 0;
+
+    /** Fills in boilerplate HTTP header field values. */
+    static
+    void
+    appendStandardFields (beast::http::message& message);
 };
 
 //------------------------------------------------------------------------------

src/ripple/server/SimpleWriter.h

@@ -0,0 +1,108 @@
+//------------------------------------------------------------------------------
+/*
+    This file is part of rippled: https://github.com/ripple/rippled
+    Copyright (c) 2012, 2013 Ripple Labs Inc.
+
+    Permission to use, copy, modify, and/or distribute this software for any
+    purpose  with  or without fee is hereby granted, provided that the above
+    copyright notice and this permission notice appear in all copies.
+
+    THE  SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+    WITH  REGARD  TO  THIS  SOFTWARE  INCLUDING  ALL  IMPLIED  WARRANTIES  OF
+    MERCHANTABILITY  AND  FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+    ANY  SPECIAL ,  DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+    WHATSOEVER  RESULTING  FROM  LOSS  OF USE, DATA OR PROFITS, WHETHER IN AN
+    ACTION  OF  CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+    OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+*/
+//==============================================================================
+
+#ifndef RIPPLE_SERVER_SIMPLEWRITER_H_INCLUDED
+#define RIPPLE_SERVER_SIMPLEWRITER_H_INCLUDED
+
+#include <ripple/server/Writer.h>
+#include <beast/asio/streambuf.h>
+#include <beast/http/message.h>
+#include <utility>
+
+namespace ripple {
+namespace HTTP {
+
+/** Writer that sends a simple HTTP response with a message body. */
+class SimpleWriter : public Writer
+{
+private:
+    beast::http::message message_;
+    beast::asio::streambuf streambuf_;
+    std::string body_;
+    bool prepared_ = false;
+
+public:
+    explicit
+    SimpleWriter(beast::http::message&& message)
+        : message_(std::forward<beast::http::message>(message))
+    {
+    }
+
+    beast::http::message&
+    message()
+    {
+        return message_;
+    }
+
+    bool
+    complete() override
+    {
+        return streambuf_.size() == 0;
+    }
+
+    void
+    consume (std::size_t bytes) override
+    {
+        streambuf_.consume(bytes);
+    }
+
+    bool
+    prepare (std::size_t bytes,
+        std::function<void(void)>) override
+    {
+        if (! prepared_)
+            do_prepare();
+        return true;
+    }
+
+    std::vector<boost::asio::const_buffer>
+    data() override
+    {
+        auto const& buf = streambuf_.data();
+        std::vector<boost::asio::const_buffer> result;
+        result.reserve(std::distance(buf.begin(), buf.end()));
+        for (auto const& b : buf)
+            result.push_back(b);
+        return result;
+    }
+
+    /** Set the content body. */
+    void
+    body (std::string const& s)
+    {
+        body_ = s;
+    }
+
+private:
+    void
+    do_prepare()
+    {
+        prepared_ = true;
+        message_.headers.erase("Content-Length");
+        message_.headers.append("Content-Length",
+            std::to_string(body_.size()));
+        write(streambuf_, message_);
+        write(streambuf_, body_;
+    }
+};
+
+}
+}
+
+#endif

src/ripple/server/impl/ServerHandlerImp.cpp

@@ -428,6 +428,13 @@ adminRole (HTTP::Port const& port,
 
 //------------------------------------------------------------------------------
 
+void
+ServerHandler::appendStandardFields (beast::http::message& message)
+{
+}
+
+//------------------------------------------------------------------------------
+
 void
 ServerHandler::Setup::makeContexts()
 {
@@ -453,42 +460,6 @@ ServerHandler::Setup::makeContexts()
 
 namespace detail {
 
-// Parse a comma-delimited list of values.
-std::vector<std::string>
-parse_csv (std::string const& in, std::ostream& log)
-{
-    auto first = in.cbegin();
-    auto const last = in.cend();
-    std::vector<std::string> result;
-    if (first != last)
-    {
-        static boost::regex const re(
-            "^"                         // start of line
-            "(?:\\s*)"                  // whitespace (optional)
-            "([a-zA-Z][_a-zA-Z0-9]*)"   // identifier
-            "(?:\\s*)"                  // whitespace (optional)
-            "(?:,?)"                    // comma (optional)
-            "(?:\\s*)"                  // whitespace (optional)
-            , boost::regex_constants::optimize
-        );
-        for(;;)
-        {
-            boost::smatch m;
-            if (! boost::regex_search(first, last, m, re,
-                boost::regex_constants::match_continuous))
-            {
-                log << "Expected <identifier>\n";
-                throw std::exception();
-            }
-            result.push_back(m[1]);
-            first = m[0].second;
-            if (first == last)
-                break;
-        }
-    }
-    return result;
-}
-
 // Intermediate structure used for parsing
 struct ParsedPort
 {

src/ripple/unity/overlay.cpp

@@ -27,5 +27,8 @@
 #include <ripple/overlay/impl/message_name.cpp>
 #include <ripple/overlay/impl/OverlayImpl.cpp>
 #include <ripple/overlay/impl/PeerImp.cpp>
-#include <ripple/overlay/tests/short_read.test.cpp>
+#include <ripple/overlay/impl/TMHello.cpp>
+
+#include <ripple/overlay/tests/short_read.test.cpp>
+#include <ripple/overlay/tests/TMHello.test.cpp>