mirror of
https://github.com/XRPLF/rippled.git
synced 2025-12-06 17:27:55 +00:00
Merge remote-tracking branch 'XRPLF/ximinez/lending-refactoring-4' into ximinez/lending-XLS-66
* XRPLF/ximinez/lending-refactoring-4: fix: Modify jobs to use '>>' instead of 'tee' for GITHUB_OUTPUT (5699) refactor: Revamp CI workflows (5661) refactor: Decouple net from xrpld and move rpc-related classes to the rpc folder (5477) Set version to 2.6.0-rc2 docs: Updates list of maintainers and reviewers (5687) fix: Change log to debug level for AMM offer retrieval and IOU payment check (5686) fix: Add -Wno-deprecated-declarations for Clang only (5680) Update .git-blame-ignore-revs for 5657 (5675) Fix BUILD.md instruction (5676) Set version to 2.6.0-rc1 fix: Improve logging of the reason to refuse a peer connection (5664) fix: Make test suite names match the directory name (5597) chore: Run prettier on all files (5657) chore: Set CONAN_REMOTE_URL also for forks (5662) chore: Cleanup bin/ directory (5660) perf: Optimize hash performance by avoiding allocating hash state object (5469)
This commit is contained in:
@@ -22,18 +22,8 @@
|
||||
|
||||
#include <xrpl/basics/contract.h>
|
||||
|
||||
#if defined(__clang__)
|
||||
#pragma clang diagnostic push
|
||||
#pragma clang diagnostic ignored "-Wdeprecated"
|
||||
#pragma clang diagnostic ignored "-Wdeprecated-declarations"
|
||||
#endif
|
||||
|
||||
#include <boost/outcome.hpp>
|
||||
|
||||
#if defined(__clang__)
|
||||
#pragma clang diagnostic pop
|
||||
#endif
|
||||
|
||||
#include <stdexcept>
|
||||
|
||||
namespace ripple {
|
||||
|
||||
@@ -4,37 +4,34 @@ Utility functions and classes.
|
||||
|
||||
ripple/basic should contain no dependencies on other modules.
|
||||
|
||||
# Choosing a rippled container.
|
||||
|
||||
Choosing a rippled container.
|
||||
=============================
|
||||
- `std::vector`
|
||||
- For ordered containers with most insertions or erases at the end.
|
||||
|
||||
* `std::vector`
|
||||
* For ordered containers with most insertions or erases at the end.
|
||||
- `std::deque`
|
||||
- For ordered containers with most insertions or erases at the start or end.
|
||||
|
||||
* `std::deque`
|
||||
* For ordered containers with most insertions or erases at the start or end.
|
||||
|
||||
* `std::list`
|
||||
* For ordered containers with inserts and erases to the middle.
|
||||
* For containers with iterators stable over insert and erase.
|
||||
* Generally slower and bigger than `std::vector` or `std::deque` except for
|
||||
- `std::list`
|
||||
- For ordered containers with inserts and erases to the middle.
|
||||
- For containers with iterators stable over insert and erase.
|
||||
- Generally slower and bigger than `std::vector` or `std::deque` except for
|
||||
those cases.
|
||||
|
||||
* `std::set`
|
||||
* For sorted containers.
|
||||
- `std::set`
|
||||
- For sorted containers.
|
||||
|
||||
* `ripple::hash_set`
|
||||
* Where inserts and contains need to be O(1).
|
||||
* For "small" sets, `std::set` might be faster and smaller.
|
||||
- `ripple::hash_set`
|
||||
- Where inserts and contains need to be O(1).
|
||||
- For "small" sets, `std::set` might be faster and smaller.
|
||||
|
||||
* `ripple::hardened_hash_set`
|
||||
* For data sets where the key could be manipulated by an attacker
|
||||
in an attempt to mount an algorithmic complexity attack: see
|
||||
- `ripple::hardened_hash_set`
|
||||
- For data sets where the key could be manipulated by an attacker
|
||||
in an attempt to mount an algorithmic complexity attack: see
|
||||
http://en.wikipedia.org/wiki/Algorithmic_complexity_attack
|
||||
|
||||
|
||||
The following container is deprecated
|
||||
|
||||
* `std::unordered_set`
|
||||
* Use `ripple::hash_set` instead, which uses a better hashing algorithm.
|
||||
* Or use `ripple::hardened_hash_set` to prevent algorithmic complexity attacks.
|
||||
- `std::unordered_set`
|
||||
- Use `ripple::hash_set` instead, which uses a better hashing algorithm.
|
||||
- Or use `ripple::hardened_hash_set` to prevent algorithmic complexity attacks.
|
||||
|
||||
@@ -24,35 +24,10 @@
|
||||
#include <boost/container/flat_set.hpp>
|
||||
#include <boost/endian/conversion.hpp>
|
||||
|
||||
/*
|
||||
|
||||
Workaround for overzealous clang warning, which trips on libstdc++ headers
|
||||
|
||||
In file included from
|
||||
/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_algo.h:61:
|
||||
/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_tempbuf.h:263:8:
|
||||
error: 'get_temporary_buffer<std::pair<ripple::Quality, const
|
||||
std::vector<std::unique_ptr<ripple::Step>> *>>' is deprecated
|
||||
[-Werror,-Wdeprecated-declarations] 263 |
|
||||
std::get_temporary_buffer<value_type>(_M_original_len));
|
||||
^
|
||||
*/
|
||||
|
||||
#if defined(__clang__)
|
||||
#pragma clang diagnostic push
|
||||
#pragma clang diagnostic ignored "-Wdeprecated"
|
||||
#pragma clang diagnostic ignored "-Wdeprecated-declarations"
|
||||
#endif
|
||||
|
||||
#include <functional>
|
||||
|
||||
#if defined(__clang__)
|
||||
#pragma clang diagnostic pop
|
||||
#endif
|
||||
|
||||
#include <array>
|
||||
#include <chrono>
|
||||
#include <cstring>
|
||||
#include <functional>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <system_error>
|
||||
|
||||
@@ -24,32 +24,110 @@
|
||||
|
||||
#include <xxhash.h>
|
||||
|
||||
#include <array>
|
||||
#include <cstddef>
|
||||
#include <new>
|
||||
#include <type_traits>
|
||||
#include <cstdint>
|
||||
#include <optional>
|
||||
#include <span>
|
||||
|
||||
namespace beast {
|
||||
|
||||
class xxhasher
|
||||
{
|
||||
private:
|
||||
// requires 64-bit std::size_t
|
||||
static_assert(sizeof(std::size_t) == 8, "");
|
||||
public:
|
||||
using result_type = std::size_t;
|
||||
|
||||
XXH3_state_t* state_;
|
||||
private:
|
||||
static_assert(sizeof(std::size_t) == 8, "requires 64-bit std::size_t");
|
||||
// Have an internal buffer to avoid the streaming API
|
||||
// A 64-byte buffer should to be big enough for us
|
||||
static constexpr std::size_t INTERNAL_BUFFER_SIZE = 64;
|
||||
|
||||
alignas(64) std::array<std::uint8_t, INTERNAL_BUFFER_SIZE> buffer_;
|
||||
std::span<std::uint8_t> readBuffer_;
|
||||
std::span<std::uint8_t> writeBuffer_;
|
||||
|
||||
std::optional<XXH64_hash_t> seed_;
|
||||
XXH3_state_t* state_ = nullptr;
|
||||
|
||||
void
|
||||
resetBuffers()
|
||||
{
|
||||
writeBuffer_ = std::span{buffer_};
|
||||
readBuffer_ = {};
|
||||
}
|
||||
|
||||
void
|
||||
updateHash(void const* data, std::size_t len)
|
||||
{
|
||||
if (writeBuffer_.size() < len)
|
||||
{
|
||||
flushToState(data, len);
|
||||
}
|
||||
else
|
||||
{
|
||||
std::memcpy(writeBuffer_.data(), data, len);
|
||||
writeBuffer_ = writeBuffer_.subspan(len);
|
||||
readBuffer_ = std::span{
|
||||
std::begin(buffer_), buffer_.size() - writeBuffer_.size()};
|
||||
}
|
||||
}
|
||||
|
||||
static XXH3_state_t*
|
||||
allocState()
|
||||
{
|
||||
auto ret = XXH3_createState();
|
||||
if (ret == nullptr)
|
||||
throw std::bad_alloc();
|
||||
throw std::bad_alloc(); // LCOV_EXCL_LINE
|
||||
return ret;
|
||||
}
|
||||
|
||||
public:
|
||||
using result_type = std::size_t;
|
||||
void
|
||||
flushToState(void const* data, std::size_t len)
|
||||
{
|
||||
if (!state_)
|
||||
{
|
||||
state_ = allocState();
|
||||
if (seed_.has_value())
|
||||
{
|
||||
XXH3_64bits_reset_withSeed(state_, *seed_);
|
||||
}
|
||||
else
|
||||
{
|
||||
XXH3_64bits_reset(state_);
|
||||
}
|
||||
}
|
||||
XXH3_64bits_update(state_, readBuffer_.data(), readBuffer_.size());
|
||||
resetBuffers();
|
||||
if (data && len)
|
||||
{
|
||||
XXH3_64bits_update(state_, data, len);
|
||||
}
|
||||
}
|
||||
|
||||
result_type
|
||||
retrieveHash()
|
||||
{
|
||||
if (state_)
|
||||
{
|
||||
flushToState(nullptr, 0);
|
||||
return XXH3_64bits_digest(state_);
|
||||
}
|
||||
else
|
||||
{
|
||||
if (seed_.has_value())
|
||||
{
|
||||
return XXH3_64bits_withSeed(
|
||||
readBuffer_.data(), readBuffer_.size(), *seed_);
|
||||
}
|
||||
else
|
||||
{
|
||||
return XXH3_64bits(readBuffer_.data(), readBuffer_.size());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public:
|
||||
static constexpr auto const endian = boost::endian::order::native;
|
||||
|
||||
xxhasher(xxhasher const&) = delete;
|
||||
@@ -58,43 +136,43 @@ public:
|
||||
|
||||
xxhasher()
|
||||
{
|
||||
state_ = allocState();
|
||||
XXH3_64bits_reset(state_);
|
||||
resetBuffers();
|
||||
}
|
||||
|
||||
~xxhasher() noexcept
|
||||
{
|
||||
XXH3_freeState(state_);
|
||||
if (state_)
|
||||
{
|
||||
XXH3_freeState(state_);
|
||||
}
|
||||
}
|
||||
|
||||
template <
|
||||
class Seed,
|
||||
std::enable_if_t<std::is_unsigned<Seed>::value>* = nullptr>
|
||||
explicit xxhasher(Seed seed)
|
||||
explicit xxhasher(Seed seed) : seed_(seed)
|
||||
{
|
||||
state_ = allocState();
|
||||
XXH3_64bits_reset_withSeed(state_, seed);
|
||||
resetBuffers();
|
||||
}
|
||||
|
||||
template <
|
||||
class Seed,
|
||||
std::enable_if_t<std::is_unsigned<Seed>::value>* = nullptr>
|
||||
xxhasher(Seed seed, Seed)
|
||||
xxhasher(Seed seed, Seed) : seed_(seed)
|
||||
{
|
||||
state_ = allocState();
|
||||
XXH3_64bits_reset_withSeed(state_, seed);
|
||||
resetBuffers();
|
||||
}
|
||||
|
||||
void
|
||||
operator()(void const* key, std::size_t len) noexcept
|
||||
{
|
||||
XXH3_64bits_update(state_, key, len);
|
||||
updateHash(key, len);
|
||||
}
|
||||
|
||||
explicit
|
||||
operator std::size_t() noexcept
|
||||
operator result_type() noexcept
|
||||
{
|
||||
return XXH3_64bits_digest(state_);
|
||||
return retrieveHash();
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
328
include/xrpl/net/AutoSocket.h
Normal file
328
include/xrpl/net/AutoSocket.h
Normal file
@@ -0,0 +1,328 @@
|
||||
//------------------------------------------------------------------------------
|
||||
/*
|
||||
This file is part of rippled: https://github.com/ripple/rippled
|
||||
Copyright (c) 2012, 2013 Ripple Labs Inc.
|
||||
|
||||
Permission to use, copy, modify, and/or distribute this software for any
|
||||
purpose with or without fee is hereby granted, provided that the above
|
||||
copyright notice and this permission notice appear in all copies.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
|
||||
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
|
||||
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
|
||||
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
|
||||
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
|
||||
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
|
||||
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||
*/
|
||||
//==============================================================================
|
||||
|
||||
#ifndef RIPPLE_WEBSOCKET_AUTOSOCKET_AUTOSOCKET_H_INCLUDED
|
||||
#define RIPPLE_WEBSOCKET_AUTOSOCKET_AUTOSOCKET_H_INCLUDED
|
||||
|
||||
#include <xrpl/basics/Log.h>
|
||||
#include <xrpl/beast/net/IPAddressConversion.h>
|
||||
|
||||
#include <boost/asio.hpp>
|
||||
#include <boost/asio/ip/tcp.hpp>
|
||||
#include <boost/asio/ssl.hpp>
|
||||
#include <boost/beast/core/bind_handler.hpp>
|
||||
|
||||
// Socket wrapper that supports both SSL and non-SSL connections.
|
||||
// Generally, handle it as you would an SSL connection.
|
||||
// To force a non-SSL connection, just don't call async_handshake.
|
||||
// To force SSL only inbound, call setSSLOnly.
|
||||
|
||||
class AutoSocket
|
||||
{
|
||||
public:
|
||||
using ssl_socket = boost::asio::ssl::stream<boost::asio::ip::tcp::socket>;
|
||||
using endpoint_type = boost::asio::ip::tcp::socket::endpoint_type;
|
||||
using socket_ptr = std::unique_ptr<ssl_socket>;
|
||||
using plain_socket = ssl_socket::next_layer_type;
|
||||
using lowest_layer_type = ssl_socket::lowest_layer_type;
|
||||
using handshake_type = ssl_socket::handshake_type;
|
||||
using error_code = boost::system::error_code;
|
||||
using callback = std::function<void(error_code)>;
|
||||
|
||||
public:
|
||||
AutoSocket(
|
||||
boost::asio::io_service& s,
|
||||
boost::asio::ssl::context& c,
|
||||
bool secureOnly,
|
||||
bool plainOnly)
|
||||
: mSecure(secureOnly)
|
||||
, mBuffer((plainOnly || secureOnly) ? 0 : 4)
|
||||
, j_{beast::Journal::getNullSink()}
|
||||
{
|
||||
mSocket = std::make_unique<ssl_socket>(s, c);
|
||||
}
|
||||
|
||||
AutoSocket(boost::asio::io_service& s, boost::asio::ssl::context& c)
|
||||
: AutoSocket(s, c, false, false)
|
||||
{
|
||||
}
|
||||
|
||||
bool
|
||||
isSecure()
|
||||
{
|
||||
return mSecure;
|
||||
}
|
||||
ssl_socket&
|
||||
SSLSocket()
|
||||
{
|
||||
return *mSocket;
|
||||
}
|
||||
plain_socket&
|
||||
PlainSocket()
|
||||
{
|
||||
return mSocket->next_layer();
|
||||
}
|
||||
|
||||
beast::IP::Endpoint
|
||||
local_endpoint()
|
||||
{
|
||||
return beast::IP::from_asio(lowest_layer().local_endpoint());
|
||||
}
|
||||
|
||||
beast::IP::Endpoint
|
||||
remote_endpoint()
|
||||
{
|
||||
return beast::IP::from_asio(lowest_layer().remote_endpoint());
|
||||
}
|
||||
|
||||
lowest_layer_type&
|
||||
lowest_layer()
|
||||
{
|
||||
return mSocket->lowest_layer();
|
||||
}
|
||||
|
||||
void
|
||||
swap(AutoSocket& s) noexcept
|
||||
{
|
||||
mBuffer.swap(s.mBuffer);
|
||||
mSocket.swap(s.mSocket);
|
||||
std::swap(mSecure, s.mSecure);
|
||||
}
|
||||
|
||||
boost::system::error_code
|
||||
cancel(boost::system::error_code& ec)
|
||||
{
|
||||
return lowest_layer().cancel(ec);
|
||||
}
|
||||
|
||||
void
|
||||
async_handshake(handshake_type type, callback cbFunc)
|
||||
{
|
||||
if ((type == ssl_socket::client) || (mSecure))
|
||||
{
|
||||
// must be ssl
|
||||
mSecure = true;
|
||||
mSocket->async_handshake(type, cbFunc);
|
||||
}
|
||||
else if (mBuffer.empty())
|
||||
{
|
||||
// must be plain
|
||||
mSecure = false;
|
||||
post(
|
||||
mSocket->get_executor(),
|
||||
boost::beast::bind_handler(cbFunc, error_code()));
|
||||
}
|
||||
else
|
||||
{
|
||||
// autodetect
|
||||
mSocket->next_layer().async_receive(
|
||||
boost::asio::buffer(mBuffer),
|
||||
boost::asio::socket_base::message_peek,
|
||||
std::bind(
|
||||
&AutoSocket::handle_autodetect,
|
||||
this,
|
||||
cbFunc,
|
||||
std::placeholders::_1,
|
||||
std::placeholders::_2));
|
||||
}
|
||||
}
|
||||
|
||||
template <typename ShutdownHandler>
|
||||
void
|
||||
async_shutdown(ShutdownHandler handler)
|
||||
{
|
||||
if (isSecure())
|
||||
mSocket->async_shutdown(handler);
|
||||
else
|
||||
{
|
||||
error_code ec;
|
||||
try
|
||||
{
|
||||
lowest_layer().shutdown(plain_socket::shutdown_both);
|
||||
}
|
||||
catch (boost::system::system_error& e)
|
||||
{
|
||||
ec = e.code();
|
||||
}
|
||||
post(
|
||||
mSocket->get_executor(),
|
||||
boost::beast::bind_handler(handler, ec));
|
||||
}
|
||||
}
|
||||
|
||||
template <typename Seq, typename Handler>
|
||||
void
|
||||
async_read_some(Seq const& buffers, Handler handler)
|
||||
{
|
||||
if (isSecure())
|
||||
mSocket->async_read_some(buffers, handler);
|
||||
else
|
||||
PlainSocket().async_read_some(buffers, handler);
|
||||
}
|
||||
|
||||
template <typename Seq, typename Condition, typename Handler>
|
||||
void
|
||||
async_read_until(Seq const& buffers, Condition condition, Handler handler)
|
||||
{
|
||||
if (isSecure())
|
||||
boost::asio::async_read_until(
|
||||
*mSocket, buffers, condition, handler);
|
||||
else
|
||||
boost::asio::async_read_until(
|
||||
PlainSocket(), buffers, condition, handler);
|
||||
}
|
||||
|
||||
template <typename Allocator, typename Handler>
|
||||
void
|
||||
async_read_until(
|
||||
boost::asio::basic_streambuf<Allocator>& buffers,
|
||||
std::string const& delim,
|
||||
Handler handler)
|
||||
{
|
||||
if (isSecure())
|
||||
boost::asio::async_read_until(*mSocket, buffers, delim, handler);
|
||||
else
|
||||
boost::asio::async_read_until(
|
||||
PlainSocket(), buffers, delim, handler);
|
||||
}
|
||||
|
||||
template <typename Allocator, typename MatchCondition, typename Handler>
|
||||
void
|
||||
async_read_until(
|
||||
boost::asio::basic_streambuf<Allocator>& buffers,
|
||||
MatchCondition cond,
|
||||
Handler handler)
|
||||
{
|
||||
if (isSecure())
|
||||
boost::asio::async_read_until(*mSocket, buffers, cond, handler);
|
||||
else
|
||||
boost::asio::async_read_until(
|
||||
PlainSocket(), buffers, cond, handler);
|
||||
}
|
||||
|
||||
template <typename Buf, typename Handler>
|
||||
void
|
||||
async_write(Buf const& buffers, Handler handler)
|
||||
{
|
||||
if (isSecure())
|
||||
boost::asio::async_write(*mSocket, buffers, handler);
|
||||
else
|
||||
boost::asio::async_write(PlainSocket(), buffers, handler);
|
||||
}
|
||||
|
||||
template <typename Allocator, typename Handler>
|
||||
void
|
||||
async_write(
|
||||
boost::asio::basic_streambuf<Allocator>& buffers,
|
||||
Handler handler)
|
||||
{
|
||||
if (isSecure())
|
||||
boost::asio::async_write(*mSocket, buffers, handler);
|
||||
else
|
||||
boost::asio::async_write(PlainSocket(), buffers, handler);
|
||||
}
|
||||
|
||||
template <typename Buf, typename Condition, typename Handler>
|
||||
void
|
||||
async_read(Buf const& buffers, Condition cond, Handler handler)
|
||||
{
|
||||
if (isSecure())
|
||||
boost::asio::async_read(*mSocket, buffers, cond, handler);
|
||||
else
|
||||
boost::asio::async_read(PlainSocket(), buffers, cond, handler);
|
||||
}
|
||||
|
||||
template <typename Allocator, typename Condition, typename Handler>
|
||||
void
|
||||
async_read(
|
||||
boost::asio::basic_streambuf<Allocator>& buffers,
|
||||
Condition cond,
|
||||
Handler handler)
|
||||
{
|
||||
if (isSecure())
|
||||
boost::asio::async_read(*mSocket, buffers, cond, handler);
|
||||
else
|
||||
boost::asio::async_read(PlainSocket(), buffers, cond, handler);
|
||||
}
|
||||
|
||||
template <typename Buf, typename Handler>
|
||||
void
|
||||
async_read(Buf const& buffers, Handler handler)
|
||||
{
|
||||
if (isSecure())
|
||||
boost::asio::async_read(*mSocket, buffers, handler);
|
||||
else
|
||||
boost::asio::async_read(PlainSocket(), buffers, handler);
|
||||
}
|
||||
|
||||
template <typename Seq, typename Handler>
|
||||
void
|
||||
async_write_some(Seq const& buffers, Handler handler)
|
||||
{
|
||||
if (isSecure())
|
||||
mSocket->async_write_some(buffers, handler);
|
||||
else
|
||||
PlainSocket().async_write_some(buffers, handler);
|
||||
}
|
||||
|
||||
protected:
|
||||
void
|
||||
handle_autodetect(
|
||||
callback cbFunc,
|
||||
error_code const& ec,
|
||||
size_t bytesTransferred)
|
||||
{
|
||||
using namespace ripple;
|
||||
|
||||
if (ec)
|
||||
{
|
||||
JLOG(j_.warn()) << "Handle autodetect error: " << ec;
|
||||
cbFunc(ec);
|
||||
}
|
||||
else if (
|
||||
(mBuffer[0] < 127) && (mBuffer[0] > 31) &&
|
||||
((bytesTransferred < 2) ||
|
||||
((mBuffer[1] < 127) && (mBuffer[1] > 31))) &&
|
||||
((bytesTransferred < 3) ||
|
||||
((mBuffer[2] < 127) && (mBuffer[2] > 31))) &&
|
||||
((bytesTransferred < 4) ||
|
||||
((mBuffer[3] < 127) && (mBuffer[3] > 31))))
|
||||
{
|
||||
// not ssl
|
||||
JLOG(j_.trace()) << "non-SSL";
|
||||
mSecure = false;
|
||||
cbFunc(ec);
|
||||
}
|
||||
else
|
||||
{
|
||||
// ssl
|
||||
JLOG(j_.trace()) << "SSL";
|
||||
mSecure = true;
|
||||
mSocket->async_handshake(ssl_socket::server, cbFunc);
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
socket_ptr mSocket;
|
||||
bool mSecure;
|
||||
std::vector<char> mBuffer;
|
||||
beast::Journal j_;
|
||||
};
|
||||
|
||||
#endif
|
||||
99
include/xrpl/net/HTTPClient.h
Normal file
99
include/xrpl/net/HTTPClient.h
Normal file
@@ -0,0 +1,99 @@
|
||||
//------------------------------------------------------------------------------
|
||||
/*
|
||||
This file is part of rippled: https://github.com/ripple/rippled
|
||||
Copyright (c) 2012, 2013 Ripple Labs Inc.
|
||||
|
||||
Permission to use, copy, modify, and/or distribute this software for any
|
||||
purpose with or without fee is hereby granted, provided that the above
|
||||
copyright notice and this permission notice appear in all copies.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
|
||||
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
|
||||
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
|
||||
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
|
||||
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
|
||||
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
|
||||
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||
*/
|
||||
//==============================================================================
|
||||
|
||||
#ifndef RIPPLE_NET_HTTPCLIENT_H_INCLUDED
|
||||
#define RIPPLE_NET_HTTPCLIENT_H_INCLUDED
|
||||
|
||||
#include <xrpl/basics/ByteUtilities.h>
|
||||
#include <xrpl/beast/utility/Journal.h>
|
||||
|
||||
#include <boost/asio/io_service.hpp>
|
||||
#include <boost/asio/streambuf.hpp>
|
||||
|
||||
#include <chrono>
|
||||
#include <deque>
|
||||
#include <functional>
|
||||
#include <string>
|
||||
|
||||
namespace ripple {
|
||||
|
||||
/** Provides an asynchronous HTTP client implementation with optional SSL.
|
||||
*/
|
||||
class HTTPClient
|
||||
{
|
||||
public:
|
||||
explicit HTTPClient() = default;
|
||||
|
||||
static constexpr auto maxClientHeaderBytes = kilobytes(32);
|
||||
|
||||
static void
|
||||
initializeSSLContext(
|
||||
std::string const& sslVerifyDir,
|
||||
std::string const& sslVerifyFile,
|
||||
bool sslVerify,
|
||||
beast::Journal j);
|
||||
|
||||
static void
|
||||
get(bool bSSL,
|
||||
boost::asio::io_service& io_service,
|
||||
std::deque<std::string> deqSites,
|
||||
unsigned short const port,
|
||||
std::string const& strPath,
|
||||
std::size_t responseMax, // if no Content-Length header
|
||||
std::chrono::seconds timeout,
|
||||
std::function<bool(
|
||||
boost::system::error_code const& ecResult,
|
||||
int iStatus,
|
||||
std::string const& strData)> complete,
|
||||
beast::Journal& j);
|
||||
|
||||
static void
|
||||
get(bool bSSL,
|
||||
boost::asio::io_service& io_service,
|
||||
std::string strSite,
|
||||
unsigned short const port,
|
||||
std::string const& strPath,
|
||||
std::size_t responseMax, // if no Content-Length header
|
||||
std::chrono::seconds timeout,
|
||||
std::function<bool(
|
||||
boost::system::error_code const& ecResult,
|
||||
int iStatus,
|
||||
std::string const& strData)> complete,
|
||||
beast::Journal& j);
|
||||
|
||||
static void
|
||||
request(
|
||||
bool bSSL,
|
||||
boost::asio::io_service& io_service,
|
||||
std::string strSite,
|
||||
unsigned short const port,
|
||||
std::function<
|
||||
void(boost::asio::streambuf& sb, std::string const& strHost)> build,
|
||||
std::size_t responseMax, // if no Content-Length header
|
||||
std::chrono::seconds timeout,
|
||||
std::function<bool(
|
||||
boost::system::error_code const& ecResult,
|
||||
int iStatus,
|
||||
std::string const& strData)> complete,
|
||||
beast::Journal& j);
|
||||
};
|
||||
|
||||
} // namespace ripple
|
||||
|
||||
#endif
|
||||
200
include/xrpl/net/HTTPClientSSLContext.h
Normal file
200
include/xrpl/net/HTTPClientSSLContext.h
Normal file
@@ -0,0 +1,200 @@
|
||||
//------------------------------------------------------------------------------
|
||||
/*
|
||||
This file is part of rippled: https://github.com/ripple/rippled
|
||||
Copyright (c) 2019 Ripple Labs Inc.
|
||||
|
||||
Permission to use, copy, modify, and/or distribute this software for any
|
||||
purpose with or without fee is hereby granted, provided that the above
|
||||
copyright notice and this permission notice appear in all copies.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
|
||||
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
|
||||
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
|
||||
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
|
||||
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
|
||||
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
|
||||
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||
*/
|
||||
//==============================================================================
|
||||
|
||||
#ifndef RIPPLE_NET_HTTPCLIENTSSLCONTEXT_H_INCLUDED
|
||||
#define RIPPLE_NET_HTTPCLIENTSSLCONTEXT_H_INCLUDED
|
||||
|
||||
#include <xrpl/basics/Log.h>
|
||||
#include <xrpl/basics/contract.h>
|
||||
#include <xrpl/beast/utility/Journal.h>
|
||||
#include <xrpl/net/RegisterSSLCerts.h>
|
||||
|
||||
#include <boost/asio.hpp>
|
||||
#include <boost/asio/ip/tcp.hpp>
|
||||
#include <boost/asio/ssl.hpp>
|
||||
#include <boost/format.hpp>
|
||||
|
||||
namespace ripple {
|
||||
|
||||
class HTTPClientSSLContext
|
||||
{
|
||||
public:
|
||||
explicit HTTPClientSSLContext(
|
||||
std::string const& sslVerifyDir,
|
||||
std::string const& sslVerifyFile,
|
||||
bool sslVerify,
|
||||
beast::Journal j,
|
||||
boost::asio::ssl::context_base::method method =
|
||||
boost::asio::ssl::context::sslv23)
|
||||
: ssl_context_{method}, j_(j), verify_{sslVerify}
|
||||
{
|
||||
boost::system::error_code ec;
|
||||
|
||||
if (sslVerifyFile.empty())
|
||||
{
|
||||
registerSSLCerts(ssl_context_, ec, j_);
|
||||
|
||||
if (ec && sslVerifyDir.empty())
|
||||
Throw<std::runtime_error>(boost::str(
|
||||
boost::format("Failed to set_default_verify_paths: %s") %
|
||||
ec.message()));
|
||||
}
|
||||
else
|
||||
{
|
||||
ssl_context_.load_verify_file(sslVerifyFile);
|
||||
}
|
||||
|
||||
if (!sslVerifyDir.empty())
|
||||
{
|
||||
ssl_context_.add_verify_path(sslVerifyDir, ec);
|
||||
|
||||
if (ec)
|
||||
Throw<std::runtime_error>(boost::str(
|
||||
boost::format("Failed to add verify path: %s") %
|
||||
ec.message()));
|
||||
}
|
||||
}
|
||||
|
||||
boost::asio::ssl::context&
|
||||
context()
|
||||
{
|
||||
return ssl_context_;
|
||||
}
|
||||
|
||||
bool
|
||||
sslVerify() const
|
||||
{
|
||||
return verify_;
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief invoked before connect/async_connect on an ssl stream
|
||||
* to setup name verification.
|
||||
*
|
||||
* If we intend to verify the SSL connection, we need to set the
|
||||
* default domain for server name indication *prior* to connecting
|
||||
*
|
||||
* @param strm asio ssl stream
|
||||
* @param host hostname to verify
|
||||
*
|
||||
* @return error_code indicating failures, if any
|
||||
*/
|
||||
template <
|
||||
class T,
|
||||
class = std::enable_if_t<
|
||||
std::is_same<
|
||||
T,
|
||||
boost::asio::ssl::stream<boost::asio::ip::tcp::socket>>::
|
||||
value ||
|
||||
std::is_same<
|
||||
T,
|
||||
boost::asio::ssl::stream<boost::asio::ip::tcp::socket&>>::
|
||||
value>>
|
||||
boost::system::error_code
|
||||
preConnectVerify(T& strm, std::string const& host)
|
||||
{
|
||||
boost::system::error_code ec;
|
||||
if (!SSL_set_tlsext_host_name(strm.native_handle(), host.c_str()))
|
||||
{
|
||||
ec.assign(
|
||||
static_cast<int>(::ERR_get_error()),
|
||||
boost::asio::error::get_ssl_category());
|
||||
}
|
||||
else if (!sslVerify())
|
||||
{
|
||||
strm.set_verify_mode(boost::asio::ssl::verify_none, ec);
|
||||
}
|
||||
return ec;
|
||||
}
|
||||
|
||||
template <
|
||||
class T,
|
||||
class = std::enable_if_t<
|
||||
std::is_same<
|
||||
T,
|
||||
boost::asio::ssl::stream<boost::asio::ip::tcp::socket>>::
|
||||
value ||
|
||||
std::is_same<
|
||||
T,
|
||||
boost::asio::ssl::stream<boost::asio::ip::tcp::socket&>>::
|
||||
value>>
|
||||
/**
|
||||
* @brief invoked after connect/async_connect but before sending data
|
||||
* on an ssl stream - to setup name verification.
|
||||
*
|
||||
* @param strm asio ssl stream
|
||||
* @param host hostname to verify
|
||||
*/
|
||||
boost::system::error_code
|
||||
postConnectVerify(T& strm, std::string const& host)
|
||||
{
|
||||
boost::system::error_code ec;
|
||||
|
||||
if (sslVerify())
|
||||
{
|
||||
strm.set_verify_mode(boost::asio::ssl::verify_peer, ec);
|
||||
if (!ec)
|
||||
{
|
||||
strm.set_verify_callback(
|
||||
std::bind(
|
||||
&rfc2818_verify,
|
||||
host,
|
||||
std::placeholders::_1,
|
||||
std::placeholders::_2,
|
||||
j_),
|
||||
ec);
|
||||
}
|
||||
}
|
||||
|
||||
return ec;
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief callback invoked for name verification - just passes through
|
||||
* to the asio rfc2818 implementation.
|
||||
*
|
||||
* @param domain hostname expected
|
||||
* @param preverified passed by implementation
|
||||
* @param ctx passed by implementation
|
||||
* @param j journal for logging
|
||||
*/
|
||||
static bool
|
||||
rfc2818_verify(
|
||||
std::string const& domain,
|
||||
bool preverified,
|
||||
boost::asio::ssl::verify_context& ctx,
|
||||
beast::Journal j)
|
||||
{
|
||||
if (boost::asio::ssl::rfc2818_verification(domain)(preverified, ctx))
|
||||
return true;
|
||||
|
||||
JLOG(j.warn()) << "Outbound SSL connection to " << domain
|
||||
<< " fails certificate verification";
|
||||
return false;
|
||||
}
|
||||
|
||||
private:
|
||||
boost::asio::ssl::context ssl_context_;
|
||||
beast::Journal const j_;
|
||||
bool const verify_;
|
||||
};
|
||||
|
||||
} // namespace ripple
|
||||
|
||||
#endif
|
||||
43
include/xrpl/net/RegisterSSLCerts.h
Normal file
43
include/xrpl/net/RegisterSSLCerts.h
Normal file
@@ -0,0 +1,43 @@
|
||||
//------------------------------------------------------------------------------
|
||||
/*
|
||||
This file is part of rippled: https://github.com/ripple/rippled
|
||||
Copyright (c) 2016 Ripple Labs Inc.
|
||||
|
||||
Permission to use, copy, modify, and/or distribute this software for any
|
||||
purpose with or without fee is hereby granted, provided that the above
|
||||
copyright notice and this permission notice appear in all copies.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
|
||||
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
|
||||
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
|
||||
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
|
||||
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
|
||||
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
|
||||
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||
*/
|
||||
//==============================================================================
|
||||
|
||||
#ifndef RIPPLE_NET_REGISTER_SSL_CERTS_H_INCLUDED
|
||||
#define RIPPLE_NET_REGISTER_SSL_CERTS_H_INCLUDED
|
||||
|
||||
#include <xrpl/basics/Log.h>
|
||||
|
||||
#include <boost/asio/ssl/context.hpp>
|
||||
|
||||
namespace ripple {
|
||||
/** Register default SSL certificates.
|
||||
|
||||
Register the system default SSL root certificates. On linux/mac,
|
||||
this just calls asio's `set_default_verify_paths` to look in standard
|
||||
operating system locations. On windows, it uses the OS certificate
|
||||
store accessible via CryptoAPI.
|
||||
*/
|
||||
void
|
||||
registerSSLCerts(
|
||||
boost::asio::ssl::context&,
|
||||
boost::system::error_code&,
|
||||
beast::Journal j);
|
||||
|
||||
} // namespace ripple
|
||||
|
||||
#endif
|
||||
@@ -79,4 +79,3 @@ mirror the JSON test as much as possible.
|
||||
Refer to the Protocol Buffers [language
|
||||
guide](https://developers.google.com/protocol-buffers/docs/proto3)
|
||||
for more detailed information about Protocol Buffers.
|
||||
|
||||
|
||||
@@ -23,9 +23,9 @@ optional fields easier to read:
|
||||
if it exists, or nothing if it doesn't." This usage of the
|
||||
tilde/bitwise NOT operator is not standard outside of the
|
||||
`rippled` codebase.
|
||||
- As a consequence of this, `x[~sfFoo] = y[~sfFoo]`
|
||||
assigns the value of Foo from y to x, including omitting
|
||||
Foo from x if it doesn't exist in y.
|
||||
- As a consequence of this, `x[~sfFoo] = y[~sfFoo]`
|
||||
assigns the value of Foo from y to x, including omitting
|
||||
Foo from x if it doesn't exist in y.
|
||||
|
||||
Typically, for things that are guaranteed to exist, you use
|
||||
`x[sfFoo]` and avoid having to deal with a container that may
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Resource::Manager #
|
||||
# Resource::Manager
|
||||
|
||||
The ResourceManager module has these responsibilities:
|
||||
|
||||
@@ -7,7 +7,7 @@ The ResourceManager module has these responsibilities:
|
||||
- Provide an interface to share load information in a cluster.
|
||||
- Warn and/or disconnect endpoints for imposing load.
|
||||
|
||||
## Description ##
|
||||
## Description
|
||||
|
||||
To prevent monopolization of server resources or attacks on servers,
|
||||
resource consumption is monitored at each endpoint. When consumption
|
||||
@@ -33,44 +33,44 @@ Although RPC connections consume resources, they are transient and
|
||||
cannot be rate limited. It is advised not to expose RPC interfaces
|
||||
to the general public.
|
||||
|
||||
## Consumer Types ##
|
||||
## Consumer Types
|
||||
|
||||
Consumers are placed into three classifications (as identified by the
|
||||
Resource::Kind enumeration):
|
||||
|
||||
- InBound,
|
||||
- OutBound, and
|
||||
- Admin
|
||||
- InBound,
|
||||
- OutBound, and
|
||||
- Admin
|
||||
|
||||
Each caller determines for itself the classification of the Consumer it is
|
||||
creating.
|
||||
Each caller determines for itself the classification of the Consumer it is
|
||||
creating.
|
||||
|
||||
## Resource Loading ##
|
||||
## Resource Loading
|
||||
|
||||
It is expected that a client will impose a higher load on the server
|
||||
when it first connects: the client may need to catch up on transactions
|
||||
it has missed, or get trust lines, or transfer fees. The Manager must
|
||||
it has missed, or get trust lines, or transfer fees. The Manager must
|
||||
expect this initial peak load, but not allow that high load to continue
|
||||
because over the long term that would unduly stress the server.
|
||||
|
||||
If a client places a sustained high load on the server, that client
|
||||
is initially given a warning message. If that high load continues
|
||||
is initially given a warning message. If that high load continues
|
||||
the Manager may tell the heavily loaded server to drop the connection
|
||||
entirely and not allow re-connection for some amount of time.
|
||||
|
||||
Each load is monitored by capturing peaks and then decaying those peak
|
||||
values over time: this is implemented by the DecayingSample class.
|
||||
|
||||
## Gossip ##
|
||||
## Gossip
|
||||
|
||||
Each server in a cluster creates a list of IP addresses of end points
|
||||
that are imposing a significant load. This list is called Gossip, which
|
||||
is passed to other nodes in that cluster. Gossip helps individual
|
||||
that are imposing a significant load. This list is called Gossip, which
|
||||
is passed to other nodes in that cluster. Gossip helps individual
|
||||
servers in the cluster identify IP addreses that might be unduly loading
|
||||
the entire cluster. Again the recourse of the individual servers is to
|
||||
the entire cluster. Again the recourse of the individual servers is to
|
||||
drop connections to those IP addresses that occur commonly in the gossip.
|
||||
|
||||
## Access ##
|
||||
## Access
|
||||
|
||||
In rippled, the Application holds a unique instance of Resource::Manager,
|
||||
which may be retrieved by calling the method
|
||||
|
||||
Reference in New Issue
Block a user