Commit Graph

70 Commits

Author SHA1 Message Date
Vinnie Falco
5d42604efd Refactor the structure of source files:
* New src/ripple/crypto and src/ripple/protocol directories
* Merged src/ripple/common into src/ripple/basics
* Move resource/api files up a level
* Add headers for "include what you use"
* Normalized include guards
* Renamed to JsonFields.h
* Remove obsolete files
* Remove net.h unity header
* Remove resource.h unity header
* Removed some deprecated unity includes
2014-11-07 13:40:43 -08:00
Vinnie Falco
ac0eaa912b Universal Port (RIPD-160):
This changes the behavior and configuration specification of the listening
ports that rippled uses to accept incoming connections for the supported
protocols: peer (Peer Protocol), http (JSON-RPC over HTTP), https (JSON-RPC)
over HTTPS, ws (Websockets Clients), and wss (Secure Websockets Clients).
Each listening port is now capable of handshaking in multiple protocols
specified in the configuration file (subject to some restrictions). Each
port can be configured to provide its own SSL certificate, or to use a
self-signed certificate. Ports can be configured to share settings, this
allows multiple ports to use the same certificate or values. The list of
ports is dynamic, administrators can open as few or as many ports as they
like. Authentication settings such as user/password or admin user/admin
password (for administrative commands on RPC or Websockets interfaces) can
also be specified per-port.

As the configuration file has changed significantly, administrators will
need to update their ripple.cfg files and carefully review the documentation
and new settings.

Changes:

* rippled-example.cfg updated with documentation and new example settings:
  All obsolete websocket, rpc, and peer configuration sections have been
  removed, the documentation updated, and a new documented set of example
  settings added.

* HTTP::Writer abstraction for sending HTTP server requests and responses
* HTTP::Handler handler improvements to support Universal Port
* HTTP::Handler handler supports legacy Peer protocol handshakes
* HTTP::Port uses shared_ptr<boost::asio::ssl::context>
* HTTP::PeerImp and Overlay use ssl_bundle to support Universal Port
* New JsonWriter to stream message and body through HTTP server
* ServerHandler refactored to support Universal Port and legacy peers
* ServerHandler Setup struct updated for Universal Port
* Refactor some PeerFinder members
* WSDoor and Websocket code stores and uses the HTTP::Port configuration
* Websocket autotls class receives the current secure/plain SSL setting
* Remove PeerDoor and obsolete Overlay peer accept code
* Remove obsolete RPCDoor and synchronous RPC handling code
* Remove other obsolete classes, types, and files
* Command line tool uses ServerHandler Setup for port and authorization info
* Fix handling of admin_user, admin_password in administrative commands
* Fix adminRole to check credentials for Universal Port
* Updated Overlay README.md

* Overlay sends IP:port redirects on HTTP Upgrade peer connection requests:
  Incoming peers who handshake using the HTTP Upgrade mechanism don't get
  a slot, and always get HTTP Status 503 redirect containing a JSON
  content-body with a set of alternate IP and port addresses to try, learned
  from PeerFinder. A future commit related to the Hub and Spoke feature will
  change the response to grant the peer a slot when there are peer slots
  available.

* HTTP responses to outgoing Peer connect requests parse redirect IP:ports:
  When the [overlay] configuration section (which is experimental) has
  http_handshake = 1, HTTP redirect responses will have the JSON content-body
  parsed to obtain the redirect IP:port addresses.

* Use a single io_service for HTTP::Server and Overlay:
  This is necessary to allow HTTP::Server to pass sockets to and from Overlay
  and eventually Websockets. Unfortunately Websockets is not so easily changed
  to use an externally provided io_service. This will be addressed in a future
  commit, and is one step necessary ease the restriction on ports configured
  to offer Websocket protocols in the .cfg file.
2014-11-06 16:10:00 -08:00
Vinnie Falco
35f9499b67 Fix Overlay stop on exit:
The stop sequence for Overlay had a race condition where autoconnect could
be called after close_all, resulting in a hang on exit. This resolves the
problem by putting the close and timer operations on a strand:
* Rename some Overlay members
* Put close on strand and tidy up members
* Use completion handler instead of coroutine for timer
* Use App io_service in PeerFinder
2014-11-03 14:11:05 -08:00
Vinnie Falco
3430be4075 Add PeerFinder onRedirects function 2014-10-31 13:27:55 -07:00
Tom Ritchford
5b667da526 Squelch some warnings in rippled and third-party code. 2014-10-27 10:00:03 -07:00
Tom Ritchford
5425a90f16 Fix tabs and trailing whitespace. 2014-10-15 19:39:30 -04:00
Vinnie Falco
af78ed608e Call Stoppable::stopped in PeerFinder onStop. 2014-10-15 19:37:25 -04:00
Vinnie Falco
4b5625fd59 Load PeerFinder database in Stoppable::onPrepare:
OverlayImpl::onStart calls into PeerFinder before PeerFinder::Manager::onStart,
causing tests to sometimes fail and the application to intermittently not start.
The order of calls to Stoppable::onStart is implementation defined and not
predictable.

This changes PeerFinder to load the database in Stoppable::onPrepare, before
threads are launched. In general, creation and initialization of resources that
are shared between classes should happen in onPrepare rather than onStart,
to solve this problem.
2014-10-10 19:38:52 -07:00
Vinnie Falco
7c0c2419f7 Refactor PeerFinder:
Previously, the PeerFinder manager constructed with a Callback object
provided by the owner which was used to perform operations like connecting,
disconnecting, and sending messages. This made it difficult to change the
overlay code because a single call into the PeerFinder could cause both
OverlayImpl and PeerImp to be re-entered one or more times, sometimes while
holding a recursive mutex. This change eliminates the callback by changing
PeerFinder functions to return values indicating the action the caller should
take.

As a result of this change the PeerFinder no longer needs its own dedicated
thread. OverlayImpl is changed to call into PeerFinder on a timer to perform
periodic activities. Furthermore the Checker class used to perform connectivity
checks has been refactored. It no longer uses an abstract base class, in order
to not type-erase the handler passed to async_connect (ensuring compatibility
with coroutines). To allow unit tests that don't need a network, the Logic
class is now templated on the Checker type. Currently the Manager provides its
own io_service. However, this can easily be changed so that the io_service is
provided upon construction.

Summary
* Remove unused SiteFiles dependency injection
* Remove Callback and update signatures for public APIs
* Remove obsolete functions
* Move timer to overlay
* Steps toward a shared io_service
* Templated, simplified Checker
* Tidy up Checker declaration
2014-10-10 15:04:37 -07:00
Vinnie Falco
5f59282ba1 Clean up Overlay and PeerFinder sources:
* Tidy up identifiers and declarations
* Merge PeerFinder headers into one file
* Merge handout classes and functions into one file
2014-10-10 15:04:36 -07:00
Vinnie Falco
9a0a434dd8 Fix incorrect address in connectivity check report:
The remoteAddress is the address as seen on the socket, which for
incoming connections has a random port chosen by the remote implementation
that is different from the port number used to accept connections by the
remote listening socket. The checkedAddress is the remote address as seen
on the socket, combined with the port advertised in the TMEndpoints message.
This fixes the reporting and metadata associated with addresses tested
for connectivity.

The README has been updated to reflect that uptime is no longer part of
the metadata associated with IP addresses saved for bootstrapping.
2014-10-09 14:48:54 -07:00
Tom Ritchford
0f71b4a378 Fix most compilation warnings for gcc, clang, release, debug. 2014-10-02 14:58:14 -07:00
Tom Ritchford
6069400538 Fix compiler warnings under gcc. 2014-10-01 08:47:55 -07:00
miguelportilla
f87a6ccc7a Fix missing includes for boost 1.56.0 2014-09-16 15:22:00 -07:00
Vinnie Falco
4239880acb Clean up and restructure sources 2014-09-16 14:15:12 -07:00
Nik Bougalis
56c18f7768 Cleanups and fixes (RIPD-532):
* Properly handle sfWalletLocator field
* Plug a tiny memory leak
* Avoid naked pointers
* Remove unused variables
* Other small cleanups
2014-09-04 16:51:31 -07:00
Vinnie Falco
295c8de858 Detect inconsistency in PeerFinder self-connects (RIPD-411) 2014-07-31 16:10:54 -07:00
Tom Ritchford
194304e544 Refactor RippleCalc:
* Rationalize method and filenames, move to subdirectory.
* Use Issue in Node.
* Restrict access to PathState variables.
* Line length and readability cleanups.
* New PathCursor stores path calculation data during rippleCalc.
* Extract methods PathCursor::node(), PathCursor::previousNode()
  and RippleCalc::addPath
2014-07-30 08:29:29 -07:00
Howard Hinnant
58547f6997 Tidy up hardened containers (RIPD-380):
* Rename hardened containers for clarity
* Fixes https://ripplelabs.atlassian.net/browse/RIPD-380
2014-07-28 09:06:35 -07:00
Vinnie Falco
3ff919ccf1 Fix version to say 'rippled' 2014-07-15 13:03:40 -07:00
Vinnie Falco
506910147f Tidy up includes:
* Replace boost with std equivalents:
  - bind, ref, cref, function, placeholders
* More "include what you use"
* Remove unnecessary includes
2014-06-15 18:26:50 -07:00
Vinnie Falco
560071bb68 Make all include paths relative to a root directory:
* Better include path support in the VSProject scons tool.
* Various manual fixes to include paths.
2014-06-02 09:16:28 -07:00
Vinnie Falco
b2d97703c6 beast cleanup and tidying:
* Replace custom types with C++11 <cstdint> types
* Fix sqlite integer intos and uses
* Fix String implicit integer constructors
* Escape the enclosing namespace in sqdb
* Replace contract checks with assert
* Make many header files compile independently
* Remove the dependence on beast_core.h in many places
* Remove unused or obsolete classes and functions
* Remove unused or obsolete macros
* Remove unused network functions and files
* Remove unused or obsolete classes:
  - ChildProcess
  - HighResolutionTimer
  - InterProcessLock
  - Throw
  - TrackedMutex
  - UntrackedMutex
  - XmlDocument
  - XmlElement
2014-03-23 15:23:55 -07:00
Vinnie Falco
a865149c65 New unit_test framework:
* Header-only!
* No external dependencies or other beast modules
* Compilation options allow for:
  - Stand-alone application to run a single test suite
  - Stand-alone application to run a set of test suites
  - Global suite of tests inline with the host application
  - Disable test suite generation completely
* Existing tests reworked to use the new classes
2014-03-21 18:00:37 -07:00
Vinnie Falco
e344cb3d62 beast cleaning and tidying:
* Consolidate small modules into one
* Remove Android-specific platform support
* Remove Chrono.h from beast_core.h
* Removed files:
  - TypeTraits.h
  - Utility.h
  - Net.h
  - Asio.h
* Remove zlib support:
  - Remove inline zlib library sources
  - Remove GZIPCompressorOutputStream
  - Remove GZIPDecompressorInputStream
* Remove obsolete or unused classes:
  - AbstractObject
  - BigInteger
  - BufferedInputStream
  - CacheLine
  - CPUMeter
  - DatagramSocket
  - DynamicObject
  - FileLogger
  - FPUFlags
  - Identifier
  - JSON
  - LocalisedStrings
  - MACAddress
  - MemoryAlignment
  - MemoryInputStream
  - MemoryMappedFile
  - NamedValueSet
  - OptionalScopedPointer
  - PerformanceCounter
  - PropertySet
  - ScopedTimeInterval
  - SparseSet
  - SpinDelay
  - StreamingSocket
  - StringPool
  - SubregionStream
  - Uuid
  - Variant
2014-03-21 07:30:50 -07:00
Howard Hinnant
cad50c68a8 General tidy and refactoring:
* Use nullptr (C++11) instead of NULL.
* Put each file into its own namespace declaration.
* Remove "using namespace" directives and add scope qualifiers.
* Control when beast's implementation of std::equal (C++14) is used.
* Tidy up some const declarations.

Conflicts:
	src/ripple_app/shamap/SHAMapSync.cpp
	src/ripple_app/tx/TransactionEngine.cpp
2014-03-19 13:42:01 -07:00
Vinnie Falco
c2fd1215f5 Refactor beast::asio:
* New tools for completion handlers:
  - wrap_handler provides composed io_service execution guarantees.
  - bind_handler rebinds arguments to handlers.
  - shared_handler type-erases any completion handler.
  - buffer_sequence type-erases templated BufferSequences
  - abstract_socket replaces Socket
  - socket_wrapper replaces SocketWrapper
  - beast::asio placeholders to work with std::bind

* Removed obsolete classes and functions
  - AbstractHandler
  - ComposedAsyncOperation
  - SharedFunction
  - SharedHandler
  - SharedHandlerAllocator
  - SharedHandlerPtr
  - SharedHandlerType
  - SocketBase
  - SocketWrapperStrand
  - wrapHandler

* Refactored classes to use new tools
  - abstract_socket
  - socket_wrapper
  - HandshakeDetector
  - HttpClientType

* Miscellanous tidying
  - socket classes moved to beast::asio namespace
  - beast asio files provide their own namespace declaration.
  - Fix IsCallPossible conflicting template parameter name
  - Use <boost/get_pointer.hpp> for C++11 compatibility.
  - Remove extraneous include path from build environment.
2014-03-14 08:03:48 -07:00
Nik Bougalis
8b1df06a94 PeerFinder fixes:
* Fix local advertisement (was missing)
* Fix Livecache histogram display
* If no [ips] are specified, use r.ripple.com
* Use different backing stores for PeerFinder and Validator databases
2014-03-04 15:43:32 -08:00
Nik Bougalis
73485d5a23 Resolve hostnames found in the [ips] config section 2014-03-01 11:53:57 -08:00
Nik Bougalis
28c7827f14 Calculate program file directories:
* Determine location of database files in Config
* Inject database directory or file path in PeerFinder and Validators
* PeerFinder and Validators will share the same sqlite file
2014-03-01 11:53:57 -08:00
Howard Hinnant
cd30e552a7 Fix clang warnings 2014-02-28 12:15:12 -08:00
Vinnie Falco
2957b688fd PeerFinder and Peers fixes:
- Fix bootcache incorrect failure on socket cancel
- Fix peer connect race on stop
- Simplify some loops
- Require boost 1.55 via static_assert
2014-02-28 12:03:33 -08:00
JoelKatz
955ce45448 Fix the assert on a self-connect 2014-02-24 22:30:04 -08:00
Vinnie Falco
995e64a205 PeerFinder fixes:
* Fix split horizon using recent address cache
* Change message tuning parameters to reduce dead messages
* Improved peer handout algorithm for addresses
* Improved handout algorithm for redirects
* Improved selection algorithm for autoconnect
* Faster autoconnection strategy
* Consolidate deadline timers
* Send empty endpoints message as a socket ping
* Fix hop count adjustments for live cache filtering
* Remove broken Peer::isConnected function
* Optimized Livecache for handouts
* Optimized Bootcache for handouts
* Remove uptime metric from Bootcache entries
* Add histogram to Livecache print output
2014-02-24 19:25:01 -08:00
Vinnie Falco
616a53888e Use forward_tuple to prevent needless copies 2014-02-22 21:33:01 -08:00
Vinnie Falco
a336cc26f9 Remove deprecated IPAddress 2014-02-22 21:33:00 -08:00
Vinnie Falco
5bb194cf89 PeerFinder fixes:
* Add Livecache property stream support
* Tidy up log output
* Move peer code to ripple_overlay module
* Remove or hide some Peer interfaces
* Fix asserts by removing isConnected which was not thread safe
2014-02-14 14:00:51 -08:00
Vinnie Falco
3a1a5d12de Refactor PeerFinder:
* Revise documentation in README.md
* Inject abstract_clock in Manager
* Introduce the Slot object as a replacement for Peer
* New bullet-proof method for slot accounting
* Replace Peer with Slot for tracking connections
* Prevent duplicate outbound connection attempts
* Improved connection and bootstrap business logic
* Refactor PeerImp, PeersImp private interfaces
* Give PeersImp access to the PeerImp interface
* Handle errors retrieving endpoints from asio sockets
* Use weak_ptr to manage PeerImp lifetime
* Better handling of socket closure in PeerImp
* Improve the orderly shutdown logic of PeersImp
2014-02-13 14:19:27 -08:00
Nik Bougalis
e60b28980a PeerFinder work and refactoring:
* Implement PeerFinder business logic.
* Support fixed peers (including DNS support).
* Add journal support to Peer and Peers.
* Refactor PeerDoor support.
* Tidy up Peers and eliminate connection functionality and timers.
* Refactor Peer interface and add journal support.
* Allow construction of incoming Peer using an existing socket.
* Remove TESTNET support.
* Allow connections from/to cluster peers without consuming slots
* Misc. cleanups.
2014-02-05 15:56:04 -08:00
Vinnie Falco
9b61a83721 Refactor, tidy up:
* Fix for msvc std::function return types
* Convert macros to constants
* Add Journal to PeerSet and use in InboundLedger
* Break lines and add annotations
* Remove some warnings
2014-01-21 10:28:33 -05:00
Vinnie Falco
087301933a General refactoring, using C++11
* Remove broken RecycledObjectPool

* Fix beast::ServiceQueue using List instead of LockFreeStack

* Add class semaphore, fixes broken Semaphore

* Move crytpo module files to new beast directory

* Use c++11 replacements for boost and beast types:
  - std::atomic instead of beast::Atomic
  - std::function instead of boost::function, beast::function
  - std::unique_ptr instead of beast::ScopedPointer
  - std::shared_ptr instead of boost::shared_ptr

* Remove modules:
  - beast_db
  - beast_crypto
  - beast_extras

* Remove unnecessary classes:
  - AbstractFifo
  - AddConst
  - AtomicCounter
  - AtomicFlag
  - AtomicPointer
  - AtomicState
  - CopyConst
  - Expression
  - ForwardList
  - IfCond
  - Interval
  - IntrusiveArray
  - KeyvaDB
  - PointerToOther
  - PointerTraits
  - RemoveConst
  - RemoveConstVolatile
  - RemoveReference
  - RemoveVolatile
  - SharedObjectArray
  - SingleThreadedSharedObject
  - SophiaDB factory
  - SortedSet
  - WeakReference
  - beast::unique_ptr
2014-01-07 15:57:45 -08:00
NATTSiM
27b771e2ba Fix clang compile 2013-12-27 17:46:51 -08:00
Vinnie Falco
9e519af887 Add missing README and TODO for all new modules 2013-11-15 11:27:26 -08:00
Vinnie Falco
a05f33f6a7 Add annotation and clean up whitespace 2013-11-12 19:30:07 -08:00
Vinnie Falco
d4d6acdf68 Add MSVC Output window Journal config setting
Conflicts:
	src/ripple/peerfinder/impl/Manager.cpp
	src/ripple/validators/impl/Manager.cpp
2013-11-10 15:06:05 -08:00
Vinnie Falco
63aa7284c4 Disable peerfinder hooks 2013-11-07 09:59:28 -08:00
Vinnie Falco
6d06cb29df Block until thread exits in manager dtors 2013-11-01 15:48:37 -07:00
Vinnie Falco
a39fa8ae5f New SiteFiles for fetching and managing ripple.txt files 2013-10-31 08:10:06 -07:00
Nik Bougalis
42b841735e PeerFinder work 2013-10-29 20:52:57 -07:00
Vinnie Falco
e710bd2183 Use IPAddressConversion 2013-10-29 20:52:57 -07:00