Commit Graph

69 Commits

Author SHA1 Message Date
Nik Bougalis
67c666b033 Clean up LedgerEntrySet and TransactionEngine:
* Reduce public interfaces
* Remove wrapper functions
* Remove freeze timed cutover code
* Return results directly instead of via ref parameters
2015-04-23 16:47:19 -04:00
Miguel Portilla
4d0ed3d857 RPC account_objects (RIPD-777)
General RPC command that can retrieve objects in the account root.
  * Add account objects integration test.
  * Support tickets.

* Add removeElement in Json::Value
2015-04-20 15:54:09 -07:00
Nik Bougalis
cefeaceef0 Signal error for incorrect configuration during unit test 2015-04-13 10:24:47 -07:00
Nik Bougalis
b2a9c79de5 Fix transaction enumeration in account_tx (RIPD-734):
In some corner cases, an incorrect resume marker could be
returned, preventing the complete enumeration of account
transactions.

* Robust markers via improved paging support
* New unit tests
* Cleanup
2015-04-10 19:00:22 -07:00
seelabs
7e25a3a942 Fix SQL in online delete cleanup:
* SQL statement is corrected to perform an implicit JOIN
* Add unit test
2015-03-31 21:50:45 -04:00
JoelKatz
9a0fa79144 Fix duplication of full below cache and tree node cache 2015-03-31 21:50:43 -04:00
Miguel Portilla
5e70db651d Improved local tx error messages (RIPD-720)
Failed local built transactions report the specific error.
2015-03-31 21:50:39 -04:00
David Schwartz
1fedede771 Remove transaction set acquire logic from consensus object
This creates a new InboundTransactions object that handles transaction sets,
removing this responsibility from the consensus object. The main benefit is
that many inbound transaction operations no longer require the master lock.

Improve logic to decide which peers to query, when to add more peers, and
when to re-query existing peers.
2015-03-31 21:50:38 -04:00
Howard Hinnant
1979846e5e Change several uses of std::list to alternative containers:
*  Performance motivated.
*  Several of these called size() which is O(N) in gcc-4.8.
*  Remove container copy from LedgerConsensusImp::playbackProposals().
*  Addresses RIPD-284.
2015-03-26 12:38:33 -04:00
Howard Hinnant
a61ffab3f9 Remove unnecessary allocation/deallocation from masterLock
* Add make_lock.
* Rename Application::LockType to Application::MutexType:
* Rename getMasterLock to getMasterMutex.
* Use getMasterMutex and make_lock.
* Remove unused code.
2015-03-26 12:38:33 -04:00
Edward Hennis
6e3f07ddce Remove unused / redundant functions. 2015-03-19 07:41:57 -07:00
JoelKatz
1c2f5d60a5 Subscribe/Unsubscribe improvements:
* Don't acquire the master lock where it's not needed
* InfoSub tracks RT and validated accounts separately
* Correctly remove accounts from the InfoSub
2015-03-19 07:41:57 -07:00
David Schwartz
60a7abcef6 Decongest the master lock:
* Reduce scope of lock in ledger accept
* Remove duplicate tracking of transaction sets
* Need master lock to secure ledger sequencing
2015-03-19 07:41:57 -07:00
JoelKatz
ff7dc0b446 Reduce chatty log outputs 2015-03-19 07:41:57 -07:00
seelabs
97623d20c5 Use soci in more places:
* Validator, peerfinder, SHAMapStore,
  RpcDB, TxnDB, LedgerDB, WalletDB use soci backend.
2015-03-18 19:39:26 -07:00
seelabs
d0ef2f7dd8 Use soci in some places:
* Brings the soci subtree into rippled.
* Validator, peerfinder, and SHAMapStore use new soci backend.
* Optional postgresql backend for soci (if POSTGRESQL_ROOT env var is set).
2015-03-18 19:37:08 -07:00
seelabs
c7cfd23580 Update sqlite3 to 3.8.8.2. 2015-03-18 19:37:03 -07:00
seelabs
6b9e842ddd Replaces StringPairArray with Section in Config. 2015-03-16 20:54:13 -04:00
Howard Hinnant
ec1e6b9385 Cleanup and simplifications to SHAMap:
SHAMapTreeNode
* Remove SHAMapTreeNode::pointer and SHAMapTreeNode::ref.
* Add std includes necessary to make the header standalone.
* Remove implementation from the SHAMapTreeNode declaration.
* Make clear what part of SHAMapTreeNode is:
  1) Truly public.
  2) Used only by SHAMap.
  3) Truly private to SHAMapTreeNode.

SHAMapItem
* Remove SHAMapItem::pointer and SHAMapItem::ref.
* Add std includes necessary to make the header standalone.
* Remove implementation from the SHAMapItem declaration.
* Make clear what part of SHAMapItem is:
  1) Truly public.
  2) Used only by SHAMapTreeNode.
  3) Truly private to SHAMapItem.

SHAMapSyncFilter
* Add override for SHAMapSyncFilter-derived functions.
* Add missing header.
* Default the destructor and delete the SHAMapSyncFilter copy members.

SHAMapNodeID
* Remove unused mHash member.
* Remove unused std::hash and boost::hash specializations.
* Remove unused constructor.
* Remove unused comparison with uint256.
* Remove unused getNodeID (int depth, uint256 const& hash).
* Remove virtual specifier from getString().
* Fix operator<= and operator>=.
* Document what API is used outside of SHAMap.
* Move inline definitions outside of the class declaration.

SHAMapMissingNode
* Make SHAMapType a enum class to prevent unwanted conversions.
* Remove needless ~SHAMapMissingNode() declaration/definition.
* Add referenced std includes.

SHAMapAddNode
* Make SHAMapAddNode (int good, int bad, int duplicate) ctor private.
* Move all member function definitions out of the class declaration.
* Remove dependence on beast::lexicalCastThrow.
* Make getGood() const.
* Make get() const.
* Add #include <string>.

SHAMap
* Remove unused enum STATE_MAP_BUCKETS.
* Remove unused getCountedObjectName().
* Remove SHAMap::pointer
* Remove SHAMap::ref
* Remove unused fetchPackEntry_t.
* Remove inline member function definitions from class declaration.
* Remove unused getTrustedPath.
* Remove unused getPath.
* Remove unused visitLeavesInternal.
* Make SHAMapState an enum class.
* Explicitly delete SHAMap copy members.
* Reduce access to nested types as much as possible.
* Normalize member data names to one style.

* Change last of the typedefs to usings under shamap.
* Reorder some includes ripple-first, beast-second.
* Declare a few constructions from make_shared with auto.
* Mark those SHAMap member functions which can be, with const.

* Add missing includes
2015-02-23 11:44:57 -08:00
Tom Ritchford
315a8b6b60 Use jss for many Json fields.
* Document JsonFields.
  * Remove some unused JsonFields values.
2015-02-23 14:36:34 -05:00
Vinnie Falco
8bda9487c6 Add shamap::Family injection bundle 2015-02-18 13:31:18 -05:00
seelabs
617d84c0ef BasicConfig support for legacy values:
* A legacy value is a config section with a single-line.
* These values may be read from the BasicConfig interface so
  the deprecated Config class does not need to be exposed to
  clients.
* Made Config class more testable.
2015-02-18 13:31:18 -05:00
Vinnie Falco
ab14123aed Remove obsolete classes:
Legacy workarounds for Visual Studio non thread-safe initialization
of function local objects with static storage duration are removed:

* Remove LeakChecked
* Remove StaticObject
* Remove SharedSingleton
2015-02-11 20:42:38 -05:00
Tom Ritchford
c3ae4da83a Fix include guards in rippled. 2015-02-02 17:01:17 -08:00
Vinnie Falco
bb4127a6fb Refactor Serializer and SerializerIterator interfaces:
* Remove unused members
* SerialIter holds only a pointer and offset now
* Use free functions for some Serializer members
* Use SerialIter in some places instead of Serializer
2015-01-28 16:34:33 -08:00
Miguel Portilla
5d6ea3d75f Combine history_ledger_index and online_delete (RIPD-774) 2015-01-28 16:34:33 -08:00
Mark Travis
43873b1b2c Initialize canDelete_ properly in the constructor. 2015-01-28 16:34:33 -08:00
Mark Travis
9430f3665b Speed up access to ledger index value that can be deleted. 2015-01-28 16:34:33 -08:00
Howard Hinnant
1e0efaffe8 Add missing includes. 2015-01-27 18:21:54 -08:00
Nik Bougalis
47593730d6 Modernize code:
* Clean STBase-derived class creation interfaces
* Annotate overriden STBase virtual functions
* Optimize path deserialization
* Prefer range-based for
* Prefer std::unique_ptr
* Remove BOOST_FOREACH
2015-01-26 19:13:40 -08:00
Scott Determan
44450bf644 Enable Amendments from config file or static data (RIPD-746):
* The rippled.cfg file has a new section called "amendments"

* Each line in this section contains two white-space separated items
** The first item is the ID of the amendment (a 256-bit hash)
** The second item is the friendly name

* Replaces config section name macros with variables

* Make addKnown arguments safer

* Added lock to addKnown
2015-01-26 19:13:02 -08:00
Miguel Portilla
a9dfb33126 Log abnormal close time offsets (RIPD-572) 2015-01-21 10:48:31 -08:00
Nik Bougalis
1af79f7960 Properly validate the configured online delete interval 2015-01-20 16:45:03 -08:00
Vinnie Falco
df54b47cd0 Tidy up includes and add modules to the classic build:
An alternative to the unity build, the classic build compiles each
translation unit individually. This adds more modules to the classic build:

* Remove unity header app.h
* Add missing includes as needed
* Remove obsolete NodeStore backend code
* Add app/, core/, crypto/, json/, net/, overlay/, peerfinder/ to classic build
2015-01-05 13:35:57 -08:00
Vinnie Falco
2e595830b3 Levelize SHAMap:
The SHAMap class is refactored into a separate module where each translation
unit compiles separate without errors. Dependencies on higher level business
logic are removed. SHAMap now depends only on basics, crypto, nodestore,
and protocol:

* Inject NodeStore::Database& to SHAMap
* Move sync filter instances to app/ledger/
* Move shamap to its own module
* Move FullBelowCache to shamap/
* Move private code to shamap/impl/
* Refactor SHAMap treatment of missing node handler
* Inject and use Journal for logging in SHAMap
2015-01-05 11:46:11 -08:00
Vinnie Falco
96fbcc9a5a Refactor NodeStore:
Manager is changed to be a Meyer's singleton, with factories automatically
registering upon construction.
2015-01-05 11:46:11 -08:00
JoelKatz
4f2d93bb65 Avoid processing transactions if we need a network ledger
Not processing tranasctions without a network ledger makes
initial network synchronization faster.
2014-12-31 01:55:10 -08:00
Nik Bougalis
b328ec2462 Prevent passing of non-POD types to POD-only interfaces:
This tidies up the code that produces random numbers to conform
to programming best practices and reduce dependencies.

* Use std::random_device instead of platform-specific code
* Remove RandomNumbers class and use free functions instead
2014-12-22 15:19:48 -08:00
Vinnie Falco
60f27178b8 Levelization, improve structure of source files:
Source files are moved between modules, includes changed and added,
and some code rewritten, with the goal of reducing cross-module dependencies
and eliminating cycles in the dependency graph of classes.

* Remove RippleAddress dependency in CKey_test
* ByteOrder.h, Blob.h, and strHex.h are moved to basics/. This makes
  the basics/ module fully independent of other ripple sources.
* types/ is merged into protocol/. The protocol module now contains
  all primitive types specific to the Ripple protocol.
* Move ErrorCodes to protocol/
* Move base_uint to basics/
* Move Base58 to crypto/
* Remove dependence on Serializer in GenerateDeterministicKey
* Eliminate unity header json.h
* Remove obsolete unity headers
* Remove unnecessary includes
2014-12-22 10:23:49 -08:00
Nik Bougalis
28b70a7b9a Remove 'Proof of Work' code 2014-12-19 11:00:29 -08:00
Tom Ritchford
fce77c9372 Configuration for yielding RPC server. 2014-12-19 11:00:28 -08:00
Tom Ritchford
1cbcc7be21 Allow the Ledger to generically output to both Json models. 2014-12-19 11:00:27 -08:00
Nik Bougalis
4a49fefdd9 Various cleanups:
* Replace SYSTEM_NAME and other macros with C++ constructs
* Remove RIPPLE_ARRAYSIZE and use std::extent or ranged for loops
* Remove old-style, unused offer crossing unit test
* Make STAmount::saFromRate free and remove default argument
2014-12-12 20:14:02 -08:00
Mark Travis
d5c3f0c9cf Stability bugfixes for online delete SHAMapStore:
The correct ledger age is necessary for checking health
status, and the previous behavior caused the online deletion process to
abort if the process took too long.

The tuning parameter added and the parameter whose default was modified both
minimize impact of SQL DELETE operations by decreasing the default batch size
for deletes and for increasing the backoff period between deletion batches.
These parameters decrease contention for the SQLite and I/O with the trade-off
of longer processing time for online delete. Online-delete is not a
time-critical function, so a little slowness in wall-clock time is not harmful.
2014-12-08 14:54:03 -08:00
Tom Ritchford
86df482842 Make sure that handlers always return Json::objectValue. 2014-12-01 17:16:24 -05:00
Vinnie Falco
55637f7508 Template abstract_clock on Clock:
The abstract_clock is now templated on a type meeting the requirements of
the Clock concept. It inherits the nested types of the Clock on which it
is based. This resolves a problem with the original design which broke the
type-safety of time_point from different abstract clocks.
2014-11-25 19:19:56 -08:00
Mark Travis
02529a0fc2 SHAMapStore Online Delete (RIPD-415):
Makes rippled configurable to support deletion of all data in its key-value
store (nodestore) and ledger and transaction SQLite databases based on
validated ledger sequence numbers. All records from a specified ledger
and forward shall remain available in the key-value store and SQLite, and
all data prior to that specific ledger may be deleted.

Additionally, the administrator may require that an RPC command be
executed to enable deletion. This is to align data deletion with local
policy.
2014-11-25 11:44:02 -08:00
Vinnie Falco
d4fd5e4fce HTTP Handshaking for Peers on Universal Port (RIPD-446):
This introduces a considerable change in the way that peers handshake. Instead
of sending the TMHello protocol message, the peer making the connection (client
role) sends an HTTP Upgrade request along with some special headers. The peer
acting in the server role sends an HTTP response completing the upgrade and
transition to RTXP (Ripple Transaction Protocol, a.k.a. peer protocol). If the
server has no available slots, then it sends a 503 Service Unavailable HTTP
response with a JSON content-body containing IP addresses of other servers to
try. The information that was previously contained in the TMHello message is
now communicated in the HTTP request and HTTP response including the secure
cookie to prevent man in the middle attacks. This information is documented
in the overlay README.md file.

To prevent disruption on the network, the handshake feature is rolled out in
two parts. This is part 1, where new servents acting in the client role will
send the old style TMHello handshake, and new servents acting in the server
role can automatically detect and accept both the old style TMHello handshake,
or the HTTP request accordingly. This detection happens in the Server module,
which supports the universal port. An experimental .cfg setting allows clients
to instead send HTTP handshakes when establishing peer connections. When this
code has reached a significant fraction of the network, these clients will be
able to establish a connection to the Ripple network using HTTP handshakes.

These changes clean up the handling of the socket for peers. It fixes a long
standing bug in the graceful close sequence, where remaining data such as the
IP addresses of other servers to try, did not get sent. Redundant state
variables for the peer are removed and the treatment of completion handlers is
streamlined. The treatment of SSL short reads and secure shutdown is also fixed.

Logging for the peers in the overlay module are divided into two partitions:
"Peer" and "Protocol". The Peer partition records activity taking place on the
socket while the Protocol partition informs about RTXP specific actions such as
transaction relay, fetch packs, and consensus rounds. The severity on the log
partitions may be adjusted independently to diagnose problems. Every log
message for peers is prefixed with a small, unique integer id in brackets,
to accurately associate log messages with peers.

HTTP handshaking is the first step in implementing the Hub and Spoke feature,
which transforms the network from a homogeneous network where all peers are
the same, into a structured network where peers with above average capabilities
in their ability to process ledgers and transactions self-assemble to form a
backbone of high powered machines which in turn serve a much larger number of
'leaves' with lower capacities with a goal to improve the number of
transactions that may be retired over time.
2014-11-21 16:47:12 -08:00
Vinnie Falco
08a81a0ab9 Tidy up the structure of sources in protocol/:
Split out and rename STValidation
Split out and rename STBlob
Split out and rename STAccount
Split out STPathSet
Split STVector256 and move UintTypes to protocol/
Rename to STBase
Rename to STLedgerEntry
Rename to SOTemplate
Rename to STTx
Remove obsolete AgedHistory
Remove types.h and add missing includes
Remove unnecessary includes in app.h
Remove unnecessary includes in app.h
Remove include app.h from app1.cpp
2014-11-20 20:15:29 -08:00
Nik Bougalis
31110c7fd9 Cleanup ripple::Ledger:
* Convert static member functions to free functions
* Adopt consistent naming convention
* De-inline code
2014-11-20 20:15:29 -08:00