Commit Graph

126 Commits

Author SHA1 Message Date
Nik Bougalis
5def79e93c Track counts of SHAMap-related objects 2020-12-04 12:45:12 -08:00
Nik Bougalis
f072469409 Simplify & modernize code:
- Simplify and consolidate code for parsing hex input.
- Replace beast::endian::order with boost::endian::order.
- Simplify CountedObject code.
- Remove pre-C++17 workarounds in favor of C++17 based solutions.
- Improve `base_uint` and simplify its hex-parsing interface by
  consolidating the `SexHex` and `SetHexExact` methods into one
  API: `parseHex` which forces callers to verify the result of
  the operation; as a result some public-facing API endpoints
  may now return errors when passed values that were previously
  accepted.
- Remove the simple fallback implementations of SHA2 and RIPEMD
  introduced to reduce our dependency on OpenSSL. The code is
  slow and rarely, if ever, exercised and we rely on OpenSSL
  functionality for Boost.ASIO as well.
2020-12-04 12:45:12 -08:00
Nik Bougalis
4a444f7d60 Simplify SHAMapTreeNode APIs:
- Provide separate functions for serializing depending on whether
  one wants a "wire" version of a node, or one suitable for hashing.
- Remove unused functions
2020-12-04 12:45:11 -08:00
Nik Bougalis
ab77444fa3 Simplify SHAMapNodeID:
The existing SHAMapNodeID object has both a valid and an invalid state
and requirs callers to verify the state of an instance prior to using
it. A simple set of changes removes that restriction and ensures that
all instances are valid, making the code more robust.

This change also:

1. Introduces a new function to construct a SHAMapNodeID from a
   serialized blob; and
2. Reduces the amount of constructors the class exposes.
2020-12-04 12:45:11 -08:00
Howard Hinnant
a3915fa5c4 Switch to boost::shared_mutex:
* Avoid std::shared_mutex bug on ubuntu 18.04
* Fixes #3663
2020-12-04 12:44:19 -08:00
Miguel Portilla
03c809371a Add Shard pool management 2020-10-14 11:17:44 -07:00
seelabs
1129110be3 Document unused SHAMap constructor parameter 2020-09-17 15:05:15 -07:00
Miguel Portilla
ab4102a632 Improve shard finalize memory utilization 2020-09-01 16:39:00 -07:00
Howard Hinnant
4ae2f06be4 Remove unused nullConstSHAMapItem
This commit fixes #3550.
2020-09-01 16:39:00 -07:00
Miguel Portilla
86e8f2e232 Add Shard Family 2020-06-30 08:52:18 -07:00
Nik Bougalis
362a017eee Cleanup SHAMap and simplify interfaces:
* Improve error reporting (more readable exception messages)
* Reduce function complexity (split oversized function to smaller pieces)
* Reduce code duplication
* Reduce buffer copying
2020-06-25 19:46:43 -07:00
seelabs
6d28f2a8d9 Cleanup code using move semantics 2020-05-26 18:39:16 -07:00
Nik Bougalis
dbee3f01b7 Clean up and modernize code:
This commit removes obsolete comments, dead or no longer useful
code, and workarounds for several issues that were present in older
compilers that we no longer support.

Specifically:

- It improves the transaction metadata handling class, simplifying
  its use and making it less error-prone.
- It reduces the footprint of the Serializer class by consolidating
  code and leveraging templates.
- It cleanups the ST* class hierarchy, removing dead code, improving
  and consolidating code to reduce complexity and code duplication.
- It shores up the handling of currency codes and the conversation
  between 160-bit currency codes and their string representation.
- It migrates beast::secure_erase to the ripple namespace and uses
  a call to OpenSSL_cleanse instead of the custom implementation.
2020-05-05 16:05:22 -07:00
Pretty Printer
50760c6935 Format first-party source according to .clang-format 2020-04-23 10:02:04 -07:00
Nik Bougalis
bdd22e4d51 Improve reporting of missing node exceptions 2020-04-14 19:42:50 -07:00
Howard Hinnant
f4c56cbd53 Update SHAMap Documentation 2020-04-07 16:25:09 -07:00
Howard Hinnant
f22fcb3b2a Rename canonicalize into two functions:
* canonicalize_replace_cache
* canonicalize_replace_client

Now it is clear at the call site that if there are
duplicate copies of the data between the cache and
the caller, which copy gets replaced.

Additionally data parameter is now const-correct.
If it is not going to be replaced (canonicalize_replace_cache),
then the shared_ptr to the client data is const.
2020-04-07 16:25:09 -07:00
Miguel Portilla
cc452dfa9b Improve shard concurrency:
* Reduce lock scope on all public functions
* Use TaskQueue to process shard finalization in separate thread
* Store shard last ledger hash and other info in backend
* Use temp SQLite DB versus control file when acquiring
* Remove boost serialization from cmake files
2020-04-06 17:22:23 -07:00
Nik Bougalis
16c659b22c Simplify HashPrefix by converting to an enum 2020-03-05 14:10:20 -05:00
Mike Ellery
82484e26f5 Add option to enable -Wextra for gcc/clang. 2019-10-15 12:01:37 -07:00
Howard Hinnant
7228b2e068 Remove SHAMap V2 2019-09-27 12:24:19 -07:00
seelabs
7912ee6f7b Use structured bindings in some places:
Most of the new uses either:
* Replace some uses of `tie`
* bind to pairs when iterating through maps
2019-08-23 11:33:59 -07:00
seelabs
5d1728cc96 Use class template argument deduction for locks 2019-08-23 08:47:43 -07:00
Howard Hinnant
c699864c85 Fix incorrect snapShot unsharing
This fixes #3020.
2019-08-19 06:58:50 -07:00
Howard Hinnant
76d5ecb595 Verify invariants when calling SHAMapInnerNodeV2::addRaw (RIPD-1700) 2018-12-28 13:32:09 -08:00
Nik Bougalis
1682fe3a39 Cleanup unused Beast bits and pieces:
This cleanup does not remove Boost.Beast code, but old-style Beast
which is no longer relevant or helpful.
2018-11-06 10:27:10 -08:00
Howard Hinnant
d257d1b2c9 Migrate more code into the chrono type system:
Changes include:

  *  Database::tune and all tune overrides
  *  TaggedCache TargetAge
  *  KeyCache TargetAge
2018-07-20 08:58:04 -07:00
Joe Loser
70d9d88cda Remove using namespace beast in base_uint.h 2018-07-20 08:58:04 -07:00
Mike Ellery
deb9e4ce3c Remove BeastConfig.h (RIPD-1167) 2018-04-08 01:52:12 -07:00
Miguel Portilla
718d217158 Implement Shards 2018-01-17 13:43:54 -08:00
Miguel Portilla
aeda2430cd Normalize SHAMap visit functions 2018-01-17 11:41:43 -08:00
Howard Hinnant
3757829f8e Convert more timing code to <chrono>
* LoadMonitor
* Entry
* Import
2017-12-01 14:15:05 -05:00
Nikolaos D. Bougalis
f0e1024ad6 Explicitly use std::deque for the missing node handler stack:
We need to ensure that pointers and/or references to existing
elements will not be invalidated during the course of element
insertion and removal.

Containers like std::vector do not offer this guarantee, and
cannot be used as the underlying container for the stack.

By choosing to explicitly specify std::deque as the underlying
cotnainer, we avoid:

- the unlikely possibility of the C++ standards committee
  changing the default template parameter to a container;
- the more likely possibility of an accidental change by
  a programmer, without fully considering the consequences.
2017-11-28 20:31:39 -05:00
David Schwartz
09a1d1a593 Improve getMissingNodes:
* Clean up and refactor
* Resume parents of nodes read asynchronously
* Resume at tip of new stack if exhausted prior stack
* No need to restart at root
2017-03-20 14:25:38 -07:00
Brad Chase
b6a01ea41c Move support test code to src/test/support (RIPD-1313) 2016-12-23 20:39:02 -05:00
Howard Hinnant
2fd0540ed4 Recognize ripplerpc 2.0 requests and respond in kind:
* Force jtx to request/receive the 2.0 API
* Force the JSON and WebSocket tests to use 2.0 API
*  This specifically allows the Websocket to create 2.0 json/ripple
   and get back a 2.0 response.
* Add test for malformed json2
* Add check for parse failure
* Add check for params to be in array form.
* Correct type-o discovered in tests due to stricter checking.
* Add API version to the WSClient & JSONRPCClient test
* Update source.dox with more headers
2016-12-23 14:36:10 -08:00
Nik Bougalis
44167a6bcb Cleanup code identified in code review 2016-11-18 12:17:30 -08:00
Miguel Portilla
19258cf980 Remove application dependency from SyncFilters 2016-10-27 12:19:26 -04:00
JoelKatz
ed02b0717e Snapshotting an unbacked SHAMap should yield an unbacked SHAMap 2016-10-17 15:01:24 -04:00
Brad Chase
8f97889176 Don't include unit test sources in code coverage (RIPD-1132):
Most files containing unit test code are moved to
src/test. JTx and the test client code are not yet moved.
2016-09-02 15:26:16 -04:00
Mike Ellery
0d803e0fa2 Refactor uses of strCopy (RIPD-1256)
Replace the sparsely used strCopy function with Slice. Change some of
the SHAMap interface to use Slice instead of Blob, which should
eliminate a copy.
2016-08-15 11:09:45 -07:00
JoelKatz
4758050444 SHAMap fixups:
* Change state_ to Invalid if inner node types mismatch
* flushDirty before unsharing to ensure modified nodes get written
* Remove unnecessary code
* Fix updateHash for V2 inner nodes
* In descend, call isInconsistentNode only if node is found
* getMissingNodes could request duplicates in some cases
* An invalid node with the right hash is useful, it proves the map invalid
2016-08-02 13:25:48 -07:00
Miguel Portilla
b343b0468a Use lock_guard over unique_lock 2016-07-29 17:57:49 -07:00
Vinnie Falco
e39316882e Add BEAST_EXPECT, replace calls to expect 2016-07-29 17:46:56 -07:00
Howard Hinnant
d88b63d4c8 Complete SHAMap V2 cutover 2016-06-28 14:53:44 -04:00
Nik Bougalis
39829a09cb Update test logging text for change in newlines 2016-05-25 15:05:32 -04:00
Edward Hennis
7e3dbce3d2 Access base_uint through public members (RIPD-898):
* Updates many (but probably not all) locations that access base_uint
  private storage.
* More calls to access base_uint through members.
* Use an iterator to write Serializer collections.
2016-05-24 10:37:01 -04:00
Howard Hinnant
1c3ee48146 Install SHAMapInnerNodeV2
* Inner node optimization.
2016-05-10 17:38:35 -04:00
Vinnie Falco
4469ff4b9a Reorganize source files 2016-04-20 12:01:25 -04:00
Vinnie Falco
203739f7a4 Remove unused or obsolete classes and files 2016-04-20 05:16:32 -04:00