Commit Graph

302 Commits

Author SHA1 Message Date
Mark Travis
002893f280 Apply transaction batches in periodic intervals (#4504)
Add new transaction submission API field, "sync", which
determines behavior of the server while submitting transactions:
1) sync (default): Process transactions in a batch immediately,
   and return only once the transaction has been processed.
2) async: Put transaction into the batch for the next processing
   interval and return immediately.
3) wait: Put transaction into the batch for the next processing
   interval and return only after it is processed.
2023-09-11 15:47:40 -07:00
Howard Hinnant
36cb5f90e2 Introduce replacement for getting and setting thread name: (#4312)
* In namespace ripple, introduces get_name function that takes a
  std:🧵:native_handle_type and returns a std::string.
* In namespace ripple, introduces get_name function that takes a
  std::thread or std::jthread and returns a std::string.
* In namespace ripple::this_thread, introduces get_name function
  that takes no parameters and returns the name of the current
  thread as a std::string.
* In namespace ripple::this_thread, introduces set_name function
  that takes a std::string_view and sets the name of the current
  thread.
* Intended to replace the beast utilities setCurrentThreadName
  and getCurrentThreadName.
2023-09-07 11:44:36 -07:00
Elliot Lee
d943c58b3d Revert "Apply transaction batches in periodic intervals (#4504)"
This reverts commit b580049ec0.
2023-08-30 15:46:12 -07:00
Mark Travis
b580049ec0 Apply transaction batches in periodic intervals (#4504)
Add new transaction submission API field, "sync", which
determines behavior of the server while submitting transactions:
1) sync (default): Process transactions in a batch immediately,
   and return only once the transaction has been processed.
2) async: Put transaction into the batch for the next processing
   interval and return immediately.
3) wait: Put transaction into the batch for the next processing
   interval and return only after it is processed.
2023-08-21 16:15:31 -07:00
Ikko Eltociear Ashimine
213491d94b refactor: fix typo in FeeUnits.h (#4644)
covert -> convert
2023-08-21 16:15:31 -07:00
John Freeman
cb09e61d2f Update dependencies (#4595)
Use the most recent versions in ConanCenter.

* Due to a bug in Clang 16, you may get a compile error:
  "call to 'async_teardown' is ambiguous"
  * A compiler flag workaround is documented in `BUILD.md`.
* At this time, building this with gcc 13 may require editing some files
  in `.conan/data`
  * A patch to support gcc13 may be added in a later PR.

---------

Co-authored-by: Scott Schurr <scott@ripple.com>
2023-07-13 13:25:08 -04:00
Chenna Keshava B S
c6fee28b92 refactor: change the return type of mulDiv to std::optional (#4243)
- Previously, mulDiv had `std::pair<bool, uint64_t>` as the output type.
  - This is an error-prone interface as it is easy to ignore when
    overflow occurs.
- Using a return type of `std::optional` should decrease the likelihood
  of ignoring overflow.
  - It also allows for the use of optional::value_or() as a way to
    explicitly recover from overflow.
- Include limits.h header file preprocessing directive in order to
  satisfy gcc's numeric_limits incomplete_type requirement.

Fix #3495

---------

Co-authored-by: John Freeman <jfreeman08@gmail.com>
2023-07-05 20:11:19 -07:00
Scott Determan
c9a586c243 Add missing includes for gcc 13.1: (#4555)
gcc 13.1 failed to compile due to missing headers. This patch adds the
needed headers.
2023-06-05 15:50:03 -07:00
Scott Determan
f709311762 Fix unaligned load and stores: (#4528) (#4531)
Misaligned load and store operations are supported by both Intel and ARM
CPUs. However, in C++, these operations are undefined behavior (UB).
Substituting these operations with a `memcpy` fixes this UB. The
compiled assembly code is equivalent to the original, so there is no
performance penalty to using memcpy.

For context: The unaligned load and store operations fixed here were
originally introduced in the slab allocator (#4218).
2023-05-31 13:28:33 -07:00
Scott Determan
ce997a6de8 Ensure that switchover vars are initialized before use: (#4527)
Global variables in different TUs are initialized in an undefined order.
At least one global variable was accessing a global switchover variable.
This caused the switchover variable to be accessed in an uninitialized
state.

Since the switchover is always explicitly set before transaction
processing, this bug can not effect transaction processing, but could
effect unit tests (and potentially the value of some global variables).
Note: at the time of this patch the offending bug is not yet in
production.
2023-05-22 19:36:46 -07:00
Nik Bougalis
b7f588b789 Introduce support for a slabbed allocator: (#4218)
When instantiating a large amount of fixed-sized objects on the heap
the overhead that dynamic memory allocation APIs impose will quickly
become significant.

In some cases, allocating a large amount of memory at once and using
a slabbing allocator to carve the large block into fixed-sized units
that are used to service requests for memory out will help to reduce
memory fragmentation significantly and, potentially, improve overall
performance.

This commit introduces a new `SlabAllocator<>` class that exposes an
API that is _similar_ to the C++ concept of an `Allocator` but it is
not meant to be a general-purpose allocator.

It should not be used unless profiling and analysis of specific memory
allocation patterns indicates that the additional complexity introduced
will improve the performance of the system overall, and subsequent
profiling proves it.

A helper class, `SlabAllocatorSet<>` simplifies handling of variably
sized objects that benefit from slab allocations.

This commit incorporates improvements suggested by Greg Popovitch
(@greg7mdp).

Commit 1 of 3 in #4218.
2023-04-10 14:22:59 -07:00
drlongle
8bfdbcbab5 Add logging for exceptions: (#4400)
Log exception messages at several locations.

Previously, these were locations where an exception was caught, but the
exception message was not logged. Logging the exception messages can be
useful for analysis or debugging. The additional logging could have a
small negative performance impact.

Fix #3213.
2023-03-30 10:13:30 -07:00
Scott Schurr
dffcdea12b fix: Expected to return a value: (#4401)
Fix a case where `ripple::Expected` returned a json array, not a value.

The problem was that `Expected` invoked the wrong constructor for the
expected type, which resulted in a constructor that took multiple
arguments being interpreted as an array.

A proposed fix was provided by @godexsoft, which involved a minor
adjustment to three constructors that replaces the use of curly braces
with parentheses. This makes `Expected` usable for
[Clio](https://github.com/XRPLF/clio).

A unit test is also included to ensure that the issue doesn't occur
again in the future.
2023-03-23 17:32:17 -07:00
drlongle
84cde3ce0b Use <=> operator for base_uint, Issue, and Book: (#4411)
- Implement the `operator==` and the `operator<=>` (aka the spaceship
  operator) in `base_uint`, `Issue`, and `Book`. 
- C++20-compliant compilers automatically provide the remaining
  comparison operators (e.g. `operator<`, `operator<=`, ...).
- Remove the function compare() because it is no longer needed.
- Maintain the same semantics as the existing code.
- Add some unit tests to gain further confidence.
- Fix #2525.
2023-03-14 20:54:54 -07:00
Howard Hinnant
5edaec2bd0 Introduce min/max observers for Number
Three static member functions are introduced with
definitions consistent with std::numeric_limits:

static constexpr Number min() noexcept;

  Returns: The minimum positive value.  This is the value closest to zero.

static constexpr Number max() noexcept;

  Returns: The maximum possible value.

static constexpr Number lowest() noexcept;

  Returns: The negative value which is less than all other values.
2023-02-07 15:43:28 -08:00
Howard Hinnant
2f1f453052 Optimize uint128_t division by 10 within Number.cpp
* Optimization includes computing remainder from division.
* Used only within Number::operator*=.
2023-02-07 15:43:28 -08:00
Howard Hinnant
6eaaa7bcfa Replace Number division algorithm
* Replace division with faster algorithm.
* Correct some rounding bugs in multiplication.
* Add tests for rounding bugs.
2023-02-07 15:43:28 -08:00
Howard Hinnant
e354497f63 Include rounding mode in XRPAmount to STAmount conversion. 2023-02-07 15:43:28 -08:00
Howard Hinnant
6fcd654bee Remove undefined behavior
* Taking the negative of a signed negative is UB, but
  taking the negative of an unsigned is not.
2023-02-07 15:43:28 -08:00
Howard Hinnant
d275a2ab72 Silence warnings 2023-02-07 15:43:28 -08:00
Howard Hinnant
3f33471220 Introduce rounding modes for Number:
You can set a thread-local flag to direct Number how to round
non-exact results with the syntax:

    Number::rounding_mode prev_mode = Number::setround(Number::towards_zero);

This flag will stay in effect for this thread only until another call
to setround.  The previously set rounding mode is returned.

You can also retrieve the current rounding mode with:

    Number::rounding_mode current_mode = Number::getround();

The available rounding modes are:

* to_nearest : Rounds to nearest representable value.  On tie, rounds
               to even.
* towards_zero : Rounds towards zero.
* downward : Rounds towards negative infinity.
* upward : Rounds towards positive infinity.

The default rounding mode is to_nearest.
2023-02-07 15:43:28 -08:00
Howard Hinnant
a82ad5ba76 Use Number for IOUAmount and STAmount arithmetic
* Guarded by amendment fixUniversalNumber
* Produces slightly better accuracy in some computations.
2023-02-07 15:43:28 -08:00
Howard Hinnant
48e804c40c Add tests 2023-02-07 15:43:28 -08:00
Howard Hinnant
c9c54c9799 Add implicit conversion from STAmount to Number 2023-02-07 15:43:28 -08:00
Howard Hinnant
24fe5f9fd0 Add clip
* Return 0 if abs(x) < limit, else returns x
2023-02-07 15:43:28 -08:00
Howard Hinnant
476ee8a479 Add conversions between Number, XRPAmount and int64_t
* Conversions to Number are implicit
* Conversions away from Number are explicit and potentially lossy
* If lossy, round to nearest, and to even on tie
2023-02-07 15:43:28 -08:00
Howard Hinnant
0ee63b7c7b AMM Add Number class and associated algorithms 2023-02-07 15:43:28 -08:00
Ed Hennis
e4b17d1cf2 XRPFees: Fee setting and handling improvements (#4247)
* Introduces amendment `XRPFees`
* Convert fee voting and protocol messages to use XRPAmounts
* Includes Validations, Change transactions, the "Fees" ledger object,
  and subscription messages

* Improve handling of 0 drop reference fee with TxQ. For use with networks that do not want to require fees
* Note that fee escalation logic is still in place, which may cause the
  open ledger fee to rise if the network is busy. 0 drop transactions
  will still queue, and fee escalation can be effectively disabled by
  modifying the configuration on all nodes

* Change default network reserves to match Mainnet

* Name the new SFields *Drops (not *XRP)
* Reserve SField IDs for Hooks

* Clarify comments explaining the ttFEE transaction field validation
2023-02-02 16:20:35 -08:00
Scott Determan
518fb6d208 Improve move semantics in Expected (#4326)
* Improve move semantics in Expected:

This patch unconditionally moves an `Unexpected<U>` value parameter as
long as `U` is not a reference. If `U` is a reference the code should
not compile. An error type that holds a reference is a strange use-case,
and an overload is not provided. If it is required in the future it can
be added.

The `Expected(U r)` overload should take a forwarding ref.

* Replace enable_if with concepts in Expected
2022-11-28 14:01:54 -08:00
greg7mdp
3726f8bf31 Release TaggedCache object memory outside the lock 2022-10-14 12:54:59 -07:00
manojsdoshi
d33df35378 Merge master (1.9.4) into develop (1.10.0-b2) 2022-10-11 16:34:36 -07:00
seelabs
9a31f321cd Allow gcc 12 compilation:
Compiling with gcc 12 on manjaro (arch variant) had compilation errors
without adding an additional include file.
2022-09-14 14:16:40 -07:00
Nik Bougalis
0ecfc7cb1a Improve self-signed certificate generation:
When starting, the code generates a new ephemeral private key and
a corresponding certificate to go along with it. This process can
take time and, while this is unlikely to matter for normal server
operations, it can have a significant impact for unit testing and
development. Profiling data suggests that ~20% of the time needed
for a unit test run can be attributed to this.

This commit does several things:

1. It restructures the code so that a new self-signed certificate
   and its corresponding private key are only initialized once at
   startup; this has minimal impact on the operation of a regular
   server.
2. It provides new default DH parameters. This doesn't impact the
   security of the connection, but those who compile from scratch
   can generate new parameters if they so choose.
3. It properly sets the version number in the certificate, fixing
   issue #4007; thanks to @donovanhide for the report.
4. It uses SHA-256 instead of SHA-1 as the hash algorithm for the
   certificate and adds some X.509 extensions as well as a random
   128-bit serial number.
5. It rounds the certificate's "start of validity" period so that
   the server's precise startup time cannot be easily deduced and
   limits the validity period to two years, down from ten years.
6. It removes some CBC-based ciphers from the default cipher list
   to avoid some potential security issues, such as CVE-2016-2107
   and CVE-2013-0169.
2022-08-25 08:49:17 -07:00
Crypto Brad Garlinghouse
83ac141f65 Remove charUnHex 2022-08-25 08:49:04 -07:00
seelabs
92d35e54c7 Switch from C++17 to C++20 2022-08-25 08:38:28 -07:00
Nik Bougalis
7e46f5342b Correct a technical flaw with the spinlock locking:
The existing spinlock code, used to protect SHAMapInnerNode
child lists, has a mistake that can allow the same child to
be repeatedly locked under some circumstances.

The bug was in the `SpinBitLock::lock` loop condition check
and would result in the loop terminating early.

This commit fixes this and further simplifies the lock loop
making the correctness of the code easier to verify without
sacrificing performance.

It also promotes the spinlock class from an implementation
detail to a more general purpose, easier to use lock class
with clearer semantics. Two different lock types now allow
developers to easily grab either a single spinlock from an
a group of spinlocks (packed in an unsigned integer) or to
grab all of the spinlocks at once.

While this commit makes spinlocks more widely available to
developers, they are rarely the best tool for the job. Use
them judiciously and only after careful consideration.
2022-07-17 22:17:34 -07:00
Nik Bougalis
9eb303f8e8 Improve STVector256 deserialization 2022-07-17 22:17:33 -07:00
seelabs
f55913dcee Add support for clang's ThreadSafetyAnalysis 2022-07-17 22:17:24 -07:00
Chenna Keshava B S
d632f9f6c8 Properly handle incorrect port numbers in parseURL (fixes #4200) 2022-07-11 09:56:39 -07:00
natenichols
d0a813a19d Adding missing include <cassert> in beast/rngfill.h to fix rippled build 2022-05-19 15:02:28 -07:00
Nik Bougalis
70779f6850 Introduce NFT support (XLS020) 2022-04-06 13:29:48 -07:00
JoelKatz
3eb8aa8b80 Negative cache support for node store
Adds support to TaggedCache to support smart replacement
(Needed to avoid race conditions with negative caching.)

Create a "hotDUMMY" object that represents the absence
of an object.

Allow DatabaseNodeImp::asyncFetch to complete immediately
if object is in cache (positive or negative).

Fix a bug in asyncFetch where the object returned may not
be the correct canonical version because we stash the
object in the results array before we canonicalize it.
2022-03-30 15:16:12 -07:00
Edward Hennis
e7e672c3f8 Incremental improvements to path finding memory usage:
* Abort background path finding when closed or disconnected
* Exit pathfinding job thread if there are no requests left
* Don't bother creating the path find job if there are no requests
* Refactor to remove circular dependency between InfoSub and PathRequest
2022-03-23 23:28:04 -07:00
Michael Legleux
a01cadbfd5 Move Beast & fix #include paths 2022-03-04 14:16:59 -08:00
Edward Hennis
0623a40f02 Refactor to fix levelization:
* Remove Application & Database dependency in PerfLog. Replace it with
  a callback passed into the constructor.
* Fixes the circular dependency between ripple/nodestore and ripple/basics
2022-03-01 14:32:14 -08:00
Nik Bougalis
416ce35d73 Detect CVE-2021-3520 when decompressing using LZ4 2022-01-10 15:29:43 -08:00
Mark Travis
7c12f01358 Parallel ledger loader & I/O performance improvements:
- Only duplicate records from archive to writable during online_delete.
- Log duration of nodestore reads.
- Include nodestore counters in perf_log output.
- Remove gratuitous nodestore activity counting.
- Report initial sync duration in server_info and perfLog.
- Report state_accounting in perfLog.
- Make state_accounting durations more accurate.
- Parallel ledger loader.
- Config parameter to load ledgers on start.
2022-01-10 15:29:21 -08:00
Mark Travis
19018e8959 Introduce partitioned unordered maps:
This commit implements partitioned unordered maps and makes it possible
to traverse such a map in parallel, allowing for more efficient use of
CPU resources.

The `CachedSLEs`, `TaggedCache`, and `KeyCache` classes make use of the
new functionality, which should improve performance.
2021-11-18 14:38:35 -05:00
Scott Schurr
c50d166c23 Add ripple::Expected which simulates std::expected:
Also integrates use of ripple::Expected into the code base.
2021-09-13 15:13:15 -07:00
CJ Cobb
3fb60a89a3 Validate data size before calling uint256::fromVoid 2021-09-13 15:13:15 -07:00