Commit Graph

57 Commits

Author SHA1 Message Date
Edward Hennis
df60e46750 Improve deterministic transaction sorting in TxQ:
* Txs with the same fee level will sort by TxID XORed with the parent
  ledger hash.
* The TxQ is re-sorted after every ledger.
* Attempt to future-proof the TxQ tie breaking test
2022-03-23 23:28:04 -07:00
Edward Hennis
8fa33795a3 Reduce TxQ logging severity in several places 2021-12-14 17:47:48 -08:00
Edward Hennis
b1c9b134dc Make transaction queue order deterministic:
* Sort by fee level (which is the current behavior) then by transaction
  ID (hash).
* Edge case when the account at the end of the queue submits a higher
  paying transaction to walk backwards and compare against the cheapest
  transaction from a different account.
* Use std::if_any to simplify the JobQueue::isOverloaded loop.
2021-12-14 17:47:48 -08:00
Edward Hennis
ae9930b87d Consensus transaction recovery/deferral completely ignores the TxQ 2021-12-14 17:47:38 -08:00
Howard Hinnant
9f75f2d522 Replace boost::clamp with std::clamp 2021-09-13 15:13:15 -07:00
Scott Schurr
3b33318dc8 Prefer std::optional over boost:optional:
Some of the boost::optionals must remain for now.  Both
boost::beast and SOCI have interfaces that require
boost::optional.
2021-03-11 14:35:31 -08:00
seelabs
c47b4f3667 Improve canonicalization of serialized amounts:
The existing code that deserialized an STAmount was sub-optimal and performed
poorly. In some rare cases the operation could result in otherwise valid
serialized amounts overflowing during deserialization. This commit will help
detect error conditions more quickly and eliminate the problematic corner cases.
2021-01-25 18:49:49 -08:00
Edward Hennis
77ec62e9c8 Always check the sequence when adding to the transaction queue:
* If multiple transactions are queued for the account, change the
  account's sequence number in a temporary view before processing the
  transaction.
* Adds a new "at()" interface to STObject which is identical to the
  operator[], but easier to write and read when dealing with ptrs.
* Split the TxQ tests into two suites to speed up parallel run times.
2020-11-18 13:25:27 -08:00
Scott Schurr
7724cca384 Implement enhanced Ticket support:
Tickets are a mechanism to allow for the "out-of-order" execution of
transactions on the XRP Ledger.

This commit, if merged, reworks the existing support for tickets and
introduces support for 'ticket batching', completing the feature set
needed for tickets.

The code is gated under the newly-introduced `TicketBatch` amendment
and the `Tickets` amendment, which is not presently active on the
network, is being removed.

The specification for this change can be found at:
https://github.com/xrp-community/standards-drafts/issues/16
2020-09-01 08:58:57 -07:00
seelabs
328e42ad42 Minor cleanups:
* Make sure variables are always initialized
* Use lround instead of adding .5 and casting
* Remove some unneeded vars
* Check for null before calling strcmp
* Remove redundant if conditions
* Remove make_TxQ factory function
2020-05-26 18:39:24 -07:00
seelabs
6d28f2a8d9 Cleanup code using move semantics 2020-05-26 18:39:16 -07:00
Pretty Printer
50760c6935 Format first-party source according to .clang-format 2020-04-23 10:02:04 -07:00
Scott Schurr
fc0a082700 Remove STAmountSO::soTime and soTime2:
STAmount::soTime and soTime2 were time based "amendment like"
switches to control small changes in behavior for STAmount.
soTime2, which was the most recent, was dated Feb 27, 2016.
That's over 3 years ago.

The main reason to retain these soTimes would be to replay
old transactions.  The likelihood of needing to replay a
transaction from over three years ago is pretty low.  So it
makes sense to remove these soTime values.

In Flow_test the testZeroOutputStep() test is removed.  That
test started to fail when the STAmount::soTimes were removed.
I checked with the original author of the test.  He said
that the code being tested by that unit test has been removed,
so it makes sense to remove the test.  That test is removed.
2020-01-30 13:22:19 -08:00
Scott Schurr
c48be14f4f Remove comments about featureFeeEscalation enabled 19May2016 2020-01-30 13:19:48 -08:00
Devon White
cd9732b47a Change how fail_hard transactions are handled.
FIXES: #2847

* Transactions that are submitted with the fail_hard flag
  and that result in any TER code besides tesSUCCESS shall
  be neither queued nor held.

[FOLD] Keep tec results out of the open ledger when fail_hard:

* Improve TransactionStatus const correctness, and remove redundant
  `local` check
* Check open ledger tx count in fail_hard tests
* Fix some wrapping
* Remove duplicate test
2020-01-10 12:40:31 -08:00
seelabs
761bb5744e Make XRPAmount constructor explicit:
Remove the implicit conversion from int64 to XRPAmount. The motivation for this
was noticing that many calls to `to_string` with an integer parameter type were
calling the wrong `to_string` function. Since the calls were not prefixed with
`std::`, and there is no ADL to call `std::to_string`, this was converting the
int to an `XRPAmount` and calling `to_string(XRPAmount)`.

Since `to_string(XRPAmount)` did the same thing as `to_string(int)` this error
went undetected.
2020-01-08 16:10:25 -08:00
Edward Hennis
e3b5b808c5 Add units to all fee calculations:
* Uses existing XRPAmount with units for drops, and a new TaggedFee for
  fee units (LoadFeeTrack), and fee levels (TxQ).
* Resolves #2451
2020-01-08 18:44:01 -05:00
p2peer
79e9085dd1 Augment "submit" command response:
If merged, this commit will report additional information in the
response to the submit command; this will make it easier for developers
to accurately track the status of transaction submission.

Fixes #2851
2020-01-01 18:12:38 -08:00
Nik Bougalis
a3a9dc26b4 Introduce support for deletable accounts:
The XRP Ledger utilizes an account model. Unlike systems based on a UTXO
model, XRP Ledger accounts are first-class objects. This design choice
allows the XRP Ledger to offer rich functionality, including the ability
to own objects (offers, escrows, checks, signer lists) as well as other
advanced features, such as key rotation and configurable multi-signing
without needing to change a destination address.

The trade-off is that accounts must be stored on ledger. The XRP Ledger
applies reserve requirements, in XRP, to protect the shared global ledger
from growing excessively large as the result of spam or malicious usage.

Prior to this commit, accounts had been permanent objects; once created,
they could never be deleted.

This commit introduces a new amendment "DeletableAccounts" which, if
enabled, will allow account objects to be deleted by executing the new
"AccountDelete" transaction. Any funds remaining in the account will
be transferred to an account specified in the deletion transaction.

The amendment changes the mechanics of account creation; previously
a new account would have an initial sequence number of 1. Accounts
created after the amendment will have an initial sequence number that
is equal to the ledger in which the account was created.

Accounts can only be deleted if they are not associated with any
obligations (like RippleStates, Escrows, or PayChannels) and if the
current ledger sequence number exceeds the account's sequence number
by at least 256 so that, if recreated, the account can be protected
from transaction replay.
2019-10-18 16:44:16 -07:00
seelabs
7912ee6f7b Use structured bindings in some places:
Most of the new uses either:
* Replace some uses of `tie`
* bind to pairs when iterating through maps
2019-08-23 11:33:59 -07:00
seelabs
5d1728cc96 Use class template argument deduction for locks 2019-08-23 08:47:43 -07:00
seelabs
b9e73b4852 Fix shadowing variables 2019-08-23 08:47:42 -07:00
Scott Schurr
64b55c0f88 Rename JsonFields.h to jss.h:
At this point all of the jss::* names are defined in the same
file.  That file has been named JsonFields.h.  That file name
has little to do with either JsonStaticStrings (which is what
jss is short for) or with jss.  The file is renamed to jss.h
so the file name better reflects what the file contains.

All includes of that file are fixed.  A few include order
issues are tidied up along the way.
2019-04-26 11:21:52 -07:00
Edward Hennis
4c06b3f86f Validate TxQ config and expected transactions range 2019-03-04 11:45:56 -05:00
Edward Hennis
2432f13903 Reserve correct vector size for fee calculations:
* Using txnsExpected_, which is influenced by both the config
  and network behavior, can reserve far too much or far too
  little memory, wasting time and resources.
* Not an issue during normal operation, but a user could
  cause problems on their local node with extreme configuration
  settings.
2018-12-11 12:51:46 -08:00
Edward Hennis
e7a69cce65 Account for minimum reserve in potential spend:
* Relevant when deciding whether an account can queue multiple
  transactions. If the potential spend of the already queued
  transactions would dip into the reserve, the reserve is
  preserved for fees.
* Also change several direct modifications of the owner count to
  call adjustOwnerCount to preserve overflow checking.
* Update related unit testcase
* Resolves #2251
2018-11-20 19:49:39 -08:00
Edward Hennis
58f786cbb4 Make the FeeEscalation amendment permanent (RIPD-1654):
The FeeEscalation amendment has been enabled on the XRP Ledger network
since May 19, 2016. The transaction which activated this amendment is:
5B1F1E8E791A9C243DD728680F108FEF1F28F21BA3B202B8F66E7833CA71D3C3.

This change removes all conditional code based around the FeeEscalation
amendment, but leaves the amendment definition itself since removing the
definition would cause nodes to think an unknown amendment was activate
causing them to become amendment blocked.

The commit also removes the redundant precomputed hashes from the
supportedAmendments vector.
2018-11-06 10:26:29 -08:00
Edward Hennis
a96cb8fc1c Remove undocumented experimental options from RPC sign (RIPD-1653):
The `x_assume_tx` and `x_queue_okay` experimental options were
associated with the transaction queue that were not officially
supported.
2018-11-06 10:26:29 -08:00
Edward Hennis
7295cf979b Grow the open ledger expected transactions quickly (RIPD-1630):
* When increasing the expected ledger size, add on an extra 20%.
* When decreasing the expected ledger size, take the minimum of the
  validated ledger size or the old expected size, and subract another 50%.
* Update fee escalation documentation.
* Refactor the FeeMetrics object to use values from Setup
2018-10-01 11:26:22 -07:00
Edward Hennis
e14f913244 Update TxQ developer docs:
* Rename a couple of member variables for clarity.
2018-10-01 11:26:22 -07:00
Edward Hennis
16b9bbb517 Retried transactions that tec move from TxQ to open ledger:
* Unit test of tec code handling.
* Extra TxQ debug logging
2018-06-25 13:52:16 -07:00
Edward Hennis
5b733fb485 Remove Transactor::mFeeDue member variable
* mFeeDue is only used in one place by one derived class, so
  only compute it as a local in that function.
* The baseFee needs to be calculated outside of the Transactor class
  because, it can change during transaction processing, and the function
  is static, so we need to be sure to call the right version
* Rename Transactor::calculateFee to minimumFee
2018-06-19 11:25:20 -07:00
Brad Chase
681df58b61 Refactor ledger replay logic (RIPD-1547):
Also switch to use ReadView for TxQ updates.
2018-05-15 09:54:00 -04:00
Howard Hinnant
db3b4dd396 Prevent accidental aggregates
*  The compiler can provide many non-explicit constructors for
   aggregate types.  This is sometimes desired, but it can
   happen accidentally, resulting in run-time errors.

*  This commit assures that no types are aggregates unless existing
   code is using aggregate initialization.
2018-04-08 01:52:11 -07:00
Edward Hennis
1853c0d678 Add STAmount switchover support to tx queue:
* RIPD-1513
* New fix1513 amendment
2017-12-01 13:49:58 -05:00
Edward Hennis
c11e186659 Grow TxQ expected size quickly, shrink slowly (RIPD-1534):
* Stores recent history of "good" ledgers. Uses the maximum as the
  expected ledger size. When a large value drops off, use a 90%
  backoff to go down to to the new maximum.
* If consensus is unhealthy, wipe the history in addition to the current
  clamping.
* Include .md doc files in xcode and VS projects
2017-11-28 20:27:54 -05:00
Nik Bougalis
dd52bdd2c4 Merge master (0.70.2) into develop (0.80.0-rc2) 2017-09-22 17:16:17 -07:00
Edward Hennis
62127d725d Recover open ledger transactions to the queue (RIPD-1530):
* If the transaction can't be queued, recover to the open ledger once,
  and drop it on the next attempt.
* New result codes for transactions that can not queue.
* Add minimum queue size.
* Remove the obsolete and incorrect SF_RETRY flag.
* fix #2215
2017-09-21 15:02:23 -04:00
Edward Hennis
89e8ea436a Fix typo in TxQ log message 2017-09-01 14:15:18 -04:00
Edward Hennis
3a973ab719 Improve TxQ locking 2017-05-17 04:06:22 -07:00
Edward Hennis
7265729446 TxQ full queue RPC info (RIPD-1404):
* RPC `ledger` command returns all queue entries in "queue_data"
  when requesting open ledger, and including boolean "queue: true".
  * Includes queue state. e.g.: fee_level, retries, last_result, tx.
  * Respects "expand" and "binary" parameters for the txs.
* Remove some unused code.
2017-03-20 16:18:48 -07:00
seelabs
f5af8b03de Add the config preset features to the view:
It is often difficult to get access to the preset features in the config. Adding
the preset features solves this problem.
2017-02-07 18:59:56 -05:00
Edward Hennis
1989b1028f Add ledger_current_index to fee RPC result (RIPD-1300) 2017-01-13 15:01:20 -08:00
Edward Hennis
c1b8efb7af Clear queue for account with high fee tx (RIPD-1246):
* If an account has any transactions in the transaction queue, submitting
  a transaction that covers the differences to the open ledger fee level
  for prior queued transactions plus itself will cause all those
  transactions to be applied to the open ledger.
* tel failures in `TxQ::accept` will leave tx in the queue to retry later.
2016-08-15 11:09:18 -07:00
Edward Hennis
d1ce07ef5d Minor refactor of LoadFeeTrack (RIPD-956):
* Load scaling functions are free, and take `Fees`.
* Move LoadFeeTrack to app/misc.
* Update naming convention.
2016-08-02 18:05:08 -04:00
Edward Hennis
e762d09e7e Tx queue enhancements and RPC info (RIPD-1205, RIPD-1206):
* Account-related queue stats (RIPD-1205). Boolean "queue" parameter to
  account_info only if requesting the open ledger.
* Account for the TxQ when autofilling sequence in sign-and-submit (RIPD-1206)
* Tweak TxQ::accept edge case when choosing which tx to try next.
* Labels for experimental "x_" submit parameters use correct separator.

=== Release Notes ===
==== New features ====

When requesting `account_info` for the open ledger, include the `queue :
true` to get extra information about any queued transactions for this
account. (RIPD-1205).

==== Bug fixes ====

When using sign-and-submit mode to autofill a transaction's sequence
number, the logic will not reuse a sequence number that is in the queue
for this account. (RIPD-1206).

Labels for experimental "x_queue_okay" and "x_assume_tx" parameters to
`sign` and `submit` updated to use correct separator.
2016-08-02 18:03:53 -04:00
Edward Hennis
177a52473a Improve TxQ edge-case handling logic (RIPD-1200):
* HashRouter: Track relay expiration separately from item lifespan.
** Renamed `swapSet` to `shouldRelay`.
** Cleaned up `HashRouter` member names and removed unused code.
** Remove `SF_RELAYED` flag.
* Fix TxQ edge case replacing dropped transactions.
* Fix TxQ bug in maximumTxnPerAccount check.
2016-07-01 17:13:11 -04:00
Edward Hennis
7f52249e40 Change fee escalation algorithms (RIPD-1177):
* Minimum factor 256*500, don't multiply by base fee
* Change autofill fee behavior to pay the open ledger fee.
** Experimental options: x-assume-tx - assume <int> more transactions in
   the open queue when computing escalated fee, x-queue-okay - if true
   and escalated fee is over limit, try with load fee.
* Port of 75af4ed.
2016-06-06 07:24:56 -07:00
Edward Hennis
7295d7f4bb RPC fee command checks open ledger rules (RIPD-1183):
* Matches internal getMetric() to avoid races.
2016-06-06 07:10:36 -07:00
Edward Hennis
9752268308 Standalone mode uses temp DB files by default (RIPD-1129):
* If a [database_path] is configured, it will always be used, and tables
  will be upgraded on startup.
2016-05-26 12:31:46 -04:00