Replace Taker.cpp with calls to the payment flow() code.
This change required a number of tweaks in the payment flow code.
These tweaks are conditionalized on whether or not offer crossing
is taking place. The flag is explicitly passed as a parameter to
the flow code.
For testing, a class was added that identifies differences in the
contents of two PaymentSandboxes. That code may be reusable in
the future.
None of the Taker offer crossing code is removed. Both versions
of the code are co-resident to support an amendment cut-over.
The code that identifies differences between Taker and Flow offer
crossing is enabled by a feature. That makes it easy to enable
or disable difference logging by changing the config file. This
approach models what was done with the payment flow code. The
differencing code should never be enabled on a production server.
Extensive offer crossing unit tests are added to examine and
verify the behavior of corner cases. The tests are currently
configured to run against both Taker and Flow offer crossing.
This gives us confidence that most cases run identically and
some of the (few) differences in behavior are documented.
Add new functionality to enforce one or more sanity checks (invariants)
on transactions. Add tests for each new invariant check. Allow
for easily adding additional invariant checks in the future.
Also Resolves
-------------
- RIPD-1426
- RIPD-1427
- RIPD-1428
- RIPD-1429
- RIPD-1430
- RIPD-1431
- RIPD-1432
Release Notes
-------------
Creates a new ammendment named "EnforceInvariants" which must be
enabled in order for these new checks to run on each transaction.
The ledger_request RPC call, under some conditions, did not
actually check that the entire ledger was present in the
database, making it unsuitable for use in cases where the
database was believed to be incorrect or incomplete.
With this change, the full ledger will be checked for
integrity unless it has already recently been checked
(according to the InboundLedgers cache).
This is a substantial refactor of the consensus code and also introduces
a basic consensus simulation and testing framework. The new generic/templated
version is in src/ripple/consensus and documents the current type requirements.
The version adapted for the RCL is in src/ripple/app/consensus. The testing
framework is in src/test/csf.
Minor behavioral changes/fixes include:
* Adjust close time offset even when not validating.
* Remove spurious proposing_ = false call at end of handleLCL.
* Remove unused functionality provided by checkLastValidation.
* Separate open and converge time
* Don't send a bow out if we're not proposing
* Prevent consensus stopping if NetworkOPs switches to disconnect mode while
consensus accepts a ledger
* Prevent a corner case in which Consensus::gotTxSet or Consensus::peerProposal
has the potential to update internal state while an dispatched accept job is
running.
* Distinguish external and internal calls to startNewRound. Only external
calls can reset the proposing_ state of consensus
The DatabaseImp has threads that asynchronously call JobQueue to
perform database reads. Formerly these threads had the same
lifespan as Database, which was until the end-of-life of
ApplicationImp. During shutdown these threads could call JobQueue
after JobQueue had already stopped. Or, even worse, occasionally
call JobQueue after JobQueue's destructor had run.
To avoid these shutdown conditions, Database is made a Stoppable,
with JobQueue as its parent. When Database stops, it shuts down
its asynchronous read threads. This prevents Database from
accessing JobQueue after JobQueue has stopped, but allows
Database to perform stores for the remainder of shutdown.
During development it was noted that the Database::close()
method was never called. So that method is removed from Database
and all derived classes.
Stoppable is also adjusted so it can be constructed using either
a char const* or a std::string.
For those files touched for other reasons, unneeded #includes
are removed.
Modify doStart Application method to specify whether or not to start the
DeadlineTimers. Specify inactive timers for jtx::Env Applications and
active timers for standard Applications.
* Sanity check on newly created strands
* Better loop detection
* Better tests (test every combination of path element pairs)
* Disallow any root issuer (even for xrp)
* Disallow compount element typs in path
* Issue was not reset when currency was XRP
* Add amendment
Escrow replaces the existing SusPay implementation with improved
code that also adds hashlock support to escrow payments, making
RCL ILP enabled.
The new functionality is under the `Escrow` amendment, which
supersedes and replaces the `SusPay` amendment.
This commit also deprecates the `CryptoConditions` amendment
which is replaced by the `CryptoConditionSuite` amendment which,
once enabled, will allow use of cryptoconditions others than
hashlocks.
All uses of beast::Thread were previously removed from the code
base, so beast::Thread is removed. One piece of beast::Thread
needed to be preserved: the ability to set the current thread's
name. So there's now a beast::CurrentThreadName that allows the
current thread's name to be set and returned.
Thread naming is also cleaned up a bit. ThreadName.h and .cpp
are removed since beast::CurrentThreadName does a better job.
ThreadEntry is also removed, but its terminateHandler() is
preserved in TerminateHandler.cpp. The revised terminateHandler()
uses beast::CurrentThreadName to recover the name of the running
thread.
Finally, the NO_LOG_UNHANDLED_EXCEPTIONS #define is removed since
it was discovered that the MacOS debugger preserves the stack
of the original throw even if the terminateHandler() rethrows.
The deferred credits table can compute a balance that's different from the
ledger balance.
Syntax:
A number written with no decimal means that number exactly. I.e. "12". A number
written with a decimal means that number has a non-zero digit at the lowest
order digit. I.e. "12.XX" means a number like "12.00000000000005"
Consider the following payment:
alice (USD) -> USD/XRP -> (XRP) Bob
Alice initially has 12.XX USD in her account.
The strand is used to debit alice the following amounts:
1) Debit alice 5
2) Debit alice 0.XX
3) Debit alice 3.XX
The next time the strand is explored, alice has a USD/XRP offer on the books,
and her account is credited:
1) Credit alice 20
When the beginning of the strand is reached, consider what happens when alice is
a limiting step. Calculate how much we can get out the step. According to the
deferred credit table this is:
12.XX - (5 + 0.XX + 3.XX)
This is also limited by alice's balance, which is large thanks to the credit she
received in the book step.
Now that the step has calculated how much we can get out, throw out the
sandbox (the one with the credit), and re-execute. However, the following error
occurs. We asked for 12.XX - (5 + 0.XX + 3.XX). However, the ledger has
calculated that alice has:
((12.XX - 5) - 0.XX) - 3.XX
That's a problem, because that number is smaller. Notice that there are two
precision losing operations in the deferred credits table:
1) The 5 + 0.XX step
2) The 12.XX - (total of debits). (Notice total of debits is < 10)
However, there is only one precision losing operation in the ledger calculation:
1) (Subtotal of 12.XX-5) - 0.XX
That means the calculation for the ledger results in a number that's smaller
than the deferred credits. Flow detects this as a re-execution error.
Allow manifest revoking validator keys to be stored in a separate
[validator_key_revocation] config field, so the validator can run
again with new keys and token.
Validator lists from configured remote sites are fetched at a regular
interval. Fetched lists are expected to be in JSON format and contain the
following fields:
* "manifest": Base64-encoded serialization of a manifest containing the
validator publisher's master and signing public keys.
* "blob": Base64-encoded JSON string containing a "sequence",
"expiration" and "validators" field. "expiration" contains the Ripple
timestamp (seconds since January 1st, 2000 (00:00 UTC)) for when the
list expires. "validators" contains an array of objects with a
"validation_public_key" field.
* "signature": Hex-encoded signature of the blob using the publisher's
signing key.
* "version": 1
* "refreshInterval" (optional)
Instead of specifying a static list of trusted validators in the config
or validators file, the configuration can now include trusted validator
list publisher keys.
The trusted validator list and quorum are now reset each consensus
round using the latest validator lists and the list of recent
validations seen. The minimum validation quorum is now only
configurable via the command line.
When started with "--start", put all known, non-vetoed
amendments in the genesis ledger. This avoids the need
to wait 256 ledgers before amendments are enabled when
testing with a fresh ledger.
Add an amendment to allow gateways to set a "tick size"
for assets they issue. There are no changes unless the
amendment is enabled (since the tick size option cannot
be set).
With the amendment enabled:
AccountSet transactions may set a "TickSize" parameter.
Legal values are 0 and 3-15 inclusive. Zero removes the
setting. 3-15 allow that many decimal digits of precision
in the pricing of offers for assets issued by this account.
For asset pairs with XRP, the tick size imposed, if any,
is the tick size of the issuer of the non-XRP asset. For
asset pairs without XRP, the tick size imposed, if any,
is the smaller of the two issuer's configured tick sizes.
The tick size is imposed by rounding the offer quality
down to nearest tick and recomputing the non-critical
side of the offer. For a buy, the amount offered is
rounded down. For a sell, the amount charged is rounded up.
Gateways must enable a TickSize on their account for this
feature to benefit them.
The primary expected benefit is the elimination of bots
fighting over the tip of the order book. This means:
- Quicker price discovery as outpricing someone by a
microscopic amount is made impossible. Currently
bots can spend hours outbidding each other with no
significant price movement.
- A reduction in offer creation and cancellation spam.
- More offers left on the books as priority means
something when you can't outbid by a microscopic amount.
A conditional suspended payment is a suspended payment where
completion of the payment is contingent upon the fulfillment
of a condition defined by the sender during creation of the
suspended payment.
This commit also introduces the "CryptoConditions" amendment
which controls whether cryptoconditions will be supported
in suspended payments. The existing "SusPay" amendment can
be used to enable suspended payments without enabling the
cryptoconditions code.
* Remove extraneous passing of transaction set hashes
* Remove recentPositions_. InboundTXs does the job now
* Move responsibility for sending "have TX set" out of consensus
The XRPEndpointStep bypassed the logic in deferred credits and
incorrectly counted funds acquired during a payment as available for
use in the payment. It also incorrectly used the current ownerCount when
calculating the reserve instead of the owner count as it was at the
beginning of the payment (reducing the owner count is analogous to
acquiring funds during a payment.)
Payment channels permit off-ledger checkpoints of XRP payments flowing
in a single direction. A channel sequesters the owner's XRP in its own
ledger entry. The owner can authorize the recipient to claim up to a
give balance by giving the receiver a signed message (off-ledger). The
recipient can use this signed message to claim any unpaid balance while
the channel remains open. The owner can top off the line as needed. If
the channel has not paid out all its funds, the owner must wait out a
delay to close the channel to give the recipient a chance to supply any
claims. The recipient can close the channel at any time. Any transaction
that touches the channel after the expiration time will close the
channel. The total amount paid increases monotonically as newer claims
are issued. When the channel is closed any remaining balance is returned
to the owner. Channels are intended to permit intermittent off-ledger
settlement of ILP trust lines as balances get substantial. For
bidirectional channels, a payment channel can be used in each direction.
cmake support in rippled. Currently supports:
* unity/nounity debug/release
* running protobuf
* sanitizer builds
* optional release build with assert turned on
* `target` variable to easily set compiler/debug/unity
(i.e. -Dtarget=gcc.debug.nounity)
* gcc/clang/visual studio/xcode
* linux/mac/win
* gcc 4 ABI, when needed
* ninja builds
* check openssl for acceptably recent release
* static builds
TBD:
* jemalloc support
* count
Notes:
* Use the -G"Visual Studio 14 2015 Win64" generator on Windows. Without
this a 32-bit project will be created. There is no way to set the
generator or force a 64-bit build in CMakeLists.txt (setting
CMAKE_GENERATOR_PLATFORM won't work). The best solution may be to
wrap cmake with a script.
* It is not possible to generate a visual studio project on linux or
mac. The visual studio generator is only available on windows.
* The visual studio project can be _either_ unity or
non-unity (selected at generation time). It does not appear possible
to disable compilation based on configuration.
* Language is _much_ worse than python, poor documentation and "quirky"
language support (for example, generator expressions can only be used
in limited contexts and seem to work differently based on
context (set_property can set multiple values, add_compile_options
can not/or is buggy)
* Could not call out to `sed` because cmake messed with the regular
expression before calling the external command. I did not see a way
around this.
* Makefile generators want to be single target. It wants a separate
directory for each target type. I saw some mentions on the web for
ways around this bug haven't look into it. The visual studio project
does support debug/release configurations in the same project (but
not unity/non-unity).
Log thread name and exception type on unhandled exceptions and use a
terminate handler to get a stack trace that includes the function that
thows the exception.
