Commit Graph

91 Commits

Author SHA1 Message Date
Mike Ellery
aea54b7230 Add RPC filters for Escrow an PayChan (RIPD-1414) 2017-03-31 12:10:48 -07:00
Howard Hinnant
1a7a6f22e2 Add 'type' param to ledger_data and ledger rpc commands (RIPD-1446):
The 'type' field allows the rpc client to specify what type of ledger
entries to retrieve. The available types are:

    "account"
    "amendments"
    "directory"
    "fee"
    "hashes"
    "offer"
    "signer_list"
    "state"
    "ticket"
2017-03-31 12:10:11 -07:00
Brad Chase
2449f9c18d Fix handleLCL consensus bug:
Consensus::checkLCL can change state_ but it was being called inside
timerEntry after a switch on the current state_.  In rare cases, this might
end up calling stateEstablish even when the state_ was open.
2017-03-31 11:54:51 -07:00
Scott Schurr
c453df927f NetworkOPs isn't stopped() until Jobs done (RIPD-1356):
A new JobCounter class is introduced.  The JobCounter keeps
a reference count of Jobs in flight to the JobQueue.  When
NetworkOPs needs to stop, in addition to other work, it calls
JobCounter::join(), which waits until all Jobs in flight
have been destroyed before returning.  This ensures that all
NetworkOPs Jobs are completed before NetworkOPs declares
itself stopped().

Also, once a JobCounter is join()ed, it refuses to produce
more counted Jobs for the JobQueue.  So, once all old Jobs
in flight are done, then NetworkOPs will add no additional
Jobs to the JobQueue.

Other classes besides NetworkOPs should also be able to use
JobCounter.  NetworkOPs is a first test case.

Also unneeded #includes were removed from files touched for
other reasons.
2017-03-21 18:55:05 -04:00
seelabs
1bb92d40aa Fix tx re-ordering bug in test:
`env.fund` requires two transactions: `pay` and `set account`. If there is a
`trust` transaction in the same set of txs, the txs may be reordered so
`pay` -> `trust` -> `set account` so the wrong `no ripple` flag would be used
on the trust line.

Adding a `close` between `env.fund` and `env.trust` resolves this problem.
2017-03-21 18:55:05 -04:00
Brad Chase
bc5a74057d Refactor consensus for simulation (RIPD-1011):
This is a substantial refactor of the consensus code and also introduces
a basic consensus simulation and testing framework.  The new generic/templated
version is in src/ripple/consensus and documents the current type requirements.
The version adapted for the RCL is in src/ripple/app/consensus.  The testing
framework is in src/test/csf.

Minor behavioral changes/fixes include:
* Adjust close time offset even when not validating.
* Remove spurious proposing_ = false call at end of handleLCL.
* Remove unused functionality provided by checkLastValidation.
* Separate open and converge time
* Don't send a bow out if we're not proposing
* Prevent consensus stopping if NetworkOPs switches to disconnect mode while
  consensus accepts a ledger
* Prevent a corner case in which Consensus::gotTxSet or Consensus::peerProposal
  has the potential to update internal state while an dispatched accept job is
  running.
* Distinguish external and internal calls to startNewRound.  Only external
  calls can reset the proposing_ state of consensus
2017-03-21 18:54:57 -04:00
Scott Schurr
9d4500cf69 Prevent low-likelihood crash on shutdown (RIPD-1392):
The DatabaseImp has threads that asynchronously call JobQueue to
perform database reads.  Formerly these threads had the same
lifespan as Database, which was until the end-of-life of
ApplicationImp.  During shutdown these threads could call JobQueue
after JobQueue had already stopped.  Or, even worse, occasionally
call JobQueue after JobQueue's destructor had run.

To avoid these shutdown conditions, Database is made a Stoppable,
with JobQueue as its parent.  When Database stops, it shuts down
its asynchronous read threads.  This prevents Database from
accessing JobQueue after JobQueue has stopped, but allows
Database to perform stores for the remainder of shutdown.

During development it was noted that the Database::close()
method was never called.  So that method is removed from Database
and all derived classes.

Stoppable is also adjusted so it can be constructed using either
a char const* or a std::string.

For those files touched for other reasons, unneeded #includes
are removed.
2017-03-20 18:08:49 -07:00
Scott Schurr
b4e765362b Remove timing window from RootStoppable (RIPD-1392):
RootStoppable was using two separate flags to identify that it
was stopping.  LoadManager was being notified when one flag was
set, but checking the other flag (not yet set) to see if we were
stopping.  There is no strong motivation for two flags.  The
timing window is closed by removing one flag and moving around
a chunk of code.
2017-03-20 17:49:16 -07:00
Mike Ellery
95aebfc38c Add timer start param to Application (RIPD 1405):
Modify doStart Application method to specify whether or not to start the
DeadlineTimers. Specify inactive timers for jtx::Env Applications and
active timers for standard Applications.
2017-03-20 16:22:26 -07:00
Edward Hennis
7265729446 TxQ full queue RPC info (RIPD-1404):
* RPC `ledger` command returns all queue entries in "queue_data"
  when requesting open ledger, and including boolean "queue: true".
  * Includes queue state. e.g.: fee_level, retries, last_result, tx.
  * Respects "expand" and "binary" parameters for the txs.
* Remove some unused code.
2017-03-20 16:18:48 -07:00
seelabs
846723d771 New rules for payment paths:
* Sanity check on newly created strands
* Better loop detection
* Better tests (test every combination of path element pairs)
* Disallow any root issuer (even for xrp)
* Disallow compount element typs in path
* Issue was not reset when currency was XRP
* Add amendment
2017-03-20 14:56:40 -07:00
Mike Ellery
80d9b0464a Add helper to modify Env configs (RIPD-1247)
Add envconfig test helper for manipulating Env config via
callables. Create new common modifiers for non-admin config,
validator config and one for using different server port values.
2017-03-20 14:38:15 -07:00
seelabs
ffa79ac6a5 Enforce rippling constraints during payments 2017-03-13 20:20:09 -04:00
wilsonianb
ffc7cf8f6c Use lower quorum for smaller validator sets 2017-03-10 16:33:24 -05:00
seelabs
f423181b94 Rename amendment featureRIPD1368 -> fix1368 2017-03-07 20:47:45 -05:00
Nik Bougalis
cfde591ac9 Add Escrow support:
Escrow replaces the existing SusPay implementation with improved
code that also adds hashlock support to escrow payments, making
RCL ILP enabled.

The new functionality is under the `Escrow` amendment, which
supersedes and replaces the `SusPay` amendment.

This commit also deprecates the `CryptoConditions` amendment
which is replaced by the `CryptoConditionSuite` amendment which,
once enabled, will allow use of cryptoconditions others than
hashlocks.
2017-03-06 14:59:32 -05:00
seelabs
35f4698aed Check for malformed public key on payment channel 2017-03-06 14:41:44 -05:00
Brad Chase
60416b18a5 Add quiet unit test reporter 2017-03-01 13:18:30 -05:00
Mike Ellery
4b0a0b0b85 Add test for transaction_entry request (RIPD-1401):
Test transaction_entry request. Remove unreachable redundant ledger
lookup check. Fix check for request against the current ledger
(disallowed).
2017-03-01 13:18:29 -05:00
Brad Chase
f1377d5d30 Publish server stream when fee changes (RIPD-1406):
Resolves #1991

Publish a server status update after every ledger close or open
ledger update if there is a change in fees.
2017-03-01 13:18:29 -05:00
seelabs
30b6e4e2e5 Do not close socket on a foreign thread:
* Closing a socket in WSClient's cleanup method was not thread safe. Force the
close to happen on the WSClient's strand.
2017-03-01 13:18:29 -05:00
Mike Ellery
9e3dadce0d Add unit test for get_counts RPC method (RIPD-1399) 2017-03-01 13:18:29 -05:00
Edward Hennis
73b4c818c5 Add more 'sign' tests:
fix #229
2017-03-01 13:18:29 -05:00
Mike Ellery
3d6a1781e7 Add tests for lookupLedger (RIPD-1268):
Cover additional input cases for lookupLedger.
2017-03-01 13:18:29 -05:00
Scott Schurr
ce9238b389 Remove beast::Thread (RIPD-1189):
All uses of beast::Thread were previously removed from the code
base, so beast::Thread is removed.  One piece of beast::Thread
needed to be preserved: the ability to set the current thread's
name.  So there's now a beast::CurrentThreadName that allows the
current thread's name to be set and returned.

Thread naming is also cleaned up a bit.  ThreadName.h and .cpp
are removed since beast::CurrentThreadName does a better job.
ThreadEntry is also removed, but its terminateHandler() is
preserved in TerminateHandler.cpp.  The revised terminateHandler()
uses beast::CurrentThreadName to recover the name of the running
thread.

Finally, the NO_LOG_UNHANDLED_EXCEPTIONS #define is removed since
it was discovered that the MacOS debugger preserves the stack
of the original throw even if the terminateHandler() rethrows.
2017-03-01 11:43:59 -05:00
seelabs
2c6b0f3193 Fix limiting step re-execute bug (RIPD-1368):
The deferred credits table can compute a balance that's different from the
ledger balance.

Syntax:
A number written with no decimal means that number exactly. I.e. "12". A number
written with a decimal means that number has a non-zero digit at the lowest
order digit. I.e. "12.XX" means a number like "12.00000000000005"

Consider the following payment:
alice (USD) -> USD/XRP -> (XRP) Bob
Alice initially has 12.XX USD in her account.
The strand is used to debit alice the following amounts:
1) Debit alice 5
2) Debit alice 0.XX
3) Debit alice 3.XX

The next time the strand is explored, alice has a USD/XRP offer on the books,
and her account is credited:

1) Credit alice 20

When the beginning of the strand is reached, consider what happens when alice is
a limiting step. Calculate how much we can get out the step. According to the
deferred credit table this is:
12.XX - (5 + 0.XX + 3.XX)

This is also limited by alice's balance, which is large thanks to the credit she
received in the book step.

Now that the step has calculated how much we can get out, throw out the
sandbox (the one with the credit), and re-execute. However, the following error
occurs. We asked for 12.XX - (5 + 0.XX + 3.XX). However, the ledger has
calculated that alice has:
((12.XX - 5) - 0.XX) - 3.XX

That's a problem, because that number is smaller. Notice that there are two
precision losing operations in the deferred credits table:
1) The 5 + 0.XX step
2) The 12.XX - (total of debits). (Notice total of debits is < 10)

However, there is only one precision losing operation in the ledger calculation:
1) (Subtotal of 12.XX-5) - 0.XX

That means the calculation for the ledger results in a number that's smaller
than the deferred credits. Flow detects this as a re-execution error.
2017-03-01 11:42:31 -05:00
wilsonianb
b4a16b165b Add validator key revocations:
Allow manifest revoking validator keys to be stored in a separate
[validator_key_revocation] config field, so the validator can run
again with new keys and token.
2017-03-01 11:41:07 -05:00
wilsonianb
a8cf5e0a5c Add validator token to config (RIPD-1386) 2017-03-01 11:41:07 -05:00
wilsonianb
2fcde0e0b6 Add SecretKey comparison operator (RIPD-1382) 2017-03-01 11:41:07 -05:00
wilsonianb
b45f45dcef Fetch validator lists from remote sites:
Validator lists from configured remote sites are fetched at a regular
interval. Fetched lists are expected to be in JSON format and contain the
following fields:

* "manifest": Base64-encoded serialization of a manifest containing the
  validator publisher's master and signing public keys.

* "blob": Base64-encoded JSON string containing a "sequence",
  "expiration" and "validators" field. "expiration" contains the Ripple
   timestamp (seconds since January 1st, 2000 (00:00 UTC)) for when the
  list expires. "validators" contains an array of objects with a
  "validation_public_key" field.

* "signature": Hex-encoded signature of the blob using the publisher's
  signing key.

* "version": 1

* "refreshInterval" (optional)
2017-03-01 11:41:07 -05:00
wilsonianb
e823e60ca0 Dynamize trusted validator list and quorum (RIPD-1220):
Instead of specifying a static list of trusted validators in the config
or validators file, the configuration can now include trusted validator
list publisher keys.

The trusted validator list and quorum are now reset each consensus
round using the latest validator lists and the list of recent
validations seen. The minimum validation quorum is now only
configurable via the command line.
2017-03-01 11:41:07 -05:00
wilsonianb
74977ab3db Consolidate parseUrl arguments into a struct 2017-03-01 11:41:07 -05:00
Edward Hennis
b514f1aae9 Config test uses unique directories for each test:
* This fixes an uncommon, but annoying, spurious failure running this
  test, particularly in release builds. This appears to be an issue with
  Windows of the FS where quickly creating and deleting the same
  directory repeatedly will eventually fail.
* RIPD-1390
2017-02-07 19:31:46 -05:00
Scott Schurr
71b42dcec5 Exercise debugLog writes in jtx unit tests (RIPD-1393) 2017-02-07 18:59:56 -05:00
seelabs
f5af8b03de Add the config preset features to the view:
It is often difficult to get access to the preset features in the config. Adding
the preset features solves this problem.
2017-02-07 18:59:56 -05:00
Mike Ellery
e01f6e7455 Use log/journal instead of std::cerr (RIPD-1377):
Change some uses of std::cerr to log or cout.
2017-02-07 18:59:56 -05:00
Mike Ellery
79149b4c0c Eliminate protocol header dependency (RIPD-1234):
Eliminate checks using sha512half, add coverage for xor and SetHex.
2017-02-01 11:42:33 -08:00
Nik Bougalis
15a30c745c Remove unused code & refactor and simplify event load timing 2017-02-01 11:42:32 -08:00
Nik Bougalis
8345475bc3 Simplify fee handling during transaction submission:
Avoid custom overflow code; simply use 128-bit math to
maintain precision and return a saturated 64-bit value
as the final result.

Disallow use of negative values in the `fee_mult_max`
and `fee_div_max` fields. This change could potentially
cause submissions with negative values that would have
previously succeeded to now fail.
2017-02-01 11:42:31 -08:00
Nik Bougalis
c7de7950c4 Correctly compare default-constructed Slice instances 2017-02-01 11:42:30 -08:00
Vinnie Falco
e05bf0844d Changes for secp256k1 2017-02-01 12:36:51 -05:00
Nik Bougalis
2c87739d6c Harden default TLS configuration (RIPD-1332, RIPD-1333, RIPD-1334):
The existing configuration includes 512 and 1024 bit DH
parameters and supports ciphers such as RC4 and 3DES and
hash algorithms like SHA-1 which are no longer considered
secure.

Going forward, use only 2048-bit DH parameters and define
a new default set of modern ciphers to use:

    HIGH:!aNULL:!MD5:!DSS:!SHA1:!3DES:!RC4:!EXPORT:!DSS

Additionally, allow administrators who wish to have different
settings to configure custom global and per-port ciphers suites
in the configuration file using the `ssl_ciphers` directive.
2017-01-17 17:19:58 -08:00
Nik Bougalis
b00b81a861 Require at least OpenSSL 1.0.1g or 1.0.2j and later (RIPD-1331) 2017-01-17 17:19:58 -08:00
Mike Ellery
be0fb67d8d Add ledger save/load test (RIPD-1378)
Provide unit test to invoke ledger load at startup.
2017-01-13 15:01:20 -08:00
Brad Chase
fb60cc9b5b Cleanup unit test support code (RIPD-1380):
* Remove `src/test/support/mao`
* Flatten `src/test/support/jtx` to `src/test/jtx`
2017-01-13 15:01:20 -08:00
Brad Chase
3c4d3b10c1 Update RPC handler role/usage (RIPD-557):
* Properly use the RPC method to determine required role for HTTP/S RPC calls.
* Charge for malformed RPC calls over HTTP/S
2017-01-13 15:01:20 -08:00
Edward Hennis
d9ef5ef98f Fix broken Intellisense (MSVC):
* MSVC Intellisense will ignore all file-level static_asserts.
2017-01-13 15:01:20 -08:00
Scott Schurr
be9c955506 Convert Workers to std::thread (RIPD-1189) 2017-01-13 15:01:20 -08:00
Edward Hennis
1989b1028f Add ledger_current_index to fee RPC result (RIPD-1300) 2017-01-13 15:01:20 -08:00
Mike Ellery
7536c53a48 Eliminate ledger data setup in test (RIPD-1372):
Change ledger-data json test fixture to simple jtx/Env setup.
2017-01-13 15:01:20 -08:00