Commit Graph

2035 Commits

Author SHA1 Message Date
Scott Schurr
eaff9a0e6a Set version to 0.90.0-b4 2018-01-17 17:19:23 -08:00
Howard Hinnant
0ec66b3dbc Limit nesting of json commands 2018-01-17 15:33:15 -08:00
Miguel Portilla
718d217158 Implement Shards 2018-01-17 13:43:54 -08:00
Miguel Portilla
aeda2430cd Normalize SHAMap visit functions 2018-01-17 11:41:43 -08:00
Brad Chase
819ea46bf0 Add RangeSet serialization 2018-01-17 11:21:53 -08:00
Scott Schurr
2d5ddbf1bf Checks (RIPD-1487):
Introduce a new ledger type: ltCHECK
Introduce three new transactions that operate on checks:

- "CheckCreate" which adds the check entry to the ledger.  The
  check is a promise from the source of the check that the
  destination of the check may cash the check and receive up to
  the SendMax specified on the check.  The check may have an
  expiration, after which the check may no longer be cashed.

- "CheckCash" is a request by the destination of the check to
  transfer a requested amount of funds, up to the check's SendMax,
  from the source to the destination.  The destination may receive
  less than the SendMax due to transfer fees.

  When cashing a check, the destination specifies the smallest
  amount of funds that will be acceptable.  If the transfer
  completes and delivers the requested amount, then the check is
  considered cashed and removed from the ledger.  If enough funds
  cannot be delivered, then the transaction fails and the check
  remains in the ledger.

  Attempting to cash the check after its expiration will fail.

- "CheckCancel" removes the check from the ledger without
  transferring funds.  Either the check's source or destination
  can cancel the check at any time.  After a check has expired,
  any account can cancel the check.

Facilities related to checks are on the "Checks" amendment.
2018-01-17 10:00:20 -08:00
Mark Travis
76ad06ef47 Control transaction dispatch rate:
Do not process a transaction received from a peer if it has
been processed within the past ten seconds.

Increase the number of transaction handlers that can be in
flight in the job queue and decrease the relative cost for
peers to share transaction and ledger data.

Additionally, make better use of resources by adjusting the
number of threads we initialize, by reverting commit
68b8ffdb63.

Performance counter modifications:
  * Create and display counters to track:
    1) Pending transaction limit overruns.
    2) Total peer disconnections.
    3) Peers disconnections due to resource consumption.

Avoid a potential double-free in Json library.
2018-01-17 09:23:29 -08:00
Nikolaos D. Bougalis
49b5c42e85 Set version to 0.90.0-b3 2018-01-12 13:42:28 -08:00
Scott Schurr
cc0ce7163a PayChan and Escrow should ignore DisallowXRP (RIPD-1462) 2018-01-10 00:12:23 -08:00
Scott Schurr
259394029a Support for lsfDepositAuth (RIPD-1487):
The DepositAuth feature allows an account to require that
it signs for any funds that are deposited to the account.
For the time being this limits the account to accepting
only XRP, although there are plans to allow IOU payments
in the future.

The lsfDepositAuth protections are not extended to offers.
If an account creates an offer it is in effect saying, “I
will accept funds from anyone who takes this offer.”
Therefore, the typical user of the lsfDepositAuth flag
will choose never to create any offers.  But they can if
they so choose.

The DepositAuth feature leaves a small gap in its
protections.  An XRP payment is allowed to a destination
account with the lsfDepositAuth flag set if:

- The Destination XRP balance is less than or equal to
  the base reserve and

- The value of the XRP Payment is less than or equal to
  the base reserve.

This exception is intended to make it impossible for an
account to wedge itself by spending all of its XRP on fees
and leave itself unable to pay the fee to get more XRP.

This commit

- adds featureDepositAuth,

- adds the lsfDepositAuth flag,

- adds support for lsfDepositAuth in SetAccount.cpp

- adds support in Payment.cpp for rejecting payments that
  don't meet the lsfDepositAuth requirements,

- adds unit tests for Payment transactions to an an account
  with lsfDepositAuth set.

- adds Escrow and PayChan support for lsfDepositAuth along
  with as unit tests.
2018-01-10 00:12:23 -08:00
seelabs
a307d2d03f Allow channel_verify to specify public key in hex (RIPD-1467) 2018-01-10 00:12:23 -08:00
seelabs
ad4ba44394 Add validated field to account_channels command (RIPD-1466) 2018-01-10 00:12:23 -08:00
Brad Chase
44e5e8bccf Make Journal assignable 2018-01-10 00:12:23 -08:00
Brad Chase
edf58820cf Cleanly report invalid [server] settings (RIPD-1562) 2018-01-10 00:12:23 -08:00
Howard Hinnant
7ff6d34a49 Force json commands to be objects
* Null json values can be objects or arrays.
* json arrays are now interpreted as batch commands.
* json objects are single commands.
* null jsons are ambiguous as to whether they are single or batch
  commands and should be avoided.
2018-01-09 16:29:39 -08:00
Nikolaos D. Bougalis
e3499b5df8 Merge master (0.81.0) into develop (0.90.0-b2) 2018-01-09 15:13:08 -08:00
Nikolaos D. Bougalis
4e8c8deeaa Set version to 0.81.0 2018-01-04 05:19:57 -08:00
Mark Travis
5e4dac41a7 Optimize SQL queries used in handling account_tx:
Profiling and research indicates that the SQLite query planner executed
our existing SQL queries sub-optimally by not using the index efficiently.

Restructuring the SQL query works around this issue and allows queries
to be executed efficiently and without unnecessary delay.
2018-01-04 04:41:00 -08:00
Brad Chase
dc9e9f498a Set version to 0.90.0-b2 2017-12-18 14:38:12 -05:00
Brad Chase
0d3ed84864 Update LZ4 support 2017-12-18 14:38:06 -05:00
Brad Chase
7d6c2229ab Update snappy support file 2017-12-18 14:37:42 -05:00
JoelKatz
80050c110f RocksDB support updates:
* Update unity build for RocksDB changes
* Log RocksDB options on startup
* Support RocksDB option strings
* Support full file bloom filters

You can now configure most RocksDB options with RocksDB's option
string scheme.

Set "filter_full" to 1 to make bloom filters for an
entire file rather than each block. More memory will be
needed during compaction but less memory will be needed
during fetching for large databases. Does nothing unless
bloom filters are enabled with "filter_bits".

Example:
options = max_compaction_bytes=64;max_bytes_for_level_multiplier=64
clock_cache_mb = 96
filter_bits = 10
filter_full = 1
2017-12-18 13:35:08 -05:00
Nikolaos D. Bougalis
d2fc4e3569 Set version to 0.80.2 2017-12-15 13:06:10 -05:00
Mark Travis
fbfb4bd74e Control transaction dispatch rate:
Do not dispatch a transaction received from a peer for
processing, if it has already been dispatched within the
past ten seconds.

Increase the number of transaction handlers that can be in
flight in the job queue and decrease the relative cost for
peers to share transaction and ledger data.

Additionally, make better use of resources by adjusting the
number of threads we initialize, by reverting commit
68b8ffdb63.
2017-12-14 16:04:04 -08:00
Brad Chase
3d6ea6737c Update soci unity build 2017-12-02 13:44:12 -05:00
seelabs
090d813065 Set version to 0.90.0-b1 2017-12-01 14:15:09 -05:00
Scott Schurr
722917e9a6 Improved error message on mistyped command [RIPD-1527]:
Previously if you mistyped the "submit_multisigned" command as
"submit_multisign", the returned message was "Internal error".  Not
very helpful.  It turns out this was caused by a small amount of
code in RPCCall.cpp.  Removing that code improves two situations:

1. It improves the situation with a mistyped command.  Now the
   command returns "Unknown method" and provides the string of
   the mistyped command.

2. The "transaction_entry", if properly entered in its command
   line form, would fire an assert.  That assert is now removed.

In the process, it was discovered that the command line form of
the "transaction_entry" command has not worked correctly for at
least a year.  Therefore support for that the command line form
of "transaction_entry" is added along with appropriate unit
tests.
2017-12-01 14:15:05 -05:00
Brad Chase
3e483cfa4c Improve JSON logging 2017-12-01 14:15:05 -05:00
JoelKatz
c42ea14531 Relay untrusted validations/proposals from cluster members (RIPD-1549)
It is common for a validator operator to connect their validator
only to other nodes under their control, using clustering. Relaying of
untrusted validations and proposals must be unreliable to prevent
denial of service attacks. But currently, they are unreliable even
within a cluster.

With this change, a cluster member's decisison to relay (or originate)
a validation or proposal is honored by other cluster members. This
ensures that validators in a cluster will get reliable relaying to
hubs outside the cluster, even if other members of the cluster do not
have that validator on their UNL.
2017-12-01 14:15:05 -05:00
Brad Chase
83dc45db62 Default account_tx RPC to validated ledger range (RIPD-1544) 2017-12-01 14:15:05 -05:00
Howard Hinnant
3757829f8e Convert more timing code to <chrono>
* LoadMonitor
* Entry
* Import
2017-12-01 14:15:05 -05:00
Howard Hinnant
cd19246937 Enable batch rpc processing
* Can be exercised from the command line with json2

* Rewrite Env::do_rpc to call the same code as
  rpc from the command line.  This puts rpc
  handling logic in one place.
2017-12-01 14:15:05 -05:00
Howard Hinnant
4e6c8d8b35 Remove use of deprecated behavior involving copy members
*  If any of the destructor, copy assignment or copy constructor
   are user-declared, both copy members should be user-declared,
   otherwise the compiler-generation of them is deprecated.
2017-12-01 14:15:05 -05:00
Mike Ellery
50b35e2090 Remove hardcoded ports in several unit tests:
Fixes: RIPD-1522
2017-12-01 14:15:05 -05:00
seelabs
e8d02c1333 Refactor FeatureBitset:
* Remove composite helper functions
* Add set difference and Bitset/uint256 operators
* Convert tests to use new feature bitset set difference operator
2017-12-01 14:15:05 -05:00
Scott Schurr
3523cee63d Minimize use of jtx::with_only_features (RIPD-1515):
In order to automatically run unit tests with newly created
amendments, prefer to start with jtx::supported_features() and
then subtract unwanted features.

These changes identified a few bugs that were hiding in
amendments.  One of those bugs, in FlowCross, is not yet fixed.
By uncommenting the test in CrossingLimits_test.cpp you can see
failures relating to that bug.  Since FlowCross is not yet
enabled on the network we can fix the bug at our convenience.
2017-12-01 14:15:05 -05:00
Edward Hennis
aa76632bb3 Add STAmount switchover support to tx queue:
* RIPD-1513
* New fix1513 amendment
2017-12-01 14:15:05 -05:00
Scott Schurr
feb7582aca Unit tests don't enable Tickets or SHAMapV2 by default:
Both Tickets and SHAMapV2 have been around for a while and don't
look like they will be enabled on the network soon.  So they are
removed from the supportedAmendments list.  This prevents Env
from automatically testing with Tickets or SHAMapV2 enabled,
although testing with those features can still be explicitly
specified.

Drive-by cleanups:

o supportedAmendments() returns a const reference rather than
  a fresh vector on each call.

o supportedAmendments() implementation moved from Amendments.cpp
  to Feature.cpp.  Amendments.cpp deleted.

o supportedAmendments() declared in Feature.h.  All other
  declarations deleted.

o preEnabledAmendments() removed, since it was empty and only
  used in one place.  It will be easy to re-add when it is needed.

o jtx::all_features_except() renamed to
  jtx::supported_features_except(), which is more descriptive.

o jtx::all_amendments() renamed to jxt::supported_amendments()

o jtx::with_features() renamed to with_only_features()

o Env_test.cpp adjusted since featureTickets is no longer
  automatically enabled for unit tests.
2017-12-01 14:15:04 -05:00
seelabs
07e3f81b76 Run unit tests in parallel 2017-12-01 14:15:04 -05:00
Brad Chase
2c13d9eb57 Redesign CSF framework (RIPD-1361):
- Separate `Scheduler` from `BasicNetwork`.
- Add an event/collector framework for monitoring invariants and calculating statistics.
- Allow distinct network and trust connections between Peers.
- Add a simple routing strategy to support broadcasting arbitrary messages.
- Add a common directed graph (`Digraph`) class for representing network and trust topologies.
- Add a `PeerGroup` class for simpler specification of the trust and network topologies.
- Add a `LedgerOracle` class to ensure distinct ledger histories and simplify branch checking.
- Add a `Submitter` to send transactions in at fixed or random intervals to fixed or random peers.

Co-authored-by: Joseph McGee
2017-12-01 14:15:04 -05:00
Edward Hennis
1853c0d678 Add STAmount switchover support to tx queue:
* RIPD-1513
* New fix1513 amendment
2017-12-01 13:49:58 -05:00
Brad Chase
6dc79c23ed Set version to 0.80.1 2017-11-29 17:58:11 -05:00
JoelKatz
5a9c3c797c Support python3 in SCons 2017-11-29 17:57:58 -05:00
Brad Chase
a4a43a4de9 Improve WorkSSL:
Support Server Name Indication
Ensure windows uses available certificates
2017-11-29 17:39:57 -05:00
wilsonianb
25b002b37f Add make_Manifest test
Reject manifest with invalid public key type

RIPD-1560
2017-11-28 20:33:33 -05:00
wilsonianb
40c39c4afb Verify validator token manifest matches private key
RIPD-1552
2017-11-28 20:33:31 -05:00
Nikolaos D. Bougalis
f0e1024ad6 Explicitly use std::deque for the missing node handler stack:
We need to ensure that pointers and/or references to existing
elements will not be invalidated during the course of element
insertion and removal.

Containers like std::vector do not offer this guarantee, and
cannot be used as the underlying container for the stack.

By choosing to explicitly specify std::deque as the underlying
cotnainer, we avoid:

- the unlikely possibility of the C++ standards committee
  changing the default template parameter to a container;
- the more likely possibility of an accidental change by
  a programmer, without fully considering the consequences.
2017-11-28 20:31:39 -05:00
Edward Hennis
c11e186659 Grow TxQ expected size quickly, shrink slowly (RIPD-1534):
* Stores recent history of "good" ledgers. Uses the maximum as the
  expected ledger size. When a large value drops off, use a 90%
  backoff to go down to to the new maximum.
* If consensus is unhealthy, wipe the history in addition to the current
  clamping.
* Include .md doc files in xcode and VS projects
2017-11-28 20:27:54 -05:00
Brad Chase
0a48916d98 Properly use ledger hash to break ties 2017-11-28 20:26:01 -05:00
Mark Travis
dffb999efb Set quorum to tolerate 2 failures for validator list of 6 members. 2017-11-28 20:23:07 -05:00