Commit Graph

906 Commits

Author SHA1 Message Date
ChronusZ
1bb32134f8 Remove censorshipMaxWarnings 2019-03-06 19:14:52 -08:00
Edward Hennis
4c06b3f86f Validate TxQ config and expected transactions range 2019-03-04 11:45:56 -05:00
seelabs
c5d215d901 Add delivered amount to the ledger RPC command 2019-02-25 13:01:12 -08:00
Nik Bougalis
2529edd2b6 Properly transition state to disconnected:
If the number of peers a server has is below the configured
minimum peer limit, this commit will properly transition the
server's state to "disconnected".

The default limit for the minimum number of peers required was
0 meaning that a server that was connected but lost all its
peers would never transition to disconnected, since it could
never drop below zero peers.

This commit redefines the default minimum number of peers to 1
and produces a warning if the server is configured in a way
that will prevent it from ever achieving sufficient connectivity.
2019-02-25 12:59:35 -08:00
Nik Bougalis
c6ab880c03 Display validator status only to admin requests:
Several commands allow a user to retrieve a server's status. Commands
will typically limit disclosure of information that can reveal that a
particular server is a validator to connections that are not verified
to make it more difficult to determine validators via fingerprinting.

Prior to this commit, servers configured to operate as validators
would, instead of simply reporting their server state as 'full',
augment their state information to indicate whether they are
'proposing' or 'validating'.

Servers will only provide this enhanced state information for
connections that have elevated privileges.

Acknowledgements:
Ripple thanks Markus Teufelberger for responsibly disclosing this issue.

Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers to responsibly
disclose any issues that they may find. For more on Ripple's Bug Bounty
program, please visit: https://ripple.com/bug-bounty
2019-02-25 12:59:31 -08:00
Howard Hinnant
148bbf4e8f Add safe_cast (RIPD-1702):
This change ensures that no overflow can occur when casting
between enums and integral types.
2019-01-18 12:13:21 -08:00
Joseph Busch
494724578a Enchance /crawl API endpoint with local server information (RIPD-1644):
The /crawl API endpoint allows developers to examine the structure of
the XRP Ledger's overlay network.

This commit adds additional information about the local server to the
/crawl endpoint, making it possible for developers to create data-rich
network-wide status dashboards.

Related:
 - https://developers.ripple.com/peer-protocol.html
 - https://github.com/ripple/rippled-network-crawler
2019-01-18 12:13:21 -08:00
Edward Hennis
2432f13903 Reserve correct vector size for fee calculations:
* Using txnsExpected_, which is influenced by both the config
  and network behavior, can reserve far too much or far too
  little memory, wasting time and resources.
* Not an issue during normal operation, but a user could
  cause problems on their local node with extreme configuration
  settings.
2018-12-11 12:51:46 -08:00
John Freeman
412a3ec710 Fix the --rpc_port command-line argument
The --rpc_port command-line option is effectively ignored. We construct
an `Endpoint` with the given port, but then drop it on the floor.
(Perhaps the author thought the `Endpoint::at_port` method is a mutation
instead of a transformation.) This small change adds the missing
assignment to hold on to the new endpoint.

Fixes #2764
2018-12-11 12:50:05 -08:00
Mark Travis
fbdbffed67 Report duration in current state. 2018-11-25 17:37:31 -08:00
Edward Hennis
c1a02440dc Load validator list from file:
* Adds local file:// URL support to the [validator_list_sites] stanza.
  The file:// URL must not contain a hostname. Allows a rippled node
  operator to "sideload" a new list if their node is unable to reach
  a validator list's web site before an old list expires. Lists
  loaded from a file will be validated in the same way a downloaded
  list is validated.
* Generalize file/dir "guards" from Config test so they can be reused
  in other tests.
* Check for error when reading validators.txt. Saves some parsing and
  checking of an empty string, and will give a more meaningful error.
* Completes RIPD-1674.
2018-11-20 19:49:39 -08:00
Edward Hennis
e7a69cce65 Account for minimum reserve in potential spend:
* Relevant when deciding whether an account can queue multiple
  transactions. If the potential spend of the already queued
  transactions would dip into the reserve, the reserve is
  preserved for fees.
* Also change several direct modifications of the owner count to
  call adjustOwnerCount to preserve overflow checking.
* Update related unit testcase
* Resolves #2251
2018-11-20 19:49:39 -08:00
Howard Hinnant
60dc949314 Remove custom terminate handler
* Reduce the amount of code we have to maintain.
* Remove the potential for degrading stack dumps.
2018-11-20 19:45:02 -08:00
JoelKatz
be70d81bd7 Perform some extra checks on ledger changes
Perform some extra checks on the close time and sequence number
of a candidate for network consensus ledger. This tightens
defenses against some "insane/hostile supermajority" attacks.
2018-11-09 07:40:41 -08:00
JoelKatz
0d2b2923da Control memory growth from slow writes
* Don't allow a write batch to grow without bound
* Don't fetch history if write load is high
2018-11-09 07:40:41 -08:00
Mike Ellery
6bdc9e7b30 Use correct manifest cache when loading ValidatorList 2018-11-08 18:58:13 -08:00
Nik Bougalis
753600a2a0 Reset the validator list fetch timer if an error occurs 2018-11-08 18:58:12 -08:00
Nik Bougalis
945493d9cf Allow servers to detect transaction censorship attempts (RIPD-1626):
The XRP Ledger is designed to be censorship resistant. Any attempt to
censor transactions would require coordinated action by a majority of
the system's validators.

Importantly, the design of the system is such that such an attempt is
detectable and can be easily proven since every validators must sign
the validations it publishes.

This commit adds an automated censorship detector. While the server is
in sync, the detector tracks all transactions that, in the view of the
server, should have been included and issues warnings of increasing
severity for any transactions which, have not after several rounds.
2018-11-08 18:58:11 -08:00
Nik Bougalis
77462b8f72 Remove deprecated 'validation_seed' RPC command:
The 'validation_seed' RPC command was used to change the validation
key used by a validator at runtime.

Its implementation was commented out with commit fa796a2eb5
which has been included in the codebase since the 0.30.0 release
and there are no plans to reintroduce the functionality at this
point.

Validator operators should migrate to using validator manifests
instead.

This fixes #2748.
2018-11-06 10:27:12 -08:00
Nik Bougalis
1682fe3a39 Cleanup unused Beast bits and pieces:
This cleanup does not remove Boost.Beast code, but old-style Beast
which is no longer relevant or helpful.
2018-11-06 10:27:10 -08:00
Edward Hennis
58f786cbb4 Make the FeeEscalation amendment permanent (RIPD-1654):
The FeeEscalation amendment has been enabled on the XRP Ledger network
since May 19, 2016. The transaction which activated this amendment is:
5B1F1E8E791A9C243DD728680F108FEF1F28F21BA3B202B8F66E7833CA71D3C3.

This change removes all conditional code based around the FeeEscalation
amendment, but leaves the amendment definition itself since removing the
definition would cause nodes to think an unknown amendment was activate
causing them to become amendment blocked.

The commit also removes the redundant precomputed hashes from the
supportedAmendments vector.
2018-11-06 10:26:29 -08:00
Edward Hennis
a96cb8fc1c Remove undocumented experimental options from RPC sign (RIPD-1653):
The `x_assume_tx` and `x_queue_okay` experimental options were
associated with the transaction queue that were not officially
supported.
2018-11-06 10:26:29 -08:00
Joe Loser
c587012e5c Inline calls to cachedRead:
Problem:
- There are only a few call sites to cachedRead, and all of them
  currently do more work than is required since we know the type in each
  case.

Solution:
- "Inline" the codepath to cachedRead, but do not check if the type is
  valid. In all such call sites, we know the keylet to read directly.

This fixes #2550
2018-11-06 10:26:29 -08:00
Howard Hinnant
146ea5d44e Remove a use after std::move
Fixes: #2538
Fixes: #2536
2018-11-06 10:26:29 -08:00
Howard Hinnant
156e8dae83 Replace WaitableEvent with portable std primitives:
The WaitableEvent class was a leftover from the pre-Boost
version of Beast and used Windows- and pthread-specific
APIs.

This refactor replaces that functionality by using only
interfaces provided by the C++ standard, making the code
more portable.

Closes #2402.
2018-11-06 10:26:29 -08:00
Markus Teufelberger
5e96da51f9 Remove the state file for the random number generator 2018-11-06 10:26:29 -08:00
Nik Bougalis
6ed2270bc9 Merge master (1.1.1) into develop (1.2.0-b4) 2018-10-23 08:21:43 -07:00
1535239824@qq.com
7b48dc36f5 Add fixTakerDryOfferRemoval amendment 2018-10-19 12:23:25 -07:00
Scott Schurr
a999894dae Allow rippled to compile with C++17:
Many of the warnings on Windows were not resolved, just
silenced with _SILENCE_ALL_CXX17_DEPRECATION_WARNINGS.
They need to be resolved in a future commit.
2018-10-19 12:21:57 -07:00
Mike Ellery
7fe1d4b9c2 Accept redirects from validator list sites:
Honor location header/redirect from validator sites. Limit retries per
refresh interval to 3. Shorten refresh interval after HTTP/network errors.

Fixes: RIPD-1669
2018-10-19 12:16:57 -07:00
Nik Bougalis
b36e11bc49 Properly handle expired validator lists when validating (RIPD-1661):
A validator that was configured to use a published validator list could
exhibit aberrent behavior if that validator list expired.

This commit introduces additional logic that makes validators operating
with an expired validator list bow out of the consensus process instead
of continuing to publish validations. Normal operation will resume once
a non-expired validator list becomes available.

This commit also enhances status reporting when using the `server_info`
and `validators` commands. Before, only the expiration time of the list
would be returned; now, its current status is also reported in a format
that is clearer.
2018-10-19 12:15:36 -07:00
Nik Bougalis
152d698957 Properly handle expired validator lists when validating (RIPD-1661):
A validator that was configured to use a published validator list could
exhibit aberrent behavior if that validator list expired.

This commit introduces additional logic that makes validators operating
with an expired validator list bow out of the consensus process instead
of continuing to publish validations. Normal operation will resume once
a non-expired validator list becomes available.

This commit also enhances status reporting when using the `server_info`
and `validators` commands. Before, only the expiration time of the list
would be returned; now, its current status is also reported in a format
that is clearer.
2018-10-19 13:08:56 -04:00
Mike Ellery
bdaad19e70 Accept redirects from validator list sites:
Honor location header/redirect from validator sites. Limit retries per
refresh interval to 3. Shorten refresh interval after HTTP/network errors.

Fixes: RIPD-1669
2018-10-11 11:08:27 -07:00
Miguel Portilla
3661dc88fe Add RPC command shard crawl (RIPD-1663) 2018-10-10 12:16:01 -04:00
Scott Schurr
0bbe6e226c Remove beast::Journal default constructor 2018-10-10 10:18:03 -04:00
Scott Schurr
6572fc8e95 Implement MultiSignReserve amendment [RIPD-1647]:
Reduces the account reserve for a multisigning SignerList from
(conditionally) 3 to 10 OwnerCounts to (unconditionally) 1
OwnerCount.  Includes a transition process.
2018-10-01 18:17:33 -07:00
Edward Hennis
7295cf979b Grow the open ledger expected transactions quickly (RIPD-1630):
* When increasing the expected ledger size, add on an extra 20%.
* When decreasing the expected ledger size, take the minimum of the
  validated ledger size or the old expected size, and subract another 50%.
* Update fee escalation documentation.
* Refactor the FeeMetrics object to use values from Setup
2018-10-01 11:26:22 -07:00
Edward Hennis
e14f913244 Update TxQ developer docs:
* Rename a couple of member variables for clarity.
2018-10-01 11:26:22 -07:00
Joe Loser
cd1c5a30dd Add user defined literals for megabytes and kilobytes 2018-10-01 11:26:22 -07:00
Scott Schurr
4104778067 Improve transaction error condition handling (RIPD-1578, RIPD-1593):
As described in #2314, when an offer executed with `Fill or Kill`
semantics, the server would return `tesSUCCESS` even if the order
couldn't be filled and was aborted. This would require additional
processing of metadata by users to determine the effects of the
transaction.

This commit introduces the `fix1578` amendment which, if enabled,
will cause the server to return the new `tecKILLED` error code
instead of `tesSUCCESS` for `Fill or Kill` orders that could not
be filled.

Additionally, the `fix1578` amendment will prevent the setting of
the `No Ripple` flag on trust lines with negative balance; trying
to set the flag on such a trust line will fail with the new error
code `tecNEGATIVE_BALANCE`.
2018-09-30 14:10:40 -07:00
Spec
4dcb3c9199 Avoid dispatching multiple fetch pack threads 2018-09-30 13:54:59 -07:00
Miguel Portilla
820546c873 Report fetch pack errors with shards 2018-09-28 09:15:06 -07:00
wilsonianb
8c14002c25 Do not use beast base64 encoding without fix:
Boost 1.67 and 1.68 are missing this fix
0439dcfa7a
2018-08-21 10:05:45 -05:00
Nik Bougalis
38c3a46a33 Deprecate commands that perform remote tx signing (RIPD-1649):
In order to facilitate transaction signing, `rippled` offers the `sign` and
`sign_for` and `submit` commands, which, given a seed, can be used to sign or
sign-and-submit transactions. These commands are accessible from the command
line, as well as over the WebSocket and RPC interfaces that `rippled` can be
configured to provide.

These commands, unfortunately, have significant security implications:

  1. They require divulging an account's seed (commonly known as a "secret
     key") to the server.
  2. When executing these commands against remote servers, the seeds can be
     transported over clear-text links.
  3. When executing these commands over the command line, the account
     seed may be visible using common tools that show running processes
     and may potentially be inadvertently stored by system monitoring
     tools or facilities designed to maintain a history of previously
     typed commands.

While this commit cannot prevent users from issuing these commands to a
server, whether locally or remotely, it restricts the `sign` and `sign_for`
commands, as well as the `submit` command when used to sign-and-submit,
so that they require administrative privileges on the server.

Server operators that want to allow unrestricted signing can do so by
adding the following stanza to their configuration file:

    [signing_support]
    true

Ripple discourages server operators from doing so and advises against using
these commands, which will be removed in a future release. If you rely on
these commands for signing, please migrate to a standalone signing solution
as soon as possible. One option is to use `ripple-lib`; documentation is
available at https://developers.ripple.com/rippleapi-reference.html#sign.

If the commands are administratively enabled, the server includes a warning
on startup and adds a new field in the resulting JSON, informing the caller
that the commands are deprecated and may become unavailable at any time.

Acknowledgements:
Jesper Wallin for reporting this issue to Ripple.

Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers to responsibly
disclose any issues that they may find. For more on Ripple's Bug Bounty
program, please visit: https://ripple.com/bug-bounty
2018-08-15 19:59:52 -07:00
Miguel Portilla
a73372cb9d Add RPC shard download 2018-08-08 21:07:54 -04:00
Miguel Portilla
658f904ce0 Add shard import support to shard database 2018-08-08 21:07:54 -04:00
Scott Schurr
2901577be7 Remove using namespace declarations at namespace scope in headers 2018-08-08 21:07:54 -04:00
Mark Travis
04745b11a8 Expand SQLite potential storage capacity:
Increase page size for SQLite transaction database upon creation
Provide diagnostics for transaction db page usage.
Shut down rippled gracefullly if transaction db is running out of pages.
Add new rippled maintenance command line option to cause new page size
to take effect.
2018-08-08 21:07:54 -04:00
Mike Ellery
63370b4441 Default to ipv4 for unit tests, add ipv6 option 2018-07-20 08:58:04 -07:00
Howard Hinnant
d98c4992dd Supply ConsensusTimer with milliseconds or finer precision 2018-07-20 08:58:04 -07:00