Commit Graph

8831 Commits

Author SHA1 Message Date
CJ Cobb
430802c1cf Add load_factor to server_info in reporting mode
* load_factor was missing from server_info when the server was running in
  reporting mode. Now, the reporting mode server calls server_info on the p2p
  node, and propagates the load_factor back to the client.
2021-05-07 14:58:29 -07:00
Nik Bougalis
9106a06579 Set version to 1.7.1 2021-04-01 13:40:41 -07:00
Nik Bougalis
79e69da364 Adjust OpenSSL defaults and mitigate CVE-2021-3499:
In order to effectively mitigate CVE-2021-3499 even when compiling
against versions of OpenSSL prior to 1.1.1k, this commit:

1) requires use of TLS 1.2 or later. Note that both TLS 1.0 and
   TLS 1.1 have been officially deprecated for over a year.
2) disables renegotiation support for TLS 1.2 connections.

Lastly, this commit also changes the default list of ciphers that
the server offers, limiting it only to ciphers that are part of
TLS 1.2.
2021-04-01 13:40:40 -07:00
CJ Cobb
73116297aa Properly encode results from the tx RPC command:
The `tx` command supports output in both "text" and "binary" modes,
controlled by the binary flag. For more details on the command and
the possible arguments, please see: https://xrpl.org/tx.html.

The existing handler would incorrectly deal with metadata when in
binary mode. This commit corrects this issue, ensuring that the
metadata is properly encoded, depending on the mode.
2021-04-01 13:21:02 -07:00
CJ Cobb
8579eb0c19 Maintain compatibility for forwarded RPC responses:
Typically, an RPC response contains a `result` field, which
contains details about the operation performed. For ease of
parsing, forwarded responses must look like a non-forwarded
response.

In some instances the response was incorrectly composed, so
that the actual `result` object would be encapsulated by an
outer `result` object, breaking existing code.

This commit, addresses this issue and correctly "folds" the
`result` field, ensuring a consistent schema for responses.
2021-04-01 10:40:05 -07:00
Mark Travis
9c8caddc5a Support HTTP health check in reporting mode. 2021-04-01 10:40:05 -07:00
manojsdoshi
c0a0b79d2d Set version to 1.7.0 2021-02-23 12:51:24 -08:00
manojsdoshi
1d5d902d28 Set version to 1.7.0-rc4 2021-02-18 18:26:17 -08:00
Mark Travis
f284a19246 Don't delete from tx table if it's not in use 2021-02-18 18:24:39 -08:00
manojsdoshi
735b8b7d48 Set version to 1.7.0-rc3 2021-02-12 13:33:37 -08:00
Scott Schurr
a2e1a7a84d TicketSequence with non-zero Sequence is an error:
Before this change any non-zero Sequence field was handled as
a non-ticketed transaction, even if a TicketSequence was
present.  We learned that this could lead to user confusion.
So the rules are tightened up.

Now if any transaction contains both a non-zero Sequence
field and a TicketSequence field then that transaction
returns a temSEQ_AND_TICKET error code.

The (deprecated) "sign" and "submit" RPC commands are tuned
up so they auto-insert a Sequence field of zero if they see
a TicketSequence in the transaction.

No amendment is needed because this change is going into
the first release that supports the TicketBatch amendment.
2021-02-12 13:33:26 -08:00
John Freeman
c138338358 Fix declaration of CassandraBackend::counters_ 2021-02-12 13:32:33 -08:00
Ikko Ashimine
e7eba93666 Fix typo in aged_unordered_container.h 2021-02-12 12:42:22 -08:00
manojsdoshi
8defb4cd28 Set version to 1.7.0-rc2 2021-02-04 19:13:14 -08:00
Miguel Portilla
d358495f02 Add database counters:
Fix bug where DatabaseRotateImp::getBackend and ::sync utilized the writable
backend without a lock. ::getBackend was replaced with ::getCounters.
2021-02-04 19:10:28 -08:00
Gregory Tsipenyuk
38dd2d6677 Fix idle peer timer:
This commit fixes  #3754
2021-02-04 19:10:01 -08:00
manojsdoshi
80bd107e57 Set version to 1.7.0-rc1 2021-01-29 12:01:56 -08:00
manojsdoshi
68286df23d Set version to 1.7.0-b12 2021-01-26 11:26:17 -08:00
seelabs
36be4856fd Fix path finding for XRP dst and sendmax:
* Fix bug where incorrect max amount was set for XRP
* Fix bug where incorrect source currencies were set when XRP was the dst and a
  sendmax amount was set
2021-01-25 18:49:49 -08:00
seelabs
c47b4f3667 Improve canonicalization of serialized amounts:
The existing code that deserialized an STAmount was sub-optimal and performed
poorly. In some rare cases the operation could result in otherwise valid
serialized amounts overflowing during deserialization. This commit will help
detect error conditions more quickly and eliminate the problematic corner cases.
2021-01-25 18:49:49 -08:00
seelabs
fe129e8e4f Optimize payment path exploration in flow:
* Use theoretical quality to order the strands
* Do not use strands below the user specified quality limit
* Stop exploring strands (at the current quality iteration) once any strand is non-dry
2021-01-25 18:49:49 -08:00
John Freeman
0a1fb4e6ca Reduce nesting and remove dead code 2021-01-25 18:49:49 -08:00
Peng Wang
7e97bfce10 Implement ledger forward replay 2021-01-25 18:49:49 -08:00
cdy20
49409dbf27 Added document describing why 16k-ledgers shards
are better than 8k- and 32k-ledgers
2021-01-25 18:49:43 -08:00
mDuo13
c3227a67ec Fix telBAD_PUBLIC_KEY message:
The previous error description was focused on keys that are too long,
but this error can occur if the key is too short or does not contain
the correct prefix.
2021-01-25 13:44:31 -08:00
manojsdoshi
c11037fd27 Set version to 1.7.0-b11 2021-01-20 16:33:19 -08:00
Miguel Portilla
114981f774 Include missing header in json_reader 2021-01-20 16:32:59 -08:00
CJ Cobb
27543170d0 Add Reporting Mode
* Add a new operating mode to rippled called reporting mode
* Add ETL mechanism for a reporting node to extract data from a p2p node
* Add new gRPC methods to faciliate ETL
* Use Postgres in place of SQLite in reporting mode
* Add Cassandra as a nodestore option
* Update logic of RPC handlers when running in reporting mode
* Add ability to forward RPCs to a p2p node
2021-01-20 11:30:03 -08:00
Mark Travis
b0a39c5f86 Add Postgres functionality 2021-01-20 10:53:24 -08:00
Nik Bougalis
70bacb349e Improve manifest relaying and domain reporting:
- The changes to manifest relaying introduced with commit f74b469e68
  will cause newly accepted manifests to be sent back to the peer from
  which they were received. This no longer happens: a newly accepted
  manifest is never sent back to the peer we received it from.
- When encountering a manifest without a domain set, the `manifest` and
  `validator_info` commands would include an empty string as the domain
  associated with the manifest. This no longer happens: if a domain is
  not present, the `domain` field will not be.
2021-01-20 10:53:24 -08:00
Nik Bougalis
012bbcfe36 Fix parsing of node public keys in manifest CLI:
The existing code attempts to validate the provided node public key
using a function that assumes that the encoded public key is for an
account. This causes the parsing to fail.

This commit fixes #3317 by letting the caller specify the type of
the public key being checked.
2021-01-20 10:51:07 -08:00
Nik Bougalis
f74b469e68 Improve manifest relaying:
The manifest relay code would only ever relay manifests from validators
on a server's UNL which means that the manifests of validators that are
not broadly trusted can fail to propagate across the network, which can
make it difficult to detect and track such validators.

This commit, if merged, propagates all manifests on a best-effort basis
resulting in broader availability of manifests on the network and avoid
the need to introduce on-ledger manifest storage or to establish one or
more manifest repositories.
2021-01-20 10:51:04 -08:00
Nik Bougalis
55dc7a252e Set version to 1.7.0-b10 2021-01-09 13:50:07 -08:00
Nik Bougalis
b015623128 Add missing include headers 2021-01-09 13:50:07 -08:00
Nik Bougalis
c76a124d14 Remove unused Beast code 2021-01-09 13:50:06 -08:00
Gregory Tsipenyuk
74d96ff4bd Add experimental validation & proposal relay reduction support:
- Add validation/proposal reduce-relay feature negotiation to
  the handshake
- Make squelch duration proportional to a number of peers that
  can be squelched
- Refactor makeRequest()/makeResponse() to facilitate handshake
  unit-testing
- Fix compression enable flag for inbound peer
- Fix compression algorithm parsing in the header parser
- Fix squelch duration in onMessage(TMSquelch)

This commit fixes 3624, fixes 3639 and fixes 3641
2021-01-09 13:49:40 -08:00
Scott Schurr
44fe0e1fc4 Add support for the 'TicketBatch' amendment:
Support for 'out-of-sequence' transaction execution was introduced
in commit 7724cca384.

The changes in that commit were gated under a feature but there was
no corresponding amendment introduced that would allow the network
to vote on this amendment.

This commit introduces 'TicketBatch' amendment as the amendment
that is associated with the tickets feature. If the amendment is
enabled, it will activate support for tickets.

This commit also removes several workarounds that are no longer
needed in unit tests.
2021-01-08 14:43:06 -05:00
John Freeman
78245a072c Clean-up the Stoppable architecture 2021-01-08 14:43:06 -05:00
Miguel Portilla
1fd1c34112 Prevent deadlock in storeSQLite 2021-01-08 14:43:06 -05:00
Edward Hennis
0dae22adf2 Prevent simultaneous outgoing async peer messages:
* Also, only send validator list on incoming connection instead of both
  incoming and outgoing.
2021-01-08 14:43:06 -05:00
Edward Hennis
4b9d3ca7de Support UNLs with future effective dates:
* Creates a version 2 of the UNL file format allowing publishers to
  pre-publish the next UNL while the current one is still valid.
* Version 1 of the UNL file format is still valid and backward
  compatible.
* Also causes rippled to lock down if it has no valid UNLs, similar to
  being amendment blocked, except reversible.
* Resolves #3548
* Resolves #3470
2021-01-08 12:35:08 -05:00
Nik Bougalis
54da532ace Set version to 1.7.0-b9 2020-12-18 15:23:31 -08:00
Gregory Tsipenyuk
4159b02753 Properly handle the "compression" flag:
- Fix compression enable flag in the inbound peer
- Fix compression algorithm fetching in protocol message
2020-12-18 15:23:26 -08:00
JoelKatz
6dd3d825c8 Adjust time resolution of genesis ledger:
Due to some quirky emergent behavior, the server can't really begin
synching until twice the default close time resolution of the genesis
ledger, which is 30 seconds, has passed. In effect, this causes a one
minute delay.

This commit adjusts the default close time resolution down to the
minimum allowed resoluion of 10 seconds, so the corresponding delay
is reduced by 67% down to 20 seconds. This should be enough time to
ensure the server has reasonable connectivity without unduly delaying
initial synch times.
2020-12-18 15:23:26 -08:00
JoelKatz
02ccdeb94e Rework deferred node logic and async fetch behavior
This comment explains this patch and the associated patches
that should be folded into it. This paragraph should be removed
when the patches are folded after review.

This change significantly improves ledger sync and fetch
times while reducing memory consumption. The change affects
the code from that begins with SHAMap::getMissingNodes and runs
through to Database::threadEntry.

The existing code issues a number of async fetches which are then
handed off to the Database's pool of read threads to execute.
The results of each read are placed in the Database's positive
and negative caches. The caller waits for all reads to complete
and then retrieves the results out of these caches.

Among other issues, this means that the results of the first read
cannot be processed until the last read completes. Additionally,
all the results must sit in memory.

This patch changes the behavior so that each read operation has a
completion handler associated with it. The completion of the read
calls the handler, allowing the results of each read to be
processed as it completes. As this was the only reason the
negative and positive caches were needed, they can now be removed.

The read generation code is also no longer needed and is removed.
The batch fetch logic was never implemented or supported and is
removed.
2020-12-17 09:11:39 -08:00
seelabs
7b192945eb Replace std::pmr with boost::pmr:
gcc's implementation of `prm::synchronized_pool_resource` showed
extremely poor performance compared with
`boost::synchronized_pool_resouece`. Boost's implementation of pmr is
now used in all cases (previously it was only used when a standard
lib, like clang's, lacked an implementation of pmr).

This patch also makes a minor change where inner nodes are constructed
with sparse arrays, unless "dense" is explicitly requested.
2020-12-17 09:11:37 -08:00
Howard Hinnant
28ed2b9e69 Persist API-configured voting settings:
Prior to this commit, the amendments that a server would vote in support
of or against could be configured both via the configuration file and
via the command line "feature" command. Changes made in the configuration
file would only be loaded once at server startup and changes made via the
command line take effect immediately but are not persisted across
restarts.

This commit deprecates management of amendments via the configuration
file and stores the relevant information in the `wallet.db` database
file.

1. On startup, the new code parses the configuration file.
2. If the `[veto_amendments]` or `[amendments]` sections are present,
   we check if the `FeatureVotes` table is present in `wallet.db`.
3. If it is not, we create the `FeatureVotes` table and transfer the
   settings from the config file.
4. Proceed normally but only reference the `FeatureVotes` table instead
   of the config file.
5. Warns if the voting table already exists in `wallet.db` and there
   exists voting sections in the config file. The config file is ignored
   in this case.

This change addresses & closes #3366
2020-12-17 05:56:17 -08:00
Nik Bougalis
24b17c6de9 Set version to 1.7.0-b8 2020-12-12 04:51:02 -08:00
Edward Hennis
6eb93e9f8a Fix nested locks in ValidatorList:
* Found several functions called under lock that take a lock. Refactor
  to require a lock as a parameter instead.
* Found several functions called under lock that don't take a lock, but
  should. Refactored those as well to require a lock as a parameter.
2020-12-12 04:50:59 -08:00
seelabs
b29812e40b Fix double counting unit test failures:
Unit tests are counting test failures, process crashes, and process exit code
failures in the count. Since a failing tests causes the process exit code to
return failure, we get extra counts. This patch removes process exit code
failures from the count.
2020-12-12 04:50:59 -08:00