Commit Graph

12556 Commits

Author SHA1 Message Date
Edward Hennis
93ea4b2f4f Fix a typo in the [sqlite] section description 2021-02-12 12:38:01 -08:00
CJ Cobb
5776c2ebe5 Document secure_gateway for gRPC 2021-02-12 12:37:32 -08:00
manojsdoshi
8defb4cd28 Set version to 1.7.0-rc2 2021-02-04 19:13:14 -08:00
Miguel Portilla
d358495f02 Add database counters:
Fix bug where DatabaseRotateImp::getBackend and ::sync utilized the writable
backend without a lock. ::getBackend was replaced with ::getCounters.
2021-02-04 19:10:28 -08:00
Gregory Tsipenyuk
38dd2d6677 Fix idle peer timer:
This commit fixes  #3754
2021-02-04 19:10:01 -08:00
manojsdoshi
80bd107e57 Set version to 1.7.0-rc1 2021-01-29 12:01:56 -08:00
manojsdoshi
68286df23d Set version to 1.7.0-b12 2021-01-26 11:26:17 -08:00
Edward Hennis
d494d7a725 Use 64-bit Visual Studio build tools in documentation and Travis 2021-01-26 11:26:02 -08:00
seelabs
36be4856fd Fix path finding for XRP dst and sendmax:
* Fix bug where incorrect max amount was set for XRP
* Fix bug where incorrect source currencies were set when XRP was the dst and a
  sendmax amount was set
2021-01-25 18:49:49 -08:00
seelabs
c47b4f3667 Improve canonicalization of serialized amounts:
The existing code that deserialized an STAmount was sub-optimal and performed
poorly. In some rare cases the operation could result in otherwise valid
serialized amounts overflowing during deserialization. This commit will help
detect error conditions more quickly and eliminate the problematic corner cases.
2021-01-25 18:49:49 -08:00
seelabs
fe129e8e4f Optimize payment path exploration in flow:
* Use theoretical quality to order the strands
* Do not use strands below the user specified quality limit
* Stop exploring strands (at the current quality iteration) once any strand is non-dry
2021-01-25 18:49:49 -08:00
John Freeman
0a1fb4e6ca Reduce nesting and remove dead code 2021-01-25 18:49:49 -08:00
Peng Wang
7e97bfce10 Implement ledger forward replay 2021-01-25 18:49:49 -08:00
cdy20
49409dbf27 Added document describing why 16k-ledgers shards
are better than 8k- and 32k-ledgers
2021-01-25 18:49:43 -08:00
mDuo13
c3227a67ec Fix telBAD_PUBLIC_KEY message:
The previous error description was focused on keys that are too long,
but this error can occur if the key is too short or does not contain
the correct prefix.
2021-01-25 13:44:31 -08:00
Elliot Lee
a4a46a491f Update validators-example.txt 2021-01-25 13:44:16 -08:00
manojsdoshi
c11037fd27 Set version to 1.7.0-b11 2021-01-20 16:33:19 -08:00
Miguel Portilla
114981f774 Include missing header in json_reader 2021-01-20 16:32:59 -08:00
CJ Cobb
27543170d0 Add Reporting Mode
* Add a new operating mode to rippled called reporting mode
* Add ETL mechanism for a reporting node to extract data from a p2p node
* Add new gRPC methods to faciliate ETL
* Use Postgres in place of SQLite in reporting mode
* Add Cassandra as a nodestore option
* Update logic of RPC handlers when running in reporting mode
* Add ability to forward RPCs to a p2p node
2021-01-20 11:30:03 -08:00
Mark Travis
b0a39c5f86 Add Postgres functionality 2021-01-20 10:53:24 -08:00
John Freeman
4fbb1699be Add start-sync-stop Python script 2021-01-20 10:53:24 -08:00
Wietse Wind
319f29da8d Update ripple.com » xrpl.org links 2021-01-20 10:53:24 -08:00
Nik Bougalis
70bacb349e Improve manifest relaying and domain reporting:
- The changes to manifest relaying introduced with commit f74b469e68
  will cause newly accepted manifests to be sent back to the peer from
  which they were received. This no longer happens: a newly accepted
  manifest is never sent back to the peer we received it from.
- When encountering a manifest without a domain set, the `manifest` and
  `validator_info` commands would include an empty string as the domain
  associated with the manifest. This no longer happens: if a domain is
  not present, the `domain` field will not be.
2021-01-20 10:53:24 -08:00
Nik Bougalis
012bbcfe36 Fix parsing of node public keys in manifest CLI:
The existing code attempts to validate the provided node public key
using a function that assumes that the encoded public key is for an
account. This causes the parsing to fail.

This commit fixes #3317 by letting the caller specify the type of
the public key being checked.
2021-01-20 10:51:07 -08:00
Nik Bougalis
f74b469e68 Improve manifest relaying:
The manifest relay code would only ever relay manifests from validators
on a server's UNL which means that the manifests of validators that are
not broadly trusted can fail to propagate across the network, which can
make it difficult to detect and track such validators.

This commit, if merged, propagates all manifests on a best-effort basis
resulting in broader availability of manifests on the network and avoid
the need to introduce on-ledger manifest storage or to establish one or
more manifest repositories.
2021-01-20 10:51:04 -08:00
Nik Bougalis
55dc7a252e Set version to 1.7.0-b10 2021-01-09 13:50:07 -08:00
Nik Bougalis
b015623128 Add missing include headers 2021-01-09 13:50:07 -08:00
Nik Bougalis
c76a124d14 Remove unused Beast code 2021-01-09 13:50:06 -08:00
Gregory Tsipenyuk
74d96ff4bd Add experimental validation & proposal relay reduction support:
- Add validation/proposal reduce-relay feature negotiation to
  the handshake
- Make squelch duration proportional to a number of peers that
  can be squelched
- Refactor makeRequest()/makeResponse() to facilitate handshake
  unit-testing
- Fix compression enable flag for inbound peer
- Fix compression algorithm parsing in the header parser
- Fix squelch duration in onMessage(TMSquelch)

This commit fixes 3624, fixes 3639 and fixes 3641
2021-01-09 13:49:40 -08:00
Scott Schurr
44fe0e1fc4 Add support for the 'TicketBatch' amendment:
Support for 'out-of-sequence' transaction execution was introduced
in commit 7724cca384.

The changes in that commit were gated under a feature but there was
no corresponding amendment introduced that would allow the network
to vote on this amendment.

This commit introduces 'TicketBatch' amendment as the amendment
that is associated with the tickets feature. If the amendment is
enabled, it will activate support for tickets.

This commit also removes several workarounds that are no longer
needed in unit tests.
2021-01-08 14:43:06 -05:00
John Freeman
78245a072c Clean-up the Stoppable architecture 2021-01-08 14:43:06 -05:00
Miguel Portilla
1fd1c34112 Prevent deadlock in storeSQLite 2021-01-08 14:43:06 -05:00
Edward Hennis
0dae22adf2 Prevent simultaneous outgoing async peer messages:
* Also, only send validator list on incoming connection instead of both
  incoming and outgoing.
2021-01-08 14:43:06 -05:00
Edward Hennis
746181cb33 Analyze and document levelization:
* Markdown explanation of what levelization is, the intended levels, as
  well as the process used to determine dependencies
* Shell script finds all dependencies, groups them, and finds cyclic
  dependencies and maps out non-cyclic dependencies.
* Github job to run the script and fail if anything changes. Should
  catch introduction of new dependencies and new problems. Will also
  detect changes if problems or dependencies are removed.
2021-01-08 14:43:02 -05:00
Edward Hennis
4b9d3ca7de Support UNLs with future effective dates:
* Creates a version 2 of the UNL file format allowing publishers to
  pre-publish the next UNL while the current one is still valid.
* Version 1 of the UNL file format is still valid and backward
  compatible.
* Also causes rippled to lock down if it has no valid UNLs, similar to
  being amendment blocked, except reversible.
* Resolves #3548
* Resolves #3470
2021-01-08 12:35:08 -05:00
Nik Bougalis
54da532ace Set version to 1.7.0-b9 2020-12-18 15:23:31 -08:00
Gregory Tsipenyuk
4159b02753 Properly handle the "compression" flag:
- Fix compression enable flag in the inbound peer
- Fix compression algorithm fetching in protocol message
2020-12-18 15:23:26 -08:00
Edward Hennis
42a068ab5e Force wipe the travis caches with "travis_clean_cache" 2020-12-18 15:23:26 -08:00
Edward Hennis
6f7d413d88 Simplify travis config:
* Move all the vcpkg windows dependency installations into one step.
* Move the unmodified `before_install` step above the matrix to improve
  readability, because this step runs before any of the matrix steps.
2020-12-18 15:23:26 -08:00
Edward Hennis
1aaafeb57b Reset corrupt / inconsistent dependency source folders:
* Also display some more information about the build
2020-12-18 15:23:26 -08:00
JoelKatz
6dd3d825c8 Adjust time resolution of genesis ledger:
Due to some quirky emergent behavior, the server can't really begin
synching until twice the default close time resolution of the genesis
ledger, which is 30 seconds, has passed. In effect, this causes a one
minute delay.

This commit adjusts the default close time resolution down to the
minimum allowed resoluion of 10 seconds, so the corresponding delay
is reduced by 67% down to 20 seconds. This should be enough time to
ensure the server has reasonable connectivity without unduly delaying
initial synch times.
2020-12-18 15:23:26 -08:00
JoelKatz
02ccdeb94e Rework deferred node logic and async fetch behavior
This comment explains this patch and the associated patches
that should be folded into it. This paragraph should be removed
when the patches are folded after review.

This change significantly improves ledger sync and fetch
times while reducing memory consumption. The change affects
the code from that begins with SHAMap::getMissingNodes and runs
through to Database::threadEntry.

The existing code issues a number of async fetches which are then
handed off to the Database's pool of read threads to execute.
The results of each read are placed in the Database's positive
and negative caches. The caller waits for all reads to complete
and then retrieves the results out of these caches.

Among other issues, this means that the results of the first read
cannot be processed until the last read completes. Additionally,
all the results must sit in memory.

This patch changes the behavior so that each read operation has a
completion handler associated with it. The completion of the read
calls the handler, allowing the results of each read to be
processed as it completes. As this was the only reason the
negative and positive caches were needed, they can now be removed.

The read generation code is also no longer needed and is removed.
The batch fetch logic was never implemented or supported and is
removed.
2020-12-17 09:11:39 -08:00
seelabs
7b192945eb Replace std::pmr with boost::pmr:
gcc's implementation of `prm::synchronized_pool_resource` showed
extremely poor performance compared with
`boost::synchronized_pool_resouece`. Boost's implementation of pmr is
now used in all cases (previously it was only used when a standard
lib, like clang's, lacked an implementation of pmr).

This patch also makes a minor change where inner nodes are constructed
with sparse arrays, unless "dense" is explicitly requested.
2020-12-17 09:11:37 -08:00
Howard Hinnant
28ed2b9e69 Persist API-configured voting settings:
Prior to this commit, the amendments that a server would vote in support
of or against could be configured both via the configuration file and
via the command line "feature" command. Changes made in the configuration
file would only be loaded once at server startup and changes made via the
command line take effect immediately but are not persisted across
restarts.

This commit deprecates management of amendments via the configuration
file and stores the relevant information in the `wallet.db` database
file.

1. On startup, the new code parses the configuration file.
2. If the `[veto_amendments]` or `[amendments]` sections are present,
   we check if the `FeatureVotes` table is present in `wallet.db`.
3. If it is not, we create the `FeatureVotes` table and transfer the
   settings from the config file.
4. Proceed normally but only reference the `FeatureVotes` table instead
   of the config file.
5. Warns if the voting table already exists in `wallet.db` and there
   exists voting sections in the config file. The config file is ignored
   in this case.

This change addresses & closes #3366
2020-12-17 05:56:17 -08:00
Nik Bougalis
24b17c6de9 Set version to 1.7.0-b8 2020-12-12 04:51:02 -08:00
Edward Hennis
6eb93e9f8a Fix nested locks in ValidatorList:
* Found several functions called under lock that take a lock. Refactor
  to require a lock as a parameter instead.
* Found several functions called under lock that don't take a lock, but
  should. Refactored those as well to require a lock as a parameter.
2020-12-12 04:50:59 -08:00
Danil Nemirovsky
8173d1f643 Add pkg-config package to linux build guide 2020-12-12 04:50:59 -08:00
seelabs
b29812e40b Fix double counting unit test failures:
Unit tests are counting test failures, process crashes, and process exit code
failures in the count. Since a failing tests causes the process exit code to
return failure, we get extra counts. This patch removes process exit code
failures from the count.
2020-12-12 04:50:59 -08:00
seelabs
183dcd08d9 Remove ReadViewFwdRange end iterator cache:
ReadViewFwdRange was storing a cached `end_` iterator that was lazily
created in an iterators `end()` function. When the cache is empty, and
the range it iterated from multiple threads, this creates a race
condition.

This change has performance consequences for "old style" for loops.
For example:

```
 // don't do this
for(auto i = tx_range.begin(); i != tx_range.end(); ++i)

```

Can call the now expensive `end()` function more often than needed.
Range-based for loop (I.e. `for(auto const& t : tx_range)`) should be
used instead.
2020-12-12 04:50:59 -08:00
Nik Bougalis
deea16e14f Correct handling of corner-cases involving custom iterators:
- Under some conditions, comparing `ReadViewFwdRange::iterators`
  for equality could derefence an empty `std::unique_ptr` which
  will result in a crash.
- Misuse of the `equal` API could result in a `std::bad_cast`
  exception being thrown from when iterating transactions or
  SLEs from the `OpenView`, `RawStateTable` and `Ledger` classes.
2020-12-12 04:50:59 -08:00