12890 Commits

Author SHA1 Message Date
Michael Legleux
95ecf296ad Install the protocol buffer deps during smoketests 2022-02-07 12:38:40 -08:00
Michael Legleux
b7e0306d0a Ensure protocol buffer prerequisites are present 2022-02-07 10:59:12 -08:00
Michael Legleux
a9ee802240 Update smoke test distributions 2022-02-07 10:59:12 -08:00
Edward Hennis
d23d37fcfd Correctly add GIT_COMMIT_HASH into version string:
Commit bf013c02ad added support
for incorporating a commit ID into the compiled version string
but did so in a way that did not follow the semantic versioning
standard.

This commit corrects that flaw by moving the commit ID into the
"metadata" part of the version string and properly handles the
case where the commit hash cannot be retrieved.
2022-02-07 10:52:31 -08:00
Nik Bougalis
289bc0afd9 Improve handling of endpoints during peer discovery 2022-02-07 10:33:02 -08:00
Nik Bougalis
c5dc00af74 Update RocksDB to version 6.27.3 2022-02-07 10:32:59 -08:00
Nik Bougalis
d49b486224 Set version to 1.8.4 1.8.4 2022-01-12 18:54:06 -08:00
Nik Bougalis
417cfc2fb0 Adjust pathfinding configuration defaults:
The pathfinding engine built into the code has several configurable
parameters to adjust the depth of the paths indexed and explored.

These parameters can dramatically impact the performance and memory
consumption of a server; higher values can result in resource usage
increasing exponentially.

These default values were decided early and somewhat arbitrarily at
a time when the network and the size of the network state were much
smaller.

This commit adjusts the default values to reduce the depth of paths
to more reasonable levels; unless explicitly overriden, the changes
mean that pathfinding operations will return fewer, shallower paths
than previous versions of the software.
2022-01-12 18:54:03 -08:00
seelabs
febbe14e6d Adjust mutex scope in walkMapParallel:
This commit corrects a technical flaw that was introduced with commit
7c12f01358: as written, a mutex that is
intended to help provide synchronization for multiple threads as they
are each walking the map, is declared so that each thread is passed a
dangling reference to a unique mutex.

This commit hoists the mutex outside the thread creation loop, so all
threads use a single mutex and eliminating the dangling reference.
2022-01-12 14:21:44 -08:00
Nik Bougalis
44514930f9 Set version to 1.8.3 1.8.3 2022-01-10 15:40:07 -08:00
Nik Bougalis
dc778536ed Provide sensible default values for nodestore cache:
The nodestore includes a built-in cache to reduce the disk I/O
load but, by default, this cache was not initialized unless it
was explicitly configured by the server operator.

This commit introduces sensible defaults based on the server's
configured node size.

It remains possible to completely disable the cache if desired
by explicitly configuring it the cache size and age parameters
to 0:

    [node_db]
    ...
    cache_size = 0
    cache_age = 0
2022-01-10 15:29:44 -08:00
Nik Bougalis
18584ef2fd Adjust number of concurrent ledger data jobs 2022-01-10 15:29:43 -08:00
Nik Bougalis
416ce35d73 Detect CVE-2021-3520 when decompressing using LZ4 2022-01-10 15:29:43 -08:00
Mark Travis
7c12f01358 Parallel ledger loader & I/O performance improvements:
- Only duplicate records from archive to writable during online_delete.
- Log duration of nodestore reads.
- Include nodestore counters in perf_log output.
- Remove gratuitous nodestore activity counting.
- Report initial sync duration in server_info and perfLog.
- Report state_accounting in perfLog.
- Make state_accounting durations more accurate.
- Parallel ledger loader.
- Config parameter to load ledgers on start.
2022-01-10 15:29:21 -08:00
JoelKatz
5a4654a0da Two small SHAMapSync improvements:
1) Don't acquire so many nodes per pass. It's likely
far more than we need.

2) Right-size the finishedReads_ vector on passes other
than just the first.
2022-01-07 04:41:58 -08:00
manojsdoshi
89766c5f21 Set version to 1.8.2 1.8.2 2021-12-20 14:50:58 -08:00
manojsdoshi
4ec11e692b Set version to 1.8.2-rc1 2021-12-16 23:23:25 -08:00
Michael Legleux
d02f0e11c5 Quiet builds and disable unity build 2021-12-16 22:45:38 -08:00
natenichols
e28989638d Enforce account RPC limits by account objects traversed 2021-12-16 11:57:08 -08:00
CJ Cobb
915fe31274 log request and duration for every RPC call 2021-12-15 12:12:28 -08:00
Mark Travis
db720a59e4 Log resource limit disconnections. 2021-12-15 11:26:13 -08:00
CJ Cobb
72752b1ee0 make cassandra io threads configurable 2021-12-15 11:25:32 -08:00
Mark Travis
c663f1f62b Make tx() function against a read-only postgres instance. 2021-12-15 11:25:10 -08:00
Scott Schurr
d54f6278bb Improve names returned by server_info counters 2021-12-15 11:21:51 -08:00
Mark Travis
fc04336caa Reporting mode always returns age in server_info. 2021-12-15 11:18:58 -08:00
CJ Cobb
47376a0cc3 Only forward to p2p nodes that are in sync 2021-12-15 11:18:07 -08:00
Nik Bougalis
45aa0142a6 Improve full & compressed inner node deserialization 2021-12-14 17:49:20 -08:00
Edward Hennis
e3acb61d57 Increase TxQ default minimum and target sizes 2021-12-14 17:47:48 -08:00
Edward Hennis
8fa33795a3 Reduce TxQ logging severity in several places 2021-12-14 17:47:48 -08:00
Edward Hennis
b1c9b134dc Make transaction queue order deterministic:
* Sort by fee level (which is the current behavior) then by transaction
  ID (hash).
* Edge case when the account at the end of the queue submits a higher
  paying transaction to walk backwards and compare against the cheapest
  transaction from a different account.
* Use std::if_any to simplify the JobQueue::isOverloaded loop.
2021-12-14 17:47:48 -08:00
Edward Hennis
ae9930b87d Consensus transaction recovery/deferral completely ignores the TxQ 2021-12-14 17:47:38 -08:00
Edward Hennis
aaa601841c Logging & minor optimizations:
* Log load fee values (at debug) received from validations.
* Log remote and cluster fee values (at trace) when changed.
* Refactor JobQueue::isOverloaded to return sooner if overloaded.
* Refactor Transactor::checkFee to only compute fee if ledger is open.
2021-12-14 17:47:38 -08:00
Scott Schurr
8ca2d98496 NuDBBackend destructor should not throw 2021-12-14 17:47:38 -08:00
Scott Schurr
ad805eb95b Improve handling of the X-Forwarded-For and Forwarded HTTP headers 2021-12-14 17:47:04 -08:00
Wo Jake
eb17325cbe Add load cost 2021-12-14 17:47:04 -08:00
wojak
b00787e161 Update README.md 2021-12-14 17:47:04 -08:00
Ikko Ashimine
81e7ec859d Fix typo in consensus.md
determing -> determining
2021-12-14 17:47:04 -08:00
Ikko Ashimine
6f6179abb4 Fix typo in RFC1751.cpp
seperated -> separated
2021-12-14 17:47:04 -08:00
Howard Hinnant
32a26a65d9 Make basic_seconds_clock::time_point atomic
* Just the rep is made atomic to workaround older compilers
2021-12-14 17:47:04 -08:00
Richard Holland
daccb5b4c0 Add ignore_default option to account_lines API:
This flag, if present, suppresses the output of incoming
trustlines in default state.

This is primarily motivated by observing that users of Xumm often
have many unwanted incoming trustlines in a default state, which are
not useful in the vast majority of cases.

Being able to suppress those when doing `account_lines` saves bandwidth
and resources.
2021-12-14 17:44:24 -08:00
Richard Holland
cf97dcb992 Make I/O and prefetch worker threads configurable 2021-12-14 17:43:50 -08:00
Richard Holland
6746b863b3 Configurable handling of untrusted validations and proposals 2021-12-14 17:43:04 -08:00
Richard Holland
bf013c02ad Add git commit hash to debug version string 2021-12-14 17:42:59 -08:00
manojsdoshi
fbedfb25ae Set version to 1.8.1 1.8.1 2021-11-24 10:32:37 -08:00
Nik Bougalis
9e877a929e Adjust RocksDB defaults 2021-11-24 10:29:49 -08:00
manojsdoshi
e0eae9725b Set version to 1.8.0 2021-11-22 18:44:47 -08:00
manojsdoshi
3083983fee Set version to 1.8.0-rc3 2021-11-22 18:42:33 -08:00
Edward Hennis
5050b366d9 Return correct error code for node_to_shard command errors 2021-11-22 18:40:32 -08:00
seelabs
f0c237e001 Set version to 1.8.0-rc2 2021-11-18 14:39:03 -05:00
CJ Cobb
970711f1fd Properly handle empty optionals in DB lookups 2021-11-18 14:38:42 -05:00