* Tally and duration counters for Job Queue tasks and RPC calls
optionally rendered by server_info and server_state, and
optionally printed to a distinct log file.
- Tally each Job Queue task as it is queued, starts, and
finishes running. Track total duration queued and running.
- Tally each RPC call as it starts and either finishes
successfully or throws an exception. Track total running
duration for each.
* Track currently executing Job Queue tasks and RPC methods
along with durations.
* Json-formatted performance log file written by a dedicated
thread, for above-described data.
* New optional parameter, "counters", for server_info and
server_state. If set, render Job Queue and RPC call counters
as well as currently executing tasks.
* New configuration section, "[perf]", to optionally control
performance logging to a file.
* Support optional sub-second periods when rendering human-readable
time points.
Each validator will generate a random cookie on startup that it will
include in each of its validations. This will allow validators to detect
when more than one validator is accidentally operating with the same
validation keys.
* The compiler can provide many non-explicit constructors for
aggregate types. This is sometimes desired, but it can
happen accidentally, resulting in run-time errors.
* This commit assures that no types are aggregates unless existing
code is using aggregate initialization.
* RIPD-1617, RIPD-1619, RIPD-1621:
Verify serialized public keys more strictly before
using them.
* RIPD-1618:
* Simplify the base58 decoder logic.
* Reduce the complexity of the base58 encoder and
eliminate a potential out-of-bounds memory access.
* Improve type safety by using an `enum class` to
enforce strict type checking for token types.
* RIPD-1616:
Avoid calling `memcpy` with a null pointer even if the
size is specified as zero, since it results in undefined
behavior.
Acknowledgements:
Ripple thanks Guido Vranken for responsibly disclosing these
issues.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers
to responsibly disclose any issues that they may find. For
more on Ripple's Bug Bounty program, please visit:
https://ripple.com/bug-bounty
* Rename isArray to isArrayOrNull
* Rename isObject to isObjectOrNull
* Introduce isArray and isObject
* Change as many uses of isArrayorNull to isArray as possible
* Change as many uses of isObjectorNull to isObject as possible
* Reject null JSON arrays for subscribe and unsubscribe
These changes augment the Validations class with a LedgerTrie to better
track the history of support for validated ledgers. This improves the
selection of the preferred working ledger for consensus. The Validations
class now tracks both full and partial validations. Partial validations
are only used to determine the working ledger; full validations are
required for any quorum related function. Validators are also now
explicitly restricted to sending validations with increasing ledger
sequence number.
Do not process a transaction received from a peer if it has
been processed within the past ten seconds.
Increase the number of transaction handlers that can be in
flight in the job queue and decrease the relative cost for
peers to share transaction and ledger data.
Additionally, make better use of resources by adjusting the
number of threads we initialize, by reverting commit
68b8ffdb63.
Performance counter modifications:
* Create and display counters to track:
1) Pending transaction limit overruns.
2) Total peer disconnections.
3) Peers disconnections due to resource consumption.
Avoid a potential double-free in Json library.
Profiling and research indicates that the SQLite query planner executed
our existing SQL queries sub-optimally by not using the index efficiently.
Restructuring the SQL query works around this issue and allows queries
to be executed efficiently and without unnecessary delay.
Do not dispatch a transaction received from a peer for
processing, if it has already been dispatched within the
past ten seconds.
Increase the number of transaction handlers that can be in
flight in the job queue and decrease the relative cost for
peers to share transaction and ledger data.
Additionally, make better use of resources by adjusting the
number of threads we initialize, by reverting commit
68b8ffdb63.
* Stores recent history of "good" ledgers. Uses the maximum as the
expected ledger size. When a large value drops off, use a 90%
backoff to go down to to the new maximum.
* If consensus is unhealthy, wipe the history in addition to the current
clamping.
* Include .md doc files in xcode and VS projects
In support of dynamic validator list, this changeset:
1. Adds a new `validator_list_expires` field to `server_info` that
indicates when the current validator list will become stale.
2. Adds a new admin only `validator_lists` RPC that returns the
current list of known validators and the most recent published validator
lists.
3. Adds a new admin only `validator_sites` RPC that returns the list of
configured validator publisher sites and when they were most recently
queried.
Manifests of validators newly added to a published validator list are
not reliably propagated to network nodes.
This solves the problem by allowing a published validator list to
include the manifest.
RIPD-1559
* If the transaction can't be queued, recover to the open ledger once,
and drop it on the next attempt.
* New result codes for transactions that can not queue.
* Add minimum queue size.
* Remove the obsolete and incorrect SF_RETRY flag.
* fix#2215
When two ledgers have the same number of validations, the code
will now use the ledger hash itself to break the tie rather than
the highest node ID supporting each validation.
* Use fixed size UNL if the total listed validators are below
threshold.
* Set quorum to provide Byzantine fault tolerance until a
threshold of total validators is exceeded, at which time
quorum is 80%.
* Ensure that a quorum of 0 cannot be configured.
The two active users of DeadlineTimer, NetworkOPs and Application,
now use asio::steady_timers rather than DeadlineTimer.
DeadlineTimer is removed since it is no longer used.
To assure that all in-flight closures on timers are done before
Stoppables call stopped(), the JobCounter is made more generic.
It's now a ClosureCounter. The ClosureCounter is currently used
to count closures in flight for the JobQueue, NetworkOPs, and the
Application.
If the JobQueue is used during shutdown then those Jobs may access
Stoppables after they have already stopped. This violates the
preconditions of Stoppables and may lead to undefined behavior.
The solution taken here is to reference count all Jobs in the
JobQueue. At stop time all Jobs already in the JobQueue are
allowed to run to completion, but no further Jobs are allowed
into the JobQueue.
If a Job is rejected from the JobQueue (because we are stopping),
then JobQueue::addJob() returns false, so the caller can make any
necessary adjustments.
Check and modify amendment blocked status with each new ledger (provided
by @wilsonianb). Honor blocked status in certain RPC commands and when
deciding whether to propose/validate.
Fixes: RIPD-1479
Fixes: RIPD-1447
Release Notes
-------------
This resolves an issue whereby an amendment blocked server would still
serve some RPC requests that are unreliable in blocked state and would
continue to publish validations.
Fixes: RIPD-1417
Fix incorrect error case messages. Fix crash in NetworkOps instance when
exiting with remaining RPC subscriptions. Add code to remove URL
subscription when requested.
Introduces a generic Validations class for storing and querying current and
recent validations. Aditionally migrates the validation related timing
constants from LedgerTiming to the new Validations code.
Introduces RCLValidations as the version of Validations adapted for use in the
RCL. This adds support for flushing/writing validations to the sqlite log and
also manages concurrent access to the Validations data.
RCLValidations::flush() no longer uses the JobQueue for its database
write at shutdown. It performs the write directly without
changing threads.
