These changes use the hash of the consensus transaction set when
characterizing the mismatch between a locally built ledger and fully
validated network ledger. This allows detection of non-determinism in
transaction process, in which consensus succeeded, but a node somehow
generated a different subsequent ledger.
The assert in LedgerHistory.cpp was incorrect since
getLedgerByHash() can return a null object. The assert has
been seen to fire inappropriately during online_delete.
The assert in LedgerMaster.cpp was simply misguided. If a
null hash were returned by getLedgerHashForHistory(), then we
should not forward that hash to peers. The assert only fires
in a debug build, so a release build would forward the request
for a null hash to peers.
If the JobQueue is used during shutdown then those Jobs may access
Stoppables after they have already stopped. This violates the
preconditions of Stoppables and may lead to undefined behavior.
The solution taken here is to reference count all Jobs in the
JobQueue. At stop time all Jobs already in the JobQueue are
allowed to run to completion, but no further Jobs are allowed
into the JobQueue.
If a Job is rejected from the JobQueue (because we are stopping),
then JobQueue::addJob() returns false, so the caller can make any
necessary adjustments.
Check and modify amendment blocked status with each new ledger (provided
by @wilsonianb). Honor blocked status in certain RPC commands and when
deciding whether to propose/validate.
Fixes: RIPD-1479
Fixes: RIPD-1447
Release Notes
-------------
This resolves an issue whereby an amendment blocked server would still
serve some RPC requests that are unreliable in blocked state and would
continue to publish validations.
Introduces a generic Validations class for storing and querying current and
recent validations. Aditionally migrates the validation related timing
constants from LedgerTiming to the new Validations code.
Introduces RCLValidations as the version of Validations adapted for use in the
RCL. This adds support for flushing/writing validations to the sqlite log and
also manages concurrent access to the Validations data.
RCLValidations::flush() no longer uses the JobQueue for its database
write at shutdown. It performs the write directly without
changing threads.
This is a substantial refactor of the consensus code and also introduces
a basic consensus simulation and testing framework. The new generic/templated
version is in src/ripple/consensus and documents the current type requirements.
The version adapted for the RCL is in src/ripple/app/consensus. The testing
framework is in src/test/csf.
Minor behavioral changes/fixes include:
* Adjust close time offset even when not validating.
* Remove spurious proposing_ = false call at end of handleLCL.
* Remove unused functionality provided by checkLastValidation.
* Separate open and converge time
* Don't send a bow out if we're not proposing
* Prevent consensus stopping if NetworkOPs switches to disconnect mode while
consensus accepts a ledger
* Prevent a corner case in which Consensus::gotTxSet or Consensus::peerProposal
has the potential to update internal state while an dispatched accept job is
running.
* Distinguish external and internal calls to startNewRound. Only external
calls can reset the proposing_ state of consensus
Instead of specifying a static list of trusted validators in the config
or validators file, the configuration can now include trusted validator
list publisher keys.
The trusted validator list and quorum are now reset each consensus
round using the latest validator lists and the list of recent
validations seen. The minimum validation quorum is now only
configurable via the command line.
* Use std::mutex instead of std::recursive_mutex
* Remove unnecessary type alias
* Use std::set instead of ripple::hash_map
* Don't reinvent virtual functions
* Revert 0efb929898
* Advisory delete setting of 0 (never) does not affect history fetching
The previous commit addressing RIPD-1112 could interact with
advisory delete and cause some history not to be acquired even
configured to acquire. This reverts that commit and provides
a better fix.
The advisory delete setting protects ledgers from being
removed by online delete by exempting them until they are
approved for purge by administrative command. However, not
connecting this with history acquisition could cause new
ledgers in the protected range not to be acquired if the
server loses sync.
With this change, the default advisory delete setting, zero (never)
causes the regular server history setting to control the acquisition
of history. Setting advisory delete to a value greater than zero,
if advisory delete is enabled, will cause the server to fetch and
maintain history back to that point.
This should produce sane behavior across server restarts, losses of
sync, and so on. You can no longer use the "hack" of setting
advisory delete to zero to tell the server to fetch and keep as much
history as possible, but you can achieve the same effect by setting
it to one.
Replace Journal public data members with member function accessors
in order to make Journal lighter weight. The change makes a
Journal cheaper to pass by value.
Also add missing stream checks (e.g., calls to JLOG) to avoid
text processing that ultimately will not be stored in the log.
* Add Validations.LedgerSeq and .InitialSeq fields.
* Clean up logging.
* Lower online delete minimum for standalone mode.
* Unit tests of online_delete.
* After successfully applying a transaction to the open ledger, resubmit any held transactions from the same account.
* All held transactions will continue to be retried after consensus round.
* Expire validations faster based on when we first saw them.
* Never jump to a ledger prior to the latest fully-valid ledger
* Drop validations with signing times too far in the future immediately
Add a LedgerMaster function to get a ledger's
close time from either its hash or sequence number.
Use this function when adding the 'date' fields to
transaction JSON. This avoids constructing large numbers
of ledgers.
If we built a different ledger from the one we ultimately
validate, log the status of the consensus round. This will
make it easier to rule out transaction processing issues
as the cause of these discrepancies and generally make them
easier to diagnose.
* Move InboundTransactions to app/ledger
* Move TransactionAcquire to app/ledger
* Move LocalTxs to app/ledger
* Move Transaction to app/misc
* Move TransactionMaster to app/ledger
The first few transactions are added to the open ledger at
the base fee (ie. 10 drops). Once enough transactions are
added, the required fee will jump dramatically. If additional
transactions are added, the fee will grow exponentially.
Transactions that don't have a high enough fee to be applied to
the ledger are added to the queue in order from highest fee to
lowest. Whenever a new ledger is accepted as validated, transactions
are first applied from the queue to the open ledger in fee order
until either all transactions are applied or the fee again jumps
too high for the remaining transactions.
Current implementation is restricted to one transaction in the
queue per account. Some groundwork has been laid to expand in
the future.
Note that this fee logic escalates independently of the load-based
fee logic (ie. LoadFeeTrack). Submitted transactions must meet
the load fee to be considered for the queue, and must meet both
fees to be put into open ledger.
* Remove cxx14 compatibility layer from ripple
* Update travis to clang 3.6 and drop gcc 4.8
* Remove unneeded beast CXX14 defines
* Do not run clang build with gdb with travis
* Update circle ci to clang 3.6 & gcc-5
* Don't run rippled in gdb, clang builds crash gdb
* Staticly link libstdc++, boost, ssl, & protobuf
* Support builds on ubuntu 15.10
* All checks flow through ripple::checkValidity, which transparently caches result flags.
* All external transaction submission code paths use checkValidity.
* SF_SIGGOOD flag no longer appears outside of HashRouter / checkValidity.
* Validity can be forced in known or trusted scenarios.
* Sanely handled specified ledger in account_tx
* Reject un-validated ledger in account_tx
* Wait to publish a ledger until it's indexed
* Add unit test for PendingSaves
* Remove ripple::RippleMutex and ripple::RippleRecursiveMutex
and use std::mutex and std::recursive_mutex respectively.
* Use std::lock_guard instead of std::unique_lock when the
additional features of std::unique_lock are not needed.
* Avoid throwing in OrderBookDB::processTxn
* Fix missing space in debug output
* Avoid duplicate lock of PathRequest in updateAll
* Avoid shadowing in insertPathRequest
* Improve indentation in runOnCoroutine
* Remove extraneous space in ServerHandlerImp::processRequest
