This commit allows server operators to reserve slots for specific
peers (identified by the peer's public node identity) and to make
changes to the reservations while the server is operating.
This commit closes#2938
At this point all of the jss::* names are defined in the same
file. That file has been named JsonFields.h. That file name
has little to do with either JsonStaticStrings (which is what
jss is short for) or with jss. The file is renamed to jss.h
so the file name better reflects what the file contains.
All includes of that file are fixed. A few include order
issues are tidied up along the way.
* Adds local file:// URL support to the [validator_list_sites] stanza.
The file:// URL must not contain a hostname. Allows a rippled node
operator to "sideload" a new list if their node is unable to reach
a validator list's web site before an old list expires. Lists
loaded from a file will be validated in the same way a downloaded
list is validated.
* Generalize file/dir "guards" from Config test so they can be reused
in other tests.
* Check for error when reading validators.txt. Saves some parsing and
checking of an empty string, and will give a more meaningful error.
* Completes RIPD-1674.
When Ed25519 support was added to ripple-lib, a way to specify
whether a seed should be used to derive a "classic" secp256k1
keypair or a "new" Ed25519 keypair was needed, and the
requirements were that:
1. previously seeds would, correctly, generate a secp256k1
keypair.
2. users would not have to know about whether the seed was
used to generate a secp256k1 or an Ed25519 keypair.
To address these requirements, the decision was made to encode
the type of key within the seed and a custom encoding was
designed.
The encoding uses a token type of 1 and prefixes the actual
seed with a 2 byte header, selected to ensure that all such
keypairs will, when encoded, begin with the string "sEd".
This custom encoding is non-standard and was not previously
documented; as a result, it is not widely supported and other
sofware may treat such keys as invalid. This can make it
difficult for users that have stored such a seed to use
wallets or other tooling that is not based on ripple-lib.
This commit adds support to rippled for automatically
detecting and properly handling such seeds.
The 'validation_seed' RPC command was used to change the validation
key used by a validator at runtime.
Its implementation was commented out with commit fa796a2eb5
which has been included in the codebase since the 0.30.0 release
and there are no plans to reintroduce the functionality at this
point.
Validator operators should migrate to using validator manifests
instead.
This fixes#2748.
Fixes: RIPD-1648
- use ExternalProject for snappy, lz4, SOCI, and sqlite3
- use FetchContent for NuDB
- update SOCI from 79e222e3c2278e6108137a2d26d3689418b37544 to
3a1f602b3021b925d38828e3ff95f9e7f8887ff7
- update lz4 from c10863b98e1503af90616ae99725ecd120265dfb to v1.8.2
- update sqlite3 from 3.21 to 3.24
- update snappy from b02bfa754ebf27921d8da3bd2517eab445b84ff9 to 1.1.7
- update NuDB from 00adc6a4f16679a376f40c967f77dfa544c179c1 to 1.0.0
Switch to target-oriented dependencies. Use imported targets for
dependencies (openssl, boost). Localize FindBoost to remove cmake
version dependence for latest boost support. Logically separate
"ripple-libpp" core sources and add install targets.
Add ninja build for msvc. Add two clang sanitizer builds. Misc script
changes to work with latest modernized cmake.
- Since we require a min Boost version of 1.67 as of recently (for
Beast), we also remove the conditional checks that existed for us
to know whether Boost.Process is available or not. We can
always assume it is available now.
- Remove runtime checks for minimum Boost and OpenSSL versions
since they are checked at CMake configure time.
The lsfDepositAuth flag limits the AccountIDs that can deposit into
the account that has the flag set. The original design only
allowed deposits to complete if the account with the flag set also
signed the transaction that caused the deposit.
The DepositPreauth ledger type allows an account with the
lsfDepositAuth flag set to preauthorize additional accounts.
This preauthorization allows them to sign deposits as well. An
account can add DepositPreauth objects to the ledger (and remove
them as well) using the DepositPreauth transaction.
* UptimeClock is a chrono-compatible seconds-precision clock.
* Like UptimeTimer, its purpose is to make it possible for clients
to query the uptime thousands of times per second without a
significant performance hit.
* UptimeClock decouples itself from LoadManager by managing its
own once-per-second update loop.
* Clients now traffic in chrono time_points and durations instead
of int.
* Tally and duration counters for Job Queue tasks and RPC calls
optionally rendered by server_info and server_state, and
optionally printed to a distinct log file.
- Tally each Job Queue task as it is queued, starts, and
finishes running. Track total duration queued and running.
- Tally each RPC call as it starts and either finishes
successfully or throws an exception. Track total running
duration for each.
* Track currently executing Job Queue tasks and RPC methods
along with durations.
* Json-formatted performance log file written by a dedicated
thread, for above-described data.
* New optional parameter, "counters", for server_info and
server_state. If set, render Job Queue and RPC call counters
as well as currently executing tasks.
* New configuration section, "[perf]", to optionally control
performance logging to a file.
* Support optional sub-second periods when rendering human-readable
time points.
Introduce a new ledger type: ltCHECK
Introduce three new transactions that operate on checks:
- "CheckCreate" which adds the check entry to the ledger. The
check is a promise from the source of the check that the
destination of the check may cash the check and receive up to
the SendMax specified on the check. The check may have an
expiration, after which the check may no longer be cashed.
- "CheckCash" is a request by the destination of the check to
transfer a requested amount of funds, up to the check's SendMax,
from the source to the destination. The destination may receive
less than the SendMax due to transfer fees.
When cashing a check, the destination specifies the smallest
amount of funds that will be acceptable. If the transfer
completes and delivers the requested amount, then the check is
considered cashed and removed from the ledger. If enough funds
cannot be delivered, then the transaction fails and the check
remains in the ledger.
Attempting to cash the check after its expiration will fail.
- "CheckCancel" removes the check from the ledger without
transferring funds. Either the check's source or destination
can cancel the check at any time. After a check has expired,
any account can cancel the check.
Facilities related to checks are on the "Checks" amendment.
* Update unity build for RocksDB changes
* Log RocksDB options on startup
* Support RocksDB option strings
* Support full file bloom filters
You can now configure most RocksDB options with RocksDB's option
string scheme.
Set "filter_full" to 1 to make bloom filters for an
entire file rather than each block. More memory will be
needed during compaction but less memory will be needed
during fetching for large databases. Does nothing unless
bloom filters are enabled with "filter_bits".
Example:
options = max_compaction_bytes=64;max_bytes_for_level_multiplier=64
clock_cache_mb = 96
filter_bits = 10
filter_full = 1
* If any of the destructor, copy assignment or copy constructor
are user-declared, both copy members should be user-declared,
otherwise the compiler-generation of them is deprecated.
Both Tickets and SHAMapV2 have been around for a while and don't
look like they will be enabled on the network soon. So they are
removed from the supportedAmendments list. This prevents Env
from automatically testing with Tickets or SHAMapV2 enabled,
although testing with those features can still be explicitly
specified.
Drive-by cleanups:
o supportedAmendments() returns a const reference rather than
a fresh vector on each call.
o supportedAmendments() implementation moved from Amendments.cpp
to Feature.cpp. Amendments.cpp deleted.
o supportedAmendments() declared in Feature.h. All other
declarations deleted.
o preEnabledAmendments() removed, since it was empty and only
used in one place. It will be easy to re-add when it is needed.
o jtx::all_features_except() renamed to
jtx::supported_features_except(), which is more descriptive.
o jtx::all_amendments() renamed to jxt::supported_amendments()
o jtx::with_features() renamed to with_only_features()
o Env_test.cpp adjusted since featureTickets is no longer
automatically enabled for unit tests.
In support of dynamic validator list, this changeset:
1. Adds a new `validator_list_expires` field to `server_info` that
indicates when the current validator list will become stale.
2. Adds a new admin only `validator_lists` RPC that returns the
current list of known validators and the most recent published validator
lists.
3. Adds a new admin only `validator_sites` RPC that returns the list of
configured validator publisher sites and when they were most recently
queried.
This commit introduces the "SortedDirectories" amendment, which
addresses two distinct issues:
First, it corrects a technical flaw that could, in some edge cases,
prevent an empty intermediate page from being deleted.
Second, it sorts directory entries within a page (other than order
book page entries, which remain strictly FIFO). This makes insert
operations deterministic, instead of pseudo-random and reliant on
temporal ordering.
Lastly, it removes the ability to perform a "soft delete" where
the page number of the item to delete need not be known if the
item is in the first 20 pages, and enforces a maximum limit to
the number of pages that a directory can span.
The two active users of DeadlineTimer, NetworkOPs and Application,
now use asio::steady_timers rather than DeadlineTimer.
DeadlineTimer is removed since it is no longer used.
To assure that all in-flight closures on timers are done before
Stoppables call stopped(), the JobCounter is made more generic.
It's now a ClosureCounter. The ClosureCounter is currently used
to count closures in flight for the JobQueue, NetworkOPs, and the
Application.
Introduces a generic Validations class for storing and querying current and
recent validations. Aditionally migrates the validation related timing
constants from LedgerTiming to the new Validations code.
Introduces RCLValidations as the version of Validations adapted for use in the
RCL. This adds support for flushing/writing validations to the sqlite log and
also manages concurrent access to the Validations data.
RCLValidations::flush() no longer uses the JobQueue for its database
write at shutdown. It performs the write directly without
changing threads.
Replace Taker.cpp with calls to the payment flow() code.
This change required a number of tweaks in the payment flow code.
These tweaks are conditionalized on whether or not offer crossing
is taking place. The flag is explicitly passed as a parameter to
the flow code.
For testing, a class was added that identifies differences in the
contents of two PaymentSandboxes. That code may be reusable in
the future.
None of the Taker offer crossing code is removed. Both versions
of the code are co-resident to support an amendment cut-over.
The code that identifies differences between Taker and Flow offer
crossing is enabled by a feature. That makes it easy to enable
or disable difference logging by changing the config file. This
approach models what was done with the payment flow code. The
differencing code should never be enabled on a production server.
Extensive offer crossing unit tests are added to examine and
verify the behavior of corner cases. The tests are currently
configured to run against both Taker and Flow offer crossing.
This gives us confidence that most cases run identically and
some of the (few) differences in behavior are documented.
Add new functionality to enforce one or more sanity checks (invariants)
on transactions. Add tests for each new invariant check. Allow
for easily adding additional invariant checks in the future.
Also Resolves
-------------
- RIPD-1426
- RIPD-1427
- RIPD-1428
- RIPD-1429
- RIPD-1430
- RIPD-1431
- RIPD-1432
Release Notes
-------------
Creates a new ammendment named "EnforceInvariants" which must be
enabled in order for these new checks to run on each transaction.
This is a substantial refactor of the consensus code and also introduces
a basic consensus simulation and testing framework. The new generic/templated
version is in src/ripple/consensus and documents the current type requirements.
The version adapted for the RCL is in src/ripple/app/consensus. The testing
framework is in src/test/csf.
Minor behavioral changes/fixes include:
* Adjust close time offset even when not validating.
* Remove spurious proposing_ = false call at end of handleLCL.
* Remove unused functionality provided by checkLastValidation.
* Separate open and converge time
* Don't send a bow out if we're not proposing
* Prevent consensus stopping if NetworkOPs switches to disconnect mode while
consensus accepts a ledger
* Prevent a corner case in which Consensus::gotTxSet or Consensus::peerProposal
has the potential to update internal state while an dispatched accept job is
running.
* Distinguish external and internal calls to startNewRound. Only external
calls can reset the proposing_ state of consensus
Escrow replaces the existing SusPay implementation with improved
code that also adds hashlock support to escrow payments, making
RCL ILP enabled.
The new functionality is under the `Escrow` amendment, which
supersedes and replaces the `SusPay` amendment.
This commit also deprecates the `CryptoConditions` amendment
which is replaced by the `CryptoConditionSuite` amendment which,
once enabled, will allow use of cryptoconditions others than
hashlocks.
All uses of beast::Thread were previously removed from the code
base, so beast::Thread is removed. One piece of beast::Thread
needed to be preserved: the ability to set the current thread's
name. So there's now a beast::CurrentThreadName that allows the
current thread's name to be set and returned.
Thread naming is also cleaned up a bit. ThreadName.h and .cpp
are removed since beast::CurrentThreadName does a better job.
ThreadEntry is also removed, but its terminateHandler() is
preserved in TerminateHandler.cpp. The revised terminateHandler()
uses beast::CurrentThreadName to recover the name of the running
thread.
Finally, the NO_LOG_UNHANDLED_EXCEPTIONS #define is removed since
it was discovered that the MacOS debugger preserves the stack
of the original throw even if the terminateHandler() rethrows.
Validator lists from configured remote sites are fetched at a regular
interval. Fetched lists are expected to be in JSON format and contain the
following fields:
* "manifest": Base64-encoded serialization of a manifest containing the
validator publisher's master and signing public keys.
* "blob": Base64-encoded JSON string containing a "sequence",
"expiration" and "validators" field. "expiration" contains the Ripple
timestamp (seconds since January 1st, 2000 (00:00 UTC)) for when the
list expires. "validators" contains an array of objects with a
"validation_public_key" field.
* "signature": Hex-encoded signature of the blob using the publisher's
signing key.
* "version": 1
* "refreshInterval" (optional)
