Dynamize trusted validator list and quorum (RIPD-1220):

Instead of specifying a static list of trusted validators in the config
or validators file, the configuration can now include trusted validator
list publisher keys.

The trusted validator list and quorum are now reset each consensus
round using the latest validator lists and the list of recent
validations seen. The minimum validation quorum is now only
configurable via the command line.
This commit is contained in:
wilsonianb
2016-08-30 09:46:24 -07:00
committed by seelabs
parent 74977ab3db
commit e823e60ca0
42 changed files with 2482 additions and 1570 deletions

View File

@@ -0,0 +1,313 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012, 2013 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_APP_MISC_MANIFEST_H_INCLUDED
#define RIPPLE_APP_MISC_MANIFEST_H_INCLUDED
#include <ripple/basics/UnorderedContainers.h>
#include <ripple/protocol/PublicKey.h>
#include <ripple/beast/utility/Journal.h>
#include <boost/optional.hpp>
#include <string>
namespace ripple {
/*
Validator key manifests
-----------------------
Suppose the secret keys installed on a Ripple validator are compromised. Not
only do you have to generate and install new key pairs on each validator,
EVERY rippled needs to have its config updated with the new public keys, and
is vulnerable to forged validation signatures until this is done. The
solution is a new layer of indirection: A master secret key under
restrictive access control is used to sign a "manifest": essentially, a
certificate including the master public key, an ephemeral public key for
verifying validations (which will be signed by its secret counterpart), a
sequence number, and a digital signature.
The manifest has two serialized forms: one which includes the digital
signature and one which doesn't. There is an obvious causal dependency
relationship between the (latter) form with no signature, the signature
of that form, and the (former) form which includes that signature. In
other words, a message can't contain a signature of itself. The code
below stores a serialized manifest which includes the signature, and
dynamically generates the signatureless form when it needs to verify
the signature.
There are two stores of information within rippled related to manifests.
An instance of ManifestCache stores, for each trusted validator, (a) its
master public key, and (b) the most senior of all valid manifests it has
seen for that validator, if any. On startup, the [validation_manifest]
config entry (which is the manifest for this validator) is decoded and
added to the manifest cache. Other manifests are added as "gossip" is
received from rippled peers.
The other data store (which does not involve manifests per se) contains
the set of active ephemeral validator keys. Keys are added to the set
when a manifest is accepted, and removed when that manifest is obsoleted.
When an ephemeral key is compromised, a new signing key pair is created,
along with a new manifest vouching for it (with a higher sequence number),
signed by the master key. When a rippled peer receives the new manifest,
it verifies it with the master key and (assuming it's valid) discards the
old ephemeral key and stores the new one. If the master key itself gets
compromised, a manifest with sequence number 0xFFFFFFFF will supersede a
prior manifest and discard any existing ephemeral key without storing a
new one. Since no further manifests for this master key will be accepted
(since no higher sequence number is possible), and no signing key is on
record, no validations will be accepted from the compromised validator.
*/
//------------------------------------------------------------------------------
struct Manifest
{
static std::size_t constexpr textLength = 288;
std::string serialized;
PublicKey masterKey;
PublicKey signingKey;
std::uint32_t sequence;
Manifest(std::string s, PublicKey pk, PublicKey spk, std::uint32_t seq);
Manifest(Manifest&& other) = default;
Manifest& operator=(Manifest&& other) = default;
inline bool
operator==(Manifest const& rhs) const
{
return sequence == rhs.sequence && masterKey == rhs.masterKey &&
signingKey == rhs.signingKey && serialized == rhs.serialized;
}
inline bool
operator!=(Manifest const& rhs) const
{
return !(*this == rhs);
}
/** Constructs Manifest from serialized string
@param s Serialized manifest string
@return `boost::none` if string is invalid
*/
static boost::optional<Manifest> make_Manifest(std::string s);
/// Returns `true` if manifest signature is valid
bool verify () const;
/// Returns hash of serialized manifest data
uint256 hash () const;
/// Returns `true` if manifest revokes master key
bool revoked () const;
/// Returns manifest signature
Blob getSignature () const;
/// Returns manifest master key signature
Blob getMasterSignature () const;
};
enum class ManifestDisposition
{
/// Manifest is valid
accepted = 0,
/// Sequence is too old
stale,
/// Timely, but invalid signature
invalid
};
class DatabaseCon;
/** Remembers manifests with the highest sequence number. */
class ManifestCache
{
private:
beast::Journal mutable j_;
std::mutex apply_mutex_;
std::mutex mutable read_mutex_;
/** Active manifests stored by master public key. */
hash_map <PublicKey, Manifest> map_;
/** Master public keys stored by current ephemeral public key. */
hash_map <PublicKey, PublicKey> signingToMasterKeys_;
public:
explicit
ManifestCache (beast::Journal j = beast::Journal())
: j_ (j)
{
};
/** Returns master key's current signing key.
@param pk Master public key
@return pk if no known signing key from a manifest
@par Thread Safety
May be called concurrently
*/
PublicKey
getSigningKey (PublicKey const& pk) const;
/** Returns ephemeral signing key's master public key.
@param pk Ephemeral signing public key
@return pk if signing key is not in a valid manifest
@par Thread Safety
May be called concurrently
*/
PublicKey
getMasterKey (PublicKey const& pk) const;
/** Returns `true` if master key has been revoked in a manifest.
@param pk Master public key
@par Thread Safety
May be called concurrently
*/
bool
revoked (PublicKey const& pk) const;
/** Add manifest to cache.
@param m Manifest to add
@return `ManifestDisposition::accepted` if successful, or
`stale` or `invalid` otherwise
@par Thread Safety
May be called concurrently
*/
ManifestDisposition
applyManifest (
Manifest m);
/** Populate manifest cache with manifests in database and config.
@param dbCon Database connection with dbTable
@param dbTable Database table
@param configManifest Base64 encoded manifest for local node's
validator keys
@par Thread Safety
May be called concurrently
*/
bool load (
DatabaseCon& dbCon, std::string const& dbTable,
std::vector<std::string> const& configManifest);
/** Populate manifest cache with manifests in database.
@param dbCon Database connection with dbTable
@param dbTable Database table
@par Thread Safety
May be called concurrently
*/
void load (
DatabaseCon& dbCon, std::string const& dbTable);
/** Save cached manifests to database.
@param dbCon Database connection with `ValidatorManifests` table
@param isTrusted Function that returns true if manifest is trusted
@par Thread Safety
May be called concurrently
*/
void save (
DatabaseCon& dbCon, std::string const& dbTable,
std::function <bool (PublicKey const&)> isTrusted);
/** Invokes the callback once for every populated manifest.
@note Undefined behavior results when calling ManifestCache members from
within the callback
@param f Function called for each manifest
@par Thread Safety
May be called concurrently
*/
template <class Function>
void
for_each_manifest(Function&& f) const
{
std::lock_guard<std::mutex> lock{read_mutex_};
for (auto const& m : map_)
{
f(m.second);
}
}
/** Invokes the callback once for every populated manifest.
@note Undefined behavior results when calling ManifestCache members from
within the callback
@param pf Pre-function called with the maximum number of times f will be
called (useful for memory allocations)
@param f Function called for each manifest
@par Thread Safety
May be called concurrently
*/
template <class PreFun, class EachFun>
void
for_each_manifest(PreFun&& pf, EachFun&& f) const
{
std::lock_guard<std::mutex> lock{read_mutex_};
pf(map_.size ());
for (auto const& m : map_)
{
f(m.second);
}
}
};
} // ripple
#endif

View File

@@ -1202,8 +1202,8 @@ public:
if (nodesUsing > v.nodesUsing)
return true;
if (nodesUsing < v.nodesUsing) return
false;
if (nodesUsing < v.nodesUsing)
return false;
return highNodeUsing > v.highNodeUsing;
}
@@ -1485,6 +1485,9 @@ bool NetworkOPsImp::beginConsensus (uint256 const& networkClosed)
assert (closingInfo.parentHash ==
m_ledgerMaster.getClosedLedger()->info().hash);
app_.validators().onConsensusStart (
app_.getValidations().getCurrentPublicKeys ());
mConsensus->startRound (
*mLedgerConsensus,
networkClosed,
@@ -2032,7 +2035,8 @@ Json::Value NetworkOPsImp::getServerInfo (bool human, bool admin)
if (mNeedNetworkLedger)
info[jss::network_ledger] = "waiting";
info[jss::validation_quorum] = m_ledgerMaster.getMinValidations ();
info[jss::validation_quorum] = static_cast<Json::UInt>(
app_.validators ().quorum ());
info[jss::io_latency_ms] = static_cast<Json::UInt> (
app_.getIOLatency().count());
@@ -2050,7 +2054,7 @@ Json::Value NetworkOPsImp::getServerInfo (bool human, bool admin)
s.reserve (Manifest::textLength);
for (auto const& line : validation_manifest.lines())
s += beast::rfc2616::trim(line);
if (auto mo = make_Manifest (beast::detail::base64_decode(s)))
if (auto mo = Manifest::make_Manifest (beast::detail::base64_decode(s)))
{
Json::Value valManifest = Json::objectValue;
valManifest [jss::master_key] = toBase58 (

View File

@@ -88,9 +88,12 @@ private:
bool addValidation (STValidation::ref val, std::string const& source) override
{
auto signer = val->getSignerPublic ();
auto hash = val->getLedgerHash ();
bool isCurrent = current (val);
if (!val->isTrusted() && app_.validators().trusted (signer))
auto pubKey = app_.validators ().getTrustedKey (signer);
if (!val->isTrusted() && pubKey)
val->setTrusted();
if (!val->isTrusted ())
@@ -98,45 +101,71 @@ private:
JLOG (j_.trace()) <<
"Node " << toBase58 (TokenType::TOKEN_NODE_PUBLIC, signer) <<
" not in UNL st=" << val->getSignTime().time_since_epoch().count() <<
", hash=" << val->getLedgerHash () <<
", hash=" << hash <<
", shash=" << val->getSigningHash () <<
" src=" << source;
}
auto hash = val->getLedgerHash ();
auto node = val->getNodeID ();
if (! pubKey)
pubKey = app_.validators ().getListedKey (signer);
if (val->isTrusted () && isCurrent)
if (isCurrent &&
(val->isTrusted () || pubKey))
{
ScopedLockType sl (mLock);
if (!findCreateSet (hash)->insert (std::make_pair (node, val)).second)
if (!findCreateSet (hash)->insert (
std::make_pair (*pubKey, val)).second)
return false;
auto it = mCurrentValidations.find (node);
auto it = mCurrentValidations.find (*pubKey);
if (it == mCurrentValidations.end ())
{
// No previous validation from this validator
mCurrentValidations.emplace (node, val);
mCurrentValidations.emplace (*pubKey, val);
}
else if (!it->second)
{
// Previous validation has expired
it->second = val;
}
else if (val->getSignTime () > it->second->getSignTime ())
{
// This is a newer validation
val->setPreviousHash (it->second->getLedgerHash ());
mStaleValidations.push_back (it->second);
it->second = val;
condWrite ();
}
else
{
// We already have a newer validation from this source
isCurrent = false;
auto const oldSeq = (*it->second)[~sfLedgerSequence];
auto const newSeq = (*val)[~sfLedgerSequence];
if (oldSeq && newSeq && *oldSeq == *newSeq)
{
JLOG (j_.warn()) <<
"Trusted node " <<
toBase58 (TokenType::TOKEN_NODE_PUBLIC, *pubKey) <<
" published multiple validations for ledger " <<
*oldSeq;
// Remove current validation for the revoked signing key
if (signer != it->second->getSignerPublic())
{
auto set = findSet (it->second->getLedgerHash ());
if (set)
set->erase (*pubKey);
}
}
if (val->getSignTime () > it->second->getSignTime () ||
signer != it->second->getSignerPublic())
{
// This is either a newer validation or a new signing key
val->setPreviousHash (it->second->getLedgerHash ());
mStaleValidations.push_back (it->second);
it->second = val;
condWrite ();
}
else
{
// We already have a newer validation from this source
isCurrent = false;
}
}
}
@@ -185,9 +214,10 @@ private:
(val->getSeenTime() < (now + VALIDATION_VALID_LOCAL)));
}
int getTrustedValidationCount (uint256 const& ledger) override
std::size_t
getTrustedValidationCount (uint256 const& ledger) override
{
int trusted = 0;
std::size_t trusted = 0;
ScopedLockType sl (mLock);
auto set = findSet (ledger);
@@ -292,6 +322,37 @@ private:
return ret;
}
hash_set<PublicKey> getCurrentPublicKeys () override
{
hash_set<PublicKey> ret;
ScopedLockType sl (mLock);
auto it = mCurrentValidations.begin ();
while (it != mCurrentValidations.end ())
{
if (!it->second) // contains no record
it = mCurrentValidations.erase (it);
else if (! current (it->second))
{
// contains a stale record
mStaleValidations.push_back (it->second);
it->second.reset ();
condWrite ();
it = mCurrentValidations.erase (it);
}
else
{
// contains a live record
ret.insert (it->first);
++it;
}
}
return ret;
}
LedgerToValidationCounter getCurrentValidations (
uint256 currentLedger,
uint256 priorLedger,
@@ -317,6 +378,8 @@ private:
condWrite ();
it = mCurrentValidations.erase (it);
}
else if (! it->second->isTrusted())
++it;
else if (! it->second->isFieldPresent (sfLedgerSequence) ||
(it->second->getFieldU32 (sfLedgerSequence) >= cutoffBefore))
{

View File

@@ -31,7 +31,7 @@ namespace ripple {
// VFALCO TODO rename and move these type aliases into the Validations interface
// nodes validating and highest node ID validating
using ValidationSet = hash_map<NodeID, STValidation::pointer>;
using ValidationSet = hash_map<PublicKey, STValidation::pointer>;
using ValidationCounter = std::pair<int, NodeID>;
using LedgerToValidationCounter = hash_map<uint256, ValidationCounter>;
@@ -47,7 +47,7 @@ public:
virtual ValidationSet getValidations (uint256 const& ledger) = 0;
virtual int getTrustedValidationCount (uint256 const& ledger) = 0;
virtual std::size_t getTrustedValidationCount (uint256 const& ledger) = 0;
/** Returns fees reported by trusted validators in the given ledger. */
virtual
@@ -57,6 +57,8 @@ public:
virtual int getNodesAfter (uint256 const& ledger) = 0;
virtual int getLoadRatio (bool overLoaded) = 0;
virtual hash_set<PublicKey> getCurrentPublicKeys () = 0;
// VFALCO TODO make a type alias for this ugly return value!
virtual LedgerToValidationCounter getCurrentValidations (
uint256 currentLedger, uint256 previousLedger,

View File

@@ -20,102 +20,437 @@
#ifndef RIPPLE_APP_MISC_VALIDATORLIST_H_INCLUDED
#define RIPPLE_APP_MISC_VALIDATORLIST_H_INCLUDED
#include <ripple/basics/BasicConfig.h>
#include <ripple/app/misc/Manifest.h>
#include <ripple/basics/Log.h>
#include <ripple/basics/UnorderedContainers.h>
#include <ripple/core/TimeKeeper.h>
#include <ripple/crypto/csprng.h>
#include <ripple/protocol/PublicKey.h>
#include <boost/optional.hpp>
#include <functional>
#include <memory>
#include <boost/iterator/counting_iterator.hpp>
#include <boost/range/adaptors.hpp>
#include <boost/thread/locks.hpp>
#include <boost/thread/shared_mutex.hpp>
#include <mutex>
#include <numeric>
namespace ripple {
enum class ListDisposition
{
/// List is valid
accepted = 0,
/// List version is not supported
unsupported_version,
/// List signed by untrusted publisher key
untrusted,
/// Trusted publisher key, but seq is too old
stale,
/// Invalid format or signature
invalid,
};
/**
Trusted Validators List
-----------------------
Rippled accepts ledger proposals and validations from trusted validator
nodes. A ledger is considered fully-validated once the number of received
trusted validations for a ledger meets or exceeds a quorum value.
This class manages the set of validation public keys the local rippled node
trusts. The list of trusted keys is populated using the keys listed in the
configuration file as well as lists signed by trusted publishers. The
trusted publisher public keys are specified in the config.
New lists are expected to include the following data:
@li @c "blob": Base64-encoded JSON string containing a @c "sequence", @c
"expiration", and @c "validators" field. @c "expiration" contains the
Ripple timestamp (seconds since January 1st, 2000 (00:00 UTC)) for when
the list expires. @c "validators" contains an array of objects with a
@c "validation_public_key" field.
@c "validation_public_key" should be the hex-encoded master public key.
@li @c "manifest": Base64-encoded serialization of a manifest containing the
publisher's master and signing public keys.
@li @c "signature": Hex-encoded signature of the blob using the publisher's
signing key.
@li @c "version": 1
Individual validator lists are stored separately by publisher. The number of
lists on which a validator's public key appears is also tracked.
The list of trusted validation public keys is reset at the start of each
consensus round to take into account the latest known lists as well as the
set of validators from whom validations are being received. Listed
validation public keys are shuffled and then sorted by the number of lists
they appear on. (The shuffling makes the order/rank of validators with the
same number of listings non-deterministic.) A quorum value is calculated for
the new trusted validator list. If there is only one list, all listed keys
are trusted. Otherwise, the trusted list size is set to 125% of the quorum.
*/
class ValidatorList
{
private:
/** The non-ephemeral public keys from the configuration file. */
hash_map<PublicKey, std::string> permanent_;
struct PublisherList
{
bool available;
std::vector<PublicKey> list;
std::size_t sequence;
std::size_t expiration;
};
/** The ephemeral public keys from manifests. */
hash_map<PublicKey, std::string> ephemeral_;
ManifestCache& validatorManifests_;
ManifestCache& publisherManifests_;
TimeKeeper& timeKeeper_;
beast::Journal j_;
boost::shared_mutex mutable mutex_;
std::mutex mutable mutex_;
beast::Journal mutable j_;
std::atomic<std::size_t> quorum_;
boost::optional<std::size_t> minimumQuorum_;
// Published lists stored by publisher master public key
hash_map<PublicKey, PublisherList> publisherLists_;
// Listed master public keys with the number of lists they appear on
hash_map<PublicKey, std::size_t> keyListings_;
// The current list of trusted master keys
hash_set<PublicKey> trustedKeys_;
PublicKey localPubKey_;
public:
explicit
ValidatorList (beast::Journal j);
ValidatorList (
ManifestCache& validatorManifests,
ManifestCache& publisherManifests,
TimeKeeper& timeKeeper,
beast::Journal j,
boost::optional<std::size_t> minimumQuorum = boost::none);
~ValidatorList ();
/** Determines whether a node is in the UNL
@return boost::none if the node isn't a member,
otherwise, the comment associated with the
node (which may be an empty string).
/** Load configured trusted keys.
@param localSigningKey This node's validation public key
@param configKeys List of trusted keys from config. Each entry consists
of a base58 encoded validation public key, optionally followed by a
comment.
@param publisherKeys List of trusted publisher public keys. Each entry
contains a base58 encoded account public key.
@par Thread Safety
May be called concurrently
@return `false` if an entry is invalid or unparsable
*/
boost::optional<std::string>
member (
PublicKey const& identity) const;
bool
load (
PublicKey const& localSigningKey,
std::vector<std::string> const& configKeys,
std::vector<std::string> const& publisherKeys);
/** Determines whether a node is in the UNL */
/** Apply published list of public keys
@param manifest base64-encoded publisher key manifest
@param blob base64-encoded json containing published validator list
@param signature Signature of the decoded blob
@param version Version of published list format
@return `ListDisposition::accepted` if list was successfully applied
@par Thread Safety
May be called concurrently
*/
ListDisposition
applyList (
std::string const& manifest,
std::string const& blob,
std::string const& signature,
std::uint32_t version);
/** Update trusted keys
Reset the trusted keys based on latest manifests, received validations,
and lists.
@param seenValidators Set of public keys used to sign recently
received validations
@par Thread Safety
May be called concurrently
*/
template<class KeySet>
void
onConsensusStart (
KeySet const& seenValidators);
/** Get quorum value for current trusted key set
The quorum is the minimum number of validations needed for a ledger to
be fully validated. It can change when the set of trusted validation
keys is updated (at the start of each consensus round) and primarily
depends on the number of trusted keys.
@par Thread Safety
May be called concurrently
@return quorum value
*/
std::size_t
quorum () const
{
return quorum_;
};
/** Returns `true` if public key is trusted
@param identity Validation public key
@par Thread Safety
May be called concurrently
*/
bool
trusted (
PublicKey const& identity) const;
/** Insert a short-term validator key published in a manifest. */
/** Returns `true` if public key is included on any lists
@param identity Validation public key
@par Thread Safety
May be called concurrently
*/
bool
insertEphemeralKey (
PublicKey const& identity,
std::string const& comment);
listed (
PublicKey const& identity) const;
/** Remove a short-term validator revoked in a manifest. */
/** Returns master public key if public key is trusted
@param identity Validation public key
@return `boost::none` if key is not trusted
@par Thread Safety
May be called concurrently
*/
boost::optional<PublicKey>
getTrustedKey (
PublicKey const& identity) const;
/** Returns listed master public if public key is included on any lists
@param identity Validation public key
@return `boost::none` if key is not listed
@par Thread Safety
May be called concurrently
*/
boost::optional<PublicKey>
getListedKey (
PublicKey const& identity) const;
/** Returns `true` if public key is a trusted publisher
@param identity Publisher public key
@par Thread Safety
May be called concurrently
*/
bool
removeEphemeralKey (
PublicKey const& identity);
trustedPublisher (
PublicKey const& identity) const;
/** Insert a long-term validator key. */
bool
insertPermanentKey (
PublicKey const& identity,
std::string const& comment);
/** Invokes the callback once for every listed validation public key.
/** Remove a long-term validator key. */
bool
removePermanentKey (
PublicKey const& identity);
/** The number of installed permanent and ephemeral keys */
std::size_t
size () const;
/** Invokes the callback once for every node in the UNL.
@note You are not allowed to insert or remove any
nodes in the UNL from within the callback.
@note Undefined behavior results when calling ValidatorList members from
within the callback
The arguments passed into the lambda are:
- The public key of the validator;
- A (possibly empty) comment.
- A boolean indicating whether this is a
permanent or ephemeral key;
@li The validation public key
@li A boolean indicating whether this is a trusted key
@par Thread Safety
May be called concurrently
*/
void
for_each (
std::function<void(PublicKey const&, std::string const&, bool)> func) const;
for_each_listed (
std::function<void(PublicKey const&, bool)> func) const;
/** Load the list of trusted validators.
private:
/** Check response for trusted valid published list
The section contains entries consisting of a base58
encoded validator public key, optionally followed by
a comment.
@return `ListDisposition::accepted` if list can be applied
@return false if an entry could not be parsed or
contained an invalid validator public key,
true otherwise.
@par Thread Safety
Calling public member function is expected to lock mutex
*/
ListDisposition
verify (
Json::Value& list,
PublicKey& pubKey,
std::string const& manifest,
std::string const& blob,
std::string const& signature);
/** Stop trusting publisher's list of keys.
@param publisherKey Publisher public key
@return `false` if key was not trusted
@par Thread Safety
Calling public member function is expected to lock mutex
*/
bool
load (
Section const& validators);
removePublisherList (PublicKey const& publisherKey);
};
//------------------------------------------------------------------------------
template<class KeySet>
void
ValidatorList::onConsensusStart (
KeySet const& seenValidators)
{
boost::unique_lock<boost::shared_mutex> lock{mutex_};
// Check that lists from all configured publishers are available
bool allListsAvailable = true;
for (auto const& list : publisherLists_)
{
// Remove any expired published lists
if (list.second.expiration &&
list.second.expiration <=
timeKeeper_.now().time_since_epoch().count())
removePublisherList (list.first);
else if (! list.second.available)
allListsAvailable = false;
}
std::multimap<std::size_t, PublicKey> rankedKeys;
bool localKeyListed = false;
// "Iterate" the listed keys in random order so that the rank of multiple
// keys with the same number of listings is not deterministic
std::vector<std::size_t> indexes (keyListings_.size());
std::iota (indexes.begin(), indexes.end(), 0);
std::shuffle (indexes.begin(), indexes.end(), crypto_prng());
for (auto const& index : indexes)
{
auto const& val = std::next (keyListings_.begin(), index);
if (validatorManifests_.revoked (val->first))
continue;
if (val->first == localPubKey_)
{
localKeyListed = val->second > 1;
rankedKeys.insert (
std::pair<std::size_t,PublicKey>(
std::numeric_limits<std::size_t>::max(), localPubKey_));
}
// If no validations are being received, use all validators.
// Otherwise, do not use validators whose validations aren't being received
else if (seenValidators.empty() ||
seenValidators.find (val->first) != seenValidators.end ())
{
rankedKeys.insert (
std::pair<std::size_t,PublicKey>(val->second, val->first));
}
}
// This quorum guarantees sufficient overlap with the trusted sets of other
// nodes using the same set of published lists.
std::size_t quorum = keyListings_.size()/2 + 1;
// Increment the quorum to prevent two unlisted validators using the same
// even number of listed validators from forking.
if (localPubKey_.size() && ! localKeyListed &&
rankedKeys.size () > 1 && keyListings_.size () % 2 != 0)
++quorum;
JLOG (j_.debug()) <<
rankedKeys.size() << " of " << keyListings_.size() <<
" listed validators eligible for inclusion in the trusted set";
auto size = rankedKeys.size();
// Use all eligible keys if there is only one trusted list.
if (publisherLists_.size() == 1)
{
// Raise the quorum to 80% of the trusted set
std::size_t const targetQuorum = std::ceil (size * 0.8);
if (targetQuorum > quorum)
quorum = targetQuorum;
}
else
{
// reduce the trusted set size so that the quorum represents
// at least 80%
size = quorum * 1.25;
}
if (minimumQuorum_ && (seenValidators.empty() ||
rankedKeys.size() < quorum))
quorum = *minimumQuorum_;
// Do not use achievable quorum until lists from all configured
// publishers are available
else if (! allListsAvailable)
quorum = std::numeric_limits<std::size_t>::max();
trustedKeys_.clear();
quorum_ = quorum;
for (auto const& val : boost::adaptors::reverse (rankedKeys))
{
if (size <= trustedKeys_.size())
break;
trustedKeys_.insert (val.second);
}
JLOG (j_.debug()) <<
"Using quorum of " << quorum_ << " for new set of " <<
trustedKeys_.size() << " trusted validators";
if (trustedKeys_.size() < quorum_)
{
JLOG (j_.warn()) <<
"New quorum of " << quorum_ <<
" exceeds the number of trusted validators (" <<
trustedKeys_.size() << ")";
}
}
} // ripple
#endif

View File

@@ -0,0 +1,373 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012, 2013 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <ripple/app/misc/Manifest.h>
#include <ripple/basics/contract.h>
#include <ripple/basics/Log.h>
#include <ripple/beast/rfc2616.h>
#include <ripple/core/DatabaseCon.h>
#include <ripple/protocol/PublicKey.h>
#include <ripple/protocol/Sign.h>
#include <beast/core/detail/base64.hpp>
#include <stdexcept>
namespace ripple {
boost::optional<Manifest>
Manifest::make_Manifest (std::string s)
{
try
{
STObject st (sfGeneric);
SerialIter sit (s.data (), s.size ());
st.set (sit);
auto const opt_pk = get<PublicKey>(st, sfPublicKey);
auto const opt_spk = get<PublicKey>(st, sfSigningPubKey);
auto const opt_seq = get (st, sfSequence);
auto const opt_sig = get (st, sfSignature);
auto const opt_msig = get (st, sfMasterSignature);
if (!opt_pk || !opt_spk || !opt_seq || !opt_sig || !opt_msig)
{
return boost::none;
}
return Manifest (std::move (s), *opt_pk, *opt_spk, *opt_seq);
}
catch (std::exception const&)
{
return boost::none;
}
}
template<class Stream>
Stream&
logMftAct (
Stream& s,
std::string const& action,
PublicKey const& pk,
std::uint32_t seq)
{
s << "Manifest: " << action <<
";Pk: " << toBase58 (TokenType::TOKEN_NODE_PUBLIC, pk) <<
";Seq: " << seq << ";";
return s;
}
template<class Stream>
Stream& logMftAct (
Stream& s,
std::string const& action,
PublicKey const& pk,
std::uint32_t seq,
std::uint32_t oldSeq)
{
s << "Manifest: " << action <<
";Pk: " << toBase58 (TokenType::TOKEN_NODE_PUBLIC, pk) <<
";Seq: " << seq <<
";OldSeq: " << oldSeq << ";";
return s;
}
Manifest::Manifest (std::string s,
PublicKey pk,
PublicKey spk,
std::uint32_t seq)
: serialized (std::move (s))
, masterKey (std::move (pk))
, signingKey (std::move (spk))
, sequence (seq)
{
}
bool Manifest::verify () const
{
STObject st (sfGeneric);
SerialIter sit (serialized.data (), serialized.size ());
st.set (sit);
if (! ripple::verify (st, HashPrefix::manifest, signingKey))
return false;
return ripple::verify (
st, HashPrefix::manifest, masterKey, sfMasterSignature);
}
uint256 Manifest::hash () const
{
STObject st (sfGeneric);
SerialIter sit (serialized.data (), serialized.size ());
st.set (sit);
return st.getHash (HashPrefix::manifest);
}
bool Manifest::revoked () const
{
/*
The maximum possible sequence number means that the master key
has been revoked.
*/
return sequence == std::numeric_limits<std::uint32_t>::max ();
}
Blob Manifest::getSignature () const
{
STObject st (sfGeneric);
SerialIter sit (serialized.data (), serialized.size ());
st.set (sit);
return st.getFieldVL (sfSignature);
}
Blob Manifest::getMasterSignature () const
{
STObject st (sfGeneric);
SerialIter sit (serialized.data (), serialized.size ());
st.set (sit);
return st.getFieldVL (sfMasterSignature);
}
PublicKey
ManifestCache::getSigningKey (PublicKey const& pk) const
{
std::lock_guard<std::mutex> lock{read_mutex_};
auto const iter = map_.find (pk);
if (iter != map_.end () && !iter->second.revoked ())
return iter->second.signingKey;
return pk;
}
PublicKey
ManifestCache::getMasterKey (PublicKey const& pk) const
{
std::lock_guard<std::mutex> lock{read_mutex_};
auto const iter = signingToMasterKeys_.find (pk);
if (iter != signingToMasterKeys_.end ())
return iter->second;
return pk;
}
bool
ManifestCache::revoked (PublicKey const& pk) const
{
std::lock_guard<std::mutex> lock{read_mutex_};
auto const iter = map_.find (pk);
if (iter != map_.end ())
return iter->second.revoked ();
return false;
}
ManifestDisposition
ManifestCache::applyManifest (Manifest m)
{
std::lock_guard<std::mutex> applyLock{apply_mutex_};
/*
before we spend time checking the signature, make sure the
sequence number is newer than any we have.
*/
auto const iter = map_.find(m.masterKey);
if (iter != map_.end() &&
m.sequence <= iter->second.sequence)
{
/*
A manifest was received for a validator we're tracking, but
its sequence number is not higher than the one already stored.
This will happen normally when a peer without the latest gossip
connects.
*/
if (auto stream = j_.debug())
logMftAct(stream, "Stale", m.masterKey, m.sequence, iter->second.sequence);
return ManifestDisposition::stale; // not a newer manifest, ignore
}
if (! m.verify())
{
/*
A manifest's signature is invalid.
This shouldn't happen normally.
*/
if (auto stream = j_.warn())
logMftAct(stream, "Invalid", m.masterKey, m.sequence);
return ManifestDisposition::invalid;
}
std::lock_guard<std::mutex> readLock{read_mutex_};
bool const revoked = m.revoked();
if (iter == map_.end ())
{
/*
This is the first received manifest for a trusted master key
(possibly our own). This only happens once per validator per
run.
*/
if (auto stream = j_.info())
logMftAct(stream, "AcceptedNew", m.masterKey, m.sequence);
if (! revoked)
signingToMasterKeys_[m.signingKey] = m.masterKey;
map_.emplace (std::make_pair(m.masterKey, std::move (m)));
}
else
{
/*
An ephemeral key was revoked and superseded by a new key.
This is expected, but should happen infrequently.
*/
if (auto stream = j_.info())
logMftAct(stream, "AcceptedUpdate",
m.masterKey, m.sequence, iter->second.sequence);
signingToMasterKeys_.erase (iter->second.signingKey);
if (! revoked)
signingToMasterKeys_[m.signingKey] = m.masterKey;
iter->second = std::move (m);
}
if (revoked)
{
/*
A validator master key has been compromised, so its manifests
are now untrustworthy. In order to prevent us from accepting
a forged manifest signed by the compromised master key, store
this manifest, which has the highest possible sequence number
and therefore can't be superseded by a forged one.
*/
if (auto stream = j_.warn())
logMftAct(stream, "Revoked", m.masterKey, m.sequence);
}
return ManifestDisposition::accepted;
}
void
ManifestCache::load (
DatabaseCon& dbCon, std::string const& dbTable)
{
// Load manifests stored in database
std::string const sql =
"SELECT RawData FROM " + dbTable + ";";
auto db = dbCon.checkoutDb ();
soci::blob sociRawData (*db);
soci::statement st =
(db->prepare << sql,
soci::into (sociRawData));
st.execute ();
while (st.fetch ())
{
std::string serialized;
convert (sociRawData, serialized);
if (auto mo = Manifest::make_Manifest (std::move (serialized)))
{
if (!mo->verify())
{
JLOG(j_.warn())
<< "Unverifiable manifest in db";
continue;
}
applyManifest (std::move(*mo));
}
else
{
JLOG(j_.warn())
<< "Malformed manifest in database";
}
}
}
bool
ManifestCache::load (
DatabaseCon& dbCon, std::string const& dbTable,
std::vector<std::string> const& configManifest)
{
load (dbCon, dbTable);
if (! configManifest.empty())
{
std::string s;
s.reserve (Manifest::textLength);
for (auto const& line : configManifest)
s += beast::rfc2616::trim(line);
auto mo = Manifest::make_Manifest (beast::detail::base64_decode(s));
if (! mo)
{
JLOG (j_.error()) << "Malformed manifest in config";
return false;
}
if (mo->revoked())
{
JLOG (j_.warn()) <<
"Configured manifest revokes public key";
}
if (applyManifest (std::move(*mo)) ==
ManifestDisposition::invalid)
{
JLOG (j_.error()) << "Manifest in config was rejected";
return false;
}
}
return true;
}
void ManifestCache::save (
DatabaseCon& dbCon, std::string const& dbTable,
std::function <bool (PublicKey const&)> isTrusted)
{
std::lock_guard<std::mutex> lock{apply_mutex_};
auto db = dbCon.checkoutDb ();
soci::transaction tr(*db);
*db << "DELETE FROM " << dbTable;
std::string const sql =
"INSERT INTO " + dbTable + " (RawData) VALUES (:rawData);";
for (auto const& v : map_)
{
if (! isTrusted (v.second.masterKey))
{
JLOG(j_.info())
<< "Untrusted manifest in cache not saved to db";
continue;
}
// soci does not support bulk insertion of blob data
// Do not reuse blob because manifest ecdsa signatures vary in length
// but blob write length is expected to be >= the last write
soci::blob rawData(*db);
convert (v.second.serialized, rawData);
*db << sql,
soci::use (rawData);
}
tr.commit ();
}
}

View File

@@ -17,117 +17,39 @@
*/
//==============================================================================
#include <BeastConfig.h>
#include <ripple/app/misc/ValidatorList.h>
#include <ripple/basics/Slice.h>
#include <ripple/basics/StringUtilities.h>
#include <ripple/json/json_reader.h>
#include <beast/core/detail/base64.hpp>
#include <boost/regex.hpp>
namespace ripple {
ValidatorList::ValidatorList (beast::Journal j)
: j_ (j)
ValidatorList::ValidatorList (
ManifestCache& validatorManifests,
ManifestCache& publisherManifests,
TimeKeeper& timeKeeper,
beast::Journal j,
boost::optional<std::size_t> minimumQuorum)
: validatorManifests_ (validatorManifests)
, publisherManifests_ (publisherManifests)
, timeKeeper_ (timeKeeper)
, j_ (j)
, quorum_ (minimumQuorum ? *minimumQuorum : 1) // Genesis ledger quorum
, minimumQuorum_ (minimumQuorum)
{
}
boost::optional<std::string>
ValidatorList::member (PublicKey const& identity) const
ValidatorList::~ValidatorList()
{
std::lock_guard <std::mutex> sl (mutex_);
auto ret = ephemeral_.find (identity);
if (ret != ephemeral_.end())
return ret->second;
ret = permanent_.find (identity);
if (ret != permanent_.end())
return ret->second;
return boost::none;
}
bool
ValidatorList::trusted (PublicKey const& identity) const
{
return static_cast<bool> (member(identity));
}
bool
ValidatorList::insertEphemeralKey (
PublicKey const& identity,
std::string const& comment)
{
std::lock_guard <std::mutex> sl (mutex_);
if (permanent_.find (identity) != permanent_.end())
{
JLOG (j_.error()) <<
toBase58 (TokenType::TOKEN_NODE_PUBLIC, identity) <<
": ephemeral key exists in permanent table!";
return false;
}
return ephemeral_.emplace (identity, comment).second;
}
bool
ValidatorList::removeEphemeralKey (
PublicKey const& identity)
{
std::lock_guard <std::mutex> sl (mutex_);
return ephemeral_.erase (identity);
}
bool
ValidatorList::insertPermanentKey (
PublicKey const& identity,
std::string const& comment)
{
std::lock_guard <std::mutex> sl (mutex_);
if (ephemeral_.find (identity) != ephemeral_.end())
{
JLOG (j_.error()) <<
toBase58 (TokenType::TOKEN_NODE_PUBLIC, identity) <<
": permanent key exists in ephemeral table!";
return false;
}
return permanent_.emplace (identity, comment).second;
}
bool
ValidatorList::removePermanentKey (
PublicKey const& identity)
{
std::lock_guard <std::mutex> sl (mutex_);
return permanent_.erase (identity);
}
std::size_t
ValidatorList::size () const
{
std::lock_guard <std::mutex> sl (mutex_);
return permanent_.size () + ephemeral_.size ();
}
void
ValidatorList::for_each (
std::function<void(PublicKey const&, std::string const&, bool)> func) const
{
std::lock_guard <std::mutex> sl (mutex_);
for (auto const& v : permanent_)
func (v.first, v.second, false);
for (auto const& v : ephemeral_)
func (v.first, v.second, true);
}
bool
ValidatorList::load (
Section const& validators)
PublicKey const& localSigningKey,
std::vector<std::string> const& configKeys,
std::vector<std::string> const& publisherKeys)
{
static boost::regex const re (
"[[:space:]]*" // skip leading whitespace
@@ -141,12 +63,61 @@ ValidatorList::load (
")?" // end optional comment block
);
boost::unique_lock<boost::shared_mutex> read_lock{mutex_};
JLOG (j_.debug()) <<
"Loading configured validators";
"Loading configured trusted validator list publisher keys";
std::size_t count = 0;
for (auto key : publisherKeys)
{
JLOG (j_.trace()) <<
"Processing '" << key << "'";
for (auto const& n : validators.values ())
auto const ret = strUnHex (key);
if (! ret.second || ! ret.first.size ())
{
JLOG (j_.error()) <<
"Invalid validator list publisher key: " << key;
return false;
}
auto id = PublicKey(Slice{ ret.first.data (), ret.first.size() });
if (validatorManifests_.revoked (id))
{
JLOG (j_.warn()) <<
"Configured validator list publisher key is revoked: " << key;
continue;
}
if (publisherLists_.count(id))
{
JLOG (j_.warn()) <<
"Duplicate validator list publisher key: " << key;
continue;
}
publisherLists_[id].available = false;
++count;
}
JLOG (j_.debug()) <<
"Loaded " << count << " keys";
localPubKey_ = validatorManifests_.getMasterKey (localSigningKey);
// Treat local validator key as though it was listed in the config
if (localPubKey_.size())
keyListings_.insert ({ localPubKey_, 1 });
JLOG (j_.debug()) <<
"Loading configured validator keys";
count = 0;
PublicKey local;
for (auto const& n : configKeys)
{
JLOG (j_.trace()) <<
"Processing '" << n << "'";
@@ -165,20 +136,23 @@ ValidatorList::load (
if (!id)
{
JLOG (j_.error()) <<
"Invalid node identity: " << match[1];
JLOG (j_.error()) << "Invalid node identity: " << match[1];
return false;
}
if (trusted (*id))
// Skip local key which was already added
if (*id == localPubKey_ || *id == localSigningKey)
continue;
auto ret = keyListings_.insert ({*id, 1});
if (! ret.second)
{
JLOG (j_.warn()) <<
"Duplicate node identity: " << match[1];
JLOG (j_.warn()) << "Duplicate node identity: " << match[1];
continue;
}
if (insertPermanentKey(*id, trim_whitespace (match[2])))
++count;
publisherLists_[local].list.emplace_back (std::move(*id));
publisherLists_[local].available = true;
++count;
}
JLOG (j_.debug()) <<
@@ -187,4 +161,244 @@ ValidatorList::load (
return true;
}
ListDisposition
ValidatorList::applyList (
std::string const& manifest,
std::string const& blob,
std::string const& signature,
std::uint32_t version)
{
if (version != 1)
return ListDisposition::unsupported_version;
boost::unique_lock<boost::shared_mutex> lock{mutex_};
Json::Value list;
PublicKey pubKey;
auto const result = verify (list, pubKey, manifest, blob, signature);
if (result != ListDisposition::accepted)
return result;
// Update publisher's list
Json::Value const& newList = list["validators"];
publisherLists_[pubKey].available = true;
publisherLists_[pubKey].sequence = list["sequence"].asUInt ();
publisherLists_[pubKey].expiration = list["expiration"].asUInt ();
std::vector<PublicKey>& publisherList = publisherLists_[pubKey].list;
std::vector<PublicKey> oldList = publisherList;
publisherList.clear ();
publisherList.reserve (newList.size ());
for (auto const& val : newList)
{
if (val.isObject () &&
val.isMember ("validation_public_key") &&
val["validation_public_key"].isString ())
{
std::pair<Blob, bool> ret (strUnHex (
val["validation_public_key"].asString ()));
if (! ret.second || ! ret.first.size ())
{
JLOG (j_.error()) <<
"Invalid node identity: " <<
val["validation_public_key"].asString ();
}
else
{
publisherList.push_back (
PublicKey(Slice{ ret.first.data (), ret.first.size() }));
}
}
}
// Update keyListings_ for added and removed keys
std::sort (
publisherList.begin (),
publisherList.end ());
auto iNew = publisherList.begin ();
auto iOld = oldList.begin ();
while (iNew != publisherList.end () ||
iOld != oldList.end ())
{
if (iOld == oldList.end () ||
(iNew != publisherList.end () &&
*iNew < *iOld))
{
// Increment list count for added keys
++keyListings_[*iNew];
++iNew;
}
else if (iNew == publisherList.end () ||
(iOld != oldList.end () && *iOld < *iNew))
{
// Decrement list count for removed keys
if (keyListings_[*iOld] == 1)
keyListings_.erase (*iOld);
else
--keyListings_[*iOld];
++iOld;
}
else
{
++iNew;
++iOld;
}
}
if (publisherList.empty())
{
JLOG (j_.warn()) <<
"No validator keys included in valid list";
}
return ListDisposition::accepted;
}
ListDisposition
ValidatorList::verify (
Json::Value& list,
PublicKey& pubKey,
std::string const& manifest,
std::string const& blob,
std::string const& signature)
{
auto m = Manifest::make_Manifest (beast::detail::base64_decode(manifest));
if (! m || ! publisherLists_.count (m->masterKey))
return ListDisposition::untrusted;
pubKey = m->masterKey;
auto const revoked = m->revoked();
auto const result = publisherManifests_.applyManifest (
std::move(*m));
if (revoked && result == ManifestDisposition::accepted)
{
removePublisherList (pubKey);
publisherLists_.erase (pubKey);
}
if (revoked || result == ManifestDisposition::invalid)
return ListDisposition::untrusted;
auto const sig = strUnHex(signature);
auto const data = beast::detail::base64_decode (blob);
if (! sig.second ||
! ripple::verify (
publisherManifests_.getSigningKey(pubKey),
makeSlice(data),
makeSlice(sig.first)))
return ListDisposition::invalid;
Json::Reader r;
if (! r.parse (data, list))
return ListDisposition::invalid;
if (list.isMember("sequence") && list["sequence"].isInt() &&
list.isMember("expiration") && list["expiration"].isInt() &&
list.isMember("validators") && list["validators"].isArray())
{
auto const sequence = list["sequence"].asUInt ();
auto const expiration = list["expiration"].asUInt ();
if (sequence <= publisherLists_[pubKey].sequence ||
expiration <= timeKeeper_.now().time_since_epoch().count())
return ListDisposition::stale;
}
else
{
return ListDisposition::invalid;
}
return ListDisposition::accepted;
}
bool
ValidatorList::listed (
PublicKey const& identity) const
{
boost::shared_lock<boost::shared_mutex> read_lock{mutex_};
auto const pubKey = validatorManifests_.getMasterKey (identity);
return keyListings_.find (pubKey) != keyListings_.end ();
}
bool
ValidatorList::trusted (PublicKey const& identity) const
{
boost::shared_lock<boost::shared_mutex> read_lock{mutex_};
auto const pubKey = validatorManifests_.getMasterKey (identity);
return trustedKeys_.find (pubKey) != trustedKeys_.end();
}
boost::optional<PublicKey>
ValidatorList::getListedKey (
PublicKey const& identity) const
{
boost::shared_lock<boost::shared_mutex> read_lock{mutex_};
auto const pubKey = validatorManifests_.getMasterKey (identity);
if (keyListings_.find (pubKey) != keyListings_.end ())
return pubKey;
return boost::none;
}
boost::optional<PublicKey>
ValidatorList::getTrustedKey (PublicKey const& identity) const
{
boost::shared_lock<boost::shared_mutex> read_lock{mutex_};
auto const pubKey = validatorManifests_.getMasterKey (identity);
if (trustedKeys_.find (pubKey) != trustedKeys_.end())
return pubKey;
return boost::none;
}
bool
ValidatorList::trustedPublisher (PublicKey const& identity) const
{
boost::shared_lock<boost::shared_mutex> read_lock{mutex_};
return identity.size() && publisherLists_.count (identity);
}
bool
ValidatorList::removePublisherList (PublicKey const& publisherKey)
{
auto const iList = publisherLists_.find (publisherKey);
if (iList == publisherLists_.end ())
return false;
JLOG (j_.debug()) <<
"Removing validator list for revoked publisher " <<
toBase58(TokenType::TOKEN_NODE_PUBLIC, publisherKey);
for (auto const& val : iList->second.list)
{
auto const& iVal = keyListings_.find (val);
if (iVal == keyListings_.end())
continue;
if (iVal->second <= 1)
keyListings_.erase (iVal);
else
--iVal->second;
}
return true;
}
void
ValidatorList::for_each_listed (
std::function<void(PublicKey const&, bool)> func) const
{
boost::shared_lock<boost::shared_mutex> read_lock{mutex_};
for (auto const& v : keyListings_)
func (v.first, trusted(v.first));
}
} // ripple