mirror of
https://github.com/XRPLF/rippled.git
synced 2026-07-31 19:10:25 +00:00
Implement negative UNL functionality:
This change can help improve the liveness of the network during periods of network instability, by allowing the network to track which validators are presently not online and to disregard them for the purposes of quorum calculations.
This commit is contained in:
Peng Wang
@@ -377,6 +377,7 @@ target_sources (rippled PRIVATE
|
||||
src/ripple/app/misc/CanonicalTXSet.cpp
|
||||
src/ripple/app/misc/FeeVoteImpl.cpp
|
||||
src/ripple/app/misc/HashRouter.cpp
|
||||
src/ripple/app/misc/NegativeUNLVote.cpp
|
||||
src/ripple/app/misc/NetworkOPs.cpp
|
||||
src/ripple/app/misc/SHAMapStoreImp.cpp
|
||||
src/ripple/app/misc/impl/AccountTxPaging.cpp
|
||||
@@ -746,6 +747,7 @@ target_sources (rippled PRIVATE
|
||||
src/test/consensus/DistributedValidatorsSim_test.cpp
|
||||
src/test/consensus/LedgerTiming_test.cpp
|
||||
src/test/consensus/LedgerTrie_test.cpp
|
||||
src/test/consensus/NegativeUNL_test.cpp
|
||||
src/test/consensus/ScaleFreeSim_test.cpp
|
||||
src/test/consensus/Validations_test.cpp
|
||||
#[===============================[
|
||||
|
||||
597
docs/0001-negative-unl/README.md
Normal file
597
docs/0001-negative-unl/README.md
Normal file
@@ -0,0 +1,597 @@
|
||||
# Negative UNL Engineering Spec
|
||||
|
||||
## The Problem Statement
|
||||
|
||||
The moment-to-moment health of the XRP Ledger network depends on the health and
|
||||
connectivity of a small number of computers (nodes). The most important nodes
|
||||
are validators, specifically ones listed on the unique node list
|
||||
([UNL](#Question-What-are-UNLs)). Ripple publishes a recommended UNL that most
|
||||
network nodes use to determine which peers in the network are trusted. Although
|
||||
most validators use the same list, they are not required to. The XRP Ledger
|
||||
network progresses to the next ledger when enough validators reach agreement
|
||||
(above the minimum quorum of 80%) about what transactions to include in the next
|
||||
ledger.
|
||||
|
||||
As an example, if there are 10 validators on the UNL, at least 8 validators have
|
||||
to agree with the latest ledger for it to become validated. But what if enough
|
||||
of those validators are offline to drop the network below the 80% quorum? The
|
||||
XRP Ledger network favors safety/correctness over advancing the ledger. Which
|
||||
means if enough validators are offline, the network will not be able to validate
|
||||
ledgers.
|
||||
|
||||
Unfortunately validators can go offline at any time for many different reasons.
|
||||
Power outages, network connectivity issues, and hardware failures are just a few
|
||||
scenarios where a validator would appear "offline". Given that most of these
|
||||
events are temporary, it would make sense to temporarily remove that validator
|
||||
from the UNL. But the UNL is updated infrequently and not every node uses the
|
||||
same UNL. So instead of removing the unreliable validator from the Ripple
|
||||
recommended UNL, we can create a second negative UNL which is stored directly on
|
||||
the ledger (so the entire network has the same view). This will help the network
|
||||
see which validators are **currently** unreliable, and adjust their quorum
|
||||
calculation accordingly.
|
||||
|
||||
*Improving the liveness of the network is the main motivation for the negative UNL.*
|
||||
|
||||
### Targeted Faults
|
||||
|
||||
In order to determine which validators are unreliable, we need clearly define
|
||||
what kind of faults to measure and analyze. We want to deal with the faults we
|
||||
frequently observe in the production network. Hence we will only monitor for
|
||||
validators that do not reliably respond to network messages or send out
|
||||
validations disagreeing with the locally generated validations. We will not
|
||||
target other byzantine faults.
|
||||
|
||||
To track whether or not a validator is responding to the network, we could
|
||||
monitor them with a “heartbeat” protocol. Instead of creating a new heartbeat
|
||||
protocol, we can leverage some existing protocol messages to mimic the
|
||||
heartbeat. We picked validation messages because validators should send one and
|
||||
only one validation message per ledger. In addition, we only count the
|
||||
validation messages that agree with the local node's validations.
|
||||
|
||||
With the negative UNL, the network could keep making forward progress safely
|
||||
even if the number of remaining validators gets to 60%. Say we have a network
|
||||
with 10 validators on the UNL and everything is operating correctly. The quorum
|
||||
required for this network would be 8 (80% of 10). When validators fail, the
|
||||
quorum required would be as low as 6 (60% of 10), which is the absolute
|
||||
***minimum quorum***. We need the absolute minimum quorum to be strictly greater
|
||||
than 50% of the original UNL so that there cannot be two partitions of
|
||||
well-behaved nodes headed in different directions. We arbitrarily choose 60% as
|
||||
the minimum quorum to give a margin of safety.
|
||||
|
||||
Consider these events in the absence of negative UNL:
|
||||
1. 1:00pm - validator1 fails, votes vs. quorum: 9 >= 8, we have quorum
|
||||
1. 3:00pm - validator2 fails, votes vs. quorum: 8 >= 8, we have quorum
|
||||
1. 5:00pm - validator3 fails, votes vs. quorum: 7 < 8, we don’t have quorum
|
||||
* **network cannot validate new ledgers with 3 failed validators**
|
||||
|
||||
We're below 80% agreement, so new ledgers cannot be validated. This is how the
|
||||
XRP Ledger operates today, but if the negative UNL was enabled, the events would
|
||||
happen as follows. (Please note that the events below are from a simplified
|
||||
version of our protocol.)
|
||||
|
||||
1. 1:00pm - validator1 fails, votes vs. quorum: 9 >= 8, we have quorum
|
||||
1. 1:40pm - network adds validator1 to negative UNL, quorum changes to ceil(9 * 0.8), or 8
|
||||
1. 3:00pm - validator2 fails, votes vs. quorum: 8 >= 8, we have quorum
|
||||
1. 3:40pm - network adds validator2 to negative UNL, quorum changes to ceil(8 * 0.8), or 7
|
||||
1. 5:00pm - validator3 fails, votes vs. quorum: 7 >= 7, we have quorum
|
||||
1. 5:40pm - network adds validator3 to negative UNL, quorum changes to ceil(7 * 0.8), or 6
|
||||
1. 7:00pm - validator4 fails, votes vs. quorum: 6 >= 6, we have quorum
|
||||
* **network can still validate new ledgers with 4 failed validators**
|
||||
|
||||
## External Interactions
|
||||
|
||||
### Message Format Changes
|
||||
This proposal will:
|
||||
1. add a new pseudo-transaction type
|
||||
1. add the negative UNL to the ledger data structure.
|
||||
|
||||
Any tools or systems that rely on the format of this data will have to be
|
||||
updated.
|
||||
|
||||
### Amendment
|
||||
This feature **will** need an amendment to activate.
|
||||
|
||||
## Design
|
||||
|
||||
This section discusses the following topics about the Negative UNL design:
|
||||
|
||||
* [Negative UNL protocol overview](#Negative-UNL-Protocol-Overview)
|
||||
* [Validator reliability measurement](#Validator-Reliability-Measurement)
|
||||
* [Format Changes](#Format-Changes)
|
||||
* [Negative UNL maintenance](#Negative-UNL-Maintenance)
|
||||
* [Quorum size calculation](#Quorum-Size-Calculation)
|
||||
* [Filter validation messages](#Filter-Validation-Messages)
|
||||
* [High level sequence diagram of code
|
||||
changes](#High-Level-Sequence-Diagram-of-Code-Changes)
|
||||
|
||||
### Negative UNL Protocol Overview
|
||||
|
||||
Every ledger stores a list of zero or more unreliable validators. Updates to the
|
||||
list must be approved by the validators using the consensus mechanism that
|
||||
validators use to agree on the set of transactions. The list is used only when
|
||||
checking if a ledger is fully validated. If a validator V is in the list, nodes
|
||||
with V in their UNL adjust the quorum and V’s validation message is not counted
|
||||
when verifying if a ledger is fully validated. V’s flow of messages and network
|
||||
interactions, however, will remain the same.
|
||||
|
||||
We define the ***effective UNL** = original UNL - negative UNL*, and the
|
||||
***effective quorum*** as the quorum of the *effective UNL*. And we set
|
||||
*effective quorum = Ceiling(80% * effective UNL)*.
|
||||
|
||||
### Validator Reliability Measurement
|
||||
|
||||
A node only measures the reliability of validators on its own UNL, and only
|
||||
proposes based on local observations. There are many metrics that a node can
|
||||
measure about its validators, but we have chosen ledger validation messages.
|
||||
This is because every validator shall send one and only one signed validation
|
||||
message per ledger. This keeps the measurement simple and removes
|
||||
timing/clock-sync issues. A node will measure the percentage of agreeing
|
||||
validation messages (*PAV*) received from each validator on the node's UNL. Note
|
||||
that the node will only count the validation messages that agree with its own
|
||||
validations.
|
||||
|
||||
We define the **PAV** as the **P**ercentage of **A**greed **V**alidation
|
||||
messages received for the last N ledgers, where N = 256 by default.
|
||||
|
||||
When the PAV drops below the ***low-water mark***, the validator is considered
|
||||
unreliable, and is a candidate to be disabled by being added to the negative
|
||||
UNL. A validator must have a PAV higher than the ***high-water mark*** to be
|
||||
re-enabled. The validator is re-enabled by removing it from the negative UNL. In
|
||||
the implementation, we plan to set the low-water mark as 50% and the high-water
|
||||
mark as 80%.
|
||||
|
||||
### Format Changes
|
||||
|
||||
The negative UNL component in a ledger contains three fields.
|
||||
* ***NegativeUNL***: The current negative UNL, a list of unreliable validators.
|
||||
* ***ToDisable***: The validator to be added to the negative UNL on the next
|
||||
flag ledger.
|
||||
* ***ToReEnable***: The validator to be removed from the negative UNL on the
|
||||
next flag ledger.
|
||||
|
||||
All three fields are optional. When the *ToReEnable* field exists, the
|
||||
*NegativeUNL* field cannot be empty.
|
||||
|
||||
A new pseudo-transaction, ***UNLModify***, is added. It has three fields
|
||||
* ***Disabling***: A flag indicating whether the modification is to disable or
|
||||
to re-enable a validator.
|
||||
* ***Seq***: The ledger sequence number.
|
||||
* ***Validator***: The validator to be disabled or re-enabled.
|
||||
|
||||
There would be at most one *disable* `UNLModify` and one *re-enable* `UNLModify`
|
||||
transaction per flag ledger. The full machinery is described further on.
|
||||
|
||||
### Negative UNL Maintenance
|
||||
|
||||
The negative UNL can only be modified on the flag ledgers. If a validator's
|
||||
reliability status changes, it takes two flag ledgers to modify the negative
|
||||
UNL. Let's see an example of the algorithm:
|
||||
|
||||
* Ledger seq = 100: A validator V goes offline.
|
||||
* Ledger seq = 256: This is a flag ledger, and V's reliability measurement *PAV*
|
||||
is lower than the low-water mark. Other validators add `UNLModify`
|
||||
pseudo-transactions `{true, 256, V}` to the transaction set which goes through
|
||||
the consensus. Then the pseudo-transaction is applied to the negative UNL
|
||||
ledger component by setting `ToDisable = V`.
|
||||
* Ledger seq = 257 ~ 511: The negative UNL ledger component is copied from the
|
||||
parent ledger.
|
||||
* Ledger seq=512: This is a flag ledger, and the negative UNL is updated
|
||||
`NegativeUNL = NegativeUNL + ToDisable`.
|
||||
|
||||
The negative UNL may have up to `MaxNegativeListed = floor(original UNL * 25%)`
|
||||
validators. The 25% is because of 75% * 80% = 60%, where 75% = 100% - 25%, 80%
|
||||
is the quorum of the effective UNL, and 60% is the absolute minimum quorum of
|
||||
the original UNL. Adding more than 25% validators to the negative UNL does not
|
||||
improve the liveness of the network, because adding more validators to the
|
||||
negative UNL cannot lower the effective quorum.
|
||||
|
||||
The following is the detailed algorithm:
|
||||
|
||||
* **If** the ledger seq = x is a flag ledger
|
||||
|
||||
1. Compute `NegativeUNL = NegativeUNL + ToDisable - ToReEnable` if they
|
||||
exist in the parent ledger
|
||||
|
||||
1. Try to find a candidate to disable if `sizeof NegativeUNL < MaxNegativeListed`
|
||||
|
||||
1. Find a validator V that has a *PAV* lower than the low-water
|
||||
mark, but is not in `NegativeUNL`.
|
||||
|
||||
1. If two or more are found, their public keys are XORed with the hash
|
||||
of the parent ledger and the one with the lowest XOR result is chosen.
|
||||
|
||||
1. If V is found, create a `UNLModify` pseudo-transaction
|
||||
`TxDisableValidator = {true, x, V}`
|
||||
|
||||
1. Try to find a candidate to re-enable if `sizeof NegativeUNL > 0`:
|
||||
|
||||
1. Find a validator U that is in `NegativeUNL` and has a *PAV* higher
|
||||
than the high-water mark.
|
||||
|
||||
1. If U is not found, try to find one in `NegativeUNL` but not in the
|
||||
local *UNL*.
|
||||
|
||||
1. If two or more are found, their public keys are XORed with the hash
|
||||
of the parent ledger and the one with the lowest XOR result is chosen.
|
||||
|
||||
1. If U is found, create a `UNLModify` pseudo-transaction
|
||||
`TxReEnableValidator = {false, x, U}`
|
||||
|
||||
1. If any `UNLModify` pseudo-transactions are created, add them to the
|
||||
transaction set. The transaction set goes through the consensus algorithm.
|
||||
|
||||
1. If have enough support, the `UNLModify` pseudo-transactions remain in the
|
||||
transaction set agreed by the validators. Then the pseudo-transactions are
|
||||
applied to the ledger:
|
||||
|
||||
1. If have `TxDisableValidator`, set `ToDisable=TxDisableValidator.V`.
|
||||
Else clear `ToDisable`.
|
||||
|
||||
1. If have `TxReEnableValidator`, set
|
||||
`ToReEnable=TxReEnableValidator.U`. Else clear `ToReEnable`.
|
||||
|
||||
* **Else** (not a flag ledger)
|
||||
|
||||
1. Copy the negative UNL ledger component from the parent ledger
|
||||
|
||||
The negative UNL is stored on each ledger because we don't know when a validator
|
||||
may reconnect to the network. If the negative UNL was stored only on every flag
|
||||
ledger, then a new validator would have to wait until it acquires the latest
|
||||
flag ledger to know the negative UNL. So any new ledgers created that are not
|
||||
flag ledgers copy the negative UNL from the parent ledger.
|
||||
|
||||
Note that when we have a validator to disable and a validator to re-enable at
|
||||
the same flag ledger, we create two separate `UNLModify` pseudo-transactions. We
|
||||
want either one or the other or both to make it into the ledger on their own
|
||||
merits.
|
||||
|
||||
Readers may have noticed that we defined several rules of creating the
|
||||
`UNLModify` pseudo-transactions but did not describe how to enforce the rules.
|
||||
The rules are actually enforced by the existing consensus algorithm. Unless
|
||||
enough validators propose the same pseudo-transaction it will not be included in
|
||||
the transaction set of the ledger.
|
||||
|
||||
### Quorum Size Calculation
|
||||
|
||||
The effective quorum is 80% of the effective UNL. Note that because at most 25%
|
||||
of the original UNL can be on the negative UNL, the quorum should not be lower
|
||||
than the absolute minimum quorum (i.e. 60%) of the original UNL. However,
|
||||
considering that different nodes may have different UNLs, to be safe we compute
|
||||
`quorum = Ceiling(max(60% * original UNL, 80% * effective UNL))`.
|
||||
|
||||
### Filter Validation Messages
|
||||
|
||||
If a validator V is in the negative UNL, it still participates in consensus
|
||||
sessions in the same way, i.e. V still follows the protocol and publishes
|
||||
proposal and validation messages. The messages from V are still stored the same
|
||||
way by everyone, used to calculate the new PAV for V, and could be used in
|
||||
future consensus sessions if needed. However V's ledger validation message is
|
||||
not counted when checking if the ledger is fully validated.
|
||||
|
||||
### High Level Sequence Diagram of Code Changes
|
||||
|
||||
The diagram below is the sequence of one round of consensus. Classes and
|
||||
components with non-trivial changes are colored green.
|
||||
|
||||
* The `ValidatorList` class is modified to compute the quorum of the effective
|
||||
UNL.
|
||||
|
||||
* The `Validations` class provides an interface for querying the validation
|
||||
messages from trusted validators.
|
||||
|
||||
* The `ConsensusAdaptor` component:
|
||||
|
||||
* The `RCLConsensus::Adaptor` class is modified for creating `UNLModify`
|
||||
Pseudo-Transactions.
|
||||
|
||||
* The `Change` class is modified for applying `UNLModify`
|
||||
Pseudo-Transactions.
|
||||
|
||||
* The `Ledger` class is modified for creating and adjusting the negative UNL
|
||||
ledger component.
|
||||
|
||||
* The `LedgerMaster` class is modified for filtering out validation messages
|
||||
from negative UNL validators when verifying if a ledger is fully
|
||||
validated.
|
||||
|
||||
|
||||
|
||||
|
||||
## Roads Not Taken
|
||||
|
||||
### Use a Mechanism Like Fee Voting to Process UNLModify Pseudo-Transactions
|
||||
|
||||
The previous version of the negative UNL specification used the same mechanism
|
||||
as the [fee voting](https://xrpl.org/fee-voting.html#voting-process.) for
|
||||
creating the negative UNL, and used the negative UNL as soon as the ledger was
|
||||
fully validated. However the timing of fully validation can differ among nodes,
|
||||
so different negative UNLs could be used, resulting in different effective UNLs
|
||||
and different quorums for the same ledger. As a result, the network's safety is
|
||||
impacted.
|
||||
|
||||
This updated version does not impact safety though operates a bit more slowly.
|
||||
The negative UNL modifications in the *UNLModify* pseudo-transaction approved by
|
||||
the consensus will take effect at the next flag ledger. The extra time of the
|
||||
256 ledgers should be enough for nodes to be in sync of the negative UNL
|
||||
modifications.
|
||||
|
||||
### Use an Expiration Approach to Re-enable Validators
|
||||
|
||||
After a validator disabled by the negative UNL becomes reliable, other
|
||||
validators explicitly vote for re-enabling it. An alternative approach to
|
||||
re-enable a validator is the expiration approach, which was considered in the
|
||||
previous version of the specification. In the expiration approach, every entry
|
||||
in the negative UNL has a fixed expiration time. One flag ledger interval was
|
||||
chosen as the expiration interval. Once expired, the other validators must
|
||||
continue voting to keep the unreliable validator on the negative UNL. The
|
||||
advantage of this approach is its simplicity. But it has a requirement. The
|
||||
negative UNL protocol must be able to vote multiple unreliable validators to be
|
||||
disabled at the same flag ledger. In this version of the specification, however,
|
||||
only one unreliable validator can be disabled at a flag ledger. So the
|
||||
expiration approach cannot be simply applied.
|
||||
|
||||
### Validator Reliability Measurement and Flag Ledger Frequency
|
||||
|
||||
If the ledger time is about 4.5 seconds and the low-water mark is 50%, then in
|
||||
the worst case, it takes 48 minutes *((0.5 * 256 + 256 + 256) * 4.5 / 60 = 48)*
|
||||
to put an offline validator on the negative UNL. We considered lowering the flag
|
||||
ledger frequency so that the negative UNL can be more responsive. We also
|
||||
considered decoupling the reliability measurement and flag ledger frequency to
|
||||
be more flexible. In practice, however, their benefits are not clear.
|
||||
|
||||
|
||||
## New Attack Vectors
|
||||
|
||||
A group of malicious validators may try to frame a reliable validator and put it
|
||||
on the negative UNL. But they cannot succeed. Because:
|
||||
|
||||
1. A reliable validator sends a signed validation message every ledger. A
|
||||
sufficient peer-to-peer network will propagate the validation messages to other
|
||||
validators. The validators will decide if another validator is reliable or not
|
||||
only by its local observation of the validation messages received. So an honest
|
||||
validator’s vote on another validator’s reliability is accurate.
|
||||
|
||||
1. Given the votes are accurate, and one vote per validator, an honest validator
|
||||
will not create a UNLModify transaction of a reliable validator.
|
||||
|
||||
1. A validator can be added to a negative UNL only through a UNLModify
|
||||
transaction.
|
||||
|
||||
Assuming the group of malicious validators is less than the quorum, they cannot
|
||||
frame a reliable validator.
|
||||
|
||||
## Summary
|
||||
|
||||
The bullet points below briefly summarize the current proposal:
|
||||
|
||||
* The motivation of the negative UNL is to improve the liveness of the network.
|
||||
|
||||
* The targeted faults are the ones frequently observed in the production
|
||||
network.
|
||||
|
||||
* Validators propose negative UNL candidates based on their local measurements.
|
||||
|
||||
* The absolute minimum quorum is 60% of the original UNL.
|
||||
|
||||
* The format of the ledger is changed, and a new *UNLModify* pseudo-transaction
|
||||
is added. Any tools or systems that rely on the format of these data will have
|
||||
to be updated.
|
||||
|
||||
* The negative UNL can only be modified on the flag ledgers.
|
||||
|
||||
* At most one validator can be added to the negative UNL at a flag ledger.
|
||||
|
||||
* At most one validator can be removed from the negative UNL at a flag ledger.
|
||||
|
||||
* If a validator's reliability status changes, it takes two flag ledgers to
|
||||
modify the negative UNL.
|
||||
|
||||
* The quorum is the larger of 80% of the effective UNL and 60% of the original
|
||||
UNL.
|
||||
|
||||
* If a validator is on the negative UNL, its validation messages are ignored
|
||||
when the local node verifies if a ledger is fully validated.
|
||||
|
||||
## FAQ
|
||||
|
||||
### Question: What are UNLs?
|
||||
|
||||
Quote from the [Technical FAQ](https://xrpl.org/technical-faq.html): "They are
|
||||
the lists of transaction validators a given participant believes will not
|
||||
conspire to defraud them."
|
||||
|
||||
### Question: How does the negative UNL proposal affect network liveness?
|
||||
|
||||
The network can make forward progress when more than a quorum of the trusted
|
||||
validators agree with the progress. The lower the quorum size is, the easier for
|
||||
the network to progress. If the quorum is too low, however, the network is not
|
||||
safe because nodes may have different results. So the quorum size used in the
|
||||
consensus protocol is a balance between the safety and the liveness of the
|
||||
network. The negative UNL reduces the size of the effective UNL, resulting in a
|
||||
lower quorum size while keeping the network safe.
|
||||
|
||||
<h3> Question: How does a validator get into the negative UNL? How is a
|
||||
validator removed from the negative UNL? </h3>
|
||||
|
||||
A validator’s reliability is measured by other validators. If a validator
|
||||
becomes unreliable, at a flag ledger, other validators propose *UNLModify*
|
||||
pseudo-transactions which vote the validator to add to the negative UNL during
|
||||
the consensus session. If agreed, the validator is added to the negative UNL at
|
||||
the next flag ledger. The mechanism of removing a validator from the negative
|
||||
UNL is the same.
|
||||
|
||||
### Question: Given a negative UNL, what happens if the UNL changes?
|
||||
|
||||
Answer: Let’s consider the cases:
|
||||
|
||||
1. A validator is added to the UNL, and it is already in the negative UNL. This
|
||||
case could happen when not all the nodes have the same UNL. Note that the
|
||||
negative UNL on the ledger lists unreliable nodes that are not necessarily the
|
||||
validators for everyone.
|
||||
|
||||
In this case, the liveness is affected negatively. Because the minimum
|
||||
quorum could be larger but the usable validators are not increased.
|
||||
|
||||
1. A validator is removed from the UNL, and it is in the negative UNL.
|
||||
|
||||
In this case, the liveness is affected positively. Because the quorum could
|
||||
be smaller but the usable validators are not reduced.
|
||||
|
||||
1. A validator is added to the UNL, and it is not in the negative UNL.
|
||||
1. A validator is removed from the UNL, and it is not in the negative UNL.
|
||||
|
||||
Case 3 and 4 are not affected by the negative UNL protocol.
|
||||
|
||||
### Question: Can we simply lower the quorum to 60% without the negative UNL?
|
||||
|
||||
Answer: No, because the negative UNL approach is safer.
|
||||
|
||||
First let’s compare the two approaches intuitively, (1) the *negative UNL*
|
||||
approach, and (2) *lower quorum*: simply lowering the quorum from 80% to 60%
|
||||
without the negative UNL. The negative UNL approach uses consensus to come up
|
||||
with a list of unreliable validators, which are then removed from the effective
|
||||
UNL temporarily. With this approach, the list of unreliable validators is agreed
|
||||
to by a quorum of validators and will be used by every node in the network to
|
||||
adjust its UNL. The quorum is always 80% of the effective UNL. The lower quorum
|
||||
approach is a tradeoff between safety and liveness and against our principle of
|
||||
preferring safety over liveness. Note that different validators don't have to
|
||||
agree on which validation sources they are ignoring.
|
||||
|
||||
Next we compare the two approaches quantitatively with examples, and apply
|
||||
Theorem 8 of [Analysis of the XRP Ledger Consensus
|
||||
Protocol](https://arxiv.org/abs/1802.07242) paper:
|
||||
|
||||
*XRP LCP guarantees fork safety if **O<sub>i,j</sub> > n<sub>j</sub> / 2 +
|
||||
n<sub>i</sub> − q<sub>i</sub> + t<sub>i,j</sub>** for every pair of nodes
|
||||
P<sub>i</sub>, P<sub>j</sub>,*
|
||||
|
||||
where *O<sub>i,j</sub>* is the overlapping requirement, n<sub>j</sub> and
|
||||
n<sub>i</sub> are UNL sizes, q<sub>i</sub> is the quorum size of P<sub>i</sub>,
|
||||
*t<sub>i,j</sub> = min(t<sub>i</sub>, t<sub>j</sub>, O<sub>i,j</sub>)*, and
|
||||
t<sub>i</sub> and t<sub>j</sub> are the number of faults can be tolerated by
|
||||
P<sub>i</sub> and P<sub>j</sub>.
|
||||
|
||||
We denote *UNL<sub>i</sub>* as *P<sub>i</sub>'s UNL*, and *|UNL<sub>i</sub>|* as
|
||||
the size of *P<sub>i</sub>'s UNL*.
|
||||
|
||||
Assuming *|UNL<sub>i</sub>| = |UNL<sub>j</sub>|*, let's consider the following
|
||||
three cases:
|
||||
|
||||
1. With 80% quorum and 20% faults, *O<sub>i,j</sub> > 100% / 2 + 100% - 80% +
|
||||
20% = 90%*. I.e. fork safety requires > 90% UNL overlaps. This is one of the
|
||||
results in the analysis paper.
|
||||
|
||||
1. If the quorum is 60%, the relationship between the overlapping requirement
|
||||
and the faults that can be tolerated is *O<sub>i,j</sub> > 90% +
|
||||
t<sub>i,j</sub>*. Under the same overlapping condition (i.e. 90%), to guarantee
|
||||
the fork safety, the network cannot tolerate any faults. So under the same
|
||||
overlapping condition, if the quorum is simply lowered, the network can tolerate
|
||||
fewer faults.
|
||||
|
||||
1. With the negative UNL approach, we want to argue that the inequation
|
||||
*O<sub>i,j</sub> > n<sub>j</sub> / 2 + n<sub>i</sub> − q<sub>i</sub> +
|
||||
t<sub>i,j</sub>* is always true to guarantee fork safety, while the negative UNL
|
||||
protocol runs, i.e. the effective quorum is lowered without weakening the
|
||||
network's fault tolerance. To make the discussion easier, we rewrite the
|
||||
inequation as *O<sub>i,j</sub> > n<sub>j</sub> / 2 + (n<sub>i</sub> −
|
||||
q<sub>i</sub>) + min(t<sub>i</sub>, t<sub>j</sub>)*, where O<sub>i,j</sub> is
|
||||
dropped from the definition of t<sub>i,j</sub> because *O<sub>i,j</sub> >
|
||||
min(t<sub>i</sub>, t<sub>j</sub>)* always holds under the parameters we will
|
||||
use. Assuming a validator V is added to the negative UNL, now let's consider the
|
||||
4 cases:
|
||||
|
||||
1. V is not on UNL<sub>i</sub> nor UNL<sub>j</sub>
|
||||
|
||||
The inequation holds because none of the variables change.
|
||||
|
||||
1. V is on UNL<sub>i</sub> but not on UNL<sub>j</sub>
|
||||
|
||||
The value of *(n<sub>i</sub> − q<sub>i</sub>)* is smaller. The value of
|
||||
*min(t<sub>i</sub>, t<sub>j</sub>)* could be smaller too. Other
|
||||
variables do not change. Overall, the left side of the inequation does
|
||||
not change, but the right side is smaller. So the inequation holds.
|
||||
|
||||
1. V is not on UNL<sub>i</sub> but on UNL<sub>j</sub>
|
||||
|
||||
The value of *n<sub>j</sub> / 2* is smaller. The value of
|
||||
*min(t<sub>i</sub>, t<sub>j</sub>)* could be smaller too. Other
|
||||
variables do not change. Overall, the left side of the inequation does
|
||||
not change, but the right side is smaller. So the inequation holds.
|
||||
|
||||
1. V is on both UNL<sub>i</sub> and UNL<sub>j</sub>
|
||||
|
||||
The value of *O<sub>i,j</sub>* is reduced by 1. The values of
|
||||
*n<sub>j</sub> / 2*, *(n<sub>i</sub> − q<sub>i</sub>)*, and
|
||||
*min(t<sub>i</sub>, t<sub>j</sub>)* are reduced by 0.5, 0.2, and 1
|
||||
respectively. The right side is reduced by 1.7. Overall, the left side
|
||||
of the inequation is reduced by 1, and the right side is reduced by 1.7.
|
||||
So the inequation holds.
|
||||
|
||||
The inequation holds for all the cases. So with the negative UNL approach,
|
||||
the network's fork safety is preserved, while the quorum is lowered that
|
||||
increases the network's liveness.
|
||||
|
||||
<h3> Question: We have observed that occasionally a validator wanders off on its
|
||||
own chain. How is this case handled by the negative UNL algorithm? </h3>
|
||||
|
||||
Answer: The case that a validator wanders off on its own chain can be measured
|
||||
with the validations agreement. Because the validations by this validator must
|
||||
be different from other validators' validations of the same sequence numbers.
|
||||
When there are enough disagreed validations, other validators will vote this
|
||||
validator onto the negative UNL.
|
||||
|
||||
In general by measuring the agreement of validations, we also measured the
|
||||
"sanity". If two validators have too many disagreements, one of them could be
|
||||
insane. When enough validators think a validator is insane, that validator is
|
||||
put on the negative UNL.
|
||||
|
||||
<h3> Question: Why would there be at most one disable UNLModify and one
|
||||
re-enable UNLModify transaction per flag ledger? </h3>
|
||||
|
||||
Answer: It is a design choice so that the effective UNL does not change too
|
||||
quickly. A typical targeted scenario is several validators go offline slowly
|
||||
during a long weekend. The current design can handle this kind of cases well
|
||||
without changing the effective UNL too quickly.
|
||||
|
||||
## Appendix
|
||||
|
||||
### Confidence Test
|
||||
|
||||
We will use two test networks, a single machine test network with multiple IP
|
||||
addresses and the QE test network with multiple machines. The single machine
|
||||
network will be used to test all the test cases and to debug. The QE network
|
||||
will be used after that. We want to see the test cases still pass with real
|
||||
network delay. A test case specifies:
|
||||
|
||||
1. a UNL with different number of validators for different test cases,
|
||||
1. a network with zero or more non-validator nodes,
|
||||
1. a sequence of validator reliability change events (by killing/restarting
|
||||
nodes, or by running modified rippled that does not send all validation
|
||||
messages),
|
||||
1. the correct outcomes.
|
||||
|
||||
For all the test cases, the correct outcomes are verified by examining logs. We
|
||||
will grep the log to see if the correct negative UNLs are generated, and whether
|
||||
or not the network is making progress when it should be. The ripdtop tool will
|
||||
be helpful for monitoring validators' states and ledger progress. Some of the
|
||||
timing parameters of rippled will be changed to have faster ledger time. Most if
|
||||
not all test cases do not need client transactions.
|
||||
|
||||
For example, the test cases for the prototype:
|
||||
1. A 10-validator UNL.
|
||||
1. The network does not have other nodes.
|
||||
1. The validators will be started from the genesis. Once they start to produce
|
||||
ledgers, we kill five validators, one every flag ledger interval. Then we
|
||||
will restart them one by one.
|
||||
1. A sequence of events (or the lack of events) such as a killed validator is
|
||||
added to the negative UNL.
|
||||
|
||||
#### Roads Not Taken: Test with Extended CSF
|
||||
|
||||
We considered testing with the current unit test framework, specifically the
|
||||
[Consensus Simulation
|
||||
Framework](https://github.com/ripple/rippled/blob/develop/src/test/csf/README.md)
|
||||
(CSF). However, the CSF currently can only test the generic consensus algorithm
|
||||
as in the paper: [Analysis of the XRP Ledger Consensus
|
||||
Protocol](https://arxiv.org/abs/1802.07242).
|
||||
79
docs/0001-negative-unl/negativeUNLSqDiagram.puml
Normal file
79
docs/0001-negative-unl/negativeUNLSqDiagram.puml
Normal file
@@ -0,0 +1,79 @@
|
||||
@startuml negativeUNL_highLevel_sequence
|
||||
|
||||
skinparam sequenceArrowThickness 2
|
||||
skinparam roundcorner 20
|
||||
skinparam maxmessagesize 160
|
||||
|
||||
actor "Rippled Start" as RS
|
||||
participant "Timer" as T
|
||||
participant "NetworkOPs" as NOP
|
||||
participant "ValidatorList" as VL #lightgreen
|
||||
participant "Consensus" as GC
|
||||
participant "ConsensusAdaptor" as CA #lightgreen
|
||||
participant "Validations" as RM #lightgreen
|
||||
|
||||
RS -> NOP: begin consensus
|
||||
activate NOP
|
||||
NOP -[#green]> VL: <font color=green>update negative UNL
|
||||
hnote over VL#lightgreen: store a copy of\nnegative UNL
|
||||
VL -> NOP
|
||||
NOP -> VL: update trusted validators
|
||||
activate VL
|
||||
VL -> VL: re-calculate quorum
|
||||
hnote over VL#lightgreen: ignore negative listed validators\nwhen calculate quorum
|
||||
VL -> NOP
|
||||
deactivate VL
|
||||
NOP -> GC: start round
|
||||
activate GC
|
||||
GC -> GC: phase = OPEN
|
||||
GC -> NOP
|
||||
deactivate GC
|
||||
deactivate NOP
|
||||
|
||||
loop at regular frequency
|
||||
T -> GC: timerEntry
|
||||
activate GC
|
||||
end
|
||||
|
||||
alt phase == OPEN
|
||||
alt should close ledger
|
||||
GC -> GC: phase = ESTABLISH
|
||||
GC -> CA: onClose
|
||||
activate CA
|
||||
alt sqn%256==0
|
||||
CA -[#green]> RM: <font color=green>getValidations
|
||||
CA -[#green]> CA: <font color=green>create UNLModify Tx
|
||||
hnote over CA#lightgreen: use validatations of the last 256 ledgers\nto figure out UNLModify Tx candidates.\nIf any, create UNLModify Tx, and add to TxSet.
|
||||
end
|
||||
CA -> GC
|
||||
GC -> CA: propose
|
||||
deactivate CA
|
||||
end
|
||||
else phase == ESTABLISH
|
||||
hnote over GC: receive peer postions
|
||||
GC -> GC : update our position
|
||||
GC -> CA : propose \n(if position changed)
|
||||
GC -> GC : check if have consensus
|
||||
alt consensus reached
|
||||
GC -> GC: phase = ACCEPT
|
||||
GC -> CA : onAccept
|
||||
activate CA
|
||||
CA -> CA : build LCL
|
||||
hnote over CA #lightgreen: copy negative UNL from parent ledger
|
||||
alt sqn%256==0
|
||||
CA -[#green]> CA: <font color=green>Adjust negative UNL
|
||||
CA -[#green]> CA: <font color=green>apply UNLModify Tx
|
||||
end
|
||||
CA -> CA : validate and send validation message
|
||||
activate NOP
|
||||
CA -> NOP : end consensus and\n<b>begin next consensus round
|
||||
deactivate NOP
|
||||
deactivate CA
|
||||
hnote over RM: receive validations
|
||||
end
|
||||
else phase == ACCEPTED
|
||||
hnote over GC: timerEntry hash nothing to do at this phase
|
||||
end
|
||||
deactivate GC
|
||||
|
||||
@enduml
|
||||
BIN
docs/0001-negative-unl/negativeUNL_highLevel_sequence.png
Normal file
BIN
docs/0001-negative-unl/negativeUNL_highLevel_sequence.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 138 KiB |
@@ -22,12 +22,14 @@
|
||||
#include <ripple/app/ledger/BuildLedger.h>
|
||||
#include <ripple/app/ledger/InboundLedgers.h>
|
||||
#include <ripple/app/ledger/InboundTransactions.h>
|
||||
#include <ripple/app/ledger/Ledger.h>
|
||||
#include <ripple/app/ledger/LedgerMaster.h>
|
||||
#include <ripple/app/ledger/LocalTxs.h>
|
||||
#include <ripple/app/ledger/OpenLedger.h>
|
||||
#include <ripple/app/misc/AmendmentTable.h>
|
||||
#include <ripple/app/misc/HashRouter.h>
|
||||
#include <ripple/app/misc/LoadFeeTrack.h>
|
||||
#include <ripple/app/misc/NegativeUNLVote.h>
|
||||
#include <ripple/app/misc/NetworkOPs.h>
|
||||
#include <ripple/app/misc/TxQ.h>
|
||||
#include <ripple/app/misc/ValidatorKeys.h>
|
||||
@@ -87,8 +89,10 @@ RCLConsensus::Adaptor::Adaptor(
|
||||
, nodeID_{validatorKeys.nodeID}
|
||||
, valPublic_{validatorKeys.publicKey}
|
||||
, valSecret_{validatorKeys.secretKey}
|
||||
, valCookie_{
|
||||
rand_int<std::uint64_t>(1, std::numeric_limits<std::uint64_t>::max())}
|
||||
, valCookie_{rand_int<std::uint64_t>(
|
||||
1,
|
||||
std::numeric_limits<std::uint64_t>::max())}
|
||||
, nUnlVote_(nodeID_, j_)
|
||||
{
|
||||
assert(valCookie_ != 0);
|
||||
|
||||
@@ -190,7 +194,7 @@ RCLConsensus::Adaptor::propose(RCLCxPeerPos::Proposal const& proposal)
|
||||
prop.set_currenttxhash(
|
||||
proposal.position().begin(), proposal.position().size());
|
||||
prop.set_previousledger(
|
||||
proposal.prevLedger().begin(), proposal.position().size());
|
||||
proposal.prevLedger().begin(), proposal.prevLedger().size());
|
||||
prop.set_proposeseq(proposal.proposeSeq());
|
||||
prop.set_closetime(proposal.closeTime().time_since_epoch().count());
|
||||
|
||||
@@ -317,18 +321,34 @@ RCLConsensus::Adaptor::onClose(
|
||||
}
|
||||
|
||||
// Add pseudo-transactions to the set
|
||||
if ((app_.config().standalone() || (proposing && !wrongLCL)) &&
|
||||
((prevLedger->info().seq % 256) == 0))
|
||||
if (app_.config().standalone() || (proposing && !wrongLCL))
|
||||
{
|
||||
// previous ledger was flag ledger, add pseudo-transactions
|
||||
auto const validations = app_.getValidations().getTrustedForLedger(
|
||||
prevLedger->info().parentHash);
|
||||
|
||||
if (validations.size() >= app_.validators().quorum())
|
||||
if (prevLedger->isFlagLedger())
|
||||
{
|
||||
feeVote_->doVoting(prevLedger, validations, initialSet);
|
||||
app_.getAmendmentTable().doVoting(
|
||||
prevLedger, validations, initialSet);
|
||||
// previous ledger was flag ledger, add fee and amendment
|
||||
// pseudo-transactions
|
||||
auto validations = app_.validators().negativeUNLFilter(
|
||||
app_.getValidations().getTrustedForLedger(
|
||||
prevLedger->info().parentHash));
|
||||
if (validations.size() >= app_.validators().quorum())
|
||||
{
|
||||
feeVote_->doVoting(prevLedger, validations, initialSet);
|
||||
app_.getAmendmentTable().doVoting(
|
||||
prevLedger, validations, initialSet);
|
||||
}
|
||||
}
|
||||
else if (
|
||||
prevLedger->isVotingLedger() &&
|
||||
prevLedger->rules().enabled(featureNegativeUNL))
|
||||
{
|
||||
// previous ledger was a voting ledger,
|
||||
// so the current consensus session is for a flag ledger,
|
||||
// add negative UNL pseudo-transactions
|
||||
nUnlVote_.doVoting(
|
||||
prevLedger,
|
||||
app_.validators().getTrustedMasterKeys(),
|
||||
app_.getValidations(),
|
||||
initialSet);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -793,7 +813,7 @@ RCLConsensus::Adaptor::validate(
|
||||
v.setFieldU64(sfCookie, valCookie_);
|
||||
|
||||
// Report our server version every flag ledger:
|
||||
if ((ledger.seq() + 1) % 256 == 0)
|
||||
if (ledger.ledger_->isVotingLedger())
|
||||
v.setFieldU64(
|
||||
sfServerVersion, BuildInfo::getEncodedVersion());
|
||||
}
|
||||
@@ -808,7 +828,7 @@ RCLConsensus::Adaptor::validate(
|
||||
|
||||
// If the next ledger is a flag ledger, suggest fee changes and
|
||||
// new features:
|
||||
if ((ledger.seq() + 1) % 256 == 0)
|
||||
if (ledger.ledger_->isVotingLedger())
|
||||
{
|
||||
// Fees:
|
||||
feeVote_->doValidation(ledger.ledger_->fees(), v);
|
||||
@@ -922,7 +942,9 @@ RCLConsensus::peerProposal(
|
||||
}
|
||||
|
||||
bool
|
||||
RCLConsensus::Adaptor::preStartRound(RCLCxLedger const& prevLgr)
|
||||
RCLConsensus::Adaptor::preStartRound(
|
||||
RCLCxLedger const& prevLgr,
|
||||
hash_set<NodeID> const& nowTrusted)
|
||||
{
|
||||
// We have a key, we do not want out of sync validations after a restart
|
||||
// and are not amendment blocked.
|
||||
@@ -961,6 +983,11 @@ RCLConsensus::Adaptor::preStartRound(RCLCxLedger const& prevLgr)
|
||||
// Notify inbound ledgers that we are starting a new round
|
||||
inboundTransactions_.newRound(prevLgr.seq());
|
||||
|
||||
// Notify NegativeUNLVote that new validators are added
|
||||
if (prevLgr.ledger_->rules().enabled(featureNegativeUNL) &&
|
||||
!nowTrusted.empty())
|
||||
nUnlVote_.newValidators(prevLgr.seq() + 1, nowTrusted);
|
||||
|
||||
// propose only if we're in sync with the network (and validating)
|
||||
return validating_ && synced;
|
||||
}
|
||||
@@ -1009,10 +1036,15 @@ RCLConsensus::startRound(
|
||||
NetClock::time_point const& now,
|
||||
RCLCxLedger::ID const& prevLgrId,
|
||||
RCLCxLedger const& prevLgr,
|
||||
hash_set<NodeID> const& nowUntrusted)
|
||||
hash_set<NodeID> const& nowUntrusted,
|
||||
hash_set<NodeID> const& nowTrusted)
|
||||
{
|
||||
std::lock_guard _{mutex_};
|
||||
consensus_.startRound(
|
||||
now, prevLgrId, prevLgr, nowUntrusted, adaptor_.preStartRound(prevLgr));
|
||||
now,
|
||||
prevLgrId,
|
||||
prevLgr,
|
||||
nowUntrusted,
|
||||
adaptor_.preStartRound(prevLgr, nowTrusted));
|
||||
}
|
||||
} // namespace ripple
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
#include <ripple/app/consensus/RCLCxPeerPos.h>
|
||||
#include <ripple/app/consensus/RCLCxTx.h>
|
||||
#include <ripple/app/misc/FeeVote.h>
|
||||
#include <ripple/app/misc/NegativeUNLVote.h>
|
||||
#include <ripple/basics/CountedObject.h>
|
||||
#include <ripple/basics/Log.h>
|
||||
#include <ripple/beast/utility/Journal.h>
|
||||
@@ -85,6 +86,7 @@ class RCLConsensus
|
||||
std::atomic<ConsensusMode> mode_{ConsensusMode::observing};
|
||||
|
||||
RCLCensorshipDetector<TxID, LedgerIndex> censorshipDetector_;
|
||||
NegativeUNLVote nUnlVote_;
|
||||
|
||||
public:
|
||||
using Ledger_t = RCLCxLedger;
|
||||
@@ -131,10 +133,13 @@ class RCLConsensus
|
||||
/** Called before kicking off a new consensus round.
|
||||
|
||||
@param prevLedger Ledger that will be prior ledger for next round
|
||||
@param nowTrusted the new validators
|
||||
@return Whether we enter the round proposing
|
||||
*/
|
||||
bool
|
||||
preStartRound(RCLCxLedger const& prevLedger);
|
||||
preStartRound(
|
||||
RCLCxLedger const& prevLedger,
|
||||
hash_set<NodeID> const& nowTrusted);
|
||||
|
||||
bool
|
||||
haveValidated() const;
|
||||
@@ -471,13 +476,16 @@ public:
|
||||
Json::Value
|
||||
getJson(bool full) const;
|
||||
|
||||
//! @see Consensus::startRound
|
||||
/** Adjust the set of trusted validators and kick-off the next round of
|
||||
consensus. For more details, @see Consensus::startRound
|
||||
*/
|
||||
void
|
||||
startRound(
|
||||
NetClock::time_point const& now,
|
||||
RCLCxLedger::ID const& prevLgrId,
|
||||
RCLCxLedger const& prevLgr,
|
||||
hash_set<NodeID> const& nowUntrusted);
|
||||
hash_set<NodeID> const& nowUntrusted,
|
||||
hash_set<NodeID> const& nowTrusted);
|
||||
|
||||
//! @see Consensus::timerEntry
|
||||
void
|
||||
|
||||
@@ -40,6 +40,7 @@
|
||||
#include <ripple/core/SociDB.h>
|
||||
#include <ripple/json/to_string.h>
|
||||
#include <ripple/nodestore/Database.h>
|
||||
#include <ripple/protocol/Feature.h>
|
||||
#include <ripple/protocol/HashPrefix.h>
|
||||
#include <ripple/protocol/Indexes.h>
|
||||
#include <ripple/protocol/PublicKey.h>
|
||||
@@ -598,6 +599,112 @@ Ledger::peek(Keylet const& k) const
|
||||
return sle;
|
||||
}
|
||||
|
||||
hash_set<PublicKey>
|
||||
Ledger::negativeUnl() const
|
||||
{
|
||||
hash_set<PublicKey> negUnl;
|
||||
if (auto sle = read(keylet::negativeUNL());
|
||||
sle && sle->isFieldPresent(sfNegativeUNL))
|
||||
{
|
||||
auto const& nUnlData = sle->getFieldArray(sfNegativeUNL);
|
||||
for (auto const& n : nUnlData)
|
||||
{
|
||||
if (n.isFieldPresent(sfPublicKey))
|
||||
{
|
||||
auto d = n.getFieldVL(sfPublicKey);
|
||||
auto s = makeSlice(d);
|
||||
if (!publicKeyType(s))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
negUnl.emplace(s);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return negUnl;
|
||||
}
|
||||
|
||||
boost::optional<PublicKey>
|
||||
Ledger::negativeUnlToDisable() const
|
||||
{
|
||||
if (auto sle = read(keylet::negativeUNL());
|
||||
sle && sle->isFieldPresent(sfNegativeUNLToDisable))
|
||||
{
|
||||
auto d = sle->getFieldVL(sfNegativeUNLToDisable);
|
||||
auto s = makeSlice(d);
|
||||
if (publicKeyType(s))
|
||||
return PublicKey(s);
|
||||
}
|
||||
|
||||
return boost::none;
|
||||
}
|
||||
|
||||
boost::optional<PublicKey>
|
||||
Ledger::negativeUnlToReEnable() const
|
||||
{
|
||||
if (auto sle = read(keylet::negativeUNL());
|
||||
sle && sle->isFieldPresent(sfNegativeUNLToReEnable))
|
||||
{
|
||||
auto d = sle->getFieldVL(sfNegativeUNLToReEnable);
|
||||
auto s = makeSlice(d);
|
||||
if (publicKeyType(s))
|
||||
return PublicKey(s);
|
||||
}
|
||||
|
||||
return boost::none;
|
||||
}
|
||||
|
||||
void
|
||||
Ledger::updateNegativeUNL()
|
||||
{
|
||||
auto sle = peek(keylet::negativeUNL());
|
||||
if (!sle)
|
||||
return;
|
||||
|
||||
bool const hasToDisable = sle->isFieldPresent(sfNegativeUNLToDisable);
|
||||
bool const hasToReEnable = sle->isFieldPresent(sfNegativeUNLToReEnable);
|
||||
|
||||
if (!hasToDisable && !hasToReEnable)
|
||||
return;
|
||||
|
||||
STArray newNUnl;
|
||||
if (sle->isFieldPresent(sfNegativeUNL))
|
||||
{
|
||||
auto const& oldNUnl = sle->getFieldArray(sfNegativeUNL);
|
||||
for (auto v : oldNUnl)
|
||||
{
|
||||
if (hasToReEnable && v.isFieldPresent(sfPublicKey) &&
|
||||
v.getFieldVL(sfPublicKey) ==
|
||||
sle->getFieldVL(sfNegativeUNLToReEnable))
|
||||
continue;
|
||||
newNUnl.push_back(v);
|
||||
}
|
||||
}
|
||||
|
||||
if (hasToDisable)
|
||||
{
|
||||
newNUnl.emplace_back(sfNegativeUNLEntry);
|
||||
newNUnl.back().setFieldVL(
|
||||
sfPublicKey, sle->getFieldVL(sfNegativeUNLToDisable));
|
||||
newNUnl.back().setFieldU32(sfFirstLedgerSequence, seq());
|
||||
}
|
||||
|
||||
if (!newNUnl.empty())
|
||||
{
|
||||
sle->setFieldArray(sfNegativeUNL, newNUnl);
|
||||
if (hasToReEnable)
|
||||
sle->makeFieldAbsent(sfNegativeUNLToReEnable);
|
||||
if (hasToDisable)
|
||||
sle->makeFieldAbsent(sfNegativeUNLToDisable);
|
||||
rawReplace(sle);
|
||||
}
|
||||
else
|
||||
{
|
||||
rawErase(sle);
|
||||
}
|
||||
}
|
||||
|
||||
//------------------------------------------------------------------------------
|
||||
bool
|
||||
Ledger::walkLedger(beast::Journal j) const
|
||||
@@ -735,6 +842,23 @@ Ledger::updateSkipList()
|
||||
rawReplace(sle);
|
||||
}
|
||||
|
||||
bool
|
||||
Ledger::isFlagLedger() const
|
||||
{
|
||||
return info_.seq % FLAG_LEDGER_INTERVAL == 0;
|
||||
}
|
||||
bool
|
||||
Ledger::isVotingLedger() const
|
||||
{
|
||||
return (info_.seq + 1) % FLAG_LEDGER_INTERVAL == 0;
|
||||
}
|
||||
|
||||
bool
|
||||
isFlagLedger(LedgerIndex seq)
|
||||
{
|
||||
return seq % FLAG_LEDGER_INTERVAL == 0;
|
||||
}
|
||||
|
||||
static bool
|
||||
saveValidatedLedger(
|
||||
Application& app,
|
||||
|
||||
@@ -329,6 +329,46 @@ public:
|
||||
void
|
||||
unshare() const;
|
||||
|
||||
/**
|
||||
* get Negative UNL validators' master public keys
|
||||
*
|
||||
* @return the public keys
|
||||
*/
|
||||
hash_set<PublicKey>
|
||||
negativeUnl() const;
|
||||
|
||||
/**
|
||||
* get the to be disabled validator's master public key if any
|
||||
*
|
||||
* @return the public key if any
|
||||
*/
|
||||
boost::optional<PublicKey>
|
||||
negativeUnlToDisable() const;
|
||||
|
||||
/**
|
||||
* get the to be re-enabled validator's master public key if any
|
||||
*
|
||||
* @return the public key if any
|
||||
*/
|
||||
boost::optional<PublicKey>
|
||||
negativeUnlToReEnable() const;
|
||||
|
||||
/**
|
||||
* update the Negative UNL ledger component.
|
||||
* @note must be called at and only at flag ledgers
|
||||
* must be called before applying UNLModify Tx
|
||||
*/
|
||||
void
|
||||
updateNegativeUNL();
|
||||
|
||||
/** Returns true if the ledger is a flag ledger */
|
||||
bool
|
||||
isFlagLedger() const;
|
||||
|
||||
/** Returns true if the ledger directly precedes a flag ledger */
|
||||
bool
|
||||
isVotingLedger() const;
|
||||
|
||||
private:
|
||||
class sles_iter_impl;
|
||||
class txs_iter_impl;
|
||||
@@ -355,6 +395,11 @@ private:
|
||||
/** A ledger wrapped in a CachedView. */
|
||||
using CachedLedger = CachedView<Ledger>;
|
||||
|
||||
std::uint32_t constexpr FLAG_LEDGER_INTERVAL = 256;
|
||||
/** Returns true if the given ledgerIndex is a flag ledgerIndex */
|
||||
bool
|
||||
isFlagLedger(LedgerIndex seq);
|
||||
|
||||
//------------------------------------------------------------------------------
|
||||
//
|
||||
// API
|
||||
|
||||
@@ -47,6 +47,11 @@ buildLedgerImpl(
|
||||
{
|
||||
auto built = std::make_shared<Ledger>(*parent, closeTime);
|
||||
|
||||
if (built->isFlagLedger() && built->rules().enabled(featureNegativeUNL))
|
||||
{
|
||||
built->updateNegativeUNL();
|
||||
}
|
||||
|
||||
// Set up to write SHAMap changes to our database,
|
||||
// perform updates, extract changes
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@
|
||||
//==============================================================================
|
||||
|
||||
#include <ripple/app/consensus/RCLValidations.h>
|
||||
#include <ripple/app/ledger/Ledger.h>
|
||||
#include <ripple/app/ledger/LedgerMaster.h>
|
||||
#include <ripple/app/ledger/OpenLedger.h>
|
||||
#include <ripple/app/ledger/OrderBookDB.h>
|
||||
@@ -314,14 +315,14 @@ LedgerMaster::setValidLedger(std::shared_ptr<Ledger const> const& l)
|
||||
|
||||
if (!standalone_)
|
||||
{
|
||||
auto const vals =
|
||||
app_.getValidations().getTrustedForLedger(l->info().hash);
|
||||
times.reserve(vals.size());
|
||||
for (auto const& val : vals)
|
||||
auto validations = app_.validators().negativeUNLFilter(
|
||||
app_.getValidations().getTrustedForLedger(l->info().hash));
|
||||
times.reserve(validations.size());
|
||||
for (auto const& val : validations)
|
||||
times.push_back(val->getSignTime());
|
||||
|
||||
if (!vals.empty())
|
||||
consensusHash = vals.front()->getConsensusHash();
|
||||
if (!validations.empty())
|
||||
consensusHash = validations.front()->getConsensusHash();
|
||||
}
|
||||
|
||||
NetClock::time_point signTime;
|
||||
@@ -359,7 +360,7 @@ LedgerMaster::setValidLedger(std::shared_ptr<Ledger const> const& l)
|
||||
"activated: server blocked.";
|
||||
app_.getOPs().setAmendmentBlocked();
|
||||
}
|
||||
else if (!app_.getOPs().isAmendmentWarned() || ((l->seq() % 256) == 0))
|
||||
else if (!app_.getOPs().isAmendmentWarned() || l->isFlagLedger())
|
||||
{
|
||||
// Amendments can lose majority, so re-check periodically (every
|
||||
// flag ledger), and clear the flag if appropriate. If an unknown
|
||||
@@ -941,8 +942,9 @@ LedgerMaster::checkAccept(uint256 const& hash, std::uint32_t seq)
|
||||
if (seq < mValidLedgerSeq)
|
||||
return;
|
||||
|
||||
valCount = app_.getValidations().numTrustedForLedger(hash);
|
||||
|
||||
auto validations = app_.validators().negativeUNLFilter(
|
||||
app_.getValidations().getTrustedForLedger(hash));
|
||||
valCount = validations.size();
|
||||
if (valCount >= app_.validators().quorum())
|
||||
{
|
||||
std::lock_guard ml(m_mutex);
|
||||
@@ -1006,8 +1008,9 @@ LedgerMaster::checkAccept(std::shared_ptr<Ledger const> const& ledger)
|
||||
return;
|
||||
|
||||
auto const minVal = getNeededValidations();
|
||||
auto const tvc =
|
||||
app_.getValidations().numTrustedForLedger(ledger->info().hash);
|
||||
auto validations = app_.validators().negativeUNLFilter(
|
||||
app_.getValidations().getTrustedForLedger(ledger->info().hash));
|
||||
auto const tvc = validations.size();
|
||||
if (tvc < minVal) // nothing we can do
|
||||
{
|
||||
JLOG(m_journal.trace())
|
||||
@@ -1162,7 +1165,8 @@ LedgerMaster::consensusBuilt(
|
||||
// This ledger cannot be the new fully-validated ledger, but
|
||||
// maybe we saved up validations for some other ledger that can be
|
||||
|
||||
auto const val = app_.getValidations().currentTrusted();
|
||||
auto validations = app_.validators().negativeUNLFilter(
|
||||
app_.getValidations().currentTrusted());
|
||||
|
||||
// Track validation counts with sequence numbers
|
||||
class valSeq
|
||||
@@ -1189,7 +1193,7 @@ LedgerMaster::consensusBuilt(
|
||||
|
||||
// Count the number of current, trusted validations
|
||||
hash_map<uint256, valSeq> count;
|
||||
for (auto const& v : val)
|
||||
for (auto const& v : validations)
|
||||
{
|
||||
valSeq& vs = count[v->getLedgerHash()];
|
||||
vs.mergeValidation(v->getFieldU32(sfLedgerSequence));
|
||||
|
||||
@@ -17,6 +17,7 @@
|
||||
*/
|
||||
//==============================================================================
|
||||
|
||||
#include <ripple/app/ledger/Ledger.h>
|
||||
#include <ripple/app/main/Application.h>
|
||||
#include <ripple/app/misc/FeeVote.h>
|
||||
#include <ripple/basics/BasicConfig.h>
|
||||
@@ -150,7 +151,7 @@ FeeVoteImpl::doVoting(
|
||||
std::shared_ptr<SHAMap> const& initialPosition)
|
||||
{
|
||||
// LCL must be flag ledger
|
||||
assert((lastClosedLedger->info().seq % 256) == 0);
|
||||
assert(isFlagLedger(lastClosedLedger->seq()));
|
||||
|
||||
detail::VotableValue baseFeeVote(
|
||||
lastClosedLedger->fees().base, target_.reference_fee);
|
||||
|
||||
350
src/ripple/app/misc/NegativeUNLVote.cpp
Normal file
350
src/ripple/app/misc/NegativeUNLVote.cpp
Normal file
@@ -0,0 +1,350 @@
|
||||
//------------------------------------------------------------------------------
|
||||
/*
|
||||
This file is part of rippled: https://github.com/ripple/rippled
|
||||
Copyright (c) 2020 Ripple Labs Inc.
|
||||
|
||||
Permission to use, copy, modify, and/or distribute this software for any
|
||||
purpose with or without fee is hereby granted, provided that the above
|
||||
copyright notice and this permission notice appear in all copies.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
|
||||
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
|
||||
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
|
||||
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
|
||||
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
|
||||
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
|
||||
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||
*/
|
||||
//==============================================================================
|
||||
|
||||
#include <ripple/app/consensus/RCLValidations.h>
|
||||
#include <ripple/app/ledger/Ledger.h>
|
||||
#include <ripple/app/misc/NegativeUNLVote.h>
|
||||
|
||||
namespace ripple {
|
||||
|
||||
NegativeUNLVote::NegativeUNLVote(NodeID const& myId, beast::Journal j)
|
||||
: myId_(myId), j_(j)
|
||||
{
|
||||
}
|
||||
|
||||
void
|
||||
NegativeUNLVote::doVoting(
|
||||
std::shared_ptr<Ledger const> const& prevLedger,
|
||||
hash_set<PublicKey> const& unlKeys,
|
||||
RCLValidations& validations,
|
||||
std::shared_ptr<SHAMap> const& initialSet)
|
||||
{
|
||||
// Voting steps:
|
||||
// -- build a reliability score table of validators
|
||||
// -- process the table and find all candidates to disable or to re-enable
|
||||
// -- pick one to disable and one to re-enable if any
|
||||
// -- if found candidates, add ttUNL_MODIFY Tx
|
||||
|
||||
// Build NodeID set for internal use.
|
||||
// Build NodeID to PublicKey map for lookup before creating ttUNL_MODIFY Tx.
|
||||
hash_set<NodeID> unlNodeIDs;
|
||||
hash_map<NodeID, PublicKey> nidToKeyMap;
|
||||
for (auto const& k : unlKeys)
|
||||
{
|
||||
auto nid = calcNodeID(k);
|
||||
nidToKeyMap.emplace(nid, k);
|
||||
unlNodeIDs.emplace(nid);
|
||||
}
|
||||
|
||||
// Build a reliability score table of validators
|
||||
if (std::optional<hash_map<NodeID, std::uint32_t>> scoreTable =
|
||||
buildScoreTable(prevLedger, unlNodeIDs, validations))
|
||||
{
|
||||
// build next negUnl
|
||||
auto negUnlKeys = prevLedger->negativeUnl();
|
||||
auto negUnlToDisable = prevLedger->negativeUnlToDisable();
|
||||
auto negUnlToReEnable = prevLedger->negativeUnlToReEnable();
|
||||
if (negUnlToDisable)
|
||||
negUnlKeys.insert(*negUnlToDisable);
|
||||
if (negUnlToReEnable)
|
||||
negUnlKeys.erase(*negUnlToReEnable);
|
||||
|
||||
hash_set<NodeID> negUnlNodeIDs;
|
||||
for (auto const& k : negUnlKeys)
|
||||
{
|
||||
auto nid = calcNodeID(k);
|
||||
negUnlNodeIDs.emplace(nid);
|
||||
if (!nidToKeyMap.count(nid))
|
||||
{
|
||||
nidToKeyMap.emplace(nid, k);
|
||||
}
|
||||
}
|
||||
|
||||
auto const seq = prevLedger->info().seq + 1;
|
||||
purgeNewValidators(seq);
|
||||
|
||||
// Process the table and find all candidates to disable or to re-enable
|
||||
auto const candidates =
|
||||
findAllCandidates(unlNodeIDs, negUnlNodeIDs, *scoreTable);
|
||||
|
||||
// Pick one to disable and one to re-enable if any, add ttUNL_MODIFY Tx
|
||||
if (!candidates.toDisableCandidates.empty())
|
||||
{
|
||||
auto n =
|
||||
choose(prevLedger->info().hash, candidates.toDisableCandidates);
|
||||
assert(nidToKeyMap.count(n));
|
||||
addTx(seq, nidToKeyMap[n], ToDisable, initialSet);
|
||||
}
|
||||
|
||||
if (!candidates.toReEnableCandidates.empty())
|
||||
{
|
||||
auto n = choose(
|
||||
prevLedger->info().hash, candidates.toReEnableCandidates);
|
||||
assert(nidToKeyMap.count(n));
|
||||
addTx(seq, nidToKeyMap[n], ToReEnable, initialSet);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void
|
||||
NegativeUNLVote::addTx(
|
||||
LedgerIndex seq,
|
||||
PublicKey const& vp,
|
||||
NegativeUNLModify modify,
|
||||
std::shared_ptr<SHAMap> const& initialSet)
|
||||
{
|
||||
STTx negUnlTx(ttUNL_MODIFY, [&](auto& obj) {
|
||||
obj.setFieldU8(sfUNLModifyDisabling, modify == ToDisable ? 1 : 0);
|
||||
obj.setFieldU32(sfLedgerSequence, seq);
|
||||
obj.setFieldVL(sfUNLModifyValidator, vp.slice());
|
||||
});
|
||||
|
||||
uint256 txID = negUnlTx.getTransactionID();
|
||||
Serializer s;
|
||||
negUnlTx.add(s);
|
||||
if (!initialSet->addGiveItem(
|
||||
std::make_shared<SHAMapItem>(txID, s.peekData()), true, false))
|
||||
{
|
||||
JLOG(j_.warn()) << "N-UNL: ledger seq=" << seq
|
||||
<< ", add ttUNL_MODIFY tx failed";
|
||||
}
|
||||
else
|
||||
{
|
||||
JLOG(j_.debug()) << "N-UNL: ledger seq=" << seq
|
||||
<< ", add a ttUNL_MODIFY Tx with txID: " << txID
|
||||
<< ", the validator to "
|
||||
<< (modify == ToDisable ? "disable: " : "re-enable: ")
|
||||
<< vp;
|
||||
}
|
||||
}
|
||||
|
||||
NodeID
|
||||
NegativeUNLVote::choose(
|
||||
uint256 const& randomPadData,
|
||||
std::vector<NodeID> const& candidates)
|
||||
{
|
||||
assert(!candidates.empty());
|
||||
static_assert(NodeID::bytes <= uint256::bytes);
|
||||
NodeID randomPad = NodeID::fromVoid(randomPadData.data());
|
||||
NodeID txNodeID = candidates[0];
|
||||
for (int j = 1; j < candidates.size(); ++j)
|
||||
{
|
||||
if ((candidates[j] ^ randomPad) < (txNodeID ^ randomPad))
|
||||
{
|
||||
txNodeID = candidates[j];
|
||||
}
|
||||
}
|
||||
return txNodeID;
|
||||
}
|
||||
|
||||
std::optional<hash_map<NodeID, std::uint32_t>>
|
||||
NegativeUNLVote::buildScoreTable(
|
||||
std::shared_ptr<Ledger const> const& prevLedger,
|
||||
hash_set<NodeID> const& unl,
|
||||
RCLValidations& validations)
|
||||
{
|
||||
// Find agreed validation messages received for
|
||||
// the last FLAG_LEDGER_INTERVAL (i.e. 256) ledgers,
|
||||
// for every validator, and fill the score table.
|
||||
|
||||
// Ask the validation container to keep enough validation message history
|
||||
// for next time.
|
||||
auto const seq = prevLedger->info().seq + 1;
|
||||
validations.setSeqToKeep(seq - 1);
|
||||
|
||||
// Find FLAG_LEDGER_INTERVAL (i.e. 256) previous ledger hashes
|
||||
auto const hashIndex = prevLedger->read(keylet::skip());
|
||||
if (!hashIndex || !hashIndex->isFieldPresent(sfHashes))
|
||||
{
|
||||
JLOG(j_.debug()) << "N-UNL: ledger " << seq << " no history.";
|
||||
return {};
|
||||
}
|
||||
auto const ledgerAncestors = hashIndex->getFieldV256(sfHashes).value();
|
||||
auto const numAncestors = ledgerAncestors.size();
|
||||
if (numAncestors < FLAG_LEDGER_INTERVAL)
|
||||
{
|
||||
JLOG(j_.debug()) << "N-UNL: ledger " << seq
|
||||
<< " not enough history. Can trace back only "
|
||||
<< numAncestors << " ledgers.";
|
||||
return {};
|
||||
}
|
||||
|
||||
// have enough ledger ancestors, build the score table
|
||||
hash_map<NodeID, std::uint32_t> scoreTable;
|
||||
for (auto const& k : unl)
|
||||
{
|
||||
scoreTable[k] = 0;
|
||||
}
|
||||
|
||||
// Query the validation container for every ledger hash and fill
|
||||
// the score table.
|
||||
for (int i = 0; i < FLAG_LEDGER_INTERVAL; ++i)
|
||||
{
|
||||
for (auto const& v : validations.getTrustedForLedger(
|
||||
ledgerAncestors[numAncestors - 1 - i]))
|
||||
{
|
||||
if (scoreTable.count(v->getNodeID()))
|
||||
++scoreTable[v->getNodeID()];
|
||||
}
|
||||
}
|
||||
|
||||
// Return false if the validation message history or local node's
|
||||
// participation in the history is not good.
|
||||
auto const myValidationCount = [&]() -> std::uint32_t {
|
||||
if (auto const it = scoreTable.find(myId_); it != scoreTable.end())
|
||||
return it->second;
|
||||
return 0;
|
||||
}();
|
||||
if (myValidationCount < negativeUnlMinLocalValsToVote)
|
||||
{
|
||||
JLOG(j_.debug()) << "N-UNL: ledger " << seq
|
||||
<< ". Local node only issued " << myValidationCount
|
||||
<< " validations in last " << FLAG_LEDGER_INTERVAL
|
||||
<< " ledgers."
|
||||
<< " The reliability measurement could be wrong.";
|
||||
return {};
|
||||
}
|
||||
else if (
|
||||
myValidationCount > negativeUnlMinLocalValsToVote &&
|
||||
myValidationCount <= FLAG_LEDGER_INTERVAL)
|
||||
{
|
||||
return scoreTable;
|
||||
}
|
||||
else
|
||||
{
|
||||
// cannot happen because validations.getTrustedForLedger does not
|
||||
// return multiple validations of the same ledger from a validator.
|
||||
JLOG(j_.error()) << "N-UNL: ledger " << seq << ". Local node issued "
|
||||
<< myValidationCount << " validations in last "
|
||||
<< FLAG_LEDGER_INTERVAL << " ledgers. Too many!";
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
NegativeUNLVote::Candidates const
|
||||
NegativeUNLVote::findAllCandidates(
|
||||
hash_set<NodeID> const& unl,
|
||||
hash_set<NodeID> const& negUnl,
|
||||
hash_map<NodeID, std::uint32_t> const& scoreTable)
|
||||
{
|
||||
// Compute if need to find more validators to disable
|
||||
auto const canAdd = [&]() -> bool {
|
||||
auto const maxNegativeListed = static_cast<std::size_t>(
|
||||
std::ceil(unl.size() * negativeUnlMaxListed));
|
||||
std::size_t negativeListed = 0;
|
||||
for (auto const& n : unl)
|
||||
{
|
||||
if (negUnl.count(n))
|
||||
++negativeListed;
|
||||
}
|
||||
bool const result = negativeListed < maxNegativeListed;
|
||||
JLOG(j_.trace()) << "N-UNL: nodeId " << myId_ << " lowWaterMark "
|
||||
<< negativeUnlLowWaterMark << " highWaterMark "
|
||||
<< negativeUnlHighWaterMark << " canAdd " << result
|
||||
<< " negativeListed " << negativeListed
|
||||
<< " maxNegativeListed " << maxNegativeListed;
|
||||
return result;
|
||||
}();
|
||||
|
||||
Candidates candidates;
|
||||
for (auto const& [nodeId, score] : scoreTable)
|
||||
{
|
||||
JLOG(j_.trace()) << "N-UNL: node " << nodeId << " score " << score;
|
||||
|
||||
// Find toDisable Candidates: check if
|
||||
// (1) canAdd,
|
||||
// (2) has less than negativeUnlLowWaterMark validations,
|
||||
// (3) is not in negUnl, and
|
||||
// (4) is not a new validator.
|
||||
if (canAdd && score < negativeUnlLowWaterMark &&
|
||||
!negUnl.count(nodeId) && !newValidators_.count(nodeId))
|
||||
{
|
||||
JLOG(j_.trace()) << "N-UNL: toDisable candidate " << nodeId;
|
||||
candidates.toDisableCandidates.push_back(nodeId);
|
||||
}
|
||||
|
||||
// Find toReEnable Candidates: check if
|
||||
// (1) has more than negativeUnlHighWaterMark validations,
|
||||
// (2) is in negUnl
|
||||
if (score > negativeUnlHighWaterMark && negUnl.count(nodeId))
|
||||
{
|
||||
JLOG(j_.trace()) << "N-UNL: toReEnable candidate " << nodeId;
|
||||
candidates.toReEnableCandidates.push_back(nodeId);
|
||||
}
|
||||
}
|
||||
|
||||
// If a negative UNL validator is removed from nodes' UNLs, it is no longer
|
||||
// a validator. It should be removed from the negative UNL too.
|
||||
// Note that even if it is still offline and in minority nodes' UNLs, it
|
||||
// will not be re-added to the negative UNL. Because the UNLModify Tx will
|
||||
// not be included in the agreed TxSet of a ledger.
|
||||
//
|
||||
// Find this kind of toReEnable Candidate if did not find any toReEnable
|
||||
// candidate yet: check if
|
||||
// (1) is in negUnl
|
||||
// (2) is not in unl.
|
||||
if (candidates.toReEnableCandidates.empty())
|
||||
{
|
||||
for (auto const& n : negUnl)
|
||||
{
|
||||
if (!unl.count(n))
|
||||
{
|
||||
candidates.toReEnableCandidates.push_back(n);
|
||||
}
|
||||
}
|
||||
}
|
||||
return candidates;
|
||||
}
|
||||
|
||||
void
|
||||
NegativeUNLVote::newValidators(
|
||||
LedgerIndex seq,
|
||||
hash_set<NodeID> const& nowTrusted)
|
||||
{
|
||||
std::lock_guard lock(mutex_);
|
||||
for (auto const& n : nowTrusted)
|
||||
{
|
||||
if (newValidators_.find(n) == newValidators_.end())
|
||||
{
|
||||
JLOG(j_.trace()) << "N-UNL: add a new validator " << n
|
||||
<< " at ledger seq=" << seq;
|
||||
newValidators_[n] = seq;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void
|
||||
NegativeUNLVote::purgeNewValidators(LedgerIndex seq)
|
||||
{
|
||||
std::lock_guard lock(mutex_);
|
||||
auto i = newValidators_.begin();
|
||||
while (i != newValidators_.end())
|
||||
{
|
||||
if (seq - i->second > newValidatorDisableSkip)
|
||||
{
|
||||
i = newValidators_.erase(i);
|
||||
}
|
||||
else
|
||||
{
|
||||
++i;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace ripple
|
||||
217
src/ripple/app/misc/NegativeUNLVote.h
Normal file
217
src/ripple/app/misc/NegativeUNLVote.h
Normal file
@@ -0,0 +1,217 @@
|
||||
//------------------------------------------------------------------------------
|
||||
/*
|
||||
This file is part of rippled: https://github.com/ripple/rippled
|
||||
Copyright (c) 2020 Ripple Labs Inc.
|
||||
|
||||
Permission to use, copy, modify, and/or distribute this software for any
|
||||
purpose with or without fee is hereby granted, provided that the above
|
||||
copyright notice and this permission notice appear in all copies.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
|
||||
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
|
||||
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
|
||||
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
|
||||
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
|
||||
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
|
||||
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||
*/
|
||||
//==============================================================================
|
||||
|
||||
#ifndef RIPPLE_APP_MISC_NEGATIVEUNLVOTE_H_INCLUDED
|
||||
#define RIPPLE_APP_MISC_NEGATIVEUNLVOTE_H_INCLUDED
|
||||
|
||||
#include <ripple/app/ledger/Ledger.h>
|
||||
#include <ripple/beast/utility/Journal.h>
|
||||
#include <ripple/protocol/Protocol.h>
|
||||
#include <ripple/protocol/PublicKey.h>
|
||||
#include <ripple/protocol/UintTypes.h>
|
||||
|
||||
#include <optional>
|
||||
|
||||
namespace ripple {
|
||||
|
||||
template <class Adaptor>
|
||||
class Validations;
|
||||
class RCLValidationsAdaptor;
|
||||
using RCLValidations = Validations<RCLValidationsAdaptor>;
|
||||
class SHAMap;
|
||||
namespace test {
|
||||
class NegativeUNLVoteInternal_test;
|
||||
class NegativeUNLVoteScoreTable_test;
|
||||
} // namespace test
|
||||
|
||||
/**
|
||||
* Manager to create NegativeUNL votes.
|
||||
*/
|
||||
class NegativeUNLVote final
|
||||
{
|
||||
public:
|
||||
/**
|
||||
* A validator is considered unreliable if its validations is less than
|
||||
* negativeUnlLowWaterMark in the last flag ledger period.
|
||||
* An unreliable validator is a candidate to be disabled by the NegativeUNL
|
||||
* protocol.
|
||||
*/
|
||||
static constexpr size_t negativeUnlLowWaterMark =
|
||||
FLAG_LEDGER_INTERVAL * 50 / 100;
|
||||
/**
|
||||
* An unreliable validator must have more than negativeUnlHighWaterMark
|
||||
* validations in the last flag ledger period to be re-enabled.
|
||||
*/
|
||||
static constexpr size_t negativeUnlHighWaterMark =
|
||||
FLAG_LEDGER_INTERVAL * 80 / 100;
|
||||
/**
|
||||
* The minimum number of validations of the local node for it to
|
||||
* participate in the voting.
|
||||
*/
|
||||
static constexpr size_t negativeUnlMinLocalValsToVote =
|
||||
FLAG_LEDGER_INTERVAL * 90 / 100;
|
||||
/**
|
||||
* We don't want to disable new validators immediately after adding them.
|
||||
* So we skip voting for disabling them for 2 flag ledgers.
|
||||
*/
|
||||
static constexpr size_t newValidatorDisableSkip = FLAG_LEDGER_INTERVAL * 2;
|
||||
/**
|
||||
* We only want to put 25% of the UNL on the NegativeUNL.
|
||||
*/
|
||||
static constexpr float negativeUnlMaxListed = 0.25;
|
||||
|
||||
/**
|
||||
* A flag indicating whether a UNLModify Tx is to disable or to re-enable
|
||||
* a validator.
|
||||
*/
|
||||
enum NegativeUNLModify {
|
||||
ToDisable, // UNLModify Tx is to disable a validator
|
||||
ToReEnable // UNLModify Tx is to re-enable a validator
|
||||
};
|
||||
|
||||
/**
|
||||
* Constructor
|
||||
*
|
||||
* @param myId the NodeID of the local node
|
||||
* @param j log
|
||||
*/
|
||||
NegativeUNLVote(NodeID const& myId, beast::Journal j);
|
||||
~NegativeUNLVote() = default;
|
||||
|
||||
/**
|
||||
* Cast our local vote on the NegativeUNL candidates.
|
||||
*
|
||||
* @param prevLedger the parent ledger
|
||||
* @param unlKeys the trusted master keys of validators in the UNL
|
||||
* @param validations the validation message container
|
||||
* @note validations is an in/out parameter. It contains validation messages
|
||||
* that will be deleted when no longer needed by other consensus logic. This
|
||||
* function asks it to keep the validation messages long enough for this
|
||||
* function to use.
|
||||
* @param initialSet the transactions set for adding ttUNL_MODIFY Tx if any
|
||||
*/
|
||||
void
|
||||
doVoting(
|
||||
std::shared_ptr<Ledger const> const& prevLedger,
|
||||
hash_set<PublicKey> const& unlKeys,
|
||||
RCLValidations& validations,
|
||||
std::shared_ptr<SHAMap> const& initialSet);
|
||||
|
||||
/**
|
||||
* Notify NegativeUNLVote that new validators are added.
|
||||
* So that they don't get voted to the NegativeUNL immediately.
|
||||
*
|
||||
* @param seq the current LedgerIndex when adding the new validators
|
||||
* @param nowTrusted the new validators
|
||||
*/
|
||||
void
|
||||
newValidators(LedgerIndex seq, hash_set<NodeID> const& nowTrusted);
|
||||
|
||||
private:
|
||||
NodeID const myId_;
|
||||
beast::Journal j_;
|
||||
mutable std::mutex mutex_;
|
||||
hash_map<NodeID, LedgerIndex> newValidators_;
|
||||
|
||||
/**
|
||||
* UNLModify Tx candidates
|
||||
*/
|
||||
struct Candidates
|
||||
{
|
||||
std::vector<NodeID> toDisableCandidates;
|
||||
std::vector<NodeID> toReEnableCandidates;
|
||||
};
|
||||
|
||||
/**
|
||||
* Add a ttUNL_MODIFY Tx to the transaction set.
|
||||
*
|
||||
* @param seq the LedgerIndex when adding the Tx
|
||||
* @param vp the master public key of the validator
|
||||
* @param modify disabling or re-enabling the validator
|
||||
* @param initialSet the transaction set
|
||||
*/
|
||||
void
|
||||
addTx(
|
||||
LedgerIndex seq,
|
||||
PublicKey const& vp,
|
||||
NegativeUNLModify modify,
|
||||
std::shared_ptr<SHAMap> const& initialSet);
|
||||
|
||||
/**
|
||||
* Pick one candidate from a vector of candidates.
|
||||
*
|
||||
* @param randomPadData the data used for picking a candidate.
|
||||
* @note Nodes must use the same randomPadData for picking the same
|
||||
* candidate. The hash of the parent ledger is used.
|
||||
* @param candidates the vector of candidates
|
||||
* @return the picked candidate
|
||||
*/
|
||||
NodeID
|
||||
choose(uint256 const& randomPadData, std::vector<NodeID> const& candidates);
|
||||
|
||||
/**
|
||||
* Build a reliability measurement score table of validators' validation
|
||||
* messages in the last flag ledger period.
|
||||
*
|
||||
* @param prevLedger the parent ledger
|
||||
* @param unl the trusted master keys
|
||||
* @param validations the validation container
|
||||
* @note validations is an in/out parameter. It contains validation messages
|
||||
* that will be deleted when no longer needed by other consensus logic. This
|
||||
* function asks it to keep the validation messages long enough for this
|
||||
* function to use.
|
||||
* @return the built scoreTable or empty optional if table could not be
|
||||
* built
|
||||
*/
|
||||
std::optional<hash_map<NodeID, std::uint32_t>>
|
||||
buildScoreTable(
|
||||
std::shared_ptr<Ledger const> const& prevLedger,
|
||||
hash_set<NodeID> const& unl,
|
||||
RCLValidations& validations);
|
||||
|
||||
/**
|
||||
* Process the score table and find all disabling and re-enabling
|
||||
* candidates.
|
||||
*
|
||||
* @param unl the trusted master keys
|
||||
* @param negUnl the NegativeUNL
|
||||
* @param scoreTable the score table
|
||||
* @return the candidates to disable and the candidates to re-enable
|
||||
*/
|
||||
Candidates const
|
||||
findAllCandidates(
|
||||
hash_set<NodeID> const& unl,
|
||||
hash_set<NodeID> const& negUnl,
|
||||
hash_map<NodeID, std::uint32_t> const& scoreTable);
|
||||
|
||||
/**
|
||||
* Purge validators that are not new anymore.
|
||||
*
|
||||
* @param seq the current LedgerIndex
|
||||
*/
|
||||
void
|
||||
purgeNewValidators(LedgerIndex seq);
|
||||
|
||||
friend class test::NegativeUNLVoteInternal_test;
|
||||
friend class test::NegativeUNLVoteScoreTable_test;
|
||||
};
|
||||
|
||||
} // namespace ripple
|
||||
|
||||
#endif
|
||||
@@ -56,6 +56,7 @@
|
||||
#include <ripple/overlay/Overlay.h>
|
||||
#include <ripple/overlay/predicates.h>
|
||||
#include <ripple/protocol/BuildInfo.h>
|
||||
#include <ripple/protocol/Feature.h>
|
||||
#include <ripple/resource/ResourceManager.h>
|
||||
#include <ripple/rpc/DeliveredAmount.h>
|
||||
#include <boost/asio/ip/host_name.hpp>
|
||||
@@ -1749,6 +1750,8 @@ NetworkOPsImp::beginConsensus(uint256 const& networkClosed)
|
||||
closingInfo.parentHash ==
|
||||
m_ledgerMaster.getClosedLedger()->info().hash);
|
||||
|
||||
if (prevLedger->rules().enabled(featureNegativeUNL))
|
||||
app_.validators().setNegativeUnl(prevLedger->negativeUnl());
|
||||
TrustChanges const changes = app_.validators().updateTrusted(
|
||||
app_.getValidations().getCurrentNodeIDs());
|
||||
|
||||
@@ -1759,7 +1762,8 @@ NetworkOPsImp::beginConsensus(uint256 const& networkClosed)
|
||||
app_.timeKeeper().closeTime(),
|
||||
networkClosed,
|
||||
prevLedger,
|
||||
changes.removed);
|
||||
changes.removed,
|
||||
changes.added);
|
||||
|
||||
const ConsensusPhase currPhase = mConsensus.phase();
|
||||
if (mLastConsensusPhase != currPhase)
|
||||
|
||||
@@ -38,6 +38,7 @@ namespace ripple {
|
||||
// predeclaration
|
||||
class Overlay;
|
||||
class HashRouter;
|
||||
class STValidation;
|
||||
|
||||
enum class ListDisposition {
|
||||
/// List is valid
|
||||
@@ -159,6 +160,9 @@ class ValidatorList
|
||||
|
||||
PublicKey localPubKey_;
|
||||
|
||||
// The master public keys of the current negative UNL
|
||||
hash_set<PublicKey> negativeUnl_;
|
||||
|
||||
// Currently supported version of publisher list format
|
||||
static constexpr std::uint32_t requiredListVersion = 1;
|
||||
static const std::string filePrefix_;
|
||||
@@ -505,6 +509,37 @@ public:
|
||||
return {quorum_, trustedSigningKeys_};
|
||||
}
|
||||
|
||||
/**
|
||||
* get the trusted master public keys
|
||||
* @return the public keys
|
||||
*/
|
||||
hash_set<PublicKey>
|
||||
getTrustedMasterKeys() const;
|
||||
|
||||
/**
|
||||
* get the master public keys of Negative UNL validators
|
||||
* @return the master public keys
|
||||
*/
|
||||
hash_set<PublicKey>
|
||||
getNegativeUnl() const;
|
||||
|
||||
/**
|
||||
* set the Negative UNL with validators' master public keys
|
||||
* @param negUnl the public keys
|
||||
*/
|
||||
void
|
||||
setNegativeUnl(hash_set<PublicKey> const& negUnl);
|
||||
|
||||
/**
|
||||
* Remove validations that are from validators on the negative UNL.
|
||||
*
|
||||
* @param validations the validations to filter
|
||||
* @return a filtered copy of the validations
|
||||
*/
|
||||
std::vector<std::shared_ptr<STValidation>>
|
||||
negativeUNLFilter(
|
||||
std::vector<std::shared_ptr<STValidation>>&& validations) const;
|
||||
|
||||
private:
|
||||
/** Get the filename used for caching UNLs
|
||||
*/
|
||||
@@ -547,12 +582,19 @@ private:
|
||||
|
||||
/** Return quorum for trusted validator set
|
||||
|
||||
@param trusted Number of trusted validator keys
|
||||
@param unlSize Number of trusted validator keys
|
||||
|
||||
@param seen Number of trusted validators that have signed
|
||||
recently received validations */
|
||||
@param effectiveUnlSize Number of trusted validator keys that are not in
|
||||
the NegativeUNL
|
||||
|
||||
@param seenSize Number of trusted validators that have signed
|
||||
recently received validations
|
||||
*/
|
||||
std::size_t
|
||||
calculateQuorum(std::size_t trusted, std::size_t seen);
|
||||
calculateQuorum(
|
||||
std::size_t unlSize,
|
||||
std::size_t effectiveUnlSize,
|
||||
std::size_t seenSize);
|
||||
};
|
||||
} // namespace ripple
|
||||
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
#include <ripple/basics/base64.h>
|
||||
#include <ripple/json/json_reader.h>
|
||||
#include <ripple/overlay/Overlay.h>
|
||||
#include <ripple/protocol/STValidation.h>
|
||||
#include <ripple/protocol/jss.h>
|
||||
#include <ripple/protocol/messages.h>
|
||||
#include <boost/regex.hpp>
|
||||
@@ -746,6 +747,16 @@ ValidatorList::getJson() const
|
||||
}
|
||||
});
|
||||
|
||||
// Negative UNL
|
||||
if (!negativeUnl_.empty())
|
||||
{
|
||||
Json::Value& jNegativeUNL = (res[jss::NegativeUNL] = Json::arrayValue);
|
||||
for (auto const& k : negativeUnl_)
|
||||
{
|
||||
jNegativeUNL.append(toBase58(TokenType::NodePublic, k));
|
||||
}
|
||||
}
|
||||
|
||||
return res;
|
||||
}
|
||||
|
||||
@@ -818,7 +829,10 @@ ValidatorList::getAvailable(boost::beast::string_view const& pubKey)
|
||||
}
|
||||
|
||||
std::size_t
|
||||
ValidatorList::calculateQuorum(std::size_t trusted, std::size_t seen)
|
||||
ValidatorList::calculateQuorum(
|
||||
std::size_t unlSize,
|
||||
std::size_t effectiveUnlSize,
|
||||
std::size_t seenSize)
|
||||
{
|
||||
// Do not use achievable quorum until lists from all configured
|
||||
// publishers are available
|
||||
@@ -858,11 +872,16 @@ ValidatorList::calculateQuorum(std::size_t trusted, std::size_t seen)
|
||||
// Oi,j > nj/2 + ni − qi + ti,j
|
||||
// ni - pi > (ni - pi + pj)/2 + ni − .8*ni + .2*ni
|
||||
// pi + pj < .2*ni
|
||||
auto quorum = static_cast<std::size_t>(std::ceil(trusted * 0.8f));
|
||||
//
|
||||
// Note that the negative UNL protocol introduced the AbsoluteMinimumQuorum
|
||||
// which is 60% of the original UNL size. The effective quorum should
|
||||
// not be lower than it.
|
||||
auto quorum = static_cast<std::size_t>(std::max(
|
||||
std::ceil(effectiveUnlSize * 0.8f), std::ceil(unlSize * 0.6f)));
|
||||
|
||||
// Use lower quorum specified via command line if the normal quorum appears
|
||||
// unreachable based on the number of recently received validations.
|
||||
if (minimumQuorum_ && *minimumQuorum_ < quorum && seen < quorum)
|
||||
if (minimumQuorum_ && *minimumQuorum_ < quorum && seenSize < quorum)
|
||||
{
|
||||
quorum = *minimumQuorum_;
|
||||
|
||||
@@ -922,7 +941,28 @@ ValidatorList::updateTrusted(hash_set<NodeID> const& seenValidators)
|
||||
<< trustedMasterKeys_.size() << " of " << keyListings_.size()
|
||||
<< " listed validators eligible for inclusion in the trusted set";
|
||||
|
||||
quorum_ = calculateQuorum(trustedMasterKeys_.size(), seenValidators.size());
|
||||
auto unlSize = trustedMasterKeys_.size();
|
||||
auto effectiveUnlSize = unlSize;
|
||||
auto seenSize = seenValidators.size();
|
||||
if (!negativeUnl_.empty())
|
||||
{
|
||||
for (auto const& k : trustedMasterKeys_)
|
||||
{
|
||||
if (negativeUnl_.count(k))
|
||||
--effectiveUnlSize;
|
||||
}
|
||||
hash_set<NodeID> negUnlNodeIDs;
|
||||
for (auto const& k : negativeUnl_)
|
||||
{
|
||||
negUnlNodeIDs.emplace(calcNodeID(k));
|
||||
}
|
||||
for (auto const& nid : seenValidators)
|
||||
{
|
||||
if (negUnlNodeIDs.count(nid))
|
||||
--seenSize;
|
||||
}
|
||||
}
|
||||
quorum_ = calculateQuorum(unlSize, effectiveUnlSize, seenSize);
|
||||
|
||||
JLOG(j_.debug()) << "Using quorum of " << quorum_ << " for new set of "
|
||||
<< trustedMasterKeys_.size() << " trusted validators ("
|
||||
@@ -939,4 +979,57 @@ ValidatorList::updateTrusted(hash_set<NodeID> const& seenValidators)
|
||||
return trustChanges;
|
||||
}
|
||||
|
||||
hash_set<PublicKey>
|
||||
ValidatorList::getTrustedMasterKeys() const
|
||||
{
|
||||
std::shared_lock lock{mutex_};
|
||||
return trustedMasterKeys_;
|
||||
}
|
||||
|
||||
hash_set<PublicKey>
|
||||
ValidatorList::getNegativeUnl() const
|
||||
{
|
||||
std::shared_lock lock{mutex_};
|
||||
return negativeUnl_;
|
||||
}
|
||||
|
||||
void
|
||||
ValidatorList::setNegativeUnl(hash_set<PublicKey> const& negUnl)
|
||||
{
|
||||
std::lock_guard lock{mutex_};
|
||||
negativeUnl_ = negUnl;
|
||||
}
|
||||
|
||||
std::vector<std::shared_ptr<STValidation>>
|
||||
ValidatorList::negativeUNLFilter(
|
||||
std::vector<std::shared_ptr<STValidation>>&& validations) const
|
||||
{
|
||||
// Remove validations that are from validators on the negative UNL.
|
||||
auto ret = std::move(validations);
|
||||
|
||||
std::shared_lock lock{mutex_};
|
||||
if (!negativeUnl_.empty())
|
||||
{
|
||||
ret.erase(
|
||||
std::remove_if(
|
||||
ret.begin(),
|
||||
ret.end(),
|
||||
[&](auto const& v) -> bool {
|
||||
if (auto const masterKey =
|
||||
getTrustedKey(v->getSignerPublic());
|
||||
masterKey)
|
||||
{
|
||||
return negativeUnl_.count(*masterKey);
|
||||
}
|
||||
else
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}),
|
||||
ret.end());
|
||||
}
|
||||
|
||||
return ret;
|
||||
}
|
||||
|
||||
} // namespace ripple
|
||||
|
||||
@@ -17,11 +17,13 @@
|
||||
*/
|
||||
//==============================================================================
|
||||
|
||||
#include <ripple/app/ledger/Ledger.h>
|
||||
#include <ripple/app/main/Application.h>
|
||||
#include <ripple/app/misc/AmendmentTable.h>
|
||||
#include <ripple/app/misc/NetworkOPs.h>
|
||||
#include <ripple/app/tx/impl/Change.h>
|
||||
#include <ripple/basics/Log.h>
|
||||
#include <ripple/protocol/Feature.h>
|
||||
#include <ripple/protocol/Indexes.h>
|
||||
#include <ripple/protocol/TxFlags.h>
|
||||
|
||||
@@ -62,6 +64,13 @@ Change::preflight(PreflightContext const& ctx)
|
||||
return temBAD_SEQUENCE;
|
||||
}
|
||||
|
||||
if (ctx.tx.getTxnType() == ttUNL_MODIFY &&
|
||||
!ctx.rules.enabled(featureNegativeUNL))
|
||||
{
|
||||
JLOG(ctx.j.warn()) << "Change: NegativeUNL not enabled";
|
||||
return temDISABLED;
|
||||
}
|
||||
|
||||
return tesSUCCESS;
|
||||
}
|
||||
|
||||
@@ -76,20 +85,32 @@ Change::preclaim(PreclaimContext const& ctx)
|
||||
return temINVALID;
|
||||
}
|
||||
|
||||
if (ctx.tx.getTxnType() != ttAMENDMENT && ctx.tx.getTxnType() != ttFEE)
|
||||
return temUNKNOWN;
|
||||
|
||||
return tesSUCCESS;
|
||||
switch (ctx.tx.getTxnType())
|
||||
{
|
||||
case ttAMENDMENT:
|
||||
case ttFEE:
|
||||
case ttUNL_MODIFY:
|
||||
return tesSUCCESS;
|
||||
default:
|
||||
return temUNKNOWN;
|
||||
}
|
||||
}
|
||||
|
||||
TER
|
||||
Change::doApply()
|
||||
{
|
||||
if (ctx_.tx.getTxnType() == ttAMENDMENT)
|
||||
return applyAmendment();
|
||||
|
||||
assert(ctx_.tx.getTxnType() == ttFEE);
|
||||
return applyFee();
|
||||
switch (ctx_.tx.getTxnType())
|
||||
{
|
||||
case ttAMENDMENT:
|
||||
return applyAmendment();
|
||||
case ttFEE:
|
||||
return applyFee();
|
||||
case ttUNL_MODIFY:
|
||||
return applyUNLModify();
|
||||
default:
|
||||
assert(0);
|
||||
return tefFAILURE;
|
||||
}
|
||||
}
|
||||
|
||||
void
|
||||
@@ -221,4 +242,130 @@ Change::applyFee()
|
||||
return tesSUCCESS;
|
||||
}
|
||||
|
||||
TER
|
||||
Change::applyUNLModify()
|
||||
{
|
||||
if (!isFlagLedger(view().seq()))
|
||||
{
|
||||
JLOG(j_.warn()) << "N-UNL: applyUNLModify, not a flag ledger, seq="
|
||||
<< view().seq();
|
||||
return tefFAILURE;
|
||||
}
|
||||
|
||||
if (!ctx_.tx.isFieldPresent(sfUNLModifyDisabling) ||
|
||||
ctx_.tx.getFieldU8(sfUNLModifyDisabling) > 1 ||
|
||||
!ctx_.tx.isFieldPresent(sfLedgerSequence) ||
|
||||
!ctx_.tx.isFieldPresent(sfUNLModifyValidator))
|
||||
{
|
||||
JLOG(j_.warn()) << "N-UNL: applyUNLModify, wrong Tx format.";
|
||||
return tefFAILURE;
|
||||
}
|
||||
|
||||
bool const disabling = ctx_.tx.getFieldU8(sfUNLModifyDisabling);
|
||||
auto const seq = ctx_.tx.getFieldU32(sfLedgerSequence);
|
||||
if (seq != view().seq())
|
||||
{
|
||||
JLOG(j_.warn()) << "N-UNL: applyUNLModify, wrong ledger seq=" << seq;
|
||||
return tefFAILURE;
|
||||
}
|
||||
|
||||
Blob const validator = ctx_.tx.getFieldVL(sfUNLModifyValidator);
|
||||
if (!publicKeyType(makeSlice(validator)))
|
||||
{
|
||||
JLOG(j_.warn()) << "N-UNL: applyUNLModify, bad validator key";
|
||||
return tefFAILURE;
|
||||
}
|
||||
|
||||
JLOG(j_.info()) << "N-UNL: applyUNLModify, "
|
||||
<< (disabling ? "ToDisable" : "ToReEnable")
|
||||
<< " seq=" << seq
|
||||
<< " validator data:" << strHex(validator);
|
||||
|
||||
auto const k = keylet::negativeUNL();
|
||||
SLE::pointer negUnlObject = view().peek(k);
|
||||
if (!negUnlObject)
|
||||
{
|
||||
negUnlObject = std::make_shared<SLE>(k);
|
||||
view().insert(negUnlObject);
|
||||
}
|
||||
|
||||
bool const found = [&] {
|
||||
if (negUnlObject->isFieldPresent(sfNegativeUNL))
|
||||
{
|
||||
auto const& negUnl = negUnlObject->getFieldArray(sfNegativeUNL);
|
||||
for (auto const& v : negUnl)
|
||||
{
|
||||
if (v.isFieldPresent(sfPublicKey) &&
|
||||
v.getFieldVL(sfPublicKey) == validator)
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}();
|
||||
|
||||
if (disabling)
|
||||
{
|
||||
// cannot have more than one toDisable
|
||||
if (negUnlObject->isFieldPresent(sfNegativeUNLToDisable))
|
||||
{
|
||||
JLOG(j_.warn()) << "N-UNL: applyUNLModify, already has ToDisable";
|
||||
return tefFAILURE;
|
||||
}
|
||||
|
||||
// cannot be the same as toReEnable
|
||||
if (negUnlObject->isFieldPresent(sfNegativeUNLToReEnable))
|
||||
{
|
||||
if (negUnlObject->getFieldVL(sfNegativeUNLToReEnable) == validator)
|
||||
{
|
||||
JLOG(j_.warn())
|
||||
<< "N-UNL: applyUNLModify, ToDisable is same as ToReEnable";
|
||||
return tefFAILURE;
|
||||
}
|
||||
}
|
||||
|
||||
// cannot be in negative UNL already
|
||||
if (found)
|
||||
{
|
||||
JLOG(j_.warn())
|
||||
<< "N-UNL: applyUNLModify, ToDisable already in negative UNL";
|
||||
return tefFAILURE;
|
||||
}
|
||||
|
||||
negUnlObject->setFieldVL(sfNegativeUNLToDisable, validator);
|
||||
}
|
||||
else
|
||||
{
|
||||
// cannot have more than one toReEnable
|
||||
if (negUnlObject->isFieldPresent(sfNegativeUNLToReEnable))
|
||||
{
|
||||
JLOG(j_.warn()) << "N-UNL: applyUNLModify, already has ToReEnable";
|
||||
return tefFAILURE;
|
||||
}
|
||||
|
||||
// cannot be the same as toDisable
|
||||
if (negUnlObject->isFieldPresent(sfNegativeUNLToDisable))
|
||||
{
|
||||
if (negUnlObject->getFieldVL(sfNegativeUNLToDisable) == validator)
|
||||
{
|
||||
JLOG(j_.warn())
|
||||
<< "N-UNL: applyUNLModify, ToReEnable is same as ToDisable";
|
||||
return tefFAILURE;
|
||||
}
|
||||
}
|
||||
|
||||
// must be in negative UNL
|
||||
if (!found)
|
||||
{
|
||||
JLOG(j_.warn())
|
||||
<< "N-UNL: applyUNLModify, ToReEnable is not in negative UNL";
|
||||
return tefFAILURE;
|
||||
}
|
||||
|
||||
negUnlObject->setFieldVL(sfNegativeUNLToReEnable, validator);
|
||||
}
|
||||
|
||||
view().update(negUnlObject);
|
||||
return tesSUCCESS;
|
||||
}
|
||||
|
||||
} // namespace ripple
|
||||
|
||||
@@ -59,6 +59,9 @@ private:
|
||||
|
||||
TER
|
||||
applyFee();
|
||||
|
||||
TER
|
||||
applyUNLModify();
|
||||
};
|
||||
|
||||
} // namespace ripple
|
||||
|
||||
@@ -365,6 +365,7 @@ LedgerEntryTypesMatch::visitEntry(
|
||||
case ltPAYCHAN:
|
||||
case ltCHECK:
|
||||
case ltDEPOSIT_PREAUTH:
|
||||
case ltNEGATIVE_UNL:
|
||||
break;
|
||||
default:
|
||||
invalidTypeAdded_ = true;
|
||||
|
||||
@@ -86,6 +86,7 @@ invoke_preflight(PreflightContext const& ctx)
|
||||
return DeleteAccount ::preflight(ctx);
|
||||
case ttAMENDMENT:
|
||||
case ttFEE:
|
||||
case ttUNL_MODIFY:
|
||||
return Change ::preflight(ctx);
|
||||
default:
|
||||
assert(false);
|
||||
@@ -173,6 +174,7 @@ invoke_preclaim(PreclaimContext const& ctx)
|
||||
return invoke_preclaim<DeleteAccount>(ctx);
|
||||
case ttAMENDMENT:
|
||||
case ttFEE:
|
||||
case ttUNL_MODIFY:
|
||||
return invoke_preclaim<Change>(ctx);
|
||||
default:
|
||||
assert(false);
|
||||
@@ -227,6 +229,7 @@ invoke_calculateBaseFee(ReadView const& view, STTx const& tx)
|
||||
return DeleteAccount::calculateBaseFee(view, tx);
|
||||
case ttAMENDMENT:
|
||||
case ttFEE:
|
||||
case ttUNL_MODIFY:
|
||||
return Change::calculateBaseFee(view, tx);
|
||||
default:
|
||||
assert(false);
|
||||
@@ -294,6 +297,7 @@ invoke_calculateConsequences(STTx const& tx)
|
||||
return invoke_calculateConsequences<DeleteAccount>(tx);
|
||||
case ttAMENDMENT:
|
||||
case ttFEE:
|
||||
case ttUNL_MODIFY:
|
||||
[[fallthrough]];
|
||||
default:
|
||||
assert(false);
|
||||
@@ -390,7 +394,8 @@ invoke_apply(ApplyContext& ctx)
|
||||
return p();
|
||||
}
|
||||
case ttAMENDMENT:
|
||||
case ttFEE: {
|
||||
case ttFEE:
|
||||
case ttUNL_MODIFY: {
|
||||
Change p(ctx);
|
||||
return p();
|
||||
}
|
||||
|
||||
@@ -322,6 +322,9 @@ class Validations
|
||||
beast::uhash<>>
|
||||
bySequence_;
|
||||
|
||||
// Sequence of the earliest validation to keep from expire
|
||||
boost::optional<Seq> toKeep_;
|
||||
|
||||
// Represents the ancestry of validated ledgers
|
||||
LedgerTrie<Ledger> trie_;
|
||||
|
||||
@@ -686,15 +689,47 @@ public:
|
||||
return ValStatus::current;
|
||||
}
|
||||
|
||||
/**
|
||||
* Set the smallest sequence number of validations to keep from expire
|
||||
* @param s the sequence number
|
||||
*/
|
||||
void
|
||||
setSeqToKeep(Seq const& s)
|
||||
{
|
||||
std::lock_guard lock{mutex_};
|
||||
toKeep_ = s;
|
||||
}
|
||||
|
||||
/** Expire old validation sets
|
||||
|
||||
Remove validation sets that were accessed more than
|
||||
validationSET_EXPIRES ago.
|
||||
validationSET_EXPIRES ago and were not asked to keep.
|
||||
*/
|
||||
void
|
||||
expire()
|
||||
{
|
||||
std::lock_guard lock{mutex_};
|
||||
if (toKeep_)
|
||||
{
|
||||
for (auto i = byLedger_.begin(); i != byLedger_.end(); ++i)
|
||||
{
|
||||
auto const& validationMap = i->second;
|
||||
if (!validationMap.empty() &&
|
||||
validationMap.begin()->second.seq() >= toKeep_)
|
||||
{
|
||||
byLedger_.touch(i);
|
||||
}
|
||||
}
|
||||
|
||||
for (auto i = bySequence_.begin(); i != bySequence_.end(); ++i)
|
||||
{
|
||||
if (i->first >= toKeep_)
|
||||
{
|
||||
bySequence_.touch(i);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
beast::expire(byLedger_, parms_.validationSET_EXPIRES);
|
||||
beast::expire(bySequence_, parms_.validationSET_EXPIRES);
|
||||
}
|
||||
|
||||
@@ -7,7 +7,7 @@ option java_multiple_files = true;
|
||||
import "org/xrpl/rpc/v1/amount.proto";
|
||||
import "org/xrpl/rpc/v1/account.proto";
|
||||
|
||||
// These fields are used in many different messsage types. They can be present
|
||||
// These fields are used in many different message types. They can be present
|
||||
// in one or more transactions, as well as metadata of one or more transactions.
|
||||
// Each is defined as its own message type with a single field "value", to
|
||||
// ensure the field is the correct type everywhere it's used
|
||||
@@ -70,6 +70,11 @@ message HighQualityOut
|
||||
uint32 value = 1;
|
||||
}
|
||||
|
||||
message FirstLedgerSequence
|
||||
{
|
||||
uint32 value = 1;
|
||||
}
|
||||
|
||||
message LastLedgerSequence
|
||||
{
|
||||
uint32 value = 1;
|
||||
@@ -351,6 +356,15 @@ message TransactionSignature
|
||||
bytes value = 1;
|
||||
}
|
||||
|
||||
message NegativeUnlToDisable
|
||||
{
|
||||
bytes value = 1;
|
||||
}
|
||||
|
||||
message NegativeUnlToReEnable
|
||||
{
|
||||
bytes value = 1;
|
||||
}
|
||||
|
||||
// *** Messages wrapping a Currency value ***
|
||||
|
||||
@@ -474,3 +488,12 @@ message SignerEntry
|
||||
|
||||
SignerWeight signer_weight = 2;
|
||||
}
|
||||
|
||||
// Next field: 3
|
||||
message NegativeUnlEntry
|
||||
{
|
||||
PublicKey public_key = 1;
|
||||
|
||||
FirstLedgerSequence ledger_sequence = 2;
|
||||
}
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ option java_multiple_files = true;
|
||||
|
||||
import "org/xrpl/rpc/v1/common.proto";
|
||||
|
||||
// Next field: 13
|
||||
// Next field: 14
|
||||
message LedgerObject
|
||||
{
|
||||
oneof object
|
||||
@@ -23,10 +23,11 @@ message LedgerObject
|
||||
PayChannel pay_channel = 10;
|
||||
RippleState ripple_state = 11;
|
||||
SignerList signer_list = 12;
|
||||
NegativeUnl negative_unl = 13;
|
||||
}
|
||||
}
|
||||
|
||||
// Next field: 13
|
||||
// Next field: 14
|
||||
enum LedgerEntryType
|
||||
{
|
||||
LEDGER_ENTRY_TYPE_UNSPECIFIED = 0;
|
||||
@@ -42,6 +43,7 @@ enum LedgerEntryType
|
||||
LEDGER_ENTRY_TYPE_PAY_CHANNEL = 10;
|
||||
LEDGER_ENTRY_TYPE_RIPPLE_STATE = 11;
|
||||
LEDGER_ENTRY_TYPE_SIGNER_LIST = 12;
|
||||
LEDGER_ENTRY_TYPE_NEGATIVE_UNL = 13;
|
||||
}
|
||||
|
||||
// Next field: 15
|
||||
@@ -329,3 +331,13 @@ message SignerList
|
||||
|
||||
SignerQuorum signer_quorum = 7;
|
||||
}
|
||||
|
||||
// Next field: 4
|
||||
message NegativeUnl
|
||||
{
|
||||
repeated NegativeUnlEntry negative_unl_entries = 1;
|
||||
|
||||
NegativeUnlToDisable validator_to_disable = 2;
|
||||
|
||||
NegativeUnlToReEnable validator_to_re_enable = 3;
|
||||
}
|
||||
@@ -112,8 +112,8 @@ class FeatureCollections
|
||||
"fix1781", // XRPEndpointSteps should be included in the circular
|
||||
// payment check
|
||||
"HardenedValidations",
|
||||
"fixAmendmentMajorityCalc"}; // Fix Amendment majority calculation
|
||||
|
||||
"fixAmendmentMajorityCalc", // Fix Amendment majority calculation
|
||||
"NegativeUNL"};
|
||||
std::vector<uint256> features;
|
||||
boost::container::flat_map<uint256, std::size_t> featureToIndex;
|
||||
boost::container::flat_map<std::string, uint256> nameToFeature;
|
||||
@@ -368,6 +368,7 @@ extern uint256 const featureRequireFullyCanonicalSig;
|
||||
extern uint256 const fix1781;
|
||||
extern uint256 const featureHardenedValidations;
|
||||
extern uint256 const fixAmendmentMajorityCalc;
|
||||
extern uint256 const featureNegativeUNL;
|
||||
|
||||
} // namespace ripple
|
||||
|
||||
|
||||
@@ -85,6 +85,10 @@ skip(LedgerIndex ledger) noexcept;
|
||||
Keylet const&
|
||||
fees() noexcept;
|
||||
|
||||
/** The (fixed) index of the object containing the ledger negativeUnl. */
|
||||
Keylet const&
|
||||
negativeUNL() noexcept;
|
||||
|
||||
/** The beginning of an order book */
|
||||
struct book_t
|
||||
{
|
||||
|
||||
@@ -86,6 +86,8 @@ enum LedgerEntryType {
|
||||
|
||||
ltDEPOSIT_PREAUTH = 'p',
|
||||
|
||||
ltNEGATIVE_UNL = 'N',
|
||||
|
||||
// No longer used or supported. Left here to prevent accidental
|
||||
// reassignment of the ledger type.
|
||||
ltNICKNAME [[deprecated]] = 'n',
|
||||
|
||||
@@ -340,6 +340,7 @@ extern SF_U8 const sfCloseResolution;
|
||||
extern SF_U8 const sfMethod;
|
||||
extern SF_U8 const sfTransactionResult;
|
||||
extern SF_U8 const sfTickSize;
|
||||
extern SF_U8 const sfUNLModifyDisabling;
|
||||
|
||||
// 16-bit integers
|
||||
extern SF_U16 const sfLedgerEntryType;
|
||||
@@ -375,7 +376,7 @@ extern SF_U32 const sfStampEscrow;
|
||||
extern SF_U32 const sfBondAmount;
|
||||
extern SF_U32 const sfLoadFee;
|
||||
extern SF_U32 const sfOfferSequence;
|
||||
extern SF_U32 const sfFirstLedgerSequence; // Deprecated: do not use
|
||||
extern SF_U32 const sfFirstLedgerSequence;
|
||||
extern SF_U32 const sfLastLedgerSequence;
|
||||
extern SF_U32 const sfTransactionIndex;
|
||||
extern SF_U32 const sfOperationLimit;
|
||||
@@ -471,6 +472,9 @@ extern SF_Blob const sfMemoFormat;
|
||||
extern SF_Blob const sfFulfillment;
|
||||
extern SF_Blob const sfCondition;
|
||||
extern SF_Blob const sfMasterSignature;
|
||||
extern SF_Blob const sfUNLModifyValidator;
|
||||
extern SF_Blob const sfNegativeUNLToDisable;
|
||||
extern SF_Blob const sfNegativeUNLToReEnable;
|
||||
|
||||
// account
|
||||
extern SF_Account const sfAccount;
|
||||
@@ -504,6 +508,7 @@ extern SField const sfMemo;
|
||||
extern SField const sfSignerEntry;
|
||||
extern SField const sfSigner;
|
||||
extern SField const sfMajority;
|
||||
extern SField const sfNegativeUNLEntry;
|
||||
|
||||
// array of objects
|
||||
// ARRAY/1 is reserved for end of array
|
||||
@@ -516,7 +521,7 @@ extern SField const sfSufficient;
|
||||
extern SField const sfAffectedNodes;
|
||||
extern SField const sfMemos;
|
||||
extern SField const sfMajorities;
|
||||
|
||||
extern SField const sfNegativeUNL;
|
||||
//------------------------------------------------------------------------------
|
||||
|
||||
} // namespace ripple
|
||||
|
||||
@@ -58,6 +58,7 @@ enum TxType {
|
||||
|
||||
ttAMENDMENT = 100,
|
||||
ttFEE = 101,
|
||||
ttUNL_MODIFY = 102,
|
||||
};
|
||||
|
||||
/** Manages the list of known transaction formats.
|
||||
|
||||
@@ -131,7 +131,9 @@ detail::supportedAmendments()
|
||||
"RequireFullyCanonicalSig",
|
||||
"fix1781",
|
||||
"HardenedValidations",
|
||||
"fixAmendmentMajorityCalc"};
|
||||
"fixAmendmentMajorityCalc",
|
||||
//"NegativeUNL" // Commented out to prevent automatic enablement
|
||||
};
|
||||
return supported;
|
||||
}
|
||||
|
||||
@@ -182,7 +184,8 @@ uint256 const
|
||||
featureRequireFullyCanonicalSig = *getRegisteredFeature("RequireFullyCanonicalSig"),
|
||||
fix1781 = *getRegisteredFeature("fix1781"),
|
||||
featureHardenedValidations = *getRegisteredFeature("HardenedValidations"),
|
||||
fixAmendmentMajorityCalc = *getRegisteredFeature("fixAmendmentMajorityCalc");
|
||||
fixAmendmentMajorityCalc = *getRegisteredFeature("fixAmendmentMajorityCalc"),
|
||||
featureNegativeUNL = *getRegisteredFeature("NegativeUNL");
|
||||
|
||||
// The following amendments have been active for at least two years. Their
|
||||
// pre-amendment code has been removed and the identifiers are deprecated.
|
||||
|
||||
@@ -58,6 +58,7 @@ enum class LedgerNameSpace : std::uint16_t {
|
||||
XRP_PAYMENT_CHANNEL = 'x',
|
||||
CHECK = 'C',
|
||||
DEPOSIT_PREAUTH = 'p',
|
||||
NEGATIVE_UNL = 'N',
|
||||
|
||||
// No longer used or supported. Left here to reserve the space
|
||||
// to avoid accidental reuse.
|
||||
@@ -162,6 +163,14 @@ fees() noexcept
|
||||
return ret;
|
||||
}
|
||||
|
||||
Keylet const&
|
||||
negativeUNL() noexcept
|
||||
{
|
||||
static Keylet const ret{
|
||||
ltNEGATIVE_UNL, indexHash(LedgerNameSpace::NEGATIVE_UNL)};
|
||||
return ret;
|
||||
}
|
||||
|
||||
Keylet
|
||||
book_t::operator()(Book const& b) const
|
||||
{
|
||||
|
||||
@@ -224,6 +224,15 @@ LedgerFormats::LedgerFormats()
|
||||
{sfPreviousTxnLgrSeq, soeREQUIRED},
|
||||
},
|
||||
commonFields);
|
||||
|
||||
add(jss::NegativeUNL,
|
||||
ltNEGATIVE_UNL,
|
||||
{
|
||||
{sfNegativeUNL, soeOPTIONAL},
|
||||
{sfNegativeUNLToDisable, soeOPTIONAL},
|
||||
{sfNegativeUNLToReEnable, soeOPTIONAL},
|
||||
},
|
||||
commonFields);
|
||||
}
|
||||
|
||||
LedgerFormats const&
|
||||
|
||||
@@ -56,6 +56,7 @@ SF_U8 const sfTransactionResult(access, STI_UINT8, 3, "TransactionResult");
|
||||
|
||||
// 8-bit integers (uncommon)
|
||||
SF_U8 const sfTickSize(access, STI_UINT8, 16, "TickSize");
|
||||
SF_U8 const sfUNLModifyDisabling(access, STI_UINT8, 17, "UNLModifyDisabling");
|
||||
|
||||
// 16-bit integers
|
||||
SF_U16 const sfLedgerEntryType(
|
||||
@@ -101,11 +102,8 @@ SF_U32 const sfStampEscrow(access, STI_UINT32, 22, "StampEscrow");
|
||||
SF_U32 const sfBondAmount(access, STI_UINT32, 23, "BondAmount");
|
||||
SF_U32 const sfLoadFee(access, STI_UINT32, 24, "LoadFee");
|
||||
SF_U32 const sfOfferSequence(access, STI_UINT32, 25, "OfferSequence");
|
||||
SF_U32 const sfFirstLedgerSequence(
|
||||
access,
|
||||
STI_UINT32,
|
||||
26,
|
||||
"FirstLedgerSequence"); // Deprecated: do not use
|
||||
SF_U32 const
|
||||
sfFirstLedgerSequence(access, STI_UINT32, 26, "FirstLedgerSequence");
|
||||
SF_U32 const sfLastLedgerSequence(access, STI_UINT32, 27, "LastLedgerSequence");
|
||||
SF_U32 const sfTransactionIndex(access, STI_UINT32, 28, "TransactionIndex");
|
||||
SF_U32 const sfOperationLimit(access, STI_UINT32, 29, "OperationLimit");
|
||||
@@ -225,6 +223,10 @@ SF_Blob const sfMasterSignature(
|
||||
"MasterSignature",
|
||||
SField::sMD_Default,
|
||||
SField::notSigning);
|
||||
SF_Blob const sfUNLModifyValidator(access, STI_VL, 19, "UNLModifyValidator");
|
||||
SF_Blob const sfNegativeUNLToDisable(access, STI_VL, 20, "ValidatorToDisable");
|
||||
SF_Blob const
|
||||
sfNegativeUNLToReEnable(access, STI_VL, 21, "ValidatorToReEnable");
|
||||
|
||||
// account
|
||||
SF_Account const sfAccount(access, STI_ACCOUNT, 1, "Account");
|
||||
@@ -263,6 +265,7 @@ SField const sfSignerEntry(access, STI_OBJECT, 11, "SignerEntry");
|
||||
SField const sfSigner(access, STI_OBJECT, 16, "Signer");
|
||||
// 17 has not been used yet...
|
||||
SField const sfMajority(access, STI_OBJECT, 18, "Majority");
|
||||
SField const sfNegativeUNLEntry(access, STI_OBJECT, 19, "DisabledValidator");
|
||||
|
||||
// array of objects
|
||||
// ARRAY/1 is reserved for end of array
|
||||
@@ -284,6 +287,7 @@ SField const sfMemos(access, STI_ARRAY, 9, "Memos");
|
||||
|
||||
// array of objects (uncommon)
|
||||
SField const sfMajorities(access, STI_ARRAY, 16, "Majorities");
|
||||
SField const sfNegativeUNL(access, STI_ARRAY, 17, "NegativeUNL");
|
||||
|
||||
SField::SField(
|
||||
private_access_tag_t,
|
||||
|
||||
@@ -527,7 +527,7 @@ isPseudoTx(STObject const& tx)
|
||||
if (!t)
|
||||
return false;
|
||||
auto tt = safe_cast<TxType>(*t);
|
||||
return tt == ttAMENDMENT || tt == ttFEE;
|
||||
return tt == ttAMENDMENT || tt == ttFEE || tt == ttUNL_MODIFY;
|
||||
}
|
||||
|
||||
} // namespace ripple
|
||||
|
||||
@@ -152,6 +152,15 @@ TxFormats::TxFormats()
|
||||
},
|
||||
commonFields);
|
||||
|
||||
add(jss::UNLModify,
|
||||
ttUNL_MODIFY,
|
||||
{
|
||||
{sfUNLModifyDisabling, soeREQUIRED},
|
||||
{sfLedgerSequence, soeREQUIRED},
|
||||
{sfUNLModifyValidator, soeREQUIRED},
|
||||
},
|
||||
commonFields);
|
||||
|
||||
add(jss::TicketCreate,
|
||||
ttTICKET_CREATE,
|
||||
{
|
||||
|
||||
@@ -82,6 +82,7 @@ JSS(PaymentChannelFund); // transaction type.
|
||||
JSS(RippleState); // ledger type.
|
||||
JSS(SLE_hit_rate); // out: GetCounts.
|
||||
JSS(SetFee); // transaction type.
|
||||
JSS(UNLModify); // transaction type.
|
||||
JSS(SettleDelay); // in: TransactionSign
|
||||
JSS(SendMax); // in: TransactionSign
|
||||
JSS(Sequence); // in/out: TransactionSign; field.
|
||||
@@ -576,8 +577,8 @@ JSS(vote); // in: Feature
|
||||
JSS(warning); // rpc:
|
||||
JSS(warnings); // out: server_info, server_state
|
||||
JSS(workers);
|
||||
JSS(write_load); // out: GetCounts
|
||||
|
||||
JSS(write_load); // out: GetCounts
|
||||
JSS(NegativeUNL); // out: ValidatorList; ledger type
|
||||
#undef JSS
|
||||
|
||||
} // namespace jss
|
||||
|
||||
@@ -485,6 +485,36 @@ populateFlags(T& to, STObject const& from)
|
||||
[&to]() { return to.mutable_flags(); }, from, sfFlags);
|
||||
}
|
||||
|
||||
template <class T>
|
||||
void
|
||||
populateFirstLedgerSequence(T& to, STObject const& from)
|
||||
{
|
||||
populateProtoPrimitive(
|
||||
[&to]() { return to.mutable_ledger_sequence(); },
|
||||
from,
|
||||
sfFirstLedgerSequence);
|
||||
}
|
||||
|
||||
template <class T>
|
||||
void
|
||||
populateNegativeUNLToDisable(T& to, STObject const& from)
|
||||
{
|
||||
populateProtoPrimitive(
|
||||
[&to]() { return to.mutable_validator_to_disable(); },
|
||||
from,
|
||||
sfNegativeUNLToDisable);
|
||||
}
|
||||
|
||||
template <class T>
|
||||
void
|
||||
populateNegativeUNLToReEnable(T& to, STObject const& from)
|
||||
{
|
||||
populateProtoPrimitive(
|
||||
[&to]() { return to.mutable_validator_to_re_enable(); },
|
||||
from,
|
||||
sfNegativeUNLToReEnable);
|
||||
}
|
||||
|
||||
template <class T>
|
||||
void
|
||||
populateLastLedgerSequence(T& to, STObject const& from)
|
||||
@@ -846,6 +876,21 @@ populateSignerEntries(T& to, STObject const& from)
|
||||
sfSignerEntry);
|
||||
}
|
||||
|
||||
template <class T>
|
||||
void
|
||||
populateNegativeUNLEntries(T& to, STObject const& from)
|
||||
{
|
||||
populateProtoArray(
|
||||
[&to]() { return to.add_negative_unl_entries(); },
|
||||
[](auto& innerObj, auto& innerProto) {
|
||||
populatePublicKey(innerProto, innerObj);
|
||||
populateFirstLedgerSequence(innerProto, innerObj);
|
||||
},
|
||||
from,
|
||||
sfNegativeUNL,
|
||||
sfNegativeUNLEntry);
|
||||
}
|
||||
|
||||
template <class T>
|
||||
void
|
||||
populateMemos(T& to, STObject const& from)
|
||||
@@ -1417,6 +1462,16 @@ convert(org::xrpl::rpc::v1::SignerList& to, STObject const& from)
|
||||
populateSignerListID(to, from);
|
||||
}
|
||||
|
||||
void
|
||||
convert(org::xrpl::rpc::v1::NegativeUnl& to, STObject const& from)
|
||||
{
|
||||
populateNegativeUNLEntries(to, from);
|
||||
|
||||
populateNegativeUNLToDisable(to, from);
|
||||
|
||||
populateNegativeUNLToReEnable(to, from);
|
||||
}
|
||||
|
||||
void
|
||||
setLedgerEntryType(
|
||||
org::xrpl::rpc::v1::AffectedNode& proto,
|
||||
@@ -1472,6 +1527,10 @@ setLedgerEntryType(
|
||||
proto.set_ledger_entry_type(
|
||||
org::xrpl::rpc::v1::LEDGER_ENTRY_TYPE_DEPOSIT_PREAUTH);
|
||||
break;
|
||||
case ltNEGATIVE_UNL:
|
||||
proto.set_ledger_entry_type(
|
||||
org::xrpl::rpc::v1::LEDGER_ENTRY_TYPE_NEGATIVE_UNL);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1517,6 +1576,9 @@ convert(T& to, STObject& from, std::uint16_t type)
|
||||
case ltDEPOSIT_PREAUTH:
|
||||
RPC::convert(*to.mutable_deposit_preauth(), from);
|
||||
break;
|
||||
case ltNEGATIVE_UNL:
|
||||
RPC::convert(*to.mutable_negative_unl(), from);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -58,6 +58,9 @@ convert(org::xrpl::rpc::v1::AccountRoot& to, STObject const& from);
|
||||
void
|
||||
convert(org::xrpl::rpc::v1::SignerList& to, STObject const& from);
|
||||
|
||||
void
|
||||
convert(org::xrpl::rpc::v1::NegativeUnl& to, STObject const& from);
|
||||
|
||||
template <class T>
|
||||
void
|
||||
convert(T& to, STAmount const& from)
|
||||
|
||||
@@ -1267,6 +1267,185 @@ private:
|
||||
}
|
||||
}
|
||||
|
||||
void
|
||||
testNegativeUNL()
|
||||
{
|
||||
testcase("NegativeUNL");
|
||||
jtx::Env env(*this);
|
||||
PublicKey emptyLocalKey;
|
||||
ManifestCache manifests;
|
||||
|
||||
auto createValidatorList =
|
||||
[&](std::uint32_t vlSize,
|
||||
boost::optional<std::size_t> minimumQuorum = {})
|
||||
-> std::shared_ptr<ValidatorList> {
|
||||
auto trustedKeys = std::make_shared<ValidatorList>(
|
||||
manifests,
|
||||
manifests,
|
||||
env.timeKeeper(),
|
||||
env.app().config().legacy("database_path"),
|
||||
env.journal,
|
||||
minimumQuorum);
|
||||
|
||||
std::vector<std::string> cfgPublishers;
|
||||
std::vector<std::string> cfgKeys;
|
||||
hash_set<NodeID> activeValidators;
|
||||
cfgKeys.reserve(vlSize);
|
||||
while (cfgKeys.size() < cfgKeys.capacity())
|
||||
{
|
||||
auto const valKey = randomNode();
|
||||
cfgKeys.push_back(toBase58(TokenType::NodePublic, valKey));
|
||||
activeValidators.emplace(calcNodeID(valKey));
|
||||
}
|
||||
if (trustedKeys->load(emptyLocalKey, cfgKeys, cfgPublishers))
|
||||
{
|
||||
trustedKeys->updateTrusted(activeValidators);
|
||||
if (trustedKeys->quorum() == std::ceil(cfgKeys.size() * 0.8f))
|
||||
return trustedKeys;
|
||||
}
|
||||
return nullptr;
|
||||
};
|
||||
|
||||
/*
|
||||
* Test NegativeUNL
|
||||
* == Combinations ==
|
||||
* -- UNL size: 34, 35, 57
|
||||
* -- nUNL size: 0%, 20%, 30%, 50%
|
||||
*
|
||||
* == with UNL size 60
|
||||
* -- set == get,
|
||||
* -- check quorum, with nUNL size: 0, 12, 30, 18
|
||||
* -- nUNL overlap: |nUNL - UNL| = 5, with nUNL size: 18
|
||||
* -- with command line minimumQuorum = 50%,
|
||||
* seen_reliable affected by nUNL
|
||||
*/
|
||||
|
||||
{
|
||||
hash_set<NodeID> activeValidators;
|
||||
//== Combinations ==
|
||||
std::array<std::uint32_t, 4> unlSizes = {34, 35, 39, 60};
|
||||
std::array<std::uint32_t, 4> nUnlPercent = {0, 20, 30, 50};
|
||||
for (auto us : unlSizes)
|
||||
{
|
||||
for (auto np : nUnlPercent)
|
||||
{
|
||||
auto validators = createValidatorList(us);
|
||||
BEAST_EXPECT(validators);
|
||||
if (validators)
|
||||
{
|
||||
std::uint32_t nUnlSize = us * np / 100;
|
||||
auto unl = validators->getTrustedMasterKeys();
|
||||
hash_set<PublicKey> nUnl;
|
||||
auto it = unl.begin();
|
||||
for (std::uint32_t i = 0; i < nUnlSize; ++i)
|
||||
{
|
||||
nUnl.insert(*it);
|
||||
++it;
|
||||
}
|
||||
validators->setNegativeUnl(nUnl);
|
||||
validators->updateTrusted(activeValidators);
|
||||
BEAST_EXPECT(
|
||||
validators->quorum() ==
|
||||
static_cast<std::size_t>(std::ceil(
|
||||
std::max((us - nUnlSize) * 0.8f, us * 0.6f))));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
//== with UNL size 60
|
||||
auto validators = createValidatorList(60);
|
||||
BEAST_EXPECT(validators);
|
||||
if (validators)
|
||||
{
|
||||
hash_set<NodeID> activeValidators;
|
||||
auto unl = validators->getTrustedMasterKeys();
|
||||
BEAST_EXPECT(unl.size() == 60);
|
||||
{
|
||||
//-- set == get,
|
||||
//-- check quorum, with nUNL size: 0, 30, 18, 12
|
||||
auto nUnlChange = [&](std::uint32_t nUnlSize,
|
||||
std::uint32_t quorum) -> bool {
|
||||
hash_set<PublicKey> nUnl;
|
||||
auto it = unl.begin();
|
||||
for (std::uint32_t i = 0; i < nUnlSize; ++i)
|
||||
{
|
||||
nUnl.insert(*it);
|
||||
++it;
|
||||
}
|
||||
validators->setNegativeUnl(nUnl);
|
||||
auto nUnl_temp = validators->getNegativeUnl();
|
||||
if (nUnl_temp.size() == nUnl.size())
|
||||
{
|
||||
for (auto& n : nUnl_temp)
|
||||
{
|
||||
if (nUnl.find(n) == nUnl.end())
|
||||
return false;
|
||||
}
|
||||
validators->updateTrusted(activeValidators);
|
||||
return validators->quorum() == quorum;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
BEAST_EXPECT(nUnlChange(0, 48));
|
||||
BEAST_EXPECT(nUnlChange(30, 36));
|
||||
BEAST_EXPECT(nUnlChange(18, 36));
|
||||
BEAST_EXPECT(nUnlChange(12, 39));
|
||||
}
|
||||
|
||||
{
|
||||
// nUNL overlap: |nUNL - UNL| = 5, with nUNL size: 18
|
||||
auto nUnl = validators->getNegativeUnl();
|
||||
BEAST_EXPECT(nUnl.size() == 12);
|
||||
std::size_t ss = 33;
|
||||
std::vector<uint8_t> data(ss, 0);
|
||||
data[0] = 0xED;
|
||||
for (int i = 0; i < 6; ++i)
|
||||
{
|
||||
Slice s(data.data(), ss);
|
||||
data[1]++;
|
||||
nUnl.emplace(s);
|
||||
}
|
||||
validators->setNegativeUnl(nUnl);
|
||||
validators->updateTrusted(activeValidators);
|
||||
BEAST_EXPECT(validators->quorum() == 39);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
//== with UNL size 60
|
||||
//-- with command line minimumQuorum = 50%,
|
||||
// seen_reliable affected by nUNL
|
||||
auto validators = createValidatorList(60, 30);
|
||||
BEAST_EXPECT(validators);
|
||||
if (validators)
|
||||
{
|
||||
hash_set<NodeID> activeValidators;
|
||||
hash_set<PublicKey> unl = validators->getTrustedMasterKeys();
|
||||
auto it = unl.begin();
|
||||
for (std::uint32_t i = 0; i < 50; ++i)
|
||||
{
|
||||
activeValidators.insert(calcNodeID(*it));
|
||||
++it;
|
||||
}
|
||||
validators->updateTrusted(activeValidators);
|
||||
BEAST_EXPECT(validators->quorum() == 48);
|
||||
hash_set<PublicKey> nUnl;
|
||||
it = unl.begin();
|
||||
for (std::uint32_t i = 0; i < 20; ++i)
|
||||
{
|
||||
nUnl.insert(*it);
|
||||
++it;
|
||||
}
|
||||
validators->setNegativeUnl(nUnl);
|
||||
validators->updateTrusted(activeValidators);
|
||||
BEAST_EXPECT(validators->quorum() == 30);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public:
|
||||
void
|
||||
run() override
|
||||
@@ -1276,6 +1455,7 @@ public:
|
||||
testApplyList();
|
||||
testUpdateTrusted();
|
||||
testExpires();
|
||||
testNegativeUNL();
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
2108
src/test/consensus/NegativeUNL_test.cpp
Normal file
2108
src/test/consensus/NegativeUNL_test.cpp
Normal file
File diff suppressed because it is too large
Load Diff
@@ -707,10 +707,18 @@ class Validations_test : public beast::unit_test::suite
|
||||
Node a = harness.makeNode();
|
||||
|
||||
Ledger ledgerA = h["a"];
|
||||
|
||||
BEAST_EXPECT(ValStatus::current == harness.add(a.validate(ledgerA)));
|
||||
BEAST_EXPECT(harness.vals().numTrustedForLedger(ledgerA.id()));
|
||||
|
||||
// Keep the validation from expire
|
||||
harness.clock().advance(harness.parms().validationSET_EXPIRES);
|
||||
harness.vals().setSeqToKeep(ledgerA.seq());
|
||||
harness.vals().expire();
|
||||
BEAST_EXPECT(harness.vals().numTrustedForLedger(ledgerA.id()));
|
||||
|
||||
// Allow the validation to expire
|
||||
harness.clock().advance(harness.parms().validationSET_EXPIRES);
|
||||
harness.vals().setSeqToKeep(++ledgerA.seq());
|
||||
harness.vals().expire();
|
||||
BEAST_EXPECT(!harness.vals().numTrustedForLedger(ledgerA.id()));
|
||||
}
|
||||
|
||||
@@ -134,6 +134,39 @@ public:
|
||||
auto const jrr = env.rpc("validator_list_sites")[jss::result];
|
||||
BEAST_EXPECT(jrr[jss::validator_sites].size() == 0);
|
||||
}
|
||||
// Negative UNL empty
|
||||
{
|
||||
auto const jrr = env.rpc("validators")[jss::result];
|
||||
BEAST_EXPECT(jrr[jss::NegativeUNL].isNull());
|
||||
}
|
||||
// Negative UNL update
|
||||
{
|
||||
hash_set<PublicKey> disabledKeys;
|
||||
auto k1 = randomKeyPair(KeyType::ed25519).first;
|
||||
auto k2 = randomKeyPair(KeyType::ed25519).first;
|
||||
disabledKeys.insert(k1);
|
||||
disabledKeys.insert(k2);
|
||||
env.app().validators().setNegativeUnl(disabledKeys);
|
||||
|
||||
auto const jrr = env.rpc("validators")[jss::result];
|
||||
auto& jrrnUnl = jrr[jss::NegativeUNL];
|
||||
auto jrrnUnlSize = jrrnUnl.size();
|
||||
BEAST_EXPECT(jrrnUnlSize == 2);
|
||||
for (std::uint32_t x = 0; x < jrrnUnlSize; ++x)
|
||||
{
|
||||
auto parsedKey = parseBase58<PublicKey>(
|
||||
TokenType::NodePublic, jrrnUnl[x].asString());
|
||||
BEAST_EXPECT(parsedKey);
|
||||
if (parsedKey)
|
||||
BEAST_EXPECT(
|
||||
disabledKeys.find(*parsedKey) != disabledKeys.end());
|
||||
}
|
||||
|
||||
disabledKeys.clear();
|
||||
env.app().validators().setNegativeUnl(disabledKeys);
|
||||
auto const jrrUpdated = env.rpc("validators")[jss::result];
|
||||
BEAST_EXPECT(jrrUpdated[jss::NegativeUNL].isNull());
|
||||
}
|
||||
}
|
||||
|
||||
void
|
||||
|
||||
Reference in New Issue
Block a user