ARCHIVE/rippled
1
0
Fork 0
mirror of https://github.com/XRPLF/rippled.git synced 2025-12-06 17:27:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

Refactor consensus for simulation (RIPD-1011):

Browse Source 
This is a substantial refactor of the consensus code and also introduces
a basic consensus simulation and testing framework.  The new generic/templated
version is in src/ripple/consensus and documents the current type requirements.
The version adapted for the RCL is in src/ripple/app/consensus.  The testing
framework is in src/test/csf.

Minor behavioral changes/fixes include:
* Adjust close time offset even when not validating.
* Remove spurious proposing_ = false call at end of handleLCL.
* Remove unused functionality provided by checkLastValidation.
* Separate open and converge time
* Don't send a bow out if we're not proposing
* Prevent consensus stopping if NetworkOPs switches to disconnect mode while
  consensus accepts a ledger
* Prevent a corner case in which Consensus::gotTxSet or Consensus::peerProposal
  has the potential to update internal state while an dispatched accept job is
  running.
* Distinguish external and internal calls to startNewRound.  Only external
  calls can reset the proposing_ state of consensus
This commit is contained in:
Brad Chase
2016-11-02 15:16:02 -07:00
parent fc0d64f5ee
commit bc5a74057d
56 changed files with 6492 additions and 3785 deletions
Download Patch File Download Diff File Expand all files Collapse all files

116
Builds/VisualStudio2015/RippleD.vcxproj
View File

@@ -819,9 +819,19 @@
</ClInclude>
<ClInclude Include="..\..\src\protobuf\vsprojects\config.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\consensus\RCLCxPos.h">
<ClCompile Include="..\..\src\ripple\app\consensus\RCLConsensus.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\consensus\RCLConsensus.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\consensus\RCLCxTraits.h">
<ClInclude Include="..\..\src\ripple\app\consensus\RCLCxLedger.h">
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\consensus\RCLCxPeerPos.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\consensus\RCLCxPeerPos.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\consensus\RCLCxTx.h">
</ClInclude>
@@ -851,22 +861,12 @@
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\ledger\BookListeners.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\ledger\Consensus.h">
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\ledger\ConsensusTransSetSF.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\ledger\ConsensusTransSetSF.h">
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\ledger\impl\ConsensusImp.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\ledger\impl\ConsensusImp.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\ledger\impl\DisputedTx.h">
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\ledger\impl\InboundLedger.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
@@ -883,20 +883,10 @@
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\ripple\app\ledger\impl\LedgerConsensusImp.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\ledger\impl\LedgerConsensusImp.h">
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\ledger\impl\LedgerMaster.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\ripple\app\ledger\impl\LedgerTiming.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\ripple\app\ledger\impl\LedgerToJson.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
@@ -933,8 +923,6 @@
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\ledger\LedgerCleaner.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\ledger\LedgerConsensus.h">
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\ledger\LedgerHistory.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
@@ -945,14 +933,6 @@
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\ledger\LedgerMaster.h">
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\ledger\LedgerProposal.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\ledger\LedgerProposal.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\ledger\LedgerTiming.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\ledger\LedgerToJson.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\ledger\LocalTxs.h">
@@ -1841,6 +1821,18 @@
</ClInclude>
<ClInclude Include="..\..\src\ripple\conditions\impl\utils.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\consensus\Consensus.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\consensus\ConsensusProposal.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\consensus\DisputedTx.h">
</ClInclude>
<ClCompile Include="..\..\src\ripple\consensus\LedgerTiming.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClInclude Include="..\..\src\ripple\consensus\LedgerTiming.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\core\Config.h">
</ClInclude>
<ClInclude Include="..\..\src\ripple\core\ConfigSections.h">
@@ -3143,6 +3135,10 @@
</ClInclude>
<ClInclude Include="..\..\src\ripple\shamap\TreeNodeCache.h">
</ClInclude>
<ClCompile Include="..\..\src\ripple\unity\app_consensus.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug.classic|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release.classic|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\ripple\unity\app_ledger.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug.classic|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release.classic|x64'">True</ExcludedFromBuild>
@@ -3173,6 +3169,10 @@
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug.classic|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release.classic|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\ripple\unity\consensus.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug.classic|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release.classic|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\ripple\unity\core.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug.classic|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release.classic|x64'">True</ExcludedFromBuild>
@@ -4397,6 +4397,14 @@
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\consensus\Consensus_test.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\consensus\LedgerTiming_test.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\core\Config_test.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
@@ -4425,6 +4433,28 @@
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClInclude Include="..\..\src\test\csf.h">
</ClInclude>
<ClInclude Include="..\..\src\test\csf\BasicNetwork.h">
</ClInclude>
<ClCompile Include="..\..\src\test\csf\BasicNetwork_test.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\csf\impl\UNL.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClInclude Include="..\..\src\test\csf\Ledger.h">
</ClInclude>
<ClInclude Include="..\..\src\test\csf\Peer.h">
</ClInclude>
<ClInclude Include="..\..\src\test\csf\Sim.h">
</ClInclude>
<ClInclude Include="..\..\src\test\csf\Tx.h">
</ClInclude>
<ClInclude Include="..\..\src\test\csf\UNL.h">
</ClInclude>
<ClCompile Include="..\..\src\test\json\json_value_test.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
@@ -4453,12 +4483,6 @@
</ClInclude>
<ClInclude Include="..\..\src\test\jtx\balance.h">
</ClInclude>
<ClInclude Include="..\..\src\test\jtx\BasicNetwork.h">
</ClInclude>
<ClCompile Include="..\..\src\test\jtx\BasicNetwork_test.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClInclude Include="..\..\src\test\jtx\basic_prop.h">
</ClInclude>
<ClInclude Include="..\..\src\test\jtx\delivermin.h">
@@ -4931,14 +4955,26 @@
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug.classic|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release.classic|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\unity\consensus_test_unity.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug.classic|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release.classic|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\unity\core_test_unity.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug.classic|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release.classic|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\unity\csf_unity.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug.classic|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release.classic|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\unity\json_test_unity.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug.classic|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release.classic|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\unity\jtx_unity.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug.classic|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release.classic|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\unity\ledger_test_unity.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug.classic|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release.classic|x64'">True</ExcludedFromBuild>
@@ -4977,10 +5013,6 @@
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug.classic|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release.classic|x64'">True</ExcludedFromBuild>
</ClCompile>
<ClCompile Include="..\..\src\test\unity\support_unity.cpp">
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='debug.classic|x64'">True</ExcludedFromBuild>
<ExcludedFromBuild Condition="'$(Configuration)|$(Platform)'=='release.classic|x64'">True</ExcludedFromBuild>
</ClCompile>
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">

130
Builds/VisualStudio2015/RippleD.vcxproj.filters
View File

@@ -214,6 +214,9 @@
<Filter Include="ripple\conditions\impl">
<UniqueIdentifier>{155DC1A3-8A60-BC74-A7E4-1AC1A679FFF9}</UniqueIdentifier>
</Filter>
<Filter Include="ripple\consensus">
<UniqueIdentifier>{75481992-909B-CAA9-859B-91F15AFD2AD0}</UniqueIdentifier>
</Filter>
<Filter Include="ripple\core">
<UniqueIdentifier>{235DCF23-2CF8-4F03-1A54-C159823A7E8D}</UniqueIdentifier>
</Filter>
@@ -448,9 +451,18 @@
<Filter Include="test\conditions">
<UniqueIdentifier>{F421E9A0-BB69-E638-F7AC-A3BD9B7D4827}</UniqueIdentifier>
</Filter>
<Filter Include="test\consensus">
<UniqueIdentifier>{330C919A-D367-9106-B0FF-BCE5B46A76B2}</UniqueIdentifier>
</Filter>
<Filter Include="test\core">
<UniqueIdentifier>{26D7F11B-5BF1-54BC-8BF5-D45F68A6A408}</UniqueIdentifier>
</Filter>
<Filter Include="test\csf">
<UniqueIdentifier>{8E0BEDEF-0473-BB6D-7AD3-877644873E62}</UniqueIdentifier>
</Filter>
<Filter Include="test\csf\impl">
<UniqueIdentifier>{2B6B8B80-5419-52D9-8CB9-758C3FB7FD0A}</UniqueIdentifier>
</Filter>
<Filter Include="test\json">
<UniqueIdentifier>{87249A3B-D8F5-1A8C-6C1D-F1CDCCF5242B}</UniqueIdentifier>
</Filter>
@@ -1323,10 +1335,19 @@
<ClInclude Include="..\..\src\protobuf\vsprojects\config.h">
<Filter>protobuf\vsprojects</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\consensus\RCLCxPos.h">
<ClCompile Include="..\..\src\ripple\app\consensus\RCLConsensus.cpp">
<Filter>ripple\app\consensus</Filter>
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\consensus\RCLConsensus.h">
<Filter>ripple\app\consensus</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\consensus\RCLCxTraits.h">
<ClInclude Include="..\..\src\ripple\app\consensus\RCLCxLedger.h">
<Filter>ripple\app\consensus</Filter>
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\consensus\RCLCxPeerPos.cpp">
<Filter>ripple\app\consensus</Filter>
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\consensus\RCLCxPeerPos.h">
<Filter>ripple\app\consensus</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\consensus\RCLCxTx.h">
@@ -1359,24 +1380,12 @@
<ClInclude Include="..\..\src\ripple\app\ledger\BookListeners.h">
<Filter>ripple\app\ledger</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\ledger\Consensus.h">
<Filter>ripple\app\ledger</Filter>
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\ledger\ConsensusTransSetSF.cpp">
<Filter>ripple\app\ledger</Filter>
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\ledger\ConsensusTransSetSF.h">
<Filter>ripple\app\ledger</Filter>
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\ledger\impl\ConsensusImp.cpp">
<Filter>ripple\app\ledger\impl</Filter>
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\ledger\impl\ConsensusImp.h">
<Filter>ripple\app\ledger\impl</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\ledger\impl\DisputedTx.h">
<Filter>ripple\app\ledger\impl</Filter>
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\ledger\impl\InboundLedger.cpp">
<Filter>ripple\app\ledger\impl</Filter>
</ClCompile>
@@ -1389,18 +1398,9 @@
<ClCompile Include="..\..\src\ripple\app\ledger\impl\LedgerCleaner.cpp">
<Filter>ripple\app\ledger\impl</Filter>
</ClCompile>
<ClCompile Include="..\..\src\ripple\app\ledger\impl\LedgerConsensusImp.cpp">
<Filter>ripple\app\ledger\impl</Filter>
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\ledger\impl\LedgerConsensusImp.h">
<Filter>ripple\app\ledger\impl</Filter>
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\ledger\impl\LedgerMaster.cpp">
<Filter>ripple\app\ledger\impl</Filter>
</ClCompile>
<ClCompile Include="..\..\src\ripple\app\ledger\impl\LedgerTiming.cpp">
<Filter>ripple\app\ledger\impl</Filter>
</ClCompile>
<ClCompile Include="..\..\src\ripple\app\ledger\impl\LedgerToJson.cpp">
<Filter>ripple\app\ledger\impl</Filter>
</ClCompile>
@@ -1437,9 +1437,6 @@
<ClInclude Include="..\..\src\ripple\app\ledger\LedgerCleaner.h">
<Filter>ripple\app\ledger</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\ledger\LedgerConsensus.h">
<Filter>ripple\app\ledger</Filter>
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\ledger\LedgerHistory.cpp">
<Filter>ripple\app\ledger</Filter>
</ClCompile>
@@ -1452,15 +1449,6 @@
<ClInclude Include="..\..\src\ripple\app\ledger\LedgerMaster.h">
<Filter>ripple\app\ledger</Filter>
</ClInclude>
<ClCompile Include="..\..\src\ripple\app\ledger\LedgerProposal.cpp">
<Filter>ripple\app\ledger</Filter>
</ClCompile>
<ClInclude Include="..\..\src\ripple\app\ledger\LedgerProposal.h">
<Filter>ripple\app\ledger</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\ledger\LedgerTiming.h">
<Filter>ripple\app\ledger</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\app\ledger\LedgerToJson.h">
<Filter>ripple\app\ledger</Filter>
</ClInclude>
@@ -2460,6 +2448,21 @@
<ClInclude Include="..\..\src\ripple\conditions\impl\utils.h">
<Filter>ripple\conditions\impl</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\consensus\Consensus.h">
<Filter>ripple\consensus</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\consensus\ConsensusProposal.h">
<Filter>ripple\consensus</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\consensus\DisputedTx.h">
<Filter>ripple\consensus</Filter>
</ClInclude>
<ClCompile Include="..\..\src\ripple\consensus\LedgerTiming.cpp">
<Filter>ripple\consensus</Filter>
</ClCompile>
<ClInclude Include="..\..\src\ripple\consensus\LedgerTiming.h">
<Filter>ripple\consensus</Filter>
</ClInclude>
<ClInclude Include="..\..\src\ripple\core\Config.h">
<Filter>ripple\core</Filter>
</ClInclude>
@@ -3738,6 +3741,9 @@
<ClInclude Include="..\..\src\ripple\shamap\TreeNodeCache.h">
<Filter>ripple\shamap</Filter>
</ClInclude>
<ClCompile Include="..\..\src\ripple\unity\app_consensus.cpp">
<Filter>ripple\unity</Filter>
</ClCompile>
<ClCompile Include="..\..\src\ripple\unity\app_ledger.cpp">
<Filter>ripple\unity</Filter>
</ClCompile>
@@ -3762,6 +3768,9 @@
<ClCompile Include="..\..\src\ripple\unity\conditions.cpp">
<Filter>ripple\unity</Filter>
</ClCompile>
<ClCompile Include="..\..\src\ripple\unity\consensus.cpp">
<Filter>ripple\unity</Filter>
</ClCompile>
<ClCompile Include="..\..\src\ripple\unity\core.cpp">
<Filter>ripple\unity</Filter>
</ClCompile>
@@ -5136,6 +5145,12 @@
<ClCompile Include="..\..\src\test\conditions\PreimageSha256_test.cpp">
<Filter>test\conditions</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\consensus\Consensus_test.cpp">
<Filter>test\consensus</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\consensus\LedgerTiming_test.cpp">
<Filter>test\consensus</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\core\Config_test.cpp">
<Filter>test\core</Filter>
</ClCompile>
@@ -5157,6 +5172,33 @@
<ClCompile Include="..\..\src\test\core\Workers_test.cpp">
<Filter>test\core</Filter>
</ClCompile>
<ClInclude Include="..\..\src\test\csf.h">
<Filter>test</Filter>
</ClInclude>
<ClInclude Include="..\..\src\test\csf\BasicNetwork.h">
<Filter>test\csf</Filter>
</ClInclude>
<ClCompile Include="..\..\src\test\csf\BasicNetwork_test.cpp">
<Filter>test\csf</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\csf\impl\UNL.cpp">
<Filter>test\csf\impl</Filter>
</ClCompile>
<ClInclude Include="..\..\src\test\csf\Ledger.h">
<Filter>test\csf</Filter>
</ClInclude>
<ClInclude Include="..\..\src\test\csf\Peer.h">
<Filter>test\csf</Filter>
</ClInclude>
<ClInclude Include="..\..\src\test\csf\Sim.h">
<Filter>test\csf</Filter>
</ClInclude>
<ClInclude Include="..\..\src\test\csf\Tx.h">
<Filter>test\csf</Filter>
</ClInclude>
<ClInclude Include="..\..\src\test\csf\UNL.h">
<Filter>test\csf</Filter>
</ClInclude>
<ClCompile Include="..\..\src\test\json\json_value_test.cpp">
<Filter>test\json</Filter>
</ClCompile>
@@ -5187,12 +5229,6 @@
<ClInclude Include="..\..\src\test\jtx\balance.h">
<Filter>test\jtx</Filter>
</ClInclude>
<ClInclude Include="..\..\src\test\jtx\BasicNetwork.h">
<Filter>test\jtx</Filter>
</ClInclude>
<ClCompile Include="..\..\src\test\jtx\BasicNetwork_test.cpp">
<Filter>test\jtx</Filter>
</ClCompile>
<ClInclude Include="..\..\src\test\jtx\basic_prop.h">
<Filter>test\jtx</Filter>
</ClInclude>
@@ -5598,12 +5634,21 @@
<ClCompile Include="..\..\src\test\unity\conditions_test_unity.cpp">
<Filter>test\unity</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\unity\consensus_test_unity.cpp">
<Filter>test\unity</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\unity\core_test_unity.cpp">
<Filter>test\unity</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\unity\csf_unity.cpp">
<Filter>test\unity</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\unity\json_test_unity.cpp">
<Filter>test\unity</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\unity\jtx_unity.cpp">
<Filter>test\unity</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\unity\ledger_test_unity.cpp">
<Filter>test\unity</Filter>
</ClCompile>
@@ -5631,8 +5676,5 @@
<ClCompile Include="..\..\src\test\unity\shamap_test_unity.cpp">
<Filter>test\unity</Filter>
</ClCompile>
<ClCompile Include="..\..\src\test\unity\support_unity.cpp">
<Filter>test\unity</Filter>
</ClCompile>
</ItemGroup>
</Project>

10
CMakeLists.txt
View File

@@ -179,12 +179,14 @@ beast_utility_unity.cpp)
prepend(ripple_unity_srcs
src/ripple/unity/
app_consensus.cpp
app_ledger.cpp
app_main.cpp
app_misc.cpp
app_paths.cpp
app_tx.cpp
conditions.cpp
consensus.cpp
core.cpp
basics.cpp
crypto.cpp
@@ -204,6 +206,7 @@ app_test_unity.cpp
basics_test_unity.cpp
beast_test_unity.cpp
conditions_test_unity.cpp
consensus_test_unity.cpp
core_test_unity.cpp
json_test_unity.cpp
ledger_test_unity.cpp
@@ -214,7 +217,8 @@ resource_test_unity.cpp
rpc_test_unity.cpp
server_test_unity.cpp
shamap_test_unity.cpp
support_unity.cpp)
jtx_unity.cpp
csf_unity.cpp)
list(APPEND rippled_src_unity ${beast_unity_srcs} ${ripple_unity_srcs} ${test_unity_srcs})
@@ -257,6 +261,7 @@ foreach(curdir
basics
conditions
crypto
consensus
json
ledger
legacy
@@ -300,7 +305,8 @@ foreach(curdir
rpc
server
shamap
jtx)
jtx
csf)
file(GLOB_RECURSE cursrcs src/test/${curdir}/*.cpp)
list(APPEND test_srcs "${cursrcs}")
endforeach()

9
SConstruct
View File

@@ -949,6 +949,7 @@ def get_classic_sources(toolchain):
append_sources(result, *list_sources('src/ripple/basics', '.cpp'))
append_sources(result, *list_sources('src/ripple/conditions', '.cpp'))
append_sources(result, *list_sources('src/ripple/crypto', '.cpp'))
append_sources(result, *list_sources('src/ripple/consensus', '.cpp'))
append_sources(result, *list_sources('src/ripple/json', '.cpp'))
append_sources(result, *list_sources('src/ripple/ledger', '.cpp'))
append_sources(result, *list_sources('src/ripple/legacy', '.cpp'))
@@ -963,6 +964,7 @@ def get_classic_sources(toolchain):
append_sources(result, *list_sources('src/test/basics', '.cpp'))
append_sources(result, *list_sources('src/test/beast', '.cpp'))
append_sources(result, *list_sources('src/test/conditions', '.cpp'))
append_sources(result, *list_sources('src/test/consensus', '.cpp'))
append_sources(result, *list_sources('src/test/core', '.cpp'))
append_sources(result, *list_sources('src/test/json', '.cpp'))
append_sources(result, *list_sources('src/test/ledger', '.cpp'))
@@ -974,6 +976,7 @@ def get_classic_sources(toolchain):
append_sources(result, *list_sources('src/test/server', '.cpp'))
append_sources(result, *list_sources('src/test/shamap', '.cpp'))
append_sources(result, *list_sources('src/test/jtx', '.cpp'))
append_sources(result, *list_sources('src/test/csf', '.cpp'))
if use_shp(toolchain):
@@ -1003,12 +1006,14 @@ def get_unity_sources(toolchain):
'src/ripple/beast/unity/beast_insight_unity.cpp',
'src/ripple/beast/unity/beast_net_unity.cpp',
'src/ripple/beast/unity/beast_utility_unity.cpp',
'src/ripple/unity/app_consensus.cpp',
'src/ripple/unity/app_ledger.cpp',
'src/ripple/unity/app_main.cpp',
'src/ripple/unity/app_misc.cpp',
'src/ripple/unity/app_paths.cpp',
'src/ripple/unity/app_tx.cpp',
'src/ripple/unity/conditions.cpp',
'src/ripple/unity/consensus.cpp',
'src/ripple/unity/core.cpp',
'src/ripple/unity/basics.cpp',
'src/ripple/unity/crypto.cpp',
@@ -1024,6 +1029,7 @@ def get_unity_sources(toolchain):
'src/test/unity/app_test_unity.cpp',
'src/test/unity/basics_test_unity.cpp',
'src/test/unity/beast_test_unity.cpp',
'src/test/unity/consensus_test_unity.cpp',
'src/test/unity/core_test_unity.cpp',
'src/test/unity/conditions_test_unity.cpp',
'src/test/unity/json_test_unity.cpp',
@@ -1035,7 +1041,8 @@ def get_unity_sources(toolchain):
'src/test/unity/rpc_test_unity.cpp',
'src/test/unity/server_test_unity.cpp',
'src/test/unity/shamap_test_unity.cpp',
'src/test/unity/support_unity.cpp'
'src/test/unity/jtx_unity.cpp',
'src/test/unity/csf_unity.cpp'
)
if use_shp(toolchain):

15
docs/source.dox
View File

@@ -106,10 +106,17 @@ INPUT = \
\
../src/ripple/protocol/STObject.h \
../src/ripple/protocol/JsonFields.h \
../src/test/support/AbstractClient.h \
../src/test/support/JSONRPCClient.h \
../src/test/support/WSClient.h \
../src/test/jtx/AbstractClient.h \
../src/test/jtx/JSONRPCClient.h \
../src/test/jtx/WSClient.h \
../src/ripple/consensus/Consensus.h \
../src/ripple/consensus/ConsensusProposal.h \
../src/ripple/consensus/DisputedTx.h \
../src/ripple/consensus/LedgerTiming.h \
../src/ripple/app/consensus/RCLCxTx.h \
../src/ripple/app/consensus/RCLCxLedger.h \
../src/ripple/app/consensus/RCLConsensus.h \
../src/ripple/app/consensus/RCLCxPeerPos.h \
INPUT_ENCODING = UTF-8
FILE_PATTERNS =

909
src/ripple/app/consensus/RCLConsensus.cpp Normal file
View File

@@ -0,0 +1,909 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012, 2013 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <BeastConfig.h>
#include <ripple/beast/core/LexicalCast.h>
#include <ripple/app/consensus/RCLConsensus.h>
#include <ripple/app/ledger/InboundTransactions.h>
#include <ripple/app/misc/NetworkOPs.h>
#include <ripple/app/ledger/LedgerMaster.h>
#include <ripple/app/ledger/InboundLedgers.h>
#include <ripple/overlay/Overlay.h>
#include <ripple/app/ledger/OpenLedger.h>
#include <ripple/protocol/digest.h>
#include <ripple/overlay/predicates.h>
#include <ripple/app/misc/AmendmentTable.h>
#include <ripple/app/misc/HashRouter.h>
#include <ripple/consensus/LedgerTiming.h>
#include <ripple/basics/make_lock.h>
#include <ripple/app/ledger/LocalTxs.h>
#include <ripple/app/misc/TxQ.h>
#include <ripple/app/tx/apply.h>
#include <ripple/protocol/Feature.h>
#include <ripple/app/misc/LoadFeeTrack.h>
#include <ripple/app/misc/ValidatorList.h>
namespace ripple {
RCLConsensus::RCLConsensus(
Application& app,
std::unique_ptr<FeeVote> && feeVote,
LedgerMaster& ledgerMaster,
LocalTxs& localTxs,
InboundTransactions& inboundTransactions,
typename Base::clock_type const & clock,
beast::Journal journal)
: Base(clock, journal)
, app_ (app)
, feeVote_ (std::move(feeVote))
, ledgerMaster_ (ledgerMaster)
, localTxs_(localTxs)
, inboundTransactions_{ inboundTransactions }
, j_ (journal)
, nodeID_{ calcNodeID(app.nodeIdentity().first) }
{
}
void
RCLConsensus::onStartRound(RCLCxLedger const & ledger)
{
inboundTransactions_.newRound(ledger.seq());
}
// First bool is whether or not we can propose
// Second bool is whether or not we can validate
std::pair <bool, bool>
RCLConsensus::getMode ()
{
bool propose = false;
bool validate = false;
if (! app_.getOPs().isNeedNetworkLedger() && (valPublic_.size() != 0))
{
// We have a key, and we have some idea what the ledger is
validate = true;
// propose only if we're in sync with the network
propose = app_.getOPs().getOperatingMode() == NetworkOPs::omFULL;
}
return { propose, validate };
}
boost::optional<RCLCxLedger>
RCLConsensus::acquireLedger(LedgerHash const & ledger)
{
// we need to switch the ledger we're working from
auto buildLCL = ledgerMaster_.getLedgerByHash(ledger);
if (! buildLCL)
{
if (acquiringLedger_ != ledger)
{
// need to start acquiring the correct consensus LCL
JLOG (j_.warn()) <<
"Need consensus ledger " << ledger;
// Tell the ledger acquire system that we need the consensus ledger
acquiringLedger_ = ledger;
auto app = &app_;
auto hash = acquiringLedger_;
app_.getJobQueue().addJob (
jtADVANCE, "getConsensusLedger",
[app, hash] (Job&) {
app->getInboundLedgers().acquire(
hash, 0, InboundLedger::fcCONSENSUS);
});
}
return boost::none;
}
assert (!buildLCL->open() && buildLCL->isImmutable ());
assert (buildLCL->info().hash == ledger);
return RCLCxLedger(buildLCL);
}
std::vector<RCLCxPeerPos>
RCLConsensus::proposals (LedgerHash const& prevLedger)
{
std::vector <RCLCxPeerPos> ret;
{
std::lock_guard <std::mutex> _(peerPositionsLock_);
for (auto const& it : peerPositions_)
for (auto const& pos : it.second)
if (pos->proposal().prevLedger() == prevLedger)
ret.emplace_back (*pos);
}
return ret;
}
void
RCLConsensus::storeProposal (
RCLCxPeerPos::ref peerPos,
NodeID const& nodeID)
{
std::lock_guard <std::mutex> _(peerPositionsLock_);
auto& props = peerPositions_[nodeID];
if (props.size () >= 10)
props.pop_front ();
props.push_back (peerPos);
}
void
RCLConsensus::relay(RCLCxPeerPos const & peerPos)
{
protocol::TMProposeSet prop;
auto const & proposal = peerPos.proposal();
prop.set_proposeseq (
proposal.proposeSeq ());
prop.set_closetime (
proposal.closeTime ().time_since_epoch().count());
prop.set_currenttxhash (
proposal.position().begin(), proposal.position().size());
prop.set_previousledger (
proposal.prevLedger().begin(), proposal.position().size());
auto const pk = peerPos.getPublicKey().slice();
prop.set_nodepubkey (pk.data(), pk.size());
auto const sig = peerPos.getSignature();
prop.set_signature (sig.data(), sig.size());
app_.overlay().relay (prop, peerPos.getSuppressionID ());
}
void
RCLConsensus::relay(DisputedTx <RCLCxTx, NodeID> const & dispute)
{
// If we didn't relay this transaction recently, relay it to all peers
auto const & tx = dispute.tx();
if (app_.getHashRouter ().shouldRelay (tx.id()))
{
auto const slice = tx.tx_.slice();
protocol::TMTransaction msg;
msg.set_rawtransaction (slice.data(), slice.size());
msg.set_status (protocol::tsNEW);
msg.set_receivetimestamp (
app_.timeKeeper().now().time_since_epoch().count());
app_.overlay ().foreach (send_always (
std::make_shared<Message> (
msg, protocol::mtTRANSACTION)));
}
}
void
RCLConsensus::propose (RCLCxPeerPos::Proposal const& proposal)
{
JLOG (j_.trace()) << "We propose: " <<
(proposal.isBowOut () ? std::string ("bowOut") :
to_string (proposal.position ()));
protocol::TMProposeSet prop;
prop.set_currenttxhash (proposal.position().begin(),
proposal.position().size());
prop.set_previousledger (proposal.prevLedger().begin(),
proposal.position().size());
prop.set_proposeseq (proposal.proposeSeq());
prop.set_closetime (
proposal.closeTime().time_since_epoch().count());
prop.set_nodepubkey (valPublic_.data(), valPublic_.size());
auto signingHash = sha512Half(
HashPrefix::proposal,
std::uint32_t(proposal.proposeSeq()),
proposal.closeTime().time_since_epoch().count(),
proposal.prevLedger(), proposal.position());
auto sig = signDigest (
valPublic_, valSecret_, signingHash);
prop.set_signature (sig.data(), sig.size());
app_.overlay().send(prop);
}
void
RCLConsensus::share (RCLTxSet const& set)
{
inboundTransactions_.giveSet (set.id(),
set.map_, false);
}
boost::optional<RCLTxSet>
RCLConsensus::acquireTxSet(RCLTxSet::ID const & setId)
{
if (auto set = inboundTransactions_.getSet(setId, true))
{
return RCLTxSet{std::move(set)};
}
return boost::none;
}
bool
RCLConsensus::hasOpenTransactions() const
{
return ! app_.openLedger().empty();
}
std::size_t
RCLConsensus::proposersValidated(LedgerHash const & h) const
{
return app_.getValidations().getTrustedValidationCount(h);
}
std::size_t
RCLConsensus::proposersFinished(LedgerHash const & h) const
{
return app_.getValidations().getNodesAfter(h);
}
uint256
RCLConsensus::getLCL (
uint256 const& currentLedger,
uint256 const& priorLedger,
bool believedCorrect)
{
// Get validators that are on our ledger, or "close" to being on
// our ledger.
auto vals =
app_.getValidations().getCurrentValidations(
currentLedger, priorLedger,
ledgerMaster_.getValidLedgerIndex());
uint256 netLgr = currentLedger;
int netLgrCount = 0;
for (auto& it : vals)
{
// Switch to ledger supported by more peers
// Or stick with ours on a tie
if ((it.second.first > netLgrCount) ||
((it.second.first == netLgrCount) && (it.first == currentLedger)))
{
netLgr = it.first;
netLgrCount = it.second.first;
}
}
if(netLgr != currentLedger)
{
if (believedCorrect)
app_.getOPs().consensusViewChange();
if (auto stream = j_.debug())
{
for (auto& it : vals)
stream << "V: " << it.first << ", " << it.second.first;
stream << getJson (true);
}
}
return netLgr;
}
void
RCLConsensus::onClose(RCLCxLedger const & ledger, bool haveCorrectLCL)
{
notify(protocol::neCLOSING_LEDGER, ledger, haveCorrectLCL);
}
std::pair <RCLTxSet, typename RCLCxPeerPos::Proposal>
RCLConsensus::makeInitialPosition (RCLCxLedger const & prevLedgerT,
bool proposing,
bool correctLCL,
NetClock::time_point closeTime,
NetClock::time_point now)
{
auto const &prevLedger = prevLedgerT.ledger_;
ledgerMaster_.applyHeldTransactions ();
// Tell the ledger master not to acquire the ledger we're probably building
ledgerMaster_.setBuildingLedger (prevLedger->info().seq + 1);
auto initialLedger = app_.openLedger().current();
auto initialSet = std::make_shared <SHAMap> (
SHAMapType::TRANSACTION, app_.family(), SHAMap::version{1});
initialSet->setUnbacked ();
// Build SHAMap containing all transactions in our open ledger
for (auto const& tx : initialLedger->txs)
{
Serializer s (2048);
tx.first->add(s);
initialSet->addItem (
SHAMapItem (tx.first->getTransactionID(), std::move (s)), true, false);
}
// Add pseudo-transactions to the set
if ((app_.config().standalone() || (proposing && correctLCL))
&& ((prevLedger->info().seq % 256) == 0))
{
// previous ledger was flag ledger, add pseudo-transactions
auto const validations =
app_.getValidations().getValidations (
prevLedger->info().parentHash);
std::size_t const count = std::count_if (
validations.begin(), validations.end(),
[](auto const& v)
{
return v.second->isTrusted();
});
if (count >= app_.validators ().quorum ())
{
feeVote_->doVoting (
prevLedger,
validations,
initialSet);
app_.getAmendmentTable ().doVoting (
prevLedger,
validations,
initialSet);
}
}
// Now we need an immutable snapshot
initialSet = initialSet->snapShot(false);
auto setHash = initialSet->getHash().as_uint256();
return std::make_pair<RCLTxSet, RCLCxPeerPos::Proposal> (
std::move (initialSet),
RCLCxPeerPos::Proposal {
initialLedger->info().parentHash,
RCLCxPeerPos::Proposal::seqJoin,
setHash,
closeTime,
now,
nodeID_ });
}
void
RCLConsensus::dispatchAccept(RCLTxSet const & txSet)
{
app_.getJobQueue().addJob(jtACCEPT, "acceptLedger",
[that = this->shared_from_this(),
consensusSet = txSet]
(auto &)
{
that->accept(consensusSet);
});
}
bool
RCLConsensus::accept(
RCLTxSet const& set,
NetClock::time_point consensusCloseTime,
bool proposing_,
bool validating_,
bool haveCorrectLCL_,
bool consensusFail_,
LedgerHash const &prevLedgerHash_,
RCLCxLedger const & previousLedger_,
NetClock::duration closeResolution_,
NetClock::time_point const & now_,
std::chrono::milliseconds const & roundTime_,
hash_map<RCLCxTx::ID, DisputedTx <RCLCxTx, NodeID>> const & disputes_,
std::map <NetClock::time_point, int> closeTimes_,
NetClock::time_point const & closeTime_
)
{
bool closeTimeCorrect;
if (consensusCloseTime == NetClock::time_point{})
{
// We agreed to disagree on the close time
consensusCloseTime = previousLedger_.closeTime() + 1s;
closeTimeCorrect = false;
}
else
{
// We agreed on a close time
consensusCloseTime = effectiveCloseTime(consensusCloseTime,
closeResolution_, previousLedger_.closeTime());
closeTimeCorrect = true;
}
JLOG (j_.debug())
<< "Report: Prop=" << (proposing_ ? "yes" : "no")
<< " val=" << (validating_ ? "yes" : "no")
<< " corLCL=" << (haveCorrectLCL_ ? "yes" : "no")
<< " fail=" << (consensusFail_ ? "yes" : "no");
JLOG (j_.debug())
<< "Report: Prev = " << prevLedgerHash_
<< ":" << previousLedger_.seq();
//--------------------------------------------------------------------------
// Put transactions into a deterministic, but unpredictable, order
CanonicalTXSet retriableTxs{ set.id() };
auto sharedLCL = buildLCL(previousLedger_, set, consensusCloseTime,
closeTimeCorrect, closeResolution_, now_, roundTime_, retriableTxs);
auto const newLCLHash = sharedLCL.id();
JLOG (j_.debug())
<< "Report: NewL = " << newLCLHash
<< ":" << sharedLCL.seq();
// Tell directly connected peers that we have a new LCL
notify (protocol::neACCEPTED_LEDGER, sharedLCL, haveCorrectLCL_);
if (validating_)
validating_ = ledgerMaster_.isCompatible(*sharedLCL.ledger_,
app_.journal("LedgerConsensus").warn(), "Not validating");
if (validating_ && ! consensusFail_)
{
validate(sharedLCL, now_, proposing_);
JLOG (j_.info())
<< "CNF Val " << newLCLHash;
}
else
JLOG (j_.info())
<< "CNF buildLCL " << newLCLHash;
// See if we can accept a ledger as fully-validated
ledgerMaster_.consensusBuilt (sharedLCL.ledger_, getJson(true));
//-------------------------------------------------------------------------
{
// Apply disputed transactions that didn't get in
//
// The first crack of transactions to get into the new
// open ledger goes to transactions proposed by a validator
// we trust but not included in the consensus set.
//
// These are done first because they are the most likely
// to receive agreement during consensus. They are also
// ordered logically "sooner" than transactions not mentioned
// in the previous consensus round.
//
bool anyDisputes = false;
for (auto& it : disputes_)
{
if (!it.second.getOurVote ())
{
// we voted NO
try
{
JLOG (j_.debug())
<< "Test applying disputed transaction that did"
<< " not get in";
SerialIter sit (it.second.tx().tx_.slice());
auto txn = std::make_shared<STTx const>(sit);
retriableTxs.insert (txn);
anyDisputes = true;
}
catch (std::exception const&)
{
JLOG (j_.debug())
<< "Failed to apply transaction we voted NO on";
}
}
}
// Build new open ledger
auto lock = make_lock(
app_.getMasterMutex(), std::defer_lock);
auto sl = make_lock(
ledgerMaster_.peekMutex (), std::defer_lock);
std::lock(lock, sl);
auto const lastVal = ledgerMaster_.getValidatedLedger();
boost::optional<Rules> rules;
if (lastVal)
rules.emplace(*lastVal, app_.config().features);
else
rules.emplace(app_.config().features);
app_.openLedger().accept(app_, *rules,
sharedLCL.ledger_, localTxs_.getTxSet(), anyDisputes, retriableTxs, tapNONE,
"consensus",
[&](OpenView& view, beast::Journal j)
{
// Stuff the ledger with transactions from the queue.
return app_.getTxQ().accept(app_, view);
});
// Signal a potential fee change to subscribers after the open ledger
// is created
app_.getOPs().reportFeeChange();
}
//-------------------------------------------------------------------------
{
ledgerMaster_.switchLCL (sharedLCL.ledger_);
// Do these need to exist?
assert (ledgerMaster_.getClosedLedger()->info().hash == sharedLCL.id());
assert (app_.openLedger().current()->info().parentHash == sharedLCL.id());
}
//-------------------------------------------------------------------------
if (haveCorrectLCL_ && ! consensusFail_)
{
// we entered the round with the network,
// see how close our close time is to other node's
// close time reports, and update our clock.
JLOG (j_.info())
<< "We closed at " << closeTime_.time_since_epoch().count();
using usec64_t = std::chrono::duration<std::uint64_t>;
usec64_t closeTotal = std::chrono::duration_cast<usec64_t>(closeTime_.time_since_epoch());
int closeCount = 1;
for (auto const& p : closeTimes_)
{
// FIXME: Use median, not average
JLOG (j_.info())
<< std::to_string(p.second)
<< " time votes for "
<< std::to_string(p.first.time_since_epoch().count());
closeCount += p.second;
closeTotal += std::chrono::duration_cast<usec64_t>(p.first.time_since_epoch()) * p.second;
}
closeTotal += usec64_t(closeCount / 2); // for round to nearest
closeTotal /= closeCount;
// Use signed times since we are subtracting
using duration = std::chrono::duration<std::int32_t>;
using time_point = std::chrono::time_point<NetClock, duration>;
auto offset = time_point{closeTotal} -
std::chrono::time_point_cast<duration>(closeTime_);
JLOG (j_.info())
<< "Our close offset is estimated at "
<< offset.count() << " (" << closeCount << ")";
app_.timeKeeper().adjustCloseTime(offset);
}
return validating_;
}
void
RCLConsensus::endConsensus(bool correctLCL)
{
app_.getOPs ().endConsensus (correctLCL);
}
void
RCLConsensus::notify(
protocol::NodeEvent ne,
RCLCxLedger const & ledger,
bool haveCorrectLCL)
{
protocol::TMStatusChange s;
if (!haveCorrectLCL)
s.set_newevent (protocol::neLOST_SYNC);
else
s.set_newevent(ne);
s.set_ledgerseq (ledger.seq());
s.set_networktime (app_.timeKeeper().now().time_since_epoch().count());
s.set_ledgerhashprevious(ledger.parentID().begin (),
std::decay_t<decltype(ledger.parentID())>::bytes);
s.set_ledgerhash (ledger.id().begin (),
std::decay_t<decltype(ledger.id())>::bytes);
std::uint32_t uMin, uMax;
if (! ledgerMaster_.getFullValidatedRange (uMin, uMax))
{
uMin = 0;
uMax = 0;
}
else
{
// Don't advertise ledgers we're not willing to serve
uMin = std::max(uMin, ledgerMaster_.getEarliestFetch ());
}
s.set_firstseq (uMin);
s.set_lastseq (uMax);
app_.overlay ().foreach (send_always (
std::make_shared <Message> (
s, protocol::mtSTATUS_CHANGE)));
JLOG (j_.trace()) << "send status change to peer";
}
/** Apply a set of transactions to a ledger.
Typically the txFilter is used to reject transactions
that already accepted in the prior ledger.
@param set set of transactions to apply
@param view ledger to apply to
@param txFilter callback, return false to reject txn
@return retriable transactions
*/
CanonicalTXSet
applyTransactions (
Application& app,
RCLTxSet const& cSet,
OpenView& view,
std::function<bool(uint256 const&)> txFilter)
{
auto j = app.journal ("LedgerConsensus");
auto& set = *(cSet.map_);
CanonicalTXSet retriableTxs (set.getHash().as_uint256());
for (auto const& item : set)
{
if (! txFilter (item.key()))
continue;
// The transaction wan't filtered
// Add it to the set to be tried in canonical order
JLOG (j.debug()) <<
"Processing candidate transaction: " << item.key();
try
{
retriableTxs.insert (
std::make_shared<STTx const>(SerialIter{item.slice()}));
}
catch (std::exception const&)
{
JLOG (j.warn()) << "Txn " << item.key() << " throws";
}
}
bool certainRetry = true;
// Attempt to apply all of the retriable transactions
for (int pass = 0; pass < LEDGER_TOTAL_PASSES; ++pass)
{
JLOG (j.debug()) << "Pass: " << pass << " Txns: "
<< retriableTxs.size ()
<< (certainRetry ? " retriable" : " final");
int changes = 0;
auto it = retriableTxs.begin ();
while (it != retriableTxs.end ())
{
try
{
switch (applyTransaction (app, view,
*it->second, certainRetry, tapNO_CHECK_SIGN, j))
{
case ApplyResult::Success:
it = retriableTxs.erase (it);
++changes;
break;
case ApplyResult::Fail:
it = retriableTxs.erase (it);
break;
case ApplyResult::Retry:
++it;
}
}
catch (std::exception const&)
{
JLOG (j.warn())
<< "Transaction throws";
it = retriableTxs.erase (it);
}
}
JLOG (j.debug()) << "Pass: "
<< pass << " finished " << changes << " changes";
// A non-retry pass made no changes
if (!changes && !certainRetry)
return retriableTxs;
// Stop retriable passes
if (!changes || (pass >= LEDGER_RETRY_PASSES))
certainRetry = false;
}
// If there are any transactions left, we must have
// tried them in at least one final pass
assert (retriableTxs.empty() || !certainRetry);
return retriableTxs;
}
RCLCxLedger
RCLConsensus::buildLCL(
RCLCxLedger const & previousLedger,
RCLTxSet const & set,
NetClock::time_point closeTime,
bool closeTimeCorrect,
NetClock::duration closeResolution,
NetClock::time_point now,
std::chrono::milliseconds roundTime,
CanonicalTXSet & retriableTxs)
{
auto replay = ledgerMaster_.releaseReplay();
if (replay)
{
// replaying, use the time the ledger we're replaying closed
closeTime = replay->closeTime_;
closeTimeCorrect = ((replay->closeFlags_ & sLCF_NoConsensusTime) == 0);
}
JLOG (j_.debug())
<< "Report: TxSt = " << set.id ()
<< ", close " << closeTime.time_since_epoch().count()
<< (closeTimeCorrect ? "" : "X");
// Build the new last closed ledger
auto buildLCL = std::make_shared<Ledger>(*previousLedger.ledger_, now);
auto const v2_enabled = buildLCL->rules().enabled(featureSHAMapV2);
auto v2_transition = false;
if (v2_enabled && !buildLCL->stateMap().is_v2())
{
buildLCL->make_v2();
v2_transition = true;
}
// Set up to write SHAMap changes to our database,
// perform updates, extract changes
JLOG (j_.debug())
<< "Applying consensus set transactions to the"
<< " last closed ledger";
{
OpenView accum(&*buildLCL);
assert(!accum.open());
if (replay)
{
// Special case, we are replaying a ledger close
for (auto& tx : replay->txns_)
applyTransaction (app_, accum, *tx.second,
false, tapNO_CHECK_SIGN, j_);
}
else
{
// Normal case, we are not replaying a ledger close
retriableTxs = applyTransactions (app_, set, accum,
[&buildLCL](uint256 const& txID)
{
return ! buildLCL->txExists(txID);
});
}
// Update fee computations.
app_.getTxQ().processClosedLedger(app_, accum,
roundTime > 5s);
accum.apply(*buildLCL);
}
// retriableTxs will include any transactions that
// made it into the consensus set but failed during application
// to the ledger.
buildLCL->updateSkipList ();
{
// Write the final version of all modified SHAMap
// nodes to the node store to preserve the new LCL
int asf = buildLCL->stateMap().flushDirty (
hotACCOUNT_NODE, buildLCL->info().seq);
int tmf = buildLCL->txMap().flushDirty (
hotTRANSACTION_NODE, buildLCL->info().seq);
JLOG (j_.debug()) << "Flushed " <<
asf << " accounts and " <<
tmf << " transaction nodes";
}
buildLCL->unshare();
// Accept ledger
buildLCL->setAccepted(closeTime, closeResolution,
closeTimeCorrect, app_.config());
// And stash the ledger in the ledger master
if (ledgerMaster_.storeLedger (buildLCL))
JLOG (j_.debug())
<< "Consensus built ledger we already had";
else if (app_.getInboundLedgers().find (buildLCL->info().hash))
JLOG (j_.debug())
<< "Consensus built ledger we were acquiring";
else
JLOG (j_.debug())
<< "Consensus built new ledger";
return RCLCxLedger{std::move(buildLCL)};
}
void
RCLConsensus::validate(
RCLCxLedger const & ledger,
NetClock::time_point now,
bool proposing)
{
auto validationTime = now;
if (validationTime <= lastValidationTime_)
validationTime = lastValidationTime_ + 1s;
lastValidationTime_ = validationTime;
// Build validation
auto v = std::make_shared<STValidation> (ledger.id(),
validationTime, valPublic_, proposing);
v->setFieldU32 (sfLedgerSequence, ledger.seq());
// Add our load fee to the validation
auto const& feeTrack = app_.getFeeTrack();
std::uint32_t fee = std::max(
feeTrack.getLocalFee(),
feeTrack.getClusterFee());
if (fee > feeTrack.getLoadBase())
v->setFieldU32(sfLoadFee, fee);
if (((ledger.seq() + 1) % 256) == 0)
// next ledger is flag ledger
{
// Suggest fee changes and new features
feeVote_->doValidation (ledger.ledger_, *v);
app_.getAmendmentTable ().doValidation (ledger.ledger_, *v);
}
auto const signingHash = v->sign (valSecret_);
v->setTrusted ();
// suppress it if we receive it - FIXME: wrong suppression
app_.getHashRouter ().addSuppression (signingHash);
app_.getValidations ().addValidation (v, "local");
Blob validation = v->getSerialized ();
protocol::TMValidation val;
val.set_validation (&validation[0], validation.size ());
// Send signed validation to all of our directly connected peers
app_.overlay().send(val);
}
PublicKey const&
RCLConsensus::getValidationPublicKey () const
{
return valPublic_;
}
void
RCLConsensus::setValidationKeys (SecretKey const& valSecret,
PublicKey const& valPublic)
{
valSecret_ = valSecret;
valPublic_ = valPublic;
}
}

383
src/ripple/app/consensus/RCLConsensus.h Normal file
View File

@@ -0,0 +1,383 @@
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012, 2013 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_APP_CONSENSUS_RCLCONSENSUS_H_INCLUDED
#define RIPPLE_APP_CONSENSUS_RCLCONSENSUS_H_INCLUDED
#include <BeastConfig.h>
#include <ripple/basics/Log.h>
#include <ripple/protocol/STValidation.h>
#include <ripple/shamap/SHAMap.h>
#include <ripple/beast/utility/Journal.h>
#include <ripple/app/misc/FeeVote.h>
#include <ripple/protocol/RippleLedgerHash.h>
#include <ripple/app/consensus/RCLCxLedger.h>
#include <ripple/app/consensus/RCLCxTx.h>
#include <ripple/app/consensus/RCLCxPeerPos.h>
#include <ripple/core/JobQueue.h>
#include <ripple/consensus/Consensus.h>
#include <ripple/basics/CountedObject.h>
#include <ripple/overlay/Message.h>
namespace ripple {
class InboundTransactions;
class LocalTxs;
class LedgerMaster;
//! Types used to adapt consensus for RCL
struct RCLCxTraits
{
//! Ledger type presented to Consensus
using Ledger_t = RCLCxLedger;
//! Peer identifier type used in Consensus
using NodeID_t = NodeID;
//! TxSet type presented to Consensus
using TxSet_t = RCLTxSet;
//! MissingTxException type neede by Consensus
using MissingTxException_t = SHAMapMissingNode;
};
/** Adapts the generic Consensus algorithm for use by RCL.
@note The enabled_shared_from_this base allows the application to properly
create a shared instance of RCLConsensus for use in the accept logic..
*/
class RCLConsensus : public Consensus<RCLConsensus, RCLCxTraits>
, public std::enable_shared_from_this <RCLConsensus>
, public CountedObject <RCLConsensus>
{
using Base = Consensus<RCLConsensus, RCLCxTraits>;
using Base::accept;
public:
//! Constructor
RCLConsensus(
Application& app,
std::unique_ptr<FeeVote> && feeVote,
LedgerMaster& ledgerMaster,
LocalTxs& localTxs,
InboundTransactions& inboundTransactions,
typename Base::clock_type const & clock,
beast::Journal journal);
RCLConsensus(RCLConsensus const&) = delete;
RCLConsensus& operator=(RCLConsensus const&) = delete;
static char const* getCountedObjectName() { return "Consensus"; }
/** Save the given consensus proposed by a peer with nodeID for later
use in consensus.
@param peerPos Proposed peer position
@param nodeID ID of peer
*/
void
storeProposal( RCLCxPeerPos::ref peerPos, NodeID const& nodeID);
/** Returns validation public key */
PublicKey const&
getValidationPublicKey () const;
/** Set validation private and public key pair. */
void
setValidationKeys (SecretKey const& valSecret, PublicKey const& valPublic);
private:
friend class Consensus<RCLConsensus, RCLCxTraits>;
//-------------------------------------------------------------------------
// Consensus type requirements.
/** Notification that a new consensus round has begun.
@param ledger The ledger we are building consensus on
*/
void
onStartRound(RCLCxLedger const & ledger);
//! @return Whether consensus should be (proposing, validating)
std::pair <bool, bool>
getMode ();
/** Attempt to acquire a specific ledger.
If not available, asynchronously acquires from the network.
@param ledger The ID/hash of the ledger acquire
@return Optional ledger, will be seated if we locally had the ledger
*/
boost::optional<RCLCxLedger>
acquireLedger(LedgerHash const & ledger);
/** Get peers' proposed positions.
@param prevLedger The base ledger which proposals are based on
@return The set of proposals
*/
std::vector<RCLCxPeerPos>
proposals (LedgerHash const& prevLedger);
/** Relay the given proposal to all peers
@param peerPos The peer position to relay.
*/
void
relay(RCLCxPeerPos const & peerPos);
/** Relay disputed transacction to peers.
Only relay if the provided transaction hasn't been shared recently.
@param dispute The disputed transaction to relay.
*/
void
relay(DisputedTx <RCLCxTx, NodeID> const & dispute);
/** Acquire the transaction set associated with a proposal.
If the transaction set is not available locally, will attempt acquire it
from the network.
@param setId The transaction set ID associated with the proposal
@return Optional set of transactions, seated if available.
*/
boost::optional<RCLTxSet>
acquireTxSet(RCLTxSet::ID const & setId);
/** Whether the open ledger has any transactions
*/
bool
hasOpenTransactions() const;
/** Number of proposers that have vallidated the given ledger
@param h The hash of the ledger of interest
@return the number of proposers that validated a ledger
*/
std::size_t
proposersValidated(LedgerHash const & h) const;
/** Number of proposers that have validated a ledger descended from requested ledger.
@param h The hash of the ledger of interest.
@return The number of validating peers that have validated a ledger
succeeding the one provided.
*/
std::size_t
proposersFinished(LedgerHash const & h) const;
/** Propose the given position to my peers.
@param proposal Our proposed position
*/
void
propose (RCLCxPeerPos::Proposal const& proposal);
/** Share the given tx set with peers.
@param set The TxSet to share.
*/
void
share (RCLTxSet const& set);
/** Get the last closed ledger (LCL) seen on the network
@param currentLedger Current ledger used in consensus
@param priorLedger Prior ledger used in consensus
@param believedCorrect Whether consensus believes currentLedger is LCL
@return The hash of the last closed network
*/
uint256
getLCL (
uint256 const& currentLedger,
uint256 const& priorLedger,
bool believedCorrect);
/** Notification that the ledger has closed.
@param ledger the ledger we are changing to
@param haveCorrectLCL whether we believe this is the correct LCL
*/
void
onClose(RCLCxLedger const & ledger, bool haveCorrectLCL);
/** Create our initial position of transactions to accept in this round
of consensus.
@param prevLedger The ledger the transactions apply to
@param isProposing Whether we are currently proposing
@param isCorrectLCL Whether we have the correct LCL
@param closeTime When we believe the ledger closed
@param now The current network adjusted time
@return Pair of (i) transactions we believe are in the ledger
(ii) the corresponding proposal of those transactions
to send to peers
*/
std::pair <RCLTxSet, typename RCLCxPeerPos::Proposal>
makeInitialPosition (
RCLCxLedger const & prevLedger,
bool isProposing,
bool isCorrectLCL,
NetClock::time_point closeTime,
NetClock::time_point now);
/** Dispatch a call to Consensus::accept
Accepting a ledger may be expensive, so this function can dispatch
that call to another thread if desired and must call the accept
method of the generic consensus algorithm.
@param txSet The transactions to accept.
*/
void
dispatchAccept(RCLTxSet const & txSet);
/** Accept a new ledger based on the given transactions.
TODO: Too many arguments, need to group related types.
@param set The set of accepted transactions
@param consensusCloseTime Consensus agreed upon close time
@param proposing_ Whether we are proposing
@param validating_ Whether we are validating
@param haveCorrectLCL_ Whether we had the correct last closed ledger
@param consensusFail_ Whether consensus failed
@param prevLedgerHash_ The hash/id of the previous ledger
@param previousLedger_ The previous ledger
@param closeResolution_ The close time resolution used this round
@param now Current network adjsuted time
@param roundTime_ Duration of this consensus round
@param disputes_ Disputed trarnsactions from this round
@param closeTimes_ Histogram of peers close times
@param closeTime Our close time
@return Whether we should continue validating
*/
bool
accept(
RCLTxSet const& set,
NetClock::time_point consensusCloseTime,
bool proposing_,
bool validating_,
bool haveCorrectLCL_,
bool consensusFail_,
LedgerHash const &prevLedgerHash_,
RCLCxLedger const & previousLedger_,
NetClock::duration closeResolution_,
NetClock::time_point const & now,
std::chrono::milliseconds const & roundTime_,
hash_map<RCLCxTx::ID, DisputedTx <RCLCxTx, NodeID>> const & disputes_,
std::map <NetClock::time_point, int> closeTimes_,
NetClock::time_point const & closeTime
);
/** Signal the end of consensus to the application, which will start the
next round.
@param correctLCL Whether we believe we have the correct LCL
*/
void
endConsensus(bool correctLCL);
//!-------------------------------------------------------------------------
// Additional members (not directly required by Consensus interface)
/** Notify peers of a consensus state change
@param ne Event type for notification
@param ledger The ledger at the time of the state change
@param haveCorrectLCL Whether we believ we have the correct LCL.
*/
void
notify(protocol::NodeEvent ne, RCLCxLedger const & ledger, bool haveCorrectLCL);
/** Build the new last closed ledger.
Accept the given the provided set of consensus transactions and build
the last closed ledger. Since consensus just agrees on which
transactions to apply, but not whether they make it into the closed
ledger, this function also populates retriableTxs with those that can
be retried in the next round.
@param previousLedger Prior ledger building upon
@param set The set of transactions to apply to the ledger
@param closeTime The the ledger closed
@param closeTimeCorrect Whether consensus agreed on close time
@param closeResolution Resolution used to determine consensus close time
@param now Current network adjusted time
@param roundTime Duration of this consensus rorund
@param retriableTxs Populate with transactions to retry in next round
@return The newly built ledger
*/
RCLCxLedger
buildLCL(
RCLCxLedger const & previousLedger,
RCLTxSet const & set,
NetClock::time_point closeTime,
bool closeTimeCorrect,
NetClock::duration closeResolution,
NetClock::time_point now,
std::chrono::milliseconds roundTime,
CanonicalTXSet & retriableTxs
);
/** Validate the given ledger and share with peers as necessary
@param ledger The ledger to validate
@param now Current network adjusted time
@param proposing Whether we were proposing transactions while generating
this ledger. If we are not proposing, a validation
can still be sent to inform peers that we know we
aren't fully participating in consensus but are still
around and trying to catch up.
*/
void
validate(
RCLCxLedger const & ledger,
NetClock::time_point now,
bool proposing);
//!-------------------------------------------------------------------------
Application& app_;
std::unique_ptr <FeeVote> feeVote_;
LedgerMaster & ledgerMaster_;
LocalTxs & localTxs_;
InboundTransactions& inboundTransactions_;
beast::Journal j_;
NodeID nodeID_;
PublicKey valPublic_;
SecretKey valSecret_;
LedgerHash acquiringLedger_;
// The timestamp of the last validation we used, in network time. This is
// only used for our own validations.
NetClock::time_point lastValidationTime_;
using PeerPositions = hash_map <NodeID, std::deque<RCLCxPeerPos::pointer>>;
PeerPositions peerPositions_;
std::mutex peerPositionsLock_;
};
}
#endif

121
src/ripple/app/consensus/RCLCxLedger.h Normal file
View File

@@ -0,0 +1,121 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2016 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_APP_CONSENSUS_RCLCXLEDGER_H_INCLUDED
#define RIPPLE_APP_CONSENSUS_RCLCXLEDGER_H_INCLUDED
#include <ripple/app/ledger/Ledger.h>
#include <ripple/ledger/ReadView.h>
#include <ripple/app/ledger/LedgerToJson.h>
#include <ripple/protocol/RippleLedgerHash.h>
#include <memory>
namespace ripple {
/** Represents a ledger in RCLConsensus.
RCLCxLedger is a thin wrapper over `std::shared_ptr<Ledger const>`.
*/
class RCLCxLedger
{
public:
//! Unique identifier of a ledger
using ID = LedgerHash;
/** Default constructor
TODO: This may not be needed if we ensure RCLConsensus is handed a valid
ledger in its constructor. Its bad now because other members are not
checking whether the ledger is valid.
*/
RCLCxLedger() = default;
/** Constructor
@param l The ledger to wrap.
*/
RCLCxLedger(std::shared_ptr<Ledger const> const & l) : ledger_{ l } {}
//! Sequence number of the ledger.
auto const &
seq() const
{
return ledger_->info().seq;
}
//! Unique identifier (hash) of this ledger.
auto const &
id() const
{
return ledger_->info().hash;
}
//! Unique identifier (hash) of this ledger's parent.
auto const &
parentID() const
{
return ledger_->info().parentHash;
}
//! Resolution used when calculating this ledger's close time.
auto
closeTimeResolution() const
{
return ledger_->info().closeTimeResolution;
}
//! Whether consensus process agreed on close time of the ledger.
bool
closeAgree() const
{
return ripple::getCloseAgree(ledger_->info());
}
//! The close time of this ledger
auto
closeTime() const
{
return ledger_->info().closeTime;
}
//! The close time of this ledger's parent.
auto
parentCloseTime() const
{
return ledger_->info().parentCloseTime;
}
//! JSON representation of this ledger.
Json::Value
getJson() const
{
return ripple::getJson(*ledger_);
}
/** The ledger instance.
TODO: Make this shared_ptr<ReadView const> .. requires ability to create
a new ledger from a readView?
*/
std::shared_ptr<Ledger const> ledger_;
};
}
#endif

91
src/ripple/app/ledger/LedgerProposal.cpp → src/ripple/app/consensus/RCLCxPeerPos.cpp
View File

@@ -18,7 +18,7 @@
//==============================================================================
#include <BeastConfig.h>
#include <ripple/app/ledger/LedgerProposal.h>
#include <ripple/app/consensus/RCLCxPeerPos.h>
#include <ripple/protocol/digest.h>
#include <ripple/core/Config.h>
#include <ripple/protocol/JsonFields.h>
@@ -28,97 +28,40 @@
namespace ripple {
// Used to construct received proposals
LedgerProposal::LedgerProposal (
uint256 const& pLgr,
std::uint32_t seq,
uint256 const& tx,
NetClock::time_point closeTime,
NetClock::time_point now,
RCLCxPeerPos::RCLCxPeerPos (
PublicKey const& publicKey,
NodeID const& nodeID,
Slice const& signature,
uint256 const& suppression)
: mPreviousLedger (pLgr)
, mCurrentHash (tx)
, mSuppression (suppression)
, mCloseTime (closeTime)
, mProposeSeq (seq)
, publicKey_ (publicKey)
, mPeerID (nodeID)
, mTime (now)
uint256 const& suppression,
Proposal && proposal)
: proposal_{ std::move(proposal)}
, mSuppression {suppression}
, publicKey_{publicKey}
, signature_{signature}
{
signature_.resize (signature.size());
std::memcpy(signature_.data(),
signature.data(), signature.size());
}
// Used to construct local proposals
// CAUTION: publicKey_ not set
LedgerProposal::LedgerProposal (
uint256 const& prevLgr,
uint256 const& position,
NetClock::time_point closeTime,
NetClock::time_point now)
: mPreviousLedger (prevLgr)
, mCurrentHash (position)
, mCloseTime (closeTime)
, mProposeSeq (seqJoin)
, mTime (now)
{
}
uint256 LedgerProposal::getSigningHash () const
uint256 RCLCxPeerPos::getSigningHash () const
{
return sha512Half(
HashPrefix::proposal,
std::uint32_t(mProposeSeq),
mCloseTime.time_since_epoch().count(),
mPreviousLedger,
mCurrentHash);
std::uint32_t(proposal().proposeSeq()),
proposal().closeTime().time_since_epoch().count(),
proposal().prevLedger(),
proposal().position());
}
bool LedgerProposal::checkSign () const
bool RCLCxPeerPos::checkSign () const
{
return verifyDigest (
publicKey_,
getSigningHash(),
makeSlice (signature_),
signature_,
false);
}
bool LedgerProposal::changePosition (
uint256 const& newPosition,
NetClock::time_point closeTime,
NetClock::time_point now)
Json::Value RCLCxPeerPos::getJson () const
{
if (mProposeSeq == seqLeave)
return false;
mCurrentHash = newPosition;
mCloseTime = closeTime;
mTime = now;
++mProposeSeq;
return true;
}
void LedgerProposal::bowOut (NetClock::time_point now)
{
mTime = now;
mProposeSeq = seqLeave;
}
Json::Value LedgerProposal::getJson () const
{
Json::Value ret = Json::objectValue;
ret[jss::previous_ledger] = to_string (mPreviousLedger);
if (mProposeSeq != seqLeave)
{
ret[jss::transaction_hash] = to_string (mCurrentHash);
ret[jss::propose_seq] = mProposeSeq;
}
ret[jss::close_time] = mCloseTime.time_since_epoch().count();
auto ret = proposal().getJson();
if (publicKey_.size())
ret[jss::peer_id] = toBase58 (

143
src/ripple/app/ledger/LedgerProposal.h → src/ripple/app/consensus/RCLCxPeerPos.h
View File

@@ -17,8 +17,8 @@
*/
//==============================================================================
#ifndef RIPPLE_APP_LEDGER_LEDGERPROPOSAL_H_INCLUDED
#define RIPPLE_APP_LEDGER_LEDGERPROPOSAL_H_INCLUDED
#ifndef RIPPLE_APP_CONSENSUS_RCLCXPEERPOS_H_INCLUDED
#define RIPPLE_APP_CONSENSUS_RCLCXPEERPOS_H_INCLUDED
#include <ripple/basics/CountedObject.h>
#include <ripple/basics/base_uint.h>
@@ -27,105 +27,84 @@
#include <ripple/protocol/PublicKey.h>
#include <ripple/protocol/SecretKey.h>
#include <ripple/beast/hash/hash_append.h>
#include <ripple/consensus/ConsensusProposal.h>
#include <chrono>
#include <cstdint>
#include <string>
namespace ripple {
class LedgerProposal
: public CountedObject <LedgerProposal>
/** A peer's signed, proposed position for use in RCLConsensus.
Carries a ConsensusProposal signed by a peer.
*/
class RCLCxPeerPos
: public CountedObject <RCLCxPeerPos>
{
private:
// A peer initial joins the consensus process
static std::uint32_t const seqJoin = 0;
// A peer wants to bow out and leave the consensus process
static std::uint32_t const seqLeave = 0xffffffff;
public:
static char const* getCountedObjectName () { return "LedgerProposal"; }
using pointer = std::shared_ptr<LedgerProposal>;
static char const* getCountedObjectName () { return "RCLCxPeerPos"; }
using pointer = std::shared_ptr<RCLCxPeerPos>;
using ref = const pointer&;
// proposal from peer
LedgerProposal (
uint256 const& prevLgr,
std::uint32_t proposeSeq,
uint256 const& propose,
NetClock::time_point closeTime,
NetClock::time_point now,
//< The type of the proposed position
using Proposal = ConsensusProposal<NodeID, uint256, uint256>;
/** Constructor
Constructs a signed peer position.
@param publicKey Public key of the peer
@param signature Signature provided with the proposal
@param suppress ????
@param proposal The consensus proposal
*/
RCLCxPeerPos (
PublicKey const& publicKey,
NodeID const& nodeID,
Slice const& signature,
uint256 const& suppress);
// Our own proposal:
LedgerProposal (
uint256 const& prevLedger,
uint256 const& position,
NetClock::time_point closeTime,
NetClock::time_point now);
uint256 const& suppress,
Proposal && proposal);
//! Create the signing hash for the proposal
uint256 getSigningHash () const;
//! Verify the signing hash of the proposal
bool checkSign () const;
NodeID const& getPeerID () const
//! Signature of the proposal (not necessarily verified)
Slice getSignature () const
{
return mPeerID;
}
uint256 const& getCurrentHash () const
{
return mCurrentHash;
}
uint256 const& getPrevLedger () const
{
return mPreviousLedger;
return signature_;
}
//! Public key of peer that sent the proposal
PublicKey const& getPublicKey () const
{
return publicKey_;
}
//! ?????
uint256 const& getSuppressionID () const
{
return mSuppression;
}
std::uint32_t getProposeSeq () const
//! The consensus proposal
Proposal const & proposal() const
{
return mProposeSeq;
}
NetClock::time_point getCloseTime () const
{
return mCloseTime;
}
NetClock::time_point getSeenTime () const
{
return mTime;
}
Blob const& getSignature () const
{
return signature_;
}
bool isInitial () const
{
return mProposeSeq == seqJoin;
}
bool isBowOut () const
{
return mProposeSeq == seqLeave;
return proposal_;
}
bool isStale (NetClock::time_point cutoff) const
/// @cond Ignore
//! Add a conversion operator to conform to the Consensus interface
operator Proposal const &() const
{
return mTime <= cutoff;
return proposal_;
}
/// @endcond
bool changePosition (
uint256 const& newPosition,
NetClock::time_point newCloseTime,
NetClock::time_point now);
void bowOut (NetClock::time_point now);
//! JSON representation of proposal
Json::Value getJson () const;
private:
@@ -135,21 +114,16 @@ private:
{
using beast::hash_append;
hash_append(h, HashPrefix::proposal);
hash_append(h, std::uint32_t(mProposeSeq));
hash_append(h, mCloseTime);
hash_append(h, mPreviousLedger);
hash_append(h, mCurrentHash);
hash_append(h, std::uint32_t(proposal().proposeSeq()));
hash_append(h, proposal().closeTime());
hash_append(h, proposal().prevLedger());
hash_append(h, proposal().position());
}
uint256 mPreviousLedger, mCurrentHash, mSuppression;
NetClock::time_point mCloseTime;
std::uint32_t mProposeSeq;
Proposal proposal_;
uint256 mSuppression;
PublicKey publicKey_;
NodeID mPeerID;
Blob signature_;
NetClock::time_point mTime;
Buffer signature_;
};
/** Calculate a unique identifier for a signed proposal.
@@ -160,6 +134,13 @@ private:
present. Recipients of the proposal will inject the last closed ledger in
order to validate the signature. If the last closed ledger is left out, then
it is considered as all zeroes for the purposes of signing.
@param proposeHash The hash of the proposed position
@param previousLedger The hash of the ledger the proposal is based upon
@param proposeSeq Sequence number of the proposal
@param closeTime Close time of the proposal
@param publicKey Signer's public key
@param signature Proposal signature
*/
uint256 proposalUniqueId (
uint256 const& proposeHash,

143
src/ripple/app/consensus/RCLCxPos.h
View File

@@ -1,143 +0,0 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2016 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_APP_CONSENSUS_RCLCXPOSITION_H_INCLUDED
#define RIPPLE_APP_CONSENSUS_RCLCXPOSITION_H_INCLUDED
#include <ripple/app/ledger/LedgerProposal.h>
#include <ripple/json/json_value.h>
#include <ripple/basics/chrono.h>
#include <ripple/protocol/UintTypes.h>
namespace ripple {
// A position taken during a consensus round
// As seen by the RCL consensus process
class RCLCxPos
{
public:
static std::uint32_t constexpr seqInitial = 0;
static std::uint32_t constexpr seqLeave = 0xffffffff;
RCLCxPos (LedgerProposal const& prop) :
proposal_ (prop)
{ }
std::uint32_t getSequence() const
{
return proposal_.getProposeSeq();
}
NetClock::time_point getCloseTime () const
{
return proposal_.getCloseTime();
}
NetClock::time_point getSeenTime() const
{
return proposal_.getSeenTime();
}
bool isStale (NetClock::time_point lastValid) const
{
return getSeenTime() < lastValid;
}
NodeID const& getNodeID() const
{
return proposal_.getPeerID();
}
LedgerHash const& getPosition() const
{
return proposal_.getCurrentHash();
}
LedgerHash const& getPrevLedger() const
{
return proposal_.getPrevLedger();
}
bool changePosition (
LedgerHash const& position,
NetClock::time_point closeTime,
NetClock::time_point now)
{
return proposal_.changePosition (position, closeTime, now);
}
bool bowOut (NetClock::time_point now)
{
if (isBowOut ())
return false;
proposal_.bowOut (now);
return true;
}
Json::Value getJson() const
{
return proposal_.getJson();
}
bool isInitial () const
{
return getSequence() == seqInitial;
}
bool isBowOut() const
{
return getSequence() == seqLeave;
}
// These three functions will be removed. New code
// should use getPosition, getSequence and getNodeID
LedgerHash const& getCurrentHash() const
{
return getPosition();
}
NodeID const& getPeerID() const
{
return getNodeID();
}
std::uint32_t getProposeSeq() const
{
return getSequence();
}
LedgerProposal const& peek() const
{
return proposal_;
}
LedgerProposal& peek()
{
return proposal_;
}
protected:
LedgerProposal proposal_;
};
}
#endif

192
src/ripple/app/consensus/RCLCxTx.h
View File

@@ -23,108 +23,160 @@
#include <ripple/basics/chrono.h>
#include <ripple/protocol/UintTypes.h>
#include <ripple/shamap/SHAMap.h>
#include <ripple/app/misc/CanonicalTXSet.h>
namespace ripple {
// Transactions, as seen by the consensus code in the rippled app
/** Represents a transaction in RCLConsensus.
RCLCxTx is a thin wrapper over the SHAMapItem that corresponds to the
transaction.
*/
class RCLCxTx
{
public:
//! Unique identifier/hash of transaction
using ID = uint256;
RCLCxTx (SHAMapItem const& txn) : txn_ (txn)
/** Constructor
@param txn The transaction to wrap
*/
RCLCxTx(SHAMapItem const& txn) : tx_{ txn }
{ }
uint256 const& getID() const
//! The unique identifier/hash of the transaction
ID const&
id() const
{
return txn_.key ();
return tx_.key ();
}
SHAMapItem const& txn() const
{
return txn_;
}
protected:
SHAMapItem const txn_;
//! The SHAMapItem that represents the transaction.
SHAMapItem const tx_;
};
class RCLTxSet;
/** Represents a set of transactions in RCLConsensus.
class MutableRCLTxSet
{
public:
MutableRCLTxSet (RCLTxSet const&);
bool
addEntry (RCLCxTx const& p)
{
return map_->addItem (
SHAMapItem {p.getID(), p.txn().peekData()},
true, false);
}
bool
removeEntry (uint256 const& entry)
{
return map_->delItem (entry);
}
std::shared_ptr <SHAMap> const& map() const
{
return map_;
}
protected:
std::shared_ptr <SHAMap> map_;
};
// Sets of transactions
// as seen by the consensus code in the rippled app
RCLTxSet is a thin wrapper over a SHAMap that stores the set of
transactions.
*/
class RCLTxSet
{
public:
//! Unique identifier/hash of the set of transactions
using ID = uint256;
//! The type that corresponds to a single transaction
using Tx = RCLCxTx;
using mutable_t = MutableRCLTxSet;
RCLTxSet (std::shared_ptr<SHAMap> map) :
map_ (std::move(map))
//< Provide a mutable view of a TxSet
class MutableTxSet
{
assert (map_);
friend class RCLTxSet;
//! The SHAMap representing the transactions.
std::shared_ptr <SHAMap> map_;
public:
MutableTxSet(RCLTxSet const & src)
: map_{ src.map_->snapShot(true) }
{
}
/** Insert a new transaction into the set.
@param t The transaction to insert.
@return Whether the transaction took place.
*/
bool
insert(Tx const& t)
{
return map_->addItem(
SHAMapItem{ t.id(), t.tx_.peekData() },
true, false);
}
/** Remove a transaction from the set.
@param entry The ID of the transaction to remove.
@return Whether the transaction was removed.
*/
bool
erase(Tx::ID const& entry)
{
return map_->delItem(entry);
}
};
/** Constructor
@param m SHAMap to wrap
*/
RCLTxSet (std::shared_ptr<SHAMap> m)
: map_{ std::move(m) }
{
assert(map_);
}
RCLTxSet (MutableRCLTxSet const& set) :
map_ (set.map()->snapShot (false))
{ }
/** Constructor from a previosly created MutableTxSet
bool hasEntry (uint256 const& entry) const
@param m MutableTxSet that will become fixed
*/
RCLTxSet(MutableTxSet const & m)
: map_{m.map_->snapShot(false)}
{
}
/** Test if a transaction is in the set.
@param entry The ID of transaction to test.
@return Whether the transaction is in the set.
*/
bool
exists(Tx::ID const& entry) const
{
return map_->hasItem (entry);
}
boost::optional <RCLCxTx const>
getEntry (uint256 const& entry) const
/** Lookup a transaction.
@param entry The ID of the transaction to find.
@return A shared pointer to the SHAMapItem.
@note Since find may not succeed, this returns a
`std::shared_ptr<const SHAMapItem>` rather than a Tx, which
cannot refer to a missing transaction. The generic consensus
code use the shared_ptr semantics to know whether the find
was succesfully and properly creates a Tx as needed.
*/
std::shared_ptr<const SHAMapItem> const &
find(Tx::ID const& entry) const
{
auto item = map_->peekItem (entry);
if (item)
return RCLCxTx(*item);
return boost::none;
return map_->peekItem (entry);
}
uint256 getID() const
//! The unique ID/hash of the transaction set
ID
id() const
{
return map_->getHash().as_uint256();
}
std::map <uint256, bool>
getDifferences (RCLTxSet const& j) const
/** Find transactions not in common between this and another transaction set.
@param j The set to compare with
@return Map of transactions in this set and `j` but not both. The key
is the transaction ID and the value is a bool of the transaction
exists in this set.
*/
std::map<Tx::ID, bool>
compare (RCLTxSet const& j) const
{
SHAMap::Delta delta;
// Bound the work we do in case of a malicious
// map from a trusted validator
// map_ from a trusted validator
map_->compare (*(j.map_), delta, 65536);
std::map <uint256, bool> ret;
@@ -138,19 +190,9 @@ public:
return ret;
}
std::shared_ptr<SHAMap> const& map() const
{
return map_;
}
protected:
//! The SHAMap representing the transactions.
std::shared_ptr <SHAMap> map_;
};
inline MutableRCLTxSet::MutableRCLTxSet (RCLTxSet const& set)
: map_ (set.map()->snapShot (true))
{ }
}
#endif

120
src/ripple/app/consensus/README.md
View File

@@ -1,115 +1,13 @@
# Consensus Algorithm
# RCL Consensus
This directory holds the types and classes needed
to connect consensus to rippled.
to connect the generic consensus algorithm to the
rippled-specific instance of consensus.
## Types
* `RCLCxTx` adapts a `SHAMapItem` transaction.
* `RCLCxTxSet` adapts a `SHAMap` to represent a set of transactions.
* `RCLCxLedger` adapts a `Ledger`.
* `RCLConsensus` is implements the requirements of the generic
`Consensus` class by connecting to the rest of the `rippled`
application.
All types must be copy constructible and assignable.
* `LgrID_t`
Represents a ledger identifier.
Typically a 256-bit hash of the ledger header.
* `TxID_t`
Represents a transaction identifier.
Typically a 256-bit hash of the transaction data.
* `TxSetID_t`
Represents an identifier of a set of transactions.
Typically a 256-bit hash of the set's root tree node.
* `NodeID_t`
Represents an identifier for a node that can take positions during
the consenus process.
* `Time_t`
Encodes absolute times. Used for the close times of ledgers and the
expiration times of positions.
* `Pos_t`
Represents a position on a consensus taken by a participant.
Typically it encodes the previous ledger identifier, the transaction
set identifier, the participant, and a sequence number. It also includes
either the time it was signed or the time it was first seen. It may also
include additional information such as the participant's public key or
signature
* `Tx_t`
Represent a transaction. Has an identifier and also whatever information
is needed to add it to a set.
* `TxSet_t`
Represents a set of transactions. It has an identifier and can report
which transactions it has and provide the actual transaction data.
If non-const, it can be modified.
## `Pos_t`
Represents a position taken by a validator during a consensus round.
Must provide:
static std::uint32_t seqInitial;
static std::uint32_t seqLeave;
std::uint32_t getSequence() const;
Time_t getCloseTime() const;
Time_t getSeenTime() const;
bool isStale (Time_t) const;
NodeID_t getNodeID() const;
TxSetID_t getPosition() const;
LgrID_t getPrevLedger() const;
bool isInitial() const;
bool isBowOut() const;
Json::Value getJson() const;
bool changePosition (TxSetID_t const& position, Time_t closeTime, Time_t now);
bool bowOut (Time_t now);
### `Tx_t`
Represents a transaction.
Must provide:
TxID_t getID() const;
### TxSet_t
Represents a set of transactions.
Must provide:
TxSet_t (TxSet_t::mutable_t const&);
TxSetID_t getID() const;
bool hasEntry (TxID_t const&) const;
bool hasEntry (Tx_t const&) const;
boost::optional <Tx_t const> const getEntry (TxID_t const&) const;
std::map <TxID_t, bool> getDifferences(TxSet_t const&) const;
## TxSet_t::mutable_t
Represents a set of transactions that can be modified.
Must provide:
TxSet_t::mutable_t (TxSet_t const &);
bool addEntry (Tx_t const&);
bool removeEntry (TxID_t const&);

101
src/ripple/app/ledger/Consensus.h
View File

@@ -1,101 +0,0 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012, 2013 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_APP_LEDGER_CONSENSUS_H_INCLUDED
#define RIPPLE_APP_LEDGER_CONSENSUS_H_INCLUDED
#include <ripple/app/ledger/LedgerConsensus.h>
#include <ripple/app/ledger/LedgerMaster.h>
#include <ripple/app/ledger/InboundTransactions.h>
#include <ripple/app/consensus/RCLCxTraits.h>
#include <ripple/app/main/Application.h>
#include <ripple/basics/Log.h>
#include <ripple/core/Config.h>
#include <memory>
namespace ripple {
class LocalTxs;
/** Implements the consensus process and provides inter-round state. */
class Consensus
{
public:
using Proposals = hash_map <NodeID, std::deque<LedgerProposal::pointer>>;
virtual
~Consensus () = default;
/** Returns whether we are issuing proposals currently. */
virtual
bool
isProposing () const = 0;
/** Returns whether we are issuing validations currently. */
virtual
bool
isValidating () const = 0;
/** Returns the number of unique proposers we observed for the LCL. */
virtual
int
getLastCloseProposers () const = 0;
/** Returns the time (in milliseconds) that the last close took. */
virtual
std::chrono::milliseconds
getLastCloseDuration () const = 0;
/** Called to create a LedgerConsensus instance */
virtual
std::shared_ptr<LedgerConsensus<RCLCxTraits>>
makeLedgerConsensus (
Application& app,
InboundTransactions& inboundTransactions,
LedgerMaster& ledgerMaster,
LocalTxs& localTxs) = 0;
/** Called when a new round of consensus is about to begin */
virtual
void
startRound (
LedgerConsensus<RCLCxTraits>& consensus,
LedgerHash const &prevLCLHash,
std::shared_ptr<Ledger const> const& previousLedger,
NetClock::time_point closeTime) = 0;
/** Specified the network time when the last ledger closed */
virtual
void
setLastCloseTime (NetClock::time_point t) = 0;
virtual
void
storeProposal (
LedgerProposal::ref proposal,
NodeID const& nodeID) = 0;
};
std::unique_ptr<Consensus>
make_Consensus (Config const& config, Logs& logs);
}
#endif

2
src/ripple/app/ledger/Ledger.cpp
View File

@@ -22,7 +22,7 @@
#include <ripple/app/ledger/AcceptedLedger.h>
#include <ripple/app/ledger/InboundLedgers.h>
#include <ripple/app/ledger/LedgerMaster.h>
#include <ripple/app/ledger/LedgerTiming.h>
#include <ripple/consensus/LedgerTiming.h>
#include <ripple/app/ledger/LedgerToJson.h>
#include <ripple/app/ledger/OrderBookDB.h>
#include <ripple/app/ledger/PendingSaves.h>

92
src/ripple/app/ledger/LedgerConsensus.h
View File

@@ -1,92 +0,0 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012, 2013 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_APP_LEDGER_LEDGERCONSENSUS_H_INCLUDED
#define RIPPLE_APP_LEDGER_LEDGERCONSENSUS_H_INCLUDED
#include <ripple/app/ledger/Ledger.h>
#include <ripple/app/ledger/LedgerProposal.h>
#include <ripple/app/ledger/InboundTransactions.h>
#include <ripple/app/consensus/RCLCxTraits.h>
#include <ripple/app/main/Application.h>
#include <ripple/app/misc/CanonicalTXSet.h>
#include <ripple/app/misc/FeeVote.h>
#include <ripple/json/json_value.h>
#include <ripple/overlay/Peer.h>
#include <ripple/protocol/RippleLedgerHash.h>
#include <chrono>
namespace ripple {
/** Manager for achieving consensus on the next ledger.
*/
template <class Traits>
class LedgerConsensus : public Traits
{
public:
using typename Traits::Time_t;
using typename Traits::Pos_t;
using typename Traits::TxSet_t;
using typename Traits::Tx_t;
using typename Traits::LgrID_t;
using typename Traits::TxID_t;
using typename Traits::TxSetID_t;
using typename Traits::NodeID_t;
virtual ~LedgerConsensus() = default;
virtual Json::Value getJson (bool full) = 0;
virtual LgrID_t getLCL () = 0;
virtual void gotMap (TxSet_t const& map) = 0;
virtual void timerEntry () = 0;
virtual bool peerPosition (Pos_t const& position) = 0;
virtual PublicKey const& getValidationPublicKey () const = 0;
virtual void setValidationKeys (
SecretKey const& valSecret, PublicKey const& valPublic) = 0;
virtual void startRound (
LgrID_t const& prevLCLHash,
std::shared_ptr<Ledger const> const& prevLedger,
Time_t closeTime,
int previousProposers,
std::chrono::milliseconds previousConvergeTime) = 0;
/** Simulate the consensus process without any network traffic.
The end result, is that consensus begins and completes as if everyone
had agreed with whatever we propose.
This function is only called from the rpc "ledger_accept" path with the
server in standalone mode and SHOULD NOT be used during the normal
consensus process.
*/
virtual void simulate (
boost::optional<std::chrono::milliseconds> consensusDelay) = 0;
};
} // ripple
#endif

183
src/ripple/app/ledger/impl/ConsensusImp.cpp
View File

@@ -1,183 +0,0 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012, 2013 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <BeastConfig.h>
#include <ripple/app/ledger/LedgerTiming.h>
#include <ripple/app/ledger/impl/ConsensusImp.h>
#include <ripple/app/ledger/impl/LedgerConsensusImp.h>
namespace ripple {
ConsensusImp::ConsensusImp (
FeeVote::Setup const& voteSetup,
Logs& logs)
: journal_ (logs.journal("Consensus"))
, feeVote_ (make_FeeVote (voteSetup,
logs.journal("FeeVote")))
, proposing_ (false)
, validating_ (false)
, lastCloseProposers_ (0)
, lastCloseConvergeTook_ (LEDGER_IDLE_INTERVAL)
, lastValidationTimestamp_ (0s)
, lastCloseTime_ (0s)
{
}
bool
ConsensusImp::isProposing () const
{
return proposing_;
}
bool
ConsensusImp::isValidating () const
{
return validating_;
}
int
ConsensusImp::getLastCloseProposers () const
{
return lastCloseProposers_;
}
std::chrono::milliseconds
ConsensusImp::getLastCloseDuration () const
{
return lastCloseConvergeTook_;
}
std::shared_ptr<LedgerConsensus<RCLCxTraits>>
ConsensusImp::makeLedgerConsensus (
Application& app,
InboundTransactions& inboundTransactions,
LedgerMaster& ledgerMaster,
LocalTxs& localTxs)
{
return make_LedgerConsensus (app, *this,
inboundTransactions, localTxs, ledgerMaster, *feeVote_);
}
void
ConsensusImp::startRound (
LedgerConsensus<RCLCxTraits>& consensus,
LedgerHash const &prevLCLHash,
std::shared_ptr<Ledger const> const& previousLedger,
NetClock::time_point closeTime)
{
consensus.startRound (
prevLCLHash,
previousLedger,
closeTime,
lastCloseProposers_,
lastCloseConvergeTook_);
}
void
ConsensusImp::setProposing (bool p, bool v)
{
proposing_ = p;
validating_ = v;
}
STValidation::ref
ConsensusImp::getLastValidation () const
{
return lastValidation_;
}
void
ConsensusImp::setLastValidation (STValidation::ref v)
{
lastValidation_ = v;
}
void
ConsensusImp::newLCL (
int proposers,
std::chrono::milliseconds convergeTime)
{
lastCloseProposers_ = proposers;
lastCloseConvergeTook_ = convergeTime;
}
NetClock::time_point
ConsensusImp::validationTimestamp (NetClock::time_point vt)
{
if (vt <= lastValidationTimestamp_)
vt = lastValidationTimestamp_ + 1s;
lastValidationTimestamp_ = vt;
return vt;
}
NetClock::time_point
ConsensusImp::getLastCloseTime () const
{
return lastCloseTime_;
}
void
ConsensusImp::setLastCloseTime (NetClock::time_point t)
{
lastCloseTime_ = t;
}
void
ConsensusImp::storeProposal (
LedgerProposal::ref proposal,
NodeID const& nodeID)
{
std::lock_guard <std::mutex> _(lock_);
auto& props = storedProposals_[nodeID];
if (props.size () >= 10)
props.pop_front ();
props.push_back (proposal);
}
std::vector <RCLCxPos>
ConsensusImp::getStoredProposals (uint256 const& prevLedger)
{
std::vector <RCLCxPos> ret;
{
std::lock_guard <std::mutex> _(lock_);
for (auto const& it : storedProposals_)
for (auto const& prop : it.second)
if (prop->getPrevLedger() == prevLedger)
ret.emplace_back (*prop);
}
return ret;
}
std::unique_ptr <Consensus>
make_Consensus (Config const& config, Logs& logs)
{
return std::make_unique<ConsensusImp> (
setup_FeeVote (config.section ("voting")),
logs);
}
}

131
src/ripple/app/ledger/impl/ConsensusImp.h
View File

@@ -1,131 +0,0 @@
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012, 2013 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_APP_LEDGER_IMPL_CONSENSUSIMP_H_INCLUDED
#define RIPPLE_APP_LEDGER_IMPL_CONSENSUSIMP_H_INCLUDED
#include <BeastConfig.h>
#include <ripple/app/ledger/Consensus.h>
#include <ripple/app/ledger/LedgerConsensus.h>
#include <ripple/app/misc/FeeVote.h>
#include <ripple/basics/Log.h>
#include <ripple/protocol/STValidation.h>
#include <ripple/shamap/SHAMap.h>
#include <ripple/beast/utility/Journal.h>
namespace ripple {
/** Implements the consensus process and provides inter-round state. */
class ConsensusImp
: public Consensus
{
public:
ConsensusImp (FeeVote::Setup const& voteSetup, Logs& logs);
~ConsensusImp () = default;
bool
isProposing () const override;
bool
isValidating () const override;
int
getLastCloseProposers () const override;
std::chrono::milliseconds
getLastCloseDuration () const override;
std::shared_ptr<LedgerConsensus<RCLCxTraits>>
makeLedgerConsensus (
Application& app,
InboundTransactions& inboundTransactions,
LedgerMaster& ledgerMaster,
LocalTxs& localTxs) override;
void
startRound (
LedgerConsensus<RCLCxTraits>& ledgerConsensus,
LedgerHash const& prevLCLHash,
std::shared_ptr<Ledger const> const& previousLedger,
NetClock::time_point closeTime) override;
void
setLastCloseTime (NetClock::time_point t) override;
void
storeProposal (
LedgerProposal::ref proposal,
NodeID const& nodeID) override;
void
setProposing (bool p, bool v);
STValidation::ref
getLastValidation () const;
void
setLastValidation (STValidation::ref v);
void
newLCL (
int proposers,
std::chrono::milliseconds convergeTime);
NetClock::time_point
validationTimestamp (NetClock::time_point vt);
NetClock::time_point
getLastCloseTime () const;
std::vector <RCLCxPos>
getStoredProposals (uint256 const& previousLedger);
private:
beast::Journal journal_;
std::unique_ptr <FeeVote> feeVote_;
bool proposing_;
bool validating_;
// A pointer to the last validation that we issued
STValidation::pointer lastValidation_;
// The number of proposers who participated in the last ledger close
int lastCloseProposers_;
// How long the last ledger close took, in milliseconds
std::chrono::milliseconds lastCloseConvergeTook_;
// The timestamp of the last validation we used, in network time. This is
// only used for our own validations.
NetClock::time_point lastValidationTimestamp_;
// The last close time
NetClock::time_point lastCloseTime_;
Consensus::Proposals storedProposals_;
// lock to protect storedProposals_
std::mutex lock_;
};
}
#endif

1906
src/ripple/app/ledger/impl/LedgerConsensusImp.cpp
View File

File diff suppressed because it is too large Load Diff

402
src/ripple/app/ledger/impl/LedgerConsensusImp.h
View File

@@ -1,402 +0,0 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012, 2013 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_APP_LEDGER_IMPL_LEDGERCONSENSUSIMP_H_INCLUDED
#define RIPPLE_APP_LEDGER_IMPL_LEDGERCONSENSUSIMP_H_INCLUDED
#include <BeastConfig.h>
#include <ripple/app/ledger/LedgerMaster.h>
#include <ripple/app/ledger/impl/ConsensusImp.h>
#include <ripple/app/ledger/impl/DisputedTx.h>
#include <ripple/app/main/Application.h>
#include <ripple/app/misc/CanonicalTXSet.h>
#include <ripple/app/misc/FeeVote.h>
#include <ripple/basics/CountedObject.h>
#include <ripple/protocol/STValidation.h>
#include <ripple/protocol/UintTypes.h>
namespace ripple {
/**
Provides the implementation for LedgerConsensus.
Achieves consensus on the next ledger.
Two things need consensus:
1. The set of transactions.
2. The close time for the ledger.
*/
template <class Traits>
class LedgerConsensusImp
: public LedgerConsensus<Traits>
, public std::enable_shared_from_this <LedgerConsensusImp<Traits>>
, public CountedObject <LedgerConsensusImp<Traits>>
{
private:
enum class State
{
// We haven't closed our ledger yet, but others might have
open,
// Establishing consensus
establish,
// We have closed on a transaction set and are
// processing the new ledger
processing,
// We have accepted / validated a new last closed ledger
// and need to start a new round
accepted,
};
public:
using typename Traits::Time_t;
using typename Traits::Pos_t;
using typename Traits::TxSet_t;
using typename Traits::Tx_t;
using typename Traits::LgrID_t;
using typename Traits::TxID_t;
using typename Traits::TxSetID_t;
using typename Traits::NodeID_t;
using Dispute_t = DisputedTx <Traits>;
/**
* The result of applying a transaction to a ledger.
*/
enum {resultSuccess, resultFail, resultRetry};
static char const* getCountedObjectName () { return "LedgerConsensus"; }
LedgerConsensusImp(LedgerConsensusImp const&) = delete;
LedgerConsensusImp& operator=(LedgerConsensusImp const&) = delete;
~LedgerConsensusImp () = default;
/**
@param localtx transactions issued by local clients
@param inboundTransactions set of inbound transaction sets
@param localtx A set of local transactions to apply
@param feeVote Our desired fee levels and voting logic.
*/
LedgerConsensusImp (
Application& app,
ConsensusImp& consensus,
InboundTransactions& inboundTransactions,
LocalTxs& localtx,
LedgerMaster& ledgerMaster,
FeeVote& feeVote);
/**
@param prevLCLHash The hash of the Last Closed Ledger (LCL).
@param previousLedger Best guess of what the LCL was.
@param closeTime Closing time point of the LCL.
@param previousProposers the number of participants in the last round
@param previousConvergeTime how long the last round took (ms)
*/
void startRound (
LgrID_t const& prevLCLHash,
std::shared_ptr<Ledger const> const& prevLedger,
Time_t closeTime,
int previousProposers,
std::chrono::milliseconds previousConvergeTime) override;
/**
Get the Json state of the consensus process.
Called by the consensus_info RPC.
@param full True if verbose response desired.
@return The Json state.
*/
Json::Value getJson (bool full) override;
/* The hash of the last closed ledger */
LgrID_t getLCL () override;
/**
We have a complete transaction set, typically acquired from the network
@param map the transaction set.
*/
void gotMap (TxSet_t const& map) override;
/**
On timer call the correct handler for each state.
*/
void timerEntry () override;
/**
A server has taken a new position, adjust our tracking
Called when a peer takes a new postion.
@param newPosition the new position
@return true if we should do delayed relay of this position.
*/
bool peerPosition (Pos_t const& newPosition) override;
void simulate(
boost::optional<std::chrono::milliseconds> consensusDelay) override;
/**
Put a transaction set where peers can find it
*/
void shareSet (TxSet_t const&);
private:
/**
Handle pre-close state.
*/
void statePreClose ();
/** We are establishing a consensus
Update our position only on the timer, and in this state.
If we have consensus, move to the finish state
*/
void stateEstablish ();
/** Check if we've reached consensus */
bool haveConsensus ();
/**
Check if our last closed ledger matches the network's.
This tells us if we are still in sync with the network.
This also helps us if we enter the consensus round with
the wrong ledger, to leave it with the correct ledger so
that we can participate in the next round.
*/
void checkLCL ();
/**
Change our view of the last closed ledger
@param lclHash Hash of the last closed ledger.
*/
void handleLCL (LgrID_t const& lclHash);
/**
We have a complete transaction set, typically acquired from the network
@param map the transaction set.
@param acquired true if we have acquired the transaction set.
*/
void mapCompleteInternal (
TxSet_t const& map,
bool acquired);
/** We have a new last closed ledger, process it. Final accept logic
@param set Our consensus set
*/
void accept (TxSet_t const& set);
/**
Compare two proposed transaction sets and create disputed
transctions structures for any mismatches
@param m1 One transaction set
@param m2 The other transaction set
*/
void createDisputes (TxSet_t const& m1,
TxSet_t const& m2);
/**
Add a disputed transaction (one that at least one node wants
in the consensus set and at least one node does not) to our tracking
@param tx The disputed transaction
*/
void addDisputedTransaction (Tx_t const& tx);
/**
Adjust the votes on all disputed transactions based
on the set of peers taking this position
@param map A disputed position
@param peers peers which are taking the position map
*/
void adjustCount (TxSet_t const& map,
std::vector<NodeID_t> const& peers);
/**
Revoke our outstanding proposal, if any, and
cease proposing at least until this round ends
*/
void leaveConsensus ();
/** Make and send a proposal
*/
void propose ();
/** Send a node status change message to our directly connected peers
@param event The event which caused the status change. This is
typically neACCEPTED_LEDGER or neCLOSING_LEDGER.
@param ledger The ledger associated with the event.
*/
void statusChange (protocol::NodeEvent event, ReadView const& ledger);
/** Determine our initial proposed transaction set based on
our open ledger
*/
std::pair <TxSet_t, Pos_t> makeInitialPosition();
/** Take an initial position on what we think the consensus set should be
*/
void takeInitialPosition ();
/**
Called while trying to avalanche towards consensus.
Adjusts our positions to try to agree with other validators.
*/
void updateOurPositions ();
/** If we radically changed our consensus context for some reason,
we need to replay recent proposals so that they're not lost.
*/
void playbackProposals ();
/** We have just decided to close the ledger. Start the consensus timer,
stash the close time, inform peers, and take a position
*/
void closeLedger ();
/**
If we missed a consensus round, we may be missing a validation.
This will send an older owed validation if we previously missed it.
*/
void checkOurValidation ();
/** We have a new LCL and must accept it */
void beginAccept (bool synchronous);
void endConsensus (bool correctLCL);
/** Add our load fee to our validation */
void addLoad(STValidation::ref val);
/** Convert an advertised close time to an effective close time */
NetClock::time_point effectiveCloseTime(NetClock::time_point closeTime);
private:
Application& app_;
ConsensusImp& consensus_;
InboundTransactions& inboundTransactions_;
LocalTxs& localTX_;
LedgerMaster& ledgerMaster_;
FeeVote& feeVote_;
std::recursive_mutex lock_;
NodeID_t ourID_;
State state_;
// The wall time this ledger closed
Time_t closeTime_;
LgrID_t prevLedgerHash_;
LgrID_t acquiringLedger_;
std::shared_ptr<Ledger const> previousLedger_;
boost::optional<Pos_t> ourPosition_;
boost::optional<TxSet_t> ourSet_;
PublicKey valPublic_;
SecretKey valSecret_;
bool proposing_, validating_, haveCorrectLCL_, consensusFail_;
// How much time has elapsed since the round started
std::chrono::milliseconds roundTime_;
// How long the close has taken, expressed as a percentage of the time that
// we expected it to take.
int closePercent_;
NetClock::duration closeResolution_;
bool haveCloseTimeConsensus_;
std::chrono::steady_clock::time_point consensusStartTime_;
int previousProposers_;
// Time it took for the last consensus round to converge
std::chrono::milliseconds previousRoundTime_;
// Convergence tracking, trusted peers indexed by hash of public key
hash_map<NodeID_t, Pos_t> peerPositions_;
// Transaction Sets, indexed by hash of transaction tree
hash_map<TxSetID_t, const TxSet_t> acquired_;
// Disputed transactions
hash_map<TxID_t, Dispute_t> disputes_;
hash_set<TxSetID_t> compares_;
// Close time estimates, keep ordered for predictable traverse
std::map <Time_t, int> closeTimes_;
// nodes that have bowed out of this consensus process
hash_set<NodeID_t> deadNodes_;
beast::Journal j_;
public:
/** Returns validation public key */
PublicKey const&
getValidationPublicKey () const override;
/** Set validation private and public key pair. */
void
setValidationKeys (
SecretKey const& valSecret, PublicKey const& valPublic) override;
};
//------------------------------------------------------------------------------
std::shared_ptr <LedgerConsensus <RCLCxTraits>>
make_LedgerConsensus (
Application& app,
ConsensusImp& consensus,
InboundTransactions& inboundTransactions,
LocalTxs& localtx,
LedgerMaster& ledgerMaster,
FeeVote& feeVote);
//------------------------------------------------------------------------------
/** Apply a set of transactions to a ledger
Typically the txFilter is used to reject transactions
that already got in the prior ledger
@param set set of transactions to apply
@param view ledger to apply to
@param txFilter callback, return false to reject txn
@return retriable transactions
*/
CanonicalTXSet
applyTransactions (
Application& app,
RCLTxSet const& set,
OpenView& view,
std::function<bool(uint256 const&)> txFilter);
extern template class LedgerConsensusImp <RCLCxTraits>;
} // ripple
#endif

4
src/ripple/app/ledger/impl/LedgerMaster.cpp
View File

@@ -828,6 +828,8 @@ LedgerMaster::consensusBuilt(
if (standalone_)
return;
mLedgerHistory.builtLedger (ledger, std::move (consensus));
if (ledger->info().seq <= mValidLedgerSeq)
{
auto stream = app_.journal ("LedgerConsensus").info();
@@ -912,8 +914,6 @@ LedgerMaster::consensusBuilt(
<< "Consensus triggered check of ledger";
checkAccept (maxLedger, maxSeq);
}
mLedgerHistory.builtLedger (ledger, std::move (consensus));
}
void

3
src/ripple/app/main/Application.cpp
View File

@@ -54,6 +54,7 @@
#include <ripple/resource/Fees.h>
#include <ripple/beast/asio/io_latency_probe.h>
#include <ripple/beast/core/LexicalCast.h>
#include <fstream>
namespace ripple {
@@ -1338,7 +1339,6 @@ ApplicationImp::startGenesisLedger()
*genesis, timeKeeper().closeTime());
next->updateSkipList ();
next->setImmutable (*config_);
m_networkOPs->setLastCloseTime (next->info().closeTime);
openLedger_.emplace(next, cachedSLEs_,
logs_->journal("OpenLedger"));
m_ledgerMaster->storeLedger(next);
@@ -1648,7 +1648,6 @@ bool ApplicationImp::loadOldLedger (
m_ledgerMaster->switchLCL (loadLedger);
loadLedger->setValidated();
m_ledgerMaster->setFullLedger(loadLedger, true, false);
m_networkOPs->setLastCloseTime (loadLedger->info().closeTime);
openLedger_.emplace(loadLedger, cachedSLEs_,
logs_->journal("OpenLedger"));

81
src/ripple/app/misc/NetworkOPs.cpp
View File

@@ -21,15 +21,14 @@
#include <ripple/app/misc/NetworkOPs.h>
#include <ripple/protocol/Quality.h>
#include <ripple/core/DatabaseCon.h>
#include <ripple/consensus/Consensus.h>
#include <ripple/app/main/Application.h>
#include <ripple/app/consensus/RCLCxTraits.h>
#include <ripple/app/ledger/Consensus.h>
#include <ripple/app/ledger/LedgerConsensus.h>
#include <ripple/app/consensus/RCLConsensus.h>
#include <ripple/app/ledger/AcceptedLedger.h>
#include <ripple/app/ledger/InboundLedger.h>
#include <ripple/app/ledger/InboundLedgers.h>
#include <ripple/app/ledger/LedgerMaster.h>
#include <ripple/app/ledger/LedgerTiming.h>
#include <ripple/consensus/LedgerTiming.h>
#include <ripple/app/ledger/LedgerToJson.h>
#include <ripple/app/ledger/LocalTxs.h>
#include <ripple/app/ledger/OpenLedger.h>
@@ -222,9 +221,14 @@ public:
, m_amendmentBlocked (false)
, m_heartbeatTimer (this)
, m_clusterTimer (this)
, mConsensus (make_Consensus (app_.config(), app_.logs()))
, mLedgerConsensus (mConsensus->makeLedgerConsensus (
app, app.getInboundTransactions(), ledgerMaster, *m_localTX))
, mConsensus (std::make_shared<RCLConsensus>(app,
make_FeeVote(setup_FeeVote (app_.config().section ("voting")),
app_.logs().journal("FeeVote")),
ledgerMaster,
*m_localTX,
app.getInboundTransactions(),
stopwatch(),
app_.logs().journal("LedgerConsensus")))
, m_ledgerMaster (ledgerMaster)
, m_job_queue (job_queue)
, m_standalone (standalone)
@@ -308,7 +312,7 @@ public:
// Ledger proposal/close functions.
void processTrustedProposal (
LedgerProposal::pointer proposal,
RCLCxPeerPos::pointer proposal,
std::shared_ptr<protocol::TMProposeSet> set,
NodeID const &node) override;
@@ -372,18 +376,14 @@ public:
}
void setAmendmentBlocked () override;
void consensusViewChange () override;
void setLastCloseTime (NetClock::time_point t) override
{
mConsensus->setLastCloseTime(t);
}
PublicKey const& getValidationPublicKey () const override
{
return mLedgerConsensus->getValidationPublicKey ();
return mConsensus->getValidationPublicKey ();
}
void setValidationKeys (
SecretKey const& valSecret, PublicKey const& valPublic) override
{
mLedgerConsensus->setValidationKeys (valSecret, valPublic);
mConsensus->setValidationKeys (valSecret, valPublic);
}
Json::Value getConsensusInfo () override;
Json::Value getServerInfo (bool human, bool admin) override;
@@ -560,8 +560,7 @@ private:
DeadlineTimer m_heartbeatTimer;
DeadlineTimer m_clusterTimer;
std::unique_ptr<Consensus> mConsensus;
std::shared_ptr<LedgerConsensus<RCLCxTraits>> mLedgerConsensus;
std::shared_ptr<RCLConsensus> mConsensus;
LedgerMaster& m_ledgerMaster;
std::shared_ptr<InboundLedger> mAcquiringLedger;
@@ -700,7 +699,8 @@ void NetworkOPsImp::processHeartbeatTimer ()
<< "Node count (" << numPeers << ") "
<< "has fallen below quorum (" << m_network_quorum << ").";
}
// We do not call mConsensus->timerEntry until there
// are enough peers providing meaningful inputs to consensus
setHeartbeatTimer ();
return;
@@ -722,7 +722,7 @@ void NetworkOPsImp::processHeartbeatTimer ()
}
mLedgerConsensus->timerEntry ();
mConsensus->timerEntry (app_.timeKeeper().closeTime());
setHeartbeatTimer ();
}
@@ -776,12 +776,12 @@ void NetworkOPsImp::processClusterTimer ()
std::string NetworkOPsImp::strOperatingMode () const
{
if (mMode == omFULL)
if (mMode == omFULL && mConsensus->haveCorrectLCL())
{
if (mConsensus->isProposing ())
if (mConsensus->proposing ())
return "proposing";
if (mConsensus->isValidating ())
if (mConsensus->validating ())
return "validating";
}
@@ -1282,8 +1282,7 @@ void NetworkOPsImp::tryStartConsensus ()
}
}
if (mMode != omDISCONNECTED)
beginConsensus (networkClosed);
beginConsensus (networkClosed);
}
bool NetworkOPsImp::checkLastClosedLedger (
@@ -1522,10 +1521,9 @@ bool NetworkOPsImp::beginConsensus (uint256 const& networkClosed)
app_.getValidations().getCurrentPublicKeys ());
mConsensus->startRound (
*mLedgerConsensus,
app_.timeKeeper().closeTime(),
networkClosed,
prevLedger,
closingInfo.closeTime);
prevLedger);
JLOG(m_journal.debug()) << "Initiating consensus engine";
return true;
@@ -1533,18 +1531,19 @@ bool NetworkOPsImp::beginConsensus (uint256 const& networkClosed)
uint256 NetworkOPsImp::getConsensusLCL ()
{
return mLedgerConsensus->getLCL ();
return mConsensus->LCL ();
}
void NetworkOPsImp::processTrustedProposal (
LedgerProposal::pointer proposal,
RCLCxPeerPos::pointer peerPos,
std::shared_ptr<protocol::TMProposeSet> set,
NodeID const& node)
{
mConsensus->storeProposal (proposal, node);
mConsensus->storeProposal (peerPos, node);
if (mLedgerConsensus->peerPosition (*proposal))
app_.overlay().relay(*set, proposal->getSuppressionID());
if (mConsensus->peerProposal (
app_.timeKeeper().closeTime(), peerPos->proposal()))
app_.overlay().relay(*set, peerPos->getSuppressionID());
else
JLOG(m_journal.info()) << "Not relaying trusted proposal";
}
@@ -1567,7 +1566,9 @@ NetworkOPsImp::mapComplete (
// We acquired it because consensus asked us to
if (fromAcquire)
mLedgerConsensus->gotMap (RCLTxSet{map});
mConsensus->gotTxSet (
app_.timeKeeper().closeTime(),
RCLTxSet{map});
}
void NetworkOPsImp::endConsensus (bool correctLCL)
@@ -1647,7 +1648,7 @@ NetworkOPsImp::ServerFeeSummary::operator !=(NetworkOPsImp::ServerFeeSummary con
em.is_initialized() != b.em.is_initialized())
return true;
if(em)
if(em && b.em)
{
return (em->minFeeLevel != b.em->minFeeLevel ||
em->expFeeLevel != b.em->expFeeLevel ||
@@ -2113,7 +2114,7 @@ bool NetworkOPsImp::recvValidation (
Json::Value NetworkOPsImp::getConsensusInfo ()
{
return mLedgerConsensus->getJson (true);
return mConsensus->getJson (true);
}
Json::Value NetworkOPsImp::getServerInfo (bool human, bool admin)
@@ -2175,17 +2176,17 @@ Json::Value NetworkOPsImp::getServerInfo (bool human, bool admin)
{
lastClose[jss::converge_time_s] =
std::chrono::duration<double>{
mConsensus->getLastCloseDuration()}.count();
mConsensus->getLastConvergeDuration()}.count();
}
else
{
lastClose[jss::converge_time] =
Json::Int (mConsensus->getLastCloseDuration().count());
Json::Int (mConsensus->getLastConvergeDuration().count());
}
info[jss::last_close] = lastClose;
// info[jss::consensus] = mLedgerConsensus->getJson();
// info[jss::consensus] = mConsensus->getJson();
if (admin)
info[jss::load] = m_job_queue.getJson ();
@@ -2757,9 +2758,11 @@ std::uint32_t NetworkOPsImp::acceptLedger (
Throw<std::runtime_error> ("Operation only possible in STANDALONE mode.");
// FIXME Could we improve on this and remove the need for a specialized
// API in LedgerConsensus?
// API in Consensus?
beginConsensus (m_ledgerMaster.getClosedLedger()->info().hash);
mLedgerConsensus->simulate (consensusDelay);
mConsensus->simulate (
app_.timeKeeper().closeTime(),
consensusDelay);
return m_ledgerMaster.getCurrentLedger ()->info().seq;
}

7
src/ripple/app/misc/NetworkOPs.h
View File

@@ -23,7 +23,7 @@
#include <ripple/core/JobQueue.h>
#include <ripple/protocol/STValidation.h>
#include <ripple/app/ledger/Ledger.h>
#include <ripple/app/ledger/LedgerProposal.h>
#include <ripple/app/consensus/RCLCxPeerPos.h>
#include <ripple/ledger/ReadView.h>
#include <ripple/net/InfoSub.h>
#include <memory>
@@ -150,7 +150,7 @@ public:
//--------------------------------------------------------------------------
// ledger proposal/close functions
virtual void processTrustedProposal (LedgerProposal::pointer proposal,
virtual void processTrustedProposal (RCLCxPeerPos::pointer peerPos,
std::shared_ptr<protocol::TMProposeSet> set,
NodeID const& node) = 0;
@@ -174,9 +174,6 @@ public:
virtual bool isAmendmentBlocked () = 0;
virtual void setAmendmentBlocked () = 0;
virtual void consensusViewChange () = 0;
// FIXME(NIKB): Remove the need for this function
virtual void setLastCloseTime (NetClock::time_point t) = 0;
virtual PublicKey const& getValidationPublicKey () const = 0;
virtual void setValidationKeys (
SecretKey const& valSecret, PublicKey const& valPublic) = 0;

2
src/ripple/app/misc/Validations.cpp
View File

@@ -21,7 +21,7 @@
#include <ripple/app/misc/Validations.h>
#include <ripple/core/DatabaseCon.h>
#include <ripple/app/ledger/LedgerMaster.h>
#include <ripple/app/ledger/LedgerTiming.h>
#include <ripple/consensus/LedgerTiming.h>
#include <ripple/app/main/Application.h>
#include <ripple/app/misc/NetworkOPs.h>
#include <ripple/app/misc/ValidatorList.h>

32
src/ripple/beast/hash/hash_append.h
View File

@@ -39,6 +39,7 @@
#include <type_traits>
#include <utility>
#include <vector>
#include <boost/container/flat_set.hpp>
namespace beast {
@@ -324,6 +325,18 @@ template <class Hasher, class Key, class Hash, class Pred, class Alloc>
void
hash_append(Hasher& h, std::unordered_set<Key, Hash, Pred, Alloc> const& s);
template <class Hasher, class Key, class Compare, class Alloc>
std::enable_if_t
<
!is_contiguously_hashable<Key, Hasher>::value
>
hash_append(Hasher& h, boost::container::flat_set<Key, Compare, Alloc> const& v) noexcept;
template <class Hasher, class Key, class Compare, class Alloc>
std::enable_if_t
<
is_contiguously_hashable<Key, Hasher>::value
>
hash_append(Hasher& h, boost::container::flat_set<Key, Compare, Alloc> const& v) noexcept;
template <class Hasher, class T0, class T1, class ...T>
void
hash_append (Hasher& h, T0 const& t0, T1 const& t1, T const& ...t) noexcept;
@@ -421,6 +434,25 @@ hash_append(Hasher& h, std::array<T, N> const& a) noexcept
hash_append(h, t);
}
template <class Hasher, class Key, class Compare, class Alloc>
std::enable_if_t
<
!is_contiguously_hashable<Key, Hasher>::value
>
hash_append(Hasher& h, boost::container::flat_set<Key, Compare, Alloc> const& v) noexcept
{
for (auto const& t : v)
hash_append(h, t);
}
template <class Hasher, class Key, class Compare, class Alloc>
std::enable_if_t
<
is_contiguously_hashable<Key, Hasher>::value
>
hash_append(Hasher& h, boost::container::flat_set<Key, Compare, Alloc> const& v) noexcept
{
h(&(v.begin()), v.size()*sizeof(Key));
}
// tuple
namespace detail

1791
src/ripple/consensus/Consensus.h Normal file
View File

File diff suppressed because it is too large Load Diff

262
src/ripple/consensus/ConsensusProposal.h Normal file
View File

@@ -0,0 +1,262 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2016 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVID_tED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_CONSENSUS_ConsensusProposal_H_INCLUDED
#define RIPPLE_CONSENSUS_ConsensusProposal_H_INCLUDED
#include <cstdint>
#include <ripple/json/json_value.h>
#include <ripple/protocol/JsonFields.h>
#include <ripple/basics/chrono.h>
namespace ripple
{
/** Represents a proposed position taken during a round of consensus.
During consensus, peers seek agreement on a set of transactions to
apply to the prior ledger to generate the next ledger. Each peer takes a
position on whether to include or exclude potential transactions.
The position on the set of transactions is proposed to its peers as an
instance of the ConsensusProposal class.
An instance of ConsensusProposal can be either our own proposal or one of
our peer's.
As consensus proceeds, peers may change their position on the transaction,
or choose to abstain. Each successive proposal includes a strictly
monotonically increasing number (or, if a peer is choosing to abstain,
the special value `seqLeave`).
Refer to @ref Consensus for requirements of the template arguments.
@tparam NodeID_t Type used to uniquely identify nodes/peers
@tparam LedgerID_t Type used to uniquely identify ledgers
@tparam Position_t Type used to represent the position taken on transactions
under consideration during this round of consensus
*/
template <
class NodeID_t,
class LedgerID_t,
class Position_t>
class ConsensusProposal
{
public:
using NodeID = NodeID_t;
//< Sequence value when a peer initially joins consensus
static std::uint32_t const seqJoin = 0;
//< Sequence number when a peer wants to bow out and leave consensus
static std::uint32_t const seqLeave = 0xffffffff;
/** Constructor
@param prevLedger The previous ledger this proposal is building on.
@param seq The sequence number of this proposal.
@param position The position taken on transactions in this round.
@param closeTime Position of when this ledger closed.
@param now Time when the proposal was taken.
@param nodeID ID of node/peer taking this position.
*/
ConsensusProposal(
LedgerID_t const& prevLedger,
std::uint32_t seq,
Position_t const& position,
NetClock::time_point closeTime,
NetClock::time_point now,
NodeID_t const& nodeID)
: previousLedger_(prevLedger)
, position_(position)
, closeTime_(closeTime)
, time_(now)
, proposeSeq_(seq)
, nodeID_(nodeID)
{
}
//! Identifying which peer took this position.
NodeID_t const&
nodeID () const
{
return nodeID_;
}
//! Get the proposed position.
Position_t const&
position () const
{
return position_;
}
//! Get the prior accepted ledger this position is based on.
LedgerID_t const&
prevLedger () const
{
return previousLedger_;
}
/** Get the sequence number of this proposal
Starting with an initial sequence number of `seqJoin`, successive
proposals from a peer will increase the sequence number.
@return the sequence number
*/
std::uint32_t
proposeSeq () const
{
return proposeSeq_;
}
//! The current position on the consensus close time.
NetClock::time_point const &
closeTime () const
{
return closeTime_;
}
//! Get when this position was taken.
NetClock::time_point const &
seenTime () const
{
return time_;
}
/** Whether this is the first position taken during the current
consensus round.
*/
bool
isInitial () const
{
return proposeSeq_ == seqJoin;
}
//! Get whether this node left the consensus process
bool
isBowOut () const
{
return proposeSeq_ == seqLeave;
}
//! Get whether this position is stale relative to the provided cutoff
bool
isStale (NetClock::time_point cutoff) const
{
return time_ <= cutoff;
}
/** Update the position during the consensus process. This will increment
the proposal's sequence number.
@param newPosition The new position taken.
@param newCloseTime The new close time.
@param now the time The new position was taken.
@return `true` if the position was updated or `false` if this node has
already left this consensus round.
*/
bool
changePosition(
Position_t const& newPosition,
NetClock::time_point newCloseTime,
NetClock::time_point now)
{
if (proposeSeq_ == seqLeave)
return false;
position_ = newPosition;
closeTime_ = newCloseTime;
time_ = now;
++proposeSeq_;
return true;
}
/** Leave consensus
Update position to indicate the node left consensus.
@param now Time when this node left consensus.
*/
void
bowOut(NetClock::time_point now)
{
time_ = now;
proposeSeq_ = seqLeave;
}
//! Get JSON representation for debugging
Json::Value
getJson () const
{
using std::to_string;
Json::Value ret = Json::objectValue;
ret[jss::previous_ledger] = to_string (prevLedger());
if (!isBowOut())
{
ret[jss::transaction_hash] = to_string (position());
ret[jss::propose_seq] = proposeSeq();
}
ret[jss::close_time] = to_string(closeTime().time_since_epoch().count());
return ret;
}
private:
//! Unique identifier of prior ledger this proposal is based on
LedgerID_t previousLedger_;
//! Unique identifier of the position this proposal is taking
Position_t position_;
//! The ledger close time this position is taking
NetClock::time_point closeTime_;
// !The time this position was last updated
NetClock::time_point time_;
//! The sequence number of these positions taken by this node
std::uint32_t proposeSeq_;
//! The identifier of the node taking this position
NodeID_t nodeID_;
};
template <class NodeID_t,
class LedgerID_t,
class Position_t>
bool
operator==(ConsensusProposal<NodeID_t, LedgerID_t, Position_t> const & a,
ConsensusProposal<NodeID_t, LedgerID_t, Position_t> const & b)
{
return a.nodeID() == b.nodeID() &&
a.proposeSeq() == b.proposeSeq() &&
a.prevLedger() == b.prevLedger() &&
a.position() == b.position() &&
a.closeTime() == b.closeTime() &&
a.seenTime() == b.seenTime();
}
}
#endif

182
src/ripple/app/ledger/impl/DisputedTx.h → src/ripple/consensus/DisputedTx.h
View File

@@ -24,6 +24,8 @@
#include <ripple/protocol/Serializer.h>
#include <ripple/basics/base_uint.h>
#include <ripple/beast/utility/Journal.h>
#include <ripple/consensus/LedgerTiming.h>
#include <ripple/basics/Log.h>
#include <memory>
namespace ripple {
@@ -35,70 +37,110 @@ namespace ripple {
the dispute.
Undisputed transactions have no corresponding @ref DisputedTx object.
Refer to @ref Consensus for details on the template type requirements.
@tparam Tx_t The type for a transaction
@tparam NodeID_t The type for a node identifier
*/
template <class Traits>
template <class Tx_t, class NodeID_t>
class DisputedTx
{
using TxID_t = typename Tx_t::ID;
public:
/** Constructor
using Tx_t = typename Traits::Tx_t;
using TxID_t = typename Traits::TxID_t;
using NodeID_t = typename Traits::NodeID_t;
@param tx The transaction under dispute
@param ourVote Our vote on whether tx should be included
@param j Journal for debugging
*/
DisputedTx (Tx_t const& tx,
bool ourVote, beast::Journal j)
: mTransactionID (tx.getID())
, mYays (0)
, mNays (0)
, mOurVote (ourVote)
, transaction (tx)
bool ourVote,
beast::Journal j)
: yays_ (0)
, nays_ (0)
, ourVote_ (ourVote)
, tx_ (tx)
, j_ (j)
{
}
TxID_t const& getID () const
//! The unique id/hash of the disputed transaction.
TxID_t
const& ID () const
{
return mTransactionID;
return tx_.id();
}
bool getOurVote () const
//! Our vote on whether the transaction should be included.
bool
getOurVote () const
{
return mOurVote;
return ourVote_;
}
Tx_t const& tx () const
//! The disputed transaction.
Tx_t
const& tx () const
{
return transaction;
return tx_;
}
void setOurVote (bool o)
//! Change our vote
void
setOurVote (bool o)
{
mOurVote = o;
ourVote_ = o;
}
void setVote (NodeID_t const& peer, bool votesYes);
void unVote (NodeID_t const& peer);
/** Change a peer's vote
bool updateVote (int percentTime, bool proposing);
Json::Value getJson ();
@param peer Identifier of peer.
@param votesYes Whether peer votes to include the disputed transaction.
*/
void
setVote (NodeID_t const& peer, bool votesYes);
/** Remove a peer's vote
@param peer Identifier of peer.
*/
void
unVote (NodeID_t const& peer);
/** Update our vote given progression of consensus.
Updates our vote on this disputed transaction based on our peers' votes
and how far along consensus has proceeded.
@param percentTime Percentage progress through consensus, e.g. 50%
through or 90%.
@param proposing Whether we are proposing to our peers in this round.
@return Whether our vote changed
*/
bool
updateVote (int percentTime, bool proposing);
//! JSON representation of dispute, used for debugging
Json::Value
getJson () const;
private:
TxID_t mTransactionID;
int mYays;
int mNays;
bool mOurVote;
Tx_t transaction;
int yays_; //< Number of yes votes
int nays_; //< Number of no votes
bool ourVote_; //< Our vote (true is yes)
Tx_t tx_; //< Transaction under dispute
hash_map <NodeID_t, bool> mVotes;
beast::Journal j_;
hash_map <NodeID_t, bool> votes_; //< Votes of our peers
beast::Journal j_; //< Debug journal
};
// Track a peer's yes/no vote on a particular disputed transaction
template <class Traits>
void DisputedTx<Traits>::setVote (NodeID_t const& peer, bool votesYes)
// Track a peer's yes/no vote on a particular disputed tx_
template <class Tx_t, class NodeID_t>
void DisputedTx<Tx_t, NodeID_t>::setVote (NodeID_t const& peer, bool votesYes)
{
auto res = mVotes.insert (std::make_pair (peer, votesYes));
auto res = votes_.insert (std::make_pair (peer, votesYes));
// new vote
if (res.second)
@@ -106,60 +148,60 @@ void DisputedTx<Traits>::setVote (NodeID_t const& peer, bool votesYes)
if (votesYes)
{
JLOG (j_.debug())
<< "Peer " << peer << " votes YES on " << mTransactionID;
++mYays;
<< "Peer " << peer << " votes YES on " << tx_.id();
++yays_;
}
else
{
JLOG (j_.debug())
<< "Peer " << peer << " votes NO on " << mTransactionID;
++mNays;
<< "Peer " << peer << " votes NO on " << tx_.id();
++nays_;
}
}
// changes vote to yes
else if (votesYes && !res.first->second)
{
JLOG (j_.debug())
<< "Peer " << peer << " now votes YES on " << mTransactionID;
--mNays;
++mYays;
<< "Peer " << peer << " now votes YES on " << tx_.id();
--nays_;
++yays_;
res.first->second = true;
}
// changes vote to no
else if (!votesYes && res.first->second)
{
JLOG (j_.debug())
<< "Peer " << peer << " now votes NO on " << mTransactionID;
++mNays;
--mYays;
<< "Peer " << peer << " now votes NO on " << tx_.id();
++nays_;
--yays_;
res.first->second = false;
}
}
// Remove a peer's vote on this disputed transasction
template <class Traits>
void DisputedTx<Traits>::unVote (NodeID_t const& peer)
template <class Tx_t, class NodeID_t>
void DisputedTx<Tx_t, NodeID_t>::unVote (NodeID_t const& peer)
{
auto it = mVotes.find (peer);
auto it = votes_.find (peer);
if (it != mVotes.end ())
if (it != votes_.end ())
{
if (it->second)
--mYays;
--yays_;
else
--mNays;
--nays_;
mVotes.erase (it);
votes_.erase (it);
}
}
template <class Traits>
bool DisputedTx<Traits>::updateVote (int percentTime, bool proposing)
template <class Tx_t, class NodeID_t>
bool DisputedTx<Tx_t, NodeID_t>::updateVote (int percentTime, bool proposing)
{
if (mOurVote && (mNays == 0))
if (ourVote_ && (nays_ == 0))
return false;
if (!mOurVote && (mYays == 0))
if (!ourVote_ && (yays_ == 0))
return false;
bool newPosition;
@@ -168,7 +210,7 @@ bool DisputedTx<Traits>::updateVote (int percentTime, bool proposing)
if (proposing) // give ourselves full weight
{
// This is basically the percentage of nodes voting 'yes' (including us)
weight = (mYays * 100 + (mOurVote ? 100 : 0)) / (mNays + mYays + 1);
weight = (yays_ * 100 + (ourVote_ ? 100 : 0)) / (nays_ + yays_ + 1);
// VFALCO TODO Rename these macros and turn them into language
// constructs. consolidate them into a class that collects
@@ -189,39 +231,41 @@ bool DisputedTx<Traits>::updateVote (int percentTime, bool proposing)
{
// don't let us outweigh a proposing node, just recognize consensus
weight = -1;
newPosition = mYays > mNays;
newPosition = yays_ > nays_;
}
if (newPosition == mOurVote)
if (newPosition == ourVote_)
{
JLOG (j_.info())
<< "No change (" << (mOurVote ? "YES" : "NO") << ") : weight "
<< "No change (" << (ourVote_ ? "YES" : "NO") << ") : weight "
<< weight << ", percent " << percentTime;
JLOG (j_.debug()) << getJson ();
return false;
}
mOurVote = newPosition;
ourVote_ = newPosition;
JLOG (j_.debug())
<< "We now vote " << (mOurVote ? "YES" : "NO")
<< " on " << mTransactionID;
<< "We now vote " << (ourVote_ ? "YES" : "NO")
<< " on " << tx_.id();
JLOG (j_.debug()) << getJson ();
return true;
}
template <class Traits>
Json::Value DisputedTx<Traits>::getJson ()
template <class Tx_t, class NodeID_t>
Json::Value DisputedTx<Tx_t, NodeID_t>::getJson () const
{
using std::to_string;
Json::Value ret (Json::objectValue);
ret["yays"] = mYays;
ret["nays"] = mNays;
ret["our_vote"] = mOurVote;
ret["yays"] = yays_;
ret["nays"] = nays_;
ret["our_vote"] = ourVote_;
if (!mVotes.empty ())
if (!votes_.empty ())
{
Json::Value votesj (Json::objectValue);
for (auto& vote : mVotes)
for (auto& vote : votes_)
votesj[to_string (vote.first)] = vote.second;
ret["votes"] = std::move (votesj);
}

70
src/ripple/app/ledger/impl/LedgerTiming.cpp → src/ripple/consensus/LedgerTiming.cpp
View File

@@ -18,68 +18,19 @@
//==============================================================================
#include <BeastConfig.h>
#include <ripple/app/ledger/LedgerTiming.h>
#include <ripple/consensus/LedgerTiming.h>
#include <ripple/basics/Log.h>
#include <algorithm>
#include <iterator>
namespace ripple {
NetClock::duration
getNextLedgerTimeResolution (
NetClock::duration previousResolution,
bool previousAgree,
std::uint32_t ledgerSeq)
{
assert (ledgerSeq);
using namespace std::chrono;
// Find the current resolution:
auto iter = std::find (std::begin (ledgerPossibleTimeResolutions),
std::end (ledgerPossibleTimeResolutions), previousResolution);
assert (iter != std::end (ledgerPossibleTimeResolutions));
// This should never happen, but just as a precaution
if (iter == std::end (ledgerPossibleTimeResolutions))
return previousResolution;
// If we did not previously agree, we try to decrease the resolution to
// improve the chance that we will agree now.
if (!previousAgree && ledgerSeq % decreaseLedgerTimeResolutionEvery == 0)
{
if (++iter != std::end (ledgerPossibleTimeResolutions))
return *iter;
}
// If we previously agreed, we try to increase the resolution to determine
// if we can continue to agree.
if (previousAgree && ledgerSeq % increaseLedgerTimeResolutionEvery == 0)
{
if (iter-- != std::begin (ledgerPossibleTimeResolutions))
return *iter;
}
return previousResolution;
}
NetClock::time_point
roundCloseTime (
NetClock::time_point closeTime,
NetClock::duration closeResolution)
{
if (closeTime == NetClock::time_point{})
return closeTime;
closeTime += (closeResolution / 2);
return closeTime - (closeTime.time_since_epoch() % closeResolution);
}
bool
shouldCloseLedger (
bool anyTransactions,
int previousProposers,
int proposersClosed,
int proposersValidated,
std::size_t previousProposers,
std::size_t proposersClosed,
std::size_t proposersValidated,
std::chrono::milliseconds previousTime,
std::chrono::milliseconds currentTime, // Time since last ledger's close time
std::chrono::milliseconds openTime, // Time waiting to close this ledger
@@ -135,7 +86,10 @@ shouldCloseLedger (
}
bool
checkConsensusReached (int agreeing, int total, bool count_self)
checkConsensusReached (
std::size_t agreeing,
std::size_t total,
bool count_self)
{
// If we are alone, we have a consensus
if (total == 0)
@@ -154,10 +108,10 @@ checkConsensusReached (int agreeing, int total, bool count_self)
ConsensusState
checkConsensus (
int previousProposers,
int currentProposers,
int currentAgree,
int currentFinished,
std::size_t previousProposers,
std::size_t currentProposers,
std::size_t currentAgree,
std::size_t currentFinished,
std::chrono::milliseconds previousAgreeTime,
std::chrono::milliseconds currentAgreeTime,
bool proposing,

313
src/ripple/app/ledger/LedgerTiming.h → src/ripple/consensus/LedgerTiming.h
View File

@@ -27,6 +27,117 @@
namespace ripple {
//------------------------------------------------------------------------------
// These are protocol parameters used to control the behavior of the system and
// they should not be changed arbitrarily.
//! The percentage threshold above which we can declare consensus.
auto constexpr minimumConsensusPercentage = 80;
using namespace std::chrono_literals;
/** Possible close time resolutions.
Values should not be duplicated.
@see getNextLedgerTimeResolution
*/
std::chrono::seconds constexpr ledgerPossibleTimeResolutions[] =
{ 10s, 20s, 30s, 60s, 90s, 120s };
#ifndef _MSC_VER
//! Initial resolution of ledger close time.
auto constexpr ledgerDefaultTimeResolution = ledgerPossibleTimeResolutions[2];
#else
// HH Remove this workaround of a VS bug when possible
//! Initial resolution of ledger close time.
auto constexpr ledgerDefaultTimeResolution = 30s;
#endif
//! How often we increase the close time resolution (in numbers of ledgers)
auto constexpr increaseLedgerTimeResolutionEvery = 8;
//! How often we decrease the close time resolution (in numbers of ledgers)
auto constexpr decreaseLedgerTimeResolutionEvery = 1;
//! The number of seconds a ledger may remain idle before closing
auto constexpr LEDGER_IDLE_INTERVAL = 15s;
/** The number of seconds a validation remains current after its ledger's close
time.
This is a safety to protect against very old validations and the time
it takes to adjust the close time accuracy window.
*/
auto constexpr VALIDATION_VALID_WALL = 5min;
/** Duration a validation remains current after first observed.
The number of seconds a validation remains current after the time we first
saw it. This provides faster recovery in very rare cases where the number
of validations produced by the network is lower than normal
*/
auto constexpr VALIDATION_VALID_LOCAL = 3min;
/** Duration pre-close in which validations are acceptable.
The number of seconds before a close time that we consider a validation
acceptable. This protects against extreme clock errors
*/
auto constexpr VALIDATION_VALID_EARLY = 3min;
//! The number of seconds we wait minimum to ensure participation
auto constexpr LEDGER_MIN_CONSENSUS = 1950ms;
//! Minimum number of seconds to wait to ensure others have computed the LCL
auto constexpr LEDGER_MIN_CLOSE = 2s;
//! How often we check state or change positions
auto constexpr LEDGER_GRANULARITY = 1s;
//! How long we consider a proposal fresh
auto constexpr PROPOSE_FRESHNESS = 20s;
//! How often we force generating a new proposal to keep ours fresh
auto constexpr PROPOSE_INTERVAL = 12s;
//------------------------------------------------------------------------------
// Avalanche tuning
//! Percentage of nodes on our UNL that must vote yes
auto constexpr AV_INIT_CONSENSUS_PCT = 50;
//! Percentage of previous close time before we advance
auto constexpr AV_MID_CONSENSUS_TIME = 50;
//! Percentage of nodes that most vote yes after advancing
auto constexpr AV_MID_CONSENSUS_PCT = 65;
//! Percentage of previous close time before we advance
auto constexpr AV_LATE_CONSENSUS_TIME = 85;
//! Percentage of nodes that most vote yes after advancing
auto constexpr AV_LATE_CONSENSUS_PCT = 70;
//! Percentage of previous close time before we are stuck
auto constexpr AV_STUCK_CONSENSUS_TIME = 200;
//! Percentage of nodes that must vote yes after we are stuck
auto constexpr AV_STUCK_CONSENSUS_PCT = 95;
//! Percentage of nodes required to reach agreement on ledger close time
auto constexpr AV_CT_CONSENSUS_PCT = 75;
/** The minimum amount of time to consider the previous round
to have taken.
The minimum amount of time to consider the previous round
to have taken. This ensures that there is an opportunity
for a round at each avalanche threshold even if the
previous consensus was very fast. This should be at least
twice the interval between proposals (0.7s) divided by
the interval between mid and late consensus ([85-50]/100).
*/
auto constexpr AV_MIN_CONSENSUS_TIME = 5s;
/** Calculates the close time resolution for the specified ledger.
The Ripple protocol uses binning to represent time intervals using only one
@@ -34,22 +145,95 @@ namespace ripple {
without the need for perfectly synchronized clocks.
The time resolution (i.e. the size of the intervals) is adjusted dynamically
based on what happened in the last ledger, to try to avoid disagreements.
@param previousResolution the resolution used for the prior ledger
@param previousAgree whether consensus agreed on the close time of the prior
ledger
@param ledgerSeq the sequence number of the new ledger
@pre previousResolution must be a valid bin
from @ref ledgerPossibleTimeResolutions
*/
NetClock::duration
getNextLedgerTimeResolution (
NetClock::duration previousResolution,
template <class duration>
duration
getNextLedgerTimeResolution(
duration previousResolution,
bool previousAgree,
std::uint32_t ledgerSeq);
std::uint32_t ledgerSeq)
{
assert (ledgerSeq);
using namespace std::chrono;
// Find the current resolution:
auto iter = std::find (std::begin (ledgerPossibleTimeResolutions),
std::end (ledgerPossibleTimeResolutions), previousResolution);
assert (iter != std::end (ledgerPossibleTimeResolutions));
// This should never happen, but just as a precaution
if (iter == std::end (ledgerPossibleTimeResolutions))
return previousResolution;
// If we did not previously agree, we try to decrease the resolution to
// improve the chance that we will agree now.
if (!previousAgree && ledgerSeq % decreaseLedgerTimeResolutionEvery == 0)
{
if (++iter != std::end (ledgerPossibleTimeResolutions))
return *iter;
}
// If we previously agreed, we try to increase the resolution to determine
// if we can continue to agree.
if (previousAgree && ledgerSeq % increaseLedgerTimeResolutionEvery == 0)
{
if (iter-- != std::begin (ledgerPossibleTimeResolutions))
return *iter;
}
return previousResolution;
}
/** Calculates the close time for a ledger, given a close time resolution.
@param closeTime The time to be rouned.
@param closeResolution The resolution
@return @b closeTime rounded to the nearest multiple of @b closeResolution.
Rounds up if @b closeTime is midway between multiples of @b closeResolution.
*/
NetClock::time_point
roundCloseTime (
NetClock::time_point closeTime,
NetClock::duration closeResolution);
template <class time_point>
time_point
roundCloseTime(
time_point closeTime,
typename time_point::duration closeResolution)
{
if (closeTime == time_point{})
return closeTime;
closeTime += (closeResolution / 2);
return closeTime - (closeTime.time_since_epoch() % closeResolution);
}
/** Calculate the effective ledger close time
After adjusting the ledger close time based on the current resolution, also
ensure it is sufficiently separated from the prior close time.
@param closeTime The raw ledger close time
@param resolution The current close time resolution
@param priorCloseTime The close time of the prior ledger
*/
template <class time_point>
time_point effectiveCloseTime(time_point closeTime,
typename time_point::duration const resolution,
time_point priorCloseTime)
{
if (closeTime == time_point{})
return closeTime;
return std::max<time_point>(
roundCloseTime (closeTime, resolution),
(priorCloseTime + 1s));
}
/** Determines whether the current ledger should close at this time.
@@ -71,9 +255,9 @@ roundCloseTime (
bool
shouldCloseLedger (
bool anyTransactions,
int previousProposers,
int proposersClosed,
int proposersValidated,
std::size_t previousProposers,
std::size_t proposersClosed,
std::size_t proposersValidated,
std::chrono::milliseconds previousTime,
std::chrono::milliseconds currentTime, // Time since last ledger's close time
std::chrono::milliseconds openTime, // Time waiting to close this ledger
@@ -91,14 +275,17 @@ shouldCloseLedger (
@return True if a consensus has been reached
*/
bool
checkConsensusReached (int agreeing, int total, bool count_self);
checkConsensusReached (
std::size_t agreeing,
std::size_t total,
bool count_self);
/** Whether we have or don't have a consensus */
enum class ConsensusState
{
No, // We do not have consensus
MovedOn, // The network has consensus without us
Yes // We have consensus along with the network
No, //!< We do not have consensus
MovedOn, //!< The network has consensus without us
Yes //!< We have consensus along with the network
};
/** Determine whether the network reached consensus and whether we joined.
@@ -116,103 +303,15 @@ enum class ConsensusState
*/
ConsensusState
checkConsensus (
int previousProposers,
int currentProposers,
int currentAgree,
int currentFinished,
std::size_t previousProposers,
std::size_t currentProposers,
std::size_t currentAgree,
std::size_t currentFinished,
std::chrono::milliseconds previousAgreeTime,
std::chrono::milliseconds currentAgreeTime,
bool proposing,
beast::Journal j);
//------------------------------------------------------------------------------
// These are protocol parameters used to control the behavior of the system and
// they should not be changed arbitrarily.
// The percentage threshold above which we can declare consensus.
auto constexpr minimumConsensusPercentage = 80;
using namespace std::chrono_literals;
// All possible close time resolutions. Values should not be duplicated.
std::chrono::seconds constexpr ledgerPossibleTimeResolutions[] =
{ 10s, 20s, 30s, 60s, 90s, 120s };
#ifndef _MSC_VER
// Initial resolution of ledger close time.
auto constexpr ledgerDefaultTimeResolution = ledgerPossibleTimeResolutions[2];
#else
// HH Remove this workaround of a VS bug when possible
auto constexpr ledgerDefaultTimeResolution = 30s;
#endif
// How often we increase the close time resolution
auto constexpr increaseLedgerTimeResolutionEvery = 8;
// How often we decrease the close time resolution
auto constexpr decreaseLedgerTimeResolutionEvery = 1;
// The number of seconds a ledger may remain idle before closing
auto constexpr LEDGER_IDLE_INTERVAL = 15s;
// The number of seconds a validation remains current after its ledger's close
// time. This is a safety to protect against very old validations and the time
// it takes to adjust the close time accuracy window
auto constexpr VALIDATION_VALID_WALL = 5min;
// The number of seconds a validation remains current after the time we first
// saw it. This provides faster recovery in very rare cases where the number
// of validations produced by the network is lower than normal
auto constexpr VALIDATION_VALID_LOCAL = 3min;
// The number of seconds before a close time that we consider a validation
// acceptable. This protects against extreme clock errors
auto constexpr VALIDATION_VALID_EARLY = 3min;
// The number of seconds we wait minimum to ensure participation
auto constexpr LEDGER_MIN_CONSENSUS = 1950ms;
// Minimum number of seconds to wait to ensure others have computed the LCL
auto constexpr LEDGER_MIN_CLOSE = 2s;
// How often we check state or change positions (in milliseconds)
auto constexpr LEDGER_GRANULARITY = 1s;
// How long we consider a proposal fresh
auto constexpr PROPOSE_FRESHNESS = 20s;
// How often we force generating a new proposal to keep ours fresh
auto constexpr PROPOSE_INTERVAL = 12s;
// Avalanche tuning
// percentage of nodes on our UNL that must vote yes
auto constexpr AV_INIT_CONSENSUS_PCT = 50;
// percentage of previous close time before we advance
auto constexpr AV_MID_CONSENSUS_TIME = 50;
// percentage of nodes that most vote yes after advancing
auto constexpr AV_MID_CONSENSUS_PCT = 65;
// percentage of previous close time before we advance
auto constexpr AV_LATE_CONSENSUS_TIME = 85;
// percentage of nodes that most vote yes after advancing
auto constexpr AV_LATE_CONSENSUS_PCT = 70;
auto constexpr AV_STUCK_CONSENSUS_TIME = 200;
auto constexpr AV_STUCK_CONSENSUS_PCT = 95;
auto constexpr AV_CT_CONSENSUS_PCT = 75;
// The minimum amount of time to consider the previous round
// to have taken. This ensures that there is an opportunity
// for a round at each avalanche threshold even if the
// previous consensus was very fast. This should be at least
// twice the interval between proposals (0.7s) divided by
// the interval between mid and late consensus ([85-50]/100).
auto constexpr AV_MIN_CONSENSUS_TIME = 5s;
} // ripple
#endif

9
src/ripple/consensus/README.md Normal file
View File

@@ -0,0 +1,9 @@
# Consensus
This directory contains the implementation of a
generic consensus algorithm. The implementation
follows a CRTP design, requiring client code to implement
specific functions and types to use consensus in their
application. The interface is undergoing refactoring and
is not yet finalized.

20
src/ripple/overlay/impl/PeerImp.cpp
View File

@@ -23,7 +23,7 @@
#include <ripple/overlay/impl/Tuning.h>
#include <ripple/app/ledger/InboundLedgers.h>
#include <ripple/app/ledger/LedgerMaster.h>
#include <ripple/app/ledger/LedgerTiming.h>
#include <ripple/consensus/LedgerTiming.h>
#include <ripple/app/ledger/InboundTransactions.h>
#include <ripple/app/misc/HashRouter.h>
#include <ripple/app/misc/LoadFeeTrack.h>
@@ -1275,10 +1275,10 @@ PeerImp::onMessage (std::shared_ptr <protocol::TMProposeSet> const& m)
JLOG(p_journal_.trace()) <<
"Proposal: " << (isTrusted ? "trusted" : "UNTRUSTED");
auto proposal = std::make_shared<LedgerProposal> (
prevLedger, set.proposeseq (), proposeHash, closeTime,
app_.timeKeeper().closeTime(), publicKey, calcNodeID(publicKey),
signature, suppression);
auto proposal = std::make_shared<RCLCxPeerPos> (
publicKey, signature, suppression,
RCLCxPeerPos::Proposal{prevLedger, set.proposeseq (), proposeHash, closeTime,
app_.timeKeeper().closeTime(),calcNodeID(publicKey)});
std::weak_ptr<PeerImp> weak = shared_from_this();
app_.getJobQueue ().addJob (
@@ -1878,7 +1878,7 @@ PeerImp::checkTransaction (int flags,
void
PeerImp::checkPropose (Job& job,
std::shared_ptr <protocol::TMProposeSet> const& packet,
LedgerProposal::pointer proposal)
RCLCxPeerPos::pointer peerPos)
{
bool isTrusted = (job.getType () == jtPROPOSAL_t);
@@ -1888,7 +1888,7 @@ PeerImp::checkPropose (Job& job,
assert (packet);
protocol::TMProposeSet& set = *packet;
if (! cluster() && ! proposal->checkSign ())
if (! cluster() && !peerPos->checkSign ())
{
JLOG(p_journal_.warn()) <<
"Proposal fails sig check";
@@ -1899,16 +1899,16 @@ PeerImp::checkPropose (Job& job,
if (isTrusted)
{
app_.getOPs ().processTrustedProposal (
proposal, packet, calcNodeID (publicKey_));
peerPos, packet, calcNodeID (publicKey_));
}
else
{
if (app_.getOPs().getConsensusLCL() == proposal->getPrevLedger())
if (app_.getOPs().getConsensusLCL() == peerPos->proposal().prevLedger())
{
// relay untrusted proposal
JLOG(p_journal_.trace()) <<
"relaying UNTRUSTED proposal";
overlay_.relay(set, proposal->getSuppressionID());
overlay_.relay(set, peerPos->getSuppressionID());
}
else
{

4
src/ripple/overlay/impl/PeerImp.h
View File

@@ -20,7 +20,7 @@
#ifndef RIPPLE_OVERLAY_PEERIMP_H_INCLUDED
#define RIPPLE_OVERLAY_PEERIMP_H_INCLUDED
#include <ripple/app/ledger/LedgerProposal.h>
#include <ripple/app/consensus/RCLCxPeerPos.h>
#include <ripple/basics/Log.h> // deprecated
#include <ripple/nodestore/Database.h>
#include <ripple/overlay/predicates.h>
@@ -452,7 +452,7 @@ private:
void
checkPropose (Job& job,
std::shared_ptr<protocol::TMProposeSet> const& packet,
LedgerProposal::pointer proposal);
RCLCxPeerPos::pointer peerPos);
void
checkValidation (STValidation::pointer val,

6
src/ripple/protocol/JsonFields.h
View File

@@ -108,7 +108,7 @@ JSS ( check_nodes ); // in: LedgerCleaner
JSS ( clear ); // in/out: FetchInfo
JSS ( close_flags ); // out: LedgerToJson
JSS ( close_time ); // in: Application, out: NetworkOPs,
// LedgerProposal, LedgerToJson
// RCLCxPeerPos, LedgerToJson
JSS ( close_time_estimated ); // in: Application, out: LedgerToJson
JSS ( close_time_human ); // out: LedgerToJson
JSS ( close_time_offset ); // out: NetworkOPs
@@ -316,7 +316,7 @@ JSS ( paths_canonical ); // out: RipplePathFind
JSS ( paths_computed ); // out: PathRequest, RipplePathFind
JSS ( peer ); // in: AccountLines
JSS ( peer_authorized ); // out: AccountLines
JSS ( peer_id ); // out: LedgerProposal
JSS ( peer_id ); // out: RCLCxPeerPos
JSS ( peers ); // out: InboundLedger, handlers/Peers, Overlay
JSS ( port ); // in: Connect
JSS ( previous_ledger ); // out: LedgerPropose
@@ -408,7 +408,7 @@ JSS ( total_coins ); // out: LedgerToJson
JSS ( transTreeHash ); // out: ledger/Ledger.cpp
JSS ( transaction ); // in: Tx
// out: NetworkOPs, AcceptedLedgerTx,
JSS ( transaction_hash ); // out: LedgerProposal, LedgerToJson
JSS ( transaction_hash ); // out: RCLCxPeerPos, LedgerToJson
JSS ( transactions ); // out: LedgerToJson,
// in: AccountTx*, Unsubscribe
JSS ( transitions ); // out: NetworkOPs

2
src/ripple/protocol/STValidation.h
View File

@@ -94,7 +94,7 @@ public:
{
mSeen = s;
}
Blob getSigned () const;
Blob getSerialized () const;
Blob getSignature () const;
// Signs the validation and returns the signing hash

2
src/ripple/protocol/impl/STValidation.cpp
View File

@@ -134,7 +134,7 @@ Blob STValidation::getSignature () const
return getFieldVL (sfSignature);
}
Blob STValidation::getSigned () const
Blob STValidation::getSerialized () const
{
Serializer s;
add (s);

39
src/ripple/app/consensus/RCLCxTraits.h → src/ripple/unity/app_consensus.cpp
View File

@@ -16,40 +16,7 @@
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <BeastConfig.h>
#ifndef RIPPLE_APP_CONSENSUS_RCLCXTRAITS_H_INCLUDED
#define RIPPLE_APP_CONSENSUS_RCLCXTRAITS_H_INCLUDED
#include <ripple/basics/chrono.h>
#include <ripple/basics/base_uint.h>
#include <ripple/protocol/UintTypes.h>
#include <ripple/protocol/RippleLedgerHash.h>
#include <ripple/app/consensus/RCLCxPos.h>
#include <ripple/app/consensus/RCLCxTx.h>
namespace ripple {
// Consensus traits class
// For adapting consensus to RCL
class RCLCxTraits
{
public:
using Time_t = NetClock::time_point;
using Pos_t = RCLCxPos;
using TxSet_t = RCLTxSet;
using Tx_t = RCLCxTx;
using LgrID_t = LedgerHash;
using TxID_t = uint256;
using TxSetID_t = uint256;
using NodeID_t = NodeID;
};
}
#endif
#include <ripple/app/consensus/RCLConsensus.cpp>
#include <ripple/app/consensus/RCLCxPeerPos.cpp>

4
src/ripple/unity/app_ledger.cpp
View File

@@ -26,18 +26,14 @@
#include <ripple/app/ledger/ConsensusTransSetSF.cpp>
#include <ripple/app/ledger/Ledger.cpp>
#include <ripple/app/ledger/LedgerHistory.cpp>
#include <ripple/app/ledger/LedgerProposal.cpp>
#include <ripple/app/ledger/OrderBookDB.cpp>
#include <ripple/app/ledger/TransactionStateSF.cpp>
#include <ripple/app/ledger/impl/ConsensusImp.cpp>
#include <ripple/app/ledger/impl/InboundLedger.cpp>
#include <ripple/app/ledger/impl/InboundLedgers.cpp>
#include <ripple/app/ledger/impl/InboundTransactions.cpp>
#include <ripple/app/ledger/impl/LedgerCleaner.cpp>
#include <ripple/app/ledger/impl/LedgerConsensusImp.cpp>
#include <ripple/app/ledger/impl/LedgerMaster.cpp>
#include <ripple/app/ledger/impl/LedgerTiming.cpp>
#include <ripple/app/ledger/impl/LocalTxs.cpp>
#include <ripple/app/ledger/impl/OpenLedger.cpp>
#include <ripple/app/ledger/impl/LedgerToJson.cpp>

21
src/ripple/unity/consensus.cpp Normal file
View File

@@ -0,0 +1,21 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2016 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <BeastConfig.h>
#include <ripple/consensus/LedgerTiming.cpp>

1
src/test/app/TxQ_test.cpp
View File

@@ -20,7 +20,6 @@
#include <ripple/app/main/Application.h>
#include <ripple/app/misc/LoadFeeTrack.h>
#include <ripple/app/misc/TxQ.h>
#include <ripple/app/ledger/LedgerConsensus.h>
#include <ripple/app/tx/apply.h>
#include <ripple/basics/Log.h>
#include <ripple/basics/mulDiv.h>

474
src/test/consensus/Consensus_test.cpp Normal file
View File

@@ -0,0 +1,474 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2016 Ripple Labs Inc->
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <BeastConfig.h>
#include <ripple/beast/unit_test.h>
#include <ripple/consensus/Consensus.h>
#include <ripple/consensus/ConsensusProposal.h>
#include <ripple/beast/clock/manual_clock.h>
#include <boost/function_output_iterator.hpp>
#include <test/csf.h>
#include <utility>
namespace ripple {
namespace test {
class Consensus_test : public beast::unit_test::suite
{
public:
void
testStandalone()
{
using namespace csf;
auto tg = TrustGraph::makeComplete(1);
Sim s(tg, topology(tg, fixed{LEDGER_GRANULARITY}));
auto & p = s.peers[0];
p.targetLedgers = 1;
p.start();
p.submit(Tx{ 1 });
s.net.step();
// Inspect that the proper ledger was created
BEAST_EXPECT(p.LCL().seq == 1);
BEAST_EXPECT(p.LCL() == p.lastClosedLedger.id());
BEAST_EXPECT(p.lastClosedLedger.id().txs.size() == 1);
BEAST_EXPECT(p.lastClosedLedger.id().txs.find(Tx{ 1 })
!= p.lastClosedLedger.id().txs.end());
BEAST_EXPECT(p.getLastCloseProposers() == 0);
}
void
testPeersAgree()
{
using namespace csf;
using namespace std::chrono;
auto tg = TrustGraph::makeComplete(5);
Sim sim(tg,
topology(tg, fixed{round<milliseconds>(0.2 * LEDGER_GRANULARITY)}));
// everyone submits their own ID as a TX and relay it to peers
for (auto & p : sim.peers)
p.submit(Tx(p.id));
// Verify all peers have the same LCL and it has all the Txs
sim.run(1);
for (auto & p : sim.peers)
{
auto const &lgrID = p.LCL();
BEAST_EXPECT(lgrID.seq == 1);
BEAST_EXPECT(p.getLastCloseProposers() == sim.peers.size() - 1);
for(std::uint32_t i = 0; i < sim.peers.size(); ++i)
BEAST_EXPECT(lgrID.txs.find(Tx{ i }) != lgrID.txs.end());
// Matches peer 0 ledger
BEAST_EXPECT(lgrID.txs == sim.peers[0].LCL().txs);
}
}
void
testSlowPeer()
{
using namespace csf;
using namespace std::chrono;
// Run two tests
// 1. The slow peer is participating inconsensus
// 2. The slow peer is just observing
for(auto isParticipant : {true, false})
{
auto tg = TrustGraph::makeComplete(5);
Sim sim(tg, topology(tg,[](PeerID i, PeerID j)
{
auto delayFactor = (i == 0 || j == 0) ? 1.1 : 0.2;
return round<milliseconds>(delayFactor* LEDGER_GRANULARITY);
}));
sim.peers[0].proposing = sim.peers[0].validating = isParticipant;
// All peers submit their own ID as a transaction and relay it to peers
for (auto & p : sim.peers)
{
p.submit(Tx{ p.id });
}
sim.run(1);
// Verify all peers have same LCL but are missing transaction 0 which
// was not received by all peers before the ledger closed
for (auto & p : sim.peers)
{
auto const &lgrID = p.LCL();
BEAST_EXPECT(lgrID.seq == 1);
// If peer 0 is participating
if(isParticipant)
{
BEAST_EXPECT(p.getLastCloseProposers()
== sim.peers.size() - 1);
// Peer 0 closes first because it sees a quorum of agreeing positions
// from all other peers in one hop (1->0, 2->0, ..)
// The other peers take an extra timer period before they find that
// Peer 0 agrees with them ( 1->0->1, 2->0->2, ...)
if(p.id != 0)
BEAST_EXPECT(p.getLastConvergeDuration()
> sim.peers[0].getLastConvergeDuration());
}
else // peer 0 is not participating
{
auto const proposers = p.getLastCloseProposers();
if(p.id == 0)
BEAST_EXPECT(proposers == sim.peers.size() - 1);
else
BEAST_EXPECT(proposers == sim.peers.size() - 2);
// so all peers should have closed together
BEAST_EXPECT(p.getLastConvergeDuration()
== sim.peers[0].getLastConvergeDuration());
}
BEAST_EXPECT(lgrID.txs.find(Tx{ 0 }) == lgrID.txs.end());
for(std::uint32_t i = 1; i < sim.peers.size(); ++i)
BEAST_EXPECT(lgrID.txs.find(Tx{ i }) != lgrID.txs.end());
// Matches peer 0 ledger
BEAST_EXPECT(lgrID.txs == sim.peers[0].LCL().txs);
}
BEAST_EXPECT(sim.peers[0].openTxs.find(Tx{ 0 })
!= sim.peers[0].openTxs.end());
}
}
void
testCloseTimeDisagree()
{
using namespace csf;
using namespace std::chrono;
// This is a very specialized test to get ledgers to disagree on
// the close time. It unfortunately assumes knowledge about current
// timing constants. This is a necessary evil to get coverage up
// pending more extensive refactorings of timing constants.
// In order to agree-to-disagree on the close time, there must be no
// clear majority of nodes agreeing on a close time. This test
// sets a relative offset to the peers internal clocks so that they
// send proposals with differing times.
// However, they have to agree on the effective close time, not the
// exact close time. The minimum closeTimeResolution is given by
// ledgerPossibleTimeResolutions[0], which is currently 10s. This means
// the skews need to be at least 10 seconds.
// Complicating this matter is that nodes will ignore proposals
// with times more than PROPOSE_FRESHNESS =20s in the past. So at
// the minimum granularity, we have at most 3 types of skews (0s,10s,20s).
// This test therefore has 6 nodes, with 2 nodes having each type of
// skew. Then no majority (1/3 < 1/2) of nodes will agree on an
// actual close time.
auto tg = TrustGraph::makeComplete(6);
Sim sim(tg,
topology(tg, fixed{round<milliseconds>(0.2 * LEDGER_GRANULARITY)}));
// Run consensus without skew until we have a short close time resolution
while(sim.peers.front().lastClosedLedger.closeTimeResolution() >=
PROPOSE_FRESHNESS)
sim.run(1);
// Introduce a shift on the time of half the peers
sim.peers[0].clockSkew = PROPOSE_FRESHNESS/2;
sim.peers[1].clockSkew = PROPOSE_FRESHNESS/2;
sim.peers[2].clockSkew = PROPOSE_FRESHNESS;
sim.peers[3].clockSkew = PROPOSE_FRESHNESS;
// Verify all peers have the same LCL and it has all the Txs
sim.run(1);
for (auto & p : sim.peers)
{
BEAST_EXPECT(! p.lastClosedLedger.closeAgree());
}
}
void
testWrongLCL()
{
using namespace csf;
using namespace std::chrono;
// Specialized test to exercise a temporary fork in which some peers
// are working on an incorrect prior ledger.
// Vary the time it takes to process validations to exercise detecting
// the wrong LCL at different phases of consensus
for(auto validationDelay : {0s, LEDGER_MIN_CLOSE})
{
// Consider 10 peers:
// 0 1 2 3 4 5 6 7 8 9
//
// Nodes 0-1 trust nodes 0-4
// Nodes 2-9 trust nodes 2-9
//
// By submitting tx 0 to nodes 0-4 and tx 1 to nodes 5-9,
// nodes 0-1 will generate the wrong LCL (with tx 0). The remaining
// nodes will instead accept the ledger with tx 1.
// Nodes 0-1 will detect this mismatch during a subsequent round
// since nodes 2-4 will validate a different ledger.
// Nodes 0-1 will acquire the proper ledger from the network and
// resume consensus and eventually generate the dominant network ledger
std::vector<UNL> unls;
unls.push_back({2,3,4,5,6,7,8,9});
unls.push_back({0,1,2,3,4});
std::vector<int> membership(10,0);
membership[0] = 1;
membership[1] = 1;
TrustGraph tg{unls, membership};
// This topology can fork, which is why we are using it for this test.
BEAST_EXPECT(tg.canFork(minimumConsensusPercentage/100.));
auto netDelay = round<milliseconds>(0.2 * LEDGER_GRANULARITY);
Sim sim(tg, topology(tg, fixed{netDelay}));
// initial round to set prior state
sim.run(1);
// Nodes in smaller UNL have seen tx 0, nodes in other unl have seen tx 1
for (auto & p : sim.peers)
{
p.validationDelay = validationDelay;
p.missingLedgerDelay = netDelay;
if (unls[1].find(p.id) != unls[1].end())
p.openTxs.insert(Tx{0});
else
p.openTxs.insert(Tx{1});
}
// Run for 2 additional rounds
// - One round to generate different ledgers
// - One round to detect different prior ledgers (but still generate
// wrong ones) and recover
sim.run(2);
bc::flat_map<int, bc::flat_set<Ledger::ID>> ledgers;
for (auto & p : sim.peers)
{
for (auto const & l : p.ledgers)
{
ledgers[l.first.seq].insert(l.first);
}
}
BEAST_EXPECT(ledgers[0].size() == 1);
BEAST_EXPECT(ledgers[1].size() == 1);
BEAST_EXPECT(ledgers[2].size() == 2);
BEAST_EXPECT(ledgers[3].size() == 1);
}
}
void
testFork()
{
using namespace csf;
using namespace std::chrono;
int numPeers = 10;
for(int overlap = 0; overlap <= numPeers; ++overlap)
{
auto tg = TrustGraph::makeClique(numPeers, overlap);
Sim sim(tg, topology(tg,
fixed{round<milliseconds>(0.2 * LEDGER_GRANULARITY)}));
// Initial round to set prior state
sim.run(1);
for (auto & p : sim.peers)
{
// Nodes have only seen transactions from their neighbors
p.openTxs.insert(Tx{p.id});
for(auto const link : sim.net.links(&p))
p.openTxs.insert(Tx{link.to->id});
}
sim.run(1);
// See if the network forked
bc::flat_set<Ledger::ID> ledgers;
for (auto & p : sim.peers)
{
ledgers.insert(p.LCL());
}
// Fork should not happen for 40% or greater overlap
// Since the overlapped nodes have a UNL that is the union of the
// two cliques, the maximum sized UNL list is the number of peers
if(overlap > 0.4 * numPeers)
BEAST_EXPECT(ledgers.size() == 1);
else // Even if we do fork, there shouldn't be more than 3 ledgers
// One for cliqueA, one for cliqueB and one for nodes in both
BEAST_EXPECT(ledgers.size() <= 3);
}
}
void
simClockSkew()
{
using namespace csf;
// Attempting to test what happens if peers enter consensus well
// separated in time. Initial round (in which peers are not staggered)
// is used to get the network going, then transactions are submitted
// together and consensus continues.
// For all the times below, the same ledger is built but the close times
// disgree. BUT THE LEDGER DOES NOT SHOW disagreeing close times.
// It is probably because peer proposals are stale, so they get ignored
// but with no peer proposals, we always assume close time consensus is
// true.
// Disabled while continuing to understand testt.
for(auto stagger : {800ms, 1600ms, 3200ms, 30000ms, 45000ms, 300000ms})
{
auto tg = TrustGraph::makeComplete(5);
Sim sim(tg, topology(tg, [](PeerID i, PeerID)
{
return 200ms * (i + 1);
}));
// all transactions submitted before starting
// Initial round to set prior state
sim.run(1);
for (auto & p : sim.peers)
{
p.openTxs.insert(Tx{ 0 });
p.targetLedgers = p.completedLedgers + 1;
}
// stagger start of consensus
for (auto & p : sim.peers)
{
p.start();
sim.net.step_for(stagger);
}
// run until all peers have accepted all transactions
sim.net.step_while([&]()
{
for(auto & p : sim.peers)
{
if(p.LCL().txs.size() != 1)
{
return true;
}
}
return false;
});
}
}
void
simScaleFree()
{
using namespace std::chrono;
using namespace csf;
// Generate a quasi-random scale free network and simulate consensus
// for a single transaction
int N = 100; // Peers
int numUNLs = 15; // UNL lists
int minUNLSize = N/4, maxUNLSize = N / 2;
double transProb = 0.5;
std::mt19937_64 rng;
auto tg = TrustGraph::makeRandomRanked(N, numUNLs,
PowerLawDistribution{1,3},
std::uniform_int_distribution<>{minUNLSize, maxUNLSize},
rng);
Sim sim{tg, topology(tg, fixed{round<milliseconds>(0.2 * LEDGER_GRANULARITY)})};
// Initial round to set prior state
sim.run(1);
std::uniform_real_distribution<> u{};
for (auto & p : sim.peers)
{
// 50-50 chance to have seen a transaction
if(u(rng) >= transProb)
p.openTxs.insert(Tx{0});
}
sim.run(1);
// See if the network forked
bc::flat_set<Ledger::ID> ledgers;
for (auto & p : sim.peers)
{
ledgers.insert(p.LCL());
}
BEAST_EXPECT(ledgers.size() == 1);
}
void
run() override
{
testStandalone();
testPeersAgree();
testSlowPeer();
testCloseTimeDisagree();
testWrongLCL();
testFork();
simClockSkew();
simScaleFree();
}
};
BEAST_DEFINE_TESTSUITE(Consensus, consensus, ripple);
} // test
} // ripple

164
src/test/consensus/LedgerTiming_test.cpp Normal file
View File

@@ -0,0 +1,164 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2016 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <BeastConfig.h>
#include <ripple/beast/unit_test.h>
#include <ripple/consensus/LedgerTiming.h>
namespace ripple {
namespace test {
class LedgerTiming_test : public beast::unit_test::suite
{
beast::Journal j;
void testGetNextLedgerTimeResolution()
{
// helper to iteratively call into getNextLedgerTimeResolution
struct test_res
{
std::uint32_t decrease = 0;
std::uint32_t equal = 0;
std::uint32_t increase = 0;
static test_res run(bool previousAgree, std::uint32_t rounds)
{
test_res res;
auto closeResolution = ledgerDefaultTimeResolution;
auto nextCloseResolution = closeResolution;
std::uint32_t round = 0;
do
{
nextCloseResolution = getNextLedgerTimeResolution(
closeResolution, previousAgree, ++round);
if (nextCloseResolution < closeResolution)
++res.decrease;
else if (nextCloseResolution > closeResolution)
++res.increase;
else
++res.equal;
std::swap(nextCloseResolution, closeResolution);
} while (round < rounds);
return res;
}
};
// If we never agree on close time, only can increase resolution
// until hit the max
auto decreases = test_res::run(false, 10);
BEAST_EXPECT(decreases.increase == 3);
BEAST_EXPECT(decreases.decrease == 0);
BEAST_EXPECT(decreases.equal == 7);
// If we always agree on close time, only can decrease resolution
// until hit the min
auto increases = test_res::run(false, 100);
BEAST_EXPECT(increases.increase == 3);
BEAST_EXPECT(increases.decrease == 0);
BEAST_EXPECT(increases.equal == 97);
}
void testRoundCloseTime()
{
// A closeTime equal to the epoch is not modified
using tp = NetClock::time_point;
tp def;
BEAST_EXPECT(def == roundCloseTime(def, 30s));
// Otherwise, the closeTime is rounded to the nearest
// rounding up on ties
BEAST_EXPECT(tp{ 0s } == roundCloseTime(tp{ 29s }, 60s));
BEAST_EXPECT(tp{ 30s } == roundCloseTime(tp{ 30s }, 1s));
BEAST_EXPECT(tp{ 60s } == roundCloseTime(tp{ 31s }, 60s));
BEAST_EXPECT(tp{ 60s } == roundCloseTime(tp{ 30s }, 60s));
BEAST_EXPECT(tp{ 60s } == roundCloseTime(tp{ 59s }, 60s));
BEAST_EXPECT(tp{ 60s } == roundCloseTime(tp{ 60s }, 60s));
BEAST_EXPECT(tp{ 60s } == roundCloseTime(tp{ 61s }, 60s));
}
void testShouldCloseLedger()
{
// Bizarre times forcibly close
BEAST_EXPECT(shouldCloseLedger(true, 10, 10, 10, -10s, 10s, 1s, 1s, j));
BEAST_EXPECT(shouldCloseLedger(true, 10, 10, 10, 100h, 10s, 1s, 1s, j));
BEAST_EXPECT(shouldCloseLedger(true, 10, 10, 10, 10s, 100h, 1s, 1s, j));
// Rest of network has closed
BEAST_EXPECT(shouldCloseLedger(true, 10, 3, 5, 10s, 10s, 10s, 10s, j));
// No transactions means wait until end of internval
BEAST_EXPECT(!shouldCloseLedger(false, 10, 0, 0, 1s, 1s, 1s, 10s, j));
BEAST_EXPECT(shouldCloseLedger(false, 10, 0, 0, 1s, 10s, 1s, 10s, j));
// Enforce minimum ledger open time
BEAST_EXPECT(!shouldCloseLedger(true, 10, 0, 0, 10s, 10s, 1s, 10s, j));
// Don't go too much faster than last time
BEAST_EXPECT(!shouldCloseLedger(true, 10, 0, 0, 10s, 10s, 3s, 10s, j));
BEAST_EXPECT(shouldCloseLedger(true, 10, 0, 0, 10s, 10s, 10s, 10s, j));
}
void testCheckConsensus()
{
// Not enough time has elapsed
BEAST_EXPECT( ConsensusState::No
== checkConsensus(10, 2, 2, 0, 3s, 2s, true, j));
// If not enough peers have propsed, ensure
// more time for proposals
BEAST_EXPECT( ConsensusState::No
== checkConsensus(10, 2, 2, 0, 3s, 4s, true, j));
// Enough time has elapsed and we all agree
BEAST_EXPECT( ConsensusState::Yes
== checkConsensus(10, 2, 2, 0, 3s, 10s, true, j));
// Enough time has elapsed and we don't yet agree
BEAST_EXPECT( ConsensusState::No
== checkConsensus(10, 2, 1, 0, 3s, 10s, true, j));
// Our peers have moved on
// Enough time has elapsed and we all agree
BEAST_EXPECT( ConsensusState::MovedOn
== checkConsensus(10, 2, 1, 8, 3s, 10s, true, j));
// No peers makes it easy to agree
BEAST_EXPECT( ConsensusState::Yes
== checkConsensus(0, 0, 0, 0, 3s, 10s, true, j));
}
void
run() override
{
testGetNextLedgerTimeResolution();
testRoundCloseTime();
testShouldCloseLedger();
testCheckConsensus();
}
};
BEAST_DEFINE_TESTSUITE(LedgerTiming, consensus, ripple);
} // test
} // ripple

26
src/test/csf.h Normal file
View File

@@ -0,0 +1,26 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2017 Ripple Labs Inc
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <test/csf/Tx.h>
#include <test/csf/Ledger.h>
#include <test/csf/BasicNetwork.h>
#include <test/csf/Peer.h>
#include <test/csf/UNL.h>
#include <test/csf/Sim.h>
#include <test/csf/Peer.h>

79
src/test/jtx/BasicNetwork.h → src/test/csf/BasicNetwork.h
View File

@@ -17,8 +17,8 @@
*/
//==============================================================================
#ifndef RIPPLE_TEST_BASICNETWORK_H_INCLUDED
#define RIPPLE_TEST_BASICNETWORK_H_INCLUDED
#ifndef RIPPLE_TEST_CSF_BASICNETWORK_H_INCLUDED
#define RIPPLE_TEST_CSF_BASICNETWORK_H_INCLUDED
#include <ripple/basics/qalloc.h>
#include <ripple/beast/clock/manual_clock.h>
@@ -35,14 +35,13 @@
#include <cassert>
#include <cstdint>
#include <iomanip>
#include <random>
#include <type_traits>
#include <unordered_map>
#include <unordered_set>
namespace ripple {
namespace test {
namespace csf {
/** Peer to peer network simulator.
The network is formed from a set of Peer objects representing
@@ -73,8 +72,8 @@ namespace test {
After creating the Peer set, constructing the network,
and establishing connections, the caller uses one or more
of the step, step_one, step_for, and step_until functions
to iterate the network,
of the step, step_one, step_for, step_until and step_while
functions to iterate the network,
Peer Requirements:
@@ -263,7 +262,6 @@ private:
// VFALCO This is an ugly wart, aged containers
// want a non-const reference to a clock.
clock_type mutable clock_;
std::mt19937_64 rng_;
std::unordered_map<Peer, links_type> links_;
public:
@@ -272,10 +270,6 @@ public:
BasicNetwork();
/** A source of pseudo-random numbers. */
std::mt19937_64&
rng();
/** Return the allocator. */
qalloc const&
alloc() const;
@@ -291,14 +285,6 @@ public:
time_point
now() const;
/** Return a random integer in range [0, n) */
std::size_t
rand (std::size_t n);
/** Return a random integer in range [first, last) */
std::size_t
rand (std::size_t first, std::size_t last);
/** Connect two peers.
The link is directed, with `from` establishing
@@ -447,6 +433,23 @@ public:
bool
step();
/** Run the network while a condition is true.
Function takes no arguments and will be called
repeatedly after each message is processed to
decide whether to continue.
Effects:
The clock is advanced to the time
of the last delivered message.
@return `true` if any message was processed.
*/
template <class Function>
bool
step_while(Function && func);
/** Run the network until the specified time.
Effects:
@@ -683,14 +686,6 @@ BasicNetwork<Peer>::BasicNetwork()
{
}
template <class Peer>
inline
std::mt19937_64&
BasicNetwork<Peer>::rng()
{
return rng_;
}
template <class Peer>
inline
qalloc const&
@@ -717,24 +712,6 @@ BasicNetwork<Peer>::now() const ->
return clock_.now();
}
template <class Peer>
inline
std::size_t
BasicNetwork<Peer>::rand(std::size_t n)
{
return std::uniform_int_distribution<
std::size_t>(0, n - 1)(rng_);
}
template <class Peer>
inline
std::size_t
BasicNetwork<Peer>::rand(
std::size_t first, std::size_t last)
{
return first + rand(last - first);
}
template <class Peer>
bool
BasicNetwork<Peer>::connect(
@@ -857,6 +834,17 @@ BasicNetwork<Peer>::step()
return true;
}
template <class Peer>
template <class Function>
bool
BasicNetwork<Peer>::step_while(Function && f)
{
bool ran = false;
while (f() && step_one())
ran = true;
return ran;
}
template <class Peer>
bool
BasicNetwork<Peer>::step_until(
@@ -922,6 +910,7 @@ BasicNetwork<Peer>::bfs(
}
}
} // csf
} // test
} // ripple

6
src/test/jtx/BasicNetwork_test.cpp → src/test/csf/BasicNetwork_test.cpp
View File

@@ -18,7 +18,7 @@
//==============================================================================
#include <BeastConfig.h>
#include <test/jtx/BasicNetwork.h>
#include <test/csf/BasicNetwork.h>
#include <ripple/beast/unit_test.h>
#include <set>
#include <vector>
@@ -26,6 +26,7 @@
namespace ripple {
namespace test {
class BasicNetwork_test : public beast::unit_test::suite
{
public:
@@ -87,8 +88,7 @@ public:
pv.emplace_back(0);
pv.emplace_back(1);
pv.emplace_back(2);
BasicNetwork<Peer*> net;
BEAST_EXPECT(net.rand(0, 1) == 0);
csf::BasicNetwork<Peer*> net;
BEAST_EXPECT(! net.connect(&pv[0], &pv[0]));
BEAST_EXPECT(net.connect(&pv[0], &pv[1], 1s));
BEAST_EXPECT(net.connect(&pv[1], &pv[2], 1s));

197
src/test/csf/Ledger.h Normal file
View File

@@ -0,0 +1,197 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2017 Ripple Labs Inc
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_TEST_CSF_LEDGER_H_INCLUDED
#define RIPPLE_TEST_CSF_LEDGER_H_INCLUDED
#include <ripple/basics/chrono.h>
#include <test/csf/Tx.h>
namespace ripple {
namespace test {
namespace csf {
/** A ledger is a set of observed transactions and a sequence number
identifying the ledger.
Peers in the consensus process are trying to agree on a set of transactions
to include in a ledger. For unit testing, each transaction is a
single integer and the ledger is a set of observed integers. This means
future ledgers have prior ledgers as subsets, e.g.
Ledger 0 : {}
Ledger 1 : {1,4,5}
Ledger 2 : {1,2,4,5,10}
....
Tx - Integer
TxSet - Set of Tx
Ledger - Set of Tx and sequence number
*/
class Ledger
{
public:
struct ID
{
std::uint32_t seq = 0;
TxSetType txs = TxSetType{};
bool operator==(ID const & o) const
{
return seq == o.seq && txs == o.txs;
}
bool operator!=(ID const & o) const
{
return !(*this == o);
}
bool operator<(ID const & o) const
{
return std::tie(seq, txs) < std::tie(o.seq, o.txs);
}
};
auto const &
id() const
{
return id_;
}
auto
seq() const
{
return id_.seq;
}
auto
closeTimeResolution() const
{
return closeTimeResolution_;
}
auto
closeAgree() const
{
return closeTimeAgree_;
}
auto
closeTime() const
{
return closeTime_;
}
auto
actualCloseTime() const
{
return actualCloseTime_;
}
auto
parentCloseTime() const
{
return parentCloseTime_;
}
auto const &
parentID() const
{
return parentID_;
}
Json::Value
getJson() const
{
Json::Value res(Json::objectValue);
res["seq"] = seq();
return res;
}
//! Apply the given transactions to this ledger
Ledger
close(TxSetType const & txs,
NetClock::duration closeTimeResolution,
NetClock::time_point const & consensusCloseTime,
bool closeTimeAgree) const
{
Ledger res{ *this };
res.id_.txs.insert(txs.begin(), txs.end());
res.id_ .seq= seq() + 1;
res.closeTimeResolution_ = closeTimeResolution;
res.actualCloseTime_ = consensusCloseTime;
res.closeTime_ = effectiveCloseTime(consensusCloseTime,
closeTimeResolution, parentCloseTime_);
res.closeTimeAgree_ = closeTimeAgree;
res.parentCloseTime_ = closeTime();
res.parentID_ = id();
return res;
}
private:
//! Unique identifier of ledger is combination of sequence number and id
ID id_;
//! Bucket resolution used to determine close time
NetClock::duration closeTimeResolution_ = ledgerDefaultTimeResolution;
//! When the ledger closed
NetClock::time_point closeTime_;
//! Whether consenssus agreed on the close time
bool closeTimeAgree_ = true;
//! Parent ledger id
ID parentID_;
//! Parent ledger close time
NetClock::time_point parentCloseTime_;
//! Close time unadjusted by closeTimeResolution
NetClock::time_point actualCloseTime_;
};
inline
std::ostream &
operator<<(std::ostream & o, Ledger::ID const & id)
{
return o << id.seq << "," << id.txs;
}
inline
std::string
to_string(Ledger::ID const & id)
{
std::stringstream ss;
ss << id;
return ss.str();
}
} // csf
} // test
} // ripple
#endif

492
src/test/csf/Peer.h Normal file
View File

@@ -0,0 +1,492 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2017 Ripple Labs Inc
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_TEST_CSF_PEER_H_INCLUDED
#define RIPPLE_TEST_CSF_PEER_H_INCLUDED
#include <boost/container/flat_map.hpp>
#include <boost/container/flat_set.hpp>
#include <test/csf/Tx.h>
#include <test/csf/Ledger.h>
#include <test/csf/UNL.h>
namespace ripple {
namespace test {
namespace csf {
/** Store validations reached by peers */
struct Validation
{
PeerID id;
Ledger::ID ledger;
Ledger::ID prevLedger;
};
namespace bc = boost::container;
class Validations
{
//< Ledgers seen by peers, saved in order received (which should be order
//< created)
bc::flat_map<Ledger::ID, bc::flat_set<PeerID>> nodesFromLedger;
bc::flat_map<Ledger::ID, bc::flat_set<PeerID>> nodesFromPrevLedger;
bc::flat_map<Ledger::ID, bc::flat_map<Ledger::ID, std::size_t>> childLedgers;
public:
void
update(Validation const & v)
{
nodesFromLedger[v.ledger].insert(v.id);
if(v.ledger.seq > 0)
{
nodesFromPrevLedger[v.prevLedger].insert(v.id);
childLedgers[v.prevLedger][v.ledger]++;
}
}
//< The number of peers who have validated this ledger
std::size_t
proposersValidated(Ledger::ID const & prevLedger) const
{
auto it = nodesFromLedger.find(prevLedger);
if(it != nodesFromLedger.end())
return it->second.size();
return 0;
}
/** The number of peers that are past this ledger, i.e.
they have a newer most recent ledger, but have this ledger
as an ancestor.
*/
std::size_t
proposersFinished(Ledger::ID const & prevLedger) const
{
auto it = nodesFromPrevLedger.find(prevLedger);
if(it != nodesFromPrevLedger.end())
return it->second.size();
return 0;
}
/** Returns the ledger starting from prevLedger with the most validations.
*/
Ledger::ID
getBestLCL(Ledger::ID const & currLedger,
Ledger::ID const & prevLedger) const
{
auto it = childLedgers.find(prevLedger);
if (it != childLedgers.end() &&
! it->second.empty())
{
std::size_t bestCount = 0;
Ledger::ID bestLedger;
for (auto const & b : it->second)
{
auto currCount = b.second;
if(currLedger == b.first)
currCount++;
if(currCount > bestCount)
bestLedger = b.first;
if(currCount == bestCount && currLedger == b.first)
bestLedger = b.first;
}
return bestLedger;
}
return currLedger;
}
};
/** Proposal is a position taken in the consensus process and is represented
directly from the generic types.
*/
using Proposal = ConsensusProposal<PeerID, Ledger::ID, TxSetType>;
struct Traits
{
using Ledger_t = Ledger;
using NodeID_t = PeerID;
using TxSet_t = TxSet;
using MissingTxException_t = MissingTx;
};
/** Represents a single node participating in the consensus process.
It implements the Callbacks required by Consensus.
*/
struct Peer : public Consensus<Peer, Traits>
{
using Base = Consensus<Peer, Traits>;
//! Our unique ID
PeerID id;
//! openTxs that haven't been closed in a ledger yet
TxSetType openTxs;
//! last ledger this peer closed
Ledger lastClosedLedger;
//! Handle to network for sending messages
BasicNetwork<Peer*> & net;
//! UNL of trusted peers
UNL unl;
//! Most recent ledger completed by peers
Validations peerValidations;
// The ledgers, proposals, TxSets and Txs this peer has seen
bc::flat_map<Ledger::ID, Ledger> ledgers;
//! Map from Ledger::ID to vector of Positions with that ledger
//! as the prior ledger
bc::flat_map<Ledger::ID, std::vector<Proposal>> peerPositions_;
bc::flat_map<TxSet::ID, TxSet> txSets;
int completedLedgers = 0;
int targetLedgers = std::numeric_limits<int>::max();
//! Skew samples from the network clock; to be refactored into a
//! clock time once it is provided separately from the network.
std::chrono::seconds clockSkew{0};
//! Delay in processing validations from remote peers
std::chrono::milliseconds validationDelay{0};
//! Delay in acquiring missing ledger from the network
std::chrono::milliseconds missingLedgerDelay{0};
bool validating = true;
bool proposing = true;
//! All peers start from the default constructed ledger
Peer(PeerID i, BasicNetwork<Peer*> & n, UNL const & u)
: Consensus<Peer, Traits>( n.clock(), beast::Journal{})
, id{i}
, net{n}
, unl(u)
{
ledgers[lastClosedLedger.id()] = lastClosedLedger;
}
// @return whether we are proposing,validating
// TODO: Bit akward that this is in callbacks, would be nice to extract
std::pair<bool, bool>
getMode()
{
// in RCL this hits NetworkOps to decide whether we are proposing
// validating
return{ proposing, validating };
}
Ledger const *
acquireLedger(Ledger::ID const & ledgerHash)
{
auto it = ledgers.find(ledgerHash);
if (it != ledgers.end())
return &(it->second);
// TODO Get from network/oracle properly!
for (auto const& link : net.links(this))
{
auto const & p = *link.to;
auto it = p.ledgers.find(ledgerHash);
if (it != p.ledgers.end())
{
schedule(missingLedgerDelay,
[this, ledgerHash, ledger = it->second]()
{
ledgers.emplace(ledgerHash, ledger);
});
if(missingLedgerDelay == 0ms)
return &ledgers[ledgerHash];
break;
}
}
return nullptr;
}
auto const &
proposals(Ledger::ID const & ledgerHash)
{
return peerPositions_[ledgerHash];
}
TxSet const *
acquireTxSet(TxSet::ID const & setId)
{
auto it = txSets.find(setId);
if(it != txSets.end())
return &(it->second);
// TODO Get from network/oracle instead!
return nullptr;
}
bool
hasOpenTransactions() const
{
return !openTxs.empty();
}
std::size_t
proposersValidated(Ledger::ID const & prevLedger)
{
return peerValidations.proposersValidated(prevLedger);
}
std::size_t
proposersFinished(Ledger::ID const & prevLedger)
{
return peerValidations.proposersFinished(prevLedger);
}
void
onStartRound(Ledger const &) {}
void
onClose(Ledger const &, bool ) {}
// don't really offload
void
dispatchAccept(TxSet const & f)
{
Base::accept(f);
}
void
share(TxSet const &s)
{
relay(s);
}
Ledger::ID
getLCL(Ledger::ID const & currLedger,
Ledger::ID const & priorLedger,
bool haveCorrectLCL)
{
// TODO: Use generic validation code
if(currLedger.seq > 0 && priorLedger.seq > 0)
return peerValidations.getBestLCL(currLedger, priorLedger);
return currLedger;
}
void
propose(Proposal const & pos)
{
if(proposing)
relay(pos);
}
void
relay(DisputedTx<Tx, PeerID> const & dispute)
{
relay(dispute.tx());
}
std::pair <TxSet, Proposal>
makeInitialPosition(
Ledger const & prevLedger,
bool isProposing,
bool isCorrectLCL,
NetClock::time_point closeTime,
NetClock::time_point now)
{
TxSet res{ openTxs };
return { res,
Proposal{prevLedger.id(), Proposal::seqJoin, res.id(), closeTime, now, id} };
}
// Process the accepted transaction set, generating the newly closed ledger
// and clearing out the openTxs that were included.
// TODO: Kinda nasty it takes so many arguments . . . sign of bad coupling
bool
accept(TxSet const& set,
NetClock::time_point consensusCloseTime,
bool proposing_,
bool validating_,
bool haveCorrectLCL_,
bool consensusFail_,
Ledger::ID const & prevLedgerHash_,
Ledger const & previousLedger_,
NetClock::duration closeResolution_,
NetClock::time_point const & now,
std::chrono::milliseconds const & roundTime_,
hash_map<Tx::ID, DisputedTx <Tx, PeerID>> const & disputes_,
std::map <NetClock::time_point, int> closeTimes_,
NetClock::time_point const & closeTime)
{
auto newLedger = previousLedger_.close(set.txs_, closeResolution_,
closeTime, consensusCloseTime != NetClock::time_point{});
ledgers[newLedger.id()] = newLedger;
lastClosedLedger = newLedger;
auto it = std::remove_if(openTxs.begin(), openTxs.end(),
[&](Tx const & tx)
{
return set.exists(tx.id());
});
openTxs.erase(it, openTxs.end());
if(validating)
relay(Validation{id, newLedger.id(), newLedger.parentID()});
return validating_;
}
void
endConsensus(bool correct)
{
// kick off the next round...
// in the actual implementation, this passes back through
// network ops
++completedLedgers;
// startRound sets the LCL state, so we need to call it once after
// the last requested round completes
// TODO: reconsider this and instead just save LCL generated here?
if(completedLedgers <= targetLedgers)
{
startRound(now(), lastClosedLedger.id(),
lastClosedLedger);
}
}
//-------------------------------------------------------------------------
// non-callback helpers
void
receive(Proposal const & p)
{
if(unl.find(p.nodeID()) == unl.end())
return;
// TODO: Be sure this is a new proposal!!!!!
auto & dest = peerPositions_[p.prevLedger()];
if(std::find(dest.begin(), dest.end(), p) != dest.end())
return;
dest.push_back(p);
peerProposal(now(), p);
}
void
receive(TxSet const & txs)
{
// save and map complete?
auto it = txSets.insert(std::make_pair(txs.id(), txs));
if(it.second)
gotTxSet(now(), txs);
}
void
receive(Tx const & tx)
{
if (openTxs.find(tx.id()) == openTxs.end())
{
openTxs.insert(tx);
// relay to peers???
relay(tx);
}
}
void
receive(Validation const & v)
{
if(unl.find(v.id) != unl.end())
{
schedule(validationDelay,
[&, v]()
{
peerValidations.update(v);
});
}
}
template <class T>
void
relay(T const & t)
{
for(auto const& link : net.links(this))
net.send(this, link.to,
[msg = t, to = link.to]
{
to->receive(msg);
});
}
// Receive and relay locally submitted transaction
void
submit(Tx const & tx)
{
receive(tx);
relay(tx);
}
void
timerEntry()
{
Base::timerEntry(now());
// only reschedule if not completed
if(completedLedgers < targetLedgers)
net.timer(LEDGER_GRANULARITY, [&]() { timerEntry(); });
}
void
start()
{
net.timer(LEDGER_GRANULARITY, [&]() { timerEntry(); });
// The ID is the one we have seen the most validations for
// In practice, we might not actually have that ledger itself yet,
// so there is no gaurantee that bestLCL == lastClosedLedger.id()
auto bestLCL = peerValidations.getBestLCL(lastClosedLedger.id(),
lastClosedLedger.parentID());
startRound(now(), bestLCL,
lastClosedLedger);
}
NetClock::time_point
now() const
{
// We don't care about the actual epochs, but do want the
// generated NetClock time to be well past its epoch to ensure
// any subtractions of two NetClock::time_point in the consensu
// code are positive. (e.g. PROPOSE_FRESHNESS)
using namespace std::chrono;
return NetClock::time_point(duration_cast<NetClock::duration>
(net.now().time_since_epoch()+ 86400s + clockSkew));
}
// Schedule the provided callback in `when` duration, but if
// `when` is 0, call immediately
template <class T>
void schedule(std::chrono::nanoseconds when, T && what)
{
if(when == 0ns)
what();
else
net.timer(when, std::forward<T>(what));
}
};
} // csf
} // test
} // ripple
#endif

104
src/test/csf/Sim.h Normal file
View File

@@ -0,0 +1,104 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2017 Ripple Labs Inc
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_TEST_CSF_SIM_H_INCLUDED
#define RIPPLE_TEST_CSF_SIM_H_INCLUDED
#include <test/csf/UNL.h>
#include <test/csf/BasicNetwork.h>
namespace ripple {
namespace test {
namespace csf {
class Sim
{
public:
/** Create a simulator for the given trust graph and network topology.
Create a simulator for consensus over the given trust graph and connect
the network links between nodes based on the provided topology.
Topology is is a functor with signature
boost::optional<std::chrono::duration> (NodeId i, NodeId j)
that returns the delay sending messages from node i to node j.
In general, this network graph is distinct from the trust graph, but
users can use adaptors to present a TrustGraph as a Topology by
specifying the delay between nodes.
@param g The trust graph between peers.
@param top The network topology between peers.
*/
template <class Topology>
Sim(TrustGraph const & g, Topology const & top)
{
peers.reserve(g.numPeers());
for(int i = 0; i < g.numPeers(); ++i)
peers.emplace_back(i, net, g.unl(i));
for(int i = 0; i < peers.size(); ++i)
{
for(int j = 0; j < peers.size(); ++j)
{
if( i != j)
{
auto d = top(i,j);
if (d)
{
net.connect(&peers[i], &peers[j], *d);
}
}
}
}
}
/** Run consensus protocol to generate the provided number of ledgers.
Has each peer run consensus until it creates `ledgers` more ledgers.
@param ledgers The number of additional ledgers to create
*/
void
run(int ledgers)
{
for (auto & p : peers)
{
if(p.completedLedgers == 0)
p.relay(Validation{p.id, p.LCL(), p.LCL()});
p.targetLedgers = p.completedLedgers + ledgers;
p.start();
}
net.step();
}
std::vector<Peer> peers;
BasicNetwork<Peer*> net;
};
} // csf
} // test
} // ripple
#endif

221
src/test/csf/Tx.h Normal file
View File

@@ -0,0 +1,221 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2017 Ripple Labs Inc
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_TEST_CSF_TX_H_INCLUDED
#define RIPPLE_TEST_CSF_TX_H_INCLUDED
#include <ripple/beast/hash/hash_append.h>
#include <boost/container/flat_set.hpp>
#include <ostream>
#include <string>
#include <map>
namespace ripple {
namespace test {
namespace csf {
//! A single transaction
class Tx
{
public:
using ID = std::uint32_t;
Tx(ID i) : id_{ i } {}
ID
id() const
{
return id_;
}
bool
operator<(Tx const & o) const
{
return id_ < o.id_;
}
bool
operator==(Tx const & o) const
{
return id_ == o.id_;
}
private:
ID id_;
};
//!-------------------------------------------------------------------------
//! All sets of Tx are represented as a flat_set.
using TxSetType = boost::container::flat_set<Tx>;
//! TxSet is a set of transactions to consider including in the ledger
class TxSet
{
public:
using ID = TxSetType;
using Tx = csf::Tx;
using MutableTxSet = TxSet;
TxSet() = default;
TxSet(TxSetType const & s) : txs_{ s } {}
bool
insert(Tx const & t)
{
return txs_.insert(t).second;
}
bool
erase(Tx::ID const & txId)
{
return txs_.erase(Tx{ txId }) > 0;
}
bool
exists(Tx::ID const txId) const
{
auto it = txs_.find(Tx{ txId });
return it != txs_.end();
}
Tx const *
find(Tx::ID const& txId) const
{
auto it = txs_.find(Tx{ txId });
if (it != txs_.end())
return &(*it);
return nullptr;
}
auto const &
id() const
{
return txs_;
}
/** @return Map of Tx::ID that are missing. True means
it was in this set and not other. False means
it was in the other set and not this
*/
std::map<Tx::ID, bool>
compare(TxSet const& other) const
{
std::map<Tx::ID, bool> res;
auto populate_diffs = [&res](auto const & a, auto const & b, bool s)
{
auto populator = [&](auto const & tx)
{
res[tx.id()] = s;
};
std::set_difference(
a.begin(), a.end(),
b.begin(), b.end(),
boost::make_function_output_iterator(
std::ref(populator)
)
);
};
populate_diffs(txs_, other.txs_, true);
populate_diffs(other.txs_, txs_, false);
return res;
}
//! The set contains the actual transactions
TxSetType txs_;
};
/** The RCL consensus process catches missing node SHAMap error
in several points. This exception is meant to represent a similar
case for the unit test.
*/
class MissingTx : public std::runtime_error
{
public:
MissingTx()
: std::runtime_error("MissingTx")
{}
};
//------------------------------------------------------------------------------
// Helper functions for debug printing
inline
std::ostream&
operator<<(std::ostream & o, const Tx & t)
{
return o << t.id();
}
template <class T>
inline
std::ostream&
operator<<(std::ostream & o, boost::container::flat_set<T> const & ts)
{
o << "{ ";
bool do_comma = false;
for (auto const & t : ts)
{
if (do_comma)
o << ", ";
else
do_comma = true;
o << t;
}
o << " }";
return o;
}
inline
std::string
to_string(TxSetType const & txs)
{
std::stringstream ss;
ss << txs;
return ss.str();
}
template <class Hasher>
inline
void
hash_append(Hasher& h, Tx const & tx)
{
using beast::hash_append;
hash_append(h, tx.id());
}
std::ostream&
operator<<(std::ostream & o, MissingTx const &m)
{
return o << m.what();
}
} // csf
} // test
} // ripple
#endif

277
src/test/csf/UNL.h Normal file
View File

@@ -0,0 +1,277 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2017 Ripple Labs Inc
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_TEST_CSF_UNL_H_INCLUDED
#define RIPPLE_TEST_CSF_UNL_H_INCLUDED
#include <boost/container/flat_set.hpp>
#include <boost/optional.hpp>
#include <vector>
#include <random>
#include <numeric>
#include <chrono>
namespace ripple {
namespace test {
namespace csf {
/** Return a randomly shuffled copy of vector based on weights w.
@param v The set of values
@param w The set of weights of each value
@param g A pseudo-random number generator
@return A vector with entries randomly sampled without replacement
from the original vector based on the provided weights.
I.e. res[0] comes from sample v[i] with weight w[i]/sum_k w[k]
*/
template <class T, class G>
std::vector<T>
random_weighted_shuffle(std::vector<T> v, std::vector<double> w, G & g)
{
using std::swap;
for (int i = 0; i < v.size() - 1; ++i)
{
// pick a random item weighted by w
std::discrete_distribution<> dd(w.begin() + i, w.end());
auto idx = dd(g);
std::swap(v[i], v[idx]);
std::swap(w[i], w[idx]);
}
return v;
}
/** Power-law distribution with PDF
P(x) = (x/xmin)^-a
for a >= 1 and xmin >= 1
*/
class PowerLawDistribution
{
double xmin_;
double a_;
double inv_;
std::uniform_real_distribution<double> uf_{0,1};
public:
PowerLawDistribution(double xmin, double a)
: xmin_{xmin}, a_{a}
{
inv_ = 1.0/(1.0 - a_);
}
template <class Generator>
inline
double
operator()(Generator & g)
{
// use inverse transform of CDF to sample
// CDF is P(X <= x): 1 - (x/xmin)^(1-a)
return xmin_ * std::pow(1 - uf_(g), inv_);
}
};
//< Unique identifier for each node in the network
using PeerID = std::uint32_t;
//< A unique node list defines a set of trusted peers used in consensus
using UNL = boost::container::flat_set<PeerID>;
/** Trust graph defining the consensus simulation
Trust is a directed relationship from a node i to node j.
If node i trusts node j, then node i has node j in its UNL.
Note that each node implicitly trusts itself but that need not be
explicitly modeled, e.g. UNLS[assignment
*/
class TrustGraph
{
//< Unique UNLs for the network
std::vector<UNL> UNLs_;
std::vector<int> assignment_;
public:
//< Constructor
TrustGraph(std::vector<UNL> UNLs, std::vector<int> assignment)
: UNLs_{UNLs}
, assignment_{assignment}
{}
//< Whether node `i` trusts node `j`
inline
bool
trusts(PeerID i, PeerID j) const
{
return unl(i).find(j) != unl(i).end();
}
//< Get the UNL for node `i`
inline
UNL const &
unl(PeerID i) const
{
return UNLs_[assignment_[i]];
}
//< Check whether this trust graph satisfies the no forking condition
bool
canFork(double quorum) const;
auto
numPeers() const
{
return assignment_.size();
}
//< Save grapviz dot file reprentation of the trust graph
void
save_dot(std::string const & fileName);
/** Generate a random trust graph based on random ranking of peers
Generate a random trust graph by
1. Randomly ranking the peers acording to RankPDF
2. Generating `numUNL` random UNLs by sampling without replacement
from the ranked nodes.
3. Restricting the size of the random UNLs according to SizePDF
@param size The number of nodes in the trust graph
@param numUNLs The number of UNLs to create
@param rankPDF Generates random positive real numbers to use as ranks
@param unlSizePDF Generates random integeres between (0,size-1) to
restrict the size of generated PDF
@param Generator The uniform random bit generator to use
@note RankPDF/SizePDF can model the full RandomDistribution concept
defined in the STL, but for the purposes of this function need
only provide:
auto operator()(Generator & g)
which should return the random sample.
*/
template <class RankPDF, class SizePDF, class Generator>
static
TrustGraph
makeRandomRanked(int size,
int numUNLs,
RankPDF rankPDF,
SizePDF unlSizePDF,
Generator & g)
{
// 1. Generate ranks
std::vector<double> weights(size);
std::generate(weights.begin(), weights.end(), [&]()
{
return rankPDF(g);
});
// 2. Generate UNLs based on sampling without replacement according
// to weights
std::vector<UNL> unls(numUNLs);
std::generate(unls.begin(), unls.end(), [&]()
{
std::vector<PeerID> ids(size);
std::iota(ids.begin(), ids.end(), 0);
auto res = random_weighted_shuffle(ids, weights, g);
return UNL(res.begin(), res.begin() + unlSizePDF(g));
});
// 3. Assign membership
std::vector<int> assignment(size);
std::uniform_int_distribution<int> u(0, numUNLs-1);
std::generate(assignment.begin(), assignment.end(),
[&]()
{
return u(g);
});
return TrustGraph(unls, assignment);
}
/** Generate a 2 UNL trust graph with some overlap.
Generates a trust graph for `size` peers formed from
two cliques with the given overlap. Nodes in the overlap
trust both all other nodes, while nodes outside the overlap
only trust nodes in their clique.
@param size The number of nodes in the trust graph
@param overlap The number of nodes trusting both cliques
*/
static
TrustGraph
makeClique(int size, int overlap);
/** Generate a complete (fully-connect) trust graph
Generatest a trust graph in which all peers trust all
other peers.
@param size The number of nodes in the trust graph
*/
static
TrustGraph
makeComplete(int size);
};
//< Make the TrustGraph into a topology with delays given by DelayModel
template <class DelayModel>
auto
topology(TrustGraph const & tg, DelayModel const & d)
{
return [&](PeerID i, PeerID j)
{
return tg.trusts(i,j) ? boost::make_optional(d(i,j)) : boost::none;
};
}
class fixed
{
std::chrono::nanoseconds d_;
public:
fixed(std::chrono::nanoseconds const & d) : d_{d} {}
inline
std::chrono::nanoseconds
operator()(PeerID const & i, PeerID const & j) const
{
return d_;
}
};
} // csf
} // test
} // ripple
#endif

135
src/test/csf/impl/UNL.cpp Normal file
View File

@@ -0,0 +1,135 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2017 Ripple Labs Inc
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <test/csf/UNL.h>
#include <boost/iterator/counting_iterator.hpp>
#include <fstream>
#include <algorithm>
namespace ripple {
namespace test {
namespace csf {
bool
TrustGraph::canFork(double quorum) const
{
// Check the forking condition by looking at intersection
// between all pairs of UNLs.
// First check if some nodes uses a UNL they are not members of, since
// this creates an implicit UNL with that ndoe.
auto uniqueUNLs = UNLs_;
for (int i = 0; i < assignment_.size(); ++i)
{
auto const & myUNL = UNLs_[assignment_[i]];
if(myUNL.find(i) == myUNL.end())
{
auto myUNLcopy = myUNL;
myUNLcopy.insert(i);
uniqueUNLs.push_back(std::move(myUNLcopy));
}
}
// Loop over all pairs of uniqueUNLs
for (int i = 0; i < uniqueUNLs.size(); ++i)
{
for (int j = (i+1); j < uniqueUNLs.size(); ++j)
{
auto const & unlA = uniqueUNLs[i];
auto const & unlB = uniqueUNLs[j];
double rhs = 2.0*(1.-quorum) *
std::max(unlA.size(), unlB.size() );
int intersectionSize = std::count_if(unlA.begin(), unlA.end(),
[&](PeerID id)
{
return unlB.find(id) != unlB.end();
});
if(intersectionSize < rhs)
return true;
}
}
return false;
}
TrustGraph
TrustGraph::makeClique(int size, int overlap)
{
using bci = boost::counting_iterator<PeerID>;
// Split network into two cliques with the given overlap
// Clique A has nodes [0,endA) and Clique B has [startB,numPeers)
// Note: Clique B will have an extra peer when numPeers - overlap
// is odd
int endA = (size + overlap)/2;
int startB = (size - overlap)/2;
std::vector<UNL> unls;
unls.emplace_back(bci(0), bci(endA));
unls.emplace_back(bci(startB), bci(size));
unls.emplace_back(bci(0), bci(size));
std::vector<int> assignment(size,0);
for (int i = 0; i < size; ++i)
{
if(i < startB)
assignment[i] = 0;
else if(i > endA)
assignment[i] = 1;
else
assignment[i] = 2;
}
return TrustGraph(unls, assignment);
}
TrustGraph
TrustGraph::makeComplete(int size)
{
UNL all{ boost::counting_iterator<PeerID>( 0 ),
boost::counting_iterator<PeerID>( size ) };
return TrustGraph(std::vector<UNL>(1,all),
std::vector<int>(size, 0));
}
inline void TrustGraph::save_dot(std::string const & fileName)
{
std::ofstream out(fileName);
out << "digraph {\n";
for (int i = 0; i < assignment_.size(); ++i)
{
for (auto & j : UNLs_[assignment_[i]])
{
out << i << " -> " << j << ";\n";
}
}
out << "}\n";
}
} // csf
} // test
} // ripple

2
src/test/jtx/impl/Env.cpp
View File

@@ -30,7 +30,7 @@
#include <test/jtx/utility.h>
#include <test/jtx/JSONRPCClient.h>
#include <ripple/app/ledger/LedgerMaster.h>
#include <ripple/app/ledger/LedgerTiming.h>
#include <ripple/consensus/LedgerTiming.h>
#include <ripple/app/misc/NetworkOPs.h>
#include <ripple/app/misc/TxQ.h>
#include <ripple/basics/contract.h>

21
src/test/unity/consensus_test_unity.cpp Normal file
View File

@@ -0,0 +1,21 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2016 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <test/consensus/Consensus_test.cpp>
#include <test/consensus/LedgerTiming_test.cpp>

23
src/test/unity/csf_unity.cpp Normal file
View File

@@ -0,0 +1,23 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2017 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <BeastConfig.h>
#include <test/csf/BasicNetwork_test.cpp>
#include <test/csf/impl/UNL.cpp>

2
src/test/unity/support_unity.cpp → src/test/unity/jtx_unity.cpp
View File

@@ -49,7 +49,5 @@
#include <test/jtx/impl/JSONRPCClient.cpp>
#include <test/jtx/impl/ManualTimeKeeper.cpp>
#include <test/jtx/impl/WSClient.cpp>
#include <test/jtx/BasicNetwork_test.cpp>
#include <test/jtx/Env_test.cpp>
#include <test/jtx/WSClient_test.cpp>