ARCHIVE/rippled
1
0
Fork 0
mirror of https://github.com/XRPLF/rippled.git synced 2025-12-06 17:27:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
b9058045945c27a01d3620507f47e55634c65d89
rippled/md_ripple_nodestore_ShardSizeTuning.html
intelliot b905804594 deploy: 1f417764c3
2023-04-13 17:45:50 +00:00

249 lines
25 KiB
HTML
Raw Blame History

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.17"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>rippled: Shard size Tuning</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td id="projectalign" style="padding-left: 0.5em;">
<div id="projectname">rippled
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.17 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
var searchBox = new SearchBox("searchBox", "search",false,'Search');
/* @license-end */
</script>
<script type="text/javascript" src="menudata.js"></script>
<script type="text/javascript" src="menu.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
$(function() {
initMenu('',true,false,'search.php','Search');
$(document).ready(function() { init_search(); });
});
/* @license-end */</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
onmouseover="return searchBox.OnSearchSelectShow()"
onmouseout="return searchBox.OnSearchSelectHide()"
onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>
<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0"
name="MSearchResults" id="MSearchResults">
</iframe>
</div>
</div><!-- top -->
<div class="PageDoc"><div class="header">
<div class="headertitle">
<div class="title">Shard size Tuning </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>The purpose of this document is to compare the sizes of shards containing varying amounts of ledgers.</p>
<h1><a class="anchor" id="autotoc_md209"></a>
Methodology</h1>
<p>One can see visually from a block explorer that a typical mainnet ledger consists of about 30 offer transactions issued by about 8 different accounts, and several transactions of other types. To simulate this situation and similar situations we have constructed deterministic shards of differenet sizes, with varying amounts of offers per ledger and varying amounts of accounts issuing these offers.</p>
<p>In the following results table, the number of ledgers per shard ranges from 256 to 16K with the size doubling the size at each step. We considered the following numbers of offers per ledger: 0, 1, 5, 10 and 30. Also we considered case of 1 and 8 accounts issuing offers. For each constructed deterministic shard we counted its size. Finally we compared doubled size of the shard with N ledgers and the size of a shard with 2*N ledgers where othere parameters such as number of offers and accounts are the same. This comparison is sufficient to determine which number of ledgers per shard leads to less storage size on the disk.</p>
<p>Note that we minimize total storage size on the disk, but not the size of each shard because data below shows that the size of a typical shard is not larger than 10G, but sizes of modern disks, even SSDs, start from 250G. So there is no problem to fit a single shard to a disk, even small.</p>
<h1><a class="anchor" id="autotoc_md210"></a>
Raw results table</h1>
<p>All sizes of constructed shards are shown in the following table. Rows corresponds to shard sizes (S) counted in ledgers, columns corresponds to numbers of offers (O) per ledger. In each cell there are two numbers: first number corresponds to the case of 1 account issuing offers, the second number corresponds to 8 accounts. Each number is a size of the shard with given parameters measured in megabytes.</p>
<table class="markdownTable">
<tr class="markdownTableHead">
<th class="markdownTableHeadNone">S\O </th><th class="markdownTableHeadNone">0 </th><th class="markdownTableHeadNone">1 </th><th class="markdownTableHeadNone">5 </th><th class="markdownTableHeadNone">10 </th><th class="markdownTableHeadNone">30 </th></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">256 </td><td class="markdownTableBodyNone">2.2/2.2 </td><td class="markdownTableBodyNone">3.4/3.3 </td><td class="markdownTableBodyNone">5.3/7.3 </td><td class="markdownTableBodyNone">7.7/10.9 </td><td class="markdownTableBodyNone">17.1/21.9 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">512 </td><td class="markdownTableBodyNone">4.4/4.5 </td><td class="markdownTableBodyNone">7.0/7.0 </td><td class="markdownTableBodyNone">11.2/15.6 </td><td class="markdownTableBodyNone">16.4/23.7 </td><td class="markdownTableBodyNone">36.9/47.9 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">1K </td><td class="markdownTableBodyNone">8.9/9.0 </td><td class="markdownTableBodyNone">14.7/14.6 </td><td class="markdownTableBodyNone">23.7/33.6 </td><td class="markdownTableBodyNone">35.0/51.0 </td><td class="markdownTableBodyNone">78.2/ 102.9 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">2K </td><td class="markdownTableBodyNone">17.8/18.0 </td><td class="markdownTableBodyNone">30.5/30.7 </td><td class="markdownTableBodyNone">50.4/72.2 </td><td class="markdownTableBodyNone">74.3/ 111.9 </td><td class="markdownTableBodyNone">166.2/ 221.0 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">4K </td><td class="markdownTableBodyNone">35.5/35.9 </td><td class="markdownTableBodyNone">63.6/64.2 </td><td class="markdownTableBodyNone">106.2/ 154.8 </td><td class="markdownTableBodyNone">156.1/ 238.7 </td><td class="markdownTableBodyNone">354.7/ 476.0 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">8K </td><td class="markdownTableBodyNone">71.1/71.9 </td><td class="markdownTableBodyNone">133.4/ 134.5 </td><td class="markdownTableBodyNone">222.2/ 328.1 </td><td class="markdownTableBodyNone">329.1/ 511.2 </td><td class="markdownTableBodyNone">754.6/ 1021.0 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">16K </td><td class="markdownTableBodyNone">142.3/ 143.9 </td><td class="markdownTableBodyNone">279/9 280.8 </td><td class="markdownTableBodyNone">465.7/ 698.1 </td><td class="markdownTableBodyNone">696.4/ 1094.2 </td><td class="markdownTableBodyNone">1590.5/ 2166.6 </td></tr>
</table>
<h1><a class="anchor" id="autotoc_md211"></a>
Preliminary conclusion</h1>
<p>If one compares a doubled size of shard with N ledgers and a size of shard with 2*N ledgers anywhere in the above table than the conlusion will be that the second number is greater. For example, the following table shows the percentage by which the second number is greater for the most interesting case of 30 offers per ledger. The first row corresponds to the case of 1 account issuing offers, and the second row corresponds to the case of 8 issuing accounts.</p>
<p>|A\N|256|512|1K|2K|4K|8K| |&mdash;|&mdash;|&mdash;|&mdash;|&mdash;|&mdash;|&mdash;| |1|8%|6%|6%|6%|7%|6%|5%| |8|9%|7%|7%|8%|6%|7%|6%|</p>
<p>The common conclusion in this model is that if one doubled the number of the ledgers in a shard then the total disk space utilized will raise by 5-9%.</p>
<h1><a class="anchor" id="autotoc_md212"></a>
Adding accounts into consideration</h1>
<p>Previous model does not take into account that there are large number of XRP accounts in the mainnet, and each shard should contain information about each of these accounts. As of January 2020, there were about 1.9 million XRP accounts, and stored information about each of them is not less than 133 bytes. The constant 133 was obtained from debug print of rippled program when it saves account object to the database. So the actual size of each shard from raw table should be increased by at least 1.9M * 133 = 252.7M. Thus we obtained the following table of shard sizes for the most interesting case (30 offers per ledger and 8 issuing accounts) where S is shard size in ledgers and M is shard size in megabytes</p>
<table class="markdownTable">
<tr class="markdownTableHead">
<th class="markdownTableHeadNone">S </th><th class="markdownTableHeadNone">256 </th><th class="markdownTableHeadNone">512 </th><th class="markdownTableHeadNone">1K </th><th class="markdownTableHeadNone">2K </th><th class="markdownTableHeadNone">4K </th><th class="markdownTableHeadNone">8K </th><th class="markdownTableHeadNone">16K </th></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">M </td><td class="markdownTableBodyNone">274.6 </td><td class="markdownTableBodyNone">300.6 </td><td class="markdownTableBodyNone">355.6 </td><td class="markdownTableBodyNone">473.7 </td><td class="markdownTableBodyNone">728.7 </td><td class="markdownTableBodyNone">1273.7 </td><td class="markdownTableBodyNone">2419.3 </td></tr>
</table>
<p>Now we can see from the last table that even considering minimum assumption about number of accounts and corresponding additional size of a shard, doubled size of shard with N ledgers is larger than size of a shard with 2*N ledgers. If number of accounts increase then this inequality will be even stronger.</p>
<h1><a class="anchor" id="autotoc_md213"></a>
Using mainnet data</h1>
<p>Next idea to improve model is to count real shard sizes from mainnet. We used real 16k-ledgers shards with indexes from 2600 to 3600 with step 100, and corresponding real 8k-ledgers shards. Each 16k-ledgers shard consists of two 8k-ledgers shards which are called "corresponding". For example, 16k-ledgers shard with index 2600 consists of two 8k-ledgers shards with indexes 5200 and 5201.</p>
<p>In the following table we compare size of a 16k-ledgers shard with sum of sizes of two corresponding 8k-ledgers shards. There we only count size of nudb.dat file, sizes are in GB. Ratio is the size of two 8k-ledgers shards divided to the size of 16k-ledgers shard.</p>
<table class="markdownTable">
<tr class="markdownTableHead">
<th class="markdownTableHeadNone">Index </th><th class="markdownTableHeadNone">16k-ledgers </th><th class="markdownTableHeadNone">8k-ledgers sum </th><th class="markdownTableHeadNone">Ratio </th></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">2600 </td><td class="markdownTableBodyNone">2.39 </td><td class="markdownTableBodyNone">1.49 + 1.63 = 3.12 </td><td class="markdownTableBodyNone">1.31 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">2700 </td><td class="markdownTableBodyNone">2.95 </td><td class="markdownTableBodyNone">1.77 + 1.94 = 3.71 </td><td class="markdownTableBodyNone">1.26 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">2800 </td><td class="markdownTableBodyNone">2.53 </td><td class="markdownTableBodyNone">1.54 + 1.75 = 3.29 </td><td class="markdownTableBodyNone">1.30 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">2900 </td><td class="markdownTableBodyNone">3.83 </td><td class="markdownTableBodyNone">2.26 + 2.35 = 4.61 </td><td class="markdownTableBodyNone">1.20 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">3000 </td><td class="markdownTableBodyNone">4.49 </td><td class="markdownTableBodyNone">2.70 + 2.59 = 5.29 </td><td class="markdownTableBodyNone">1.18 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">3100 </td><td class="markdownTableBodyNone">3.79 </td><td class="markdownTableBodyNone">2.35 + 2.25 = 4.60 </td><td class="markdownTableBodyNone">1.21 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">3200 </td><td class="markdownTableBodyNone">4.15 </td><td class="markdownTableBodyNone">2.54 + 2.43 = 4.97 </td><td class="markdownTableBodyNone">1.20 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">3300 </td><td class="markdownTableBodyNone">5.19 </td><td class="markdownTableBodyNone">3.23 + 2.80 = 6.03 </td><td class="markdownTableBodyNone">1.16 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">3400 </td><td class="markdownTableBodyNone">4.18 </td><td class="markdownTableBodyNone">2.53 + 2.51 = 5.04 </td><td class="markdownTableBodyNone">1.21 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">3500 </td><td class="markdownTableBodyNone">5.06 </td><td class="markdownTableBodyNone">2.90 + 3.04 = 5.94 </td><td class="markdownTableBodyNone">1.17 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">3600 </td><td class="markdownTableBodyNone">4.18 </td><td class="markdownTableBodyNone">2.56 + 2.51 = 5.07 </td><td class="markdownTableBodyNone">1.21 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">Average </td><td class="markdownTableBodyNone">3.89 </td><td class="markdownTableBodyNone">2.35 + 2.35 = 4.70 </td><td class="markdownTableBodyNone">1.21 </td></tr>
</table>
<p>Note that shard on the disk consists of 4 files each of which can be large too. These files are nudb.dat, nudb.key, ledger.db, transaction.db. Next table is similar to previous with the following exception: each number is total size of these 2 files: nudb.dat and nudb.key. We decided not to count sizes of ledger.db and transaction.db since these sizes are not permanent instead of sizes of nudb.* which are permanent for deterministic shards.</p>
<table class="markdownTable">
<tr class="markdownTableHead">
<th class="markdownTableHeadNone">Index </th><th class="markdownTableHeadNone">16k-ledgers </th><th class="markdownTableHeadNone">8k-ledgers sum </th><th class="markdownTableHeadNone">Ratio </th></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">2600 </td><td class="markdownTableBodyNone">2.76 </td><td class="markdownTableBodyNone">1.73 + 1.89 = 3.62 </td><td class="markdownTableBodyNone">1.31 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">2700 </td><td class="markdownTableBodyNone">3.40 </td><td class="markdownTableBodyNone">2.05 + 2.25 = 4.30 </td><td class="markdownTableBodyNone">1.26 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">2800 </td><td class="markdownTableBodyNone">2.91 </td><td class="markdownTableBodyNone">1.79 + 2.02 = 3.81 </td><td class="markdownTableBodyNone">1.31 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">2900 </td><td class="markdownTableBodyNone">4.40 </td><td class="markdownTableBodyNone">2.62 + 2.71 = 5.33 </td><td class="markdownTableBodyNone">1.21 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">3000 </td><td class="markdownTableBodyNone">5.09 </td><td class="markdownTableBodyNone">3.09 + 2.96 = 6.05 </td><td class="markdownTableBodyNone">1.19 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">3100 </td><td class="markdownTableBodyNone">4.29 </td><td class="markdownTableBodyNone">2.69 + 2.57 = 5.26 </td><td class="markdownTableBodyNone">1.23 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">3200 </td><td class="markdownTableBodyNone">4.69 </td><td class="markdownTableBodyNone">2.90 + 2.78 = 5.68 </td><td class="markdownTableBodyNone">1.21 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">3300 </td><td class="markdownTableBodyNone">5.92 </td><td class="markdownTableBodyNone">3.72 + 3.21 = 6.93 </td><td class="markdownTableBodyNone">1.17 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">3400 </td><td class="markdownTableBodyNone">4.77 </td><td class="markdownTableBodyNone">2.91 + 2.89 = 5.80 </td><td class="markdownTableBodyNone">1.22 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">3500 </td><td class="markdownTableBodyNone">5.73 </td><td class="markdownTableBodyNone">3.31 + 3.47 = 6.78 </td><td class="markdownTableBodyNone">1.18 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">3600 </td><td class="markdownTableBodyNone">4.77 </td><td class="markdownTableBodyNone">2.95 + 2.90 = 5.85 </td><td class="markdownTableBodyNone">1.23 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">Average </td><td class="markdownTableBodyNone">4.43 </td><td class="markdownTableBodyNone">2.70 + 2.70 = 5.40 </td><td class="markdownTableBodyNone">1.22 </td></tr>
</table>
<p>We can see that in all tables ratio is greater then 1, so using shards with 16 ledgers is preferred.</p>
<h1><a class="anchor" id="autotoc_md214"></a>
Compare 16K shards and 32K shards</h1>
<p>To claim that shards with 16K ledgers are the best choice, we also assembled shards with 32k ledgers per shard with indexes from 1300 to 1800 with step 50 and corresponding shards with 16k ledgers per shard. For example, 32k-ledgers shard 1800 correnspond to 16k-ledgers shards with indexes 3600 and 3601 etc.</p>
<p>Here are result tables for these shards similar to tables from previous part. In the first table we only take into consideration sizes of nudb.dat files.</p>
<table class="markdownTable">
<tr class="markdownTableHead">
<th class="markdownTableHeadNone">Index </th><th class="markdownTableHeadNone">32k-ledgers </th><th class="markdownTableHeadNone">16k-ledgers sum </th><th class="markdownTableHeadNone">Ratio </th></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">1300 </td><td class="markdownTableBodyNone">4.00 </td><td class="markdownTableBodyNone">2.39 + 2.32 = 4.71 </td><td class="markdownTableBodyNone">1.18 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">1350 </td><td class="markdownTableBodyNone">5.23 </td><td class="markdownTableBodyNone">2.95 + 3.02 = 5.97 </td><td class="markdownTableBodyNone">1.14 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">1400 </td><td class="markdownTableBodyNone">4.37 </td><td class="markdownTableBodyNone">2.53 + 2.59 = 5.12 </td><td class="markdownTableBodyNone">1.17 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">1450 </td><td class="markdownTableBodyNone">7.02 </td><td class="markdownTableBodyNone">3.83 + 3.98 = 7.81 </td><td class="markdownTableBodyNone">1.11 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">1500 </td><td class="markdownTableBodyNone">7.53 </td><td class="markdownTableBodyNone">4.49 + 3.86 = 8.35 </td><td class="markdownTableBodyNone">1.11 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">1550 </td><td class="markdownTableBodyNone">6.85 </td><td class="markdownTableBodyNone">3.79 + 3.89 = 7.68 </td><td class="markdownTableBodyNone">1.12 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">1600 </td><td class="markdownTableBodyNone">7.28 </td><td class="markdownTableBodyNone">4.15 + 3.99 = 8.14 </td><td class="markdownTableBodyNone">1.12 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">1650 </td><td class="markdownTableBodyNone">8.10 </td><td class="markdownTableBodyNone">5.19 + 3.76 = 8.95 </td><td class="markdownTableBodyNone">1.10 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">1700 </td><td class="markdownTableBodyNone">7.58 </td><td class="markdownTableBodyNone">4.18 + 4.27 = 8.45 </td><td class="markdownTableBodyNone">1.11 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">1750 </td><td class="markdownTableBodyNone">8.95 </td><td class="markdownTableBodyNone">5.06 + 4.77 = 9.83 </td><td class="markdownTableBodyNone">1.10 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">1800 </td><td class="markdownTableBodyNone">7.29 </td><td class="markdownTableBodyNone">4.18 + 4.02 = 8.20 </td><td class="markdownTableBodyNone">1.12 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">Average </td><td class="markdownTableBodyNone">6.75 </td><td class="markdownTableBodyNone">3.88 + 3.68 = 7.56 </td><td class="markdownTableBodyNone">1.12 </td></tr>
</table>
<p>In the second table we take into consideration total sizes of files nudb.dat and nudb.key.</p>
<table class="markdownTable">
<tr class="markdownTableHead">
<th class="markdownTableHeadNone">Index </th><th class="markdownTableHeadNone">32k-ledgers </th><th class="markdownTableHeadNone">16k-ledgers sum </th><th class="markdownTableHeadNone">Ratio </th></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">1300 </td><td class="markdownTableBodyNone">4.59 </td><td class="markdownTableBodyNone">2.76 + 2.68 = 5.44 </td><td class="markdownTableBodyNone">1.19 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">1350 </td><td class="markdownTableBodyNone">5.98 </td><td class="markdownTableBodyNone">3.40 + 3.47 = 6.87 </td><td class="markdownTableBodyNone">1.15 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">1400 </td><td class="markdownTableBodyNone">4.99 </td><td class="markdownTableBodyNone">2.91 + 2.98 = 5.89 </td><td class="markdownTableBodyNone">1.18 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">1450 </td><td class="markdownTableBodyNone">8.02 </td><td class="markdownTableBodyNone">4.40 + 4.56 = 8.96 </td><td class="markdownTableBodyNone">1.12 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">1500 </td><td class="markdownTableBodyNone">8.51 </td><td class="markdownTableBodyNone">5.09 + 4.39 = 9.48 </td><td class="markdownTableBodyNone">1.11 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">1550 </td><td class="markdownTableBodyNone">7.73 </td><td class="markdownTableBodyNone">4.29 + 4.42 = 8.71 </td><td class="markdownTableBodyNone">1.13 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">1600 </td><td class="markdownTableBodyNone">8.20 </td><td class="markdownTableBodyNone">4.69 + 4.52 = 9.21 </td><td class="markdownTableBodyNone">1.12 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">1650 </td><td class="markdownTableBodyNone">9.20 </td><td class="markdownTableBodyNone">5.92 + 4.29 = 10.21 </td><td class="markdownTableBodyNone">1.11 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">1700 </td><td class="markdownTableBodyNone">8.61 </td><td class="markdownTableBodyNone">4.77 + 4.87 = 9.64 </td><td class="markdownTableBodyNone">1.12 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">1750 </td><td class="markdownTableBodyNone">10.09 </td><td class="markdownTableBodyNone">5.73 + 5.41 = 11.14 </td><td class="markdownTableBodyNone">1.10 </td></tr>
<tr class="markdownTableRowOdd">
<td class="markdownTableBodyNone">1800 </td><td class="markdownTableBodyNone">8.27 </td><td class="markdownTableBodyNone">4.77 + 4.59 = 9.36 </td><td class="markdownTableBodyNone">1.13 </td></tr>
<tr class="markdownTableRowEven">
<td class="markdownTableBodyNone">Average </td><td class="markdownTableBodyNone">7.69 </td><td class="markdownTableBodyNone">4.43 + 4.20 = 8.63 </td><td class="markdownTableBodyNone">1.12 </td></tr>
</table>
<h1><a class="anchor" id="autotoc_md215"></a>
Conclusion</h1>
<p>We showed that using shards with 8k ledgers leads to raising required disk size by 22% in comparison with using shards with 16k ledgers. In the same way, using shards with 16k ledgers leads to raising required disk space by 12% in comparison with using shards with 32k ledgers. Note that increase ratio 12% is much less than 22% so using 32k-ledgers shards will bring us not so much economy in disk space.</p>
<p>At the same time, size is one thing to compare but there are other aspects. Smaller shards have an advantage that they take less time to acquire and finalize. They also make for smaller archived shards which take less time to download and import. Having more/smaller shards might also lead to better database concurrency/performance.</p>
<p>It is hard to maintain both size and time parameters by a single optimization formulae because different choices for weights of size and time may lead to different results. But using "common sense" arguments we can compare 16k-ledgers shards and 32k-ledgers as follows: using 32k-ledgers shards give us 12% advantage in size, and about 44% disadvantage in time, because average size of 16k-ledgers shard is about 56% of average 32k-ledgers shard. At the same, if we compare 16k-ledgers shards with 8k-ledgers, then the first has 22% advantage in size and 39% disadvantage in time. So the balance of advantages/disadvantages is better when we use 16k-ledgers shards.</p>
<p>Thus we recommend use shards with 16K ledgers. </p>
</div></div><!-- contents -->
</div><!-- PageDoc -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/>
</a> 1.8.17
</small></address>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink