ARCHIVE/xrpl-dev-portal
1
0
Fork 0
mirror of https://github.com/XRPLF/xrpl-dev-portal.git synced 2026-06-03 08:46:39 +00:00
Code Issues Packages Projects Releases Wiki Activity

Move storage disk requirements table to capacity planning doc

Browse Source
This commit is contained in:
mDuo13
2018-12-13 15:54:22 -08:00
parent 00cc820384
commit 0bec401963
2 changed files with 23 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
content/concepts/the-rippled-server/online-deletion.md
View File

@@ -98,27 +98,7 @@ The following settings relate to online deletion:
![Ledger versions older than fetch_depth are not served to peers](img/fetch_depth.png)
### Relation to Real Time
Online deletion settings must be configured in terms of the number of ledger versions, not in terms of real time. You can estimate the amount of time represented by a number of ledger versions using the rate of ledger progress. Ledger versions typically close 3-4 seconds apart, with the overall rate of ledger closures ranging from about 20,000 to 27,000 per day throughout the years 2017-2018. There are hard limits of at least 2 seconds and no more than 20 seconds between ledgers as long as consensus is operating properly.
The following table approximates the requirements for different amounts of history:
| Real Time Amount | Number of Ledger Versions | Disk Space Required (RocksDB) | Disk Space Required (NuDB) |
|:-----------------|:--------------------------|:------------------------------|:--|
| 1 day | 25,000 | 8 GB | 12 GB |
| 14 days | 350,000 | 112 GB | 168 GB |
| 30 days | 750,000 | 240 GB | 360 GB |
| 90 days | 2,250,000 | 720 GB | 1 TB |
| 1 year | 10,000,000 | 3 TB | 4.5 TB |
| 2 years | 20,000,000 | 6 TB | 9 TB |
These numbers are estimates and depend on several factors:
- The rate of agreement among trusted validators. More disagreement slows the rate of new ledger creation. Disagreement can occur because of random chance, network latency, or validators running different software versions.
- The volume of transactions in the network. A high rate of transactions can increase the work needed to close a new ledger version, reducing the number of separate ledgers closed. An extremely low rate of transactions can also cause ledgers to close further apart (because there is nothing to be done).
- Changes and optimizations to the XRP Ledger's consensus algorithm can affect the speed of consensus and the rate of disagreement either positively or negatively.
For estimates of how much disk space is required to store different amounts of history, see [Capacity Planning](capacity-planning.html#historical-data).
### Advisory Deletion

28
content/tutorials/manage-the-rippled-server/installation/capacity-planning.md
View File

@@ -82,18 +82,34 @@ Non-validator production servers should be configured to use NuDB and to store t
NuDB does not have performance-related configuration options available in `rippled.cfg`.
#### History Sharding
`rippled` offers a history sharding feature that allows you to store a randomized range of ledgers in a separate shard store. You can use the `[shard_db]` stanza to configure the shard store to use a different type of key-value store than the one you defined for the ledger store using the `[node_db]` stanza. For more information about how to use this feature, see [History Sharding](history-sharding.html).
### Historical Data
The amount of historical data that a `rippled` server keeps online is a major contributor to required storage space. At the time of writing (2018-10-29), a `rippled` server stores about 12GB of data per day and requires 8.4TB to store the full history of the XRP Ledger. You can expect this amount to grow as transaction volume increases across the XRP Ledger network. You can control how much data you keep with the `online_delete` and `advisory_delete` fields.
The amount of historical data that a `rippled` server keeps available locally is a major contributor to the amount of disk space `rippled` requires. The server does not need to store more than the most recent 256 ledger versions to follow the consensus process and report the complete state of the ledger, but you can only query your server for transactions that executed in ledger versions it has stored locally. You can control how much data you keep with [online deletion](online-deletion.html); the default config file has the server keep the latest 2000 ledger versions. Without online deletion, the server's disk requirements grow without bounds.
Online deletion enables the purging of `rippled` ledgers from databases without any disruption of service. It removes only records that are not part of the current ledgers. Data in current ledgers means any data that's used by ledger versions that are new enough not to be deleted. Without online deletion, those databases grow without bounds. Freeing disk space requires stopping the process and manually removing database files. For more information, see [`[node_db]`: `online_delete`](https://github.com/ripple/rippled/blob/develop/cfg/rippled-example.cfg#L832).
The following table approximates the requirements for different amounts of history, at the time of writing (2018-12-13):
<!-- {# ***TODO***: Add link to online_delete section, when complete, per https://ripplelabs.atlassian.net/browse/DOC-1313 #} -->
| Real Time Amount | Number of Ledger Versions | Disk Space Required (RocksDB) | Disk Space Required (NuDB) |
|:-----------------|:--------------------------|:------------------------------|:--|
| 1 day | 25,000 | 8 GB | 12 GB |
| 14 days | 350,000 | 112 GB | 168 GB |
| 30 days | 750,000 | 240 GB | 360 GB |
| 90 days | 2,250,000 | 720 GB | 1 TB |
| 1 year | 10,000,000 | 3 TB | 4.5 TB |
| 2 years | 20,000,000 | 6 TB | 9 TB |
| Full history (through 2018) | 43,000,000+ | 9 TB | (Unknown) |
These numbers are estimates. They depend on several factors, most importantly the volume of transactions in the network. As transaction volume increases, each ledger version stores more unique data.
The `online_delete` setting tells the server how many ledger versions to keep after deleting old history. You should plan for enough disk space to store twice that many ledger versions at maximum (right before online deletion runs).
For instructions on how to change the amount of history you keep, see [Configure Online Deletion](configure-online-deletion.html).
#### History Sharding
`rippled` offers a history sharding feature that allows you to store a randomized range of ledgers in a separate shard store. You can use the `[shard_db]` stanza to configure the shard store to use a different type of key-value store than the one you defined for the ledger store using the `[node_db]` stanza. For more information about how to use this feature, see [History Sharding](history-sharding.html).
### Log Level