mirror of
https://github.com/XRPLF/xrpl-dev-portal.git
synced 2025-11-23 13:15:49 +00:00
Online deletion revisions
- Split ledger history section into a parent concept page w/ online deletion and history sharding as child pages - Clarified how much data is stored in general - Covered the special case for backfilling with advisory_delete enabled (including updating the diagram) - Changed full history recommendation to NuDB only
This commit is contained in:
mDuo13
@@ -1,6 +1,6 @@
|
||||
# Configure Advisory Deletion
|
||||
|
||||
In its default configuration, the `rippled` server automatically deletes outdated history of XRP Ledger state and transactions as new ledger versions become available. If your server uses most of its hardware resources during peak hours, you can configure the server to delete ledgers only when prompted by a command scheduled to run during off-peak hours to reduce the chances of online deletion
|
||||
The default config file sets `rippled` to automatically delete outdated history of XRP Ledger state and transactions as new ledger versions become available. If your server uses most of its hardware resources during peak hours, you can configure the server to delete ledgers only when prompted by a command scheduled to run during off-peak hours to reduce the chances of online deletion.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
@@ -37,7 +37,7 @@ To configure advisory deletion with a daily schedule, perform the following step
|
||||
online_delete=2000
|
||||
advisory_delete=1
|
||||
|
||||
- Set `advisory_delete` to `1` to require approval before running online deletion. (Set it to `0` to run online deletion automatically as new ledger versions become available.)
|
||||
- Set `advisory_delete` to `1` to run online deletion only when prompted. (Set it to `0` to run online deletion automatically as new ledger versions become available.)
|
||||
- Set `online_delete` to the minimum number of ledger versions to keep after running online deletion. The server accumulates more history than this until online deletion runs.
|
||||
|
||||
{% include '_snippets/conf-file-location.md' %}<!--_ -->
|
||||
@@ -57,6 +57,8 @@ To configure advisory deletion with a daily schedule, perform the following step
|
||||
}
|
||||
}
|
||||
|
||||
The server only deletes those ledger versions if the number of _newer_ validated ledger versions it has is equal to or greater than the `online_delete` setting.
|
||||
|
||||
3. Configure your `cron` daemon to run the `can_delete` method you tested in the previous step at a scheduled time.
|
||||
|
||||
Edit your `cron` configuration:
|
||||
@@ -67,9 +69,9 @@ To configure advisory deletion with a daily schedule, perform the following step
|
||||
|
||||
5 1 * * * rippled --conf /etc/opt/ripple/rippled.cfg can_delete now
|
||||
|
||||
Be sure that you schedule the command to run based on the time zone your server's clock is configured to use. Ripple recommends running all production servers with the UTC time zone.
|
||||
Be sure that you schedule the command to run based on your server's configured time zone.
|
||||
|
||||
**Tip:** You do not need to schedule a `cron` job to run online deletion if you have `advisory_delete` disabled. In that case, `rippled` runs online deletion automatically when the server's oldest available ledger is twice as old as the number of ledgers to keep after deletion.
|
||||
**Tip:** You do not need to schedule a `cron` job to run online deletion if you have `advisory_delete` disabled. In that case, `rippled` runs online deletion automatically when the server has approximately twice the number of ledgers to keep after deletion.
|
||||
|
||||
4. Start (or restart) the `rippled` service.
|
||||
|
||||
|
||||
@@ -20,23 +20,17 @@ To configure your server to acquire and store full history, complete the followi
|
||||
|
||||
$ sudo systemctl stop rippled
|
||||
|
||||
0. Remove (or comment out) the `online_delete` and `advisory_delete` settings from the `[node_db]` stanza of your server's config file:
|
||||
0. Remove (or comment out) the `online_delete` and `advisory_delete` settings from the `[node_db]` stanza of your server's config file, and change the type to `NuDB` if you haven't already:
|
||||
|
||||
[node_db]
|
||||
type=RocksDB
|
||||
path=/var/lib/rippled/db/rocksdb
|
||||
open_files=2000
|
||||
filter_bits=12
|
||||
cache_mb=256
|
||||
file_size_mb=8
|
||||
file_size_mult=2
|
||||
compression=1
|
||||
type=NuDB
|
||||
path=/var/lib/rippled/db/nudb
|
||||
#online_delete=2000
|
||||
#advisory_delete=0
|
||||
|
||||
You can store full history with either RocksDB or NuDB as the key-value store behind the ledger store. NuDB requires less RAM but RocksDB requires less disk space. For more information, see [Capacity Planning](capacity-planning.html).
|
||||
On a full-history server, you should use NuDB for the ledger store, because RocksDB requires too much RAM when the database is that large. For more information, see [Capacity Planning](capacity-planning.html). You can remove the following performance-related configuration options from the default `[node_db]` stanza, because they only apply to RocksDB: `open_files`, `filter_bits`, `cache_mb`, `file_size_mb`, and `file_size_mult.`
|
||||
|
||||
If you are using RocksDB, add `compression=1` to enable Snappy compression, which reduces the disk space necessary to store full history (at a cost of greater CPU usage when reading or writing history).
|
||||
**Caution:** If you have any history already downloaded with RocksDB, you must either delete that data or change the `path` field when you switch to NuDB. Otherwise, the server may [fail to start](server-wont-start.html). <!--{# TODO: link a specific case for this error #}-->
|
||||
|
||||
{% include '_snippets/conf-file-location.md' %}<!--_ -->
|
||||
|
||||
|
||||
@@ -64,14 +64,18 @@ RocksDB has performance-related configuration options that you can set in `rippl
|
||||
```
|
||||
[node_db]
|
||||
type=RocksDB
|
||||
path={path_to_ledger_store}
|
||||
path=/var/lib/rippled/db/rocksdb
|
||||
open_files=512
|
||||
filter_bits=12
|
||||
cache_mb=512
|
||||
file_size_mb=64
|
||||
file_size_mult=2
|
||||
online_delete=2000
|
||||
advisory_delete=0
|
||||
```
|
||||
|
||||
(Adjust the `path` to the directory where you want to keep the ledger store on disk. Adjust the `online_delete` and `advisory_delete` settings as desired for your configuration.)
|
||||
|
||||
#### More About Using NuDb
|
||||
|
||||
[NuDB](https://github.com/vinniefalco/nudb#introduction) is an append-only key-value store that is optimized for SSD drives.
|
||||
@@ -80,8 +84,17 @@ NuDB has nearly constant performance and memory footprints regardless of the amo
|
||||
|
||||
Non-validator production servers should be configured to use NuDB and to store the amount of historical data required for the use case.
|
||||
|
||||
NuDB does not have performance-related configuration options available in `rippled.cfg`.
|
||||
NuDB does not have performance-related configuration options available in `rippled.cfg`. Here is the recommended configuration for a `rippled` server using NuDB:
|
||||
|
||||
```
|
||||
[node_db]
|
||||
type=NuDB
|
||||
path=/var/lib/rippled/db/nudb
|
||||
online_delete=2000
|
||||
advisory_delete=0
|
||||
```
|
||||
|
||||
(Adjust the `path` to the directory where you want to keep the ledger store on disk. Adjust the `online_delete` and `advisory_delete` settings as desired for your configuration.)
|
||||
|
||||
|
||||
### Historical Data
|
||||
@@ -98,7 +111,7 @@ The following table approximates the requirements for different amounts of histo
|
||||
| 90 days | 2,250,000 | 720 GB | 1 TB |
|
||||
| 1 year | 10,000,000 | 3 TB | 4.5 TB |
|
||||
| 2 years | 20,000,000 | 6 TB | 9 TB |
|
||||
| Full history (through 2018) | 43,000,000+ | 9 TB | (Unknown) |
|
||||
| Full history (through 2018) | 43,000,000+ | (Not recommended) | ~9 TB |
|
||||
|
||||
These numbers are estimates. They depend on several factors, most importantly the volume of transactions in the network. As transaction volume increases, each ledger version stores more unique data.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user