ARCHIVE/xrpl-dev-portal
1
0
Fork 0
mirror of https://github.com/XRPLF/xrpl-dev-portal.git synced 2025-11-22 04:35:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

Advisory delete, full history tutorials. History disk space estimates.

Browse Source
This commit is contained in:
mDuo13
2018-12-11 20:03:29 -08:00
parent 45e7b2aca0
commit a31fd23d6a
4 changed files with 241 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

98
content/tutorials/manage-the-rippled-server/configuration/configure-full-history.md Normal file
View File

@@ -0,0 +1,98 @@
# Configure Full History
In its default configuration, the `rippled` server automatically deletes outdated history of XRP Ledger state and transactions as new ledger versions become available. This is sufficient for most servers, which do not need older history to know the current state and process transactions. However, it can be useful for the network if some servers provide as much history of the XRP Ledger as possible.
## Warnings
Storing full history is expensive. As of 2018-12-11, the full history of the XRP Ledger occupies approximately **9 terabytes** of disk space, which must be entirely stored on fast solid state disk drives for proper server performance. Such a large amount of solid state storage is not cheap, and the total amount of history you must store increases by approximately 12 GB per day.
Acquiring full history from the peer-to-peer network takes a long time (several months) and requires that your server has sufficient system and network resources to acquire older history while keeping up with new ledger progress. To get a faster start on acquiring ledger history, you may want to find another server operator who has a large amount of history already downloaded, who can give you a database dump or at least allow your server to explicitly peer with theirs for a long time to acquire history. The server can load ledger history from a file and verify the integrity of the historical ledgers it imports.
You do not need a full history server to participate in the network, validate transactions, or know the current state of the network. Full history is only useful for knowing the outcome of transactions that occurred in the past, or the state of the ledger at a given time in the past. To get such information, you must rely on other servers having the history you need.
If you want to contribute to storing the history of the XRP Ledger network without storing the full history, you can [configure history sharding](configure-history-sharding.html) to store randomly-selected chunks of ledger history instead.
## Configuration Steps
To configure your server to acquire and store full history, complete the following steps:
1. Stop the `rippled` server if it is running.
$ sudo systemctl stop rippled
0. Remove (or comment out) the `online_delete` and `advisory_delete` settings from the `[node_db]` stanza of your server's config file:
[node_db]
type=RocksDB
path=/var/lib/rippled/db/rocksdb
open_files=2000
filter_bits=12
cache_mb=256
file_size_mb=8
file_size_mult=2
compression=1
#online_delete=2000
#advisory_delete=0
You can store full history with either RocksDB or NuDB as the key-value store behind the ledger store. NuDB requires less RAM but RocksDB requires less disk space. For more information, see [Capacity Planning](capacity-planning.html).
If you are using RocksDB, add `compression=1` to enable Snappy compression, which reduces the disk space necessary to store full history (at a cost of greater CPU usage when reading or writing history).
{% include '_snippets/conf-file-location.md' %}<!--_ -->
0. Set the `[ledger_history]` stanza of your server's config file to `full`:
[ledger_history]
full
0. Set the `[ips_fixed]` stanza of your server's config file to explicitly peer with at least one server that has full history available.
[ips_fixed]
169.55.164.20
50.22.123.215
Your server can only download historical data from the peer-to-peer network if one its direct peers has the data available. The easiest way to ensure you can download full history is to peer with a server that already has full history.
**Tip:** Ripple makes a pool of full history servers publicly available. You can resolve the domain `s2.ripple.com` a few times to get the IP addresses of these servers. Ripple offers these servers as a public service, so be aware that their availability to peer with other servers is limited and you may be blocked if you abuse them.
0. If you have a database dump from another full-history server to use as a basis, set the `[import_db]` stanza of your server's config file to point to the data to be imported. (Otherwise, skip this step.)
[import_db]
type=RocksDB
path=/tmp/full_history_dump/
0. Remove your server's existing database files, if you have any from previously running `rippled`.
After disabling online deletion, the server ignores any data that was downloaded while online deletion was enabled, so you may as well clear up the disk space:
rm -r /var/lib/rippled/db/rocksdb/*
rm /var/lib/rippled/db/*.db
rm /var/lib/rippled/db/*.sqlite
0. Start the `rippled` server, importing the database dump if you have one available:
If you have a database dump to load configured in `[import_db]`, start the server explicitly and include the `--import` [commandline option](commandline-usage.html#daemon-mode-options):
$ /opt/ripple/bin/rippled --conf /etc/opt/ripple/rippled.cfg --import
Importing a large database dump may take several minutes or even hours. During this time, the server is not fully started and synced with the network. Watch the server logs to see the status of the import.
If you are not importing a database dump, start the server normally:
$ sudo systemctl start rippled
0. If you added an `[import_db]` stanza to your server's config file, remove it after the import completes.
Otherwise, your server may try to import the same data again the next time it is restarted.
0. Monitor your server's available history with the [server_info method][].
The range of available ledgers reported in the `complete_ledgers` field should increase over time.
The earliest available ledger version in the production XRP Ledger's history is ledger index **32570**. The first two weeks or so of ledger history was lost due to a bug in the server at the time. Test nets and other chains generally have history going back to ledger index **1**.
<!--{# common link defs #}-->
{% include '_snippets/rippled-api-links.md' %}
{% include '_snippets/tx-type-links.md' %}
{% include '_snippets/rippled_versions.md' %}

96
content/tutorials/manage-the-rippled-server/configuration/configure-online-deletion.md Normal file
View File

@@ -0,0 +1,96 @@
# Configure Online Deletion with Advisory Delete
In its default configuration, the `rippled` server automatically deletes outdated history of XRP Ledger state and transactions as new ledger versions become available. If your server uses most of its hardware resources during peak hours, you can configure the server to delete ledgers only when prompted by a command scheduled to run during off-peak hours to reduce the chances of online deletion
## Prerequisites
This tutorial assumes your server meets the following prerequisites:
- You are on a supported operating system: Ubuntu Linux, Red Hat Enterprise Linux (RHEL), or CentOS.
- The `rippled` server is already [installed](install-rippled.html) and [online deletion](online-deletion.html) is enabled.
The default configuration file enables online deletion after
- A `cron` daemon is installed and running.
Ubuntu Linux runs a `cron` daemon by default.
On RHEL or CentOS, you can install the `cronie` package:
$ sudo yum install cronie
- Your server has enough disk space to store your chosen amount of history in its ledger store.
See [Capacity Planning](capacity-planning.html) for details of how much storage is required for different configuration. With advisory deletion enabled, the maximum history a server may accumulate before deletion is equal to the number of ledger versions configured in the `online_delete` setting **plus** the amount of time between online deletion prompts.
- You know which hours are least busy for your server.
## Configuration Steps
To configure advisory deletion with a daily schedule, perform the following steps:
1. Enable `advisory_delete` in the `[node_db]` stanza of your `rippled`'s config file.
[node_db]
# Other settings unchanged ...
online_delete=2000
advisory_delete=1
- Set `advisory_delete` to `1` to require approval before running online deletion. (Set it to `0` to run online deletion automatically as new ledger versions become available.)
- Set `online_delete` to the minimum number of ledger versions to keep after running online deletion. The server accumulates more history than this until online deletion runs.
{% include '_snippets/conf-file-location.md' %}<!--_ -->
2. Test running the [can_delete method][] to prompt the server to run online deletion.
You can use the [`rippled` commandline interface](get-started-with-the-rippled-api.html#commandline) to run this command. For example:
$ rippled --conf=/etc/opt/ripple/rippled.cfg can_delete now
The response indicates the maximum ledger index that may the server may delete from its ledger store. For example, the following message indicates that ledger versions up to and including ledger index 43633667 are to be deleted:
{
"result": {
"can_delete": 43633667,
"status": "success"
}
}
3. Configure your `cron` daemon to run the `can_delete` method you tested in the previous step at a scheduled time.
Edit your `cron` configuration:
$ crontab -e
The following example sets the server to run deletion at 1:05 AM server time daily:
5 1 * * * rippled --conf /etc/opt/ripple/rippled.cfg can_delete now
Be sure that you schedule the command to run based on the time zone your server's clock is configured to use. Ripple recommends running all production servers with the UTC time zone.
**Tip:** You do not need to schedule a `cron` job to run online deletion if you have `advisory_delete` disabled. In that case, `rippled` runs online deletion automatically when the server's oldest available ledger is twice as old as the number of ledgers to keep after deletion.
4. Start (or restart) the `rippled` service.
$ sudo systemctl restart rippled
5. Periodically check your server's `complete_ledgers` range using the [server_info method][] to confirm that ledgers are being deleted as scheduled.
The lowest ledger index in `complete_ledgers` should increase after online deletion.
Deletion may take several minutes to complete when it runs, depending on how busy your server is and how much history you delete at a time.
## Troubleshooting
If online deletion does not seem to be running after configuring it, try the following:
- Check that the user who configured the `cron` job has permissions to run the `rippled` server as a commandline client.
- Check the syntax of your cron job and the time when it is supposed to run.
- Check that the `rippled` executable is available at the path specified in your `cron` configuration. If necessary, specify the absolute path to the executable, such as `/opt/ripple/bin/rippled`.
- Check your `rippled` logs for messages that begin with `SHAMapStore::WRN`. This can indicate that [online deletion is being interrupted](online-deletion.html#interrupting-online-deletion) because your server fell out of sync with the network.
<!--{# common link defs #}-->
{% include '_snippets/rippled-api-links.md' %}
{% include '_snippets/tx-type-links.md' %}
{% include '_snippets/rippled_versions.md' %}