Online deletion: edits per reviews, new diagram of how it works

content/concepts/the-rippled-server/ledger-history/ledger-history.md

@@ -2,7 +2,7 @@
 
 The [consensus process](intro-to-consensus.html) creates a chain of [validated ledger versions](ledgers.html), each derived from the previous one by applying a set of transactions. Every `rippled` server stores ledger versions and transaction history locally. The amount of transaction history a server stores depends on how long that server has been online and how much history it is configured to fetch and keep.
 
-Servers in the peer-to-peer XRP Ledger network share transactions and other data with each other as part of the consensus process. Each server each independently builds each new ledger version and compares results with its trusted validators to ensure consistency. (If a consensus of trusted validators disagrees with a server's results, that server fetches the necessary data from its peers to achieve consistency.) Servers can download older data from their peers to fill gaps in their available history. The structure of the ledger uses cryptographic [hashes](basic-data-types.html#hashes) of the data so that any server can verify the integrity and consistency of the data.
+Servers in the peer-to-peer XRP Ledger network share transactions and other data with each other as part of the consensus process. Each server independently builds each new ledger version and compares results with its trusted validators to ensure consistency. (If a consensus of trusted validators disagrees with a server's results, that server fetches the necessary data from its peers to achieve consistency.) Servers can download older data from their peers to fill gaps in their available history. The structure of the ledger uses cryptographic [hashes](basic-data-types.html#hashes) of the data so that any server can verify the integrity and consistency of the data.
 
 ## Databases
 
@@ -18,7 +18,7 @@ The [server_info method][] reports how many ledger versions your server has avai
 
 ## Fetching History
 
-When it starts, a `rippled` server's first priority is to get a complete copy of the latest validated ledger. From there, it keeps up with advances in the ledger progress. If configured to do so, the server also backfills ledger history up to a configured amount, which must be equal or less than the cutoff beyond which online deletion is configured to delete.
+When it starts, a `rippled` server's first priority is to get a complete copy of the latest validated ledger. From there, it keeps up with advances in the ledger progress. If configured to do so, the server also backfills ledger history up to a configured amount, which must be equal to or less than the cutoff beyond which online deletion is configured to delete.
 
 The server can backfill history from before it became synced, as well as filling in any gaps in the history it has collected after syncing. (Gaps in ledger history can occur if a server temporarily becomes too busy to keep up with the network, loses its network connection, or suffers other temporary issues.) To backfill history, the server requests data from its peer `rippled` servers. The amount the server tries to backfill is defined by the `[ledger_history]` setting.
 
@@ -39,7 +39,7 @@ Ripple provides a set of public full-history servers as a public service at `s2.
 
 **Tip:** Unlike some cryptocurrency networks, servers in the XRP Ledger do not need full history to know the current state and keep up with current transactions.
 
-For instructions setting up full history, see [Configure Full History](configure-full-history.html).
+For instructions on setting up full history, see [Configure Full History](configure-full-history.html).
 
 ## History Sharding
 

content/concepts/the-rippled-server/ledger-history/online-deletion.md

@@ -14,7 +14,7 @@ The default config file sets the `rippled` server to keep the most recent 2000 l
 
 The `rippled` server stores [ledger history](ledger-history.html) in its _ledger store_. This data accumulates over time.
 
-Inside the ledger store, ledger data is "de-duplicated": data that doesn't change from version to version is only stored once. The records themselves in the ledger store do not indicate which ledger version(s) contain them; part of the work of online deletion is identifying which records are only used by outdated ledger versions. This process is time consuming and affects the disk I/O and application cache, so it is not feasible to delete old data on every ledger close.
+Inside the ledger store, ledger data is "de-duplicated". In other words, data that doesn't change from version to version is only stored once. The records themselves in the ledger store do not indicate which ledger version(s) contain them; part of the work of online deletion is identifying which records are only used by outdated ledger versions. This process is time consuming and affects the disk I/O and application cache, so it is not feasible to delete old data on every ledger close.
 
 
 ## Online Deletion Behavior
@@ -25,12 +25,16 @@ The online deletion settings configure how many ledger versions the `rippled` se
 - The server may store up to twice the configured number of ledger versions if online deletion is set to run automatically. (Each time it runs, it reduces the number of stored ledger versions to approximately the configured number.)
 - If advisory deletion is enabled, the server stores all the ledger versions that it has acquired and built until its administrator calls the [can_delete method][].
 
-    If you call [can_delete][can_delete method] on a regular basis, the maximum amount of data the server stores is either of the following, whichever is larger:
+    The amount of data the server stores depends on how often you call [can_delete][can_delete method] and the how big an interval of time your `online_delete` setting represents:
 
-    - A number of ledger versions approximately equal to twice the `online_delete` value. (After deletion, this is reduced to approximately the `online_delete` value.)
-    - Ledger versions spanning an amount of time that is approximately twice the interval between `can_delete` calls. (After deletion, this is reduced to approximately one interval's worth of data.)
+    - If you call `can_delete` _more often_ than your `online_delete` interval, the server stores at most a number of ledger versions approximately equal to **twice the `online_delete` value**. (After deletion, this is reduced to approximately the `online_delete` value.)
+
+        For example, if you call `can_delete` with a value of `now` once per day and an `online_delete` value of 50,000, the server typically stores up to 100,000 ledger versions before running deletion. After running deletion, the server keeps at least 50,000 ledger versions (about two days' worth). With this configuration, approximately every other `can_delete` call results in no change because the server does not have enough ledger versions to delete.
+
+    - If you call `can_delete` _less often_ than your `online_delete` interval, the server stores at most ledger versions spanning an amount of time that is approximately **twice the interval between `can_delete` calls**. (After deletion, this is reduced to approximately one interval's worth of data.)
+
+        For example, if you call `can_delete` with a value of `now` once per day and an `online_delete` value of 2000, the server typically stores up to two full days' worth of ledger versions before running deletion. After running deletion, the server keeps approximately one day's worth (about 25,000 ledger versions), but never fewer than 2000 ledger versions.
 
-    For example, if you call `can_delete` with a value of `now` once per day and an `online_delete` value of 2000, the server typically stores up two full day's worth of ledger versions before running deletion. After running deletion, the server keeps approximately 1 day's worth, but never less than 2000 ledger versions.
 
 With online deletion enabled and running automatically (that is, with advisory delete disabled), the total amount of ledger data stored should remain at minimum equal to the number of ledger versions the server is configured to keep, with the maximum being roughly twice that many.
 
@@ -44,7 +48,7 @@ Online deletion automatically stops if the [server state](rippled-server-states.
 
 If you stop the server or it crashes while online deletion is running, online deletion resumes after the server is restarted and the server becomes fully synced.
 
-To temporarily disable online deletion, you can use the [can_delete method][] with an argument of `never`. This change persists until you re-enable online deletion by calling [can_delete][can_delete method] again, even if you change the . For more information on controlling when online deletion happens, see [Advisory Deletion](#advisory-deletion).
+To temporarily disable online deletion, you can use the [can_delete method][] with an argument of `never`. This change persists until you re-enable online deletion by calling [can_delete][can_delete method] again. For more information on controlling when online deletion happens, see [Advisory Deletion](#advisory-deletion).
 
 
 ## Configuration
@@ -55,7 +59,7 @@ The following settings relate to online deletion:
 
     The default config file specifies 2000 for this value. This cannot be less than 256, because some events like [Fee Voting](fee-voting.html) and the [Amendment Process](amendments.html#amendment-process) update only every 256 ledgers.
 
-    **Caution:** If you run `rippled` with `online_delete` disabled, then later enable `online_delete` and restart the server, the server disregards but does not delete existing ledger history that your server already downloaded while `online_delete` was disabled. To save disk space, delete your existing history before re-starting the server with `online_delete` back on.
+    **Caution:** If you run `rippled` with `online_delete` disabled, then later enable `online_delete` and restart the server, the server disregards but does not delete existing ledger history that your server already downloaded while `online_delete` was disabled. To save disk space, delete your existing history before re-starting the server after changing the `online_delete` setting.
 
 - **`[ledger_history]`** - Specify a number of validated ledgers, equal to or less than `online_delete`. If the server does not have at least this many validated ledger versions, it attempts to backfill them by fetching the data from peers.
 
@@ -79,23 +83,31 @@ The following settings relate to online deletion:
 
     ![Ledger versions older than fetch_depth are not served to peers](img/fetch_depth.png)
 
-For estimates of how much disk space is required to store different amounts of history, see [Capacity Planning](capacity-planning.html#disk-storage).
+For estimates of how much disk space is required to store different amounts of history, see [Capacity Planning](capacity-planning.html#disk-space).
 
 ### Advisory Deletion
 
-By default, online deletion happens automatically and periodically. If the `advisory_delete` setting is enabled in the config file, online deletion only happens when an administrator triggers it using the [can_delete method][].
+The default config file schedules online deletion to happen automatically and periodically. If the config file does not specify an `online_delete` interval, online deletion does not occur. If config file enables the `advisory_delete` setting, online deletion only happens when an administrator triggers it using the [can_delete method][].
 
 You can use advisory deletion with a scheduled job to trigger automatic deletion based on clock time instead of the number of ledger versions closed. If your server is heavily used, the extra load from online deletion can cause your server to fall behind and temporarily de-sync from the consensus network. If this is the case, you can use advisory deletion and schedule online deletion to happen only during off-peak times.
 
-Alternatively, you can use advisory deletion for other reasons, such as if you want to manually confirm that transaction data is backed up to a separate server before deleting it.
+You can use advisory deletion for other reasons. For example, you may want to manually confirm that transaction data is backed up to a separate server before deleting it. Alternatively, you may want to manually confirm that a separate task has finished processing transaction data before you delete that data.
 
 The `can_delete` API method can enable or disable automatic deletion, in general or up to a specific ledger version, as long as `advisory_delete` is enabled in the config file. These settings changes persist even if you restart the `rippled` server, unless you disable `advisory_delete` in the config file before restarting.
 
 
+## How It Works
+
+Online deletion works by creating two databases: at any given time, there is an "old" database, which is read-only, and a "current" database, which is writable. The `rippled` server can read objects from either database, so current ledger versions may contain objects in either one. If an object in a ledger does not change from ledger version to ledger version, only one copy of that object remains in the database, so the server does not store redundant copies of that object. When a new ledger version modifies an object, the server stores the modified object in the "new" database, while the previous version of the object (which is still used by previous ledger versions) remains in the "old" database.
+
+When it comes time for online deletion, the server first walks through the oldest ledger version to keep, and copies all objects in that ledger version from the read-only "old" database into the "current" database. This guarantees that the "current" database now contains all objects used in the chosen ledger version and all newer versions. Then, the server deletes the "old" database, and changes the existing "current" database to become "old" and read-only. The server starts a new "current" database to contain any newer changes after this point.
+
+![Diagram showing how online deletion uses two databases](img/online-deletion-process.png)
+
 ## See Also
 
 - [Capacity Planning](capacity-planning.html)
-- [can_delete method][] API reference documentation
+- [can_delete method][] - API reference documentation
 - [Configure Online Deletion](configure-online-deletion.html)
 - [Configure Advisory Deletion](configure-advisory-deletion.html)
 - [Configure Full History](configure-full-history.html)

content/references/rippled-api/public-rippled-methods/server-info-methods/server_info.md

@@ -283,7 +283,7 @@ The `info` object may have some arrangement of the following fields:
 | `amendment_blocked`                 | Boolean                   | _(May be omitted)_ If `true`, this server is [amendment blocked](amendments.html#amendment-blocked). If the server is not amendment blocked, the response omits this field. [New in: rippled 0.80.0][] |
 | `build_version`                     | String                    | The version number of the running `rippled` version. |
 | `closed_ledger`                     | Object                    | (May be omitted) Information on the most recently closed ledger that has not been validated by consensus. If the most recently validated ledger is available, the response omits this field and includes `validated_ledger` instead. The member fields are the same as the `validated_ledger` field. |
-| `complete_ledgers`                  | String                    | Range expression indicating the sequence numbers of the ledger versions the local rippled has in its database. This may be a disjoint sequence, for example `24900901-24900984,24901116-24901158`. If the server does not have any complete ledgers (for example, it just started syncing with the network), this is the string `empty`. |
+| `complete_ledgers`                  | String                    | Range expression indicating the sequence numbers of the ledger versions the local `rippled` has in its database. This may be a disjoint sequence, for example `24900901-24900984,24901116-24901158`. If the server does not have any complete ledgers (for example, it just started syncing with the network), this is the string `empty`. |
 | `hostid`                            | String                    | On an admin request, returns the hostname of the server running the `rippled` instance; otherwise, returns a unique four letter word. |
 | `io_latency_ms`                     | Number                    | Amount of time spent waiting for I/O operations, in milliseconds. If this number is not very, very low, then the `rippled` server is probably having serious load issues. |
 | `last_close`                        | Object                    | Information about the last time the server closed a ledger, including the amount of time it took to reach a consensus and the number of trusted validators participating. |

content/tutorials/manage-the-rippled-server/configuration/configure-advisory-deletion.md

@@ -1,6 +1,6 @@
 # Configure Advisory Deletion
 
-The default config file sets `rippled` to automatically delete outdated history of XRP Ledger state and transactions as new ledger versions become available. If your server uses most of its hardware resources during peak hours, you can configure the server to delete ledgers only when prompted by a command scheduled to run during off-peak hours to reduce the chances of online deletion.
+The default config file sets `rippled` to automatically delete outdated history of XRP Ledger state and transactions as new ledger versions become available. If your server uses most of its hardware resources during peak hours, you can configure the server to delete ledgers only when prompted by a command scheduled to run during off-peak hours, so that online deletion is less likely to impact server performance.
 
 ## Prerequisites
 
@@ -22,7 +22,7 @@ This tutorial assumes your server meets the following prerequisites:
 
 - Your server has enough disk space to store your chosen amount of history in its ledger store.
 
-    See [Capacity Planning](capacity-planning.html) for details of how much storage is required for different configuration. With advisory deletion enabled, the maximum history a server may accumulate before deletion is equal to the number of ledger versions configured in the `online_delete` setting **plus** the amount of time between online deletion prompts.
+    See [Capacity Planning](capacity-planning.html) for details of how much storage is required for different configurations. With advisory deletion enabled, the maximum history a server may accumulate before deletion is equal to the number of ledger versions configured in the `online_delete` setting **plus** the amount of time between online deletion prompts.
 
 - You know which hours are least busy for your server.
 
@@ -48,7 +48,7 @@ To configure advisory deletion with a daily schedule, perform the following step
 
         $ rippled --conf=/etc/opt/ripple/rippled.cfg can_delete now
 
-    The response indicates the maximum ledger index that may the server may delete from its ledger store. For example, the following message indicates that ledger versions up to and including ledger index 43633667 are to be deleted:
+    The response indicates the maximum ledger index that the server may delete from its ledger store. For example, the following message indicates that ledger versions up to and including ledger index 43633667 can be deleted:
 
         {
           "result": {
@@ -71,7 +71,7 @@ To configure advisory deletion with a daily schedule, perform the following step
 
     Be sure that you schedule the command to run based on your server's configured time zone.
 
-    **Tip:** You do not need to schedule a `cron` job to run online deletion if you have `advisory_delete` disabled. In that case, `rippled` runs online deletion automatically when the server has approximately twice the number of ledgers to keep after deletion.
+    **Tip:** You do not need to schedule a `cron` job to run online deletion if you have `advisory_delete` disabled. In that case, `rippled` runs online deletion automatically when the difference between the server's oldest and current validated ledger versions is at least the value of `online_delete`.
 
 4. Start (or restart) the `rippled` service.
 

content/tutorials/manage-the-rippled-server/configuration/configure-full-history.md

@@ -52,12 +52,12 @@ To configure your server to acquire and store full history, complete the followi
 0. If you have a database dump from another full-history server to use as a basis, set the `[import_db]` stanza of your server's config file to point to the data to be imported. (Otherwise, skip this step.)
 
         [import_db]
-      	type=RocksDB
+      	type=NuDB
       	path=/tmp/full_history_dump/
 
 0. Remove your server's existing database files, if you have any from previously running `rippled`.
 
-    After disabling online deletion, the server ignores any data that was downloaded while online deletion was enabled, so you may as well clear up the disk space:
+    After disabling online deletion, the server ignores any data that was downloaded while online deletion was enabled, so you may as well clear up the disk space. For example:
 
         rm -r /var/lib/rippled/db/rocksdb/*
         rm /var/lib/rippled/db/*.db

content/tutorials/manage-the-rippled-server/configuration/configure-online-deletion.md

@@ -21,7 +21,7 @@ To change the amount of history your server stores, perform the following steps:
 
 1. Decide how many ledger versions' worth of history to store.
 
-    New ledger versions are usually validated 3 to 4 seconds apart, so the number of ledger versions corresponds roughly to the amount of time you want to store. See [Capacity Planning](capacity-planning.html) for details of how much storage is required for different configuration.
+    New ledger versions are usually validated 3 to 4 seconds apart, so the number of ledger versions corresponds roughly to the amount of time you want to store. See [Capacity Planning](capacity-planning.html) for details of how much storage is required for different configurations.
 
     Online deletion is based on how many ledger versions to keep _after_ deleting history, so you should have enough disk space to store twice as many ledgers as you set it to keep.
 

content/tutorials/manage-the-rippled-server/installation/capacity-planning.md

@@ -53,7 +53,7 @@ The example `rippled-example.cfg` file has the `type` field in the `[node_db]` s
 
 #### More About Using RocksDB
 
-[RocksDB](https://rocksdb.org/docs/getting-started.html) is an embeddable persistent key-value store that is optimized for rotational disks.
+[RocksDB](https://rocksdb.org/docs/getting-started.html) is an embeddable persistent key-value store that works well with both rotational disks and solid state disks.
 
 RocksDB requires approximately one-third less [disk storage](#disk-space) than NuDB and provides better I/O latency. However, the better I/O latency comes as result of the large amount of RAM RocksDB requires to store data indexes.