ARCHIVE/xrpl-dev-portal
1
0
Fork 0
mirror of https://github.com/XRPLF/xrpl-dev-portal.git synced 2025-11-20 11:45:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

WIP - initial draft for Clio

Browse Source
This commit is contained in:
Amarantha Kulkarni
2022-06-26 08:23:13 -07:00
parent c06fd3c6f9
commit 6930657686
5 changed files with 761 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

230
content/tutorials/manage-the-clio-server/build-run-clio-ubuntu.md Normal file
View File

@@ -0,0 +1,230 @@
---
html: build-run-rippled-ubuntu.html
parent: install-rippled.html
blurb: Compile rippled yourself on Ubuntu Linux.
labels:
- Core Server
---
# Build and Run rippled on Ubuntu
[`rippled`](the-rippled-server.html) is the reference implementation of an XRP Ledger peer-to-peer server. This server can connect to a network of peers, relay cryptographically signed transactions, and maintain a local copy of the complete shared global ledger. Use these instructions to build a `rippled` executable from source. These instructions were tested on Ubuntu 18.04 LTS.
For information about building for other platforms, see [Builds](https://github.com/ripple/rippled/tree/develop/Builds) in the `rippled` GitHub repository.
## Prerequisites
Before you compile or install `rippled`, you must meet the [System Requirements](system-requirements.html).
## 1. Build `rippled`
These instructions use Ubuntu's APT (Advanced Packaging Tool) to install `rippled`'s build dependencies.
1. Update the list of packages that are available for `apt-get` to install or upgrade.
sudo apt-get update
2. Upgrade currently installed packages.
sudo apt-get -y upgrade
3. Install dependencies.
sudo apt-get -y install git pkg-config protobuf-compiler libprotobuf-dev libssl-dev wget build-essential doxygen
4. Install CMake.
Version 1.9.0 of `rippled` requires CMake 3.16.3 or higher.
To install CMake 3.16.3:
wget https://github.com/Kitware/CMake/releases/download/v3.16.3/cmake-3.16.3-Linux-x86_64.sh
sudo sh cmake-3.16.3-Linux-x86_64.sh --prefix=/usr/local --exclude-subdir
Use `cmake --version` to verify that the installation worked.
5. Compile Boost.
Version 1.9.0 of `rippled` requires the Boost library and is compatible with Boost versions 1.71.0 to 1.77.0. The Ubuntu 18.04 (or 20.04) software repositories don't have a compatible Boost version, so you must compile it yourself. The following examples use Boost 1.75.0.
If you have previously built Boost 1.75.0 for `rippled` and configured the `BOOST_ROOT` environment variable, you can skip these steps.
1. Download Boost 1.75.0.
wget https://boostorg.jfrog.io/artifactory/main/release/1.75.0/source/boost_1_75_0.tar.gz
2. Extract `boost_1_75_0.tar.gz`.
tar xvzf boost_1_75_0.tar.gz
3. Change to the new `boost_1_75_0` directory.
cd boost_1_75_0
4. Prepare the Boost.Build system for use.
./bootstrap.sh
5. Build the separately-compiled Boost libraries. This may take about 10 minutes, depending on your hardware specs.
./b2 -j 4
**Tip:** This example uses 4 processes to build in parallel. The best number of processes to use depends on how many CPU cores your hardware has available. You can use `cat /proc/cpuinfo` to get information about your hardware's processor.
6. Set the environment variable `BOOST_ROOT` to point to the new `boost_1_75_0` directory. It's best to put this environment variable in your `.profile`, or equivalent, file for your shell so it's automatically set when you log in. Add the following line to the file (change `my_user` to your username):
export BOOST_ROOT=/home/my_user/boost_1_75_0
7. Source your updated `.profile` file. For example:
source ~/.profile
6. From a working directory, get the `rippled` source code. The `master` branch has the latest released version.
git clone https://github.com/ripple/rippled.git
cd rippled
git checkout master
7. Check the commit log to be sure you're compiling the right code. The most recent commit should be signed by a well-known Ripple developer and should set the version number to the latest released version. The [release announcements for `rippled`](https://xrpl.org/blog/label/rippled-release-notes.html) generally show the exact commit to expect for that release.
$ git log -1
8. If you previously built, or (more importantly) tried and failed to build the XRPL server, you should delete the `my_build/` directory (or whatever you named it) to start clean before moving on to the next step. Otherwise, you may get unexpected behavior, like an executable that crashes due to a segmentation fault (segfault). <!-- SPELLING_IGNORE: segfault -->
If this is your first time building `rippled`, you won't have a `my_build/` directory and can move on to the next step.
9. Use CMake to build a `rippled` binary executable from source code. This creates a `rippled` binary executable in the `my_build` directory.
1. Generate the build system. Builds should be performed in a directory that is separate from the source tree root. In this example, we'll use a `my_build` directory that is a subdirectory of `rippled`.
mkdir my_build
cd my_build
cmake ..
**Tip:** The default build includes debugging symbols, which can be useful for development but are inefficient in production. To build `rippled` for use on production servers, add the `-DCMAKE_BUILD_TYPE=Release` flag when running the `cmake` command.
2. Build the `rippled` executable. This may take about 30 minutes, depending on your hardware specs.
cmake --build .
10. _(Optional)_ Run `rippled` unit tests. If there are no test failures, you can be fairly certain that your `rippled` executable compiled correctly.
./rippled -u
## 2. Configure `rippled`
Complete the following configurations that are required for `rippled` to start up successfully. All other configuration is optional and can be tweaked after you have a working server.
1. Create a copy of the example config file (assumes you're in the `rippled` folder already). Saving the config file to this location enables you to run `rippled` as a non-root user (recommended).
mkdir -p ~/.config/ripple
cp cfg/rippled-example.cfg ~/.config/ripple/rippled.cfg
2. Edit the config file to set necessary file paths. The user you plan to run `rippled` as must have write permissions to all of the paths you specify here.
1. Set the `[node_db]`'s path to the location where you want to store the ledger database.
2. Set the `[database_path]` to the location where you want to store other database data. (This includes an SQLite database with configuration data, and is typically one level above the `[node_db]` path field.)
3. Set the `[debug_logfile]` to a path where `rippled` can write logging information.
3. Copy the example `validators.txt` file to the same folder as `rippled.cfg`:
cp cfg/validators-example.txt ~/.config/ripple/validators.txt
**Warning:** The `validators.txt` file contains settings that determine how your server declares a ledger to be validated. If you are not careful, changes to this file could cause your server to diverge from the rest of the network and report out of date, incomplete, or inaccurate data. Acting on such data can cause you to lose money.
## 3. Run `rippled`
To run your `rippled` server from the executable you built:
```sh
./rippled
```
### What to Expect
Once you've run `rippled`, here are excerpts of what you can expect to see in your terminal.
```text
Loading: "/home/ubuntu/.config/ripple/rippled.cfg"
Watchdog: Launching child 1
2018-Jun-06 00:51:35.094331139 JobQueue:NFO Auto-tuning to 4 validation/transaction/proposal threads.
2018-Jun-06 00:51:35.100607625 Amendments:DBG Amendment 4C97EBA926031A7CF7D7B36FDE3ED66DDA5421192D63DE53FFB46E43B9DC8373 is supported.
2018-Jun-06 00:51:35.101226904 Amendments:DBG Amendment 6781F8368C4771B83E8B821D88F580202BCB4228075297B19E4FDC5233F1EFDC is supported.
2018-Jun-06 00:51:35.101354503 Amendments:DBG Amendment 42426C4D4F1009EE67080A9B7965B44656D7714D104A72F9B4369F97ABF044EE is supported.
2018-Jun-06 00:51:35.101503304 Amendments:DBG Amendment 08DE7D96082187F6E6578530258C77FAABABE4C20474BDB82F04B021F1A68647 is supported.
2018-Jun-06 00:51:35.101624717 Amendments:DBG Amendment 740352F2412A9909880C23A559FCECEDA3BE2126FED62FC7660D628A06927F11 is supported.
...
2018-Jun-06 00:51:35.106970906 OrderBookDB:DBG Advancing from 0 to 3
2018-Jun-06 00:51:35.107158071 OrderBookDB:DBG OrderBookDB::update>
2018-Jun-06 00:51:35.107380722 OrderBookDB:DBG OrderBookDB::update< 0 books found
2018-Jun-06 00:51:35.168875072 ManifestCache:NFO Manifest: AcceptedNew;Pk: nHBARBMi2MC3LJYuvs9Rhp94WcfbxoQD5BGhwN3jaHBsPkbNpoZq;Seq: 1;
2018-Jun-06 00:51:35.172099325 ManifestCache:NFO Manifest: AcceptedNew;Pk: nHB57Sey9QgaB8CubTPvMZLkLAzfJzNMWBCCiDRgazWJujRdnz13;Seq: 1;
2018-Jun-06 00:51:35.175302816 ManifestCache:NFO Manifest: AcceptedNew;Pk: nHDsPCxoBHZS9KNNfsd7iVaQXBSitNtbqXfB6BS1iEmJwwEKLhhQ;Seq: 1;
2018-Jun-06 00:51:35.178486951 ManifestCache:NFO Manifest: AcceptedNew;Pk: nHBQ3CT3EWYZ4uzbnL3k6TRf9bBPhWRFVcK1F5NjtwCBksMEt5yy;Seq: 2;
2018-Jun-06 00:51:35.181681868 ManifestCache:NFO Manifest: AcceptedNew;Pk: nHU5egMCYs1g7YRVKrKjEqVYFL12mFWwkqVFTiz2Zi4Z8jppPgxU;Seq: 2;
2018-Jun-06 00:51:35.184864291 ManifestCache:NFO Manifest: AcceptedNew;Pk: nHBbiP5ua5dUqCTz5i5vd3ia9jg3KJthohDjgKxnc7LxtmnauW7Z;Seq: 2;
...
2018-Jun-06 00:51:35.317972033 LedgerConsensus:NFO Entering consensus process, watching, synced=no
2018-Jun-06 00:51:35.318155351 LedgerConsensus:NFO Consensus mode change before=observing, after=observing
2018-Jun-06 00:51:35.318360468 NetworkOPs:DBG Initiating consensus engine
2018-Jun-06 00:51:35.358673488 Server:NFO Opened 'port_rpc_admin_local' (ip=127.0.0.1:5005, admin IPs:127.0.0.1, http)
2018-Jun-06 00:51:35.359296222 Server:NFO Opened 'port_peer' (ip=0.0.0.0:51235, peer)
2018-Jun-06 00:51:35.359778994 Server:NFO Opened 'port_ws_admin_local' (ip=127.0.0.1:6006, admin IPs:127.0.0.1, ws)
2018-Jun-06 00:51:35.360240190 Application:FTL Startup RPC:
{
"command" : "log_level",
"severity" : "warning"
}
...
2018-Jun-06 00:52:32.385295633 NetworkOPs:WRN We are not running on the consensus ledger
2018-Jun-06 00:52:32.388552023 LedgerConsensus:WRN Need consensus ledger 84726E8C5B346E28C21ADE6AAD703E65F802322EDAA5B76446A4D0C5206AB2DB
2018-Jun-06 00:52:33.379448561 LedgerConsensus:WRN View of consensus changed during open status=open, mode=wrongLedger
2018-Jun-06 00:52:33.379541915 LedgerConsensus:WRN 84726E8C5B346E28C21ADE6AAD703E65F802322EDAA5B76446A4D0C5206AB2DB to 1720162AE3BA8CD953BFB40EB746D7B78D13E1C97905E8C553E0B573F1B6A517
2018-Jun-06 00:52:33.379747629 LedgerConsensus:WRN {"accepted":true,"account_hash":"CC1F1EC08E76BC9FE843BBF9C6068C5B73192E6957B9CC1174DCB2B94DD2025A","close_flags":0,"close_time":581561550,"close_time_human":"2018-Jun-06 00:52:30.000000000","close_time_resolution":30,"closed":true,"hash":"94354A7FECAB638C29BBC79B18CFDBDC05E4FF72647AD62F072DB4D23A5E0317","ledger_hash":"94354A7FECAB638C29BBC79B18CFDBDC05E4FF72647AD62F072DB4D23A5E0317","ledger_index":"3","parent_close_time":581561490,"parent_hash":"80BF92A69F65F5C543E962DF4B41715546FDD97FC6988028E5ACBB46654756CA","seqNum":"3","totalCoins":"100000000000000000","total_coins":"100000000000000000","transaction_hash":"0000000000000000000000000000000000000000000000000000000000000000"}
...
2018-Jun-06 00:53:50.568965045 LedgerConsensus:WRN {"accepted":true,"account_hash":"A79E6754544F9C8FC74870C95A39CED1D45CC1206DDA4C113E51F9DB6DDB0E76","close_flags":0,"close_time":581561630,"close_time_human":"2018-Jun-06 00:53:50.000000000","close_time_resolution":10,"closed":true,"hash":"6294118F39F5F2B8349E7CC6D4D5931011622E78DD4E34D91372651E9F453E2F","ledger_hash":"6294118F39F5F2B8349E7CC6D4D5931011622E78DD4E34D91372651E9F453E2F","ledger_index":"29","parent_close_time":581561623,"parent_hash":"5F57870CE5160D6B53271955F26E3BE63696D1127B91BC7943F9A199B313CB85","seqNum":"29","totalCoins":"100000000000000000","total_coins":"100000000000000000","transaction_hash":"0000000000000000000000000000000000000000000000000000000000000000"}
2018-Jun-06 00:53:50.569776678 LedgerConsensus:WRN Need consensus ledger 6A0DE66550B6BA9636E3F8FDB71C2E924D182A1835E4143B2170DAA1D33CAE8D
2018-Jun-06 00:53:51.576778862 NetworkOPs:WRN We are not running on the consensus ledger
2018-Jun-06 00:53:53.576524564 LedgerConsensus:WRN View of consensus changed during establish status=establish, mode=wrongLedger
2018-Jun-06 00:53:53.576783663 LedgerConsensus:WRN 6A0DE66550B6BA9636E3F8FDB71C2E924D182A1835E4143B2170DAA1D33CAE8D to 1CB9C9A1C27403CBAB9DFCFA61E1F915059DFE4FA93524537B885CC190DB5C6B
2018-Jun-06 00:53:53.577079124 LedgerConsensus:WRN {"accepted":true,"account_hash":"5CAB3E4F5F2AC1A764106D7CC0729E6E7D1F7F93C65B7D8CB04C8DE2FC2C1305","close_flags":0,"close_time":581561631,"close_time_human":"2018-Jun-06 00:53:51.000000000","close_time_resolution":10,"closed":true,"hash":"201E147BD195CE3C56B0C0B8DF58386FC7BFF450E1E5B286A29AB856926D5F79","ledger_hash":"201E147BD195CE3C56B0C0B8DF58386FC7BFF450E1E5B286A29AB856926D5F79","ledger_index":"30","parent_close_time":581561630,"parent_hash":"6294118F39F5F2B8349E7CC6D4D5931011622E78DD4E34D91372651E9F453E2F","seqNum":"30","totalCoins":"100000000000000000","total_coins":"100000000000000000","transaction_hash":"0000000000000000000000000000000000000000000000000000000000000000"}
```
## Explore Next Steps
* Now that you have a stock `rippled` server running, you may want to consider running it as a validating server. For information about validating servers and why you might want to run one, see the [rippled Setup Tutorial](install-rippled.html).
* For information about communicating with your `rippled` server using the `rippled` API, see the [`rippled` API reference](rippled-api.html).
* As a development best practice, you may want to build a `rippled` `.deb` package. You can use the CMake build's deb package target to build a `deb` package directly from the source tree. The build machine must have [Docker installed](https://docs.docker.com/install/#supported-platforms). This process may take more than an hour to complete. To build the `deb` package:
mkdir -p build/pkg && cd build/pkg
cmake -Dpackages_only=ON ../..
cmake --build . --target dpkg
* You may also want to install a `systemd` unit. For more information, see [systemd for Upstart Users](https://wiki.ubuntu.com/SystemdForUpstartUsers). You can use the [official `rippled` system unit file](https://github.com/ripple/rippled/blob/master/Builds/containers/shared/rippled.service) or modify it to suit your needs.
## See Also
- **Concepts:**
- [The `rippled` Server](the-rippled-server.html)
- [Introduction to Consensus](intro-to-consensus.html)
- **Tutorials:**
- [Configure rippled](configure-rippled.html)
- [Troubleshoot rippled](troubleshoot-the-rippled-server.html)
- [Get Started with the rippled API](get-started-using-http-websocket-apis.html)
- **References:**
- [rippled API Reference](rippled-api.html)
- [`rippled` Commandline Usage](commandline-usage.html)
- [server_info method][]
<!--{# common link defs #}-->
{% include '_snippets/rippled-api-links.md' %}
{% include '_snippets/tx-type-links.md' %}
{% include '_snippets/rippled_versions.md' %}

347
content/tutorials/manage-the-clio-server/build-run-rippled-in-reporting-mode.md Normal file
View File

@@ -0,0 +1,347 @@
---
html: build-run-rippled-in-reporting-mode.html
parent: install-rippled.html
blurb: Build and run a special operating mode of rippled that handles remote procedure calls (RPC) for validated data.
labels:
- Core Server
- Blockchain
top_nav_grouping: Popular Pages
---
# Build and Run `rippled` in Reporting Mode
[Reporting mode](rippled-server-modes.html) is a mode of the XRP Ledger core server specialized for serving [HTTP and WebSocket APIs](rippled-api.html).
In reporting mode, the server does not connect to the peer-to-peer network. Instead, it uses gRPC to get validated data from one or more trusted servers that are connected to the P2P network.
It can then efficiently handle API calls, reducing the load on `rippled` servers running in P2P mode.
{{ include_svg("img/reporting-mode-basic-architecture.svg", "Figure 1: Working of `rippled` in reporting mode") }}
The reporting mode of `rippled` uses two datastores:
* The primary persistent datastore for `rippled` that includes transaction metadata, account states, and ledger headers. You can use NuDB (included with the source) or [Cassandra](https://cassandra.apache.org/) as the primary persistent datastore. If you use Cassandra, multiple reporting mode servers can share access to data in a single Cassandra instance or cluster.
* [PostgreSQL](https://www.postgresql.org/) database to hold relational data, which is used mainly by [tx method][] and [account_tx method][].
When a reporting mode server receives an API request, it loads the data from these data stores if possible. For requests that require data from the P2P network, the reporting mode forwards the request to a P2P server, and then passes the response back to the client.
Multiple reporting mode servers can share access to the same network accessible databases (PostgreSQL and Cassandra); at any given time, only one reporting mode server writes to the databases, while all the others read from the databases.
## How to Run Reporting Mode
### Prerequisites
1. Ensure that your system meets the [system requirements](system-requirements.html).
**Note:** If you choose to use Cassandra as the database, the disk requirements for `rippled` will be lower as the data will not be stored on your local disk.
2. In order to run reporting mode, you also need to run one or more `rippled` servers in [P2P mode](install-rippled.html). Ensure that you have at least one `rippled` server running in P2P mode.
3. A compatible version of CMake must be installed.
4. Install and configure the datastores required to run `rippled` in reporting mode.
1. Install PostgreSQL.
2. Install and configure the database to be used as the primary persistent datastore. You can choose to use Cassandra or NuDB.
3. On macOS, you need to manually install the Cassandra cpp driver. On all other platforms, the Cassandra driver is built as part of the `rippled` build.
brew install cassandra-cpp-driver
#### Install PostgreSQL
**Install PostgreSQL on Linux**
1. Download and [install PostgreSQL on Linux](https://www.postgresqltutorial.com/install-postgresql-linux/).
2. Connect to the PostgreSQL Database Server using `psql`, and create a user `newuser` and a database `reporting`.
psql postgres
CREATE ROLE newuser WITH LOGIN PASSWORD ‘password’;
ALTER ROLE newuser CREATEDB;
\q
psql postgres -U newuser
postgres=# create database reporting;
**Install PostgreSQL on macOS**
1. Download and install PostgreSQL on macOS.
brew install postgres
brew services start postgres
2. Connect to the PostgreSQL Database Server using `psql` and create a user `newuser` and a database `reporting`.
psql postgres
CREATE ROLE newuser WITH LOGIN PASSWORD ‘password’;
ALTER ROLE newuser CREATEDB;
\q
psql postgres -U newuser
postgres=# create database reporting;
#### Install and Configure the Primary Persistent Datastore
**Cassandra**
Install Cassandra and then create a keyspace for `rippled`, with replication. <!-- SPELLING_IGNORE: keyspace -->
While a replication factor of 3 is recommended, when running locally, replication is not needed and you can set `replication_factor` to 1.
$ cqlsh [host] [port]
> CREATE KEYSPACE `rippled` WITH REPLICATION =
{'class' : 'SimpleStrategy', 'replication_factor' : 1 };
**NuDB**
If you’re running `rippled` in reporting mode for your local network, you can choose to use NuDB instead of Cassandra as your backend database.
NuDB is installed as part of your `rippled` build setup and does not require any additional installation steps.
### Procedure
1. Build `rippled` for reporting mode on [Ubuntu](build-run-rippled-ubuntu.html) or [macOS](build-run-rippled-macos.html).
<!-- MULTICODE_BLOCK_START -->
*Linux*
wget https://github.com/Kitware/CMake/releases/download/v3.16.3/cmake-3.16.3-Linux-x86_64.sh
sudo sh cmake-3.16.3-Linux-x86_64.sh --prefix=/usr/local --exclude-subdir
cmake -B build -Dreporting=ON -DCMAKE_BUILD_TYPE=Debug
cmake --build build --parallel $(nproc)
*macOS*
cmake -B build -G "Unix Makefiles" -Dreporting=ON -DCMAKE_BUILD_TYPE=Debug
cmake --build build --parallel $(nproc)
<!-- MULTICODE_BLOCK_END -->
2. Create a configuration file to run `rippled` in reporting mode.
Make a copy of the example config file, `rippled-example.cfg`, and save it as `rippled-reporting-mode.cfg` in a location that enables you to run `rippled` as a non-root user. For example:
mkdir -p $HOME/.config/ripple
cp <RIPPLED_SOURCE>/cfg/rippled-example.cfg $HOME/.config/ripple/rippled-reporting-mode.cfg
3. Edit rippled-reporting-mode.cfg to set necessary file paths. The user you plan to run `rippled` as must have write permissions to all of the paths you specify here.
1. Set the `[node_db]` path to the location where you want to store the ledger database.
2. Set the `[database_path]` to the location where you want to store other database data. (This includes an SQLite database with configuration data, and is typically one level above the `[node_db]` path field.)
3. Set the `[debug_logfile]` to a path where `rippled` can write logging information.
Note that these are the only configurations required for `rippled` to start up successfully. All other configurations are optional and can be tweaked after you have a working server.
4. Edit the `rippled-reporting-mode.cfg` file to enable reporting mode:
1. Uncomment the `[reporting]` stanza or add a new one:
[reporting]
etl_source
read_only=0
2. List the `rippled` sources (ETL sources) to extract data from. These `rippled` servers must have gRPC enabled.
NOTE: Only include servers that you trust as reporting mode does not connect to the P2P network and hence cannot verify that the data actually matches the network consensus ledger.
[etl_source]
source_grpc_port=50051
source_ws_port=6006
source_ip=127.0.0.1
5. Configure the databases
1. Specify the Postgres DB for `[ledger_tx_tables]`:
[ledger_tx_tables]
conninfo = postgres://newuser:password@127.0.0.1/reporting
use_tx_tables=1
2. Specify the database for `[node_db]`.
<!-- MULTICODE_BLOCK_START -->
*NuDB*
[node_db]
type=NuDB
path=/home/ubuntu/ripple/
[ledger_history]
1000000
*Cassandra*
[node_db]
type=Cassandra
[ledger_history]
1000000
<!-- MULTICODE_BLOCK_END -->
6. Modify the configuration for `rippled` to open up ports.
1. Open the public websocket port:
[port_ws_admin_local]
port = 6006
ip = 127.0.0.1
admin = 127.0.0.1
protocol = ws
2. Open the gRPC port:
[port_grpc]
port = 60051
ip = 0.0.0.0
3. Add a secured gateway to the IP of your reporting system:
secure_gateway = 127.0.0.1
7. Run `rippled` in reporting mode:
./rippled --conf /home/ubuntu/.config/ripple/rippled-reporting-example.cfg
### What to Expect
Here are the excerpts of what you can expect to see on your terminal.
```text
Loading: "/home/ubuntu/.config/ripple/rippled-reporting-example.cfg"
2021-Dec-09 21:31:52.245577 UTC JobQueue:NFO Using 10 threads
2021-Dec-09 21:31:52.255422 UTC LedgerConsensus:NFO Consensus engine started (cookie: 17859050541656985684)
2021-Dec-09 21:31:52.256542 UTC ReportingETL::ETLSource:NFO Using IP to connect to ETL source: 127.0.0.1:50051
2021-Dec-09 21:31:52.257784 UTC ReportingETL::ETLSource:NFO Made stub for remote = { validated_ledger : , ip : 127.0.0.1 , web socket port : 6006, grpc port : 50051 }
2021-Dec-09 21:31:52.258032 UTC ReportingETL::LoadBalancer:NFO add : added etl source - { validated_ledger : , ip : 127.0.0.1 , web socket port : 6006, grpc port : 50051 }
2021-Dec-09 21:31:52.258327 UTC Application:NFO process starting: rippled-1.8.1+DEBUG
2021-Dec-09 21:31:52.719186 UTC PgPool:DBG max_connections: 18446744073709551615, timeout: 600, connection params: port: 5432, hostaddr: 127.0.0.1, user: newuser, password: *, channel_binding: prefer, dbname: reporting_test_core, host: 127.0.0.1, options: , sslmode: prefer, sslcompression: 0, sslsni: 1, ssl_min_protocol_version: TLSv1.2, gssencmode: prefer, krbsrvname: postgres, target_session_attrs: any
2021-Dec-09 21:31:52.788851 UTC PgPool:NFO server message: NOTICE: relation "version" already exists, skipping
2021-Dec-09 21:31:53.282807 UTC TaggedCache:DBG LedgerCache target size set to 384
2021-Dec-09 21:31:53.282892 UTC TaggedCache:DBG LedgerCache target age set to 240000000000
2021-Dec-09 21:31:53.283741 UTC Amendments:DBG Amendment 98DECF327BF79997AEC178323AD51A830E457BFC6D454DAF3E46E5EC42DC619F (CheckCashMakesTrustLine) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.283836 UTC Amendments:DBG Amendment 157D2D480E006395B76F948E3E07A45A05FE10230D88A7993C71F97AE4B1F2D1 (Checks) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.283917 UTC Amendments:DBG Amendment 1562511F573A19AE9BD103B5D6B9E01B3B46805AEC5D3C4805C902B514399146 (CryptoConditions) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.283975 UTC Amendments:DBG Amendment 86E83A7D2ECE3AD5FA87AB2195AE015C950469ABF0B72EAACED318F74886AE90 (CryptoConditionsSuite) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284016 UTC Amendments:DBG Amendment 30CD365592B8EE40489BA01AE2F7555CAC9C983145871DC82A42A31CF5BAE7D9 (DeletableAccounts) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284062 UTC Amendments:DBG Amendment F64E1EABBE79D55B3BB82020516CEC2C582A98A6BFE20FBE9BB6A0D233418064 (DepositAuth) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284099 UTC Amendments:DBG Amendment 3CBC5C4E630A1B82380295CDA84B32B49DD066602E74E39B85EF64137FA65194 (DepositPreauth) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284126 UTC Amendments:DBG Amendment DC9CA96AEA1DCF83E527D1AFC916EFAF5D27388ECA4060A88817C1238CAEE0BF (EnforceInvariants) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284153 UTC Amendments:DBG Amendment 07D43DCE529B15A10827E5E04943B496762F9A88E3268269D69C44BE49E21104 (Escrow) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284189 UTC Amendments:DBG Amendment 42426C4D4F1009EE67080A9B7965B44656D7714D104A72F9B4369F97ABF044EE (FeeEscalation) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284216 UTC Amendments:DBG Amendment 740352F2412A9909880C23A559FCECEDA3BE2126FED62FC7660D628A06927F11 (Flow) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284241 UTC Amendments:DBG Amendment 3012E8230864E95A58C60FD61430D7E1B4D3353195F2981DC12B0C7C0950FFAC (FlowCross) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284284 UTC Amendments:DBG Amendment AF8DF7465C338AE64B1E937D6C8DA138C0D63AD5134A68792BBBE1F63356C422 (FlowSortStrands) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284337 UTC Amendments:DBG Amendment 1F4AFA8FA1BC8827AD4C0F682C03A8B671DCDF6B5C4DE36D44243A684103EF88 (HardenedValidations) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284412 UTC Amendments:DBG Amendment 4C97EBA926031A7CF7D7B36FDE3ED66DDA5421192D63DE53FFB46E43B9DC8373 (MultiSign) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284455 UTC Amendments:DBG Amendment 586480873651E106F1D6339B0C4A8945BA705A777F3F4524626FF1FC07EFE41D (MultiSignReserve) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284491 UTC Amendments:DBG Amendment B4E4F5D2D6FB84DF7399960A732309C9FD530EAE5941838160042833625A6076 (NegativeUNL) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284528 UTC Amendments:DBG Amendment 08DE7D96082187F6E6578530258C77FAABABE4C20474BDB82F04B021F1A68647 (PayChan) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284592 UTC Amendments:DBG Amendment 00C1FC4A53E60AB02C864641002B3172F38677E29C26C5406685179B37E1EDAC (RequireFullyCanonicalSig) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284649 UTC Amendments:DBG Amendment CC5ABAE4F3EC92E94A59B1908C2BE82D2228B6485C00AFF8F22DF930D89C194E (SortedDirectories) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284703 UTC Amendments:DBG Amendment 532651B4FD58DF8922A49BA101AB3E996E5BFBF95A913B3E392504863E63B164 (TickSize) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284787 UTC Amendments:DBG Amendment 955DF3FA5891195A9DAEFA1DDC6BB244B545DDE1BAA84CBB25D5F12A8DA68A0C (TicketBatch) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284950 UTC Amendments:DBG Amendment 6781F8368C4771B83E8B821D88F580202BCB4228075297B19E4FDC5233F1EFDC (TrustSetAuth) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.284997 UTC Amendments:DBG Amendment B4D44CC3111ADD964E846FC57760C8B50FFCD5A82C86A72756F6B058DDDF96AD (fix1201) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285025 UTC Amendments:DBG Amendment E2E6F2866106419B88C50045ACE96368558C345566AC8F2BDF5A5B5587F0E6FA (fix1368) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285067 UTC Amendments:DBG Amendment 42EEA5E28A97824821D4EF97081FE36A54E9593C6E4F20CBAE098C69D2E072DC (fix1373) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285103 UTC Amendments:DBG Amendment 6C92211186613F9647A89DFFBAB8F94C99D4C7E956D495270789128569177DA1 (fix1512) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285129 UTC Amendments:DBG Amendment 67A34F2CF55BFC0F93AACD5B281413176FEE195269FA6D95219A2DF738671172 (fix1513) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285153 UTC Amendments:DBG Amendment 5D08145F0A4983F23AFFFF514E83FAD355C5ABFBB6CAB76FB5BC8519FF5F33BE (fix1515) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285176 UTC Amendments:DBG Amendment B9E739B8296B4A1BB29BE990B17D66E21B62A300A909F25AC55C22D6C72E1F9D (fix1523) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285202 UTC Amendments:DBG Amendment 1D3463A5891F9E589C5AE839FFAC4A917CE96197098A1EF22304E1BC5B98A454 (fix1528) is supported and will be down voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285256 UTC Amendments:DBG Amendment CA7C02118BA27599528543DFE77BA6838D1B0F43B447D4D7F53523CE6A0E9AC2 (fix1543) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285290 UTC Amendments:DBG Amendment 7117E2EC2DBF119CA55181D69819F1999ECEE1A0225A7FD2B9ED47940968479C (fix1571) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285343 UTC Amendments:DBG Amendment FBD513F1B893AC765B78F250E6FFA6A11B573209D1842ADC787C850696741288 (fix1578) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285381 UTC Amendments:DBG Amendment 58BE9B5968C4DA7C59BA900961828B113E5490699B21877DEF9A31E9D0FE5D5F (fix1623) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285424 UTC Amendments:DBG Amendment 25BA44241B3BD880770BFA4DA21C7180576831855368CBEC6A3154FDE4A7676E (fix1781) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285464 UTC Amendments:DBG Amendment 4F46DF03559967AC60F2EB272FEFE3928A7594A45FF774B87A7E540DB0F8F068 (fixAmendmentMajorityCalc) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285500 UTC Amendments:DBG Amendment 8F81B066ED20DAECA20DF57187767685EEF3980B228E0667A650BAF24426D3B4 (fixCheckThreading) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285527 UTC Amendments:DBG Amendment C4483A1896170C66C098DEA5B0E024309C60DC960DE5F01CD7AF986AA3D9AD37 (fixMasterKeyAsRegularKey) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285550 UTC Amendments:DBG Amendment 621A0B264970359869E3C0363A899909AAB7A887C8B73519E4ECF952D33258A8 (fixPayChanRecipientOwnerDir) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285575 UTC Amendments:DBG Amendment 89308AF3B8B10B7192C4E613E1D2E4D9BA64B2EE2D5232402AE82A6A7220D953 (fixQualityUpperBound) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285614 UTC Amendments:DBG Amendment B6B3EEDC0267AB50491FDC450A398AF30DBCD977CECED8BEF2499CAB5DAC19E2 (fixRmSmallIncreasedQOffers) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285651 UTC Amendments:DBG Amendment 452F5906C46D46F407883344BFDD90E672B672C5E9943DB4891E3A34FEEEB9DB (fixSTAmountCanonicalize) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.285725 UTC Amendments:DBG Amendment 2CD5286D8D687E98B41102BDD797198E81EA41DF7BD104E6561FEB104EFF2561 (fixTakerDryOfferRemoval) is supported and will be up voted if not enabled on the ledger.
2021-Dec-09 21:31:53.290446 UTC Server:NFO Opened 'port_rpc_admin_local' (ip=127.0.0.1:7005, admin IPs:127.0.0.1, http)
2021-Dec-09 21:31:53.290834 UTC Server:NFO Opened 'port_ws_admin_local' (ip=127.0.0.1:7006, admin IPs:127.0.0.1, ws)
2021-Dec-09 21:31:53.290984 UTC Application:WRN Running in standalone mode
2021-Dec-09 21:31:53.291048 UTC NetworkOPs:NFO STATE->full
2021-Dec-09 21:31:53.291192 UTC Application:FTL Startup RPC:
{
"command" : "log_level",
"severity" : "debug"
}
2021-Dec-09 21:31:53.291347 UTC RPCHandler:DBG RPC call log_level completed in 2.2e-08seconds
2021-Dec-09 21:31:53.291440 UTC Application:FTL Result:
{
"warnings" :
[
{
"id" : 1004,
"message" : "This is a reporting server. The default behavior of a reporting server is to only return validated data. If you are looking for not yet validated data, include \"ledger_index : current\" in your request, which will cause this server to forward the request to a p2p node. If the forward is successful the response will include \"forwarded\" : \"true\""
}
]
}
2021-Dec-09 21:31:53.291502 UTC ReportingETL:NFO Starting reporting etl
2021-Dec-09 21:31:53.291605 UTC Application:NFO Application starting. Version is 1.8.1+DEBUG
2021-Dec-09 21:31:53.291747 UTC LoadManager:DBG Starting
2021-Dec-09 21:31:53.291846 UTC gRPC Server:NFO Starting gRPC server at 0.0.0.0:60051
2021-Dec-09 21:31:53.293246 UTC LedgerCleaner:DBG Started
2021-Dec-09 21:31:53.295543 UTC ReportingETL::ETLSource:DBG handleMessage : Received a message on ledger subscription stream. Message : {
"result" : {},
"status" : "success",
"type" : "response"
}
- { validated_ledger : , ip : 127.0.0.1 , web socket port : 6006, grpc port : 50051 }
2021-Dec-09 21:31:53.368075 UTC ReportingETL:NFO monitor : Database is empty. Will download a ledger from the network.
2021-Dec-09 21:31:53.368183 UTC ReportingETL:NFO monitor : Waiting for next ledger to be validated by network...
```
## Frequently Asked Questions
**Do I need to run more than one instance of `rippled` to use reporting mode?**
Yes. A `rippled` server running in reporting mode does not connect to the peer-to-peer network, but instead extracts validated data from one or more `rippled` servers that are connected to the network, so you need to run at least one P2P mode server.
**I’ve already installed `rippled`. Can I update the configuration file to enable reporting mode and restart `rippled`?**
No. Currently, you need to download the source and build `rippled` for reporting mode. There are initiatives in progress to provide packages for reporting mode.
**To run `rippled` in reporting mode, I need at least one `rippled` server running in P2P mode too. Does this mean I need double the disk space?**
The answer depends on the location of your primary data store. If you use Cassandra as the primary data store, the reporting mode server stores much less data on its local disk. The PostgreSQL server can be remote as well. You can have multiple reporting mode servers share the same data this way.
Lastly, the P2P mode server only needs to keep very recent history, while the reporting mode server keeps long term history.
For more information on system requirements to run `rippled`, see the [`rippled` system requirements](system-requirements.html).
**How can I confirm the validity of the data that comes from the PostgreSQL or Cassandra database?**
When `rippled` runs in reporting mode, it only serves validated data from the ETL source specified in the config file. If you are using someone else's `rippled` server in P2P mode as the ETL source, you are implicitly trusting that server. If not, you need to run your own `rippled` node in P2P mode.
**Is it possible to make traditional SQL queries to the relational database rather than using the API?**
Technically, you *can* directly access the database if you want. However, the data is stored as binary blobs and you have to decode the blobs to access the component fields. This makes traditional SQL queries much less useful since they cannot find and filter the individual fields of the data.
<!--{# common link defs #}-->
{% include '_snippets/rippled-api-links.md' %}
{% include '_snippets/tx-type-links.md' %}
{% include '_snippets/rippled_versions.md' %}

123
content/tutorials/manage-the-clio-server/install-clio-on-ubuntu.md Normal file
View File

@@ -0,0 +1,123 @@
---
html: install-clio-on-ubuntu.html
parent: install-clio.html
blurb: Install a precompiled Clio binary on Ubuntu Linux.
labels:
- Clio Server
---
# Install Clio on Ubuntu Linux
This page describes the recommended instructions for installing the latest stable version of Clio on **Ubuntu Linux 20.04 or higher** using the [`apt`](https://ubuntu.com/server/docs) utility.
These instructions install a binary that has been compiled by Ripple.
## Prerequisites
Before you install Clio, you must meet the following requirements.
1. Ensure that your system meets the [system requirements](system-requirements.html).
**Note:** Clio uses Cassandra as the database and the disk requirements for Clio are lower than the requirements for `rippled` as the data will not be stored on your local disk.
2. A compatible version of CMake must be installed. Clio requires C++20 and Boost 1.75.0 or higher.
3. Access to a Cassandra cluster that is running locally or remote. You can choose to install and configure a Cassandra cluster manually by following the installation instructions, or run Cassandra on a Docker container using the following commands.
1. If you choose to persist Clio data, run Cassandra in a Docker container and specify an empty directory to store Clio data:
docker run --rm -it --network=host --name cassandra -v $PWD/cassandra_data:/var/lib/
cassandra cassandra:4.0.4
2. If you do not wish to persist Clio data, run the following command:
docker run --rm -it --network=host --name cassandra cassandra:4.0.4
5. In order to run Clio, you also need to run one or more `rippled` servers in [P2P mode](install-rippled.html). The `rippled` servers can either be local or remote. Ensure that you have at least one `rippled` server running in P2P mode.
## Installation Steps
1. Update repositories:
sudo apt -y update
2. Install utilities:
sudo apt -y install apt-transport-https ca-certificates wget gnupg
3. Add Ripple's package-signing GPG key to your list of trusted keys:
sudo mkdir /usr/local/share/keyrings/
wget -q -O - "https://repos.ripple.com/repos/api/gpg/key/public" | gpg --dearmor > ripple-key.gpg
sudo mv ripple-key.gpg /usr/local/share/keyrings
4. Check the fingerprint of the newly-added key:
gpg /usr/local/share/keyrings/ripple-key.gpg
The output should include an entry for Ripple such as the following:
gpg: WARNING: no command supplied. Trying to guess what you mean ...
pub rsa3072 2019-02-14 [SC] [expires: 2026-02-17]
C0010EC205B35A3310DC90DE395F97FFCCAFD9A2
uid TechOps Team at Ripple <techops+rippled@ripple.com>
sub rsa3072 2019-02-14 [E] [expires: 2026-02-17]
In particular, make sure that the fingerprint matches. (In the above example, the fingerprint is on the third line, starting with `C001`.)
4. Add the appropriate Ripple repository for your operating system version:
echo "deb [signed-by=/usr/local/share/keyrings/ripple-key.gpg] https://repos.ripple.com/repos/rippled-deb focal stable" | \
sudo tee -a /etc/apt/sources.list.d/ripple.list
The above example is appropriate for **Ubuntu 20.04 Focal Fossa**.
5. Fetch the Ripple repository.
sudo apt -y update
6. Install the Clio software package:
sudo apt -y install rippled
7. Check the status of the Clio service:
systemctl status rippled.service
The `rippled` service should start automatically. If not, you can start it manually:
sudo systemctl start rippled.service
To configure it to start automatically on boot:
sudo systemctl enable rippled.service
8. Optional: allow `rippled` to bind to privileged ports.
This allows you to serve incoming API requests on port 80 or 443. (If you want to do so, you must also update the config file's port settings.)
sudo setcap 'cap_net_bind_service=+ep' /opt/ripple/bin/rippled
<!--_ -->
## See Also
- **Concepts:**
- [The Clio Server](the-clio-server.html)
- **Tutorials:**
- [Build and Run Clio Server](build-run-clio-ubuntu.html)
- **References:**
- [rippled and Clio API Reference](rippled-api.html)
<!--{# common link defs #}-->
{% include '_snippets/rippled-api-links.md' %}
{% include '_snippets/tx-type-links.md' %}
{% include '_snippets/rippled_versions.md' %}