From 4e8fe0f205c0ab2138bf89da9e2ecd3e94cef000 Mon Sep 17 00:00:00 2001 From: mDuo13 Date: Wed, 9 May 2018 14:53:30 -0700 Subject: [PATCH] Reorg 'The rippled Server' concepts --- .../concepts/the-rippled-server/clustering.md | 23 +++ .../the-rippled-server/peer-protocol.md | 177 ++++++++++++++++++ .../the-rippled-server/the-rippled-server.md | 12 -- .../rippled-api/rippled-old-monolith.md | 171 ----------------- .../rippled-setup.md | 21 --- dactyl-config.yml | 38 +++- 6 files changed, 237 insertions(+), 205 deletions(-) create mode 100644 content/concepts/the-rippled-server/clustering.md create mode 100644 content/concepts/the-rippled-server/peer-protocol.md diff --git a/content/concepts/the-rippled-server/clustering.md b/content/concepts/the-rippled-server/clustering.md new file mode 100644 index 0000000000..3e25dbab4e --- /dev/null +++ b/content/concepts/the-rippled-server/clustering.md @@ -0,0 +1,23 @@ +# Clustering + +If you are running multiple [`rippled` servers](the-rippled-server.html) in a single datacenter, you can configure those servers into a cluster to maximize efficiency. Running your `rippled` servers in a cluster provides the following benefits: + +- Clustered `rippled` servers share the work of cryptography. If one server has verified the authenticity of a message, the other servers in the cluster trust it and do not re-verify. +- Clustered servers share information about peers and API clients that are misbehaving or abusing the network. This makes it harder to attack all servers of the cluster at once. +- Clustered servers always propagate transactions throughout the cluster, even if the transaction does not meet the current load-based transaction fee on some of them. + +If you are running a validator as a [private peer](peer-protocol.html#private-peers), Ripple recommends using a cluster of `rippled` servers as proxy servers. + +## Configuring Clustering + +To enable clustering, change the following sections of your [config file](https://github.com/ripple/rippled/blob/d7def5509d8338b1e46c0adf309b5912e5168af0/doc/rippled-example.cfg#L297-L346) for each server: + +1. List the IP address and port of each other server under the `[ips_fixed]` section. The port should be the one from the other servers' `protocol = peer` setting in their `rippled.cfg`. Example: + + [ips_fixed] + 192.168.0.1 51235 + 192.168.0.2 51235 + +2. Generate a unique seed (using the [validation_create method][]) for each of your servers, and configure it under the `[node_seed]` section. The `rippled` server uses this key to sign its messages to other servers in the peer-to-peer network. + +3. Add each node's peer-communication public key (generated in step 2) to the `[cluster_nodes]` section of each of your other servers' config files. diff --git a/content/concepts/the-rippled-server/peer-protocol.md b/content/concepts/the-rippled-server/peer-protocol.md new file mode 100644 index 0000000000..e96bcd427d --- /dev/null +++ b/content/concepts/the-rippled-server/peer-protocol.md @@ -0,0 +1,177 @@ +# Peer Protocol + +Servers in the XRP Ledger communicate to each other using the XRP Ledger peer protocol, also known as RTXP. Peer servers connect via HTTPS and then do an [HTTP upgrade](https://tools.ietf.org/html/rfc7230#section-6.7) to switch to RTXP. (For more information, see the [Overlay Network](https://github.com/ripple/rippled/blob/906ef761bab95f80b0a7e0cab3b4c594b226cf57/src/ripple/overlay/README.md#handshake) article in the [`rippled` repository](https://github.com/ripple/rippled).) + +## Configuring the Peer Protocol + +To participate in the XRP Ledger, `rippled` servers connects to arbitrary peers using the peer protocol. (All such peers are treated as untrusted, unless they are [clustered](tutorial-rippled-setup.html#clustering) with the current server.) + +Ideally, the server should be able to send _and_ receive connections on the peer port. You should forward the port used for the peer protocol through your firewall to the `rippled` server. The [default `rippled` configuration file](https://github.com/ripple/rippled/blob/release/doc/rippled-example.cfg) listens for incoming peer protocol connections on port 51235 on all network interfaces. You can change the port used by editing the appropriate stanza in your `rippled.cfg` file. + +Example: + +``` +[port_peer] +port = 51235 +ip = 0.0.0.0 +protocol = peer +``` + +## Peer Crawler + +The Peer Crawler asks a `rippled` server to report information about the other `rippled` servers it is connected to as peers. The [`peers` command](#peers) in the [WebSocket and JSON-RPC APIs](#websocket-and-json-rpc-apis) also returns a similar, more comprehensive set of information, but requires [administrative access](#connecting-to-rippled) to the server. The Peer Crawler response is available to other servers on a non-privileged basis through the Peer Protocol (RTXP) port. + +#### Request Format + +To request the Peer Crawler information, make the following HTTP request: + +* **Protocol:** https +* **HTTP Method:** GET +* **Host:** (any `rippled` server, by hostname or IP address) +* **Port:** (port number where the `rippled` server uses the Peer Protocol, typically 51235) +* **Path:** `/crawl` +* **Notes:** Most `rippled` servers use a self-signed certificate to respond to the request. By default, most tools (including web browsers) flag or block such responses for being untrusted. You must ignore the certificate checking (for example, if using cURL, add the `--insecure` flag) to display a response from those servers. + +#### Response Format + +The response has the status code **200 OK** and a JSON object in the message body. + +The JSON object has the following fields: + +| `Field` | Value | Description | +|:-----------------|:------|:--------------------------------------------------| +| `overlay.active` | Array | Array of Peer Objects, where each member is a peer that is connected to this server. | + +Each member of the `active` array is a Peer Object with the following fields: + +| `Field` | Value | Description | +|:-------------|:-------------------------|:-----------------------------------| +| `ip` | String (IPv4 Address) | The IP address of this connected peer. | +| `port` | String (Number) | The port number on the peer server that serves RTXP. Typically 51235. | +| `public_key` | String (Base-64 Encoded) | The public key of the ECDSA key pair used by this peer to sign RTXP messages. (This is the same data as the `pubkey_node` reported in the peer server's [`server_info` command](#server-info).) | +| `type` | String | The value `in` or `out`, indicating whether the TCP connection to the peer is incoming or outgoing. | +| `uptime` | Number | The number of seconds the server has been has been connected to this peer. | +| `version` | String | The `rippled` version number the peer reports to be using. | + +#### Example + +Request: + + + +*HTTP* + +``` +GET https://s1.ripple.com:51235/crawl +``` + +*cURL* + +``` +curl -k https://s1.ripple.com:51235/crawl +``` + + + +Response: + +```json +200 OK +{ + "overlay": { + "active": [{ + "ip": "208.43.252.243", + "port": "51235", + "public_key": "A2GayQNaj7qbqLFiCFW2UXtAnEPghP/KWVqix2gUa6dM", + "type": "out", + "uptime": 107926, + "version": "rippled-0.31.0-rc1" + }, { + "ip": "184.172.237.226", + "port": "51235", + "public_key": "Asv/wKq/dqMWbP2M4eR+QvkuojYMLRXhKhIPnW40bsaF", + "type": "out", + "uptime": 247376, + "version": "rippled-0.31.0-rc1" + }, { + "ip": "54.186.73.52", + "port": "51235", + "public_key": "AjikFnq0P2XybCyREr2KPiqXqJteqwPwVRVbVK+93+3o", + "type": "out", + "uptime": 328776, + "version": "rippled-0.31.0-rc1" + }, { + "ip": "169.53.155.59", + "port": "51235", + "public_key": "AyIcVhAhOGnP0vtfCt+HKUrx9B2fDvP/4XUkOtVQ37g/", + "type": "out", + "uptime": 328776, + "version": "rippled-0.31.1" + }, { + "ip": "169.53.155.44", + "port": "51235", + "public_key": "AuVZszWXgMgM8YuTVhQsGE9OciEeBD8aMVe1mFid3n63", + "type": "out", + "uptime": 328776, + "version": "rippled-0.32.0-b12" + }, { + "ip": "184.173.45.39", + "port": "51235", + "public_key": "Ao2GbGbp2QYQ2B4S9ckCtON7CsZdXqdK5Yon4x7qmWFm", + "type": "out", + "uptime": 63336, + "version": "rippled-0.31.0-rc1" + }, { + "ip": "169.53.155.34", + "port": "51235", + "public_key": "A3inNJsIQzO7z7SS7uB9DyvN0wsiS9it/RGY/kNx6KEG", + "type": "out", + "uptime": 328776, + "version": "rippled-0.31.0-rc1" + }, { + "ip": "169.53.155.45", + "port": "51235", + "public_key": "AglUUjwXTC2kUlK41WjDs2eAVN0SnlMpzYA9lEgB0UGP", + "type": "out", + "uptime": 65443, + "version": "rippled-0.31.0-rc1" + }, { + "ip": "99.110.49.91", + "port": "51301", + "public_key": "AuQDH0o+4fpl2n+pR5U0Y4FTj0oGr4iEKe0MObPcSYj9", + "type": "out", + "uptime": 328776, + "version": "rippled-0.32.0-b9" + }, { + "ip": "169.53.155.36", + "port": "51235", + "public_key": "AsR4xub7MLg2Zl5Fwd/n7dTz7mhbBoSyCc/v9bnubrVy", + "type": "out", + "uptime": 328776, + "version": "rippled-0.31.0-rc1" + }] + } +} +``` + +## Private Peers + + + +You can use the `[peer_private]` stanza of the `rippled` config file to request that peer servers do not report your IP address in the Peer Crawler response. Servers you do not control can be modified not to respect this setting. However, you can use this to hide the IP addresses of `rippled` servers you control that _only_ connect to peers you control (using `[ips_fixed]` and a firewall). In this way, you can use public rippled servers you control as proxies for your private rippled servers. + +To protect an important [validating server](tutorial-rippled-setup.html#types-of-rippled-servers), Ripple recommends configuring one or more public tracking servers to act as proxies, while the validating server is protected by a firewall and only connects to the public tracking servers. + + + +Example configuration: + +``` +# Configuration on a private server that only connects through +# a second rippled server at IP address 10.1.10.55 +[ips_fixed] +10.1.10.55 + +[peer_private] +1 +``` diff --git a/content/concepts/the-rippled-server/the-rippled-server.md b/content/concepts/the-rippled-server/the-rippled-server.md index b1cbcceece..07bb179aa6 100644 --- a/content/concepts/the-rippled-server/the-rippled-server.md +++ b/content/concepts/the-rippled-server/the-rippled-server.md @@ -2,18 +2,6 @@ `rippled` is the core peer-to-peer server that manages the XRP Ledger. This section covers concepts that help you learn the "what" and "why" behind fundamental aspects of the `rippled` server. -Other pages in this category: - -* **[History Sharding](history-sharding.html)** - - Historical sharding distributes the transaction history of the XRP Ledger into segments, called shards, across servers in the XRP Ledger network. Understand the purpose of history sharding and when to use it. - - -* **[Clustering](clustering.html)** - - If you are running multiple `rippled` servers in a single datacenter, you can configure those servers into a cluster to maximize efficiency. Understand the purpose of clustering and when to use it. - - ## Types of rippled Servers The `rippled` server software can run in several modes depending on its configuration, including: diff --git a/content/references/rippled-api/rippled-old-monolith.md b/content/references/rippled-api/rippled-old-monolith.md index ed0f1252f7..10322cd249 100755 --- a/content/references/rippled-api/rippled-old-monolith.md +++ b/content/references/rippled-api/rippled-old-monolith.md @@ -730,177 +730,6 @@ The `rippled` server also provides a few commands purely for convenience. {% include 'rippled-api-methods/stop.md' %} -# Peer Protocol - -Servers in the XRP Ledger communicate to each other using the XRP Ledger peer protocol, also known as RTXP. Peer servers connect via HTTPS and then do an [HTTP upgrade](https://tools.ietf.org/html/rfc7230#section-6.7) to switch to RTXP. (For more information, see the [Overlay Network](https://github.com/ripple/rippled/blob/906ef761bab95f80b0a7e0cab3b4c594b226cf57/src/ripple/overlay/README.md#handshake) article in the [`rippled` repository](https://github.com/ripple/rippled).) - -## Configuring the Peer Protocol - -To participate in the XRP Ledger, `rippled` servers connects to arbitrary peers using the peer protocol. (All such peers are treated as untrusted, unless they are [clustered](tutorial-rippled-setup.html#clustering) with the current server.) - -Ideally, the server should be able to send _and_ receive connections on the peer port. You should forward the port used for the peer protocol through your firewall to the `rippled` server. The [default `rippled` configuration file](https://github.com/ripple/rippled/blob/release/doc/rippled-example.cfg) listens for incoming peer protocol connections on port 51235 on all network interfaces. You can change the port used by editing the appropriate stanza in your `rippled.cfg` file. - -Example: - -``` -[port_peer] -port = 51235 -ip = 0.0.0.0 -protocol = peer -``` - -## Peer Crawler - -The Peer Crawler asks a `rippled` server to report information about the other `rippled` servers it is connected to as peers. The [`peers` command](#peers) in the [WebSocket and JSON-RPC APIs](#websocket-and-json-rpc-apis) also returns a similar, more comprehensive set of information, but requires [administrative access](#connecting-to-rippled) to the server. The Peer Crawler response is available to other servers on a non-privileged basis through the Peer Protocol (RTXP) port. - -#### Request Format - -To request the Peer Crawler information, make the following HTTP request: - -* **Protocol:** https -* **HTTP Method:** GET -* **Host:** (any `rippled` server, by hostname or IP address) -* **Port:** (port number where the `rippled` server uses the Peer Protocol, typically 51235) -* **Path:** `/crawl` -* **Notes:** Most `rippled` servers use a self-signed certificate to respond to the request. By default, most tools (including web browsers) flag or block such responses for being untrusted. You must ignore the certificate checking (for example, if using cURL, add the `--insecure` flag) to display a response from those servers. - -#### Response Format - -The response has the status code **200 OK** and a JSON object in the message body. - -The JSON object has the following fields: - -| `Field` | Value | Description | -|:-----------------|:------|:--------------------------------------------------| -| `overlay.active` | Array | Array of Peer Objects, where each member is a peer that is connected to this server. | - -Each member of the `active` array is a Peer Object with the following fields: - -| `Field` | Value | Description | -|:-------------|:-------------------------|:-----------------------------------| -| `ip` | String (IPv4 Address) | The IP address of this connected peer. | -| `port` | String (Number) | The port number on the peer server that serves RTXP. Typically 51235. | -| `public_key` | String (Base-64 Encoded) | The public key of the ECDSA key pair used by this peer to sign RTXP messages. (This is the same data as the `pubkey_node` reported in the peer server's [`server_info` command](#server-info).) | -| `type` | String | The value `in` or `out`, indicating whether the TCP connection to the peer is incoming or outgoing. | -| `uptime` | Number | The number of seconds the server has been has been connected to this peer. | -| `version` | String | The `rippled` version number the peer reports to be using. | - -#### Example - -Request: - - - -*HTTP* - -``` -GET https://s1.ripple.com:51235/crawl -``` - -*cURL* - -``` -curl -k https://s1.ripple.com:51235/crawl -``` - - - -Response: - -```json -200 OK -{ - "overlay": { - "active": [{ - "ip": "208.43.252.243", - "port": "51235", - "public_key": "A2GayQNaj7qbqLFiCFW2UXtAnEPghP/KWVqix2gUa6dM", - "type": "out", - "uptime": 107926, - "version": "rippled-0.31.0-rc1" - }, { - "ip": "184.172.237.226", - "port": "51235", - "public_key": "Asv/wKq/dqMWbP2M4eR+QvkuojYMLRXhKhIPnW40bsaF", - "type": "out", - "uptime": 247376, - "version": "rippled-0.31.0-rc1" - }, { - "ip": "54.186.73.52", - "port": "51235", - "public_key": "AjikFnq0P2XybCyREr2KPiqXqJteqwPwVRVbVK+93+3o", - "type": "out", - "uptime": 328776, - "version": "rippled-0.31.0-rc1" - }, { - "ip": "169.53.155.59", - "port": "51235", - "public_key": "AyIcVhAhOGnP0vtfCt+HKUrx9B2fDvP/4XUkOtVQ37g/", - "type": "out", - "uptime": 328776, - "version": "rippled-0.31.1" - }, { - "ip": "169.53.155.44", - "port": "51235", - "public_key": "AuVZszWXgMgM8YuTVhQsGE9OciEeBD8aMVe1mFid3n63", - "type": "out", - "uptime": 328776, - "version": "rippled-0.32.0-b12" - }, { - "ip": "184.173.45.39", - "port": "51235", - "public_key": "Ao2GbGbp2QYQ2B4S9ckCtON7CsZdXqdK5Yon4x7qmWFm", - "type": "out", - "uptime": 63336, - "version": "rippled-0.31.0-rc1" - }, { - "ip": "169.53.155.34", - "port": "51235", - "public_key": "A3inNJsIQzO7z7SS7uB9DyvN0wsiS9it/RGY/kNx6KEG", - "type": "out", - "uptime": 328776, - "version": "rippled-0.31.0-rc1" - }, { - "ip": "169.53.155.45", - "port": "51235", - "public_key": "AglUUjwXTC2kUlK41WjDs2eAVN0SnlMpzYA9lEgB0UGP", - "type": "out", - "uptime": 65443, - "version": "rippled-0.31.0-rc1" - }, { - "ip": "99.110.49.91", - "port": "51301", - "public_key": "AuQDH0o+4fpl2n+pR5U0Y4FTj0oGr4iEKe0MObPcSYj9", - "type": "out", - "uptime": 328776, - "version": "rippled-0.32.0-b9" - }, { - "ip": "169.53.155.36", - "port": "51235", - "public_key": "AsR4xub7MLg2Zl5Fwd/n7dTz7mhbBoSyCc/v9bnubrVy", - "type": "out", - "uptime": 328776, - "version": "rippled-0.31.0-rc1" - }] - } -} -``` - -### Concealing Peer Information - -You can use the `[peer_private]` stanza of the `rippled` config file to request that peer servers do not report your IP address in the Peer Crawler response. You cannot force peers you do not control to follow this request, if they run custom software. However, you can use this to hide the IP addresses of `rippled` servers you control that _only_ connect to peers you control (using `[ips_fixed]` and a firewall). This can help to protect important [validating servers](tutorial-rippled-setup.html#types-of-rippled-servers). - -Example: - -``` -# Configuration on a private server that only connects through -# a second rippled server at IP address 10.1.10.55 -[ips_fixed] -10.1.10.55 - -[peer_private] -1 -``` {% include '_snippets/rippled_versions.md' %} diff --git a/content/tutorials/manage-the-rippled-server/rippled-setup.md b/content/tutorials/manage-the-rippled-server/rippled-setup.md index fb0b1ac6b5..194fb6edcf 100644 --- a/content/tutorials/manage-the-rippled-server/rippled-setup.md +++ b/content/tutorials/manage-the-rippled-server/rippled-setup.md @@ -248,24 +248,3 @@ Over time, there may also be smaller, temporary test networks for specific purpo ### Parallel Networks and Consensus There is no `rippled` setting that defines which network it uses. Instead, it uses the consensus of validators it trusts to know which ledger to accept as the truth. When different consensus groups of `rippled` instances only trust other members of the same group, each group continues as a parallel network. Even if malicious or misbehaving computers connect to both networks, the consensus process overrides the confusion as long as the members of each network are not configured to trust members of another network in excess of their quorum settings. - - -## Clustering - -If you are running multiple `rippled` servers in a single datacenter, you can configure those servers into a cluster to maximize efficiency. Running your `rippled` servers in a cluster provides the following benefits: - -* Clustered `rippled` servers share the work of cryptography. If one server has verified the authenticity of a message, the other servers in the cluster trust it and do not re-verify. -* Clustered servers share information about peers and API clients that are misbehaving or abusing the network. This makes it harder to attack all servers of the cluster at once. -* Clustered servers always propagate transactions throughout the cluster, even if the transaction does not meet the current load-based transaction fee on some of them. - -To enable clustering, change the following sections of your [config file](https://github.com/ripple/rippled/blob/d7def5509d8338b1e46c0adf309b5912e5168af0/doc/rippled-example.cfg#L297-L346) for each server: - -* List the IP address and port of each other server under the `[ips_fixed]` section. The port should be the one from the other servers' `protocol = peer` setting in their `rippled.cfg`. Example: - - [ips_fixed] - 192.168.0.1 51235 - 192.168.0.2 51235 - -* Generate a unique seed (using the [validation_create method][]) for each of your servers, and configure it under the `[node_seed]` section. The `rippled` server uses this key to sign its messages to other servers in the peer-to-peer network. - -* Add the public keys (for peer communication) of each of your other servers under the `[cluster_nodes]` section. diff --git a/dactyl-config.yml b/dactyl-config.yml index c302af3577..80ea430054 100644 --- a/dactyl-config.yml +++ b/dactyl-config.yml @@ -264,11 +264,47 @@ pages: funnel: Docs doc_type: Concepts category: The rippled Server + template: template-landing-children.html blurb: rippled is the core peer-to-peer server that manages the XRP Ledger. This section covers concepts that help you learn the "what" and "why" behind fundamental aspects of the rippled server. targets: - local - # TODO: history sharding, stand-alone mode, clustering, peer protocol + - md: concepts/the-rippled-server/clustering.md + html: clustering.html + funnel: Docs + doc_type: Concepts + category: The rippled Server + blurb: Run rippled servers in a cluster to share the load of cryptography between them. + targets: + - local + + - md: concepts/the-rippled-server/history-sharding.md + html: history-sharding.html + funnel: Docs + doc_type: Concepts + category: The rippled Server + blurb: History sharding divides the work of keeping historical ledger data among rippled servers. + targets: + - local + + - md: concepts/the-rippled-server/peer-protocol.md + html: peer-protocol.html + funnel: Docs + doc_type: Concepts + category: The rippled Server + blurb: The peer protocol specifies the language rippled servers speak to each other. + targets: + - local + + - md: concepts/the-rippled-server/stand-alone-mode.md + html: stand-alone-mode.html + funnel: Docs + doc_type: Concepts + category: The rippled Server + blurb: Stand-alone mode provides a way to test rippled servers without connecting to a network of peers. + targets: + - local + # Tutorials --------------------------------------------------------------------