Merge pull request #785 from mDuo13/rate-limiting

Rate limiting

content/references/rippled-api/api-conventions/rate-limiting.md

@@ -0,0 +1,62 @@
+# Rate Limiting
+
+The `rippled` server limits the rate at which API clients can make requests on public APIs. Rate limiting is based on the IP address of the client, so clients behind [network address translation](https://en.wikipedia.org/wiki/Network_address_translation) share a limit based on their public IP address.
+
+**Tip:** Rate limiting does not apply when the client is connected [as an admin](get-started-with-the-rippled-api.html#admin-access).
+
+When a client is approaching the rate limit, the server adds the field `"warning": "load"` at the top level of an [API response](response-formatting.html). This warning is not added to every response, but the server may send several such warnings before it disconnects a client.
+
+If a client goes past the rate limit, the server disconnects that client and does not serve further requests from the client's API address for a while. The WebSocket and JSON-RPC APIs use different disconnect messages.
+
+## WebSocket API Disconnect Message
+
+For the WebSocket API, the server closes the connection and provides a close message and code. The way you access these messages depends on your WebSocket client implementation. For example, using the [Node.js ws library](https://github.com/websockets/ws), the following code prints the close reason when disconnected:
+
+```js
+const WebSocket = require('ws')
+const ws = new WebSocket('ws://localhost:6007/')
+ws.on('close', (code,reason) => {
+  console.log("Disconnected. \ncode: ", code, "\nreason: ", reason)
+})
+
+// If rate limited, prints:
+// Disconnected.
+// code:  1008
+// reason:  threshold exceeded
+```
+
+If the connection is closed because of rate limiting, the close code is `1008` and the close message is the string `threshold exceeded`.
+
+## JSON-RPC Rate Limited Error
+
+For a JSON-RPC API request, the server responds with the HTTP status code **503 Service Unavailable** when the client is over the rate limit. This response has a text (not JSON) body stating that the server is overloaded:
+
+```text
+503 Service Unavailable
+
+Server is overloaded
+```
+
+## Rate Per Request
+[[Source]](https://github.com/ripple/rippled/blob/master/src/ripple/resource/Fees.h "Source")
+
+The server calculates a client's usage rate based on the number of requests made over time, and weighs different types of requests based on approximately how much work the server must do to serve them. Follow-up messages from the server for the [subscribe method][] and [path_find method][] also count towards a client's usage rate.
+
+The usage rate drops off exponentially over time, so a client that does not make requests automatically has its access restored after a period of seconds to minutes.
+
+## See Also
+
+- **Concepts:**
+    - [The `rippled` Server](the-rippled-server.html)
+    - [Software Ecosystem](software-ecosystem.html)
+- **Tutorials:**
+    - [Getting Started with XRP Ledger APIs](get-started-with-the-rippled-api.html)
+    - [Troubleshooting rippled](troubleshoot-the-rippled-server.html)
+- **References:**
+    - [rippled API Reference](rippled-api.html)
+        - [Error Formatting](error-formatting.html)
+
+<!--{# common link defs #}-->
+{% include '_snippets/rippled-api-links.md' %}
+{% include '_snippets/tx-type-links.md' %}
+{% include '_snippets/rippled_versions.md' %}

content/references/rippled-api/api-conventions/response-formatting.md

@@ -11,6 +11,7 @@ The fields of a successful response include:
 | `result.status` | String   | (JSON-RPC and Commandline) The value `success` indicates the request was successfully received and understood by the server. |
 | `type`          | String   | (WebSocket only) The value `response` indicates a successful response to a command. [Asynchronous notifications](subscribe.html) use a different value such as `ledgerClosed` or `transaction`. |
 | `result`        | Object   | The result of the query; contents vary depending on the command. |
+| `warning`       | String   | _(May be omitted)_ If this field is provided, the value is the string `load`. This means the client is approaching the [rate limiting](rate-limiting.html) threshold where the server will disconnect this client. |
 
 
 ## Example Successful Response

content/tutorials/manage-the-rippled-server/troubleshooting/understanding-log-messages.md

@@ -55,9 +55,23 @@ Losing connections from time to time is normal for any peer-to-peer network. **O
 A large number of these messages around the same time may indicate a problem, such as:
 
 - Your internet connection to one or more specific peers was cut off.
-- Your server may have been overloading the peer with requests, causing it to drop your server.
+- Your server may have been overloading the peer with requests, causing the peer to disconnect your server.
 
 
+## Consumer entry dropped with balance at or above drop threshold
+
+The following log message indicates that a client to the server's public API has been dropped as a result of [rate limiting](rate-limiting.html):
+
+```text
+2020-Feb-24 23:05:35.566312806 Resource:WRN Consumer entry 169.55.164.21 dropped with balance 15970 at or above drop threshold 15000
+```
+
+The entry contains the IP address of the client that exceeded its rate limit, and the client's "balance", which is a score estimating the rate at which the client has been using the API. The threshold for dropping a client is [hardcoded to a score of 15000](https://github.com/ripple/rippled/blob/06c371544acc3b488b9d9c057cee4e51f6bef7a2/src/ripple/resource/impl/Tuning.h#L34-L35).
+
+If you see frequent messages from the same IP address, you may want to block those IP addresses from your network to reduce the load on your server's public API. (For example, you may be able to configure your firewall to block those IP addresses.)
+
+To avoid being dropped by rate limiting on your own server, [connect as an admin](get-started-with-the-rippled-api.html#admin-access).
+
 ## InboundLedger 11 timeouts for ledger
 
 ```text

dactyl-config.yml

@@ -285,7 +285,8 @@ targets:
             # Fix links from untranslated disable-master-key-pair.html.
             "transaction-basics.html#authorizing-transactions": "transaction-basics.html#トランザクションの承認"
             "transaction-common-fields.html#auto-fillable-fields": "transaction-common-fields.html#自動入力可能なフィールド"
-
+            # Fix link from untranslated rate-limiting.html
+            "get-started-with-the-rippled-api.html#admin-access": "get-started-with-the-rippled-api.html" # The translated doc hasn't been updated to add the "Admin Access" section, but the details are mentioned near the top of the page
 
     -   name: xrp-api-only
         lang: en
@@ -2995,6 +2996,17 @@ pages:
         targets:
             - ja
 
+    -   md: references/rippled-api/api-conventions/rate-limiting.md
+        html: rate-limiting.html
+        funnel: Docs
+        doc_type: References
+        supercategory: rippled API
+        category: API Conventions
+        blurb: Information on how public APIs limit clients from making too many requests.
+        targets:
+            - en
+            - ja
+
     -   md: references/rippled-api/api-conventions/request-formatting.md
         html: request-formatting.html
         funnel: Docs