Merge pull request #445 from mDuo13/simple-consensus

Introductory consensus article

content/concepts/consensus-network/consensus-protections.md

@@ -0,0 +1,67 @@
+# Consensus Protections Against Attacks and Failure Modes
+
+The XRP Ledger Consensus Protocol is a _byzantine fault tolerant_ consensus mechanism, which means it's designed to operate even if all kinds of things can go wrong: participants depend on an unreliable open network to communicate, and malicious actors may be attempting to control or interrupt the system at any given time. On top of that, the set of participants in the XRP Ledger Consensus Protocol isn't known in advance and can change over time.
+
+Confirming transactions in a timely manner while maintaining [the desired properties of the network](intro-to-consensus.html#consensus-protocol-properties) is a complex challenge, and it's impossible to build a perfect system. The XRP Ledger Consensus Protocol is designed to work as well as it can in most situations, and to fail as gracefully as possible in the situations where it can't.
+
+This page describes some of the types of challenges that the XRP Ledger Consensus Protocol faces and how it handles them.
+
+## Individual Validators Misbehaving
+
+_Validators_ are servers that actively contribute to the process of deciding each new ledger version. Validators only have an influence over servers configured to trust them (including indirectly). Consensus can continue even if some validators are misbehaving, including a large variety of failure cases, such as:
+
+- Being unavailable or overloaded.
+- Being partially disconnected from the network, so their messages reach only a subset of participants without delay.
+- Intentionally behaving with intent to defraud others or halt the network.
+- Behaving maliciously as a result of pressure from outside factors, such as threats from an oppressive government.
+- Accidentally sending confusing or malformed messages due to a bug or outdated software.
+
+In general, consensus can continue without problems as long as only a small percentage (less than about 20%) of trusted validators are misbehaving at a given time. (For the exact percentages and the math behind them, see the latest [Consensus Research](consensus-research.html).)
+
+If more than about 20% of validators are unreachable or not behaving properly, the network fails to reach a consensus. During this time, new transactions can be tentatively processed, but new ledger versions cannot be validated, so those transactions' final outcomes are not certain. In this situation, it would become immediately obvious that the XRP Ledger is unhealthy, prompting intervention from human participants who can decide whether to wait, or reconfigure their set of trusted validators.
+
+The only way to confirm an invalid transaction would be to get at least 80% of trusted validators to approve of the transaction and agree on its exact outcome. (Invalid transactions include those spending money that has already been spent, or otherwise breaking the rules of the network.) In other words, a large majority of trusted validators would have to _collude_. With dozens of trusted validators run by different people and businesses in different parts of the world, this would be very difficult to achieve intentionally.
+
+
+## Software Vulnerabilities
+
+As with any software system, bugs (or intentionally malicious code) in the implementation of the XRP Ledger Consensus Protocol, commonly deployed software packages, or their dependencies, are a problem to be taken seriously. Even bugs that simply cause a server to crash when it sees carefully crafted inputs can be abused to disrupt the progress of the network. Ripple has a number of precautions in place to address this threat, including:
+
+- An [open-source code base](https://github.com/ripple/rippled/), so any member of the public can review, compile, and independently test the relevant software.
+- A thorough and robust code review process for all changes to the official XRP Ledger repositories.
+- Digital signatures from Ripple employees on all releases and official software packages.
+- Regularly-commissioned professional reviews for security vulnerabilities and insecurities.
+- A [bug bounty program](https://ripple.com/bug-bounty/) that rewards security researchers who responsibly disclose vulnerabilities.
+
+
+## Sybil Attacks
+
+A _[Sybil attack](https://en.wikipedia.org/wiki/Sybil_attack)_ is an attempt to take control of a network using a large number of fake identities. In the XRP Ledger, a Sybil attack would take the form of running a large number of validators, then convincing others to trust those validators. This sort of attack is theoretically possible, but would be very difficult to accomplish because human intervention is necessary for validators to become trusted.
+
+No matter how many validating servers a would-be attacker runs, those servers have no say on what the existing participants consider validated unless those participants choose to trust the attacker's validators. Other servers only listen to the validators they are configured to trust, either through a validator list or explicit configuration. (See [validator overlap requirements](#validator-overlap-requirements) for a summary of how the default validator list works.)
+
+This trust does not happen automatically, so performing a successful Sybil attack would involve the difficult work of convincing targeted humans and businesses to reconfigure their XRP Ledger servers to trust the attacker's validators. Even in the case that one individual entity was fooled into doing so, this would have a minimal impact on others who do not change their configurations.
+
+
+## 51% Attack
+
+A "51% attack" is an attack on a blockchain system where one party controls more than 50% of all mining or voting power. (Technically, the attack is slightly misnamed because _any_ amount over 50% is enough.) The XRP Ledger is not vulnerable to a 51% attack because it does not use mining in its consensus mechanism. The next closest analogue for the XRP Ledger is a [Sybil attack](#sybil-attacks), which would also be difficult to accomplish.
+
+
+## Validator Overlap Requirements
+
+For all participants in the XRP Ledger to agree on what they consider validated, they must start by choosing a set of trusted validators that are fairly similar to the sets chosen by everyone else. In the worst case, less than about 90% overlap could cause some participants to diverge from each other. For that reason, Ripple publishes a signed list of recommended validators, including trustworthy and well-maintained servers run by the company, industry, and community.
+
+By default, XRP Ledger servers are configured to use a validator list site run by Ripple. The site provides a list of recommended validators (also known as a recommended _Unique Node List_, or UNL), which Ripple updates periodically. Servers configured this way trust all validators in the latest version of the list, which ensures 100% overlap with others also using the same list. The default configuration includes a public key that verifies the authenticity of the site's contents. In case the site goes down, servers in the XRP Ledger's peer-to-peer network can directly relay the signed updates to the list among themselves.
+
+Technically, if you run a server, you can configure your own list site or explicitly choose validators to trust on an individual basis, but Ripple does not recommended doing so. If your chosen set of validators does not have enough overlap with others, your server may diverge from the rest of the network, and you could lose money by taking action based on your server's divergent state.
+
+Research is ongoing to design an improved consensus protocol that allows more heterogeneous validator lists. For more information, see the [Consensus Research](consensus-research.html) page.
+
+
+## See Also
+
+- For an **intro-level overview** of consensus, see [Intro to Consensus](intro-to-consensus.html).
+- For a **detailed description** of the consensus protocol, see [Consensus](consensus.html).
+- For an explanation of the **design decisions and background** behind the consensus protocol, see [Consensus Principles and Rules](consensus-principles-and-rules.html).
+- For **academic research** exploring the properties and limitations of the protocol, see [Consensus Research](consensus-research.html).

content/concepts/introduction/intro-to-consensus.md

@@ -0,0 +1,68 @@
+# Introduction to Consensus
+
+_Consensus_ is the most important property of any decentralized payment system. In traditional centralized payment systems, one authoritative administrator gets the final say in how and when payments occur. Decentralized systems, by definition, don't have an administrator to do that. Instead, decentralized systems like the XRP Ledger define a set of rules all participants follow, so every participant can agree on the exact same series of events and their outcome at any point in time. We call this set of rules a _consensus protocol_.
+
+
+## Consensus Protocol Properties
+
+The XRP Ledger uses a consensus protocol unlike any digital asset that came before it. This protocol, known as the XRP Ledger Consensus Protocol, is designed to have the following important properties:
+
+- Everyone who uses the XRP Ledger can agree on the latest state, and which transactions have occurred in which order.
+- All valid transactions are processed without needing a central operator or having a single point of failure.
+- The ledger can make progress even if some participants join, leave, or behave inappropriately.
+- If too many participants are unreachable or misbehaving, the network fails to make progress rather than diverging or confirming invalid transactions.
+- Confirming transactions does not require wasteful or competitive use of resources, unlike most other blockchain systems.
+
+These properties are sometimes summarized as the following principles, in order of priority: **Correctness, Agreement, Forward Progress**.
+
+This protocol is still evolving, as is our knowledge of its limits and possible failure cases. For academic research on the protocol itself, see [Consensus Research](consensus-research.html).
+
+## Background
+
+Consensus protocols are a solution to the _double-spend problem_: the challenge of preventing someone from successfully spending the same digital money twice. The hardest part about this problem is putting transactions in order: without a central authority, it can be difficult to resolve disputes about which transaction comes first when you have two or more mutually-exclusive transactions sent around the same time. For a detailed analysis of the double-spend problem, how the XRP Ledger Consensus Protocol solves this problem, and the tradeoffs and limitations involved, see [Consensus Principles and Rules](consensus-principles-and-rules.html).
+
+
+## Ledger History
+
+The XRP Ledger processes transactions in blocks called "ledger versions", or "ledgers" for short. Each ledger version contains three pieces:
+
+- The current state of all balances and objects stored in the ledger.
+- The set of transactions that have been applied to the previous ledger to result in this one.
+- Metadata about the current ledger version, such as its ledger index, a [cryptographic hash](https://en.wikipedia.org/wiki/Cryptographic_hash_function) that uniquely identifies its contents, and information about the parent ledger that was used as a basis for building this one.
+
+[![Figure 1: Anatomy of a ledger version, which includes transactions, state, and metadata](img/anatomy-of-a-ledger-simplified.png)](img/anatomy-of-a-ledger-simplified.png)
+<!--{# Diagram source: https://docs.google.com/presentation/d/1mg2jZQwgfLCIhOU8Mr5aOiYpIgbIgk3ymBoDb2hh7_s/ #}-->
+
+Each ledger version is numbered with a _ledger index_ and builds on a previous ledger version whose index is one less, going all the way back to a starting point called the _genesis ledger_ with ledger index 1. Like Bitcoin and other blockchain technologies, this forms a public history of all transactions and their results. Unlike many blockchain technologies, each new "block" in the XRP Ledger contains the entirety of the current state, so you don't need to collect the entire history to know what's happening now.[¹](#footnote-1)
+
+The main job of the XRP Ledger Consensus Protocol is to agree on a set of transactions to apply to the previous ledger, apply them in a well-defined order, then confirm that everyone got the same results. When this happens successfully, a ledger version is considered _validated_, and final. From there, the process continues by building the next ledger version.
+
+
+## Trust-Based Validation
+
+The core principle behind the XRP Ledger's consensus mechanism is that a little trust goes a long way. Each participant in the network chooses a set of _validators_, servers [specifically configured to participate actively in consensus](run-a-rippled-validator.html), run by different parties who are expected to behave honestly most of the time. More importantly, the set of chosen validators should not be likely to collude with one another to break the rules in the exact same way. This list is sometimes called a _Unique Node List_, or UNL.
+
+As the network progresses, each server listens to its trusted validators[²](#footnote-2); as long as a large enough percentage of them agree that a set of transactions should occur and that a given ledger is the result, the server declares a consensus. If they don't agree, validators modify their proposals to more closely match the other validators they trust, repeating the process in several rounds until they reach a consensus.
+
+[![Figure 2: Consensus rounds. Validators revise their proposals to match other validators they trust](img/consensus-rounds.png)](img/consensus-rounds.png)
+
+It's OK if a small proportion of validators don't operate properly all the time. As long as fewer than 20% of trusted validators are faulty, consensus can continue unimpeded; and confirming an invalid transaction would require over 80% of trusted validators to collude. If more than 20% but less than 80% of trusted validators are faulty, the network simply stops making progress.
+
+For a longer exploration of how the XRP Ledger Consensus Protocol responds to various challenges, attacks, and failure cases, see [Consensus Protections Against Attacks and Failure Modes](consensus-protections.html).
+
+
+## See Also
+
+- [Consensus Network Concepts](consensus-network.html) for several articles describing the mechanics of the XRP Ledger Consensus Protocol in greater depth.
+- [Run `rippled` as a Validator](run-rippled-as-a-validator.html) for instructions on running your own validator.
+- [Decentralization Strategy Update (Ripple Dev Blog)](https://ripple.com/dev-blog/decentralization-strategy-update/) for a description of Ripple's goals and plans for decentralizing the XRP Ledger.
+
+<!--{# TODO: Replace the Decent Strategy Update w/ a newer article when we have one #}-->
+
+----
+
+## Footnotes
+
+1. <a id="footnote-1"></a> In Bitcoin, the current state is sometimes called the set of "UTXOs" (unspent transaction outputs). Unlike the XRP Ledger, a Bitcoin server must download the entire transaction history to know the full set of UTXOs and process new transactions. Currently (as of 2018), there have been some proposals to modify Bitcoin's consensus mechanism to periodically summarize the latest UTXOs so new servers would not need to do this. Etherium, which also uses a UTXO model, has already implemented a similar approach with a summary of the current state, called a _state root_, in each block.
+
+2. <a id="footnote-2"></a> A server does not need a direct connection to its trusted validators to hear from them. The XRP Ledger peer-to-peer network uses a _gossip protocol_ where servers identify each other by public keys and relay digitally-signed messages from others.

dactyl-config.yml

@@ -105,12 +105,12 @@ pages:
             - local
 
         #TODO: DOC-1503: replace this with a more concise intro to consensus
-    -   md: concepts/introduction/consensus.md
-        html: consensus.html
+    -   md: concepts/introduction/intro-to-consensus.md
+        html: intro-to-consensus.html
         funnel: Docs
         doc_type: Concepts
         category: Introduction
-        blurb: Understand the role of consensus in the XRP Ledger.
+        blurb: Develop a basic understanding of the XRP Ledger's consensus mechanism.
         targets:
             - local
 
@@ -435,14 +435,12 @@ pages:
         targets:
             - local
 
-    # TODO: validators & UNLs
-
-    -   md: concepts/consensus-network/amendments.md
-        html: amendments.html
+    -   md: concepts/consensus-network/consensus.md
+        html: consensus.html
         funnel: Docs
         doc_type: Concepts
         category: Consensus Network
-        blurb: Amendments represent new features or other changes to transaction processing. Validators coordinate through consensus to apply these upgrades to the XRP Ledger in an orderly fashion.
+        blurb: Understand the role of consensus in the XRP Ledger.
         targets:
             - local
 
@@ -455,6 +453,24 @@ pages:
         targets:
           - local
 
+    -   md: concepts/consensus-network/consensus-protections.md
+        html: consensus-protections.html
+        funnel: Docs
+        doc_type: Concepts
+        category: Consensus Network
+        blurb: Learn how the XRP Ledger Consensus Protocol is protected against various problems and attacks that may occur in a decentralized financial system.
+        targets:
+            - local
+
+    -   md: concepts/consensus-network/amendments.md
+        html: amendments.html
+        funnel: Docs
+        doc_type: Concepts
+        category: Consensus Network
+        blurb: Amendments represent new features or other changes to transaction processing. Validators coordinate through consensus to apply these upgrades to the XRP Ledger in an orderly fashion.
+        targets:
+            - local
+
     -   md: concepts/consensus-network/fee-voting.md
         html: fee-voting.html
         funnel: Docs