Incomplete draft of simple consensus and detailed consensus challenges

content/concepts/consensus-network/consensus-protections.md

@@ -0,0 +1,67 @@
+# Consensus Protections Against Attacks and Failure Modes
+
+The XRP Ledger Consensus Protocol is a _byzantine fault tolerant_ consensus mechanism, which means it's designed to operate even if all kinds of things can go wrong: participants depend on an unreliable open network to communicate, and malicious actors may be attempting to control or interrupt the system at any given time. On top of that, the set of participants in the XRP Ledger Consensus Protocol isn't known in advance and can change over time.
+
+Confirming transactions in a timely manner while maintaining [the desired properties of the network](intro-to-consensus.html#consensus-protocol-properties) is a complex challenge, and it's impossible to build a perfect system. The XRP Ledger Consensus Protocol is designed to work as well as it can in most situations, and to fail as gracefully as possible in the situations where it can't.
+
+This page describes some of the types of challenges that the XRP Ledger Consensus Protocol faces and how it handles them.
+
+## Individual Validators Misbehaving
+
+_Validators_ are servers who actively contribute to the process of deciding each new ledger version. Validators only have an influence over servers configured to trust them (including indirectly). Consensus can continue even if some validators are misbehaving, including a large variety of failure cases, such as:
+
+- Being unavailable or overloaded.
+- Being partially disconnected from the network, so their messages reach only a subset of participants without delay.
+- Intentionally behaving with intent to defraud others or halt the network.
+- Behaving maliciously as a result of pressure from outside factors, such as threats from an oppressive government.
+- Accidentally sending confusing or malformed messages due to a bug or outdated software.
+
+In general, consensus can continue without problems as long as only a small percentage (less than about 20%) of trusted validators are misbehaving at a given time. (For the exact percentages and the math behind them, see the latest [Consensus Research](consensus-research.html).)
+
+If more than about 20% of validators are unreachable or not behaving properly, the network fails to reach a consensus. During this time, new transactions can be tentatively processed, but new ledger versions cannot be validated, so those transactions' final outcomes are not certain. In this situation, it would become immediately obvious that the XRP Ledger is unhealthy, prompting intervention from human participants who can decide whether to wait, or reconfigure their set of trusted validators.
+
+The only way to confirm invalid transactions—such as those spending money that has already been spent, or otherwise breaking the rules of the network—would be if more than about 80% of trusted validators colluded to confirm those transactions. In other words, a large majority of trusted validators would have to misbehave in the exact same way. With dozens of trusted validators run by different people and businesses in different parts of the world, this would be very difficult to achieve intentionally.
+
+
+## Software Vulnerabilities
+
+As with any software system, bugs (or intentionally malicious code) in the implementation of the XRP Ledger Consensus Protocol, commonly deployed software packages, or their dependencies, are a problem to be taken seriously. Even bugs that simply cause a server to crash when it sees carefully-crafted inputs can be abused to disrupt the progress of the network. Ripple has a number of precautions in place to address this threat, including:
+
+- An [open-source code base](https://github.com/ripple/rippled/), so any member of the public can review, compile, and independently test the relevant software.
+- A thorough and robust code review process for all changes to the official XRP Ledger repositories.
+- Digital signatures from Ripple employees on all releases and official software packages.
+- Regularly-commissioned professional reviews for security vulnerabilities and insecurities.
+- A [bug bounty program](https://ripple.com/bug-bounty/) that rewards security researchers who responsibly disclose vulnerabilities.
+
+
+## Sybil Attacks
+
+A _[sybil attack](https://en.wikipedia.org/wiki/Sybil_attack)_ is an attempt to take control of a network using a large number of fake identities. In the XRP Ledger, a sybil attack would take the form of running a large number of validators, then convincing others to trust those validators. This sort of attack is theoretically possible, but would be very difficult to accomplish because human intervention is necessary for validators to become trusted.
+
+No matter how many validating servers a would-be attacker runs, those servers have no say on what the existing participants consider validated unless those participants choose to trust the attacker's validators. Other servers only listen to the validators they are configured to trust, either through a validator list or explicit configuration. (See [validator overlap requirements](#validator-overlap-requirements) for a summary of how the default validator list works.)
+
+This trust does not happen automatically, so performing a successful sybil attack would involve the difficult work of convincing targeted humans and businesses to reconfigure their XRP Ledger servers to trust the attacker's validators. Even in the case that one individual entity was fooled into doing so, this would have a minimal impact on others who do not change their configurations.
+
+
+## 51% Attack
+
+A "51% attack" is an attack on a blockchain system where one party controls more than 50% of all mining or voting power. (Technically, the attack is slightly misnamed because _any_ amount over 50% is enough.) The XRP Ledger is not vulnerable to a 51% attack because it does not use mining in its consensus mechanism. The next closest analogue for the XRP Ledger is a [sybil attack](#sybil-attacks), which would also be difficult to accomplish.
+
+
+## Validator Overlap Requirements
+
+For all participants in the XRP Ledger to agree on what they consider validated, they must start by choosing a set of trusted validators that are fairly similar to the sets chosen by everyone else. In the worst case, less than about 90% overlap could cause some participants to diverge from each other. For that reason, Ripple publishes a signed list of recommended validators, including trustworthy and well-maintained servers run by the company, industry, and community.
+
+By default, servers in the network are configured with the URL of Ripple's validator list site, and a cryptographic key that can be used to verify the integrity of lists published to that site. In case the site goes down, the signed updates to the list can also be relayed directly among servers in the XRP Ledger's peer-to-peer network.
+
+Participants can configure their own list sites or explicitly choose validators to trust on an individual basis, but Ripple does not recommended because doing so could cause you to diverge from the rest of the network and lose money as a result.
+
+Research is ongoing to design an improved consensus protocol that allow more heterogeneous validator lists. For more information, see the [Consensus Research](consensus-research.html) page.
+
+
+## See Also
+
+- For an **intro-level overview** of consensus, see [Intro to Consensus](intro-to-consensus.html).
+- For a **detailed description** of the consensus protocol, see [Consensus](consensus.html).
+- For an explanation of the **design decisions and background** behind the consensus protocol, see [Consensus Principles and Rules](consensus-principles-and-rules.html).
+- For **academic research** exploring the properties and limitations of the protocol, see [Consensus Research](consensus-research.html).

content/concepts/introduction/intro-to-consensus.md

@@ -0,0 +1,54 @@
+# Introduction to Consensus
+
+_Consensus_ is the most important property of any decentralized payment system. In traditional centralized payment systems, one authoritative administrator gets the final say in how and when payments occur. Decentralized systems, by definition, don't have an operator to do that. Instead, decentralized systems like the XRP Ledger define a set of rules all participants follow, so every participant can agree on the exact same series of events and their outcome at any point in time. We call this set of rules a _consensus protocol_.
+
+## Consensus Protocol Properties
+
+The XRP Ledger uses a consensus protocol unlike any digital asset that came before it. This protocol, known as the XRP Ledger Consensus Protocol, is designed to have the following important properties:
+
+- Everyone who uses the XRP Ledger can agree on the latest state, and which transactions have occurred in which order.
+- All valid transactions are processed without needing a central operator or having a single point of failure.
+- The ledger can make progress even if some participants join, leave, or behave inappropriately.
+- If too many participants are unreachable or misbehaving, the network fails to make progress rather than diverging or confirming invalid transactions.
+- Confirming transactions does not require wasteful or competitive use of resources, unlike most blockchain systems.
+
+This protocol is still evolving, as is our knowledge of its limits and possible failure cases. For academic research on the protocol itself, see [Consensus Research](consensus-research.html).
+
+## Background
+
+Consensus protocols are a solution to the _double-spend problem_: the challenge to prevent someone from successfully spending the same digital money in two places. For a detailed analysis of the double-spend problem, how the XRP Ledger Consensus Protocol solves this problem, and the tradeoffs and limitations involved, see [Consensus Principles and Rules](consensus-principles-and-rules.html).
+
+
+## Ledger History
+
+The XRP Ledger processes transactions in blocks called "ledger versions", or "ledgers" for short. Each ledger version is numbered with a _ledger index_ and builds on a previous ledger version whose index is one less. This goes all the way back to a starting point called the _genesis ledger_ with ledger index 1.
+
+<!-- TODO: diagram(s) of ledger, ledger history -->
+
+Each ledger version contains three pieces:
+
+- The current state of all balances and objects stored in the ledger.
+- The set of transactions applied to the previous ledger to result in this one.
+- Metadata about the current ledger version, such as its ledger index, a [cryptographic hash](https://en.wikipedia.org/wiki/Cryptographic_hash_function) that uniquely identifies its contents, and information on the parent ledger was used as a basis for building this one.
+
+Like Bitcoin and other blockchain technologies, this forms a public history of all transactions and their results. Unlike most blockchain technologies, each new "block" in the XRP Ledger contains the entirety of the current state, so you don't need to collect the entire history to know what's happening now.[¹](#footnote-1)
+
+The main job of the XRP Ledger Consensus Protocol is to agree on set of transactions to apply to the previous ledger, apply them in a well-defined order, then confirm that everyone got the same results. When this happens successfully, a ledger version is considered _validated_, and final. From there, the process continues by building the next ledger version.
+
+
+## Trust-Based Validation
+
+The core principle behind the XRP Ledger's consensus mechanism is that a little trust goes a long way. Each participant in the network chooses a set of _validators_, servers [specifically configured to participate actively in consensus](run-a-rippled-validator.html), run by different parties who are expected to behave honestly most of the time. More importantly, the set of chosen validators should not be likely to collude with one another to break the rules in the exact same way.
+
+As the network progresses, each server listens to its trusted validators[²](#footnote-2); as long as a large percentage of them agree that a set of transactions should occur and that a given ledger is the result, the server declares a consensus. If they don't agree, validators modify their suggestions to more closely match the other validators they trust, repeating the process in several rounds until they reach a consensus.
+
+It's OK if a small proportion of validators don't operate properly all the time. As long as fewer than 20% of trusted validators are faulty, consensus can continue unimpeded; and confirming an invalid transaction would require over 80% of trusted validators to collude. (Anywhere in between, and the network simply stops making progress.)
+
+For a longer exploration of how the XRP Ledger Consensus Protocol responds to various challenges, attacks, and failure cases, see [Consensus Protections Against Attacks and Failure Modes](consensus-protections.html).
+
+
+## Footnotes
+
+1. <a id="footnote-1"></a> In Bitcoin, the current state is sometimes called the set of "UTXOs" (unspent transaction outputs). Despite proposals that Bitcoin's consensus mechanism could vouch for the latest UTXOs to save new servers the effort of collecting state going back to the beginning of the network, it currently (as of 2018) does not do this.
+
+2. <a id="footnote-2"></a> A server does not need a direct connection to its trusted validators to hear from them. The XRP Ledger peer-to-peer network uses a _gossip protocol_ where servers identify each other by public keys and relay digitally-signed messages from others.