From 03ac96b69a992c8c1f29760319b1255be78e35b0 Mon Sep 17 00:00:00 2001 From: mDuo13 Date: Thu, 6 Dec 2018 18:09:42 -0800 Subject: [PATCH 1/5] Hashes - clarify hash prefixes, SHA-512Half size. --- content/_snippets/data_types/hash.md | 2 +- content/_snippets/rippled-api-links.md | 2 +- .../api-conventions/basic-data-types.md | 18 ++++++++++++++++++ .../ledger-data-formats/ledger-header.md | 2 +- .../ledger-data-formats/ledger-object-ids.md | 5 +++-- 5 files changed, 24 insertions(+), 5 deletions(-) diff --git a/content/_snippets/data_types/hash.md b/content/_snippets/data_types/hash.md index 9cdf6ddb01..adea0c8abc 100644 --- a/content/_snippets/data_types/hash.md +++ b/content/_snippets/data_types/hash.md @@ -1,4 +1,4 @@ -Many objects in the XRP Ledger, particularly transactions and ledgers, are uniquely identified by a 256-bit hash value. This value is typically calculated as a "SHA-512Half", which calculates a [SHA-512](http://dx.doi.org/10.6028/NIST.FIPS.180-4) hash from some contents, then takes the first 64 characters of the hexadecimal representation. Since the hash of an object is derived from the contents in a way that is extremely unlikely to produce collisions, two objects with the same hash can be considered the same. +Many objects in the XRP Ledger, particularly transactions and ledgers, are uniquely identified by a 256-bit hash value. This value is typically calculated as a "SHA-512Half", which calculates a [SHA-512](http://dx.doi.org/10.6028/NIST.FIPS.180-4) hash from some contents, then takes the first half of the output. (That's 256 bits, which is 32 bytes, or 64 characters of the hexadecimal representation.) Since the hash of an object is derived from the contents in a way that is extremely unlikely to produce collisions, two objects with the same hash can be considered the same. An XRP Ledger hash value has the following characteristics: diff --git a/content/_snippets/rippled-api-links.md b/content/_snippets/rippled-api-links.md index e6a2f8f15f..c49153e348 100644 --- a/content/_snippets/rippled-api-links.md +++ b/content/_snippets/rippled-api-links.md @@ -11,7 +11,7 @@ [result code]: transaction-results.html [seconds since the Ripple Epoch]: basic-data-types.html#specifying-time [Sequence Number]: basic-data-types.html#account-sequence -[SHA-512Half]: ledger-object-ids.html#sha512half +[SHA-512Half]: basic-data-types.html#hashes [Specifying Currency Amounts]: basic-data-types.html#specifying-currency-amounts [Specifying Ledgers]: basic-data-types.html#specifying-ledgers [Specifying Time]: basic-data-types.html#specifying-time diff --git a/content/references/rippled-api/api-conventions/basic-data-types.md b/content/references/rippled-api/api-conventions/basic-data-types.md index 244797190a..4e582b0a02 100644 --- a/content/references/rippled-api/api-conventions/basic-data-types.md +++ b/content/references/rippled-api/api-conventions/basic-data-types.md @@ -21,6 +21,24 @@ Each closed [Ledger](ledger-data-formats.html) has a [Ledger Index][] and a [Has {% include '_snippets/data_types/hash.md' %} +### Hash Prefixes +[[Source]
](https://github.com/ripple/rippled/blob/master/src/ripple/protocol/HashPrefix.h "Source") + +In many cases, the XRP Ledger prefixes an object's binary data with a 4-byte code before calculating its hash, so that objects of different types have different hashes even if their binary formats are the same. The 4-byte codes are structured as three bytes of ASCII followed by a zero byte. + +The following table shows the hash prefixes for some common types of data: + +| Object Type | Hash Prefix (Hex) | Hash Prefix (Text) | +|:----------------------------------------------------|:------------------|:---| +| Transaction (Signed) | `0x54584E00` | `TXN\0` | +| Ledger Version (`ledger_hash`) | `0x4C575200` | `LWR\0` | +| [Ledger state tree information](ledger-header.html) | `0x4D4C4E00` | `MLN\0` | +| Transaction (Unsigned, for single-signing) | `0x53545800` | `STX\0` | +| Transaction (Unsigned, for multi-signing) | `0x534D5400` | `SMT\0` | +| Validation vote | `0x56414C00` | `VAL\0` | + +[Ledger objects IDs](ledger-object-ids.html) are calculated in a similar way, with 16-bit "space key" prefixes for each object type. + ## Account Sequence [Sequence Number]: #account-sequence diff --git a/content/references/rippled-api/ledger-data-formats/ledger-header.md b/content/references/rippled-api/ledger-data-formats/ledger-header.md index cc4715f87a..64b6d3ef7c 100644 --- a/content/references/rippled-api/ledger-data-formats/ledger-header.md +++ b/content/references/rippled-api/ledger-data-formats/ledger-header.md @@ -6,7 +6,7 @@ Every ledger version has a unique header that describes the contents. You can lo | Field | JSON Type | [Internal Type][] | Description | |:-------------------------------|:----------|:------------------|:------------| | [`ledger_index`][Ledger Index] | String | UInt32 | The sequence number of the ledger. Some API methods display this as a quoted integer; some display it as a native JSON number. | -| `ledger_hash` | String | Hash256 | The [SHA-512Half][] of the ledger header, excluding the `ledger_hash` itself. This serves as a unique identifier for this ledger and all its contents. | +| `ledger_hash` | String | Hash256 | The [SHA-512Half][] of this ledger version. This serves as a unique identifier for this ledger and all its contents. | | `account_hash` | String | Hash256 | The [SHA-512Half][] of this ledger's state tree information. | | `close_time` | Number | UInt32 | The approximate time this ledger closed, as the number of seconds since the Ripple Epoch of 2000-01-01 00:00:00. This value is rounded based on the `close_time_resolution`, so later ledgers can have the same value. | | `closed` | Boolean | bool | If true, this ledger version is no longer accepting new transactions. (However, unless this ledger version is validated, it might be replaced by a different ledger version with a different set of transactions.) | diff --git a/content/references/rippled-api/ledger-data-formats/ledger-object-ids.md b/content/references/rippled-api/ledger-data-formats/ledger-object-ids.md index 0053a6c9b5..388fa49a14 100644 --- a/content/references/rippled-api/ledger-data-formats/ledger-object-ids.md +++ b/content/references/rippled-api/ledger-data-formats/ledger-object-ids.md @@ -1,14 +1,15 @@ # Ledger Object IDs -All objects in a ledger's state tree have a unique ID. This field is returned as the `index` field in JSON, at the same level as the object's contents. The ID is derived by hashing important contents of the object, along with a [namespace identifier](https://github.com/ripple/rippled/blob/master/src/ripple/protocol/LedgerFormats.h#L99). The [ledger object type](ledger-object-types.html) determines which namespace identifier to use and which contents to include in the hash. This ensures every ID is unique. To calculate the hash, `rippled` uses SHA-512 and then truncates the result to the first 256 bytes. This algorithm, informally called **SHA-512Half**, provides an output that has comparable security to SHA-256, but runs faster on 64-bit processors. +All objects in a ledger's state tree have a unique ID. This field is returned as the `index` field in JSON, at the same level as the object's contents. The ID is derived by hashing important contents of the object, along with a [namespace identifier](https://github.com/ripple/rippled/blob/master/src/ripple/protocol/LedgerFormats.h#L99). The [ledger object type](ledger-object-types.html) determines which namespace identifier to use and which contents to include in the hash. This ensures every ID is unique. To calculate the hash, `rippled` uses SHA-512 and then truncates the result to the first 256 bits. This algorithm, informally called **SHA-512Half**, provides an output that has comparable security to SHA-256, but runs faster on 64-bit processors. ![Diagram: rippled uses SHA-512Half to generate IDs for ledger objects. The space key prevents IDs for different object types from colliding.](img/ledger-indexes.png) ## See Also -For ledger basics, see [Ledgers](ledgers.html). +- For more information how the XRP Ledger creates and uses hashes, see [Hashes](basic-data-types.html#hashes). +- For ledger basics, see [Ledgers](ledgers.html). From 47dd83e3c5a5e56bf600a488a8d564e063648286 Mon Sep 17 00:00:00 2001 From: mDuo13 Date: Mon, 10 Dec 2018 15:32:50 -0800 Subject: [PATCH 2/5] Revise hash prefixes table --- .../api-conventions/basic-data-types.md | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/content/references/rippled-api/api-conventions/basic-data-types.md b/content/references/rippled-api/api-conventions/basic-data-types.md index 4e582b0a02..493b3e8905 100644 --- a/content/references/rippled-api/api-conventions/basic-data-types.md +++ b/content/references/rippled-api/api-conventions/basic-data-types.md @@ -26,18 +26,17 @@ Each closed [Ledger](ledger-data-formats.html) has a [Ledger Index][] and a [Has In many cases, the XRP Ledger prefixes an object's binary data with a 4-byte code before calculating its hash, so that objects of different types have different hashes even if their binary formats are the same. The 4-byte codes are structured as three bytes of ASCII followed by a zero byte. -The following table shows the hash prefixes for some common types of data: +The following table shows the prefixes for some hash types that appear in API requests and responses: -| Object Type | Hash Prefix (Hex) | Hash Prefix (Text) | -|:----------------------------------------------------|:------------------|:---| -| Transaction (Signed) | `0x54584E00` | `TXN\0` | -| Ledger Version (`ledger_hash`) | `0x4C575200` | `LWR\0` | -| [Ledger state tree information](ledger-header.html) | `0x4D4C4E00` | `MLN\0` | -| Transaction (Unsigned, for single-signing) | `0x53545800` | `STX\0` | -| Transaction (Unsigned, for multi-signing) | `0x534D5400` | `SMT\0` | -| Validation vote | `0x56414C00` | `VAL\0` | +| Object Type | Hash Prefix (Hex) | Hash Prefix (Text) | +|:----------------------------------------------------------------------------|:------------------|:--| +| Signed Transaction (`hash` field of transactions) | `0x54584E00` | `TXN\0` | +| Ledger Version (`ledger_hash` fields) | `0x4C575200` | `LWR\0` | +| [Ledger state tree information (`account_state` field)](ledger-header.html) | `0x4D4C4E00` | `MLN\0` | -[Ledger objects IDs](ledger-object-ids.html) are calculated in a similar way, with 16-bit "space key" prefixes for each object type. +**Note:** There are several other hash prefixes which are only used when calculating the hash as the first step of signing a specific type of data. For the complete list, see [the source code](https://github.com/ripple/rippled/blob/master/src/ripple/protocol/impl/HashPrefix.cpp). + +[Ledger objects IDs](ledger-object-ids.html) are calculated in a similar way, but they use a 2-byte prefix called a "space key" instead of a prefix in the form described here. ## Account Sequence From a19b3c2e4247aeefdd39075583e81b1634b7fb4e Mon Sep 17 00:00:00 2001 From: mDuo13 Date: Mon, 10 Dec 2018 17:08:10 -0800 Subject: [PATCH 3/5] Hash prefixes: re-add unsigned TX prefixes These prefixes are relevant for implementing offline signing so they're important to include. (The serialization docs want to know them.) --- .../references/rippled-api/api-conventions/basic-data-types.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/references/rippled-api/api-conventions/basic-data-types.md b/content/references/rippled-api/api-conventions/basic-data-types.md index 493b3e8905..87467bf2e0 100644 --- a/content/references/rippled-api/api-conventions/basic-data-types.md +++ b/content/references/rippled-api/api-conventions/basic-data-types.md @@ -33,6 +33,8 @@ The following table shows the prefixes for some hash types that appear in API re | Signed Transaction (`hash` field of transactions) | `0x54584E00` | `TXN\0` | | Ledger Version (`ledger_hash` fields) | `0x4C575200` | `LWR\0` | | [Ledger state tree information (`account_state` field)](ledger-header.html) | `0x4D4C4E00` | `MLN\0` | +| Unsigned Transaction (For single-signing) | `0x53545800` | `STX\0` | +| Unsigned Transaction (For multi-signing) | `0x534D5400` | `SMT\0` | **Note:** There are several other hash prefixes which are only used when calculating the hash as the first step of signing a specific type of data. For the complete list, see [the source code](https://github.com/ripple/rippled/blob/master/src/ripple/protocol/impl/HashPrefix.cpp). From 13e5f183bb7d6bc8b07a903453f4f59a7dfbdfb9 Mon Sep 17 00:00:00 2001 From: mDuo13 Date: Mon, 10 Dec 2018 17:36:19 -0800 Subject: [PATCH 4/5] Hash prefixes: actually just re-add all prefixes --- .../api-conventions/basic-data-types.md | 26 ++++++++++++------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/content/references/rippled-api/api-conventions/basic-data-types.md b/content/references/rippled-api/api-conventions/basic-data-types.md index 87467bf2e0..4aacdd8f09 100644 --- a/content/references/rippled-api/api-conventions/basic-data-types.md +++ b/content/references/rippled-api/api-conventions/basic-data-types.md @@ -26,17 +26,25 @@ Each closed [Ledger](ledger-data-formats.html) has a [Ledger Index][] and a [Has In many cases, the XRP Ledger prefixes an object's binary data with a 4-byte code before calculating its hash, so that objects of different types have different hashes even if their binary formats are the same. The 4-byte codes are structured as three bytes of ASCII followed by a zero byte. -The following table shows the prefixes for some hash types that appear in API requests and responses: +Some types of hash appear in API requests and responses. Others are only calculated as the first step of signing a certain type of data, or calculating a higher-level hash. The following table shows all 4-byte hash prefixes the XRP Ledger uses: -| Object Type | Hash Prefix (Hex) | Hash Prefix (Text) | -|:----------------------------------------------------------------------------|:------------------|:--| -| Signed Transaction (`hash` field of transactions) | `0x54584E00` | `TXN\0` | -| Ledger Version (`ledger_hash` fields) | `0x4C575200` | `LWR\0` | -| [Ledger state tree information (`account_state` field)](ledger-header.html) | `0x4D4C4E00` | `MLN\0` | -| Unsigned Transaction (For single-signing) | `0x53545800` | `STX\0` | -| Unsigned Transaction (For multi-signing) | `0x534D5400` | `SMT\0` | +| Object Type | API Fields | Hash Prefix (Hex) | Hash Prefix (Text) | +|:--------------------------------------|:-------------------------------------|:------------------|:--| +| Consensus proposal | N/A | `0x50525000` | `PRP\0` | +| Ledger Version | `ledger_hash` | `0x4C575200` | `LWR\0` | +| Ledger state data | `account_state` in [ledger header][] | `0x4D4C4E00` | `MLN\0` | +| Ledger data inner node | N/A | `0x4D494E00` | `MIN\0` | +| Ledger data inner node ([SHAMapv2][]) | N/A | `0x494E5200` | `INR\0` | +| Payment Channel Claim | N/A | `0x434C4D00` | `CLM\0` | +| Signed Transaction | `hash` of transactions | `0x54584E00` | `TXN\0` | +| Transaction with metadata | N/A | `0x534E4400` | `SND\0` | +| Unsigned Transaction (Single-signing) | N/A | `0x53545800` | `STX\0` | +| Unsigned Transaction (Multi-signing) | N/A | `0x534D5400` | `SMT\0` | +| Validation vote | N/A | `0x56414C00` | `VAL\0` | +| Validator ephemeral key update | N/A | `0x4D414E00` | `MAN\0` | -**Note:** There are several other hash prefixes which are only used when calculating the hash as the first step of signing a specific type of data. For the complete list, see [the source code](https://github.com/ripple/rippled/blob/master/src/ripple/protocol/impl/HashPrefix.cpp). +[ledger header]: ledger-header.html +[SHAMapv2]: known-amendments.html#shamapv2 [Ledger objects IDs](ledger-object-ids.html) are calculated in a similar way, but they use a 2-byte prefix called a "space key" instead of a prefix in the form described here. From a6b82ff59960fcf0ac955014ed8e6b3ca81caa70 Mon Sep 17 00:00:00 2001 From: mDuo13 Date: Mon, 17 Dec 2018 14:10:32 -0800 Subject: [PATCH 5/5] Hash prefix clarifications per review --- .../rippled-api/api-conventions/basic-data-types.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/references/rippled-api/api-conventions/basic-data-types.md b/content/references/rippled-api/api-conventions/basic-data-types.md index 4aacdd8f09..6faaf69731 100644 --- a/content/references/rippled-api/api-conventions/basic-data-types.md +++ b/content/references/rippled-api/api-conventions/basic-data-types.md @@ -24,7 +24,7 @@ Each closed [Ledger](ledger-data-formats.html) has a [Ledger Index][] and a [Has ### Hash Prefixes [[Source]
](https://github.com/ripple/rippled/blob/master/src/ripple/protocol/HashPrefix.h "Source") -In many cases, the XRP Ledger prefixes an object's binary data with a 4-byte code before calculating its hash, so that objects of different types have different hashes even if their binary formats are the same. The 4-byte codes are structured as three bytes of ASCII followed by a zero byte. +In many cases, the XRP Ledger prefixes an object's binary data with a 4-byte code before calculating its hash, so that objects of different types have different hashes even if their binary formats are the same. The existing 4-byte codes are structured as three alphabetic characters, encoded as ASCII, followed by a zero byte. Some types of hash appear in API requests and responses. Others are only calculated as the first step of signing a certain type of data, or calculating a higher-level hash. The following table shows all 4-byte hash prefixes the XRP Ledger uses: @@ -41,7 +41,7 @@ Some types of hash appear in API requests and responses. Others are only calcula | Unsigned Transaction (Single-signing) | N/A | `0x53545800` | `STX\0` | | Unsigned Transaction (Multi-signing) | N/A | `0x534D5400` | `SMT\0` | | Validation vote | N/A | `0x56414C00` | `VAL\0` | -| Validator ephemeral key update | N/A | `0x4D414E00` | `MAN\0` | +| Validator subkey authorization ("validator manifest") | N/A | `0x4D414E00` | `MAN\0` | [ledger header]: ledger-header.html [SHAMapv2]: known-amendments.html#shamapv2