Compare commits

...

12 Commits

Author SHA1 Message Date
Richard Holland
60caaea495 try again 2026-09-13 23:40:06 +10:00
RichardAHBot
4bbcc2d953 fix: regenerate hook/sfcodes.h on jsontx branch
Adds sfTime (UINT32/96), sfJsonTxDelta (VL/96), and sfAppLoader/sfAppLoaderID
which were added to sfields.macro but not propagated to the generated hook
header, causing verify-generated-headers CI failure.
2026-09-13 22:39:31 +10:00
RichardAHBot
72d793d912 style: clang-format-18 for jsontx files 2026-09-13 22:39:31 +10:00
RichardAHBot
db25f9799c test: add 15 more error path tests for delta decoder and timestamp parser
Cover unsanitize_jsontx error paths: truncated delta, unmerged literal/run,
empty delta, undersize copy, unmerged copy run, bad literal length, overlong
varint, and delta value out of range. Also add timestamp edge cases: malformed
ISO format, out-of-range month/day, non-leap Feb 29, and before-epoch dates.
2026-09-13 22:39:31 +10:00
RichardAHBot
e470d8a70b fix: tighten jsontx_exact integer boundary at 2^53
jsontx_exact: change the double boundary check from exclusive to inclusive
(d > max -> d >= max, d < -max -> d <= -max). Doubles cannot uniquely
represent odd integers at or above 2^53, so values like 2^53+1 silently
round to 2^53, corrupting the canonical form. The safe ceiling for
round-trip-exact integers is 2^53 - 1.

tests: add coverage for the 2^53 boundary (positive and negative sides,
and the silent rounding case).
2026-09-13 22:39:31 +10:00
RichardAHBot
4fb88f5f80 fix: reject \u escapes in jsontx_strict; expand tests
jsontx_strict: the comment claimed \u sequences were rejected but the
code only tracked backslashes without checking if they introduced a
unicode escape. Now properly throws on \u inside strings, which also
covers NUL-byte injection (\u0000) and prevents canonical form
mismatches caused by jsoncpp silently decoding \uXXXX.

tests: add coverage for \u rejection, jsontx_u64 negative/non-integer/
infinity rejection, delta size scaling with transaction complexity,
and sanitize rejection of unicode escapes.

Fixes a subtle security concern: without this check, a signer could
send a preimage with \uXXXX that jsoncpp would decode into a
different character, causing the canonical form to diverge from what
the signer actually signed while still passing the delta round-trip.
2026-09-13 22:39:31 +10:00
RichardAHBot
b2cdcf429e improve: add sig parameter check and expand json-tx test coverage
Submit.cpp: validate jss::sig presence before use so missing-signature
errors are reported clearly rather than as a misleading 'bad signature'.

JSONTxSignatures_test: 40 test sections covering:
- strict JSON parsing (comments, unicode, trailing commas, single quotes)
- ISO-8601 timestamp round-trips and overflow
- sanitize/unsanitize delta encoding round-trips
- unknown field and duplicate field rejection
- NUL byte rejection in string values
- document size limit enforcement
- nested objects and canonical field reordering
- delta decoding edge cases (varint bounds, too many ops, copy past end,
  unknown ops, non-minimal varints)
- multiple round-trip stability
- u64 formatting edge cases
2026-09-13 22:39:31 +10:00
RichardAHBot
3fd6ee472a add: unit tests for JSONTxSignatures
Comprehensive test coverage for jsontx_strict, jsontx_iso round-tripping,
sanitize/unsanitize delta encoding, jsontx_num edge cases, jsontx_field
lookup, and full pipeline integration tests.

Tests cover:
- Valid and invalid strict JSON parsing (comments, unicode, syntax)
- ISO-8601 timestamp round-trips and overflow rejection
- Delta encoding round-trips for various transaction shapes
- Bounds checking on malformed deltas
- Multiple round-trip stability
- Whitespace normalization producing identical canonical forms
- Delta size bounded by jsontx_max_diff
2026-09-13 22:39:31 +10:00
Richard Holland
b2aeb1cd92 more 2026-09-11 19:06:34 +10:00
Richard Holland
0f56d2d807 Merge branch 'dev' into jsontx 2026-09-11 16:18:41 +10:00
Richard Holland
e904290e4e jsontx stuff 2026-08-05 12:43:25 +10:00
Richard Holland
723103150c init jsontxsig amendment 2026-07-31 12:36:39 +10:00
10 changed files with 1478 additions and 31 deletions

View File

@@ -105,6 +105,7 @@
#define sfMPTAmount ((3U << 16U) + 26U)
#define sfIssuerNode ((3U << 16U) + 27U)
#define sfSubjectNode ((3U << 16U) + 28U)
#define sfTime ((3U << 16U) + 96U)
#define sfTouchCount ((3U << 16U) + 97U)
#define sfAccountIndex ((3U << 16U) + 98U)
#define sfAccountCount ((3U << 16U) + 99U)
@@ -222,6 +223,7 @@
#define sfProvider ((7U << 16U) + 30U)
#define sfMPTokenMetadata ((7U << 16U) + 31U)
#define sfCredentialType ((7U << 16U) + 32U)
#define sfJsonTxDelta ((7U << 16U) + 96U)
#define sfHookName ((7U << 16U) + 97U)
#define sfRemarkValue ((7U << 16U) + 98U)
#define sfRemarkName ((7U << 16U) + 99U)

View File

@@ -0,0 +1,739 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2014 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_PROTOCOL_JSONTXSIGNATURES_H_INCLUDED
#define RIPPLE_PROTOCOL_JSONTXSIGNATURES_H_INCLUDED
#include <xrpl/json/json_reader.h>
#include <xrpl/json/json_writer.h>
#include <xrpl/protocol/PublicKey.h>
#include <xrpl/protocol/SField.h>
#include <xrpl/protocol/STParsedJSON.h>
#include <xrpl/protocol/STTx.h>
#include <boost/algorithm/string.hpp>
#include <algorithm>
#include <charconv>
#include <cmath>
#include <cstdint>
#include <functional>
#include <map>
#include <stdexcept>
#include <string>
#include <string_view>
#include <unordered_map>
#include <utility>
#include <vector>
namespace ripple {
//------------------------------------------------------------------------------
// jsontx: plaintext-JSON signing support (featureJsonTx)
//
// The delta is attacker-controlled: it arrives over the wire beside a binary
// transaction and is not covered by the signature it helps reconstruct. Every
// bound below is therefore explicit, and unsanitize_jsontx accepts only the
// exact encoding sanitize_jsontx would have produced.
//------------------------------------------------------------------------------
// Every constant below decides whether a transaction is valid, so each is a
// consensus rule from the moment featureJsonTx activates and cannot be tuned
// afterwards without a further amendment. jsontx_min_copy and jsontx_max_cand
// are normative encoder parameters: sanitize_jsontx's output is compared byte
// for byte in jsontx_verify, so changing either changes which deltas verify.
//
// jsontx_max_diff bounds what an ordinary signer can send, not just what a
// node will accept. A pretty-printed document costs roughly 8 bytes and 2.3
// ops per line of delta (12 bytes / 2.3 ops at 4-space indent with CRLF), so
// 2048 bytes covers about 170 lines - a payment with a dozen memos, indented.
// sanitize_jsontx enforces both caps itself so that a document the encoder
// accepts is never one unsanitize_jsontx then refuses.
inline constexpr std::size_t jsontx_max_text = 8192; // canonical and original
inline constexpr std::size_t jsontx_max_diff = 2048; // delta bytes
inline constexpr std::size_t jsontx_max_ops =
jsontx_max_diff / 2; // delta instructions
inline constexpr std::size_t jsontx_min_copy = 4; // encoder match threshold
inline constexpr std::size_t jsontx_max_cand = 64; // encoder candidate cap
// Case-insensitive field-name -> canonical SField. Built once from
// SField::knownCodeToField, the same table doServerDefinitions publishes, using
// its serializability filter (useful, binary, non-pseudo). sfInvalid if
// unknown.
inline SField const&
jsontx_field(std::string const& name)
{
static auto const tbl = [] {
std::unordered_map<std::string, SField const*> m;
for (auto const& [code, f] : SField::knownCodeToField)
if (f->isUseful() && f->isBinary() && f->fieldType < 10000 &&
!f->fieldName.empty())
m.emplace(boost::algorithm::to_lower_copy(f->fieldName), f);
return m;
}();
auto const i = tbl.find(boost::algorithm::to_lower_copy(name));
return i == tbl.end() ? sfInvalid : *i->second;
}
// Civil calendar arithmetic (Howard Hinnant's algorithm), used in both
// directions. Pure integer maths - no strptime, no timegm, no locale, no
// tzdata - because these conversions decide whether a signature verifies and
// so must give the same answer on every node forever.
inline constexpr std::int64_t
jsontx_days(int y, unsigned m, unsigned d) // days from 1970-01-01
{
y -= m <= 2;
std::int64_t const era = (y >= 0 ? y : y - 399) / 400;
unsigned const yoe = static_cast<unsigned>(y - era * 400);
unsigned const doy = (153 * (m + (m > 2 ? -3 : 9)) + 2) / 5 + d - 1;
unsigned const doe = yoe * 365 + yoe / 4 - yoe / 100 + doy;
return era * 146097 + doe - 719468;
}
inline constexpr std::int64_t jsontx_epoch_day = jsontx_days(2000, 1, 1);
// 9999-12-31T23:59:59.999Z: past this toISOString() switches to expanded years
// (+275760-09-13T...) and the fixed 24 character shape no longer holds
inline constexpr std::uint64_t jsontx_max_time =
(static_cast<std::uint64_t>(jsontx_days(9999, 12, 31) - jsontx_epoch_day) *
86400 +
86399) *
1000 +
999;
static_assert(jsontx_epoch_day == 10957); // matches chrono.h epoch_offset
// Strict Date().toISOString() -> milliseconds since the ripple epoch. Exactly
// YYYY-MM-DDTHH:MM:SS.sssZ, always UTC, always three fractional digits.
inline std::uint64_t
jsontx_iso(std::string const& s)
{
static constexpr char pat[] = "0000-00-00T00:00:00.000Z";
if (s.size() != 24)
throw std::runtime_error("jsontx: Time must be an ISO 8601 instant");
// not std::isdigit: that consults the locale, and this comparison decides
// whether a signature verifies
auto const digit = [](char c) { return c >= '0' && c <= '9'; };
for (std::size_t i = 0; i < 24; ++i)
if (pat[i] == '0' ? !digit(s[i]) : s[i] != pat[i])
throw std::runtime_error("jsontx: malformed Time");
auto const n = [&s](std::size_t i, std::size_t c) {
int v = 0;
while (c--)
v = v * 10 + (s[i++] - '0');
return v;
};
int const y = n(0, 4), mo = n(5, 2), d = n(8, 2), h = n(11, 2),
mi = n(14, 2), se = n(17, 2), ms = n(20, 3);
if (mo < 1 || mo > 12)
throw std::runtime_error("jsontx: Time month out of range");
bool const leap = (y % 4 == 0 && y % 100 != 0) || y % 400 == 0;
int const dim =
mo == 2 ? (leap ? 29 : 28) : ((mo % 2 == 1) == (mo <= 7) ? 31 : 30);
// 60 is rejected: JS cannot emit a leap second and the ledger cannot
// represent one
if (d < 1 || d > dim || h > 23 || mi > 59 || se > 59)
throw std::runtime_error("jsontx: Time out of range");
std::int64_t const t = (jsontx_days(y, mo, d) - jsontx_epoch_day) * 86400 +
h * 3600 + mi * 60 + se;
if (t < 0)
throw std::runtime_error("jsontx: Time precedes the ripple epoch");
return static_cast<std::uint64_t>(t) * 1000 + ms;
}
// The exact inverse. Total over [0, jsontx_max_time] and injective, so sfTime
// and its ISO spelling are two views of one value and the delta carries
// nothing for the field.
inline std::string
jsontx_iso_str(std::uint64_t ms)
{
if (ms > jsontx_max_time)
throw std::runtime_error("jsontx: Time out of range");
std::int64_t const z =
static_cast<std::int64_t>(ms / 86400000) + jsontx_epoch_day + 719468;
unsigned const tod = static_cast<unsigned>(ms / 1000 % 86400);
std::int64_t const era = (z >= 0 ? z : z - 146096) / 146097;
unsigned const doe = static_cast<unsigned>(z - era * 146097);
unsigned const yoe = (doe - doe / 1460 + doe / 36524 - doe / 146096) / 365;
unsigned const doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
unsigned const mp = (5 * doy + 2) / 153;
unsigned const d = doy - (153 * mp + 2) / 5 + 1;
unsigned const m = mp + (mp < 10 ? 3 : -9);
std::int64_t const y =
static_cast<std::int64_t>(yoe) + era * 400 + (m <= 2);
// jsontx_max_time already bounds y to [2000, 9999], so every field below
// fits its width. Hand-rolled rather than formatted: this spelling is part
// of the signature preimage and a library's padding rules are not.
char buf[24];
auto const pad = [&buf](std::size_t at, std::uint64_t v, std::size_t w) {
while (w--)
{
buf[at + w] = static_cast<char>('0' + v % 10);
v /= 10;
}
};
pad(0, static_cast<std::uint64_t>(y), 4);
buf[4] = '-';
pad(5, m, 2);
buf[7] = '-';
pad(8, d, 2);
buf[10] = 'T';
pad(11, tod / 3600, 2);
buf[13] = ':';
pad(14, tod / 60 % 60, 2);
buf[16] = ':';
pad(17, tod % 60, 2);
buf[19] = '.';
pad(20, ms % 1000, 3);
buf[23] = 'Z';
return std::string(buf, sizeof(buf));
}
//------------------------------------------------------------------------------
// jsoncpp compatibility
//
// This is xrpl's vendored jsoncpp, which predates JSON_HAS_INT64: Json::Int is
// int and Json::UInt is unsigned int, both 32 bit, and there is no isInt64 /
// asInt64 / asUInt64. Reader::decodeNumber yields intValue or uintValue only
// while the digits fit in 32 bits; on overflow, and for any token carrying a
// '.' or an exponent, it falls through to decodeDouble and the number arrives
// as a realValue.
//
// So a large integer is not lost, but it is no longer held as an integer, and
// the digits the signer wrote are recoverable only while the double is an
// exact integer view of them. That holds to 2^53; above it consecutive doubles
// are more than 1 apart and distinct decimal integers collapse onto the same
// double. Past that point the value is refused rather than guessed at.
//------------------------------------------------------------------------------
inline constexpr double jsontx_exact_max = 9007199254740992.0; // 2^53
// Exact integer view of a json number. False if v is not a number at all, or
// is one this build cannot reproduce digit for digit.
inline bool
jsontx_exact(Json::Value const& v, std::int64_t& out)
{
switch (v.type())
{
case Json::intValue:
out = v.asInt();
return true;
case Json::uintValue:
out = static_cast<std::int64_t>(v.asUInt());
return true;
case Json::realValue:
break;
default: // including booleanValue, which isIntegral() would admit
return false;
}
double const d = v.asDouble();
if (!std::isfinite(d) || d != std::trunc(d) || d <= -jsontx_exact_max ||
d >= jsontx_exact_max)
return false;
out = static_cast<std::int64_t>(d);
return true;
}
// STI_UINT64 renders as a fixed 16 digit hex string and so needs the value
// unsigned. A negative is refused rather than wrapped: the old
// static_cast<std::uint64_t>(v.asDouble()) was undefined for a negative or
// oversized double, and a silent wrap would let -1 and 18446744073709551615
// canonicalize to the same bytes.
inline std::uint64_t
jsontx_u64(Json::Value const& v)
{
std::int64_t n = 0;
if (!jsontx_exact(v, n) || n < 0)
throw std::runtime_error(
"jsontx: UInt64 must be an exact non-negative integer");
return static_cast<std::uint64_t>(n);
}
// STUInt64::getJson renders through std::to_chars, so: lowercase, unpadded,
// and base ten for sMD_BaseTen fields. Any other spelling is one the node can
// never re-derive from its own transaction, which would stop the canonical
// form being a fixed point.
inline std::string
jsontx_u64_str(SField const& f, std::uint64_t v)
{
char buf[20];
auto const r = std::to_chars(
buf, buf + sizeof(buf), v, f.shouldMeta(SField::sMD_BaseTen) ? 10 : 16);
return std::string(buf, r.ptr);
}
// Renders a javascript number as an exact integer. Anything this build cannot
// reproduce digit for digit - a fraction, an infinity, a magnitude past 2^53 -
// is rejected outright: shortest-round-trip rendering of a double is not
// portable enough to sit in a consensus preimage, and nothing in a transaction
// needs one. Fractional amounts arrive as strings, which is what the ledger
// wants anyway.
inline std::string
jsontx_num(Json::Value const& v)
{
std::int64_t n = 0;
if (!jsontx_exact(v, n))
throw std::runtime_error("jsontx: number must be an exact integer");
return std::to_string(n);
}
// The vendored jsoncpp accepts both comment styles, and stops looking once it
// has a root value rather than requiring end of input. So
//
// {"TransactionType":"Payment",...} /* sign this to log in */
//
// parses, and the trailing text is reproduced verbatim by the delta and so
// sits inside the signed preimage. For a feature whose whole premise is that
// the signer reads what they sign, text outside the object cannot be allowed
// to ride along. Reject it before the parser ever sees the document.
//
// This validates framing only - one bracketed value, no comments, nothing
// after it. Whether the contents are legal json is still the parser's job.
inline void
jsontx_strict(std::string_view raw)
{
std::size_t depth = 0;
bool str = false, esc = false, closed = false;
for (std::size_t i = 0; i < raw.size(); ++i)
{
char const c = raw[i];
if (str)
{
if (esc)
{
if (c == 'u')
throw std::runtime_error(
"jsontx: \\u escapes are not allowed");
esc = false;
}
else if (c == '\\')
esc = true;
else if (c == '"')
str = false;
continue;
}
switch (c)
{
// json's whitespace set, exactly
case ' ':
case '\t':
case '\r':
case '\n':
continue;
case '"':
str = true;
break;
case '{':
case '[':
++depth;
break;
case '}':
case ']':
if (depth == 0)
throw std::runtime_error("jsontx: unbalanced document");
if (--depth == 0)
closed = true;
continue;
case '/':
// a bare '/' is not legal json either way, so leave that to
// the parser and reject only what the parser would accept
if (i + 1 < raw.size() &&
(raw[i + 1] == '/' || raw[i + 1] == '*'))
throw std::runtime_error(
"jsontx: comments are not allowed");
break;
default:
break;
}
if (closed)
throw std::runtime_error("jsontx: trailing data after document");
}
if (str || depth || !closed)
throw std::runtime_error("jsontx: malformed json");
}
// Returns { sanitized, diff }. `sanitized` is the canonical form: whitespace
// stripped, field names capitalized to their xahau spelling, members reordered
// by field code, numbers reformatted per field type. `diff` is a binary delta
// which, applied to `sanitized` by unsanitize_jsontx, reproduces `raw` byte for
// byte. Throws on anything it cannot canonicalize.
inline std::pair<std::string, std::string>
sanitize_jsontx(std::string_view raw)
{
if (raw.size() > jsontx_max_text)
throw std::runtime_error("jsontx: document too large");
jsontx_strict(raw);
Json::Value jv;
if (Json::Reader r;
!r.parse(raw.data(), raw.data() + raw.size(), jv) || !jv.isObject())
throw std::runtime_error("jsontx: malformed json");
// (a plain recursive lambda; deducing-this would drop the std::function)
std::function<void(
Json::Value const&, SerializedTypeID, SField const*, std::string&)>
emit = [&](Json::Value const& v,
SerializedTypeID ty,
SField const* fld,
std::string& o) {
if (v.isObject())
{
auto keys = v.getMemberNames();
o += '{';
if (ty == STI_OBJECT) // keys are xahau fields
{
std::vector<std::pair<SField const*, std::string>> ks;
for (auto const& k : keys)
{
auto const& f = jsontx_field(k);
if (f == sfInvalid)
throw std::runtime_error(
"jsontx: unknown field '" + k + "'");
ks.emplace_back(&f, k);
}
std::sort(
ks.begin(), ks.end(), [](auto const& a, auto const& b) {
return a.first->fieldCode < b.first->fieldCode;
});
for (std::size_t n = 0; n < ks.size(); ++n)
{
auto const& [f, k] = ks[n];
if (n && f == ks[n - 1].first) // e.g. "Fee" and "fee"
throw std::runtime_error(
"jsontx: duplicate field '" + f->fieldName +
"'");
if (o.back() != '{')
o += ',';
o += Json::valueToQuotedString(f->fieldName.c_str()) +
':';
emit(v[k], f->fieldType, f, o);
}
}
else // amount / issue style subobject: lexicographic, quoted
{
std::sort(keys.begin(), keys.end());
for (auto const& k : keys)
{
if (o.back() != '{')
o += ',';
o += Json::valueToQuotedString(k.c_str()) + ':';
emit(v[k], STI_NOTPRESENT, nullptr, o);
}
}
o += '}';
}
else if (v.isArray())
{
o += '[';
for (auto const& e : v)
{
if (o.back() != '[')
o += ',';
emit(e, STI_OBJECT, nullptr, o);
}
o += ']';
}
else if (v.isString())
{
// Time is spelled Date().toISOString() in the preimage and
// stored as an sfTime u64 of milliseconds. Re-emitting the
// round-tripped spelling rather than the input is what makes
// the canonical form a fixed point: any string that is not
// exactly what jsontx_iso_str produces is rejected here.
if (fld && *fld == sfTime)
o += Json::valueToQuotedString(
jsontx_iso_str(jsontx_iso(v.asString())).c_str());
else
{
// valueToQuotedString takes a char const* and so stops at
// an embedded NUL, which would let two different strings
// canonicalize to the same bytes. Nothing in a
// transaction needs one, so refuse rather than truncate.
auto const t = v.asString();
if (t.find('\0') != std::string::npos)
throw std::runtime_error("jsontx: NUL in string value");
o += Json::valueToQuotedString(t.c_str());
}
}
else if (v.isBool())
o += v.asBool() ? "true" : "false";
else if (v.isNull())
throw std::runtime_error("jsontx: null value");
else if (ty == STI_UINT8 || ty == STI_UINT16 || ty == STI_UINT32)
o += jsontx_num(v); // small ints stay bare
else if (ty == STI_UINT64)
{
if (!fld)
throw std::runtime_error("jsontx: unnamed UInt64");
o += '"' + jsontx_u64_str(*fld, jsontx_u64(v)) + '"';
}
else // amounts, u64, everything else the ledger wants as a string
o += Json::valueToQuotedString(jsontx_num(v).c_str());
};
std::string out;
emit(jv, STI_OBJECT, nullptr, out);
// Greedy copy/insert delta over `raw`, sourcing from `out`.
// op 0x00 <varint len> <bytes> literal
// op 0x01 <varint off> <varint len> copy from sanitized
std::string diff, lit;
std::size_t ops = 0;
auto const gram = [](std::string_view s, std::size_t i) {
return std::uint32_t(std::uint8_t(s[i])) << 24 |
std::uint32_t(std::uint8_t(s[i + 1])) << 16 |
std::uint32_t(std::uint8_t(s[i + 2])) << 8 |
std::uint32_t(std::uint8_t(s[i + 3]));
};
auto const varint = [](std::string& o, std::uint64_t v) {
do
{
std::uint8_t const c = v & 0x7F;
v >>= 7;
o += static_cast<char>(c | (v ? 0x80 : 0));
} while (v);
};
auto const flush = [&] {
if (lit.empty())
return;
diff += char(0);
varint(diff, lit.size());
diff += lit;
lit.clear();
++ops;
};
// This encoder is normative - unsanitize_jsontx only accepts its exact
// output - so it must emit identical bytes on every node and every stdlib.
// An unordered container would not: bucket order for equal keys is
// unspecified, and with a candidate cap that changes which match wins.
std::map<std::uint32_t, std::vector<std::uint32_t>> idx;
for (std::size_t i = 0; i + jsontx_min_copy <= out.size(); ++i)
idx[gram(out, i)].push_back(i);
for (std::size_t i = 0; i < raw.size();)
{
std::size_t bo = 0, bl = 0;
if (i + jsontx_min_copy <= raw.size())
if (auto const it = idx.find(gram(raw, i)); it != idx.end())
{
std::size_t tried = 0;
for (auto const off : it->second) // ascending, so ties keep
{ // the lowest offset
if (++tried > jsontx_max_cand)
break;
std::size_t l = 0;
while (i + l < raw.size() && off + l < out.size() &&
out[off + l] == raw[i + l])
++l;
if (l > bl)
bl = l, bo = off;
}
}
if (bl >= jsontx_min_copy)
{
flush();
diff += char(1);
varint(diff, bo);
varint(diff, bl);
++ops;
i += bl;
}
else
lit += raw[i++];
}
flush();
// The same two bounds unsanitize_jsontx applies, applied here so that a
// document this function accepts is never one the verifier then refuses.
// Almost always this means "too much whitespace to encode": the canonical
// form itself is well inside jsontx_max_text.
if (diff.size() > jsontx_max_diff || ops > jsontx_max_ops)
throw std::runtime_error(
"jsontx: formatting differs too much from canonical form");
return {std::move(out), std::move(diff)};
}
// Applies an UNTRUSTED delta to a canonical form the node derived itself.
// Copies read only from `sanitized`, never from the output being built, so a
// short delta cannot expand geometrically. Offsets and lengths are range
// checked before use, varints are length- and minimality-bounded, and the two
// encodings the encoder can never emit - an unmerged literal run, and a copy
// abutting the previous copy in the source - are rejected. Throws on anything
// else.
inline std::string
unsanitize_jsontx(std::string_view sanitized, std::string_view diff)
{
if (sanitized.size() > jsontx_max_text || diff.size() > jsontx_max_diff)
throw std::runtime_error("jsontx: oversize delta input");
std::string out;
std::size_t p = 0, ops = 0, prevEnd = 0;
int prev = -1;
auto const varint = [&](std::uint64_t max) -> std::uint64_t {
std::uint64_t v = 0;
for (int s = 0; s <= 21; s += 7) // four bytes; caps far under a shift
{ // wide enough to be undefined
if (p >= diff.size())
throw std::runtime_error("jsontx: truncated delta");
std::uint8_t const c = diff[p++];
v |= std::uint64_t(c & 0x7F) << s;
if (c & 0x80)
continue;
if (s && !(c & 0x7F))
throw std::runtime_error("jsontx: non-minimal varint");
if (v > max)
throw std::runtime_error("jsontx: delta value out of range");
return v;
}
throw std::runtime_error("jsontx: overlong varint");
};
while (p < diff.size())
{
if (++ops > jsontx_max_ops)
throw std::runtime_error("jsontx: too many delta ops");
std::uint8_t const op = diff[p++];
if (op > 1)
throw std::runtime_error("jsontx: unknown delta op");
std::size_t n = 0;
if (op == 0) // literal
{
if (prev == 0)
throw std::runtime_error("jsontx: unmerged literal run");
n = varint(jsontx_max_text);
if (n == 0 || n > diff.size() - p)
throw std::runtime_error("jsontx: bad literal length");
if (out.size() + n > jsontx_max_text)
throw std::runtime_error("jsontx: delta expands too far");
out += diff.substr(p, n);
p += n;
}
else // copy from the canonical form
{
auto const off = varint(sanitized.size());
n = varint(sanitized.size() - off);
if (n < jsontx_min_copy)
throw std::runtime_error("jsontx: undersize copy");
if (prev == 1 && off == prevEnd)
throw std::runtime_error("jsontx: unmerged copy run");
if (out.size() + n > jsontx_max_text)
throw std::runtime_error("jsontx: delta expands too far");
out += sanitized.substr(off, n);
prevEnd = off + n;
}
prev = op;
}
if (out.empty())
throw std::runtime_error("jsontx: empty delta");
return out;
}
// The complete untrusted-side check, in one place so the RPC path and the
// relay/consensus path cannot drift. Takes the transaction exactly as it came
// off the wire and returns the reconstructed preimage.
inline std::string
jsontx_verify(STTx const& stx, std::string_view diff)
{
if (!stx.isFieldPresent(sfTxnSignature) || stx.isFieldPresent(sfSigners))
throw std::runtime_error("jsontx: expects a lone TxnSignature");
// Out comes everything the signer did not have in front of them: the
// signature, and the delta carrier. SigningPubKey stays. It being inside
// the preimage is what binds key to signature and stops a third party
// re-signing a captured preimage under their own key.
auto txj = stx.STObject::getJson(JsonOptions::none);
txj.removeMember(sfTxnSignature.fieldName);
txj.removeMember(sfJsonTxDelta.fieldName);
// sfTime is a u64 of milliseconds on the wire and an ISO 8601 instant in
// the preimage. The two are a bijection over the representable range, so
// this is a rewrite rather than a reconstruction and the delta carries
// nothing for the field.
if (stx.isFieldPresent(sfTime))
txj[sfTime.fieldName] = jsontx_iso_str(stx.getFieldU64(sfTime));
auto const pkb = stx.getSigningPubKey();
if (publicKeyType(makeSlice(pkb)) != KeyType::ed25519)
throw std::runtime_error("jsontx: SigningPubKey must be ed25519");
// canonical form, derived only from data the node has already validated
auto const san = sanitize_jsontx(Json::FastWriter{}.write(txj)).first;
// reconstruct the signed preimage under the caps above
auto const raw = unsanitize_jsontx(san, diff);
// Bind the preimage to the transaction. This is the load-bearing check,
// not a sanity check: a delta of pure literals can reconstruct ANY text,
// so without it any ed25519 signature the key ever produced over anything
// at all would authorise this transaction. Comparing the delta too - not
// just the canonical form - makes the delta a pure function of the
// preimage, which rules out a second delta reconstructing the same bytes
// and yielding a second valid transaction id.
auto const [san2, diff2] = sanitize_jsontx(raw);
if (san2 != san || diff2 != diff)
throw std::runtime_error("jsontx: preimage does not match transaction");
if (!verify(
PublicKey(makeSlice(pkb)),
makeSlice(raw),
makeSlice(stx.getFieldVL(sfTxnSignature))))
throw std::runtime_error("jsontx: signature does not verify");
return raw;
}
// The relay and consensus entry point: same check, with the delta taken from
// the transaction rather than passed alongside it. This is what checkValidity
// calls in place of STTx::checkSign for a transaction carrying a delta.
inline std::string
jsontx_verify(STTx const& stx)
{
if (!stx.isFieldPresent(sfJsonTxDelta))
throw std::runtime_error("jsontx: no delta");
auto const delta = stx.getFieldVL(sfJsonTxDelta);
return jsontx_verify(
stx,
std::string_view(
reinterpret_cast<char const*>(delta.data()), delta.size()));
}
} // namespace ripple
#endif

View File

@@ -34,6 +34,7 @@
// If you add an amendment here, then do not forget to increment `numFeatures`
// in include/xrpl/protocol/Feature.h.
XRPL_FEATURE(JsonTx, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(OnChainManifests, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FIX (HookMap, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FIX (GuardDepth32, Supported::yes, VoteBehavior::DefaultNo)

View File

@@ -153,6 +153,7 @@ TYPED_SFIELD(sfOutstandingAmount, UINT64, 25, SField::sMD_BaseTen|SFie
TYPED_SFIELD(sfMPTAmount, UINT64, 26, SField::sMD_BaseTen|SField::sMD_Default)
TYPED_SFIELD(sfIssuerNode, UINT64, 27)
TYPED_SFIELD(sfSubjectNode, UINT64, 28)
TYPED_SFIELD(sfTime, UINT64, 96)
TYPED_SFIELD(sfTouchCount, UINT64, 97)
TYPED_SFIELD(sfAccountIndex, UINT64, 98)
TYPED_SFIELD(sfAccountCount, UINT64, 99)
@@ -294,6 +295,7 @@ TYPED_SFIELD(sfAssetClass, VL, 29)
TYPED_SFIELD(sfProvider, VL, 30)
TYPED_SFIELD(sfMPTokenMetadata, VL, 31)
TYPED_SFIELD(sfCredentialType, VL, 32)
TYPED_SFIELD(sfJsonTxDelta, VL, 96, SField::sMD_Default, SField::notSigning)
TYPED_SFIELD(sfHookName, VL, 97)
TYPED_SFIELD(sfRemarkValue, VL, 98)
TYPED_SFIELD(sfRemarkName, VL, 99)

View File

@@ -631,6 +631,7 @@ JSS(server_status); // out: NetworkOPs
JSS(server_version); // out: NetworkOPs
JSS(settle_delay); // out: AccountChannels
JSS(severity); // in: LogLevel
JSS(sig);
JSS(signature); // out: NetworkOPs, ChannelAuthorize
JSS(signature_verified); // out: ChannelVerify
JSS(signing_key); // out: NetworkOPs

View File

@@ -49,6 +49,8 @@ TxFormats::TxFormats()
{sfNetworkID, soeOPTIONAL},
{sfHookParameters, soeOPTIONAL},
{sfHookName, soeOPTIONAL},
{sfTime, soeOPTIONAL},
{sfJsonTxDelta, soeOPTIONAL},
};
#pragma push_macro("UNWRAP")

View File

@@ -0,0 +1,558 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2012-2014 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <xrpl/beast/unit_test.h>
#include <xrpl/protocol/JSONTxSignatures.h>
#include <cstdint>
#include <exception>
#include <limits>
#include <string>
#include <string_view>
namespace ripple {
/**
* Unit tests for JsonTx plaintext transaction signature support.
*
* Covers:
* - jsontx_strict: document framing (no comments, no \u, nothing outside)
* - jsontx_iso / jsontx_iso_str: ISO 8601 <-> ripple-epoch milliseconds
* - sanitize_jsontx / unsanitize_jsontx: canonical form + delta round-trip
* - unsanitize_jsontx: rejection of every delta the encoder cannot emit
* - jsontx_exact / jsontx_num / jsontx_u64 / jsontx_u64_str: number handling
* - jsontx_field: case-insensitive canonical field lookup
*
* All of these report failure by throwing rather than by returning a status,
* so the helpers below adapt them to the boolean BEAST_EXPECT wants.
*/
class JSONTxSignatures_test : public beast::unit_test::suite
{
// True if f() threw. Used in place of a throws-macro so the expectation
// still reports the failing file and line through BEAST_EXPECT.
template <class F>
static bool
threw(F&& f)
{
try
{
f();
}
catch (std::exception const&)
{
return true;
}
return false;
}
static bool
strictOk(std::string_view s)
{
return !threw([&] { jsontx_strict(s); });
}
// sanitize then unsanitize must reproduce the input byte for byte; this is
// the property jsontx_verify rests on.
static bool
roundTrips(std::string const& raw)
{
auto const [san, diff] = sanitize_jsontx(raw);
return unsanitize_jsontx(san, diff) == raw;
}
static bool
unsanitizeThrows(std::string_view san, std::string const& diff)
{
return threw([&] { (void)unsanitize_jsontx(san, diff); });
}
void
testStrictAccepts()
{
testcase("jsontx_strict accepts");
BEAST_EXPECT(strictOk("{}"));
BEAST_EXPECT(strictOk(R"({"key":"value"})"));
BEAST_EXPECT(strictOk(R"({"n":0})"));
BEAST_EXPECT(strictOk(R"({"n":12345})"));
BEAST_EXPECT(strictOk(R"({"n":-1})"));
BEAST_EXPECT(strictOk(R"({"outer":{"inner":true}})"));
BEAST_EXPECT(strictOk(R"({"a":[1,2,3]})"));
BEAST_EXPECT(strictOk(R"({"b":true})"));
BEAST_EXPECT(strictOk(R"({"b":false})"));
BEAST_EXPECT(strictOk(R"({"n":null})"));
// json's whitespace set, between tokens and around the document
BEAST_EXPECT(strictOk(" { \"key\" : \"val\" } "));
BEAST_EXPECT(strictOk("{\r\n\t\"key\":\"val\"\r\n}"));
// an escaped backslash ends the escape, so the 'u' after it is an
// ordinary character and not a \u escape
BEAST_EXPECT(strictOk(R"({"key":"\\utest"})"));
// a solidus that does not open a comment is the parser's business
BEAST_EXPECT(strictOk(R"({"key":"a/b"})"));
// This validates framing only. Trailing commas, single quotes,
// unquoted keys and non-object roots are all rejected too, but by the
// parser or by sanitize_jsontx rather than here.
BEAST_EXPECT(strictOk(R"({"key":"val",})"));
BEAST_EXPECT(strictOk(R"({'key':'val'})"));
BEAST_EXPECT(strictOk(R"({key:"val"})"));
BEAST_EXPECT(strictOk(R"([1,2,3])"));
}
void
testStrictRejects()
{
testcase("jsontx_strict rejects");
// comments outside the document, in either style
BEAST_EXPECT(!strictOk("{\"key\":\"value\"}\n// real comment"));
BEAST_EXPECT(!strictOk("{\"key\":\"value\"} /* real comment */"));
BEAST_EXPECT(!strictOk("// leading comment\n{\"key\":\"value\"}"));
// ...but a comment marker inside a string is just text
BEAST_EXPECT(strictOk(R"({"key":"value // not a comment"})"));
// \u escapes never appear in a preimage: the signer has to be able to
// read what they are signing
BEAST_EXPECT(!strictOk(R"({"key":"\u0048ello"})"));
BEAST_EXPECT(!strictOk(R"({"key":"hello\u0041"})"));
BEAST_EXPECT(!strictOk(R"({"key":"\u0000"})"));
// nothing may ride along after the document
BEAST_EXPECT(!strictOk(R"({"a":"b"} "trailing")"));
BEAST_EXPECT(!strictOk(R"({"a":"b"} {"c":"d"})"));
// unbalanced brackets
BEAST_EXPECT(!strictOk(R"({"a":"b")"));
BEAST_EXPECT(!strictOk(R"({"a":"b"}})"));
// unterminated string
BEAST_EXPECT(!strictOk(R"({"a":"b})"));
// a bare scalar is not a bracketed document
BEAST_EXPECT(!strictOk(R"("just a string")"));
BEAST_EXPECT(!strictOk("42"));
BEAST_EXPECT(!strictOk("true"));
BEAST_EXPECT(!strictOk("null"));
BEAST_EXPECT(!strictOk(""));
BEAST_EXPECT(!strictOk(" "));
}
void
testIsoRoundTrip()
{
testcase("ISO 8601 round-trip");
BEAST_EXPECT(jsontx_iso("2000-01-01T00:00:00.000Z") == 0);
BEAST_EXPECT(jsontx_iso_str(0) == "2000-01-01T00:00:00.000Z");
auto rt = [](char const* s) {
return jsontx_iso_str(jsontx_iso(s)) == s;
};
BEAST_EXPECT(rt("2000-01-01T00:00:00.000Z"));
BEAST_EXPECT(rt("2025-06-15T12:30:45.123Z"));
BEAST_EXPECT(rt("2000-02-29T00:00:00.000Z")); // leap year
BEAST_EXPECT(rt("2400-02-29T00:00:00.000Z")); // 400-year leap year
BEAST_EXPECT(rt("9999-12-31T23:59:59.999Z"));
// the last instant with a 24 character spelling
BEAST_EXPECT(jsontx_iso("9999-12-31T23:59:59.999Z") == jsontx_max_time);
BEAST_EXPECT(
jsontx_iso_str(jsontx_max_time) == "9999-12-31T23:59:59.999Z");
BEAST_EXPECT(threw([] { (void)jsontx_iso_str(jsontx_max_time + 1); }));
// milliseconds are carried, not truncated
BEAST_EXPECT(jsontx_iso("2000-01-01T00:00:00.001Z") == 1);
BEAST_EXPECT(jsontx_iso("2000-01-02T00:00:00.000Z") == 86400000);
}
void
testIsoRejects()
{
testcase("ISO 8601 rejection");
auto bad = [](char const* s) {
return threw([&] { (void)jsontx_iso(s); });
};
// the shape is exactly YYYY-MM-DDTHH:MM:SS.sssZ, always 24 characters
BEAST_EXPECT(bad("2000-01-01T00:00:00")); // no fraction, no Z
BEAST_EXPECT(bad("2000-01-01T00:00:00.Z")); // no fraction
BEAST_EXPECT(bad("2000-01-01T00:00:00.000")); // no Z
BEAST_EXPECT(bad("2000-01-01T00:00:00.000+00:00")); // offset
BEAST_EXPECT(bad("2000-01-01 00:00:00.000Z")); // space for T
BEAST_EXPECT(bad("2000/01/01T00:00:00.000Z"));
BEAST_EXPECT(bad("not-a-date"));
BEAST_EXPECT(bad(""));
// out of range components
BEAST_EXPECT(bad("2000-00-01T00:00:00.000Z")); // month 0
BEAST_EXPECT(bad("2000-13-01T00:00:00.000Z")); // month 13
BEAST_EXPECT(bad("2000-01-00T00:00:00.000Z")); // day 0
BEAST_EXPECT(bad("2000-02-30T00:00:00.000Z")); // Feb 30
BEAST_EXPECT(bad("2001-02-29T00:00:00.000Z")); // non-leap Feb 29
BEAST_EXPECT(bad("1900-02-29T00:00:00.000Z")); // century non-leap
BEAST_EXPECT(bad("2000-01-01T24:00:00.000Z")); // hour 24
BEAST_EXPECT(bad("2000-01-01T00:60:00.000Z")); // minute 60
BEAST_EXPECT(bad("2000-01-01T00:00:60.000Z")); // leap second
// before the ripple epoch
BEAST_EXPECT(bad("1999-12-31T23:59:59.999Z"));
}
void
testSanitizeRoundTrip()
{
testcase("sanitize/unsanitize round-trip");
// already canonical
BEAST_EXPECT(roundTrips(
R"({"TransactionType":"Payment","Sequence":1,"Fee":"10","SigningPubKey":"ED0000","Account":"rTest"})"));
// source order is irrelevant: the canonical form sorts by field code
BEAST_EXPECT(roundTrips(
R"({"Account":"rTest","Fee":"10","Sequence":1,"SigningPubKey":"ED0000","TransactionType":"Payment"})"));
// sfTime is an ISO instant in the preimage and a u64 on the wire
BEAST_EXPECT(roundTrips(
R"({"Account":"rTest","Fee":"10","Sequence":1,"SigningPubKey":"ED0000","Time":"2000-01-01T00:00:00.000Z","TransactionType":"Payment"})"));
// interior whitespace is carried by the delta
BEAST_EXPECT(roundTrips(R"({ "Account" : "rTest" , "Fee" : "10" })"));
// pretty-printed, which is what a signer actually reads
BEAST_EXPECT(
roundTrips("{\n"
" \"Account\": \"rTest\",\n"
" \"Fee\": \"10\",\n"
" \"Sequence\": 1,\n"
" \"SigningPubKey\": \"ED\",\n"
" \"Time\": \"2000-01-01T00:00:00.000Z\",\n"
" \"TransactionType\": \"Payment\"\n"
"}"));
// CRLF, which is what a browser textarea produces
BEAST_EXPECT(
roundTrips("{\r\n"
" \"Account\": \"rTest\",\r\n"
" \"Fee\": \"10\",\r\n"
" \"TransactionType\": \"Payment\"\r\n"
"}"));
// nested objects and arrays
BEAST_EXPECT(roundTrips(
R"({"Account":"rTest","Fee":"10","Memos":[{"Memo":{"MemoData":"48656C6C6F"}}],"Sequence":1,"TransactionType":"Payment"})"));
// a u32 above the signed range stays bare and exact
BEAST_EXPECT(roundTrips(
R"({"Account":"rTest","Fee":"10","Flags":2147483648,"Sequence":1})"));
// field names match case-insensitively and re-emit canonically
BEAST_EXPECT(roundTrips(R"({"account":"rTest","FEE":"10"})"));
}
void
testCanonicalForm()
{
testcase("canonical form");
auto const compact =
sanitize_jsontx(R"({"Account":"rTest","Fee":"10"})");
auto const spaced =
sanitize_jsontx(R"({ "Account" : "rTest" , "Fee" : "10" })").first;
auto const reordered =
sanitize_jsontx(R"({"Fee":"10","Account":"rTest"})").first;
auto const cased =
sanitize_jsontx(R"({"account":"rTest","fee":"10"})").first;
// whitespace, member order and spelling all wash out
BEAST_EXPECT(compact.first == spaced);
BEAST_EXPECT(compact.first == reordered);
BEAST_EXPECT(compact.first == cased);
// the canonical form is a fixed point, which is what lets jsontx_verify
// compare a node-derived form against a signer-derived one
BEAST_EXPECT(sanitize_jsontx(compact.first).first == compact.first);
// and both halves sit inside the caps the verifier applies
BEAST_EXPECT(compact.first.size() <= jsontx_max_text);
BEAST_EXPECT(compact.second.size() <= jsontx_max_diff);
}
void
testSanitizeRejects()
{
testcase("sanitize_jsontx rejection");
auto bad = [](std::string const& raw) {
return threw([&] { (void)sanitize_jsontx(raw); });
};
// not json at all
BEAST_EXPECT(bad("not json"));
BEAST_EXPECT(bad(""));
// a preimage is an object
BEAST_EXPECT(bad(R"("just a string")"));
BEAST_EXPECT(bad(R"([1,2,3])"));
// every member must name a serializable field
BEAST_EXPECT(bad(R"({"Account":"rTest","Fee":"10","BogusField":"x"})"));
// two spellings of one field would canonicalize to the same member
BEAST_EXPECT(bad(R"({"Fee":"10","fee":"20"})"));
// null has no canonical spelling as a field value
BEAST_EXPECT(bad(R"({"Account":"rTest","Fee":null})"));
// framing violations reach sanitize_jsontx through jsontx_strict
BEAST_EXPECT(bad("{\"Account\":\"rTest\"} // hi"));
BEAST_EXPECT(
bad(R"({"Account":"rTest","Fee":"10","Domain":"\u0041"})"));
// over the text cap
{
std::string large = "{\"Account\":\"";
large.append(jsontx_max_text, 'r');
large += "\"}";
BEAST_EXPECT(bad(large));
}
// an sfTime that is not exactly what jsontx_iso_str would produce
BEAST_EXPECT(bad(R"({"Account":"rTest","Time":"2000-01-01"})"));
}
void
testDeltaRejection()
{
testcase("unsanitize_jsontx rejection");
// unsanitize_jsontx never parses its first argument, so any text will
// do as a source for the decoder's own error paths
std::string const san = "abcdefghij";
// a delta must reconstruct something
BEAST_EXPECT(unsanitizeThrows(san, ""));
// only two ops exist
BEAST_EXPECT(unsanitizeThrows(san, std::string("\x02", 1)));
BEAST_EXPECT(unsanitizeThrows(san, std::string("\x7f", 1)));
// truncated: an op with no varint after it
BEAST_EXPECT(unsanitizeThrows(san, std::string("\x00", 1)));
// zero-length literal
BEAST_EXPECT(unsanitizeThrows(san, std::string("\x00\x00", 2)));
// literal claiming more bytes than the delta holds
BEAST_EXPECT(
unsanitizeThrows(san, std::string("\x00\x81\x01", 3) + "short"));
// non-minimal varint: a continuation byte followed by zero
BEAST_EXPECT(unsanitizeThrows(san, std::string("\x00\x80\x00", 3)));
// overlong varint: more than four bytes
BEAST_EXPECT(
unsanitizeThrows(san, std::string("\x00\xff\xff\xff\xff\x01", 6)));
// two literals in a row; the encoder always merges them
BEAST_EXPECT(unsanitizeThrows(
san,
std::string("\x00\x02", 2) + "ab" + std::string("\x00\x02", 2) +
"cd"));
// a copy shorter than the encoder's match threshold
BEAST_EXPECT(unsanitizeThrows(san, std::string("\x01\x00\x01", 3)));
// a copy abutting the previous copy; the encoder always merges them
BEAST_EXPECT(
unsanitizeThrows(san, std::string("\x01\x00\x04\x01\x04\x04", 6)));
// offset past the end of the canonical form
BEAST_EXPECT(unsanitizeThrows(san, std::string("\x01\xff\xff\x04", 4)));
// length running past the end from a valid offset
BEAST_EXPECT(unsanitizeThrows(san, std::string("\x01\x08\x08", 3)));
// oversize inputs are refused before any decoding
BEAST_EXPECT(
unsanitizeThrows(san, std::string(jsontx_max_diff + 1, '\x00')));
BEAST_EXPECT(unsanitizeThrows(
std::string(jsontx_max_text + 1, 'x'),
std::string("\x01\x00\x04", 3)));
// a well-formed copy is accepted, so the cases above fail for their
// own reasons and not because the fixture itself is malformed
BEAST_EXPECT(
unsanitize_jsontx(san, std::string("\x01\x00\x05", 3)) == "abcde");
}
void
testNumbers()
{
testcase("number handling");
std::int64_t out = 0;
// small integers arrive as intValue / uintValue
BEAST_EXPECT(jsontx_exact(Json::Value(0), out) && out == 0);
BEAST_EXPECT(jsontx_exact(Json::Value(-1), out) && out == -1);
BEAST_EXPECT(jsontx_exact(Json::Value(42u), out) && out == 42);
// larger ones fall through to realValue and stay exact below 2^53
BEAST_EXPECT(
jsontx_exact(Json::Value(9007199254740991.0), out) &&
out == 9007199254740991LL);
BEAST_EXPECT(
jsontx_exact(Json::Value(-9007199254740991.0), out) &&
out == -9007199254740991LL);
// at and past 2^53 distinct decimal integers share a double
BEAST_EXPECT(!jsontx_exact(Json::Value(9007199254740992.0), out));
BEAST_EXPECT(!jsontx_exact(Json::Value(-9007199254740992.0), out));
// fractions, infinities and non-numbers are not integers
BEAST_EXPECT(!jsontx_exact(Json::Value(1.5), out));
BEAST_EXPECT(!jsontx_exact(
Json::Value(std::numeric_limits<double>::infinity()), out));
BEAST_EXPECT(!jsontx_exact(Json::Value(true), out));
BEAST_EXPECT(!jsontx_exact(Json::Value("7"), out));
// jsontx_num renders what jsontx_exact accepts
BEAST_EXPECT(jsontx_num(Json::Value(0)) == "0");
BEAST_EXPECT(jsontx_num(Json::Value(1)) == "1");
BEAST_EXPECT(jsontx_num(Json::Value(-1)) == "-1");
BEAST_EXPECT(jsontx_num(Json::Value(42)) == "42");
BEAST_EXPECT(jsontx_num(Json::Value(1000000)) == "1000000");
BEAST_EXPECT(
jsontx_num(Json::Value(9007199254740991.0)) == "9007199254740991");
BEAST_EXPECT(
threw([] { (void)jsontx_num(Json::Value(9007199254740992.0)); }));
BEAST_EXPECT(threw([] { (void)jsontx_num(Json::Value(1.5)); }));
// jsontx_u64 additionally refuses negatives rather than wrapping them
BEAST_EXPECT(jsontx_u64(Json::Value(0)) == 0);
BEAST_EXPECT(jsontx_u64(Json::Value(100)) == 100);
BEAST_EXPECT(threw([] { (void)jsontx_u64(Json::Value(-1)); }));
BEAST_EXPECT(threw([] { (void)jsontx_u64(Json::Value(1.5)); }));
BEAST_EXPECT(threw([] {
(void)jsontx_u64(
Json::Value(std::numeric_limits<double>::infinity()));
}));
// STUInt64::getJson renders hex, unpadded and lowercase, except for
// the sMD_BaseTen fields which render base ten
BEAST_EXPECT(jsontx_u64_str(sfTime, 0) == "0");
BEAST_EXPECT(jsontx_u64_str(sfTime, 255) == "ff");
BEAST_EXPECT(jsontx_u64_str(sfTime, 1000) == "3e8");
BEAST_EXPECT(jsontx_u64_str(sfMaximumAmount, 255) == "255");
BEAST_EXPECT(jsontx_u64_str(sfMaximumAmount, 1000) == "1000");
}
void
testFieldLookup()
{
testcase("jsontx_field");
auto const* account = &jsontx_field("Account");
BEAST_EXPECT(account->fieldName == "Account");
BEAST_EXPECT(account->fieldCode > 0);
// the lookup is case-insensitive and always yields the one canonical
// SField, which is what makes the canonical spelling well defined
BEAST_EXPECT(&jsontx_field("account") == account);
BEAST_EXPECT(&jsontx_field("ACCOUNT") == account);
BEAST_EXPECT(&jsontx_field("AcCoUnT") == account);
BEAST_EXPECT(jsontx_field("Fee").fieldName == "Fee");
BEAST_EXPECT(
jsontx_field("TransactionType").fieldName == "TransactionType");
BEAST_EXPECT(jsontx_field("Time") == sfTime);
BEAST_EXPECT(jsontx_field("JsonTxDelta") == sfJsonTxDelta);
// anything unknown is sfInvalid, which sanitize_jsontx turns into a
// rejection rather than a silently dropped member
BEAST_EXPECT(jsontx_field("NoSuchField") == sfInvalid);
BEAST_EXPECT(jsontx_field("") == sfInvalid);
}
void
testBounds()
{
testcase("consensus bounds");
// These are consensus rules from the moment featureJsonTx activates,
// so pin them: changing one changes which deltas verify.
BEAST_EXPECT(jsontx_max_text == 8192);
BEAST_EXPECT(jsontx_max_diff == 2048);
BEAST_EXPECT(jsontx_max_ops == 1024);
BEAST_EXPECT(jsontx_min_copy == 4);
BEAST_EXPECT(jsontx_max_cand == 64);
BEAST_EXPECT(jsontx_epoch_day == 10957);
// sanitize_jsontx applies the delta caps itself, so a document it
// accepts is never one unsanitize_jsontx would then refuse. This is
// reached well before the text cap: indentation costs delta bytes
// that the canonical form does not pay.
auto indented = [](int memos) {
std::string raw =
"{\n \"Account\": \"rTest\",\n \"Memos\": [";
for (int i = 0; i < memos; ++i)
raw += std::string(i ? "," : "") +
"\n {\n"
" \"Memo\": {\n"
" \"MemoData\": \"AA\"\n"
" }\n"
" }";
raw += "\n ]\n}";
return raw;
};
BEAST_EXPECT(roundTrips(indented(20)));
auto const tooFormatted = indented(40);
BEAST_EXPECT(tooFormatted.size() < jsontx_max_text);
BEAST_EXPECT(threw([&] { (void)sanitize_jsontx(tooFormatted); }));
}
public:
void
run() override
{
testStrictAccepts();
testStrictRejects();
testIsoRoundTrip();
testIsoRejects();
testSanitizeRoundTrip();
testCanonicalForm();
testSanitizeRejects();
testDeltaRejection();
testNumbers();
testFieldLookup();
testBounds();
}
};
BEAST_DEFINE_TESTSUITE(JSONTxSignatures, protocol, ripple);
} // namespace ripple

View File

@@ -37,6 +37,7 @@
#include <xrpl/json/to_string.h>
#include <xrpl/protocol/Feature.h>
#include <xrpl/protocol/Indexes.h>
#include <xrpl/protocol/JSONTxSignatures.h>
#include <xrpl/protocol/Protocol.h>
#include <xrpl/protocol/STAccount.h>
#include <xrpl/protocol/UintTypes.h>
@@ -102,6 +103,22 @@ preflight1(PreflightContext const& ctx)
return temMALFORMED;
}
// sfTime and sfJsonTxDelta are common fields, so every transaction type
// can carry them the moment this binary ships. A node on an older build
// cannot parse them, so accepting one into a ledger before the amendment
// activates would split consensus.
if (ctx.tx.isFieldPresent(sfTime) || ctx.tx.isFieldPresent(sfJsonTxDelta))
{
if (!ctx.rules.enabled(featureJsonTx))
return temDISABLED;
// unsanitize_jsontx refuses anything larger, so a bigger delta is
// only ever ledger weight that can never verify
if (ctx.tx.isFieldPresent(sfJsonTxDelta) &&
ctx.tx.getFieldVL(sfJsonTxDelta).size() > jsontx_max_diff)
return temMALFORMED;
}
auto const ret = preflight0(ctx);
if (!isTesSuccess(ret))
return ret;

View File

@@ -23,6 +23,7 @@
#include <xrpld/app/tx/applySteps.h>
#include <xrpl/basics/Log.h>
#include <xrpl/protocol/Feature.h>
#include <xrpl/protocol/JSONTxSignatures.h>
namespace ripple {
@@ -97,6 +98,44 @@ checkValidity(
return {Validity::Valid, ""};
}
if (rules.enabled(featureJsonTx) && tx.isFieldPresent(sfJsonTxDelta))
{
// A JsonTx signs a plaintext preimage, so STTx::checkSign below -- a
// check against the binary signing hash -- can never succeed for it.
// The delta cannot be stripped to downgrade to a binary check either:
// that leaves a TxnSignature over text the binary hash does not match.
//
// This proves only that the key signed the preimage. Tying that key to
// sfAccount is still Transactor::checkSingleSign's job, exactly as for
// a binary-signed transaction.
// jsontx_verify canonicalizes twice and then verifies ed25519, and
// checkValidity runs on every relay, so honour the cache the binary
// path below uses rather than redoing that work per peer.
if (flags & SF_SIGBAD)
return {Validity::SigBad, "Transaction has bad signature."};
if (!(flags & SF_SIGGOOD))
{
try
{
(void)jsontx_verify(tx);
}
catch (std::exception const& e)
{
router.setFlags(id, SF_SIGBAD);
return {Validity::SigBad, e.what()};
}
router.setFlags(id, SF_SIGGOOD);
}
std::string reason;
if (!passesLocalChecks(tx, reason))
return {Validity::SigGoodOnly, reason};
return {Validity::Valid, ""};
}
if (flags & SF_SIGBAD)
// Signature is known bad
return {Validity::SigBad, "Transaction has bad signature."};

View File

@@ -30,9 +30,15 @@
#include <xrpld/rpc/detail/RPCHelpers.h>
#include <xrpld/rpc/detail/TransactionSign.h>
#include <xrpl/basics/strHex.h>
#include <xrpl/json/json_reader.h>
#include <xrpl/json/json_writer.h>
#include <xrpl/protocol/ErrorCodes.h>
#include <xrpl/protocol/Feature.h>
#include <xrpl/protocol/JSONTxSignatures.h>
#include <xrpl/protocol/PublicKey.h>
#include <xrpl/protocol/RPCErr.h>
#include <xrpl/protocol/SField.h>
#include <xrpl/protocol/STParsedJSON.h>
#include <xrpl/resource/Fees.h>
namespace ripple {
@@ -88,8 +94,10 @@ doInject(RPC::JsonContext& context)
}
// {
// tx_blob: <string> XOR tx_json: <object>,
// secret: <secret>
// tx_blob: <string>
// OR tx: <json text> together with sig: <hex>
// OR manifest: <hex>
// OR tx_json: <object> together with secret: <secret> (deprecated)
// }
Json::Value
doSubmit(RPC::JsonContext& context)
@@ -98,17 +106,37 @@ doSubmit(RPC::JsonContext& context)
context.loadType = Resource::feeMediumBurdenRPC;
auto const view = context.app.openLedger().current();
bool const isJsonTx =
context.params.isMember(jss::tx) && context.params.isMember(jss::sig);
bool const hasManifest = context.params.isMember(jss::manifest);
bool const hasTxBlob = context.params.isMember(jss::tx_blob);
if (hasManifest && hasTxBlob)
{
// Both of these carry authority that only their amendment teaches the
// network to honour, so without the amendment the submitter is told their
// transaction is unsigned, which reads as their mistake. It isn't -- the
// feature is not live yet -- so say so before touching the payload at all.
if (isJsonTx && !view->rules().enabled(featureJsonTx))
return RPC::make_error(
rpcINVALID_PARAMS,
"Specify exactly one of either `tx_blob` or `manifest`");
}
else if (!hasTxBlob && !hasManifest)
rpcNOT_ENABLED,
"The JsonTx amendment is not enabled on this network. "
"Plaintext-JSON submission will work once it activates; nothing "
"is wrong with this request.");
if (hasManifest && !view->rules().enabled(featureOnChainManifests))
return RPC::make_error(
rpcNOT_ENABLED,
"The OnChainManifests amendment is not enabled on this "
"network. Manifest submission will work once it activates; "
"nothing is wrong with this request.");
int const count =
(hasTxBlob ? 1 : 0) + (isJsonTx ? 1 : 0) + (hasManifest ? 1 : 0);
if (!count)
{
// legacy signing code
auto const failType = getFailHard(context);
if (context.role != Role::ADMIN && !context.app.config().canSign())
@@ -132,30 +160,22 @@ doSubmit(RPC::JsonContext& context)
return ret;
}
else if (count != 1)
{
return RPC::make_error(
rpcINVALID_PARAMS,
"Specify exactly one of `tx_blob`, `manifest`, or `tx` together "
"with `sig`");
}
// execution to here means exactly one of isJsonTx, hasManifest or
// hasTxBlob is true
std::string txBlob =
hasTxBlob ? context.params[jss::tx_blob].asString() : "";
if (hasManifest)
{
// OnChainManifests amendment accepts a manifest submission here; turn
// it into the transaction that carries it and drop through to normal
// tx_blob processing below.
auto const view = context.app.openLedger().current();
// The transaction built below carries no account signature; its
// authority is the manifest's own master and ephemeral signatures,
// which checkValidity() only honours once the amendment is active.
// Without this the submitter is told their transaction is unsigned,
// which reads as their mistake. It isn't -- the feature is not live
// yet -- so say so before touching the manifest at all.
if (!view->rules().enabled(featureOnChainManifests))
return RPC::make_error(
rpcNOT_ENABLED,
"The OnChainManifests amendment is not enabled on this "
"network. Manifest submission will work once it activates; "
"nothing is wrong with this request.");
auto const raw = strUnHex(context.params[jss::manifest].asString());
if (!raw || raw->empty())
return rpcError(rpcINVALID_PARAMS);
@@ -175,18 +195,84 @@ doSubmit(RPC::JsonContext& context)
txBlob = *hex;
}
auto ret = strUnHex(txBlob);
std::optional<Blob> ret;
if (!ret || !ret->size())
return rpcError(rpcINVALID_PARAMS);
if (!isJsonTx)
{
ret = strUnHex(txBlob);
SerialIter sitTrans(makeSlice(*ret));
if (!ret || ret->empty())
return rpcError(rpcINVALID_PARAMS);
}
std::shared_ptr<STTx const> stTx;
try
{
stTx = std::make_shared<STTx const>(std::ref(sitTrans));
if (!isJsonTx)
{
SerialIter sitTrans(makeSlice(*ret));
stTx = std::make_shared<STTx const>(std::ref(sitTrans));
}
else
{
if (!context.params.isMember(jss::sig))
throw std::runtime_error("JsonTx: missing sig parameter");
std::string const raw = context.params[jss::tx].asString();
auto const [san, diff] = sanitize_jsontx(raw);
auto const sig = strUnHex(context.params[jss::sig].asString());
if (!sig || sig->empty())
throw std::runtime_error("JsonTx: bad signature");
Json::Value jv;
if (Json::Reader r; !r.parse(san, jv))
throw std::runtime_error("JsonTx: unparsable canonical form");
// The preimage carries the key but not the signature over itself.
for (auto const& n :
{sfTxnSignature.fieldName, sfSigners.fieldName})
if (jv.isMember(n))
throw std::runtime_error(
"JsonTx: " + n + " must not appear in tx");
if (!jv.isMember(sfSigningPubKey.fieldName))
throw std::runtime_error("JsonTx: tx must carry SigningPubKey");
// Hand the parser the u64 rather than teaching STUInt64 a second
// spelling; the ISO form only ever exists in the preimage.
std::optional<std::uint64_t> ms;
if (jv.isMember(sfTime.fieldName))
{
ms = jsontx_iso(jv[sfTime.fieldName].asString());
jv.removeMember(sfTime.fieldName);
}
STParsedJSONObject parsed("tx_json", jv);
if (!parsed.object)
throw std::runtime_error(
parsed.error[jss::error_message].asString());
if (ms)
parsed.object->setFieldU64(sfTime, *ms);
parsed.object->setFieldVL(sfTxnSignature, *sig);
// The delta rides along in the transaction. Without it a relaying
// node has nothing to reconstruct the preimage from, the binary
// TxnSignature check fails there, and the transaction never
// propagates past this node.
parsed.object->setFieldVL(sfJsonTxDelta, makeSlice(diff));
stTx = std::make_shared<STTx const>(std::move(*parsed.object));
// Round-trip the binary codec and run the check a relaying node
// will run, so this path cannot accept anything the network would
// later reject -- and so the caller gets the real reason rather
// than a bare "fails local checks" from checkValidity below.
Serializer ser;
stTx->add(ser);
SerialIter si(ser.slice());
STTx const rt{si};
if (jsontx_verify(rt) != raw)
throw std::runtime_error("JsonTx: does not round-trip");
}
}
catch (std::exception& e)
{