Compare commits

..

22 Commits

Author SHA1 Message Date
Nicholas Dudfield
10b901ea0b style: clang-format for external-env-tests Logs/HooksTrace changes
CI clang-format (18) flagged formatting drift in the env-tests
logging path; apply the generated patch.
2026-07-30 11:33:51 +07:00
Nicholas Dudfield
1025952cb6 build(test): force canonical hook recompilation 2026-07-30 07:12:18 +07:00
Nicholas Dudfield
ddd3c28d8b Merge remote-tracking branch 'origin/dev' into external-env-tests-wd
# Conflicts:
#	src/test/app/AMMClawback_test.cpp
#	src/test/app/AMM_test.cpp
#	src/test/jtx/AMMTest.h
#	src/test/rpc/AMMInfo_test.cpp
#	src/xrpld/app/hook/applyHook.h
#	src/xrpld/app/hook/detail/applyHook.cpp
#	src/xrpld/app/tx/detail/AMMClawback.cpp
#	src/xrpld/app/tx/detail/InvariantCheck.cpp
#	src/xrpld/app/tx/detail/InvariantCheck.h
2026-07-05 13:45:46 +07:00
Nicholas Dudfield
8e2c69deb2 Scope hook test include path to external sources 2026-04-01 12:29:25 +07:00
Nicholas Dudfield
ff763a500c feat: log transform for test output (r-address → Account(name), prefix)
- Log.h: add setTransform/applyTransform on Logs for message rewriting
- SuiteJournal.h: wire transform into SuiteJournalSink so test output
  goes through it (previously bypassed Logs::write entirely)
- Env.h: pass Logs* to SuiteJournalSink
- TestEnv.h: add setPrefix() for per-phase log labels, prepend prefix
  in transform

usage:
  auto env = makeEnv(features);
  auto const& alice = env.account("alice");
  env.setPrefix("deposit phase");
  // logs: TRC:HooksTrace [deposit phase] HookTrace[Account(alice)-...]: ...

  TESTENV_LOGGING="HooksTrace=trace,View=debug"
2026-03-31 17:45:44 +07:00
Nicholas Dudfield
a605aec57a chore: remove unused SuiteLogsWithOverrides.h 2026-03-31 16:44:26 +07:00
Nicholas Dudfield
bfcbbc3c5e feat: migrate coverage from sancov to hookz __on_source_line
replace sancov-based coverage instrumentation with hookz's DWARF-based
__on_source_line(line, col) approach. line/col arrive as direct arguments
so no post-processing symbolication step is needed.

- Guard.h: allow result_count == 0 for void-returning whitelisted imports
- Enum.h: replace sancov whitelist entries with __on_source_line
- applyHook.h: replace sancov callbacks with onSourceLine, emit line:col
- SetHook.cpp: re-enable guard validation (was disabled for sancov testing)
- CMake: use hookz build-test-hooks, add HOOKS_FORCE_RECOMPILE support
- remove obsolete HookCoverage sancov test files
2026-03-31 16:38:28 +07:00
Nicholas Dudfield
d782f8cab4 feat: snapshot cmake change 2026-03-31 13:11:17 +07:00
Nicholas Dudfield
8a61dd44e0 feat: chorse: 2026-03-27 22:54:03 +07:00
Nicholas Dudfield
a8ca62a148 feat: add TestEnv with named accounts, log transform, and env-var logging
TestEnv wraps Env with:
- account("name"): auto-registers r-address → Account(name) in logs
- TESTENV_LOGGING env var: "HooksTrace=trace,View=debug" sets
  per-partition log levels without code changes
2026-03-27 22:19:13 +07:00
Nicholas Dudfield
b7aeff95a9 feat: add log transform to Logs for test-time message rewriting
Logs::setTransform(fn) installs a function that transforms every log
message before output. Useful in tests to replace raw r-addresses
with human-readable account names.

Usage:
  env.app().logs().setTransform([&](std::string const& text) {
      std::string out = text;
      // replace rG1QQv2... with Account(alice)
      boost::algorithm::replace_all(out, toBase58(alice.id()), "Account(alice)");
      return out;
  });
  // Pass nullptr to clear:
  env.app().logs().setTransform(nullptr);
2026-03-27 21:59:31 +07:00
tequ
b880c80c2b Fix BEAST_ENHANCED_LOGGING not working and restore original behavior 2026-03-27 21:27:38 +07:00
Nicholas Dudfield
8666cdfb71 fix: remove stdout duplicate from StderrJournalSink 2026-03-27 20:53:22 +07:00
Nicholas Dudfield
6d2a0b4e8b feat: also write overridden journal output to stdout with prefix 2026-03-27 20:43:47 +07:00
Nicholas Dudfield
739ebfaba4 rename: HooksApi journal → HooksTrace 2026-03-27 20:28:38 +07:00
Nicholas Dudfield
65166a9329 feat: route hook trace output to dedicated HooksApi journal
- Macro.h: add `jh` journal for HooksApi partition in HOOK_SETUP()
- applyHook.cpp: trace, trace_num, trace_float now use jh + JLOG macro
  for line numbers and separate partition filtering
- SuiteLogsWithOverrides.h: per-partition severity overrides for tests

Usage in tests:
  Env env{*this, envconfig(), features,
      std::make_unique<SuiteLogsWithOverrides>(*this,
          SuiteLogsWithOverrides::Overrides{{"HooksApi", Sev::kTrace}})};
2026-03-27 20:10:56 +07:00
Nicholas Dudfield
ca469b5d22 feat: wasm hook coverage instrumentation support
- Enum.h: add sancov callbacks to import whitelist with void_t return
- applyHook.h: sancov host callbacks (trace guard + init), global
  coverage accumulator with label support, coverageReset/Hits/Dump API
- SetHook.cpp: bypass guard validation for coverage-instrumented hooks
- RippledCore.cmake: HOOKS_TEST_DIR, HOOKS_C_DIR, HOOKS_COVERAGE,
  HOOKS_TEST_ONLY env vars for external hook test compilation
2026-03-27 19:32:43 +07:00
tequ
8cfee6c8a3 Merge fixAMMClawbackRounding amendment into featureAMMClawback amendment 2026-02-25 19:07:45 +10:00
yinyiqian1
8673599d2b fixAMMClawbackRounding: adjust last holder's LPToken balance (#5513)
Due to rounding, the LPTokenBalance of the last LP might not match the LP's trustline balance. This was fixed for `AMMWithdraw` in `fixAMMv1_1` by adjusting the LPTokenBalance to be the same as the trustline balance. Since `AMMClawback` is also performing a withdrawal, we need to adjust LPTokenBalance as well in `AMMClawback.`

This change includes:
1. Refactored `verifyAndAdjustLPTokenBalance` function in `AMMUtils`, which both`AMMWithdraw` and `AMMClawback` call to adjust LPTokenBalance.
2. Added the unit test `testLastHolderLPTokenBalance` to test the scenario.
3. Modify the existing unit tests for `fixAMMClawbackRounding`.
2026-02-25 19:07:45 +10:00
tequ
ec65e622aa Merge fixAMMv1_3 amendment into featureAMM amendment 2026-02-25 16:20:43 +10:00
Gregory Tsipenyuk
65837f49e1 fix: Add AMMv1_3 amendment (#5203)
* Add AMM bid/create/deposit/swap/withdraw/vote invariants:
  - Deposit, Withdrawal invariants: `sqrt(asset1Balance * asset2Balance) >= LPTokens`.
  - Bid: `sqrt(asset1Balance * asset2Balance) > LPTokens` and the pool balances don't change.
  - Create: `sqrt(asset1Balance * assetBalance2) == LPTokens`.
  - Swap: `asset1BalanceAfter * asset2BalanceAfter >= asset1BalanceBefore * asset2BalanceBefore`
     and `LPTokens` don't change.
  - Vote: `LPTokens` and pool balances don't change.
  - All AMM and swap transactions: amounts and tokens are greater than zero, except on withdrawal if all tokens
    are withdrawn.
* Add AMM deposit and withdraw rounding to ensure AMM invariant:
  - On deposit, tokens out are rounded downward and deposit amount is rounded upward.
  - On withdrawal, tokens in are rounded upward and withdrawal amount is rounded downward.
* Add Order Book Offer invariant to verify consumed amounts. Consumed amounts are less than the offer.
* Fix Bid validation. `AuthAccount` can't have duplicate accounts or the submitter account.
2026-02-25 16:20:43 +10:00
RichardAH
e5b21f026e Merge pull request #692 from Xahau/sync-2.4.0-rebased
Sync: Ripple(d) 2.4.0
2026-02-24 16:07:09 +10:00
38 changed files with 590 additions and 1374 deletions

View File

@@ -179,7 +179,121 @@ if(xrpld)
file(GLOB_RECURSE sources CONFIGURE_DEPENDS
"${CMAKE_CURRENT_SOURCE_DIR}/src/test/*.cpp"
)
if(HOOKS_TEST_ONLY OR DEFINED ENV{HOOKS_TEST_ONLY})
# Keep test infra but drop the individual *_test.cpp files
list(FILTER sources EXCLUDE REGEX "_test\\.cpp$")
message(STATUS "HOOKS_TEST_ONLY: excluded *_test.cpp from src/test/")
endif()
target_sources(rippled PRIVATE ${sources})
# Optional: include external hook test sources from another directory.
# Set via -DHOOKS_TEST_DIR=/path/to/tests or env HOOKS_TEST_DIR.
# Optionally set HOOKS_C_DIR to pass --hooks-c-dir args to the compiler
# (e.g. "tipbot=/path/to/hooks" — multiple values separated by ";").
#
# hookz build-test-hooks must be on PATH. It auto-compiles hooks referenced
# in each *_test.cpp and generates *_test_hooks.h next to the test file.
if(NOT HOOKS_TEST_DIR AND DEFINED ENV{HOOKS_TEST_DIR})
set(HOOKS_TEST_DIR $ENV{HOOKS_TEST_DIR})
endif()
if(NOT HOOKS_C_DIR AND DEFINED ENV{HOOKS_C_DIR})
set(HOOKS_C_DIR $ENV{HOOKS_C_DIR})
endif()
if(HOOKS_TEST_DIR AND EXISTS "${HOOKS_TEST_DIR}")
file(GLOB EXTERNAL_HOOK_TESTS CONFIGURE_DEPENDS
"${HOOKS_TEST_DIR}/*_test.cpp"
)
if(EXTERNAL_HOOK_TESTS)
# Build extra args for hookz build-test-hooks
set(_hooks_extra_args "")
set(_hooks_source_deps "")
if(HOOKS_C_DIR)
foreach(_dir ${HOOKS_C_DIR})
list(APPEND _hooks_extra_args "--hooks-c-dir" "${_dir}")
string(REGEX REPLACE "^[^=]+=" "" _hook_dir "${_dir}")
if(EXISTS "${_hook_dir}")
file(GLOB_RECURSE _hook_dir_deps CONFIGURE_DEPENDS
"${_hook_dir}/*.c"
"${_hook_dir}/*.h"
)
if(HOOKS_TEST_DIR)
list(FILTER _hook_dir_deps EXCLUDE REGEX "^${HOOKS_TEST_DIR}/")
endif()
list(APPEND _hooks_source_deps ${_hook_dir_deps})
endif()
endforeach()
list(REMOVE_DUPLICATES _hooks_source_deps)
endif()
if(HOOKS_COVERAGE OR DEFINED ENV{HOOKS_COVERAGE})
list(APPEND _hooks_extra_args "--hook-coverage")
message(STATUS "Hook coverage enabled: compiling hooks with hookz")
endif()
set(_hooks_always_run OFF)
if(HOOKS_FORCE_RECOMPILE OR DEFINED ENV{HOOKS_FORCE_RECOMPILE})
list(APPEND _hooks_extra_args "--force-write" "--no-cache")
set(_hooks_always_run ON)
message(STATUS "Hook force recompile enabled (cache bypassed)")
endif()
if(HOOKZ_BUILDBOX OR "$ENV{HOOKZ_BUILDBOX}" STREQUAL "1")
# Remote evidence must observe the service on every build. An
# OUTPUT-cached header may otherwise preserve locally built WASM
# after the compiler mode changes.
list(APPEND _hooks_extra_args "--buildbox")
if(NOT _hooks_always_run)
list(APPEND _hooks_extra_args "--force-write")
endif()
set(_hooks_always_run ON)
message(STATUS "Canonical buildbox enabled (always recompiled)")
endif()
# Run hookz build-test-hooks on each test file before compilation
foreach(_test_file ${EXTERNAL_HOOK_TESTS})
get_filename_component(_stem ${_test_file} NAME_WE)
set(_hooks_header "${HOOKS_TEST_DIR}/${_stem}_hooks.h")
if(_hooks_always_run)
# Always run — no DEPENDS, no OUTPUT caching
add_custom_target(compile_hooks_${_stem} ALL
COMMAND hookz build-test-hooks "${_test_file}" ${_hooks_extra_args}
WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}"
COMMENT "Compiling hooks for ${_stem} (forced)"
VERBATIM
)
list(APPEND EXTERNAL_HOOK_TARGETS compile_hooks_${_stem})
else()
add_custom_command(
OUTPUT "${_hooks_header}"
COMMAND hookz build-test-hooks "${_test_file}" ${_hooks_extra_args}
DEPENDS "${_test_file}" ${_hooks_source_deps}
WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}"
COMMENT "Compiling hooks for ${_stem}"
VERBATIM
)
list(APPEND EXTERNAL_HOOK_HEADERS "${_hooks_header}")
endif()
endforeach()
# Ensure headers are generated before rippled compiles
if(_hooks_always_run)
foreach(_tgt ${EXTERNAL_HOOK_TARGETS})
add_dependencies(rippled ${_tgt})
endforeach()
else()
add_custom_target(compile_external_hooks DEPENDS ${EXTERNAL_HOOK_HEADERS})
add_dependencies(rippled compile_external_hooks)
endif()
target_sources(rippled PRIVATE ${EXTERNAL_HOOK_TESTS})
# Keep the generated hook-header include path scoped to the external
# test sources so changing HOOKS_TEST_DIR doesn't invalidate the
# compile command for the rest of rippled.
set_property(
SOURCE ${EXTERNAL_HOOK_TESTS}
APPEND PROPERTY INCLUDE_DIRECTORIES "${HOOKS_TEST_DIR}"
)
message(STATUS "Including external hook tests from: ${HOOKS_TEST_DIR}")
endif()
endif()
endif()
target_link_libraries(rippled

View File

@@ -25,6 +25,7 @@
#include <boost/beast/core/string.hpp>
#include <boost/filesystem.hpp>
#include <fstream>
#include <functional>
#include <map>
#include <memory>
#include <mutex>
@@ -165,6 +166,7 @@ private:
beast::severities::Severity thresh_;
File file_;
bool silent_ = false;
std::function<std::string(std::string const&)> transform_;
public:
Logs(beast::severities::Severity level);
@@ -203,6 +205,33 @@ public:
std::string const& text,
bool console);
/** Set a transform applied to every log message before output.
* Useful in tests to replace raw account IDs with human-readable names.
* Pass nullptr to clear.
*
* TODO: This is test-only infrastructure (used by TestEnv). Consider
* moving to SuiteLogs or a test-specific subclass if the Logs interface
* needs to stay clean for production.
*/
void
setTransform(std::function<std::string(std::string const&)> fn)
{
std::lock_guard lock(mutex_);
transform_ = std::move(fn);
}
/** Apply the current transform to text (or return as-is if none set). */
std::string const&
applyTransform(std::string const& text) const
{
if (!transform_)
return text;
// Store in thread_local to return a const ref
thread_local std::string buf;
buf = transform_(text);
return buf;
}
std::string
rotate();

View File

@@ -417,6 +417,7 @@ getImportWhitelist(Rules const& rules)
#define int64_t 0x7EU
#define int32_t 0x7FU
#define uint32_t 0x7FU
#define void_t 0x00U
#define HOOK_WRAP_PARAMS(...) __VA_ARGS__
@@ -428,11 +429,15 @@ getImportWhitelist(Rules const& rules)
#include "hook_api.macro"
// Coverage callback: void __on_source_line(uint32_t line, uint32_t col)
whitelist["__on_source_line"] = {void_t, uint32_t, uint32_t};
#undef HOOK_API_DEFINITION
#undef HOOK_WRAP_PARAMS
#undef int64_t
#undef int32_t
#undef uint32_t
#undef void_t
#pragma pop_macro("HOOK_API_DEFINITION")
return whitelist;

View File

@@ -1383,21 +1383,51 @@ validateGuards(
int result_count = parseLeb128(wasm, i, &i);
CHECK_SHORT_HOOK();
// this needs a reliable hook cleaner otherwise it will catch
// most compilers out
if (result_count != 1)
if (j == hook_type_idx)
{
GUARDLOG(hook::log::FUNC_RETURN_COUNT)
<< "Malformed transaction. "
<< "Hook declares a function type that returns fewer "
"or more than one value. "
<< "\n";
return {};
// hook/cbak must return exactly one value (i64)
if (result_count != 1)
{
GUARDLOG(hook::log::FUNC_RETURN_COUNT)
<< "Malformed transaction. "
<< "hook/cbak function type must return exactly "
"one value. "
<< "\n";
return {};
}
}
else if (first_signature)
{
// For whitelisted imports, check expected return count.
// void_t (0x00) means 0 return values.
uint8_t expected_return = (*first_signature).get()[0];
int expected_result_count =
(expected_return == 0x00U) ? 0 : 1;
if (result_count != expected_result_count)
{
GUARDLOG(hook::log::FUNC_RETURN_COUNT)
<< "Malformed transaction. "
<< "Hook API: " << *first_name
<< " has wrong return count "
<< "(expected " << expected_result_count << ", got "
<< result_count << ")."
<< "\n";
return {};
}
}
else
{
if (result_count != 1)
{
GUARDLOG(hook::log::FUNC_RETURN_COUNT)
<< "Malformed transaction. "
<< "Hook declares a function type that returns "
"fewer or more than one value. "
<< "\n";
return {};
}
}
// this can only ever be 1 in production, but in testing it may
// also be 0 or >1 so for completeness this loop is here but can
// be taken out in prod
for (int k = 0; k < result_count; ++k)
{
int result_type = parseLeb128(wasm, i, &i);

View File

@@ -150,22 +150,23 @@
WasmEdge_CallingFrameContext const& frameCtx __VA_OPT__( \
COMMA __VA_ARGS__))
#define HOOK_SETUP() \
using enum hook_api::hook_return_code; \
try \
{ \
[[maybe_unused]] ApplyContext& applyCtx = hookCtx.applyCtx; \
[[maybe_unused]] auto& view = applyCtx.view(); \
[[maybe_unused]] auto j = applyCtx.app.journal("View"); \
[[maybe_unused]] WasmEdge_MemoryInstanceContext* memoryCtx = \
WasmEdge_CallingFrameGetMemoryInstance(&frameCtx, 0); \
[[maybe_unused]] unsigned char* memory = \
WasmEdge_MemoryInstanceGetPointer(memoryCtx, 0, 0); \
[[maybe_unused]] const uint64_t memory_length = \
WasmEdge_MemoryInstanceGetPageSize(memoryCtx) * \
WasmEdge_kPageSize; \
[[maybe_unused]] auto& api = hookCtx.api(); \
if (!memoryCtx || !memory || !memory_length) \
#define HOOK_SETUP() \
using enum hook_api::hook_return_code; \
try \
{ \
[[maybe_unused]] ApplyContext& applyCtx = hookCtx.applyCtx; \
[[maybe_unused]] auto& view = applyCtx.view(); \
[[maybe_unused]] auto j = applyCtx.app.journal("View"); \
[[maybe_unused]] auto jh = applyCtx.app.journal("HooksTrace"); \
[[maybe_unused]] WasmEdge_MemoryInstanceContext* memoryCtx = \
WasmEdge_CallingFrameGetMemoryInstance(&frameCtx, 0); \
[[maybe_unused]] unsigned char* memory = \
WasmEdge_MemoryInstanceGetPointer(memoryCtx, 0, 0); \
[[maybe_unused]] const uint64_t memory_length = \
WasmEdge_MemoryInstanceGetPageSize(memoryCtx) * \
WasmEdge_kPageSize; \
[[maybe_unused]] auto& api = hookCtx.api(); \
if (!memoryCtx || !memory || !memory_length) \
return INTERNAL_ERROR;
#define HOOK_TEARDOWN() \

View File

@@ -1,100 +0,0 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2025 XRPL Labs
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_PROTOCOL_APPLOADER_H_INCLUDED
#define RIPPLE_PROTOCOL_APPLOADER_H_INCLUDED
#include <cstddef>
#include <cstdint>
namespace ripple {
/** Validation of the AccountRoot `AppLoader` blob (sfAppLoader).
The blob carries the bootstrap document of a Progressive Web App that is
served from the account itself. Consensus cannot afford to run a real
HTML parser, so enforcement here is deliberately *shallow*: it rejects
payloads that plainly are not a UTF-8 HTML document, and nothing more.
It is a sanity gate, not a conformance checker -- a blob that passes is
not thereby guaranteed to render, and clients must still treat the
contents as wholly untrusted input.
A conforming blob must:
- be non-empty and no longer than `maxAppLoaderLength` bytes;
- be well-formed UTF-8 (no overlongs, surrogates, or scalar values
above U+10FFFF);
- contain no C0 control characters other than TAB, LF and CR, and no
NUL or DEL;
- begin (after an optional BOM and leading whitespace) with either an
HTML doctype or an `<html` start tag;
- contain an `<html` start tag and a matching `</html>` end tag, in
that order, with the end tag being the last non-whitespace content.
*/
namespace appLoader {
/** Reasons an AppLoader blob may be rejected.
Returned by `validate` so that callers can log something more useful
than a bare failure. Not part of the protocol: only `ok` vs. everything
else is consensus-relevant.
*/
enum class Result : std::uint8_t {
ok = 0,
empty, // zero-length blob
tooLarge, // exceeds maxAppLoaderLength
badUTF8, // malformed UTF-8 sequence
badControlChar, // disallowed control character
noDoctype, // does not open with a doctype or <html tag
noHtmlElement, // missing <html ...> start tag
unclosed, // missing </html> end tag, or it precedes <html
trailingGarbage // non-whitespace content after </html>
};
/** Validate a UTF-8 byte sequence as an AppLoader document.
@param data Pointer to the first byte; may be nullptr iff size is 0.
@param size Length of the blob in bytes.
@return Result::ok if the blob is acceptable, otherwise the first
problem encountered.
@note Total: every input yields a defined result, including inputs
whose final bytes form a truncated multi-byte sequence.
*/
Result
validate(std::uint8_t const* data, std::size_t size);
/** Human-readable description of a validation result, for logging. */
char const*
to_string(Result r);
/** Check that a byte sequence is well-formed UTF-8.
Rejects overlong encodings, UTF-16 surrogate halves (U+D800..U+DFFF),
and scalar values above U+10FFFF. Noncharacters such as U+FFFE and
U+FFFF are accepted: they are well-formed UTF-8 and are permitted in
interchange.
*/
bool
isValidUTF8(std::uint8_t const* data, std::size_t size);
} // namespace appLoader
} // namespace ripple
#endif

View File

@@ -328,14 +328,6 @@ xChainCreateAccountClaimID(STXChainBridge const& bridge, std::uint64_t seq);
Keylet
did(AccountID const& account) noexcept;
/** An AppLoader belonging to an account.
There is at most one per account, so the key is derived from the owner's
AccountID alone.
*/
Keylet
appLoader(AccountID const& account) noexcept;
Keylet
oracle(AccountID const& account, std::uint32_t const& documentID) noexcept;
@@ -410,7 +402,7 @@ struct keyletDesc
// This list should include all of the keylet functions that take a single
// AccountID parameter.
std::array<keyletDesc<AccountID const&>, 7> const directAccountKeylets{
std::array<keyletDesc<AccountID const&>, 6> const directAccountKeylets{
{{&keylet::account, jss::AccountRoot, false},
{&keylet::ownerDir, jss::DirectoryNode, true},
{&keylet::signers, jss::SignerList, true},
@@ -418,8 +410,7 @@ std::array<keyletDesc<AccountID const&>, 7> const directAccountKeylets{
// test it anyway, since the invariant checks for it.
{&keylet::nftpage_min, jss::NFTokenPage, true},
{&keylet::nftpage_max, jss::NFTokenPage, true},
{&keylet::did, jss::DID, true},
{&keylet::appLoader, jss::AppLoader, true}}};
{&keylet::did, jss::DID, true}}};
MPTID
makeMptID(std::uint32_t sequence, AccountID const& account);

View File

@@ -96,25 +96,6 @@ std::size_t constexpr maxDIDAttestationLength = 256;
/** The maximum length of a domain */
std::size_t constexpr maxDomainLength = 256;
/** The maximum length of an AppLoader document.
The AppLoader holds the bootstrap document of a Progressive Web App
served from an account. It is deliberately capped: the blob lives in a
ledger object that carries no owner reserve at all, so the only things
standing between it and unbounded ledger growth are this limit and the
per-byte transaction fee below.
*/
std::size_t constexpr maxAppLoaderLength = 4096;
/** Drops of transaction fee charged per byte of AppLoader document.
Storage is paid for through the fee rather than through an owner
reserve, so that a loader can be published by an account sitting at its
reserve floor. The fee is charged on every AccountSet that carries the
field, including overwrites.
*/
std::uint64_t constexpr appLoaderFeeDropsPerByte = 1;
/** The maximum length of a URI inside a Credential */
std::size_t constexpr maxCredentialURILength = 256;

View File

@@ -34,7 +34,6 @@
// If you add an amendment here, then do not forget to increment `numFeatures`
// in include/xrpl/protocol/Feature.h.
XRPL_FEATURE(PWALoader, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FIX (HookMap, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FIX (GuardDepth32, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(NamedHooks, Supported::yes, VoteBehavior::DefaultNo)

View File

@@ -223,29 +223,6 @@ LEDGER_ENTRY(ltURI_TOKEN, 0x0055, URIToken, uri_token, ({
{sfPreviousTxnLgrSeq, soeREQUIRED},
}))
/** A ledger object holding the bootstrap document of a Progressive Web App.
There is at most one of these per account, keyed on the owner's AccountID.
It is linked directly from the owning AccountRoot via sfAppLoaderID and is
deliberately NOT placed in the owner directory: it takes no owner reserve
and contributes nothing to OwnerCount, because its storage is paid for by
a per-byte fee on the AccountSet that publishes it.
Two consequences follow from staying out of the directory. It is invisible
to account_objects, so it must be reached via sfAppLoaderID (or by
recomputing the keylet). And DeleteAccount's directory walk cannot find
it, so DeleteAccount erases it explicitly; without that, account deletion
would orphan it and trip AccountRootsDeletedClean.
\sa keylet::appLoader
*/
LEDGER_ENTRY(ltAPP_LOADER, 'L', AppLoader, app_loader, ({
{sfOwner, soeREQUIRED},
{sfAppLoader, soeREQUIRED},
{sfPreviousTxnID, soeREQUIRED},
{sfPreviousTxnLgrSeq, soeREQUIRED},
}))
/** A ledger object which describes an account.
\sa keylet::account
@@ -285,7 +262,6 @@ LEDGER_ENTRY(ltACCOUNT_ROOT, 0x0061, AccountRoot, account, ({
{sfHookStateScale, soeOPTIONAL},
{sfCron, soeOPTIONAL},
{sfAMMID, soeOPTIONAL},
{sfAppLoaderID, soeOPTIONAL},
}))
/** A ledger object which contains a list of object identifiers.

View File

@@ -210,7 +210,6 @@ TYPED_SFIELD(sfOfferID, UINT256, 34)
TYPED_SFIELD(sfEscrowID, UINT256, 35)
TYPED_SFIELD(sfURITokenID, UINT256, 36)
TYPED_SFIELD(sfDomainID, UINT256, 37)
TYPED_SFIELD(sfAppLoaderID, UINT256, 92)
TYPED_SFIELD(sfHookOnOutgoing, UINT256, 93)
TYPED_SFIELD(sfHookOnIncoming, UINT256, 94)
TYPED_SFIELD(sfCron, UINT256, 95)
@@ -294,7 +293,6 @@ TYPED_SFIELD(sfAssetClass, VL, 29)
TYPED_SFIELD(sfProvider, VL, 30)
TYPED_SFIELD(sfMPTokenMetadata, VL, 31)
TYPED_SFIELD(sfCredentialType, VL, 32)
TYPED_SFIELD(sfAppLoader, VL, 96)
TYPED_SFIELD(sfHookName, VL, 97)
TYPED_SFIELD(sfRemarkValue, VL, 98)
TYPED_SFIELD(sfRemarkName, VL, 99)

View File

@@ -74,7 +74,6 @@ TRANSACTION(ttACCOUNT_SET, 3, AccountSet, ({
{sfTickSize, soeOPTIONAL},
{sfNFTokenMinter, soeOPTIONAL},
{sfHookStateScale, soeOPTIONAL},
{sfAppLoader, soeOPTIONAL},
}))
/** This transaction type cancels an existing escrow. */

View File

@@ -23,9 +23,6 @@
#include <xrpl/json/Output.h>
#include <xrpl/json/json_value.h>
#include <string>
#include <vector>
namespace ripple {
void
@@ -33,9 +30,7 @@ HTTPReply(
int nStatus,
std::string const& strMsg,
Json::Output const&,
beast::Journal j,
std::string const& contentType = "application/json; charset=UTF-8",
std::vector<std::string> const& extraHeaders = {});
beast::Journal j);
} // namespace ripple

View File

@@ -196,9 +196,10 @@ Logs::write(
std::string const& text,
bool console)
{
std::string s;
format(s, text, level, partition);
std::lock_guard lock(mutex_);
std::string const& transformed = transform_ ? transform_(text) : text;
std::string s;
format(s, transformed, level, partition);
file_.writeln(s);
if (!silent_)
std::cerr << s << '\n';

View File

@@ -1,286 +0,0 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2025 XRPL Labs
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <xrpl/protocol/AppLoader.h>
#include <xrpl/protocol/Protocol.h>
#include <string_view>
namespace ripple {
namespace appLoader {
namespace {
// HTML "ASCII whitespace" per the WHATWG spec: TAB, LF, FF, CR, SPACE.
inline bool
isSpace(std::uint8_t c) noexcept
{
return c == 0x09 || c == 0x0A || c == 0x0C || c == 0x0D || c == 0x20;
}
// Control characters we refuse to store. Everything in C0 that is not HTML
// whitespace, plus DEL. These have no business in a loader document and
// their presence is a strong smell of a non-HTML payload being smuggled
// through as "text".
inline bool
isForbiddenControl(std::uint8_t c) noexcept
{
if (isSpace(c))
return false;
return c < 0x20 || c == 0x7F;
}
inline std::uint8_t
asciiLower(std::uint8_t c) noexcept
{
return (c >= 'A' && c <= 'Z') ? static_cast<std::uint8_t>(c + 32) : c;
}
// True if `lit` (which must be lowercase ASCII) occurs at `pos`, compared
// case-insensitively. A match that would run past the end is no match.
bool
matchAt(
std::uint8_t const* data,
std::size_t size,
std::size_t pos,
std::string_view lit) noexcept
{
if (pos > size || size - pos < lit.size())
return false;
for (std::size_t i = 0; i < lit.size(); ++i)
if (asciiLower(data[pos + i]) != static_cast<std::uint8_t>(lit[i]))
return false;
return true;
}
// Index of the first case-insensitive occurrence of `lit` at or after
// `from`, or `size` if absent.
std::size_t
findFrom(
std::uint8_t const* data,
std::size_t size,
std::size_t from,
std::string_view lit) noexcept
{
if (lit.empty() || lit.size() > size)
return size;
for (std::size_t i = from; i + lit.size() <= size; ++i)
if (matchAt(data, size, i, lit))
return i;
return size;
}
// A tag name must be terminated by whitespace, '>' or '/', otherwise
// "<htmlish>" would be mistaken for "<html>".
inline bool
isTagNameEnd(std::uint8_t c) noexcept
{
return isSpace(c) || c == '>' || c == '/';
}
} // namespace
bool
isValidUTF8(std::uint8_t const* data, std::size_t size)
{
// Table-free decoder in the style of Markus Kuhn's utf8_check.c. Each
// branch establishes that the whole sequence is present before reading
// any continuation byte, so a truncated tail is simply invalid.
std::size_t i = 0;
while (i < size)
{
std::uint8_t const c0 = data[i];
if (c0 < 0x80)
{
// 0xxxxxxx
++i;
continue;
}
if ((c0 & 0xE0) == 0xC0)
{
// 110xxxxx 10xxxxxx
if (size - i < 2)
return false;
if ((data[i + 1] & 0xC0) != 0x80)
return false;
if ((c0 & 0xFE) == 0xC0) // overlong
return false;
i += 2;
continue;
}
if ((c0 & 0xF0) == 0xE0)
{
// 1110xxxx 10xxxxxx 10xxxxxx
if (size - i < 3)
return false;
std::uint8_t const c1 = data[i + 1];
if ((c1 & 0xC0) != 0x80 || (data[i + 2] & 0xC0) != 0x80)
return false;
if (c0 == 0xE0 && (c1 & 0xE0) == 0x80) // overlong
return false;
if (c0 == 0xED && (c1 & 0xE0) == 0xA0) // UTF-16 surrogate
return false;
i += 3;
continue;
}
if ((c0 & 0xF8) == 0xF0)
{
// 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
if (size - i < 4)
return false;
std::uint8_t const c1 = data[i + 1];
if ((c1 & 0xC0) != 0x80 || (data[i + 2] & 0xC0) != 0x80 ||
(data[i + 3] & 0xC0) != 0x80)
return false;
if (c0 == 0xF0 && (c1 & 0xF0) == 0x80) // overlong
return false;
if (c0 > 0xF4 || (c0 == 0xF4 && c1 > 0x8F)) // > U+10FFFF
return false;
i += 4;
continue;
}
// Stray continuation byte or 0xF8..0xFF.
return false;
}
return true;
}
Result
validate(std::uint8_t const* data, std::size_t size)
{
if (size == 0 || data == nullptr)
return Result::empty;
if (size > maxAppLoaderLength)
return Result::tooLarge;
if (!isValidUTF8(data, size))
return Result::badUTF8;
for (std::size_t i = 0; i < size; ++i)
if (isForbiddenControl(data[i]))
return Result::badControlChar;
// Skip an optional UTF-8 BOM, then any leading whitespace.
std::size_t begin = 0;
if (size >= 3 && data[0] == 0xEF && data[1] == 0xBB && data[2] == 0xBF)
begin = 3;
while (begin < size && isSpace(data[begin]))
++begin;
if (begin == size)
return Result::empty;
// The document must open with a doctype or with the <html> tag itself.
// We accept "<!doctype" followed by whitespace, because that is the
// only doctype form any browser treats as standards mode for HTML.
bool const opensWithDoctype = matchAt(data, size, begin, "<!doctype") &&
begin + 9 < size && isSpace(data[begin + 9]);
bool const opensWithHtml = matchAt(data, size, begin, "<html") &&
begin + 5 < size && isTagNameEnd(data[begin + 5]);
if (!opensWithDoctype && !opensWithHtml)
return Result::noDoctype;
// Locate the <html> start tag.
std::size_t open = begin;
for (;;)
{
open = findFrom(data, size, open, "<html");
if (open == size)
return Result::noHtmlElement;
if (open + 5 < size && isTagNameEnd(data[open + 5]))
break;
++open;
}
// Locate the last </html> end tag. Scanning backwards means a literal
// "</html>" inside a script string does not shadow the real one.
std::size_t close = size;
std::size_t closeEnd = size;
{
std::size_t probe = open;
for (;;)
{
std::size_t const found = findFrom(data, size, probe, "</html");
if (found == size)
break;
// Allow whitespace between the tag name and '>'.
std::size_t j = found + 6;
while (j < size && isSpace(data[j]))
++j;
if (j < size && data[j] == '>')
{
close = found;
closeEnd = j + 1;
}
probe = found + 1;
}
}
if (close == size)
return Result::unclosed;
if (close < open)
return Result::unclosed;
// Nothing but whitespace may follow the closing tag.
for (std::size_t i = closeEnd; i < size; ++i)
if (!isSpace(data[i]))
return Result::trailingGarbage;
return Result::ok;
}
char const*
to_string(Result r)
{
switch (r)
{
case Result::ok:
return "ok";
case Result::empty:
return "empty document";
case Result::tooLarge:
return "document exceeds maxAppLoaderLength";
case Result::badUTF8:
return "malformed UTF-8";
case Result::badControlChar:
return "forbidden control character";
case Result::noDoctype:
return "does not open with a doctype or <html> tag";
case Result::noHtmlElement:
return "missing <html> start tag";
case Result::unclosed:
return "missing or misplaced </html> end tag";
case Result::trailingGarbage:
return "content after </html>";
}
return "unknown";
}
} // namespace appLoader
} // namespace ripple

View File

@@ -81,7 +81,6 @@ enum class LedgerNameSpace : std::uint16_t {
IMPORT_VLSEQ = 'I',
UNL_REPORT = 'R',
CRON = 'L',
APP_LOADER = 'W',
AMM = 'A',
BRIDGE = LEDGER_NAMESPACE2(0x01, 'H'),
XCHAIN_CLAIM_ID = 'Q',
@@ -613,12 +612,6 @@ did(AccountID const& account) noexcept
return {ltDID, indexHash(LedgerNameSpace::DID, account)};
}
Keylet
appLoader(AccountID const& account) noexcept
{
return {ltAPP_LOADER, indexHash(LedgerNameSpace::APP_LOADER, account)};
}
Keylet
oracle(AccountID const& account, std::uint32_t const& documentID) noexcept
{

View File

@@ -25,9 +25,6 @@
#include <xrpl/server/detail/JSONRPCUtil.h>
#include <boost/algorithm/string.hpp>
#include <string>
#include <vector>
namespace ripple {
std::string
@@ -60,9 +57,7 @@ HTTPReply(
int nStatus,
std::string const& content,
Json::Output const& output,
beast::Journal j,
std::string const& contentType,
std::vector<std::string> const& extraHeaders)
beast::Journal j)
{
JLOG(j.trace()) << "HTTP Reply " << nStatus << " " << content;
@@ -148,11 +143,9 @@ HTTPReply(
// output ("Access-Control-Allow-Origin: *\r\n");
output(std::to_string(content.size() + 2));
output("\r\n");
output("Content-Type: " + contentType + "\r\n");
for (auto const& h : extraHeaders)
output(h + "\r\n");
output(
"\r\n"
"Content-Type: application/json; charset=UTF-8\r\n");
output("Server: " + systemName() + "-json-rpc/");
output(BuildInfo::getFullVersionString());

View File

@@ -1,523 +0,0 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2025 XRPL Labs
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#include <test/jtx.h>
#include <xrpl/basics/StringUtilities.h>
#include <xrpl/protocol/AppLoader.h>
#include <xrpl/protocol/Feature.h>
#include <xrpl/protocol/Indexes.h>
#include <xrpl/protocol/Protocol.h>
#include <xrpl/protocol/jss.h>
#include <string>
namespace ripple {
namespace test {
struct PWALoader_test : public beast::unit_test::suite
{
// A minimal, realistic loader: enough markup to mount the app, no more.
static std::string
goodDoc()
{
return "<!DOCTYPE html>\n"
"<html lang=\"en\">\n"
"<head><meta charset=\"utf-8\"><title>xapp</title></head>\n"
"<body><div id=\"root\"></div>"
"<script src=\"/app.js\"></script></body>\n"
"</html>\n";
}
static appLoader::Result
check(std::string const& s)
{
return appLoader::validate(
reinterpret_cast<std::uint8_t const*>(s.data()), s.size());
}
// Build a valid document padded out to exactly `len` bytes.
static std::string
docOfSize(std::size_t len)
{
std::string const head = "<!DOCTYPE html><html><body>";
std::string const tail = "</body></html>";
if (len < head.size() + tail.size())
return head + tail;
return head + std::string(len - head.size() - tail.size(), 'x') + tail;
}
//--------------------------------------------------------------------------
// Validator unit tests. These drive appLoader::validate directly, with no
// ledger involved.
//--------------------------------------------------------------------------
void
testValidator()
{
testcase("validator");
using R = appLoader::Result;
// --- accepted ---------------------------------------------------
BEAST_EXPECT(check(goodDoc()) == R::ok);
BEAST_EXPECT(check("<!DOCTYPE html><html></html>") == R::ok);
BEAST_EXPECT(check("<!doctype html><html></html>") == R::ok);
BEAST_EXPECT(check("<!DoCtYpE html><html></html>") == R::ok);
// No doctype, but opens with <html> directly.
BEAST_EXPECT(check("<html></html>") == R::ok);
BEAST_EXPECT(check("<html lang=\"en\"></html>") == R::ok);
// Leading and trailing whitespace is tolerated.
BEAST_EXPECT(check(" \n\t<html></html>\n\n ") == R::ok);
// Whitespace inside the closing tag is legal HTML.
BEAST_EXPECT(check("<html></html >") == R::ok);
// A UTF-8 BOM is skipped.
BEAST_EXPECT(check("\xEF\xBB\xBF<html></html>") == R::ok);
// Multi-byte UTF-8 in the body.
BEAST_EXPECT(
check("<html><body>\xE6\xBC\xA2\xE5\xAD\x97"
"</body></html>") == R::ok);
// A literal </html> inside a script must not shadow the real one.
BEAST_EXPECT(
check("<html><script>var s=\"</html>\";</script></html>") == R::ok);
// TAB / LF / CR / FF are HTML whitespace and are fine.
BEAST_EXPECT(check("<html>\t\n\r\f</html>") == R::ok);
// --- rejected ---------------------------------------------------
BEAST_EXPECT(check("") == R::empty);
BEAST_EXPECT(check(" \n\t ") == R::empty);
// Plain text, JSON, and other non-documents.
BEAST_EXPECT(check("hello world") == R::noDoctype);
BEAST_EXPECT(check("{\"a\":1}") == R::noDoctype);
BEAST_EXPECT(check("<?xml version=\"1.0\"?>") == R::noDoctype);
BEAST_EXPECT(check("<body></body>") == R::noDoctype);
// A fragment is not a document.
BEAST_EXPECT(check("<div>hi</div>") == R::noDoctype);
// Doctype must be followed by whitespace.
BEAST_EXPECT(check("<!doctypehtml><html></html>") == R::noDoctype);
// Tag name must actually end.
BEAST_EXPECT(check("<htmlish></htmlish>") == R::noDoctype);
// Structure problems.
BEAST_EXPECT(check("<!DOCTYPE html>") == R::noHtmlElement);
BEAST_EXPECT(check("<!DOCTYPE html><p>orphan</p>") == R::noHtmlElement);
BEAST_EXPECT(check("<html>") == R::unclosed);
BEAST_EXPECT(check("<html><body>no end tag</body>") == R::unclosed);
// Missing '>' on the end tag.
BEAST_EXPECT(check("<html></html") == R::unclosed);
BEAST_EXPECT(check("<html></html>trailing") == R::trailingGarbage);
BEAST_EXPECT(
check("<html></html><script>x</script>") == R::trailingGarbage);
// Control characters.
BEAST_EXPECT(
check("<html>\x01"
"</html>") == R::badControlChar);
// Embedded NUL.
BEAST_EXPECT(
check(std::string("<html>\0</html>", 14)) == R::badControlChar);
BEAST_EXPECT(
check("<html>\x7F"
"</html>") == R::badControlChar);
// Size, at the 4 KiB boundary.
BEAST_EXPECT(maxAppLoaderLength == 4096);
BEAST_EXPECT(check(docOfSize(maxAppLoaderLength - 1)) == R::ok);
BEAST_EXPECT(check(docOfSize(maxAppLoaderLength)) == R::ok);
BEAST_EXPECT(check(docOfSize(maxAppLoaderLength + 1)) == R::tooLarge);
}
void
testUTF8()
{
testcase("utf8");
using R = appLoader::Result;
auto const wrap = [](std::string const& mid) {
return "<html>" + mid + "</html>";
};
// Well-formed sequences of each length.
BEAST_EXPECT(check(wrap("\x24")) == R::ok); // U+0024
BEAST_EXPECT(check(wrap("\xC2\xA2")) == R::ok); // U+00A2
BEAST_EXPECT(check(wrap("\xE0\xA4\xB9")) == R::ok); // U+0939
BEAST_EXPECT(check(wrap("\xF0\x90\x8D\x88")) == R::ok); // U+10348
BEAST_EXPECT(check(wrap("\xF4\x8F\xBF\xBF")) == R::ok); // U+10FFFF
// U+FFFE / U+FFFF are noncharacters but are valid UTF-8, and are
// permitted in interchange.
BEAST_EXPECT(check(wrap("\xEF\xBF\xBE")) == R::ok);
BEAST_EXPECT(check(wrap("\xEF\xBF\xBF")) == R::ok);
// Overlong encodings.
BEAST_EXPECT(check(wrap("\xC0\xAF")) == R::badUTF8);
BEAST_EXPECT(check(wrap("\xC1\xBF")) == R::badUTF8);
BEAST_EXPECT(check(wrap("\xE0\x80\xAF")) == R::badUTF8);
BEAST_EXPECT(check(wrap("\xF0\x80\x80\xAF")) == R::badUTF8);
// UTF-16 surrogate halves.
BEAST_EXPECT(check(wrap("\xED\xA0\x80")) == R::badUTF8); // D800
BEAST_EXPECT(check(wrap("\xED\xBF\xBF")) == R::badUTF8); // DFFF
// Beyond U+10FFFF.
BEAST_EXPECT(check(wrap("\xF4\x90\x80\x80")) == R::badUTF8);
BEAST_EXPECT(check(wrap("\xF5\x80\x80\x80")) == R::badUTF8);
BEAST_EXPECT(check(wrap("\xF8\x88\x80\x80\x80")) == R::badUTF8);
// Stray continuation bytes.
BEAST_EXPECT(check(wrap("\x80")) == R::badUTF8);
BEAST_EXPECT(check(wrap("\xBF")) == R::badUTF8);
// Truncated sequences at the very end of the buffer.
BEAST_EXPECT(check("<html></html>\xC2") == R::badUTF8);
BEAST_EXPECT(check("<html></html>\xE0\xA4") == R::badUTF8);
BEAST_EXPECT(check("<html></html>\xF0\x90\x8D") == R::badUTF8);
BEAST_EXPECT(check(std::string("\xC2", 1)) == R::badUTF8);
}
//--------------------------------------------------------------------------
// Ledger-level tests.
//--------------------------------------------------------------------------
void
testEnabled(FeatureBitset features)
{
testcase("enabled");
using namespace jtx;
for (bool const withLoader : {false, true})
{
auto const amend =
withLoader ? features : features - featurePWALoader;
Env env{*this, amend};
auto const alice = Account("alice");
env.fund(XRP(1000), alice);
env.close();
auto jt = noop(alice);
jt[sfAppLoader.fieldName] = strHex(goodDoc());
if (withLoader)
{
env(jt, fee(XRP(1)));
env.close();
BEAST_EXPECT(env.le(keylet::appLoader(alice.id())));
}
else
{
env(jt, ter(temDISABLED));
env.close();
BEAST_EXPECT(!env.le(keylet::appLoader(alice.id())));
}
}
// Removal is gated too: an empty blob is still a present field.
{
Env env{*this, features - featurePWALoader};
auto const alice = Account("alice");
env.fund(XRP(1000), alice);
env.close();
auto jt = noop(alice);
jt[sfAppLoader.fieldName] = "";
env(jt, ter(temDISABLED));
}
}
void
testSetAndRemove(FeatureBitset features)
{
testcase("set and remove");
using namespace jtx;
Env env{*this, features};
auto const alice = Account("alice");
env.fund(XRP(1000), alice);
env.close();
std::string const doc = goodDoc();
auto const klLoader = keylet::appLoader(alice.id());
// Nothing to begin with.
BEAST_EXPECT(!env.le(klLoader));
if (auto const root = env.le(alice); BEAST_EXPECT(root))
BEAST_EXPECT(!root->isFieldPresent(sfAppLoaderID));
// --- create ---
auto jt = noop(alice);
jt[sfAppLoader.fieldName] = strHex(doc);
env(jt, fee(XRP(1)));
env.close();
{
auto const sle = env.le(klLoader);
if (BEAST_EXPECT(sle))
{
BEAST_EXPECT((*sle)[sfAppLoader] == makeSlice(doc));
BEAST_EXPECT((*sle)[sfOwner] == alice.id());
}
// The AccountRoot points at it...
auto const root = env.le(alice);
if (BEAST_EXPECT(root))
{
BEAST_EXPECT((*root)[sfAppLoaderID] == klLoader.key);
// ...and the blob is NOT inline on the AccountRoot.
BEAST_EXPECT(!root->isFieldPresent(sfAppLoader));
}
}
// No reserve is taken, and because the object is not in the owner
// directory it is invisible to account_objects. Reaching it requires
// sfAppLoaderID (or recomputing the keylet); this asserts the
// trade-off deliberately rather than by omission.
{
auto const jrr = env.rpc(
"json",
"account_objects",
R"({"account":")" + alice.human() + R"("})");
BEAST_EXPECT(jrr[jss::result][jss::account_objects].size() == 0);
// ledger_entry by index still resolves it.
auto const le = env.rpc(
"json",
"ledger_entry",
R"({"index":")" + to_string(klLoader.key) + R"("})");
BEAST_EXPECT(
le[jss::result][jss::node][sfLedgerEntryType.fieldName] ==
jss::AppLoader);
}
// --- overwrite in place ---
std::string const doc2 = "<html><body>v2</body></html>";
jt[sfAppLoader.fieldName] = strHex(doc2);
env(jt, fee(XRP(1)));
env.close();
{
auto const sle = env.le(klLoader);
if (BEAST_EXPECT(sle))
BEAST_EXPECT((*sle)[sfAppLoader] == makeSlice(doc2));
// The pointer is unchanged by an overwrite.
if (auto const root = env.le(alice); BEAST_EXPECT(root))
BEAST_EXPECT((*root)[sfAppLoaderID] == klLoader.key);
}
// --- remove with an empty blob ---
jt[sfAppLoader.fieldName] = "";
env(jt, fee(XRP(1)));
env.close();
BEAST_EXPECT(!env.le(klLoader));
if (auto const root = env.le(alice); BEAST_EXPECT(root))
BEAST_EXPECT(!root->isFieldPresent(sfAppLoaderID));
// Removing when already absent is a no-op, not an error.
env(jt, fee(XRP(1)));
env.close();
BEAST_EXPECT(!env.le(klLoader));
// An AccountSet that does not mention AppLoader leaves it alone.
jt[sfAppLoader.fieldName] = strHex(doc);
env(jt, fee(XRP(1)));
env.close();
env(noop(alice));
env.close();
if (auto const sle = env.le(klLoader); BEAST_EXPECT(sle))
BEAST_EXPECT((*sle)[sfAppLoader] == makeSlice(doc));
}
void
testFee(FeatureBitset features)
{
testcase("fee");
using namespace jtx;
Env env{*this, features};
auto const alice = Account("alice");
env.fund(XRP(1000), alice);
env.close();
auto const base = env.current()->fees().base;
std::string const doc = docOfSize(1024);
// The required fee is base + one drop per byte.
auto const expected = base + XRPAmount{1024};
auto jt = noop(alice);
jt[sfAppLoader.fieldName] = strHex(doc);
// One drop short is rejected.
env(jt, fee(expected - XRPAmount{1}), ter(telINSUF_FEE_P));
env.close();
BEAST_EXPECT(!env.le(keylet::appLoader(alice.id())));
// Exactly the required fee succeeds, and that much is burned.
auto const before = env.balance(alice);
env(jt, fee(expected));
env.close();
BEAST_EXPECT(env.le(keylet::appLoader(alice.id())));
BEAST_EXPECT(before - env.balance(alice) == drops(expected));
// Removal carries an empty blob, so it costs only the base fee.
auto jtDel = noop(alice);
jtDel[sfAppLoader.fieldName] = "";
env(jtDel, fee(base));
env.close();
BEAST_EXPECT(!env.le(keylet::appLoader(alice.id())));
}
void
testMalformed(FeatureBitset features)
{
testcase("malformed");
using namespace jtx;
Env env{*this, features};
auto const alice = Account("alice");
env.fund(XRP(1000), alice);
env.close();
auto reject = [&](std::string const& doc) {
auto jt = noop(alice);
jt[sfAppLoader.fieldName] = strHex(doc);
env(jt, fee(XRP(1)), ter(temMALFORMED));
env.close();
BEAST_EXPECT(!env.le(keylet::appLoader(alice.id())));
};
reject("hello world");
reject("<div>fragment</div>");
reject("<!DOCTYPE html>");
reject("<html>");
reject("<html></html>trailing");
reject(std::string("<html>\0</html>", 14));
reject("<html>\xC0\xAF</html>");
reject(docOfSize(maxAppLoaderLength + 1));
// The largest legal document is accepted.
auto jt = noop(alice);
jt[sfAppLoader.fieldName] = strHex(docOfSize(maxAppLoaderLength));
env(jt, fee(XRP(1)));
env.close();
BEAST_EXPECT(env.le(keylet::appLoader(alice.id())));
}
void
testAccountDelete(FeatureBitset features)
{
testcase("account delete");
using namespace jtx;
Env env{*this, features};
auto const alice = Account("alice");
auto const bob = Account("bob");
env.fund(XRP(1000), alice, bob);
env.close();
auto jt = noop(alice);
jt[sfAppLoader.fieldName] = strHex(goodDoc());
env(jt, fee(XRP(1)));
env.close();
BEAST_EXPECT(env.le(keylet::appLoader(alice.id())));
// AccountDelete requires the account to be well seasoned.
for (int i = 0; i < 256; ++i)
env.close();
// The AppLoader is not in the owner directory, so it must not block
// deletion -- and DeleteAccount must still erase it, or
// AccountRootsDeletedClean fires on the orphan.
env(acctdelete(alice, bob),
fee(drops(env.current()->fees().increment)));
env.close();
BEAST_EXPECT(!env.le(alice));
BEAST_EXPECT(!env.le(keylet::appLoader(alice.id())));
}
void
testURITokenUTF8Gate(FeatureBitset features)
{
testcase("uritoken utf8 gate");
using namespace jtx;
// U+FFFF is well-formed UTF-8, but the pre-amendment URIToken check
// rejected it. Under featurePWALoader it is accepted. This is the
// observable behaviour change the amendment gate exists to cover.
std::string const uri = "ipfs://x\xEF\xBF\xBF";
for (bool const withLoader : {false, true})
{
auto const amend =
withLoader ? features : features - featurePWALoader;
Env env{*this, amend};
auto const alice = Account("alice");
env.fund(XRP(1000), alice);
env.close();
env(uritoken::mint(alice, uri),
fee(XRP(1)),
ter(withLoader ? TER{tesSUCCESS} : TER{temMALFORMED}));
env.close();
}
// Genuinely malformed UTF-8 is rejected either side of the amendment.
for (bool const withLoader : {false, true})
{
auto const amend =
withLoader ? features : features - featurePWALoader;
Env env{*this, amend};
auto const alice = Account("alice");
env.fund(XRP(1000), alice);
env.close();
env(uritoken::mint(alice, std::string("bad\xC0\xAF")),
fee(XRP(1)),
ter(temMALFORMED));
env.close();
}
}
void
testWithFeats(FeatureBitset features)
{
testEnabled(features);
testSetAndRemove(features);
testFee(features);
testMalformed(features);
testAccountDelete(features);
testURITokenUTF8Gate(features);
}
public:
void
run() override
{
using namespace test::jtx;
testValidator();
testUTF8();
testWithFeats(supported_amendments());
}
};
BEAST_DEFINE_TESTSUITE(PWALoader, app, ripple);
} // namespace test
} // namespace ripple

View File

@@ -106,7 +106,8 @@ public:
std::string const& partition,
beast::severities::Severity threshold) override
{
return std::make_unique<SuiteJournalSink>(partition, threshold, suite_);
return std::make_unique<SuiteJournalSink>(
partition, threshold, suite_, this);
}
};

148
src/test/jtx/TestEnv.h Normal file
View File

@@ -0,0 +1,148 @@
#ifndef TEST_JTX_TESTENV_H_INCLUDED
#define TEST_JTX_TESTENV_H_INCLUDED
#include <test/jtx/Env.h>
#include <xrpl/basics/Log.h>
#include <xrpl/protocol/AccountID.h>
#include <cstdlib>
#include <cstring>
#include <map>
#include <sstream>
#include <string>
namespace ripple {
namespace test {
namespace jtx {
/**
* TestEnv wraps Env with:
* - Named account registry: env.account("alice")
* - Auto log transform: replaces r-addresses with Account(name) in log output
* - Env-var driven per-partition log levels via TESTENV_LOGGING
*
* Usage:
* TestEnv env{suite, features};
* auto const& alice = env.account("alice");
* auto const& bob = env.account("bob");
* env.fund(XRP(10000), alice, bob);
* // Logs now show Account(alice), Account(bob) instead of r-addresses
*
* Log levels via env var:
* TESTENV_LOGGING="HooksTrace=trace,View=debug"
*
* Valid levels: trace, debug, info, warning, error, fatal
*/
class TestEnv : public Env
{
std::map<std::string, Account> accounts_;
std::string prefix_;
public:
TestEnv(beast::unit_test::suite& suite, FeatureBitset features)
: Env(suite, features)
{
installTransform();
applyLoggingEnvVar();
}
TestEnv(
beast::unit_test::suite& suite,
std::unique_ptr<Config> config,
FeatureBitset features,
std::unique_ptr<Logs> logs = nullptr,
beast::severities::Severity thresh = beast::severities::kError)
: Env(suite, std::move(config), features, std::move(logs), thresh)
{
installTransform();
applyLoggingEnvVar();
}
~TestEnv()
{
app().logs().setTransform(nullptr);
}
/// Get or create a named account.
/// First call creates the Account; subsequent calls return the same one.
Account const&
account(std::string const& name)
{
auto [it, inserted] = accounts_.try_emplace(name, name);
return it->second;
}
/// Set a prefix that appears at the start of every log line.
/// Useful for visually separating test phases in trace output.
/// Pass empty string to clear.
void
setPrefix(std::string const& prefix)
{
prefix_ = prefix.empty() ? "" : "[" + prefix + "] ";
}
private:
static beast::severities::Severity
parseSeverity(std::string const& s)
{
if (s == "trace")
return beast::severities::kTrace;
if (s == "debug")
return beast::severities::kDebug;
if (s == "info")
return beast::severities::kInfo;
if (s == "warning")
return beast::severities::kWarning;
if (s == "error")
return beast::severities::kError;
if (s == "fatal")
return beast::severities::kFatal;
return beast::severities::kError;
}
void
applyLoggingEnvVar()
{
// Parse TESTENV_LOGGING="Partition1=level,Partition2=level"
auto const* envVal = std::getenv("TESTENV_LOGGING");
if (!envVal || !envVal[0])
return;
std::istringstream ss(envVal);
std::string pair;
while (std::getline(ss, pair, ','))
{
auto eq = pair.find('=');
if (eq == std::string::npos)
continue;
auto partition = pair.substr(0, eq);
auto level = pair.substr(eq + 1);
app().logs().get(partition).threshold(parseSeverity(level));
}
}
void
installTransform()
{
app().logs().setTransform([this](std::string const& text) {
std::string out = prefix_ + text;
for (auto const& [name, acc] : accounts_)
{
auto raddr = toBase58(acc.id());
std::string::size_type pos = 0;
std::string replacement = "Account(" + name + ")";
while ((pos = out.find(raddr, pos)) != std::string::npos)
{
out.replace(pos, raddr.size(), replacement);
pos += replacement.size();
}
}
return out;
});
}
};
} // namespace jtx
} // namespace test
} // namespace ripple
#endif

View File

@@ -19,6 +19,7 @@
#ifndef TEST_UNIT_TEST_SUITE_JOURNAL_H
#define TEST_UNIT_TEST_SUITE_JOURNAL_H
#include <xrpl/basics/Log.h>
#include <xrpl/beast/unit_test.h>
#include <xrpl/beast/utility/Journal.h>
#include <mutex>
@@ -31,13 +32,18 @@ class SuiteJournalSink : public beast::Journal::Sink
{
std::string partition_;
beast::unit_test::suite& suite_;
Logs* logs_ = nullptr;
public:
SuiteJournalSink(
std::string const& partition,
beast::severities::Severity threshold,
beast::unit_test::suite& suite)
: Sink(threshold, false), partition_(partition + " "), suite_(suite)
beast::unit_test::suite& suite,
Logs* logs = nullptr)
: Sink(threshold, false)
, partition_(partition + " ")
, suite_(suite)
, logs_(logs)
{
}
@@ -97,11 +103,12 @@ SuiteJournalSink::writeAlways(
// Only write the string if the level at least equals the threshold.
if (level >= threshold())
{
std::string const& output = logs_ ? logs_->applyTransform(text) : text;
// std::endl flushes → sync() → str()/str("") race in shared buffer →
// crashes
static std::mutex log_mutex;
std::lock_guard lock(log_mutex);
suite_.log << s << partition_ << text << std::endl;
suite_.log << s << partition_ << output << std::endl;
}
}

View File

@@ -12,9 +12,11 @@
#include <xrpl/protocol/TER.h>
#include <xrpl/protocol/digest.h>
#include <any>
#include <fstream>
#include <memory>
#include <optional>
#include <queue>
#include <set>
#include <utility>
#include <vector>
#include <wasmedge/wasmedge.h>
@@ -305,6 +307,130 @@ static WasmEdge_String hookFunctionName =
// see: lib/system/allocator.cpp
#define WasmEdge_kPageSize 65536ULL
// --- Coverage infrastructure ---
//
// Global coverage accumulator keyed by hook hash. Persists across all hook
// executions in the process. Each __on_source_line call records a (line, col)
// pair under the executing hook's hash.
//
// Test API:
// hook::coverageReset() — clear all accumulated data
// hook::coverageHits(hookHash) — get hits for a specific hook
// hook::coverageLabel(hash, label) — register a human-readable label
// hook::coverageDump(path) — write all data to a file
//
// The dump file format is:
// [label or hash]
// hits=<line:col>,<line:col>,...
struct CoverageData
{
std::set<uint32_t> hits{};
};
// Global accumulator — survives across HookContext lifetimes
inline std::map<ripple::uint256, CoverageData>&
coverageMap()
{
static std::map<ripple::uint256, CoverageData> map;
return map;
}
// Hash → label mapping (e.g. hash → "file:tipbot/tip.c")
inline std::map<ripple::uint256, std::string>&
coverageLabels()
{
static std::map<ripple::uint256, std::string> labels;
return labels;
}
inline void
coverageReset()
{
coverageMap().clear();
coverageLabels().clear();
}
inline void
coverageLabel(ripple::uint256 const& hookHash, std::string const& label)
{
coverageLabels()[hookHash] = label;
}
inline std::set<uint32_t> const*
coverageHits(ripple::uint256 const& hookHash)
{
auto& map = coverageMap();
auto it = map.find(hookHash);
if (it == map.end())
return nullptr;
return &it->second.hits;
}
inline bool
coverageDump(std::string const& path)
{
auto& map = coverageMap();
if (map.empty())
return false;
auto& labels = coverageLabels();
std::ofstream out(path);
if (!out)
return false;
for (auto const& [hash, data] : map)
{
auto it = labels.find(hash);
if (it != labels.end())
out << "[" << it->second << "]\n";
else
out << "[" << to_string(hash) << "]\n";
out << "hits=";
bool first = true;
for (auto key : data.hits)
{
if (!first)
out << ",";
out << (key >> 16) << ":" << (key & 0xFFFF);
first = false;
}
out << "\n\n";
}
return true;
}
// --- Coverage host callback ---
inline WasmEdge_Result
onSourceLine(
void* data_ptr,
const WasmEdge_CallingFrameContext* frameCtx,
const WasmEdge_Value* in,
WasmEdge_Value* out)
{
// Called by hookz-instrumented WASM at each DWARF source location.
// in[0] = line number, in[1] = column number.
(void)out;
(void)frameCtx;
auto* hookCtx = reinterpret_cast<HookContext*>(data_ptr);
if (!hookCtx)
return WasmEdge_Result_Success;
uint32_t line = WasmEdge_ValueGetI32(in[0]);
uint32_t col = WasmEdge_ValueGetI32(in[1]);
// Pack (line, col) into a single uint32_t key.
// Limits: line < 65536, col < 65536 — more than sufficient for hooks.
uint32_t key = (line << 16) | (col & 0xFFFF);
coverageMap()[hookCtx->result.hookHash].hits.insert(key);
return WasmEdge_Result_Success;
}
/**
* HookExecutor is effectively a two-part function:
* The first part sets up the Hook Api inside the wasm import, ready for use
@@ -483,6 +609,22 @@ public:
#undef HOOK_WRAP_PARAMS
#pragma pop_macro("HOOK_API_DEFINITION")
// Coverage callback: void __on_source_line(i32 line, i32 col)
// Registered unconditionally — production hooks don't import it,
// so it's harmless. Instrumented hooks call it at each DWARF
// source location to record line:col coverage hits.
{
static WasmEdge_ValType paramsOSL[] = {
WasmEdge_ValType_I32, WasmEdge_ValType_I32};
static auto* ftOSL =
WasmEdge_FunctionTypeCreate(paramsOSL, 2, nullptr, 0);
auto* hfOSL = WasmEdge_FunctionInstanceCreate(
ftOSL, hook::onSourceLine, (void*)(&ctx), 0);
static auto nameOSL =
WasmEdge_StringCreateByCString("__on_source_line");
WasmEdge_ModuleInstanceAddFunction(importObj, nameOSL, hfOSL);
}
WasmEdge_TableInstanceContext* hostTable =
WasmEdge_TableInstanceCreate(tableType);
WasmEdge_ModuleInstanceAddTable(importObj, tableName, hostTable);

View File

@@ -1111,7 +1111,7 @@ DEFINE_HOOK_FUNCTION(
if (NOT_IN_BOUNDS(read_ptr, read_len, memory_length))
return OUT_OF_BOUNDS;
if (!j.trace())
if (!jh.trace())
return 0ULL;
if (read_len > 128)
@@ -1125,16 +1125,16 @@ DEFINE_HOOK_FUNCTION(
if (read_len > 0)
{
j.trace() << "HookTrace[" << HC_ACC() << "]: "
<< std::string_view(
(const char*)memory + read_ptr, read_len)
<< ": " << number;
JLOG(jh.trace())
<< "HookTrace[" << HC_ACC() << "]: "
<< std::string_view((const char*)memory + read_ptr, read_len)
<< ": " << number;
return 0ULL;
}
}
j.trace() << "HookTrace[" << HC_ACC() << "]: " << number;
JLOG(jh.trace()) << "HookTrace[" << HC_ACC() << "]: " << number;
return 0ULL;
HOOK_TEARDOWN();
}
@@ -1154,7 +1154,7 @@ DEFINE_HOOK_FUNCTION(
NOT_IN_BOUNDS(dread_ptr, dread_len, memory_length))
return OUT_OF_BOUNDS;
if (!j.trace())
if (!jh.trace())
return 0ULL;
if (mread_len > 128)
@@ -1214,8 +1214,9 @@ DEFINE_HOOK_FUNCTION(
if (out_len > 0)
{
j.trace() << "HookTrace[" << HC_ACC() << "]: "
<< std::string_view((const char*)output_storage, out_len);
JLOG(jh.trace()) << "HookTrace[" << HC_ACC() << "]: "
<< std::string_view(
(const char*)output_storage, out_len);
}
return 0ULL;
@@ -3403,7 +3404,7 @@ DEFINE_HOOK_FUNCTION(
if (NOT_IN_BOUNDS(read_ptr, read_len, memory_length))
return OUT_OF_BOUNDS;
if (!j.trace())
if (!jh.trace())
return 0ULL;
if (read_len > 128)
@@ -3420,8 +3421,8 @@ DEFINE_HOOK_FUNCTION(
if (float1 == 0)
{
j.trace() << "HookTrace[" << HC_ACC() << "]: " << messageKey
<< ": Float 0*10^(0) <ZERO>";
JLOG(jh.trace()) << "HookTrace[" << HC_ACC() << "]: " << messageKey
<< ": Float 0*10^(0) <ZERO>";
return 0ULL;
}
@@ -3432,14 +3433,14 @@ DEFINE_HOOK_FUNCTION(
man.value() > maxMantissa || exp.value() < minExponent ||
exp.value() > maxExponent)
{
j.trace() << "HookTrace[" << HC_ACC() << "]: " << messageKey
<< ": Float <INVALID>";
JLOG(jh.trace()) << "HookTrace[" << HC_ACC() << "]: " << messageKey
<< ": Float <INVALID>";
return 0ULL;
}
j.trace() << "HookTrace[" << HC_ACC() << "]:" << messageKey << ": Float "
<< (neg ? "-" : "") << man.value() << "*10^(" << exp.value()
<< ")";
JLOG(jh.trace()) << "HookTrace[" << HC_ACC() << "]:" << messageKey
<< ": Float " << (neg ? "-" : "") << man.value() << "*10^("
<< exp.value() << ")";
return 0ULL;
HOOK_TEARDOWN();

View File

@@ -428,19 +428,6 @@ DeleteAccount::doApply()
if (src->isFieldPresent(sfHookNamespaces) || src->isFieldPresent(sfHooks))
return tecHAS_OBLIGATIONS;
// The AppLoader is linked directly from the AccountRoot via sfAppLoaderID
// rather than through the owner directory, so the cleanup walk below will
// never visit it. Erase it here, or account deletion would orphan it --
// which AccountRootsDeletedClean would flag, since keylet::appLoader is
// registered in directAccountKeylets.
//
// Not gated on featurePWALoader: on a ledger where the amendment is not
// active no such object can exist, so the peek simply finds nothing. That
// is deliberately safer than gating, which would orphan the object in the
// event the rule ever read as disabled while one existed.
if (auto sleLoader = view().peek(keylet::appLoader(account_)))
view().erase(sleLoader);
// Delete all of the entries in the account directory.
Keylet const ownerDirKeylet{keylet::ownerDir(account_)};
auto const ter = cleanupOnAccountDelete(

View File

@@ -612,7 +612,6 @@ LedgerEntryTypesMatch::visitEntry(
case ltMPTOKEN:
case ltCREDENTIAL:
case ltPERMISSIONED_DOMAIN:
case ltAPP_LOADER:
break;
default:
invalidTypeAdded_ = true;

View File

@@ -201,7 +201,7 @@ Remit::preflight(PreflightContext const& ctx)
return temMALFORMED;
}
if (!URIToken::validateUTF8(uri, ctx.rules.enabled(featurePWALoader)))
if (!URIToken::validateUTF8(uri))
{
JLOG(ctx.j.warn())
<< "Malformed transaction: Invalid UTF8 inside MintURIToken.";

View File

@@ -21,7 +21,6 @@
#include <xrpld/core/Config.h>
#include <xrpld/ledger/View.h>
#include <xrpl/basics/Log.h>
#include <xrpl/protocol/AppLoader.h>
#include <xrpl/protocol/Feature.h>
#include <xrpl/protocol/Indexes.h>
#include <xrpl/protocol/PublicKey.h>
@@ -56,26 +55,6 @@ SetAccount::makeTxConsequences(PreflightContext const& ctx)
return TxConsequences{ctx.tx, getTxConsequencesCategory(ctx.tx)};
}
XRPAmount
SetAccount::calculateBaseFee(ReadView const& view, STTx const& tx)
{
XRPAmount extraFee{0};
if (view.rules().enabled(featurePWALoader))
{
if (auto const loader = tx[~sfAppLoader])
{
// One drop per byte. Charged on every AccountSet that carries the
// field, including overwrites; removal carries an empty blob and
// so costs nothing extra.
extraFee += XRPAmount{static_cast<std::int64_t>(
loader->size() * appLoaderFeeDropsPerByte)};
}
}
return Transactor::calculateBaseFee(view, tx) + extraFee;
}
NotTEC
SetAccount::preflight(PreflightContext const& ctx)
{
@@ -205,28 +184,6 @@ SetAccount::preflight(PreflightContext const& ctx)
return temMALFORMED;
}
// AppLoader
if (auto const loader = tx[~sfAppLoader])
{
if (!ctx.rules.enabled(featurePWALoader))
return temDISABLED;
// An empty blob is the removal sentinel, matching Domain and
// MessageKey. It is not validated as a document.
if (!loader->empty())
{
auto const result =
appLoader::validate(loader->data(), loader->size());
if (result != appLoader::Result::ok)
{
JLOG(j.trace()) << "Malformed transaction: AppLoader: "
<< appLoader::to_string(result);
return temMALFORMED;
}
}
}
// HookStateScale
if (tx.isFieldPresent(sfHookStateScale))
{
@@ -608,57 +565,6 @@ SetAccount::doApply()
}
}
//
// AppLoader
//
// The document lives in its own ltAPP_LOADER object rather than inline on
// the AccountRoot, so that a multi-kilobyte blob is not dragged into
// memory every time the account is touched. The AccountRoot keeps only a
// pointer.
//
// The object is not placed in the owner directory and takes no reserve:
// its storage is paid for by the per-byte fee added in calculateBaseFee.
// DeleteAccount therefore has to erase it explicitly, since its directory
// walk will never see it.
//
if (view().rules().enabled(featurePWALoader) &&
tx.isFieldPresent(sfAppLoader))
{
Blob const loader = tx.getFieldVL(sfAppLoader);
Keylet const klLoader = keylet::appLoader(account_);
auto sleLoader = view().peek(klLoader);
if (loader.empty())
{
// Removal.
if (sleLoader)
{
JLOG(j_.trace()) << "unset app loader";
view().erase(sleLoader);
}
if (sle->isFieldPresent(sfAppLoaderID))
sle->makeFieldAbsent(sfAppLoaderID);
}
else if (sleLoader)
{
// Overwrite in place; the pointer already stands.
JLOG(j_.trace()) << "update app loader";
sleLoader->setFieldVL(sfAppLoader, loader);
view().update(sleLoader);
}
else
{
// Create.
JLOG(j_.trace()) << "set app loader";
sleLoader = std::make_shared<SLE>(klLoader);
(*sleLoader)[sfOwner] = account_;
sleLoader->setFieldVL(sfAppLoader, loader);
view().insert(sleLoader);
sle->setFieldH256(sfAppLoaderID, klLoader.key);
}
}
//
// TransferRate
//

View File

@@ -41,14 +41,6 @@ public:
static TxConsequences
makeTxConsequences(PreflightContext const& ctx);
/** Adds appLoaderFeeDropsPerByte per byte of AppLoader document.
The AppLoader ledger object carries no owner reserve, so the cost of
the storage is recovered here instead.
*/
static XRPAmount
calculateBaseFee(ReadView const& view, STTx const& tx);
static NotTEC
preflight(PreflightContext const& ctx);

View File

@@ -518,8 +518,7 @@ SetHook::validateHookSetEntry(SetHookCtx& ctx, STObject const& hookSetObj)
if (hookSetObj.isFieldPresent(sfHookName))
{
auto name = hookSetObj.getFieldVL(sfHookName);
if (!validateHookName(
name, ctx.rules.enabled(featurePWALoader), ctx.j))
if (!validateHookName(name, ctx.j))
return false;
}
@@ -547,7 +546,7 @@ SetHook::validateHookSetEntry(SetHookCtx& ctx, STObject const& hookSetObj)
}
auto result = validateGuards(
hook, // wasm to verify
hook,
logger,
hsacc,
hook_api::getImportWhitelist(ctx.rules),
@@ -623,10 +622,7 @@ SetHook::validateHookSetEntry(SetHookCtx& ctx, STObject const& hookSetObj)
}
bool
SetHook::validateHookName(
Blob const& name,
bool permitNoncharacters,
beast::Journal const& j)
SetHook::validateHookName(Blob const& name, beast::Journal const& j)
{
if (name.size() != 0 && (name.size() < 4 || 16 < name.size()))
{
@@ -634,7 +630,7 @@ SetHook::validateHookName(
<< "sfHookName must be between 8 and 32 hex characters.";
return false;
}
if (!URIToken::validateUTF8(name, permitNoncharacters))
if (!URIToken::validateUTF8(name))
{
JLOG(j.trace()) << "sfHookName must be a valid UTF-8 string.";
return false;

View File

@@ -92,10 +92,7 @@ public:
validateHookSetEntry(SetHookCtx& ctx, STObject const& hookSetObj);
static bool
validateHookName(
Blob const& name,
bool permitNoncharacters,
beast::Journal const& j);
validateHookName(Blob const& name, beast::Journal const& j);
static uint32_t
computeHookReserve(STObject const& hookObj);

View File

@@ -155,10 +155,7 @@ preflight1(PreflightContext const& ctx)
!ctx.rules.enabled(featureNamedHooks))
return temMALFORMED;
if (!SetHook::validateHookName(
ctx.tx.getFieldVL(sfHookName),
ctx.rules.enabled(featurePWALoader),
ctx.j))
if (!SetHook::validateHookName(ctx.tx.getFieldVL(sfHookName), ctx.j))
return temMALFORMED;
}

View File

@@ -103,7 +103,7 @@ URIToken::preflight(PreflightContext const& ctx)
return temMALFORMED;
}
if (!validateUTF8(uri, ctx.rules.enabled(featurePWALoader)))
if (!validateUTF8(uri))
{
JLOG(ctx.j.warn()) << "Malformed transaction. URI must be a "
"valid utf-8 string.";

View File

@@ -23,8 +23,6 @@
#include <xrpld/app/ledger/Ledger.h>
#include <xrpld/app/tx/detail/Transactor.h>
#include <xrpl/basics/Log.h>
#include <xrpl/protocol/AppLoader.h>
#include <xrpl/protocol/Feature.h>
#include <xrpl/protocol/Indexes.h>
namespace ripple {
@@ -32,37 +30,53 @@ namespace ripple {
class URIToken : public Transactor
{
public:
/** Validate a byte sequence as UTF-8.
Two things differ across featurePWALoader. Before the amendment,
U+FFFE and U+FFFF were rejected; they are well-formed UTF-8, and
noncharacters are permitted in interchange, so the amendment accepts
them. Because that changes which transactions preflight successfully,
it rides an amendment rather than landing directly.
@param permitNoncharacters Pass rules.enabled(featurePWALoader).
Defaults to the post-amendment behaviour so that a caller who
omits it is at worst wrong before activation, never after.
*/
bool inline static validateUTF8(
std::vector<uint8_t> const& u,
bool permitNoncharacters = true)
bool inline static validateUTF8(std::vector<uint8_t> const& u)
{
if (!appLoader::isValidUTF8(u.data(), u.size()))
return false;
if (permitNoncharacters)
return true;
// Pre-amendment behaviour: reject U+FFFE and U+FFFF, which encode as
// EF BF BE and EF BF BF. UTF-8 is self-synchronising, so in a
// sequence already known to be well-formed these bytes can only be
// those two code points.
for (std::size_t i = 0; i + 2 < u.size(); ++i)
if (u[i] == 0xEF && u[i + 1] == 0xBF &&
(u[i + 2] == 0xBE || u[i + 2] == 0xBF))
// this code is from
// https://www.cl.cam.ac.uk/~mgk25/ucs/utf8_check.c
uint8_t const* s = (uint8_t const*)u.data();
uint8_t const* end = s + u.size();
while (s < end)
{
if (*s < 0x80)
/* 0xxxxxxx */
s++;
else if ((s[0] & 0xe0) == 0xc0)
{
/* 110XXXXx 10xxxxxx */
if ((s[1] & 0xc0) != 0x80 ||
(s[0] & 0xfe) == 0xc0) /* overlong? */
return false;
else
s += 2;
}
else if ((s[0] & 0xf0) == 0xe0)
{
/* 1110XXXX 10Xxxxxx 10xxxxxx */
if ((s[1] & 0xc0) != 0x80 || (s[2] & 0xc0) != 0x80 ||
(s[0] == 0xe0 && (s[1] & 0xe0) == 0x80) || /* overlong? */
(s[0] == 0xed && (s[1] & 0xe0) == 0xa0) || /* surrogate? */
(s[0] == 0xef && s[1] == 0xbf &&
(s[2] & 0xfe) == 0xbe)) /* U+FFFE or U+FFFF? */
return false;
else
s += 3;
}
else if ((s[0] & 0xf8) == 0xf0)
{
/* 11110XXX 10XXxxxx 10xxxxxx 10xxxxxx */
if ((s[1] & 0xc0) != 0x80 || (s[2] & 0xc0) != 0x80 ||
(s[3] & 0xc0) != 0x80 ||
(s[0] == 0xf0 && (s[1] & 0xf0) == 0x80) || /* overlong? */
(s[0] == 0xf4 && s[1] > 0x8f) ||
s[0] > 0xf4) /* > U+10FFFF? */
return false;
else
s += 4;
}
else
return false;
}
return true;
}

View File

@@ -309,12 +309,6 @@ public:
// Enable the beta API version
bool BETA_RPC_API = false;
// Serve on-ledger AppLoader documents as HTML from the http/https ports
// under /pwa/<account>. Off by default: it turns the node into a web
// host for content it does not control. See ServerHandler for the
// restrictions that apply when it is on.
bool PWA_ENABLED = false;
// First, attempt to load the latest ledger directly from disk.
bool FAST_LOAD = false;
// When starting rippled with existing database it do not know it has those

View File

@@ -69,7 +69,6 @@ struct ConfigSection
#define SECTION_PATH_SEARCH "path_search"
#define SECTION_PATH_SEARCH_FAST "path_search_fast"
#define SECTION_PATH_SEARCH_MAX "path_search_max"
#define SECTION_PWA "pwa"
#define SECTION_PEER_PRIVATE "peer_private"
#define SECTION_PEERS_MAX "peers_max"
#define SECTION_PEERS_IN_MAX "peers_in_max"

View File

@@ -849,9 +849,6 @@ Config::loadFromString(std::string const& fileContents)
if (getSingleSection(secConfig, SECTION_BETA_RPC_API, strTemp, j_))
BETA_RPC_API = beast::lexicalCastThrow<bool>(strTemp);
if (getSingleSection(secConfig, SECTION_PWA, strTemp, j_))
PWA_ENABLED = beast::lexicalCastThrow<bool>(strTemp);
// Do not load trusted validator configuration for standalone mode
do
{

View File

@@ -217,16 +217,6 @@ private:
std::shared_ptr<Session> const&,
std::shared_ptr<JobQueue::Coro> coro);
/** Serve an account's on-ledger AppLoader document as HTML.
Only reached when PWA_ENABLED is set and the request was a GET of
/pwa/<account>. Runs on the job queue because it reads the ledger.
*/
void
processPWARequest(
std::shared_ptr<Session> const& session,
AccountID const& account);
void
processRequest(
Port const& port,

View File

@@ -19,7 +19,6 @@
#include <xrpld/rpc/ServerHandler.h>
#include <xrpld/app/ledger/LedgerMaster.h>
#include <xrpld/app/main/Application.h>
#include <xrpld/app/misc/NetworkOPs.h>
#include <xrpld/core/ConfigSections.h>
@@ -38,9 +37,7 @@
#include <xrpl/beast/rfc2616.h>
#include <xrpl/json/json_reader.h>
#include <xrpl/json/to_string.h>
#include <xrpl/protocol/AccountID.h>
#include <xrpl/protocol/ErrorCodes.h>
#include <xrpl/protocol/Indexes.h>
#include <xrpl/protocol/RPCErr.h>
#include <xrpl/resource/Fees.h>
#include <xrpl/resource/ResourceManager.h>
@@ -53,10 +50,7 @@
#include <boost/beast/http/string_body.hpp>
#include <algorithm>
#include <optional>
#include <stdexcept>
#include <string>
#include <vector>
namespace ripple {
@@ -300,118 +294,6 @@ buffers_to_string(ConstBufferSequence const& bs)
return s;
}
// Parse "/pwa/<account>" and return the account, or nullopt if the target is
// not a PWA request or the account is unparseable. A trailing slash and any
// query string or fragment are ignored, so /pwa/rXXX/?v=2 resolves the same
// as /pwa/rXXX.
static std::optional<AccountID>
parsePWATarget(boost::beast::string_view target)
{
static constexpr char prefix[] = "/pwa/";
static constexpr std::size_t prefixLen = sizeof(prefix) - 1;
if (target.size() <= prefixLen || target.substr(0, prefixLen) != prefix)
return std::nullopt;
std::string rest{target.substr(prefixLen)};
if (auto const cut = rest.find_first_of("?#"); cut != std::string::npos)
rest.erase(cut);
while (!rest.empty() && rest.back() == '/')
rest.pop_back();
// A base58 r-address contains no path separators; reject anything with
// further path structure rather than silently taking the first segment.
if (rest.empty() || rest.find('/') != std::string::npos)
return std::nullopt;
return parseBase58<AccountID>(rest);
}
void
ServerHandler::processPWARequest(
std::shared_ptr<Session> const& session,
AccountID const& account)
{
auto const j = app_.journal("PWA");
auto out = makeOutput(*session);
// Serving third-party HTML from the same origin as the JSON-RPC endpoint
// is dangerous: script in the document could POST commands back to the
// node. Two things guard against that.
//
// First, refuse outright if this connection would be granted elevated
// privileges, or if the port is password protected (in which case a
// browser would attach the credentials to same-origin requests).
if (!session->port().user.empty() || !session->port().password.empty())
{
HTTPReply(403, "Forbidden", out, j);
session->close(true);
return;
}
if (isUnlimited(requestRole(
Role::GUEST,
session->port(),
Json::objectValue,
session->remoteAddress().at_port(0),
"")))
{
JLOG(j.debug()) << "refusing to serve PWA content to a privileged "
<< "connection from "
<< session->remoteAddress().to_string();
HTTPReply(403, "Forbidden", out, j);
session->close(true);
return;
}
// Second, sandbox the document. "sandbox allow-scripts" without
// allow-same-origin puts the page in an opaque origin, so its scripts
// cannot reach this node's RPC endpoint at all. Note the trade-off: an
// opaque origin has no storage and no service workers, so a document
// served this way is not a fully functional PWA. Hosting one properly
// needs an origin per account, which is out of scope here.
static std::vector<std::string> const securityHeaders{
"Content-Security-Policy: sandbox allow-scripts allow-forms "
"allow-popups",
"X-Content-Type-Options: nosniff",
"X-Frame-Options: DENY",
"Referrer-Policy: no-referrer",
"Cross-Origin-Resource-Policy: same-origin"};
auto const ledger = app_.getLedgerMaster().getClosedLedger();
if (!ledger)
{
HTTPReply(503, "Service Unavailable", out, j);
session->close(true);
return;
}
auto const sle = ledger->read(keylet::appLoader(account));
if (!sle || !sle->isFieldPresent(sfAppLoader))
{
HTTPReply(404, "Not Found", out, j);
session->close(true);
return;
}
Blob const blob = sle->getFieldVL(sfAppLoader);
JLOG(j.trace()) << "serving AppLoader for " << toBase58(account) << " ("
<< blob.size() << " bytes) from ledger " << ledger->seq();
HTTPReply(
200,
std::string(blob.begin(), blob.end()),
out,
j,
"text/html; charset=utf-8",
securityHeaders);
session->close(true);
}
void
ServerHandler::onRequest(Session& session)
{
@@ -424,35 +306,6 @@ ServerHandler::onRequest(Session& session)
return;
}
// PWA content: GET /pwa/<account>. Handled ahead of the RPC path because
// it is a plain document request, not a JSON-RPC call.
if (app_.config().PWA_ENABLED &&
session.request().method() == boost::beast::http::verb::get)
{
if (auto const account = parsePWATarget(session.request().target()))
{
std::shared_ptr<Session> detachedSession = session.detach();
auto const postResult = m_jobQueue.postCoro(
jtCLIENT_RPC,
"PWA-Client",
[this, detachedSession, account = *account](
std::shared_ptr<JobQueue::Coro>) {
processPWARequest(detachedSession, account);
});
if (postResult == nullptr)
{
HTTPReply(
503,
"Service Unavailable",
makeOutput(*detachedSession),
app_.journal("PWA"));
detachedSession->close(true);
}
return;
}
}
// Check user/password authorization
if (!authorized(session.port(), build_map(session.request())))
{