Compare commits

...

65 Commits

Author SHA1 Message Date
Ayaz Salikhov
00a178fb92 chore: Bump version to 3.3.0 2026-08-06 17:34:39 +01:00
Bart
39c8c293b3 chore: Bump version to 3.3.0-rc7 2026-08-04 17:11:00 -04:00
Pratik Mankawde
54cfdda00b fix: Increase manifest protocol message size cap and fix manifests relay
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-08-04 17:08:43 -04:00
Ed Hennis
15d08770bf Merge branch 'release/3.2.x' into ximinez/merge-321-to-330
* release/3.2.x:
  chore: Bump version to 3.2.1
  chore: Bump version to 3.2.1-rc1
  fix: Cap untrusted manifests per message and drop oversized ones
  fix: Reject oversized validator manifest before decoding
  fix: Reduce untrusted manifest cache cap to 100
  fix: Bound untrusted manifest cache
2026-08-04 13:28:52 -04:00
Pratik Mankawde
8461ded0d8 fix: Cap untrusted manifests per message and drop oversized ones 2026-08-03 12:00:11 -04:00
Ed Hennis
d4c1359921 chore: Bump version to 3.2.1 2026-07-31 19:46:17 -04:00
Bart
a88ba66fce chore: Bump version to 3.2.1-rc1 2026-07-31 19:43:19 -04:00
Valentin Balaschenko
4bd1d1ca2f fix: Cap untrusted manifests per message and drop oversized ones
Bound the number of manifests carried in a single TMManifests message
(kMaxManifestsPerMessage). Trusted manifests are always included and
processed; untrusted gossip is capped per message on both send and
receive, and the sender is charged only when untrusted entries are
actually skipped. Oversized TMManifests messages are dropped without
penalty at the protocol layer so an unpatched peer is not disconnected.

Complements the cache bound from #276/#323.
2026-07-31 19:43:18 -04:00
Bart
0cce5a06d9 fix: Reject oversized validator manifest before decoding 2026-07-31 19:43:18 -04:00
Pratik Mankawde
32a9cc4038 fix: Reduce untrusted manifest cache cap to 100 2026-07-31 19:43:15 -04:00
Pratik Mankawde
587505ef18 fix: Bound untrusted manifest cache 2026-07-31 19:43:04 -04:00
Bart
4eece4003d chore: Bump version to 3.3.0-rc6 2026-07-30 16:34:46 +01:00
Ayaz Salikhov
3ad6ce236e feat: Package validator-keys inside rippled 2026-07-30 11:29:37 -04:00
Bart
ccb9db0bc7 chore: Bump version to 3.3.0-rc5 2026-07-29 14:25:13 -04:00
Mayukha Vadari
24b6dad287 fix: Switch SponsorshipSet to use a delta for sfFeeAmount 2026-07-29 14:24:55 -04:00
Bart
e290005db5 fix: Re-revert "fix: Set request size limits and differential pricing for get-object-by-hash calls" 2026-07-28 14:02:49 -04:00
Bart
6668b7e8d0 chore: Bump version to 3.3.0-rc4 2026-07-24 18:40:02 -04:00
Bart
a5cc339d7b fix: Revert "fix: Set request size limits and differential pricing for get-object-by-hash calls" 2026-07-24 18:39:35 -04:00
Bart
1653f0c80f chore: Bump version to 3.3.0-rc3 2026-07-23 16:39:50 -04:00
Pratik Mankawde
c50edf507c fix: Reduce untrusted manifest cache cap to 100 2026-07-23 16:27:45 -04:00
Valentin Balaschenko
9cd531659a fix: Revert "fix: Reject oversized SHAMap nodes in gotStaleData and fetch-pack path" 2026-07-23 14:53:15 -04:00
Bart
982bf36dd8 chore: Bump version to 3.3.0-rc2 2026-07-17 17:09:13 -04:00
Denis Angell
faca302adf fix: Check transaction type before RawTransactions 2026-07-17 17:08:57 -04:00
Shawn Xie
6b3eaf091b fix: Change ConfidentialMPTConvert to no delegate 2026-07-17 16:06:17 -04:00
Pratik Mankawde
68a765d929 fix: Bound untrusted manifest cache 2026-07-17 14:07:57 -04:00
yinyiqian1
033dca2f0e feat: Make DynamicMPT opt-in-immutable 2026-07-17 14:02:35 -04:00
Ayaz Salikhov
bf65e5fa7b chore: Upload codecov for whole XRPLF org 2026-07-17 11:20:28 -04:00
Valentin Balaschenko
90b2a68da8 fix: Reject oversized SHAMap nodes in gotStaleData and fetch-pack path 2026-07-17 08:07:42 -04:00
Ayaz Salikhov
06a9b1b617 chore: Update mpt-crypto to 1.0.2 2026-07-17 08:07:01 -04:00
Jingchen
7d3611df2a fix: Compute validation suppression key over canonical serialisation 2026-07-17 07:33:44 -04:00
Bart
7877ee42a0 fix: Reject oversized validator manifest before decoding 2026-07-16 16:33:03 -04:00
Bart
1dcaf4b54e fix: Bound and offload per-connection subscription cleanup 2026-07-16 16:33:03 -04:00
Bart
5ab95748d4 refactor: Clean up pong replies 2026-07-16 16:33:03 -04:00
Bart
4a9ee54c88 fix: Handle malformed ledger replay responses 2026-07-16 16:33:03 -04:00
Bart
981c256933 fix: Use weighted median for close-time offset aggregation 2026-07-16 16:33:03 -04:00
Bart
a5af6b4e4a fix: Charge kFeeHeavyBurdenRpc in doChannelVerify 2026-07-16 16:33:03 -04:00
Bart
e1dae5f775 refactor: Use tecPSEUDO_ACCOUNT instead of tecNO_PERMISSION where relevant 2026-07-16 16:32:54 -04:00
Timothy Banks
6c793edf72 fix: Reject oversized TMPing messages 2026-07-16 15:07:36 -04:00
Timothy Banks
d60955e2fc fix: Acquire lock on getClosedLedgerHash 2026-07-16 12:41:34 -04:00
Timothy Banks
846369cbe7 fix: Use hashmap for quicker lookup in assembleAdd 2026-07-16 12:16:09 -04:00
Mayukha Vadari
d569f7db9e fix: Prevent credentials from being created for pseudo-accounts 2026-07-16 11:42:27 -04:00
Mayukha Vadari
a12ab0496c fix: Add missing pseudo-account checks 2026-07-16 11:18:26 -04:00
Bart
18e311e1e2 chore: Bump version to 3.3.0-rc1 (#7806)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2026-07-16 13:54:12 +00:00
Denis Angell
69b70d7a0d fix: Refactor Batch Transaction IDs (#7736)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Mayukha Vadari <mvadari@ripple.com>
2026-07-16 13:15:59 +00:00
Mayukha Vadari
b42cde3e85 refactor: Remove redundant enable checks in ConfidentialMPT txs (#7809) 2026-07-16 09:51:19 +00:00
Peter Chen
cd38c0e800 chore: Update mpt-crypto-lib to 0.4.0-rc4 (#7813) 2026-07-15 23:47:22 +00:00
Vlad
433e5f6896 fix: Reject zero CheckID in CheckCancel and CheckCash (#7685) 2026-07-15 22:08:45 +00:00
Ayaz Salikhov
781ab723af ci: Fix workflow launch on matrix-unrelated labels (#7812) 2026-07-15 18:24:31 +00:00
Ed Hennis
a24e543af3 fix: Allocate TaggedCache::getKeys() memory outside of lock (#7567)
Co-authored-by: xrplf-ai-reviewer[bot] <266832837+xrplf-ai-reviewer[bot]@users.noreply.github.com>
2026-07-15 13:30:20 +00:00
Sophia Xie
a0fd1cce54 fix: Re-store nodes missing from both backends during online_delete rotation (#7763)
Co-authored-by: Valentin Balaschenko <13349202+vlntb@users.noreply.github.com>
2026-07-14 23:42:40 +00:00
Kassaking7
cda63d00a2 fix: Add amendment sponsor for AccountRootsDeletedClean (#7801) 2026-07-14 19:41:53 +00:00
Ed Hennis
530e09dbe8 fix: Update base_uint and test changes released in 3.1.3 (#7570)
Co-authored-by: Sergey Kuznetsov <skuznetsov@ripple.com>
Co-authored-by: xrplf-ai-reviewer[bot] <266832837+xrplf-ai-reviewer[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2026-07-14 18:48:10 +00:00
Ed Hennis
f10dd7b450 fix: Handle rounding just above kMaxRep more accurately (#7389)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Vito Tumas <5780819+Tapanito@users.noreply.github.com>
Co-authored-by: xrplf-ai-reviewer[bot] <266832837+xrplf-ai-reviewer[bot]@users.noreply.github.com>
2026-07-14 18:47:41 +00:00
Ed Hennis
0a4676d947 fix: Document and assert "after" is never null in invariants (#7354)
Co-authored-by: Bart <bthomee@users.noreply.github.com>
2026-07-14 18:16:46 +00:00
Ayaz Salikhov
0dc942508e ci: Run full matrix only on Ready to merge or Full CI build labeled PRs (#7689)
Co-authored-by: Bart <bthomee@users.noreply.github.com>
2026-07-14 16:09:20 +00:00
Gregory Tsipenyuk
2403670da9 fix: Strengthen Clawback invariant checks for MPT balances (#7285) 2026-07-14 14:31:06 +00:00
Jingchen
c621136748 test: Add unit tests for IP address related functions (#7744)
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Ed Hennis <ed@ripple.com>
2026-07-14 14:29:18 +00:00
Sergey Kuznetsov
2e25435a4a ci: Add Rust to Nix docker image (#7571) 2026-07-14 13:28:55 +00:00
Ayaz Salikhov
ab3ff66cd9 docs: Add more information about pre-commit hooks and how to set them up (#7802) 2026-07-14 12:55:34 +00:00
Alex Kremer
73e97b8b84 test: Add JSON array size tests (#7592) 2026-07-14 12:35:04 +00:00
Ayaz Salikhov
e1d4f357dc chore: Enable most readability checks (#7772) 2026-07-14 12:21:40 +00:00
Bart
acd54fd627 ci: Do not run conflict checker when label is applied (#7774)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-07-14 10:22:49 +00:00
Ayaz Salikhov
cd06ee221d chore: Run clang_tidy_check with pass_filenames: false from pre-commit (#7800) 2026-07-13 19:19:18 +00:00
Peter Chen
752dab8b30 feat: Add delegate filter param for account_tx RPC (#6126) 2026-07-13 18:44:59 +00:00
Mayukha Vadari
86583bc34e refactor: Move jss.h include out of Indexes.h (#7799) 2026-07-13 17:44:47 +00:00
215 changed files with 8837 additions and 3121 deletions

View File

@@ -56,32 +56,17 @@ Checks: "-*,
readability-*,
-readability-avoid-const-params-in-decls,
-readability-avoid-unconditional-preprocessor-if,
-readability-container-data-pointer,
-readability-delete-null-pointer,
-readability-function-cognitive-complexity,
-readability-function-size,
-readability-identifier-length,
-readability-inconsistent-declaration-parameter-name,
-readability-isolate-declaration,
-readability-magic-numbers,
-readability-misplaced-array-index,
-readability-named-parameter,
-readability-operators-representation,
-readability-qualified-auto,
-readability-redundant-access-specifiers,
-readability-redundant-control-flow,
-readability-redundant-function-ptr-dereference,
-readability-redundant-preprocessor,
-readability-redundant-smartptr-get,
-readability-redundant-string-cstr,
-readability-simplify-subscript-expr,
-readability-static-accessed-through-instance,
-readability-string-compare,
-readability-uniqueptr-delete-release,
-readability-uppercase-literal-suffix,
-readability-use-anyofallof,
-readability-use-concise-preprocessor-directives
-readability-uppercase-literal-suffix
"
# ---
# bugprone-narrowing-conversions, # This will break a lot of code but we should enable it in the future because it can eliminate a lot of bugs

View File

@@ -65,6 +65,7 @@ words:
- Btrfs
- Buildx
- canonicality
- canonicalised
- changespq
- checkme
- choco
@@ -72,6 +73,7 @@ words:
- citardauq
- clawback
- clawbacks
- clippy
- cmaketoolchain
- coeffs
- coldwallet
@@ -128,6 +130,7 @@ words:
- godexsoft
- gpgcheck
- gpgkey
- Hinnant
- hotwallet
- hwaddress
- hwrap
@@ -161,6 +164,7 @@ words:
- llection
- LOCALGOOD
- logwstream
- Lombrozo
- lseq
- lsmf
- ltype
@@ -198,6 +202,7 @@ words:
- nftokens
- nftpage
- nikb
- Nikolaos
- nixfmt
- nixos
- nixpkgs
@@ -259,6 +264,7 @@ words:
- rocksdb
- Rohrs
- roundings
- rustc
- sahyadri
- Satoshi
- scons

View File

@@ -159,6 +159,7 @@ test.peerfinder > xrpl.protocol
test.protocol > test.jtx
test.protocol > test.unit_test
test.protocol > xrpl.basics
test.protocol > xrpld.core
test.protocol > xrpl.json
test.protocol > xrpl.protocol
test.rpc > test.jtx

View File

@@ -25,24 +25,16 @@ def get_cmake_args(build_type: str, extra_args: str) -> str:
return " ".join(args)
def runs_on_event(exclude_event_types: list[str], event: str | None) -> bool:
"""Whether a config should run for the current event.
'exclude_event_types' is a list of GitHub event names (e.g.
["pull_request"]) on which the config should NOT run; an empty list means
the config runs on every event. When no event is given (event is None), no
filtering is applied.
"""
if event is None:
return True
return event not in exclude_event_types
# ---------------------------------------------------------------------------
# Input types — shapes of the JSON config files
# ---------------------------------------------------------------------------
# Every config must declare 'minimal'. Minimal configs form the reduced matrix
# built for pull requests by default; the full matrix adds the rest. Packaging
# configs declare it too, but packaging is gated in the workflow, not by it.
@dataclasses.dataclass
class LinuxConfig:
"""One entry in linux.json's 'configs' or 'package_configs' arrays."""
@@ -50,13 +42,11 @@ class LinuxConfig:
compiler: list[str]
build_type: list[str]
arch: list[str]
minimal: bool
sanitizers: list[str] = dataclasses.field(default_factory=list)
suffix: str = ""
extra_cmake_args: str = ""
image: str = "" # only used by package_configs entries
# List of GitHub event names (e.g. "pull_request") on which this config
# should NOT run. Empty means it runs on every event.
exclude_event_types: list[str] = dataclasses.field(default_factory=list)
@dataclasses.dataclass
@@ -89,11 +79,9 @@ class PlatformConfig:
"""One entry in macos.json's or windows.json's 'configs' array."""
build_type: list[str]
minimal: bool
build_only: bool = False # if true, skip tests (e.g. macos/Windows Debug)
extra_cmake_args: str = ""
# List of GitHub event names (e.g. "pull_request") on which this config
# should NOT run. Empty means it runs on every event.
exclude_event_types: list[str] = dataclasses.field(default_factory=list)
def __post_init__(self) -> None:
if isinstance(self.build_type, str):
@@ -148,7 +136,8 @@ class MatrixEntry:
class PackagingEntry:
"""One entry in the generated packaging strategy matrix."""
artifact_name: str
xrpld_artifact_name: str
validator_keys_artifact_name: str
image: str
distro: str # e.g. "debian" or "rhel"; drives package-format-specific steps
@@ -168,20 +157,18 @@ _ARCHS: dict[str, Architecture] = {
}
def expand_linux_matrix(
linux: LinuxFile, event: str | None = None
) -> list[MatrixEntry]:
def expand_linux_matrix(linux: LinuxFile, minimal: bool) -> list[MatrixEntry]:
"""Expand a LinuxFile into a flat list of matrix entries.
Each config entry is expanded over the cross-product of its
compiler, build_type, sanitizers, and architecture lists. Configs that
exclude the current event are skipped.
compiler, build_type, sanitizers, and architecture lists. When 'minimal' is
true, only configs flagged as minimal are included.
"""
entries: list[MatrixEntry] = []
for distro, configs in linux.configs.items():
for cfg in configs:
if not runs_on_event(cfg.exclude_event_types, event):
if minimal and not cfg.minimal:
continue
# An empty sanitizers list means "one entry with no sanitizer".
effective_sanitizers = cfg.sanitizers or [""]
@@ -224,14 +211,19 @@ def expand_linux_packaging(linux: LinuxFile) -> list[PackagingEntry]:
the nix-based build images, because deb/rpm tooling (debhelper, rpm-build)
is taken from the distro's archive rather than from nixpkgs. Each config
entry carries its own 'image'.
The artifact names must match what the build job uploads: one artifact per
binary, each named after the build config.
"""
entries = []
for distro, configs in linux.package_configs.items():
for cfg in configs:
for compiler, build_type in itertools.product(cfg.compiler, cfg.build_type):
config_name = f"{distro}-{compiler}-{build_type.lower()}-amd64"
entries.append(
PackagingEntry(
artifact_name=f"xrpld-{distro}-{compiler}-{build_type.lower()}-amd64",
xrpld_artifact_name=f"xrpld-{config_name}",
validator_keys_artifact_name=f"validator-keys-{config_name}",
image=cfg.image,
distro=distro,
)
@@ -240,19 +232,17 @@ def expand_linux_packaging(linux: LinuxFile) -> list[PackagingEntry]:
return entries
def expand_platform_matrix(
pf: PlatformFile, event: str | None = None
) -> list[MatrixEntry]:
def expand_platform_matrix(pf: PlatformFile, minimal: bool) -> list[MatrixEntry]:
"""Expand a PlatformFile (macOS or Windows) into matrix entries.
Configs that exclude the current event are skipped.
When 'minimal' is true, only configs flagged as minimal are included.
"""
platform_name, arch = pf.platform.split("/")
is_windows = platform_name == "windows"
entries: list[MatrixEntry] = []
for cfg in pf.configs:
if not runs_on_event(cfg.exclude_event_types, event):
if minimal and not cfg.minimal:
continue
for build_type in cfg.build_type:
entries.append(
@@ -292,12 +282,12 @@ if __name__ == "__main__":
action="store_true",
)
parser.add_argument(
"-e",
"--event",
help="The GitHub event name that triggered the workflow (e.g. 'push', "
"'pull_request'). Configs are filtered by their 'event_type'. If "
"omitted, no filtering is applied.",
default=None,
"-m",
"--minimal",
help="Emit only the minimal matrix (the configs flagged 'minimal'), "
"used for pull requests by default. If omitted, the full matrix is "
"emitted.",
action="store_true",
)
args = parser.parse_args()
@@ -308,15 +298,15 @@ if __name__ == "__main__":
else:
if args.config in ("linux", None):
matrix += expand_linux_matrix(
LinuxFile.load(THIS_DIR / "linux.json"), args.event
LinuxFile.load(THIS_DIR / "linux.json"), args.minimal
)
if args.config in ("macos", None):
matrix += expand_platform_matrix(
PlatformFile.load(THIS_DIR / "macos.json"), args.event
PlatformFile.load(THIS_DIR / "macos.json"), args.minimal
)
if args.config in ("windows", None):
matrix += expand_platform_matrix(
PlatformFile.load(THIS_DIR / "windows.json"), args.event
PlatformFile.load(THIS_DIR / "windows.json"), args.minimal
)
print(f"matrix={json.dumps({'include': [dataclasses.asdict(e) for e in matrix]})}")

View File

@@ -2,16 +2,30 @@
"image_tag": "sha-e29b523",
"configs": {
"ubuntu": [
{
"compiler": ["clang"],
"build_type": ["Release"],
"arch": ["amd64"],
"minimal": true
},
{
"compiler": ["gcc"],
"build_type": ["Release"],
"arch": ["amd64"],
"minimal": false
},
{
"compiler": ["gcc", "clang"],
"build_type": ["Debug", "Release"],
"arch": ["amd64", "arm64"]
"arch": ["arm64"],
"minimal": false
},
{
"compiler": ["gcc", "clang"],
"build_type": ["Debug", "Release"],
"arch": ["amd64"],
"minimal": false,
"sanitizers": ["address", "undefinedbehavior"]
},
@@ -19,6 +33,7 @@
"compiler": ["gcc"],
"build_type": ["Debug"],
"arch": ["amd64"],
"minimal": true,
"suffix": "coverage",
"extra_cmake_args": "-DUNIT_TEST_REFERENCE_FEE=500 -Dcoverage=ON -Dcoverage_format=xml -DCODE_COVERAGE_VERBOSE=ON -DCMAKE_C_FLAGS=-O0 -DCMAKE_CXX_FLAGS=-O0"
},
@@ -26,6 +41,7 @@
"compiler": ["clang"],
"build_type": ["Debug"],
"arch": ["amd64"],
"minimal": false,
"suffix": "voidstar",
"extra_cmake_args": "-Dvoidstar=ON"
},
@@ -33,6 +49,7 @@
"compiler": ["clang"],
"build_type": ["Release"],
"arch": ["amd64"],
"minimal": false,
"suffix": "reffee",
"extra_cmake_args": "-DUNIT_TEST_REFERENCE_FEE=1000"
},
@@ -40,9 +57,9 @@
"compiler": ["gcc"],
"build_type": ["Debug"],
"arch": ["amd64"],
"minimal": false,
"suffix": "unity",
"extra_cmake_args": "-Dunity=ON",
"exclude_event_types": ["pull_request"]
"extra_cmake_args": "-Dunity=ON"
}
],
@@ -50,7 +67,9 @@
{
"compiler": ["gcc"],
"build_type": ["Release"],
"arch": ["amd64"]
"arch": ["amd64"],
"minimal": false,
"extra_cmake_args": "-Dvalidator_keys=ON"
}
],
@@ -58,7 +77,9 @@
{
"compiler": ["gcc"],
"build_type": ["Release"],
"arch": ["amd64"]
"arch": ["amd64"],
"minimal": false,
"extra_cmake_args": "-Dvalidator_keys=ON"
}
]
},
@@ -68,6 +89,7 @@
"compiler": ["gcc"],
"build_type": ["Release"],
"arch": ["amd64"],
"minimal": false,
"image": "ghcr.io/xrplf/xrpld/packaging-debian:sha-577d745"
}
],
@@ -77,6 +99,7 @@
"compiler": ["gcc"],
"build_type": ["Release"],
"arch": ["amd64"],
"minimal": false,
"image": "ghcr.io/xrplf/xrpld/packaging-rhel:sha-577d745"
}
]

View File

@@ -4,13 +4,14 @@
"configs": [
{
"build_type": "Release",
"extra_cmake_args": "-DCMAKE_POLICY_VERSION_MINIMUM=3.5"
"extra_cmake_args": "-DCMAKE_POLICY_VERSION_MINIMUM=3.5",
"minimal": true
},
{
"build_type": "Debug",
"extra_cmake_args": "-DCMAKE_POLICY_VERSION_MINIMUM=3.5",
"build_only": true,
"exclude_event_types": ["pull_request"]
"minimal": false
}
]
}

View File

@@ -2,11 +2,11 @@
"platform": "windows/amd64",
"runner": ["self-hosted", "Windows", "dev-box-windows-2026"],
"configs": [
{ "build_type": "Release" },
{ "build_type": "Release", "minimal": true },
{
"build_type": "Debug",
"build_only": true,
"exclude_event_types": ["pull_request"]
"minimal": false
}
]
}

View File

@@ -14,6 +14,7 @@ permissions:
jobs:
main:
if: ${{ !contains(github.event.pull_request.labels.*.name, 'IgnoreConflicts') }}
runs-on: ubuntu-latest
steps:
- name: Check if PRs are dirty

View File

@@ -1,7 +1,11 @@
# This workflow runs all workflows to check, build and test the project on
# various Linux flavors, as well as on MacOS and Windows, on every push to a
# user branch. However, it will not run if the pull request is a draft unless it
# has the 'DraftRunCI' label. For commits to PRs that target a release branch,
# This workflow runs workflows to check, build and test the project
# on every meaningful change on pull_request.
# However, it will not run if the PR is a draft
# unless it has the 'DraftRunCI' or 'Full CI build' label.
#
# By default a PR builds only a minimal matrix.
# The full matrix runs once the PR is labeled "Ready to merge" or "Full CI build".
# For commits to PRs that target a release branch,
# it also uploads the libxrpl recipe to the Conan remote.
name: PR
@@ -15,8 +19,16 @@ on:
- reopened
- synchronize
- ready_for_review
# Trigger on label changes so toggling "Ready to merge" or "Full CI build"
# switches between the minimal and full matrix without needing a new push.
- labeled
- unlabeled
concurrency:
# A single per-ref group with cancel-in-progress means any newer run (a push
# or a label change) supersedes the in-progress one for that ref. Keeping
# exactly one authoritative run per ref ensures a fast do-nothing run can never
# mask a real build's checks.
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -25,11 +37,18 @@ defaults:
shell: bash
jobs:
# This job determines whether the rest of the workflow should run. It runs
# when the PR is not a draft (which should also cover merge-group) or
# has the 'DraftRunCI' label.
# This job determines whether the rest of the workflow should run at all,
# based on the current set of labels: it runs when the PR is not a draft
# (which should also cover merge-group) or has the 'DraftRunCI' or
# 'Full CI build' label. Whether a build then happens, and whether it is the
# minimal or full matrix, is decided further below and in the strategy matrix.
should-run:
if: ${{ !github.event.pull_request.draft || contains(github.event.pull_request.labels.*.name, 'DraftRunCI') }}
if: >-
${{
!github.event.pull_request.draft
|| contains(github.event.pull_request.labels.*.name, 'DraftRunCI')
|| contains(github.event.pull_request.labels.*.name, 'Full CI build')
}}
runs-on: ubuntu-latest
steps:
- name: Checkout repository
@@ -91,15 +110,17 @@ jobs:
# least one of:
# * Any of the files checked in the `changes` step were modified
# * The PR is NOT a draft and is labeled "Ready to merge"
# * The PR is labeled "Full CI build" (draft or not)
# * The workflow is running from the merge queue
id: go
env:
FILES: ${{ steps.changes.outputs.any_changed }}
DRAFT: ${{ github.event.pull_request.draft }}
READY: ${{ contains(github.event.pull_request.labels.*.name, 'Ready to merge') }}
FULL: ${{ contains(github.event.pull_request.labels.*.name, 'Full CI build') }}
MERGE: ${{ github.event_name == 'merge_group' }}
run: |
echo "go=${{ (env.DRAFT != 'true' && env.READY == 'true') || env.FILES == 'true' || env.MERGE == 'true' }}" >>"${GITHUB_OUTPUT}"
echo "go=${{ (env.DRAFT != 'true' && env.READY == 'true') || env.FULL == 'true' || env.FILES == 'true' || env.MERGE == 'true' }}" >>"${GITHUB_OUTPUT}"
cat "${GITHUB_OUTPUT}"
outputs:
go: ${{ steps.go.outputs.go == 'true' }}
@@ -142,7 +163,10 @@ jobs:
package:
needs: [should-run, build-test]
if: ${{ needs.should-run.outputs.go == 'true' }}
# Packaging consumes the debian/rhel release binaries, which are only built
# by the full matrix. Skip it for pull requests that ran only the minimal
# matrix (i.e. not yet labeled "Ready to merge" or "Full CI build").
if: ${{ needs.should-run.outputs.go == 'true' && (github.event_name != 'pull_request' || contains(github.event.pull_request.labels.*.name, 'Ready to merge') || contains(github.event.pull_request.labels.*.name, 'Full CI build')) }}
uses: ./.github/workflows/reusable-package.yml
upload-recipe:

View File

@@ -100,9 +100,10 @@ jobs:
# header files are copied into separate directories by CMake, which will
# otherwise result in cache misses.
CCACHE_SLOPPINESS: include_file_ctime,include_file_mtime
# Determine if coverage and voidstar should be enabled.
# Determine if coverage, voidstar and validator-keys should be enabled.
COVERAGE_ENABLED: ${{ contains(inputs.cmake_args, '-Dcoverage=ON') }}
VOIDSTAR_ENABLED: ${{ contains(inputs.cmake_args, '-Dvoidstar=ON') }}
VALIDATOR_KEYS_ENABLED: ${{ contains(inputs.cmake_args, '-Dvalidator_keys=ON') }}
SANITIZERS_ENABLED: ${{ inputs.sanitizers != '' }}
steps:
- name: Cleanup workspace (macOS and Windows)
@@ -124,7 +125,7 @@ jobs:
- name: Check tools
env:
CHECK_TOOLS_SKIP_CLONE: "1"
run: ./bin/check-tools.sh
run: ./bin/check-tools.sh || true
- name: Print build environment
uses: XRPLF/actions/print-build-env@59dec886e4afb05a1724443af08baccbc045b574
@@ -247,6 +248,22 @@ jobs:
retention-days: 3
if-no-files-found: error
- name: Run the validator-keys tests
if: ${{ env.VALIDATOR_KEYS_ENABLED == 'true' }}
working-directory: ${{ env.BUILD_DIR }}
run: ./validator-keys --unittest
- name: Upload the validator-keys binary
if: ${{ github.event.repository.visibility == 'public' && env.VALIDATOR_KEYS_ENABLED == 'true' }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: validator-keys-${{ inputs.config_name }}
path: |
${{ env.BUILD_DIR }}/validator-keys
${{ env.BUILD_DIR }}/validator-keys-LICENSE
retention-days: 3
if-no-files-found: error
- name: Upload the test binary (Linux)
if: ${{ github.event.repository.visibility == 'public' && runner.os == 'Linux' }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
@@ -366,7 +383,7 @@ jobs:
--target coverage
- name: Upload coverage report
if: ${{ github.repository == 'XRPLF/rippled' && !inputs.build_only && env.COVERAGE_ENABLED == 'true' }}
if: ${{ github.repository_owner == 'XRPLF' && !inputs.build_only && env.COVERAGE_ENABLED == 'true' }}
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
disable_search: true

View File

@@ -1,7 +1,7 @@
# Build Linux packages (DEB and RPM) from pre-built binary artifacts.
# Discovers which configurations to package from linux.json (configs in
# "package_configs") and fans out one job per distro. Only linux/amd64 is
# supported; the runner is hardcoded in the job below.
# Build Linux packages (DEB and RPM) from pre-built binary artifacts (xrpld and
# validator-keys). Discovers which configurations to package from linux.json
# (configs in "package_configs") and fans out one job per distro. Only
# linux/amd64 is supported; the runner is hardcoded in the job below.
name: Package
on:
@@ -45,7 +45,7 @@ jobs:
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.generate-matrix.outputs.matrix) }}
name: "${{ matrix.artifact_name }}"
name: "${{ matrix.xrpld_artifact_name }}"
permissions:
contents: read
runs-on: ["self-hosted", "Linux", "X64", "heavy"]
@@ -56,14 +56,20 @@ jobs:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Download pre-built binary
- name: Download pre-built xrpld binary
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: ${{ matrix.artifact_name }}
name: ${{ matrix.xrpld_artifact_name }}
path: ${{ env.BUILD_DIR }}
- name: Make binary executable
run: chmod +x "${BUILD_DIR}/xrpld"
- name: Download pre-built validator-keys binary
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: ${{ matrix.validator_keys_artifact_name }}
path: ${{ env.BUILD_DIR }}
- name: Make binaries executable
run: chmod +x "${BUILD_DIR}/xrpld" "${BUILD_DIR}/validator-keys"
- name: Build package
env:
@@ -73,7 +79,7 @@ jobs:
- name: Upload package artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ matrix.artifact_name }}-pkg
name: ${{ matrix.xrpld_artifact_name }}-pkg
path: |
${{ env.BUILD_DIR }}/debbuild/*.deb
${{ env.BUILD_DIR }}/debbuild/*.ddeb

View File

@@ -35,5 +35,8 @@ jobs:
id: generate
env:
GENERATE_CONFIG: ${{ inputs.os != '' && format('--config={0}', inputs.os) || '' }}
GENERATE_EVENT: ${{ github.event_name }}
run: ./generate.py ${GENERATE_CONFIG} --event="${GENERATE_EVENT}" >>"${GITHUB_OUTPUT}"
# Run only the minimal matrix for pull requests that are not yet
# labeled "Ready to merge" or "Full CI build". Any other event (merge
# queue, push, schedule, manual dispatch) runs the full matrix.
GENERATE_MINIMAL: ${{ (github.event_name == 'pull_request' && !contains(github.event.pull_request.labels.*.name, 'Ready to merge') && !contains(github.event.pull_request.labels.*.name, 'Full CI build')) && '--minimal' || '' }}
run: ./generate.py ${GENERATE_CONFIG} ${GENERATE_MINIMAL} >>"${GITHUB_OUTPUT}"

View File

@@ -32,6 +32,11 @@ repos:
# as standalone translation units, so they have no compile_commands.json
# entry to lint (verify_headers checks them transitively).
exclude: '^include/xrpl/protocol_autogen|\.ipp$'
# run-clang-tidy --fix may edit headers included by files it is not run on,
# so pre-commit must not split the files across parallel hook invocations.
# The script determines the staged files itself and lets run-clang-tidy
# handle parallelism internally.
pass_filenames: false
- id: fix-include-style
name: fix include style
entry: ./bin/pre-commit/fix_include_style.py

View File

@@ -28,6 +28,9 @@ This section contains changes targeting a future version.
### Additions
- `account_tx`: Added an optional `delegate` request object to filter delegated transactions. The object requires `delegate_filter`, which must be either `actor` for transactions owned by the requested account but signed by another account, or `authorizer` for transactions signed by the requested account on behalf of another account. The optional `counter_party` account narrows the results to a specific signer/delegate for `actor` or a specific owner/delegator for `authorizer`. Malformed `delegate`, `delegate_filter`, and `counter_party` values return standard invalid field errors, and invalid account IDs return `actMalformed`.
When paginating delegate-filtered queries, a marker from a delegate-filtered query includes a `delegate` flag and is only valid for follow-up requests that also supply `delegate` (mixing marker conventions returns `invalidParams`). Because filtering is applied after the ledger scan, a page may contain fewer results than `limit` (possibly zero) while still returning a marker, so callers must continue until no marker is present.
- `ledger_entry`, `account_objects`: The `Delegate` ledger entry now includes an optional `DestinationNode` field, which stores the index into the authorized account's owner directory. This field is present on entries created after bidirectional directory tracking was introduced and may appear in RPC responses for those entries. ([#6681](https://github.com/XRPLF/rippled/pull/6681))
- `server_definitions`: Added the following new sections to the response ([#6321](https://github.com/XRPLF/rippled/pull/6321)):

View File

@@ -138,8 +138,10 @@ endif()
include(XrplCore)
include(XrplProtocolAutogen)
include(XrplInstall)
include(XrplPackaging)
include(XrplValidatorKeys)
# Must come after XrplValidatorKeys: the 'package' target depends on the
# validator-keys target existing.
include(XrplPackaging)
if(tests)
include(CTest)

View File

@@ -84,7 +84,9 @@ If you create new source files, they must be organized as follows:
- All other non-test files must go under `src/xrpld`.
- All test source files must go under `src/test`.
The source must be formatted according to the style guide below.
The source must be formatted according to the style guide below. The easiest
way to satisfy this is to install the [`pre-commit`](#pre-commit-hooks) hooks,
which format and lint your changes automatically on every commit.
Header includes must be [levelized](.github/scripts/levelization).
@@ -212,13 +214,61 @@ This is a non-exhaustive list of recommended style guidelines. These are
not always strictly enforced and serve as a way to keep the codebase
coherent rather than a set of _thou shalt not_ commandments.
## Pre-commit hooks
We use the [`pre-commit`](https://pre-commit.com/) framework to run the
formatting and linting tools that keep the codebase consistent. `pre-commit`
runs each tool configured in
[`.pre-commit-config.yaml`](./.pre-commit-config.yaml) in its own isolated
environment, so you don't need to install most of the individual tools
yourself. The version of each hook sourced from an external repository
(`clang-format`, `gersemi`, etc.) is pinned in that file, so running the hooks
locally uses exactly the same versions as CI. A few `local` hooks — most notably
`clang-tidy` — run tools from your own environment; see
[Installing clang-tidy](#installing-clang-tidy) for how to get those.
To get started, install `pre-commit` and enable the git hook scripts:
```bash
pip install pre-commit
pre-commit install
```
Once installed, the hooks run automatically on your staged files every time you
`git commit`. You can also run them on demand:
```bash
# Run all hooks against only the staged files
pre-commit run
# Run all hooks against every file in the repository
pre-commit run --all-files
# Run a single hook (e.g. clang-format) against all files
pre-commit run clang-format --all-files
```
The hooks configured in this repository include, among others:
- `clang-format` — C++/proto formatting (see [Formatting](#formatting))
- `clang-tidy` — C++ static analysis (see [Clang-tidy](#clang-tidy)); opt in with `TIDY=1`
- `fix-include-style`, `fix-pragma-once`, `check-doxygen-style` — C++ hygiene
- `gersemi` — CMake formatting
- `prettier`, `black`, `shfmt` — formatting for JavaScript/JSON/Markdown, Python, and shell
- `cspell` — spell checking
The same hooks run in CI on every pull request, so running them locally before
you push helps you avoid CI failures.
## Formatting
All code must conform to `clang-format` version 22,
according to the settings in [`.clang-format`](./.clang-format),
unless the result would be unreasonably difficult to read or maintain.
To demarcate lines that should be left as-is, surround them with comments like
this:
All code must conform to `clang-format`, according to the settings in
[`.clang-format`](./.clang-format), unless the result would be unreasonably
difficult to read or maintain. The `clang-format` version is pinned in
[`.pre-commit-config.yaml`](./.pre-commit-config.yaml), so the
[`pre-commit`](#pre-commit-hooks) hook always formats with the same version as
CI. To demarcate lines that should be left as-is, surround them with comments
like this:
```
// clang-format off
@@ -226,9 +276,21 @@ this:
// clang-format on
```
You can format individual files in place by running `clang-format -i <file>...`
The easiest way to format your changes is to let the `pre-commit` hook run
automatically on commit, or to run it manually:
```bash
pre-commit run clang-format --all-files
```
You can also format individual files in place by running `clang-format -i <file>...`
from any directory within this project.
> [!NOTE]
> This uses whatever `clang-format` version is installed locally, which may
> differ from the pinned version used by `pre-commit` and CI, so the results
> can vary.
There is a Continuous Integration job that runs clang-format on pull requests. If the code doesn't comply, a patch file that corrects auto-fixable formatting issues is generated.
To download the patch file:
@@ -239,13 +301,6 @@ To download the patch file:
4. Download the zip file and extract it to your local git repository. Run `git apply [patch-file-name]`.
5. Commit and push.
You can install a pre-commit hook to automatically run `clang-format` before every commit:
```
pip3 install pre-commit
pre-commit install
```
## Clang-tidy
All code must pass `clang-tidy` checks according to the settings in [`.clang-tidy`](./.clang-tidy).
@@ -267,7 +322,7 @@ Before running clang-tidy, you must build the project to generate required files
#### Via pre-commit (recommended)
If you have already installed the pre-commit hooks (see above), you can run clang-tidy on your staged files using:
If you have already installed the [`pre-commit`](#pre-commit-hooks) hooks, you can run clang-tidy on your staged files using:
```
TIDY=1 pre-commit run clang-tidy

View File

@@ -110,6 +110,23 @@ if [ "${os}" = "linux" ] || [ "${os}" = "macos" ]; then
fi
fi
# Rust toolchain. Part of the Nix commonPackages, so available on both Linux
# and macOS. The cargo plugins are invoked through cargo (`cargo <sub>`), which
# resolves the matching `cargo-<sub>` binary on PATH; `--version` is offline and
# does not need a Cargo project.
if [ "${os}" = "linux" ] || [ "${os}" = "macos" ]; then
echo
echo "Rust toolchain:"
check cargo
check cargo-audit cargo audit --version
check cargo-llvm-cov cargo llvm-cov --version
check cargo-nextest cargo nextest --version
check clippy clippy-driver --version
check rust-analyzer
check rustc
check rustfmt
fi
# GCC is the default compiler on Linux. macOS uses the system Apple Clang
# instead, so GCC/g++/gcov are not expected there.
if [ "${os}" = "linux" ]; then

View File

@@ -1,27 +1,46 @@
#!/usr/bin/env python3
"""Pre-commit hook that runs clang-tidy on changed files using run-clang-tidy.
"""Pre-commit hook that runs clang-tidy on staged files using run-clang-tidy.
The set of files is chosen by pre-commit (see .pre-commit-config.yaml), which
filters to C/C++ sources and excludes `.ipp` fragments. Headers are linted
directly: the `verify_headers` build option (ON by default) compiles every
`.h`/`.hpp` on its own, so each header is the main file of its own
compile_commands.json entry and run-clang-tidy can analyse it just like a
`.cpp`.
The script determines the staged files itself (see `pass_filenames: false` in
.pre-commit-config.yaml) so run-clang-tidy is run once and handles parallelism
internally: pre-commit would otherwise split the files across parallel hook
invocations that race when fixes edit a shared header.
Fixes are collected with `-export-fixes` and applied by clang-apply-replacements
in a separate step rather than with run-clang-tidy's `-fix`. The `add_module`
build isolates each module's headers behind a per-module symlink directory
(build/modules/<module>/...), so a header reachable from several translation
units is referenced through different paths that all resolve to the same source
file. clang-apply-replacements deduplicates identical replacements by their
literal path, so those paths must be canonicalised to the real source path
first; otherwise the same fix is applied once per path and corrupts the header.
"""
from __future__ import annotations
import os
import re
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
CLANG_TIDY_VERSION = 22
# Extensions run-clang-tidy can analyse: `.cpp` translation units and, thanks to
# the `verify_headers` build option, `.h`/`.hpp` headers (each has its own
# compile_commands.json entry). `.ipp` fragments have no entry and are skipped.
TIDY_EXTENSIONS = {".cpp", ".h", ".hpp"}
def find_run_clang_tidy() -> str | None:
for candidate in (f"run-clang-tidy-{CLANG_TIDY_VERSION}", "run-clang-tidy"):
# A single-quoted `FilePath:` entry in an -export-fixes YAML file, allowing the
# `- ` marker that precedes it inside a `Replacements:` sequence. clang-tidy
# emits paths single-quoted and doubles any embedded quote per YAML rules.
FILEPATH_RE = re.compile(r"^(\s*(?:-\s+)?FilePath:\s*)'((?:[^']|'')*)'\s*$")
def find_tool(name: str) -> str | None:
for candidate in (f"{name}-{CLANG_TIDY_VERSION}", name):
if path := shutil.which(candidate):
return path
return None
@@ -35,23 +54,43 @@ def find_build_dir(repo_root: Path) -> Path | None:
return None
def staged_files(repo_root: Path) -> list[Path]:
"""Return absolute paths of staged, lint-able C/C++ files.
`--diff-filter=d` excludes deletions so we never lint a removed file.
"""
output = subprocess.check_output(
["git", "diff", "--staged", "--name-only", "--diff-filter=d", "--"]
+ [f"*{ext}" for ext in TIDY_EXTENSIONS],
text=True,
cwd=repo_root,
)
return [repo_root / rel for rel in output.splitlines() if rel]
def canonicalize_fix_paths(fixes_dir: Path) -> None:
"""Rewrite every `FilePath` in the exported fixes to its real source path.
A header included through a module's isolation symlink is recorded under that
symlink's path; collapsing all paths to the same real file lets
clang-apply-replacements recognise the per-translation-unit duplicates and
apply each fix once.
"""
for yaml in fixes_dir.glob("*.yaml"):
lines = []
for line in yaml.read_text().splitlines():
if m := FILEPATH_RE.match(line):
path = m.group(2).replace("''", "'")
real = os.path.realpath(path).replace("'", "''")
line = f"{m.group(1)}'{real}'"
lines.append(line)
yaml.write_text("\n".join(lines) + "\n")
def main():
if not os.environ.get("TIDY"):
return 0
files = sys.argv[1:]
if not files:
return 0
run_clang_tidy = find_run_clang_tidy()
if not run_clang_tidy:
print(
f"clang-tidy check failed: TIDY is enabled but neither "
f"'run-clang-tidy-{CLANG_TIDY_VERSION}' nor 'run-clang-tidy' was found in PATH.",
file=sys.stderr,
)
return 1
repo_root = Path(
subprocess.check_output(
["git", "rev-parse", "--show-toplevel"],
@@ -59,6 +98,29 @@ def main():
text=True,
).strip()
)
files = staged_files(repo_root)
if not files:
return 0
run_clang_tidy = find_tool("run-clang-tidy")
clang_apply_replacements = find_tool("clang-apply-replacements")
missing = [
name
for name, path in (
("run-clang-tidy", run_clang_tidy),
("clang-apply-replacements", clang_apply_replacements),
)
if not path
]
if missing:
print(
f"clang-tidy check failed: TIDY is enabled but {' and '.join(missing)} "
f"was not found in PATH (tried the '-{CLANG_TIDY_VERSION}' suffix too).",
file=sys.stderr,
)
return 1
build_dir = find_build_dir(repo_root)
if not build_dir:
print(
@@ -68,11 +130,23 @@ def main():
)
return 1
result = subprocess.run(
[run_clang_tidy, "-quiet", "-p", str(build_dir), "-fix", "-allow-no-checks"]
+ files
)
return result.returncode
with tempfile.TemporaryDirectory() as fixes_dir:
result = subprocess.run(
[
run_clang_tidy,
"-quiet",
"-p",
build_dir,
"-export-fixes",
fixes_dir,
"-allow-no-checks",
]
+ files
)
canonicalize_fix_paths(Path(fixes_dir))
applied = subprocess.run([clang_apply_replacements, fixes_dir])
return result.returncode or applied.returncode
if __name__ == "__main__":

View File

@@ -488,6 +488,17 @@
# Must be a number between 100 and 1000, defaults to 250
#
#
# [max_subscriptions_per_connection]
#
# Maximum number of account, real-time account, and account-history
# subscriptions a single client connection may hold at once. Bounds the
# per-connection state torn down when the connection disconnects. Book
# subscriptions are tracked separately and are not counted here.
#
# Defaults to 100000 if not set; large enough for legitimate power users
# such as block explorers.
#
#
# [overlay]
#
# Controls settings related to the peer to peer overlay.
@@ -538,6 +549,45 @@
# only be used for local testing and debugging. Do not disable
# on mainnet.
#
# max_untrusted_count = <number>
#
# The number of manifests the server keeps for validators it does not
# list, and the number it sends and processes in a single peer protocol
# message. Once the server holds this many, a manifest for a new
# unlisted validator is rejected, so peer gossip cannot grow the cache
# without end.
#
# This option can take any value between 50 and 1000, inclusive. If
# the option is not present the server uses its built-in value.
#
# The current default (which is subject to change) is 300.
#
# max_trusted_count = <number>
#
# The number of manifests for listed validators to allow for when
# sizing peer protocol messages. Manifests for listed validators are
# never dropped, whether sending or receiving, because doing so would
# delay a validator key change reaching this server. Set this above the
# number of validators the server lists.
#
# Together the two counts above set the largest manifest message the
# server accepts: bigger messages are discarded without reading them,
# and without penalising the sender. Raising either means the server
# accepts and sends bigger messages than a peer using the defaults, and
# those peers will discard what this server sends. Lowering either below
# what peers send makes this server discard their manifest messages,
# which it does without recording anything.
#
# This option can take any value between 50 and 1000, inclusive. If
# the option is not present the server uses its built-in value.
#
# The current default (which is subject to change) is 300.
#
# NOTE: These two options (max_untrusted_count and max_trusted_count)
# are transitional. They exist to bound manifest-message size and cache
# growth during the network upgrade. They may be removed in a future
# release once the fleet has upgraded, and should not be relied upon as
# stable configuration.
#
# [transaction_queue] EXPERIMENTAL
#

View File

@@ -3,9 +3,9 @@
The Nix-based CI image links binaries against an ELF interpreter (loader)
that lives in the Nix store, so the resulting binaries don't run elsewhere
(including once installed from the .deb package). `patch_nix_binary` adds a
POST_BUILD step that resets the interpreter to the system default loader and
drops the rpath.
(including once installed from the .deb package). `patch_nix_binary` resets
the interpreter to the system default loader and drops the rpath, once the
binary has been linked.
This is only active inside the Nix-based image, detected by the presence of
/tmp/loader-path.sh (shipped by that image, resolves the default loader). It
@@ -41,13 +41,38 @@ function(patch_nix_binary target)
if(NOT PATCH_NIX_BINARIES)
return()
endif()
add_custom_command(
TARGET ${target}
POST_BUILD
COMMAND
"${PATCHELF_COMMAND}" --set-interpreter "${DEFAULT_LOADER_PATH}"
--remove-rpath "$<TARGET_FILE:${target}>"
COMMENT "Patching ${target}: set default loader, remove rpath"
VERBATIM
set(patch_command
"${PATCHELF_COMMAND}"
--set-interpreter
"${DEFAULT_LOADER_PATH}"
--remove-rpath
"$<TARGET_FILE:${target}>"
)
set(comment "Patching ${target}: set default loader, remove rpath")
# POST_BUILD is the cheap way to do this: it runs only when the binary is
# relinked. It is also only available in the directory that defined the
# target, so for a target from elsewhere (e.g. a FetchContent subproject)
# fall back to a custom target that runs after the binary is linked. That
# one runs on every build, which is harmless because patchelf is idempotent.
get_target_property(target_source_dir ${target} SOURCE_DIR)
if("${target_source_dir}" STREQUAL "${CMAKE_CURRENT_SOURCE_DIR}")
add_custom_command(
TARGET ${target}
POST_BUILD
COMMAND ${patch_command}
COMMENT "${comment}"
VERBATIM
)
else()
add_custom_target(
${target}-patch-nix
ALL
COMMAND ${patch_command}
COMMENT "${comment}"
VERBATIM
)
add_dependencies(${target}-patch-nix ${target})
endif()
endfunction()

View File

@@ -25,6 +25,19 @@ if(NOT (RPMBUILD_EXECUTABLE OR DPKG_BUILDPACKAGE_EXECUTABLE))
return()
endif()
if(NOT TARGET xrpld)
message(STATUS "xrpld=ON is required; 'package' target not available")
return()
endif()
if(NOT TARGET validator-keys)
message(
STATUS
"validator_keys=ON is required; 'package' target not available"
)
return()
endif()
set(package_env
SRC_DIR=${CMAKE_SOURCE_DIR}
BUILD_DIR=${CMAKE_BINARY_DIR}
@@ -37,7 +50,7 @@ add_custom_target(
${CMAKE_COMMAND} -E env ${package_env}
${CMAKE_SOURCE_DIR}/package/build_pkg.sh
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
DEPENDS xrpld
DEPENDS xrpld validator-keys
COMMENT "Building Linux package (deb/rpm inferred from host tooling)"
VERBATIM
)

View File

@@ -5,22 +5,39 @@ option(
)
if(validator_keys)
git_branch(current_branch)
# default to tracking VK master branch unless we are on release
if(NOT (current_branch STREQUAL "release"))
set(current_branch "master")
endif()
message(STATUS "Tracking ValidatorKeys branch: ${current_branch}")
# Own the install destination below rather than relying on another module
# having pulled this in first.
include(GNUInstallDirs)
# Pinned to an exact commit, not a branch: the tool ships inside our
# packages, so the same xrpld version must always package the same
# validator-keys. Bump this deliberately.
set(validator_keys_commit "4c0fb75eec9601c711645998c904507e87e910ae")
message(STATUS "Using ValidatorKeys commit: ${validator_keys_commit}")
FetchContent_Declare(
validator_keys
GIT_REPOSITORY https://github.com/ripple/validator-keys-tool.git
GIT_TAG "${current_branch}"
GIT_TAG "${validator_keys_commit}"
)
FetchContent_MakeAvailable(validator_keys)
# The tool's own CMakeLists excludes the target from 'all' when it is built
# as a subproject. Undo that, so validator_keys=ON really does build it.
set_target_properties(
validator-keys
PROPERTIES RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}"
PROPERTIES
RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}"
EXCLUDE_FROM_ALL OFF
EXCLUDE_FROM_DEFAULT_BUILD OFF
)
# We ship this binary, so like xrpld it must not keep the Nix store's ELF
# loader, or it cannot run on the target distro at all.
patch_nix_binary(validator-keys)
configure_file(
"${validator_keys_SOURCE_DIR}/LICENSE"
"${CMAKE_BINARY_DIR}/validator-keys-LICENSE"
COPYONLY
)
install(TARGETS validator-keys RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
endif()

View File

@@ -10,22 +10,22 @@
"rocksdb/10.5.1#4a197eca381a3e5ae8adf8cffa5aacd0%1782392413.075713",
"re2/20251105#8579cfd0bda4daf0683f9e3898f964b4%1782392402.431897",
"protobuf/6.33.5#ff253ead763bd8d9904a52979cd21e81%1782392410.233933",
"openssl/3.6.3#1163d4ddc603907084d08a6a0c6e580f%1782307150.583886",
"openssl/3.6.3#f806de8933e3bf6f01016c6a888cee2e%1783945160.863288",
"nudb/2.0.9#11149c73f8f2baff9a0198fe25971fc7%1782392402.297166",
"mpt-crypto/0.4.0-rc2#a580f2f9ad0e795de696aa62d54fb9af%1782425834.488828",
"mpt-crypto/1.0.2#b313cef0c1a493eb970ad185b2e9bab7%1784285108.866483",
"lz4/1.10.0#982d9b673900f665a1da109e09c17cab%1782392402.164188",
"libiconv/1.17#9923bc6dc6f106646d6967e0039a5ada%1782392792.775744",
"libbacktrace/cci.20210118#a7691bfccd8caaf66309df196790a5a1%1782392402.420732",
"libarchive/3.8.7#c446109bd1f1d8ba7936c94189bc50e6%1782392403.066892",
"jemalloc/5.3.1#1fc58d55316041f10fbc1e8a2eae632a%1776700028.228",
"gtest/1.17.0#5224b3b3ff3b4ce1133cbdd27d53ee7d%1782392402.791979",
"grpc/1.81.1#5217e6ef0544c42b46f4af35d5e7f649%1782307148.845616",
"grpc/1.81.1#f729f6d75992d20f9c72828e9142d62f%1783945160.094135",
"ed25519/2015.03#ae761bdc52730a843f0809bdf6c1b1f6%1782307148.15562",
"date/3.0.4#862e11e80030356b53c2c38599ceb32b%1782392402.538492",
"c-ares/1.34.6#545240bb1c40e2cacd4362d6b8967650%1782392402.681654",
"bzip2/1.0.8#c470882369c2d95c5c77e970c0c7e321%1782392402.296732",
"boost/1.91.0#ea540ca2133d831b560036aa24dece3c%1782392419.475605",
"abseil/20250127.0#bb0baf1f362bc4a725a24eddd419b8f7%1782307147.395833"
"abseil/20250127.0#9ef01c1451a8340f9022e46238c0fbb6%1783945159.651047"
],
"build_requires": [
"zlib/1.3.2#1cb806da49011867778ffb6ac7190fcb%1782392402.122708",
@@ -38,7 +38,7 @@
"b2/5.4.2#ffd6084a119587e70f11cd45d1a386e2%1782392402.624226",
"automake/1.16.5#b91b7c384c3deaa9d535be02da14d04f%1755524470.56",
"autoconf/2.71#51077f068e61700d65bb05541ea1e4b0%1731054366.86",
"abseil/20250127.0#bb0baf1f362bc4a725a24eddd419b8f7%1782307147.395833"
"abseil/20250127.0#9ef01c1451a8340f9022e46238c0fbb6%1783945159.651047"
],
"python_requires": [],
"overrides": {

View File

@@ -134,7 +134,7 @@ class Xrpl(ConanFile):
if self.options.jemalloc:
self.requires("jemalloc/5.3.1")
self.requires("lz4/1.10.0", force=True)
self.requires("mpt-crypto/0.4.0-rc2", transitive_headers=True)
self.requires("mpt-crypto/1.0.2", transitive_headers=True)
self.requires("protobuf/6.33.5", force=True)
if self.options.rocksdb:
self.requires("rocksdb/10.5.1")

View File

@@ -364,6 +364,8 @@ public:
static constexpr internalrep kMaxRep = std::numeric_limits<rep>::max();
static_assert(kMaxRep == 9'223'372'036'854'775'807);
static_assert(-kMaxRep == std::numeric_limits<rep>::min() + 1);
static constexpr internalrep kMaxRepUp = ((kMaxRep / 10) + 1) * 10;
static_assert(kMaxRepUp == 9'223'372'036'854'775'810ULL);
// May need to make unchecked private
struct Unchecked
@@ -591,6 +593,13 @@ public:
std::pair<T, int>
normalizeToRange() const;
// Safely convert rep (int64) mantissa to internalrep (uint64). If the rep
// is negative, returns the positive value. This takes a little extra work
// because converting std::numeric_limits<std::int64_t>::min() flirts with
// UB, and can vary across compilers.
static internalrep
externalToInternal(rep mantissa);
private:
static thread_local RoundingMode mode;
// The available ranges for mantissa
@@ -645,13 +654,6 @@ private:
// exponent could go out of range, so it will be checked.
[[nodiscard]] Number
shiftExponent(int exponentDelta) const;
// Safely convert rep (int64) mantissa to internalrep (uint64). If the rep
// is negative, returns the positive value. This takes a little extra work
// because converting std::numeric_limits<std::int64_t>::min() flirts with
// UB, and can vary across compilers.
static internalrep
externalToInternal(rep mantissa);
};
constexpr Number::Number(bool negative, internalrep mantissa, int exponent, Unchecked) noexcept

View File

@@ -3,6 +3,9 @@
#include <xrpl/basics/IntrusivePointer.ipp>
#include <xrpl/basics/Log.h> // IWYU pragma: keep
#include <xrpl/basics/TaggedCache.h>
#include <xrpl/basics/scope.h>
#include <algorithm>
namespace xrpl {
@@ -601,8 +604,42 @@ TaggedCache<Key, T, IsKeyCache, SharedWeakUnionPointer, SharedPointerType, Hash,
std::vector<key_type> v;
{
std::scoped_lock const lock(mutex_);
v.reserve(cache_.size());
// Keep track of how many iterations are needed. Exit the loop if the number of retries gets
// absurd. (Note that if this somehow ever happens, one more allocation will be done under
// lock, which is undesirable, but really should be almost impossible.)
std::size_t allocationIterations = 0;
std::unique_lock lock(mutex_);
for (auto size = cache_.size(); v.capacity() < size && allocationIterations < 20;
size = cache_.size())
{
ScopeUnlock const unlock(lock);
if (allocationIterations > 0)
{
JLOG(journal_.info())
<< "getKeys(): Cache grew beyond allocated capacity after "
<< allocationIterations << " prior attempt(s). Have " << v.capacity()
<< ", need " << size << ". Retrying allocation";
}
// Allocate the current size plus a little extra, in case the cache grows while
// allocating. Each time another allocation is needed, the extra also gets bigger until
// it ultimately doubles the size + 1.
constexpr std::size_t baseShift = 5;
auto const bufferOffset = std::min(allocationIterations, std::size_t{baseShift});
auto const bufferShift = baseShift - bufferOffset;
size += (size >> bufferShift) + 1;
v.reserve(size);
++allocationIterations;
}
if (v.capacity() < cache_.size())
{
// LCOV_EXCL_START
UNREACHABLE("xrpl::TaggedCache::getKeys(): failed to allocate sufficient capacity");
v.reserve(cache_.size());
// LCOV_EXCL_STOP
}
XRPL_ASSERT(lock.owns_lock(), "xrpl::TaggedCache::getKeys(): owns lock");
XRPL_ASSERT(
v.capacity() >= cache_.size(), "xrpl::TaggedCache::getKeys(): sufficient capacity");
for (auto const& _ : cache_)
v.push_back(_.first);
}

View File

@@ -41,6 +41,35 @@
namespace xrpl {
namespace base64 {
/**
* Returns the maximum number of characters needed to base64-encode @p nBytes bytes.
*
* @param nBytes Number of input bytes.
* @return Size of the encoded string, including padding.
*/
constexpr std::size_t
encodedSize(std::size_t const nBytes)
{
return 4 * ((nBytes + 2) / 3);
}
/**
* Returns the maximum number of bytes a base64 string of @p numChars characters
* decodes to.
*
* @param numChars Number of base64 characters.
* @return Upper bound on the number of decoded bytes.
*/
constexpr std::size_t
decodedSize(std::size_t const numChars)
{
return ((numChars / 4) * 3) + 2;
}
} // namespace base64
std::string
base64Encode(std::uint8_t const* data, std::size_t len);

View File

@@ -308,7 +308,9 @@ public:
XRPL_ASSERT(
c.size() * sizeof(typename Container::value_type) == size(),
"xrpl::BaseUInt::fromRaw(Container auto) : input size match");
std::memcpy(result.data_.data(), c.data(), size());
std::size_t const canCopy =
std::min(size(), c.size() * sizeof(typename Container::value_type));
std::memcpy(result.data_.data(), c.data(), canCopy);
return result;
}
@@ -322,7 +324,11 @@ public:
XRPL_ASSERT(
c.size() * sizeof(typename Container::value_type) == size(),
"xrpl::BaseUInt::operator=(Container auto) : input size match");
std::memcpy(data_.data(), c.data(), size());
std::size_t const canCopy =
std::min(size(), c.size() * sizeof(typename Container::value_type));
if (canCopy < size())
*this = beast::kZero;
std::memcpy(data_.data(), c.data(), canCopy);
return *this;
}

View File

@@ -358,12 +358,13 @@ template <class = void>
bool
tokenInList(boost::string_ref const& value, boost::string_ref const& token)
{
for (auto const& item : makeList(value))
{
if (ciEqual(item, token))
return true;
}
return false;
auto const list = makeList(value);
// ListIterator is not default-constructible, so it does not model a std::ranges
// sentinel/range; the classic std::any_of (which only needs an input iterator)
// is used instead.
// NOLINTNEXTLINE(modernize-use-ranges)
return std::any_of(
list.begin(), list.end(), [&token](auto const& item) { return ciEqual(item, token); });
}
template <bool IsRequest, class Body, class Fields>

View File

@@ -25,6 +25,7 @@ struct Sections
static constexpr auto kLedgerHistory = "ledger_history";
static constexpr auto kLedgerReplay = "ledger_replay";
static constexpr auto kLedgerTxTables = "ledger_tx_tables";
static constexpr auto kMaxSubscriptionsPerConnection = "max_subscriptions_per_connection";
static constexpr auto kMaxTransactions = "max_transactions";
static constexpr auto kNetworkId = "network_id";
static constexpr auto kNetworkQuorum = "network_quorum";
@@ -118,7 +119,9 @@ struct Keys
static constexpr auto kLogInterval = "log_interval";
static constexpr auto kMaxDivergedTime = "max_diverged_time";
static constexpr auto kMaxLedgerCountsToStore = "max_ledger_counts_to_store";
static constexpr auto kMaxTrustedCount = "max_trusted_count";
static constexpr auto kMaxUnknownTime = "max_unknown_time";
static constexpr auto kMaxUntrustedCount = "max_untrusted_count";
static constexpr auto kMaximumTxnInLedger = "maximum_txn_in_ledger";
static constexpr auto kMaximumTxnPerAccount = "maximum_txn_per_account";
static constexpr auto kMemoryLevel = "memory_level";

View File

@@ -41,6 +41,19 @@ public:
std::unique_ptr<NodeStore::Backend>&& newBackend,
std::function<void(std::string const& writableName, std::string const& archiveName)> const&
f) = 0;
/**
* Marks an online-delete rotation as in progress (or completed).
*
* While in flight, a read served by the archive backend is copied
* forward into the writable backend even for ordinary
* (duplicate == false) fetches: the archive is about to be deleted,
* and a node body canonicalized into caches during the rotation
* window would otherwise survive only in RAM once the archive is
* dropped.
*/
virtual void
setRotationInFlight(bool inFlight) = 0;
};
} // namespace xrpl::NodeStore

View File

@@ -9,6 +9,7 @@
#include <xrpl/nodestore/NodeObject.h>
#include <xrpl/nodestore/Scheduler.h>
#include <atomic>
#include <cstdint>
#include <functional>
#include <memory>
@@ -69,11 +70,22 @@ public:
void
sweep() override;
void
setRotationInFlight(bool inFlight) override;
private:
std::shared_ptr<Backend> writableBackend_;
std::shared_ptr<Backend> archiveBackend_;
mutable std::mutex mutex_;
// True between SHAMapStore starting the cache-freshen phase and the
// completion of rotate(). While true, archive hits on ordinary
// (duplicate == false) fetches are copied forward into the writable
// backend; copyForwardCount_ tallies them per rotation for the
// summary line logged at swap.
std::atomic<bool> rotationInFlight_{false};
std::atomic<std::uint64_t> copyForwardCount_{0};
std::shared_ptr<NodeObject>
fetchNodeObject(uint256 const& hash, std::uint32_t, FetchReport& fetchReport, bool duplicate)
override;

View File

@@ -293,14 +293,15 @@ message TMLedgerData {
}
message TMPing {
// Previously used - don't reuse.
reserved 3, 4;
enum pingType {
ptPING = 0; // we want a reply
ptPONG = 1; // this is a reply
}
required pingType type = 1;
optional uint32 seq = 2; // detect stale replies, ensure other side is reading
optional uint64 pingTime = 3; // know when we think we sent the ping
optional uint64 netTime = 4;
optional uint32 seq = 2; // detect stale replies, ensure other side is reading
}
message TMSquelch {

View File

@@ -13,7 +13,6 @@
#include <xrpl/protocol/Protocol.h>
#include <xrpl/protocol/STXChainBridge.h>
#include <xrpl/protocol/UintTypes.h>
#include <xrpl/protocol/jss.h>
#include <array>
#include <cstdint>
@@ -423,21 +422,8 @@ struct KeyletDesc
bool includeInTests{};
};
// This list should include all of the keylet functions that take a single
// AccountID parameter.
std::array<KeyletDesc<AccountID const&>, 6> const kDirectAccountKeylets{
{{.function = &keylet::account, .expectedLEName = jss::AccountRoot, .includeInTests = false},
{.function = &keylet::ownerDir, .expectedLEName = jss::DirectoryNode, .includeInTests = true},
{.function = &keylet::signerList, .expectedLEName = jss::SignerList, .includeInTests = true},
// It's normally impossible to create an item at nftpage_min, but
// test it anyway, since the invariant checks for it.
{.function = &keylet::nftokenPageMin,
.expectedLEName = jss::NFTokenPage,
.includeInTests = true},
{.function = &keylet::nftokenPageMax,
.expectedLEName = jss::NFTokenPage,
.includeInTests = true},
{.function = &keylet::did, .expectedLEName = jss::DID, .includeInTests = true}}};
// This list should include all of the keylet functions that take a single AccountID parameter.
extern std::array<KeyletDesc<AccountID const&>, 6> const kDirectAccountKeylets;
MPTID
makeMptID(std::uint32_t sequence, AccountID const& account);

View File

@@ -190,17 +190,6 @@ enum LedgerEntryType : std::uint16_t {
LSF_FLAG(lsfMPTCanClawback, 0x00000040) \
LSF_FLAG(lsfMPTCanHoldConfidentialBalance, 0x00000080)) \
\
LEDGER_OBJECT(MPTokenIssuanceMutable, \
LSF_FLAG(lsmfMPTCanEnableCanLock, 0x00000002) \
LSF_FLAG(lsmfMPTCanEnableRequireAuth, 0x00000004) \
LSF_FLAG(lsmfMPTCanEnableCanEscrow, 0x00000008) \
LSF_FLAG(lsmfMPTCanEnableCanTrade, 0x00000010) \
LSF_FLAG(lsmfMPTCanEnableCanTransfer, 0x00000020) \
LSF_FLAG(lsmfMPTCanEnableCanClawback, 0x00000040) \
LSF_FLAG(lsmfMPTCannotEnableCanHoldConfidentialBalance, 0x00000080) \
LSF_FLAG(lsmfMPTCanMutateMetadata, 0x00010000) \
LSF_FLAG(lsmfMPTCanMutateTransferFee, 0x00020000)) \
\
LEDGER_OBJECT(MPToken, \
LSF_FLAG2(lsfMPTLocked, 0x00000001) \
LSF_FLAG(lsfMPTAuthorized, 0x00000002) \
@@ -294,6 +283,17 @@ getAllLedgerFlags()
#pragma pop_macro("TO_MAP")
#pragma pop_macro("ALL_LEDGER_FLAGS")
// MPTokenIssuance ImmutableFlags (sfImmutableFlags)
inline constexpr std::uint32_t lsifMPTCanLock = 0x00000002;
inline constexpr std::uint32_t lsifMPTRequireAuth = 0x00000004;
inline constexpr std::uint32_t lsifMPTCanEscrow = 0x00000008;
inline constexpr std::uint32_t lsifMPTCanTrade = 0x00000010;
inline constexpr std::uint32_t lsifMPTCanTransfer = 0x00000020;
inline constexpr std::uint32_t lsifMPTCanClawback = 0x00000040;
inline constexpr std::uint32_t lsifMPTCanHoldConfidentialBalance = 0x00000080;
inline constexpr std::uint32_t lsifMPTMetadata = 0x00010000;
inline constexpr std::uint32_t lsifMPTTransferFee = 0x00020000;
//------------------------------------------------------------------------------
/**

View File

@@ -90,7 +90,11 @@ public:
operator=(STObject&& other);
STObject(SOTemplate const& type, SField const& name);
STObject(SOTemplate const& type, SerialIter& sit, SField const& name);
STObject(
SOTemplate const& type,
SerialIter& sit,
SField const& name,
bool requireCanonicalOrder = false);
STObject(SerialIter& sit, SField const& name, int depth = 0);
STObject(SerialIter&& sit, SField const& name);
explicit STObject(SField const& name);
@@ -123,7 +127,7 @@ public:
set(SOTemplate const&);
bool
set(SerialIter& u, int depth = 0);
set(SerialIter& u, int depth = 0, bool requireCanonicalOrder = false);
[[nodiscard]] SerializedTypeID
getSType() const override;
@@ -229,13 +233,6 @@ public:
[[nodiscard]] AccountID
getAccountID(SField const& field) const;
/**
* The account responsible for the authorization: the delegate when
* sfDelegate is present, otherwise the account.
*/
[[nodiscard]] AccountID
getInitiator() const;
[[nodiscard]] Blob
getFieldVL(SField const& field) const;
[[nodiscard]] STAmount const&

View File

@@ -1,6 +1,7 @@
#pragma once
#include <xrpl/basics/CountedObject.h>
#include <xrpl/basics/UnorderedContainers.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/json/json_value.h>
#include <xrpl/protocol/AccountID.h>
@@ -108,6 +109,9 @@ public:
[[nodiscard]] bool
isType(Type const& pe) const;
[[nodiscard]] size_t
getHash() const;
bool
operator==(STPathElement const& t) const;
@@ -171,12 +175,23 @@ public:
reserve(size_t s);
};
template <class Hasher>
void
hash_append(Hasher& h, STPath const& p) noexcept
{
for (auto const& e : p)
{
beast::hash_append(h, e.getHash());
}
}
//------------------------------------------------------------------------------
// A set of zero or more payment paths
class STPathSet final : public STBase, public CountedObject<STPathSet>
{
std::vector<STPath> value_;
xrpl::hardened_hash_set<STPath> seenHashes_;
public:
STPathSet() = default;
@@ -205,9 +220,6 @@ public:
std::vector<STPath>::const_reference
operator[](std::vector<STPath>::size_type n) const;
std::vector<STPath>::reference
operator[](std::vector<STPath>::size_type n);
[[nodiscard]] std::vector<STPath>::const_iterator
begin() const;
@@ -227,6 +239,9 @@ public:
void
emplaceBack(Args&&... args);
[[nodiscard]] bool
contains(STPath const& path) const;
private:
STBase*
copy(std::size_t n, void* buf) const override;
@@ -515,12 +530,6 @@ STPathSet::operator[](std::vector<STPath>::size_type n) const
return value_[n];
}
inline std::vector<STPath>::reference
STPathSet::operator[](std::vector<STPath>::size_type n)
{
return value_[n];
}
inline std::vector<STPath>::const_iterator
STPathSet::begin() const
{
@@ -549,6 +558,7 @@ inline void
STPathSet::pushBack(STPath const& e)
{
value_.push_back(e);
seenHashes_.emplace(value_.back());
}
template <typename... Args>
@@ -556,6 +566,13 @@ inline void
STPathSet::emplaceBack(Args&&... args)
{
value_.emplace_back(std::forward<Args>(args)...);
seenHashes_.emplace(value_.back());
}
inline bool
STPathSet::contains(STPath const& path) const
{
return seenHashes_.contains(path);
}
} // namespace xrpl

View File

@@ -142,9 +142,26 @@ public:
TxnSql status,
std::string const& escapedMetaData) const;
[[nodiscard]] std::vector<uint256> const&
/**
* The IDs of the inner transactions of a Batch.
*/
[[nodiscard]] std::vector<uint256>
getBatchTransactionIDs() const;
/**
* The inner transactions of a Batch, built and validated at construction.
* Always seated for Batch STTx instances (construction throws if oversized).
*/
[[nodiscard]] std::vector<std::shared_ptr<STTx const>> const&
getBatchTransactions() const;
/**
* The account responsible for the authorization: the delegate when
* sfDelegate is present, otherwise the account.
*/
[[nodiscard]] AccountID
getInitiator() const;
[[nodiscard]] AccountID
getFeePayerID() const;
@@ -166,13 +183,16 @@ private:
checkMultiSign(Rules const& rules, STObject const& sigObject) const;
[[nodiscard]] std::expected<void, std::string>
checkBatchSingleSign(STObject const& batchSigner) const;
checkBatchSingleSign(STObject const& batchSigner, std::vector<uint256> const& txIds) const;
[[nodiscard]] std::expected<void, std::string>
checkBatchMultiSign(STObject const& batchSigner, Rules const& rules) const;
checkBatchMultiSign(
STObject const& batchSigner,
Rules const& rules,
std::vector<uint256> const& txIds) const;
void
buildBatchTxnIds();
buildBatchTxns();
STBase*
copy(std::size_t n, void* buf) const override;
@@ -180,11 +200,11 @@ private:
move(std::size_t n, void* buf) override;
friend class detail::STVar;
std::optional<std::vector<uint256>> batchTxnIds_;
std::optional<std::vector<std::shared_ptr<STTx const>>> batchTxns_;
};
bool
passesLocalChecks(STObject const& st, std::string&);
passesLocalChecks(STTx const& tx, std::string&);
/**
* Sterilize a transaction.

View File

@@ -54,6 +54,22 @@ class STValidation final : public STObject, public CountedObject<STValidation>
NetClock::time_point seenTime_;
public:
/**
* @struct DeserializeOptions
* @brief Options controlling deserialization of a STValidation.
* @var DeserializeOptions::checkSignature
* Whether to verify the data was signed properly
*
* @var DeserializeOptions::requireCanonicalOrder
* Whether to require the fields to be in canonical order
*/
struct DeserializeOptions
{
bool checkSignature;
bool requireCanonicalOrder;
};
/**
* Construct a STValidation from a peer from serialized data.
*
@@ -64,12 +80,12 @@ public:
* that signed the validation. For manifest based
* validators, this should be the NodeID of the master
* public key.
* @param checkSignature Whether to verify the data was signed properly
* @param options Options controlling deserialization
*
* @note Throws if the object is not valid
*/
template <class LookupNodeID>
STValidation(SerialIter& sit, LookupNodeID&& lookupNodeID, bool checkSignature);
STValidation(SerialIter& sit, LookupNodeID&& lookupNodeID, DeserializeOptions options);
/**
* Construct, sign and trust a new STValidation issued by this node.
@@ -163,8 +179,8 @@ private:
};
template <class LookupNodeID>
STValidation::STValidation(SerialIter& sit, LookupNodeID&& lookupNodeID, bool checkSignature)
: STObject(validationFormat(), sit, sfValidation)
STValidation::STValidation(SerialIter& sit, LookupNodeID&& lookupNodeID, DeserializeOptions options)
: STObject(validationFormat(), sit, sfValidation, options.requireCanonicalOrder)
, signingPubKey_([this]() {
auto const spk = getFieldVL(sfSigningPubKey);
@@ -175,7 +191,7 @@ STValidation::STValidation(SerialIter& sit, LookupNodeID&& lookupNodeID, bool ch
}())
, nodeID_(lookupNodeID(signingPubKey_))
{
if (checkSignature && !isValid())
if (options.checkSignature && !isValid())
{
JLOG(debugLog().error()) << "Invalid signature in validation: "
<< getJson(JsonOptions::Values::None);

View File

@@ -152,7 +152,14 @@ inline constexpr FlagValue tfUniversalMask = ~tfUniversal;
\
TRANSACTION(MPTokenIssuanceSet, \
TF_FLAG(tfMPTLock, 0x00000001) \
TF_FLAG(tfMPTUnlock, 0x00000002), \
TF_FLAG(tfMPTUnlock, 0x00000002) \
TF_FLAG(tfMPTSetCanLock, 0x00000004) \
TF_FLAG(tfMPTSetRequireAuth, 0x00000008) \
TF_FLAG(tfMPTSetCanEscrow, 0x00000010) \
TF_FLAG(tfMPTSetCanTrade, 0x00000020) \
TF_FLAG(tfMPTSetCanTransfer, 0x00000040) \
TF_FLAG(tfMPTSetCanClawback, 0x00000080) \
TF_FLAG(tfMPTSetCanHoldConfidentialBalance, 0x00000100), \
MASK_ADJ(0)) \
\
TRANSACTION(NFTokenCreateOffer, \
@@ -356,38 +363,26 @@ inline constexpr FlagValue tfMPTPaymentMask = ~(tfUniversal | tfPartialPayment);
inline constexpr FlagValue tfTrustSetPermissionMask =
~(tfUniversal | tfSetfAuth | tfSetFreeze | tfClearFreeze);
// MPTokenIssuanceCreate MutableFlags:
// Indicating specific fields or flags may be changed after issuance.
inline constexpr FlagValue tmfMPTCanEnableCanLock = lsmfMPTCanEnableCanLock;
inline constexpr FlagValue tmfMPTCanEnableRequireAuth = lsmfMPTCanEnableRequireAuth;
inline constexpr FlagValue tmfMPTCanEnableCanEscrow = lsmfMPTCanEnableCanEscrow;
inline constexpr FlagValue tmfMPTCanEnableCanTrade = lsmfMPTCanEnableCanTrade;
inline constexpr FlagValue tmfMPTCanEnableCanTransfer = lsmfMPTCanEnableCanTransfer;
inline constexpr FlagValue tmfMPTCanEnableCanClawback = lsmfMPTCanEnableCanClawback;
inline constexpr FlagValue tmfMPTCanMutateMetadata = lsmfMPTCanMutateMetadata;
inline constexpr FlagValue tmfMPTCanMutateTransferFee = lsmfMPTCanMutateTransferFee;
inline constexpr FlagValue tmfMPTCannotEnableCanHoldConfidentialBalance =
lsmfMPTCannotEnableCanHoldConfidentialBalance;
inline constexpr FlagValue tmfMPTokenIssuanceCreateMutableMask =
~(tmfMPTCanEnableCanLock | tmfMPTCanEnableRequireAuth | tmfMPTCanEnableCanEscrow |
tmfMPTCanEnableCanTrade | tmfMPTCanEnableCanTransfer | tmfMPTCanEnableCanClawback |
tmfMPTCanMutateMetadata | tmfMPTCanMutateTransferFee |
tmfMPTCannotEnableCanHoldConfidentialBalance);
// MPTokenIssuanceCreate / MPTokenIssuanceSet ImmutableFlags:
// Defines the immutable fields and flags specific to MPTokenIssuance.
inline constexpr FlagValue tifMPTCanLock = lsifMPTCanLock;
inline constexpr FlagValue tifMPTRequireAuth = lsifMPTRequireAuth;
inline constexpr FlagValue tifMPTCanEscrow = lsifMPTCanEscrow;
inline constexpr FlagValue tifMPTCanTrade = lsifMPTCanTrade;
inline constexpr FlagValue tifMPTCanTransfer = lsifMPTCanTransfer;
inline constexpr FlagValue tifMPTCanClawback = lsifMPTCanClawback;
inline constexpr FlagValue tifMPTMetadata = lsifMPTMetadata;
inline constexpr FlagValue tifMPTTransferFee = lsifMPTTransferFee;
inline constexpr FlagValue tifMPTCanHoldConfidentialBalance = lsifMPTCanHoldConfidentialBalance;
inline constexpr FlagValue tifMPTokenIssuanceImmutableMask =
~(tifMPTCanLock | tifMPTRequireAuth | tifMPTCanEscrow | tifMPTCanTrade | tifMPTCanTransfer |
tifMPTCanClawback | tifMPTMetadata | tifMPTTransferFee | tifMPTCanHoldConfidentialBalance);
// MPTokenIssuanceSet MutableFlags:
// Enable mutable capability flags. These flags are one-way: once enabled,
// the corresponding capability cannot be disabled by MPTokenIssuanceSet.
inline constexpr FlagValue tmfMPTSetCanLock = 0x00000001;
inline constexpr FlagValue tmfMPTSetRequireAuth = 0x00000002;
inline constexpr FlagValue tmfMPTSetCanEscrow = 0x00000004;
inline constexpr FlagValue tmfMPTSetCanTrade = 0x00000008;
inline constexpr FlagValue tmfMPTSetCanTransfer = 0x00000010;
inline constexpr FlagValue tmfMPTSetCanClawback = 0x00000020;
inline constexpr FlagValue tmfMPTSetCanHoldConfidentialBalance = 0x00000040;
inline constexpr FlagValue tmfMPTokenIssuanceSetMutableMask =
~(tmfMPTSetCanLock | tmfMPTSetRequireAuth | tmfMPTSetCanEscrow | tmfMPTSetCanTrade |
tmfMPTSetCanTransfer | tmfMPTSetCanClawback | tmfMPTSetCanHoldConfidentialBalance);
// MPTokenIssuanceSet set of flags that is used to enable capabilities on an MPTokenIssuance.
// Used as `txFlags & tfMPTokenIssuanceSetEnableFlagMask` to extract the capability-enabling bits.
inline constexpr FlagValue tfMPTokenIssuanceSetEnableFlagMask = tfMPTSetCanLock |
tfMPTSetRequireAuth | tfMPTSetCanEscrow | tfMPTSetCanTrade | tfMPTSetCanTransfer |
tfMPTSetCanClawback | tfMPTSetCanHoldConfidentialBalance;
// Prior to fixRemoveNFTokenAutoTrustLine, transfer of an NFToken between accounts allowed a
// TrustLine to be added to the issuer of that token without explicit permission from that issuer.

View File

@@ -404,7 +404,7 @@ LEDGER_ENTRY(ltMPTOKEN_ISSUANCE, 0x007e, MPTokenIssuance, mpt_issuance, ({
{sfPreviousTxnID, SoeRequired},
{sfPreviousTxnLgrSeq, SoeRequired},
{sfDomainID, SoeOptional},
{sfMutableFlags, SoeDefault},
{sfImmutableFlags, SoeDefault},
{sfReferenceHolding, SoeOptional},
{sfIssuerEncryptionKey, SoeOptional},
{sfAuditorEncryptionKey, SoeOptional},

View File

@@ -97,7 +97,7 @@ TYPED_SFIELD(sfVoteWeight, UINT32, 48)
TYPED_SFIELD(sfFirstNFTokenSequence, UINT32, 50)
TYPED_SFIELD(sfOracleDocumentID, UINT32, 51)
TYPED_SFIELD(sfPermissionValue, UINT32, 52)
TYPED_SFIELD(sfMutableFlags, UINT32, 53)
TYPED_SFIELD(sfImmutableFlags, UINT32, 53)
TYPED_SFIELD(sfStartDate, UINT32, 54)
TYPED_SFIELD(sfPaymentInterval, UINT32, 55)
TYPED_SFIELD(sfGracePeriod, UINT32, 56)
@@ -238,6 +238,7 @@ TYPED_SFIELD(sfManagementFeeOutstanding, NUMBER, 17, SField::kSmdNeedsAsset
// int32
TYPED_SFIELD(sfLoanScale, INT32, 1)
TYPED_SFIELD(sfRemainingOwnerCountDelta, INT32, 2)
// currency amount (common)
TYPED_SFIELD(sfAmount, AMOUNT, 1)
@@ -277,6 +278,7 @@ TYPED_SFIELD(sfMinAccountCreateAmount, AMOUNT, 30)
TYPED_SFIELD(sfLPTokenBalance, AMOUNT, 31)
TYPED_SFIELD(sfFeeAmount, AMOUNT, 32)
TYPED_SFIELD(sfMaxFee, AMOUNT, 33)
TYPED_SFIELD(sfFeeAmountDelta, AMOUNT, 34)
// variable length (common)
TYPED_SFIELD(sfPublicKey, VL, 1)

View File

@@ -705,7 +705,7 @@ TRANSACTION(ttMPTOKEN_ISSUANCE_CREATE, 54, MPTokenIssuanceCreate,
{sfMaximumAmount, SoeOptional},
{sfMPTokenMetadata, SoeOptional},
{sfDomainID, SoeOptional},
{sfMutableFlags, SoeOptional},
{sfImmutableFlags, SoeOptional},
}))
/** This transaction type destroys a MPTokensIssuance instance */
@@ -734,7 +734,7 @@ TRANSACTION(ttMPTOKEN_ISSUANCE_SET, 56, MPTokenIssuanceSet,
{sfDomainID, SoeOptional},
{sfMPTokenMetadata, SoeOptional},
{sfTransferFee, SoeOptional},
{sfMutableFlags, SoeOptional},
{sfImmutableFlags, SoeOptional},
{sfIssuerEncryptionKey, SoeOptional},
{sfAuditorEncryptionKey, SoeOptional},
}))
@@ -1085,7 +1085,7 @@ TRANSACTION(ttLOAN_PAY, 84, LoanPay,
# include <xrpl/tx/transactors/token/ConfidentialMPTConvert.h>
#endif
TRANSACTION(ttCONFIDENTIAL_MPT_CONVERT, 85, ConfidentialMPTConvert,
Delegation::Delegable,
Delegation::NotDelegable,
featureConfidentialTransfer,
NoPriv,
({
@@ -1189,9 +1189,9 @@ TRANSACTION(ttSPONSORSHIP_SET, 91, SponsorshipSet,
({
{sfCounterpartySponsor, SoeOptional},
{sfSponsee, SoeOptional},
{sfFeeAmount, SoeOptional},
{sfFeeAmountDelta, SoeOptional},
{sfMaxFee, SoeOptional},
{sfRemainingOwnerCount, SoeOptional},
{sfRemainingOwnerCountDelta, SoeOptional},
}))
/** This system-generated transaction type is used to update the status of the various amendments.

View File

@@ -110,6 +110,7 @@ JSS(accounts); // in: LedgerEntry, Subscribe, handlers/Ledger
JSS(accounts_proposed); // in: Subscribe, Unsubscribe
JSS(action); //
JSS(active); // out: OverlayImpl
JSS(actor); // in/out: AccountTx
JSS(acquiring); // out: LedgerRequest
JSS(address); // out: PeerImp
JSS(affected); // out: AcceptedLedgerTx
@@ -133,6 +134,7 @@ JSS(attestation_reward_account); //
JSS(auction_slot); // out: amm_info
JSS(authorized); // out: AccountLines
JSS(authorize); // out: delegate
JSS(authorizer); // in/out: AccountTx
JSS(authorized_credentials); // in: ledger_entry DepositPreauth
JSS(auth_accounts); // out: amm_info
JSS(auth_change); // out: AccountInfo
@@ -191,6 +193,7 @@ JSS(converge_time); // out: NetworkOPs
JSS(converge_time_s); // out: NetworkOPs
JSS(cookie); // out: NetworkOPs
JSS(count); // in: AccountTx*, ValidatorList
JSS(counter_party); // in/out: AccountTx
JSS(counters); // in/out: retrieve counters
JSS(credentials); // in: deposit_authorized
JSS(credential_type); // in: LedgerEntry DepositPreauth
@@ -270,6 +273,7 @@ JSS(freeze); // out: AccountLines
JSS(freeze_peer); // out: AccountLines
JSS(deep_freeze); // out: AccountLines
JSS(deep_freeze_peer); // out: AccountLines
JSS(delegate_filter); // in/out: AccountTx
JSS(frozen_balances); // out: GatewayBalances
JSS(full); // in: LedgerClearer, handlers/Ledger
JSS(full_reply); // out: PathFind

View File

@@ -256,27 +256,27 @@ public:
}
/**
* @brief Get sfMutableFlags (SoeDefault)
* @brief Get sfImmutableFlags (SoeDefault)
* @return The field value, or std::nullopt if not present.
*/
[[nodiscard]]
protocol_autogen::Optional<SF_UINT32::type::value_type>
getMutableFlags() const
getImmutableFlags() const
{
if (hasMutableFlags())
return this->sle_->at(sfMutableFlags);
if (hasImmutableFlags())
return this->sle_->at(sfImmutableFlags);
return std::nullopt;
}
/**
* @brief Check if sfMutableFlags is present.
* @brief Check if sfImmutableFlags is present.
* @return True if the field is present, false otherwise.
*/
[[nodiscard]]
bool
hasMutableFlags() const
hasImmutableFlags() const
{
return this->sle_->isFieldPresent(sfMutableFlags);
return this->sle_->isFieldPresent(sfImmutableFlags);
}
/**
@@ -557,13 +557,13 @@ public:
}
/**
* @brief Set sfMutableFlags (SoeDefault)
* @brief Set sfImmutableFlags (SoeDefault)
* @return Reference to this builder for method chaining.
*/
MPTokenIssuanceBuilder&
setMutableFlags(std::decay_t<typename SF_UINT32::type::value_type> const& value)
setImmutableFlags(std::decay_t<typename SF_UINT32::type::value_type> const& value)
{
object_[sfMutableFlags] = value;
object_[sfImmutableFlags] = value;
return *this;
}

View File

@@ -19,7 +19,7 @@ class ConfidentialMPTConvertBuilder;
* @brief Transaction: ConfidentialMPTConvert
*
* Type: ttCONFIDENTIAL_MPT_CONVERT (85)
* Delegable: Delegation::Delegable
* Delegable: Delegation::NotDelegable
* Amendment: featureConfidentialTransfer
* Privileges: NoPriv
*

View File

@@ -178,29 +178,29 @@ public:
}
/**
* @brief Get sfMutableFlags (SoeOptional)
* @brief Get sfImmutableFlags (SoeOptional)
* @return The field value, or std::nullopt if not present.
*/
[[nodiscard]]
protocol_autogen::Optional<SF_UINT32::type::value_type>
getMutableFlags() const
getImmutableFlags() const
{
if (hasMutableFlags())
if (hasImmutableFlags())
{
return this->tx_->at(sfMutableFlags);
return this->tx_->at(sfImmutableFlags);
}
return std::nullopt;
}
/**
* @brief Check if sfMutableFlags is present.
* @brief Check if sfImmutableFlags is present.
* @return True if the field is present, false otherwise.
*/
[[nodiscard]]
bool
hasMutableFlags() const
hasImmutableFlags() const
{
return this->tx_->isFieldPresent(sfMutableFlags);
return this->tx_->isFieldPresent(sfImmutableFlags);
}
};
@@ -302,13 +302,13 @@ public:
}
/**
* @brief Set sfMutableFlags (SoeOptional)
* @brief Set sfImmutableFlags (SoeOptional)
* @return Reference to this builder for method chaining.
*/
MPTokenIssuanceCreateBuilder&
setMutableFlags(std::decay_t<typename SF_UINT32::type::value_type> const& value)
setImmutableFlags(std::decay_t<typename SF_UINT32::type::value_type> const& value)
{
object_[sfMutableFlags] = value;
object_[sfImmutableFlags] = value;
return *this;
}

View File

@@ -163,29 +163,29 @@ public:
}
/**
* @brief Get sfMutableFlags (SoeOptional)
* @brief Get sfImmutableFlags (SoeOptional)
* @return The field value, or std::nullopt if not present.
*/
[[nodiscard]]
protocol_autogen::Optional<SF_UINT32::type::value_type>
getMutableFlags() const
getImmutableFlags() const
{
if (hasMutableFlags())
if (hasImmutableFlags())
{
return this->tx_->at(sfMutableFlags);
return this->tx_->at(sfImmutableFlags);
}
return std::nullopt;
}
/**
* @brief Check if sfMutableFlags is present.
* @brief Check if sfImmutableFlags is present.
* @return True if the field is present, false otherwise.
*/
[[nodiscard]]
bool
hasMutableFlags() const
hasImmutableFlags() const
{
return this->tx_->isFieldPresent(sfMutableFlags);
return this->tx_->isFieldPresent(sfImmutableFlags);
}
/**
@@ -341,13 +341,13 @@ public:
}
/**
* @brief Set sfMutableFlags (SoeOptional)
* @brief Set sfImmutableFlags (SoeOptional)
* @return Reference to this builder for method chaining.
*/
MPTokenIssuanceSetBuilder&
setMutableFlags(std::decay_t<typename SF_UINT32::type::value_type> const& value)
setImmutableFlags(std::decay_t<typename SF_UINT32::type::value_type> const& value)
{
object_[sfMutableFlags] = value;
object_[sfImmutableFlags] = value;
return *this;
}

View File

@@ -100,29 +100,29 @@ public:
}
/**
* @brief Get sfFeeAmount (SoeOptional)
* @brief Get sfFeeAmountDelta (SoeOptional)
* @return The field value, or std::nullopt if not present.
*/
[[nodiscard]]
protocol_autogen::Optional<SF_AMOUNT::type::value_type>
getFeeAmount() const
getFeeAmountDelta() const
{
if (hasFeeAmount())
if (hasFeeAmountDelta())
{
return this->tx_->at(sfFeeAmount);
return this->tx_->at(sfFeeAmountDelta);
}
return std::nullopt;
}
/**
* @brief Check if sfFeeAmount is present.
* @brief Check if sfFeeAmountDelta is present.
* @return True if the field is present, false otherwise.
*/
[[nodiscard]]
bool
hasFeeAmount() const
hasFeeAmountDelta() const
{
return this->tx_->isFieldPresent(sfFeeAmount);
return this->tx_->isFieldPresent(sfFeeAmountDelta);
}
/**
@@ -152,29 +152,29 @@ public:
}
/**
* @brief Get sfRemainingOwnerCount (SoeOptional)
* @brief Get sfRemainingOwnerCountDelta (SoeOptional)
* @return The field value, or std::nullopt if not present.
*/
[[nodiscard]]
protocol_autogen::Optional<SF_UINT32::type::value_type>
getRemainingOwnerCount() const
protocol_autogen::Optional<SF_INT32::type::value_type>
getRemainingOwnerCountDelta() const
{
if (hasRemainingOwnerCount())
if (hasRemainingOwnerCountDelta())
{
return this->tx_->at(sfRemainingOwnerCount);
return this->tx_->at(sfRemainingOwnerCountDelta);
}
return std::nullopt;
}
/**
* @brief Check if sfRemainingOwnerCount is present.
* @brief Check if sfRemainingOwnerCountDelta is present.
* @return True if the field is present, false otherwise.
*/
[[nodiscard]]
bool
hasRemainingOwnerCount() const
hasRemainingOwnerCountDelta() const
{
return this->tx_->isFieldPresent(sfRemainingOwnerCount);
return this->tx_->isFieldPresent(sfRemainingOwnerCountDelta);
}
};
@@ -243,13 +243,13 @@ public:
}
/**
* @brief Set sfFeeAmount (SoeOptional)
* @brief Set sfFeeAmountDelta (SoeOptional)
* @return Reference to this builder for method chaining.
*/
SponsorshipSetBuilder&
setFeeAmount(std::decay_t<typename SF_AMOUNT::type::value_type> const& value)
setFeeAmountDelta(std::decay_t<typename SF_AMOUNT::type::value_type> const& value)
{
object_[sfFeeAmount] = value;
object_[sfFeeAmountDelta] = value;
return *this;
}
@@ -265,13 +265,13 @@ public:
}
/**
* @brief Set sfRemainingOwnerCount (SoeOptional)
* @brief Set sfRemainingOwnerCountDelta (SoeOptional)
* @return Reference to this builder for method chaining.
*/
SponsorshipSetBuilder&
setRemainingOwnerCount(std::decay_t<typename SF_UINT32::type::value_type> const& value)
setRemainingOwnerCountDelta(std::decay_t<typename SF_INT32::type::value_type> const& value)
{
object_[sfRemainingOwnerCount] = value;
object_[sfRemainingOwnerCountDelta] = value;
return *this;
}

View File

@@ -46,6 +46,22 @@ struct LedgerRange
uint32_t max;
};
/**
* @brief Enumeration of possible delegate types that can occur during filtering in account_tx
*/
enum class DelegateType {
Actor, ///< Another account signed and submitted transactions on behalf of this account (this
///< account is the owner/delegator).
Authorizer ///< This account signed and submitted transactions on behalf of another account
///< (this account is the signer/delegatee).
};
struct DelegateFilter
{
DelegateType type = DelegateType::Actor;
std::optional<AccountID> counterparty;
};
class RelationalDatabase
{
public:
@@ -82,6 +98,7 @@ public:
std::optional<AccountTxMarker> marker;
std::uint32_t limit = 0;
bool bAdmin = false;
std::optional<DelegateFilter> delegate;
};
using AccountTx = std::pair<std::shared_ptr<Transaction>, std::shared_ptr<TxMeta>>;
@@ -101,6 +118,7 @@ public:
bool forward = false;
uint32_t limit = 0;
std::optional<AccountTxMarker> marker;
std::optional<DelegateFilter> delegate;
};
struct AccountTxResult
@@ -109,6 +127,7 @@ public:
LedgerRange ledgerRange{};
uint32_t limit = 0;
std::optional<AccountTxMarker> marker;
std::optional<DelegateFilter> delegate;
};
virtual ~RelationalDatabase() = default;

View File

@@ -13,7 +13,7 @@
#include <soci/session.h>
#include <memory>
#if defined(__clang__)
#ifdef __clang__
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wdeprecated"
#endif
@@ -120,6 +120,6 @@ makeCheckpointer(std::uintptr_t id, std::weak_ptr<soci::session>, JobQueue&, Ser
} // namespace xrpl
#if defined(__clang__)
#ifdef __clang__
#pragma clang diagnostic pop
#endif

View File

@@ -13,6 +13,7 @@ extern Charge const kFeeRequestNoReply; // A request that we cannot satisfy.
extern Charge const kFeeInvalidSignature; // An object whose signature we had to check that failed.
extern Charge const kFeeUselessData; // Data we have no use for.
extern Charge const kFeeInvalidData; // Data we have to verify before rejecting.
extern Charge const kFeeMalformedData; // Data that no honest peer would send.
// RPC loads
extern Charge const kFeeMalformedRpc; // An RPC request that we can immediately tell is invalid.

View File

@@ -11,6 +11,7 @@
#include <xrpl/server/Manifest.h>
#include <atomic>
#include <cstddef>
#include <cstdint>
#include <functional>
#include <memory>
@@ -22,6 +23,39 @@ namespace xrpl {
// Operations that clients may wish to perform against the network
// Master operational handler, server sequencer, network tracker
/**
* Maximum number of subscriptions a single client connection may hold at once.
*
* Applies to the account, real-time account, and account-history subscriptions
* tracked on one InfoSub (the sets counted by totalSubscriptionCount), bounding
* the disconnect-time cleanup of those sets. Book subscriptions are tracked
* separately (OrderBookDB) and are not counted here. Generous enough for
* legitimate power users such as block explorers.
*/
constexpr std::size_t kMaxSubscriptionsPerConnection = 100'000;
/**
* Whether adding @p additional subscriptions to a connection already holding
* @p current would exceed the cap.
*
* Pure arithmetic split out so it can be unit-tested without a live
* connection. The first term avoids underflow in the subtraction.
*
* @param current Subscriptions already tracked on the connection.
* @param additional Subscriptions a request would add.
* @param cap The effective per-connection cap. Defaults to the
* built-in limit; callers may pass a configured override.
* @return true if the request must be rejected to stay within the cap.
*/
[[nodiscard]] constexpr bool
exceedsSubscriptionCap(
std::size_t current,
std::size_t additional,
std::size_t cap = kMaxSubscriptionsPerConnection)
{
return additional > cap || current > cap - additional;
}
class InfoSubRequest : public CountedObject<InfoSubRequest>
{
public:
@@ -44,12 +78,12 @@ public:
* map.
*
* @note Lifetime contract: every `InfoSub` instance MUST be destroyed
* before the backing `Source`. NetworkOPsImp shutdown drops all
* subscriber strong refs before its own teardown to satisfy this.
* before the backing `Source`. NetworkOPsImp shutdown drops all
* subscriber strong refs before its own teardown to satisfy this.
* @note Thread-safety: per-instance state is guarded by `lock_`. The
* destructor reads tracking sets without taking `lock_` because
* the strong-pointer ref-count is zero at destruction time, so
* no other thread can be calling the public mutators.
* destructor reads tracking sets without taking `lock_` because
* the strong-pointer ref-count is zero at destruction time, so
* no other thread can be calling the public mutators.
*/
class InfoSub : public CountedObject<InfoSub>
{
@@ -117,6 +151,34 @@ public:
AccountID const& account,
bool historyOnly) = 0;
/**
* Schedule the server-side teardown of a disconnecting connection's
* account subscriptions off the destructor thread.
*
* The implementation posts a low-priority JobQueue task that erases the
* entries in bounded chunks, so `~InfoSub` returns immediately instead
* of running the erase loop inline. The sets are taken by value so the
* job owns its copies and never references the destroyed `InfoSub`.
* Cleanup is keyed on `seq` (unique per connection), so deferring it
* cannot disturb a reconnected client reusing the same accounts.
*
* @param seq The disconnecting connection's unique subscription id.
* @param rtAccounts Real-time account subscriptions to remove.
* @param normalAccounts Normal account subscriptions to remove.
* @param historyAccounts Account-history subscriptions to remove.
*
* @note The implementing `Source` must outlive any job it posts. If the
* JobQueue is already stopping (process shutdown), the job is not
* enqueued; the cleanup is skipped because the server-side maps
* are about to be destroyed and no publishing can run.
*/
virtual void
scheduleAccountCleanup(
std::uint64_t seq,
hash_set<AccountID> rtAccounts,
hash_set<AccountID> normalAccounts,
hash_set<AccountID> historyAccounts) = 0;
// VFALCO TODO Document the bool return value
virtual bool
subLedger(ref ispListener, json::Value& jvResult) = 0;
@@ -153,12 +215,12 @@ public:
* @param ispListener The subscriber requesting removal.
* @param book The order book to unsubscribe from.
* @return true if the entry was present and removed, false if the
* subscriber was not subscribed to @p book.
* subscriber was not subscribed to @p book.
*
* @note Thread-safety: acquires subLock_ internally.
* @note Thread-safety: acquires bookLock_ internally.
* @note Do NOT call from ~InfoSub(). Use unsubBookInternal instead
* to avoid a redundant write-back to bookSubscriptions_ on a
* partially-destroyed object.
* to avoid a redundant write-back to bookSubscriptions_ on a
* partially-destroyed object.
*/
virtual bool
unsubBook(ref ispListener, Book const&) = 0;
@@ -173,9 +235,9 @@ public:
* @param uListener The sequence number of the subscriber being torn down.
* @param book The order book entry to remove.
* @return true if the entry was present and removed, false otherwise
* (e.g., already removed by a concurrent RPC unsubscribe).
* (e.g., already removed by a concurrent RPC unsubscribe).
*
* @note Thread-safety: acquires subLock_ internally.
* @note Thread-safety: acquires bookLock_ internally.
*/
virtual bool
unsubBookInternal(std::uint64_t uListener, Book const&) = 0;
@@ -221,8 +283,8 @@ public:
/**
* Journal used by InfoSub for diagnostics that occur after the
* owning subsystem (e.g. application-level Logs) is the only
* surviving sink — primarily destructor-time cleanup failures.
* owning subsystem (e.g. application-level Logs) is the only
* surviving sink — primarily destructor-time cleanup failures.
*/
[[nodiscard]] virtual beast::Journal const&
journal() const = 0;
@@ -243,6 +305,56 @@ public:
[[nodiscard]] std::uint64_t
getSeq() const;
/**
* Return the number of subscriptions currently tracked on this
* connection.
*
* The combined size of the per-connection account, real-time account, and
* account-history subscription sets. `doSubscribe` reads this to enforce
* the per-connection subscription cap before admitting more.
*
* @return The total tracked subscription count for this connection.
*
* @note Thread-safe: takes `lock_` for the read; read-only.
*/
[[nodiscard]] std::size_t
totalSubscriptionCount() const;
/**
* Enforce the cap and reserve a request's net-new accounts, atomically.
*
* Under one hold of `lock_`: count the net-new entries in the two sets,
* check the total against @p cap, and insert them only if it fits.
* All-or-nothing. Doing check and insert together stops two concurrent
* requests sharing an InfoSub (the admin subscribe-by-url path) from both
* passing the check before either records its accounts. The server-side
* maps are populated afterwards by subAccount, whose re-insert is a no-op.
*
* @param proposedAccounts Real-time (accounts_proposed) ids to reserve.
* @param normalAccounts Normal (accounts) ids to reserve.
* @param cap The effective per-connection cap.
* @return true if reserved; false if the request must be rejected.
* @note Thread-safe: takes `lock_`.
*/
[[nodiscard]] bool
tryReserveAccountSubscriptions(
hash_set<AccountID> const& proposedAccounts,
hash_set<AccountID> const& normalAccounts,
std::size_t cap);
/**
* Whether this connection already tracks an account-history for @p account.
*
* `doSubscribe` reads this to charge the cap for an account_history_tx_stream
* only when it is net-new, matching the account branches.
*
* @param account The account an account_history_tx_stream would add.
* @return true if @p account is already in the account-history set.
* @note Thread-safe: takes `lock_`; read-only.
*/
[[nodiscard]] bool
hasAccountHistorySubscription(AccountID const& account) const;
void
onSendEmpty();
@@ -302,7 +414,9 @@ public:
getApiVersion() const noexcept;
protected:
std::mutex lock_;
// Mutable so the read-only totalSubscriptionCount() accessor can lock it
// from a const method; locking semantics are otherwise unchanged.
mutable std::mutex lock_;
private:
Consumer consumer_;

View File

@@ -3,12 +3,14 @@
#include <xrpl/basics/Blob.h>
#include <xrpl/basics/Slice.h>
#include <xrpl/basics/UnorderedContainers.h>
#include <xrpl/basics/base64.h>
#include <xrpl/basics/base_uint.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/protocol/PublicKey.h>
#include <xrpl/protocol/SecretKey.h>
#include <atomic>
#include <cstddef>
#include <cstdint>
#include <functional>
#include <optional>
@@ -43,12 +45,15 @@ namespace xrpl {
dynamically generates the signatureless form when it needs to verify
the signature.
An instance of ManifestCache stores, for each trusted validator, (a) its
An instance of ManifestCache stores, for each known validator, (a) its
master public key, and (b) the most senior of all valid manifests it has
seen for that validator, if any. On startup, the [validator_token] config
entry (which contains the manifest for this validator) is decoded and
added to the manifest cache. Other manifests are added as "gossip"
received from xrpld peers.
received from xrpld peers, including ones for validators this node does not
trust. Manifests for untrusted validators are capped (kMaxUntrustedCount)
so peer gossip cannot grow the cache without bound; trusted validators are
not capped. Entries are never evicted, so a stored revocation is permanent.
When an ephemeral key is compromised, a new signing key pair is created,
along with a new manifest vouching for it (with a higher sequence number),
@@ -164,6 +169,100 @@ struct Manifest
std::string
to_string(Manifest const& m);
/**
* Largest a valid manifest can be, in decoded bytes.
*
* A manifest has a fixed set of fields. Each is serialized as a field header
* (1-2 bytes), an optional length prefix (1 byte for these sizes), and the
* field body. Taking every field at its largest gives the maximum below, so
* anything larger cannot be a valid manifest.
*
* Field header + length + body = bytes
* sfVersion (U16) 2 0 2 4
* sfSequence (U32) 1 0 4 5
* sfPublicKey (33) 1 1 33 35
* sfSigningPubKey (33) 1 1 33 35
* sfSignature (72) 1 1 72 74
* sfMasterSignature (72) 2 1 72 75
* sfDomain (128) 1 1 128 130
* -----
* 358
*/
constexpr std::size_t kMaxManifestBytes = 358;
/**
* Largest a valid manifest can be, in base64 characters.
*
* base64 encodes 3 bytes as 4 characters, so this is the encoded form of
* @ref kMaxManifestBytes. Callers that receive a base64 manifest should
* reject anything longer than this before decoding, to avoid allocating
* memory for an oversized input.
*/
constexpr std::size_t kMaxManifestBase64 = base64::encodedSize(kMaxManifestBytes);
/**
* Default number of untrusted manifests to store in cache and allowed
* in one Manifest message.
*
* Bounds unlisted validators two ways. In the cache, a manifest for a
* brand-new unlisted key is rejected once this many are held, so peer gossip
* cannot grow the cache without end. In a TMManifests message, this many are
* sent and processed, so a peer sending its whole cache cannot force unbounded
* work.
*
* Operators can override this with `[overlay] max_untrusted_count`. Both users
* read the configured value and fall back to this default.
*/
constexpr std::size_t kMaxUntrustedCount = 300;
/**
* Default number of trusted manifests allowed in a Manifest message.
* Not used atm while creating the message, but used to calculate the higher limit on
* received message size. Introduced to maintain consistency. Future implementation
* will use this limit.
*
* Trusted manifests are never dropped: every one this node holds is sent, and
* every one received is processed, since dropping one would delay a validator
* key rotation. This count only sizes the largest message accepted, so it must
* stay above any realistic validator list. Cap can be increased in the config
* file if messages get rejected with actual trusted manifest count crossing
* configured(or else default) value.
* Operators can override this with `[overlay] max_trusted_count`.
*/
constexpr std::size_t kMaxTrustedCount = 300;
/**
* Number of untrusted manifests to store in cache and allowed
* in one Manifest message..
*
* Returns the operator's override when one is configured, otherwise
* @ref kMaxUntrustedCount. Config stores an override rather than the default
* itself because the core module cannot depend on this module.
*
* @param configured The value from `[overlay] max_untrusted_count`, or
* `std::nullopt` when the operator did not set it.
*/
constexpr std::size_t
untrustedManifestCount(std::optional<std::size_t> const& configured)
{
return configured.value_or(kMaxUntrustedCount);
}
/**
* Number of trusted manifests allowed in a Manifest message.
*
* Not a cap on how many are sent or processed; see @ref kMaxTrustedCount.
* but used to calculate the higher limit on received message size.
*
* @param configured The value from `[overlay] max_trusted_count`, or
* `std::nullopt` when the operator did not set it.
*/
constexpr std::size_t
trustedManifestCount(std::optional<std::size_t> const& configured)
{
return configured.value_or(kMaxTrustedCount);
}
/**
* Constructs Manifest from serialized string
*
@@ -172,7 +271,7 @@ to_string(Manifest const& m);
* @return `std::nullopt` if string is invalid
*
* @note This does not verify manifest signatures.
* `Manifest::verify` should be called after constructing manifest.
* `Manifest::verify` should be called after constructing manifest.
*/
/** @{ */
std::optional<Manifest>
@@ -225,30 +324,17 @@ loadValidatorToken(
beast::Journal journal = beast::Journal(beast::Journal::getNullSink()));
enum class ManifestDisposition {
/**
* Manifest is valid
*/
Accepted = 0,
Accepted = 0, ///< Manifest is valid
/**
* Sequence is too old
*/
Stale,
Stale, ///< Sequence is too old
/**
* The master key is not acceptable to us
*/
BadMasterKey,
BadMasterKey, ///< The master key is not acceptable to us
/**
* The ephemeral key is not acceptable to us
*/
BadEphemeralKey,
BadEphemeralKey, ///< The ephemeral key is not acceptable to us
/**
* Timely, but invalid signature
*/
Invalid
Invalid, ///< Timely, but invalid signature
UntrustedCapacity ///< Unlisted and limit reached
};
inline std::string
@@ -266,11 +352,25 @@ to_string(ManifestDisposition m)
return "badEphemeralKey";
case ManifestDisposition::Invalid:
return "invalid";
case ManifestDisposition::UntrustedCapacity:
return "untrustedCapacity";
default:
return "unknown";
}
}
/**
* Whether a manifest counts against the 'untrusted' cache cap.
*
* Passed to `ManifestCache::applyManifest` with no default, so every caller
* must choose. `Capped` is the safe, flood-resistant value; only listed or
* configured keys should use `Uncapped`.
*/
enum class ManifestRateLimitCapPolicy : std::uint8_t {
Capped, ///< Subject to the untrusted cap (unlisted peer gossip)
Uncapped ///< Bypasses the cap (listed/trusted or config manifests)
};
class DatabaseCon;
/**
@@ -294,8 +394,51 @@ private:
std::atomic<std::uint32_t> seq_{0};
/**
* Master keys of cached manifests for validators this node does not list.
*
* One entry per capped key in `map_`; its size enforces the cap below.
* A key is added when first cached under `Capped` and removed when it
* becomes listed (see `promoteToTrusted`) or an `Uncapped` update arrives,
* never re-added on de-listing. Uncapped keys are not tracked here.
*/
hash_set<PublicKey> untrustedKeys_;
/**
* Maximum number of untrusted master keys kept in the cache.
*
* Once reached, a manifest for a brand-new unlisted key is rejected. Set
* from the config, defaulting to @ref kMaxUntrustedCount.
*/
std::size_t const maxUntrustedCount_;
/**
* Running count of manifests rejected because the untrusted cap was full.
*
* Drives throttled logging (see `kUntrustedRejectCount`). Atomic because
* `applyManifest` may run concurrently.
*/
std::atomic<std::uint64_t> untrustedRejectCount_{0};
/**
* Number of cap rejections between summary warnings.
*
* @see untrustedRejectCount_
*/
static constexpr std::uint64_t kUntrustedRejectCount = 10000;
public:
explicit ManifestCache(beast::Journal j = beast::Journal(beast::Journal::getNullSink())) : j_(j)
/**
* @param j Journal for logging.
*
* @param maxUntrustedCount Untrusted master keys to keep. Pass the
* configured value; defaults to @ref kMaxUntrustedCount. Taken as a
* parameter because this module cannot depend on the config.
*/
explicit ManifestCache(
beast::Journal j = beast::Journal(beast::Journal::getNullSink()),
std::size_t maxUntrustedCount = kMaxUntrustedCount)
: j_(j), maxUntrustedCount_(maxUntrustedCount)
{
}
@@ -378,17 +521,44 @@ public:
/**
* Add manifest to cache.
*
* A brand-new unlisted key is rejected once the untrusted cap is full;
* updates to a cached key and `Uncapped` manifests bypass the cap. The
* caller decides `cap` before calling so the cache lock is not held while
* consulting the validator list, which would risk a lock-ordering deadlock.
*
* @param m Manifest to add
*
* @return `ManifestDisposition::accepted` if successful, or
* `stale` or `invalid` otherwise
* @param cap `Uncapped` skips the untrusted cap; use it for keys that are
* listed, configured, or loaded from the DB. Note `Uncapped` does not
* assert the key is currently trusted (a DB entry may predate a
* de-listing). Callers must state this explicitly so a manifest is
* never left uncapped by omission.
*
* @return `Accepted` if stored, `Stale` if superseded, `Invalid`/
* `BadEphemeralKey` if malformed, or `UntrustedCapacity` if the
* untrusted cap is full.
*
* @par Thread Safety
*
* May be called concurrently
*/
ManifestDisposition
applyManifest(Manifest m);
applyManifest(Manifest m, ManifestRateLimitCapPolicy cap);
/**
* Stop counting a master key against the untrusted cap.
*
* Called when a cached untrusted key becomes listed, freeing its slot.
* Idempotent and a no-op for keys that were never counted.
*
* @param pk Master public key that is now listed/trusted
*
* @par Thread Safety
*
* May be called concurrently
*/
void
promoteToTrusted(PublicKey const& pk);
/**
* Populate manifest cache with manifests in database and config.

View File

@@ -71,9 +71,18 @@ public:
/**
* @brief called for each ledger entry in the current transaction.
*
* @param isDelete true if the SLE is being deleted
* @param before ledger entry before modification by the transaction
* @param after ledger entry after modification by the transaction
* @param isDelete true if the SLE is being deleted.
* @param before ledger entry before modification by the transaction. `before` will be null if
* the entry is new.
* @param after ledger entry after modification by the transaction. Always non-null. When
* deleting, `after` may differ from `before`. Whether that is important is up to the
* individual invariant check.
*
* @note `after` IS NEVER NULL. `isDelete` is the only correct way to check for deletions.
* Do not make logic or branching decisions on whether on `after` is set, because it will
* always be set. Treat a null `after` as a programming error (with XRPL_ASSERT). An
* invariant MAY check for null defensively, if it makes more sense, but an assertion is
* preferred for new invariants.
*/
void
visitEntry(bool isDelete, SLE::const_ref before, SLE::const_ref after);
@@ -316,17 +325,26 @@ public:
};
/**
* @brief Invariant: Token holder's trustline balance cannot be negative after
* Clawback.
* @brief Invariant: Token holder's trustline/MPT balance cannot be invalid
* after Clawback.
*
* We iterate all the trust lines affected by this transaction and ensure
* that no more than one trustline is modified, and also holder's balance is
* non-negative.
* non-negative. When featureMPTokensV2 is enabled, also verify the holder's
* raw trustline/MPToken balance decreased by the clawed amount.
*/
class ValidClawback
{
struct EntryChange
{
SLE::const_pointer before;
SLE::const_pointer after;
};
std::uint32_t trustlinesChanged_ = 0;
std::uint32_t mptokensChanged_ = 0;
EntryChange iou_;
EntryChange mpt_;
public:
void
@@ -440,7 +458,7 @@ using InvariantChecks = std::tuple<
ValidLoan,
ValidVault,
ValidConfidentialMPToken,
ValidMPTPayment,
ValidMPTBalanceChanges,
ValidAmounts,
ValidMPTTransfer,
ObjectHasPseudoAccount,

View File

@@ -87,7 +87,7 @@ public:
* OutstandingAmount after application equals OutstandingAmount before
* application plus the net holder balance delta.
*/
class ValidMPTPayment
class ValidMPTBalanceChanges
{
enum class Order { Before = 0, After = 1 };
struct MPTData

View File

@@ -2,6 +2,8 @@
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/ledger/ReadView.h>
#include <xrpl/protocol/AccountID.h>
#include <xrpl/protocol/Keylet.h>
#include <xrpl/protocol/STLedgerEntry.h>
#include <xrpl/protocol/STTx.h>
#include <xrpl/protocol/TER.h>
@@ -16,7 +18,7 @@ namespace xrpl {
class SponsorshipSet : public Transactor
{
public:
static constexpr auto kConsequencesFactory = ConsequencesFactoryType::Normal;
static constexpr auto kConsequencesFactory = ConsequencesFactoryType::Custom;
explicit SponsorshipSet(ApplyContext& ctx) : Transactor(ctx)
{
@@ -47,6 +49,15 @@ public:
XRPAmount fee,
ReadView const& view,
beast::Journal const& j) override;
private:
TER
createSponsorship(
Keylet const& sponsorshipKeylet,
AccountID const& sponsorID,
AccountID const& sponseeID,
SLE::ref sponsorAccSle,
SLE::ref reserveSponsorAccSle);
};
} // namespace xrpl

View File

@@ -12,6 +12,7 @@
#include <array>
#include <cstdint>
#include <optional>
namespace xrpl {
@@ -39,6 +40,9 @@ public:
static NotTEC
checkSign(PreclaimContext const& ctx);
static TER
preclaim(PreclaimContext const& ctx);
TER
doApply() override;
@@ -76,6 +80,10 @@ private:
// only be reached through Batch::checkSign.
static NotTEC
checkBatchSign(PreclaimContext const& ctx);
// nullopt on overflow or oversized signer arrays.
static std::optional<XRPAmount>
calculateBaseFeeImpl(ReadView const& view, STTx const& tx);
};
} // namespace xrpl

View File

@@ -32,7 +32,7 @@ struct MPTCreateArgs
std::optional<std::uint16_t> transferFee = std::nullopt;
std::optional<Slice> const& metadata{};
std::optional<uint256> domainId = std::nullopt;
std::optional<std::uint32_t> mutableFlags = std::nullopt;
std::optional<std::uint32_t> immutableFlags = std::nullopt;
// Set only by callers that issue an MPT representing a wrapped asset
// (e.g. VaultCreate's share token). The keylet must point to an
// existing MPToken or RippleState owned by `account`. Surfaces on

View File

@@ -3,12 +3,15 @@
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/core/ServiceRegistry.h>
#include <xrpl/ledger/ReadView.h>
#include <xrpl/protocol/LedgerFormats.h>
#include <xrpl/protocol/STTx.h>
#include <xrpl/protocol/TER.h>
#include <xrpl/protocol/TxFlags.h>
#include <xrpl/protocol/XRPAmount.h>
#include <xrpl/tx/ApplyContext.h>
#include <xrpl/tx/Transactor.h>
#include <array>
#include <cstdint>
namespace xrpl {
@@ -22,6 +25,37 @@ public:
{
}
// Maps each MPTokenIssuanceSet set flag(e.g., tfMPTSetCanLock), to the issuance's
// corresponding immutable flag (e.g., lsifMPTCanLock) and the target ledger flag (e.g.,
// lsfMPTCanLock).
struct FlagMapping
{
std::uint32_t setFlag;
std::uint32_t immutableFlag;
std::uint32_t ledgerFlag;
};
static constexpr std::array<FlagMapping, 7> flagMapping = {
{{.setFlag = tfMPTSetCanLock, .immutableFlag = lsifMPTCanLock, .ledgerFlag = lsfMPTCanLock},
{.setFlag = tfMPTSetRequireAuth,
.immutableFlag = lsifMPTRequireAuth,
.ledgerFlag = lsfMPTRequireAuth},
{.setFlag = tfMPTSetCanEscrow,
.immutableFlag = lsifMPTCanEscrow,
.ledgerFlag = lsfMPTCanEscrow},
{.setFlag = tfMPTSetCanTrade,
.immutableFlag = lsifMPTCanTrade,
.ledgerFlag = lsfMPTCanTrade},
{.setFlag = tfMPTSetCanTransfer,
.immutableFlag = lsifMPTCanTransfer,
.ledgerFlag = lsfMPTCanTransfer},
{.setFlag = tfMPTSetCanClawback,
.immutableFlag = lsifMPTCanClawback,
.ledgerFlag = lsfMPTCanClawback},
{.setFlag = tfMPTSetCanHoldConfidentialBalance,
.immutableFlag = lsifMPTCanHoldConfidentialBalance,
.ledgerFlag = lsfMPTCanHoldConfidentialBalance}}};
static bool
checkExtraFeatures(PreflightContext const& ctx);

View File

@@ -75,9 +75,13 @@ COPY bin/check-tools.sh /tmp/check-tools.sh
RUN /tmp/check-tools.sh
# Sanity-check that the g++/clang++ are able to build binaries, including sanitizer-instrumented ones.
COPY nix/docker/test_files/cpp_sources/ /tmp/cpp_sources/
COPY nix/docker/test_files/compile-cpp-sources.sh /tmp/compile-cpp-sources.sh
RUN /tmp/compile-cpp-sources.sh /tmp/cpp_sources /tmp/bins
COPY nix/docker/test_files/cpp/ /tmp/test_files/cpp/
RUN /tmp/test_files/cpp/compile-sources.sh /tmp/test_files/cpp/sources /tmp/cpp-bins
# Sanity-check that rustc is able to build binaries, including ones that rely on
# the runtime overflow check.
COPY nix/docker/test_files/rust/ /tmp/test_files/rust/
RUN /tmp/test_files/rust/compile-sources.sh /tmp/test_files/rust/sources /tmp/rust-bins
# Tester: start from a clean BASE_IMAGE, install sanitizer runtime libraries,
# and run the compiled test binaries to verify they execute correctly.
@@ -94,15 +98,18 @@ SHELL ["/bin/bash", "-e", "-o", "pipefail", "-c"]
# Sanity-check that the built binaries run correctly in the vanilla base image, with the necessary sanitizer runtime libraries installed.
COPY bin/install-sanitizer-libs.sh /tmp/install-sanitizer-libs.sh
COPY nix/docker/test_files/run-test-binaries.sh /tmp/run-test-binaries.sh
COPY --from=final /tmp/bins /tmp/bins
COPY nix/docker/test_files/cpp/run-binaries.sh /tmp/test_files/cpp/run-binaries.sh
COPY nix/docker/test_files/rust/run-binaries.sh /tmp/test_files/rust/run-binaries.sh
COPY --from=final /tmp/cpp-bins /tmp/cpp-bins
COPY --from=final /tmp/rust-bins /tmp/rust-bins
RUN <<EOF
if echo "${BASE_IMAGE}" | grep -qiE 'nixos'; then
echo "Skipping runnning binaries on NixOS."
else
/tmp/install-sanitizer-libs.sh
/tmp/run-test-binaries.sh /tmp/bins
/tmp/test_files/cpp/run-binaries.sh /tmp/cpp-bins
/tmp/test_files/rust/run-binaries.sh /tmp/rust-bins
fi
touch /tmp/tests-passed
EOF

View File

@@ -42,9 +42,12 @@ work without `ca-certificates` being installed in the base image.
- installs the dynamic linker if the base image lacks one (see
[How libc is handled](#how-libc-is-handled)),
- runs [`bin/check-tools.sh`](../../bin/check-tools.sh) to verify every
expected tool is present and runnable, and
expected tool is present and runnable.
- compiles the C++ test programs in
[`test_files/`](./test_files) with both `g++` and `clang++`, and sanitizers.
[`test_files/cpp/sources/`](./test_files/cpp/sources) with both `g++` and
`clang++`, and sanitizers, and
- compiles the Rust test programs in
[`test_files/rust/sources/`](./test_files/rust/sources) with `rustc`.
3. **`tester`** — Start again from a clean `BASE_IMAGE` (no Nix toolchain),
install only the sanitizer runtime libraries
([`install-sanitizer-libs.sh`](./install-sanitizer-libs.sh)), and run the
@@ -73,11 +76,12 @@ toolchain being present at runtime. Two pieces make that work:
[`loader-path.sh`](./loader-path.sh) reports the expected loader path for the
current architecture, so we can patch the binaries to use the correct loader.
The build then verifies all of this end to end: the test programs in
`test_files/` (a regular binary plus ASan/TSan/UBSan variants) are compiled in
`final`, their `PT_INTERP` is patched to the target loader, and they are run in
the clean `tester` stage to confirm each emits the expected sanitizer
diagnostic on a stock base image.
The build then verifies all of this end to end: the C++ test programs in
`test_files/cpp/sources/` (a regular binary plus ASan/TSan/UBSan variants) and
the Rust test programs in `test_files/rust/sources/` (a hello binary plus panic
and overflow-check variants) are compiled in `final`, their `PT_INTERP` is
patched to the target loader, and they are run in the clean `tester` stage to
confirm each emits the expected diagnostic on a stock base image.
## Files
@@ -85,6 +89,7 @@ diagnostic on a stock base image.
| ----------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| [`./Dockerfile`](./Dockerfile) | Multi-stage build described above. |
| [`./loader-path.sh`](./loader-path.sh) | Print the dynamic-linker (`PT_INTERP`) path for the current architecture. |
| [`./test_files/`](./test_files) | C++ sources and scripts to compile and run the sanitizer smoke tests. |
| [`./test_files/cpp/`](./test_files/cpp) | C++ sanitizer smoke test: sources + compile/run scripts. |
| [`./test_files/rust/`](./test_files/rust) | Rust rustc smoke test: sources + compile/run scripts. |
| [`/bin/check-tools.sh`](../../bin/check-tools.sh) | Verify every expected tools are present and runnable. |
| [`/bin/install-sanitizer-libs.sh`](../../bin/install-sanitizer-libs.sh) | Install `libasan`/`libtsan`/`libubsan` runtimes on the supported base images. |

View File

@@ -0,0 +1,45 @@
#!/bin/bash
# Compile all Rust test binaries during the Docker image build.
# Each binary has the target system's ELF PT_INTERP (dynamic-linker path)
# baked in so it can run on the (potentially minimal) final BASE_IMAGE.
set -eo pipefail
src_dir="${1:?usage: $0 <src_dir> <dst_dir>}"
dst_dir="${2:?usage: $0 <src_dir> <dst_dir>}"
loader="$(/tmp/loader-path.sh)"
mkdir -p "${dst_dir}"
function compile() {
local name="${1}"
local extra_flags="${2:-}"
local src="${src_dir}/${name}.rs"
local binary="${dst_dir}/${name}"
echo "=== Compiling ${name} with rustc ==="
# -O optimizes (opt-level 2); Rust statically links its own std, so the
# only dynamic dependency left is the system glibc (+ libgcc_s), exactly
# like the C++ binaries.
local compile_cmd="rustc --edition 2021 -O -g ${extra_flags} \
${src} -o ${binary}"
echo "Compile cmd: ${compile_cmd}"
eval "${compile_cmd}"
echo "=== Patching ${binary} to use ${loader} as PT_INTERP ==="
local patch_cmd="patchelf --set-interpreter ${loader} --remove-rpath ${binary}"
echo "Patch cmd: ${patch_cmd}"
eval "${patch_cmd}"
}
# `-O` disables overflow checks by default, so `overflow` re-enables them
# explicitly to exercise the runtime overflow check.
compile hello
compile panic
compile overflow "-C overflow-checks=on"
echo "=== All binaries compiled ==="
ls -la "${dst_dir}"

View File

@@ -0,0 +1,74 @@
#!/bin/bash
# Run pre-compiled Rust binaries and confirm each emits its expected diagnostic.
# Binaries must already exist in <bins_dir> as <name> for name in
# {hello,panic,overflow}.
set -eo pipefail
bins_dir="${1:?usage: $0 <bins_dir>}"
failed_binaries=()
# Run a binary and verify its exit code and output.
# Usage: run <binary> <expected_output> <expected_rc>
function run() {
local binary="${1}"
local expected_output="${2}"
local expected_rc="${3}"
local out_file
out_file="$(mktemp)"
echo "=== Run ${binary} ==="
set +e
"${binary}" >"${out_file}" 2>&1
local rc=$?
set -e
cat "${out_file}"
local failed=0
if [ "${expected_rc}" = "nonzero" ]; then
if [ "${rc}" -eq 0 ]; then
echo "ERROR: expected non-zero exit code from ${binary}, got ${rc}" >&2
failed=1
fi
elif [ "${rc}" -ne "${expected_rc}" ]; then
echo "ERROR: expected exit code ${expected_rc} from ${binary}, got ${rc}" >&2
failed=1
fi
if ! grep -q "${expected_output}" "${out_file}"; then
echo "ERROR: expected '${expected_output}' from ${binary}" >&2
failed=1
fi
if [ "${failed}" -eq 0 ]; then
echo "OK: '${expected_output}' detected"
else
failed_binaries+=("${binary}")
fi
}
declare -A expect=(
[hello]="Hello from main thread"
[panic]="explicit panic from test"
[overflow]="attempt to add with overflow"
)
for name in hello panic overflow; do
binary="${bins_dir}/${name}"
if [ "${name}" = "hello" ]; then
expected_rc=0
else
expected_rc=nonzero
fi
run "${binary}" "${expect[$name]}" "${expected_rc}"
done
if [ "${#failed_binaries[@]}" -gt 0 ]; then
echo "ERROR: the following binaries failed:" >&2
printf ' %s\n' "${failed_binaries[@]}" >&2
exit 1
fi

View File

@@ -0,0 +1,16 @@
use std::thread;
fn main() {
const NUM_THREADS: usize = 10;
let mut handles = Vec::with_capacity(NUM_THREADS);
for id in 0..NUM_THREADS {
handles.push(thread::spawn(move || {
println!("Hello from thread {id}");
}));
}
for handle in handles {
handle.join().expect("worker thread panicked");
}
println!("Hello from main thread");
}

View File

@@ -0,0 +1,13 @@
use std::hint::black_box;
// Rust analogue of the C++ UBSan check: with overflow checks enabled the
// compiler inserts a runtime check that panics on signed integer overflow.
// `black_box` keeps the operands opaque so the addition is evaluated at
// runtime rather than being rejected by the compile-time overflow lint.
fn main() {
let max = black_box(i32::MAX);
let one = black_box(1);
println!("Current max: {max}");
let overflowed = max + one;
println!("Overflowed result: {overflowed}");
}

View File

@@ -0,0 +1,5 @@
fn main() {
// Verify the panic runtime works: a panic must print its message to stderr
// and exit with a non-zero status (Rust's default panic exit code is 101).
panic!("explicit panic from test");
}

View File

@@ -62,5 +62,15 @@ in
runClangTidy
vim
zip
# Rust packages
cargo
cargo-audit
cargo-llvm-cov
cargo-nextest
clippy
corrosion
rust-analyzer
rustc
rustfmt
];
}

View File

@@ -1,6 +1,8 @@
# Linux Packaging
This directory contains all files needed to build RPM and Debian packages for `xrpld`.
This directory contains all files needed to build RPM and Debian packages for
`xrpld`. The packages also ship the `validator-keys` tool, so packaging requires
a build configured with `-Dvalidator_keys=ON`.
## Directory layout
@@ -46,17 +48,28 @@ To print the full packaging matrix (artifact names and images) for the current
Caller workflows (`on-pr.yml`, `on-tag.yml`, `on-trigger.yml`) call
`reusable-package.yml`. That workflow generates its own packaging matrix from
`package_configs` in `linux.json` (via `generate.py --packaging`) and fans out
one job per distro. Each job downloads the pre-built `xrpld` binary artifact and
runs in that distro's container, so the package format follows from the
container's package manager. The packaging script derives the package version
from the downloaded binary's `xrpld --version` output; no CMake configure or
build step is needed inside the packaging job.
one job per distro. Each job downloads the pre-built `xrpld` and `validator-keys`
binary artifacts and runs in that distro's container, so the package format
follows from the container's package manager. The packaging script derives the
package version from the downloaded binary's `xrpld --version` output; no CMake
configure or build step is needed inside the packaging job.
The binaries come from the `debian` and `rhel` build configurations in
`linux.json`'s `configs` section, which pass `-Dvalidator_keys=ON` so that the
build job produces `validator-keys` next to `xrpld` and uploads it as the
`validator-keys-<config name>` artifact. The packaging entry for a distro names
both artifacts (`xrpld_artifact_name` and `validator_keys_artifact_name`), so a
packaged configuration must keep `-Dvalidator_keys=ON`.
`validator-keys` is fetched from an exact commit pinned in
[`cmake/XrplValidatorKeys.cmake`](../cmake/XrplValidatorKeys.cmake), so a given
`xrpld` version always packages the same tool; bump that commit deliberately.
### Locally (mirrors CI)
With an `xrpld` binary already built at `build/xrpld`, run the packaging step
inside the same container CI uses. The image tag is derived from `linux.json`
so you don't need to hardcode a SHA.
With `xrpld` and `validator-keys` binaries already built at `build/xrpld` and
`build/validator-keys`, run the packaging step inside the same container CI uses.
The image tag is derived from `linux.json` so you don't need to hardcode a SHA.
```bash
# From the repo root. Each distro's container image is the `image` field of its
@@ -87,6 +100,7 @@ needed, but the host toolchain replaces the pinned CI image:
```bash
cmake \
-Dxrpld=ON \
-Dvalidator_keys=ON \
-Dpkg_release=1 \
-Dtests=OFF \
..
@@ -95,9 +109,11 @@ cmake --build . --target package # deb on Debian/Ubuntu, rpm on RHEL
```
The `cmake/XrplPackaging.cmake` module defines the `package` target only if at
least one of `rpmbuild` / `dpkg-buildpackage` is present; `build_pkg.sh` then
infers the package format from the host's package manager. The packaging script
installs to FHS-standard paths (`/usr/bin`, `/etc/xrpld`, etc.) regardless of
least one of `rpmbuild` / `dpkg-buildpackage` is present and both the `xrpld` and
`validator-keys` targets exist (`-Dxrpld=ON -Dvalidator_keys=ON`); the target
builds both binaries before packaging. `build_pkg.sh` then infers the package
format from the host's package manager. The packaging script installs to
FHS-standard paths (`/usr/bin`, `/etc/xrpld`, etc.) regardless of
`CMAKE_INSTALL_PREFIX`.
The package version is not a CMake input on this path: `build_pkg.sh` derives it
@@ -156,13 +172,17 @@ CMake/CI integration. The CI workflow and the CMake `package` target both invoke
and lets the script use defaults for the rest.
It resolves `SRC_DIR` and `BUILD_DIR` to absolute paths, then calls
`stage_common()` to copy the binary, config files, and shared support files
into the staging area, and invokes the platform build tool.
`stage_common()` to copy the `xrpld` and `validator-keys` binaries, config files,
and shared support files into the staging area, and invokes the platform build
tool. Both binaries must be present in `BUILD_DIR` and must run in the packaging
environment; a missing or non-runnable one fails early. That runtime check is
what catches a binary still linked against the Nix store's ELF loader (see
`patch_nix_binary` in `cmake/PatchNixBinary.cmake`).
### RPM
1. Creates the standard `rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}` tree inside the build directory.
2. Copies `xrpld.spec` and all shared source files (binary, configs, service files) into `SOURCES/`.
2. Copies `xrpld.spec` and all shared source files (binaries, configs, service files) into `SOURCES/`.
3. Runs `rpmbuild -bb`, passing the normalized package metadata version as the
`pkg_version` RPM macro and `PKG_RELEASE` as the `pkg_release` RPM macro.
The spec uses manual `install` commands to place files, disables `dwz`, and
@@ -182,7 +202,8 @@ service restart.
### DEB
1. Creates a staging source tree at `debbuild/source/` inside the build directory.
2. Stages the binary, configs, `README.md`, and `LICENSE.md`.
2. Stages the binaries, configs, `README.md`, `LICENSE.md`, and
`validator-keys-LICENSE`.
3. Copies `package/debian/` control files into `debbuild/source/debian/`.
4. Copies shared service/sysusers/tmpfiles into `debian/` where `dh_installsystemd`, `dh_installsysusers`, and `dh_installtmpfiles` pick them up automatically.
5. Generates a minimal `debian/changelog` using `${pkg_version}-${PKG_RELEASE}`,

View File

@@ -1,7 +1,8 @@
#!/usr/bin/env bash
set -euo pipefail
# Build an RPM or Debian package from a pre-built xrpld binary.
# Build an RPM or Debian package from the pre-built xrpld and validator-keys
# binaries.
#
# Flags override env vars; env vars override defaults.
@@ -11,7 +12,9 @@ Usage: build_pkg.sh [options]
Options (each can also be set via the env var shown):
--src-dir DIR repo root [SRC_DIR; default: ${PWD}]
--build-dir DIR directory holding xrpld [BUILD_DIR; default: ${PWD}/build]
--build-dir DIR directory holding the
xrpld and validator-keys
binaries [BUILD_DIR; default: ${PWD}/build]
--pkg-release N package release iteration [PKG_RELEASE; default: 1]
--source-date-epoch SECS reproducibility timestamp [SOURCE_DATE_EPOCH; latest git ctime; fallback: current time]
-h, --help show this help and exit
@@ -69,15 +72,44 @@ SRC_DIR="$(cd "${SRC_DIR:-${PWD}}" && pwd)"
BUILD_DIR="${BUILD_DIR:-${PWD}/build}"
if [[ ! -d "${BUILD_DIR}" ]]; then
echo "build_pkg.sh: build directory not found: ${BUILD_DIR}" >&2
echo "Build xrpld before packaging, or set BUILD_DIR to the directory containing xrpld." >&2
echo "Build the binaries before packaging, or set BUILD_DIR to the directory containing them." >&2
exit 1
fi
BUILD_DIR="$(cd "${BUILD_DIR}" && pwd)"
xrpld_binary="${BUILD_DIR}/xrpld"
if [[ ! -x "${xrpld_binary}" ]]; then
echo "build_pkg.sh: expected executable xrpld binary at ${xrpld_binary}." >&2
echo "Build xrpld before packaging, or set BUILD_DIR to the directory containing xrpld." >&2
validator_keys_binary="${BUILD_DIR}/validator-keys"
# Report both binaries at once: they share a single BUILD_DIR, so telling the
# reader to point it at one of them in isolation is advice they cannot follow.
missing=()
[[ -x "${xrpld_binary}" ]] || missing+=(xrpld)
[[ -x "${validator_keys_binary}" ]] || missing+=(validator-keys)
if [[ ${#missing[@]} -gt 0 ]]; then
echo "build_pkg.sh: missing or not executable in ${BUILD_DIR}: ${missing[*]}" >&2
echo "Both binaries come from a single CMake build directory configured with" >&2
echo "-Dxrpld=ON -Dvalidator_keys=ON. Build them, then point BUILD_DIR at that" >&2
echo "directory." >&2
exit 1
fi
# Shipping validator-keys means shipping its notice, so treat it as required
# rather than letting a package go out without the attribution.
validator_keys_license="${BUILD_DIR}/validator-keys-LICENSE"
if [[ ! -f "${validator_keys_license}" ]]; then
echo "build_pkg.sh: missing ${validator_keys_license}." >&2
echo "cmake/XrplValidatorKeys.cmake copies it out of the fetched" >&2
echo "validator-keys-tool source, so reconfigure with -Dvalidator_keys=ON." >&2
exit 1
fi
# The binary must also *run* here. Packaging happens in a vanilla distro
# container, so this is what catches a binary still pointing at the Nix store's
# ELF loader (see patch_nix_binary in cmake/PatchNixBinary.cmake); xrpld is
# covered implicitly by the version query below.
if ! "${validator_keys_binary}" --version >/dev/null; then
echo "build_pkg.sh: ${validator_keys_binary} exists but does not run here." >&2
exit 1
fi
@@ -150,7 +182,9 @@ stage_common() {
local dest="$1"
mkdir -p "${dest}"
cp "${BUILD_DIR}/xrpld" "${dest}/xrpld"
cp "${xrpld_binary}" "${dest}/xrpld"
cp "${validator_keys_binary}" "${dest}/validator-keys"
cp "${validator_keys_license}" "${dest}/validator-keys-LICENSE"
cp "${SRC_DIR}/cfg/xrpld-example.cfg" "${dest}/xrpld.cfg"
cp "${SRC_DIR}/cfg/validators-example.txt" "${dest}/validators.txt"
cp "${SRC_DIR}/LICENSE.md" "${dest}/LICENSE.md"

View File

@@ -18,6 +18,8 @@ Depends:
${shlibs:Depends},
${misc:Depends}
Description: XRP Ledger daemon
Reference implementation of the XRP Ledger protocol.
Participates in the peer-to-peer network, processes transactions,
and maintains a local ledger copy.
xrpld is the reference implementation of the XRP Ledger protocol. It
participates in the peer-to-peer XRP Ledger network, processes
transactions, and maintains the ledger database.
This package also includes the validator-keys tool for validator key
management.

View File

@@ -4,6 +4,25 @@ Source: https://github.com/XRPLF/rippled
Files: *
Copyright: 2011-present, the XRP Ledger developers
License: ISC
Files: validator-keys
Copyright: 2016, Ripple Labs Inc.
2011, Arthur Britto, David Schwartz, Jed McCaleb, Vinnie Falco, Bob Way,
Eric Lombrozo, Nikolaos D. Bougalis, Howard Hinnant
2013, Raw Material Software Ltd.
2003-2011, Christopher M. Kohlhoff
2009-2010, Satoshi Nakamoto
2011, The Bitcoin developers
2003-2005, Tom Wu
License: ISC
Comment: Built from https://github.com/ripple/validator-keys-tool at the commit
pinned in cmake/XrplValidatorKeys.cmake. Besides ISC-licensed code it
incorporates work under the Boost Software License 1.0 (ASIO), the MIT/X11
license (Bitcoin) and Tom Wu's license, whose terms require its notice to be
retained intact. The complete upstream notice is therefore shipped verbatim as
/usr/share/doc/xrpld/validator-keys-LICENSE.
License: ISC
Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above

View File

@@ -18,6 +18,7 @@ override_dh_installsysusers:
override_dh_install:
install -D -m 0755 xrpld debian/xrpld/usr/bin/xrpld
install -D -m 0755 validator-keys debian/xrpld/usr/bin/validator-keys
install -D -m 0644 xrpld.cfg debian/xrpld/etc/xrpld/xrpld.cfg
install -D -m 0644 validators.txt debian/xrpld/etc/xrpld/validators.txt

View File

@@ -1 +1,2 @@
README.md
validator-keys-LICENSE

View File

@@ -32,6 +32,8 @@ BuildRequires: systemd-rpm-macros
xrpld is the reference implementation of the XRP Ledger protocol. It
participates in the peer-to-peer XRP Ledger network, processes
transactions, and maintains the ledger database.
This package also includes the validator-keys tool for validator key
management.
%prep
:
@@ -41,6 +43,7 @@ transactions, and maintains the ledger database.
%install
install -Dm0755 %{_sourcedir}/xrpld %{buildroot}%{_bindir}/%{name}
install -Dm0755 %{_sourcedir}/validator-keys %{buildroot}%{_bindir}/validator-keys
install -Dm0644 %{_sourcedir}/xrpld.cfg %{buildroot}%{_sysconfdir}/%{name}/xrpld.cfg
install -Dm0644 %{_sourcedir}/validators.txt %{buildroot}%{_sysconfdir}/%{name}/validators.txt
@@ -59,6 +62,8 @@ install -Dm0644 %{_sourcedir}/xrpld.logrotate %{buildroot}%{_sysconfdir}/lo
# Docs
install -Dm0644 %{_sourcedir}/LICENSE.md %{buildroot}%{_docdir}/%{name}/LICENSE.md
install -Dm0644 %{_sourcedir}/README.md %{buildroot}%{_docdir}/%{name}/README.md
# Upstream notice for the bundled validator-keys tool.
install -Dm0644 %{_sourcedir}/validator-keys-LICENSE %{buildroot}%{_docdir}/%{name}/validator-keys-LICENSE
# Legacy compatibility for pre-FHS package layouts.
# TODO: remove after rippled fully deprecated.
@@ -80,11 +85,13 @@ systemd-tmpfiles --create %{_tmpfilesdir}/xrpld.conf || :
%files
%license %{_docdir}/%{name}/LICENSE.md
%license %{_docdir}/%{name}/validator-keys-LICENSE
%doc %{_docdir}/%{name}/README.md
%dir %{_sysconfdir}/%{name}
%{_bindir}/%{name}
%{_bindir}/validator-keys
%config(noreplace) %{_sysconfdir}/%{name}/xrpld.cfg
%config(noreplace) %{_sysconfdir}/%{name}/validators.txt

View File

@@ -29,7 +29,7 @@ src:test/beast/beast_PropertyStream_test.cpp
src:src/test/app/Invariants_test.cpp
# ASan false positive: stack-use-after-scope in ErrorCodes.h inline functions.
# When Clang inlines the StaticString overloads (e.g. invalid_field_error(StaticString)),
# When Clang inlines the StaticString overloads (e.g. invalidFieldError(StaticString)),
# ASan scope-poisons the temporary std::string before the inlined callee finishes reading
# through the const ref. This corrupts the coroutine stack and crashes the Simulate test.
# See asan.supp comments for full explanation and planned fix.

View File

@@ -277,6 +277,25 @@ public:
void
doDropDigit(T& mantissa, int& exponent) noexcept;
// Modify the result to the correctly rounded value
template <UnsignedMantissa T>
void
doRoundUp(bool& negative, T& mantissa, int& exponent, std::string location);
// Modify the result to the correctly rounded value
template <UnsignedMantissa T>
void
doRoundDown(bool& negative, T& mantissa, int& exponent) const;
// Modify the result to the correctly rounded value
void
doRound(rep& drops, std::string location) const;
private:
template <UnsignedMantissa T>
void
pushOverflow(T mantissa);
enum class Round {
// The result is exact. No rounding is needed. Only used if cuspRoundingFix is Enabled330 or
// higher.
@@ -289,37 +308,22 @@ public:
// The result was exactly half-way between two integers. This will round to even.
Even = 0,
// Round up. Always adds 1 (or subtracts 1 in some cases if cuspRoundingFix is not
// Enabled)
// Enabled330)
Up = 1,
};
// Indicate round direction: 1 is up, -1 is down, 0 is even
// Indicate round direction. See Round enum above.
// This enables the client to round towards nearest, and on
// tie, round towards even.
[[nodiscard]] Round
round() const noexcept;
// Modify the result to the correctly rounded value
template <UnsignedMantissa T>
void
doRoundUp(bool& negative, T& mantissa, int& exponent, std::string location);
// Modify the result to the correctly rounded value
template <UnsignedMantissa T>
void
doRoundDown(bool& negative, T& mantissa, int& exponent);
// Modify the result to the correctly rounded value
void
doRound(rep& drops, std::string location) const;
private:
void
doPush(unsigned d) noexcept;
template <UnsignedMantissa T>
void
bringIntoRange(bool& negative, T& mantissa, int& exponent);
bringIntoRange(bool& negative, T& mantissa, int& exponent) const;
};
inline void
@@ -349,6 +353,7 @@ Number::Guard::isNegative() const noexcept
inline void
Number::Guard::doPush(unsigned d) noexcept
{
XRPL_ASSERT(d < 10, "xrpl::Number::Guard::doPush : valid digit");
xbit_ = xbit_ || ((digits_ & 0x0000'0000'0000'000F) != 0);
digits_ >>= 4;
digits_ |= (d & 0x0000'0000'0000'000FULL) << 60;
@@ -396,10 +401,69 @@ Number::Guard::doDropDigit<uint128_t>(uint128_t& mantissa, int& exponent) noexce
++exponent;
}
template <UnsignedMantissa T>
void
Number::Guard::pushOverflow(T mantissa)
{
XRPL_ASSERT(mantissa <= kMaxRepUp, "xrpl::Number::Guard::pushOverflow : valid mantissa");
if (cuspRoundingFix >= MantissaRange::CuspRoundingFix::Enabled330 && mantissa >= kMaxRep &&
mantissa < kMaxRepUp)
{
// Special case rounding rules for the values in the range [kMaxRep, kMaxRepUp).
auto constexpr spread = kMaxRepUp - kMaxRep;
static_assert(spread == 3);
// Round in two steps.
// The first step uses the digits _already_ in the Guard to possibly round the mantissa up.
// Ultimately, the purpose of this step is to capture rounding where the stored digits would
// change the decision without those digits. (e.g. From just _below_ the midpoint to just
// _above_ the midpoint for ToNearest, or from kMaxRep into the in-between for Upward. Make
// an exception if the final digit is 9, because it can only get larger, and we don't want
// to bump up to kMaxRepUp.
if (mantissa % 10 < 9)
{
// Intentionally use integer math to get the largest value under the midpoint.
auto constexpr kMidpoint = kMaxRep + (spread / 2);
static_assert(kMidpoint == kMaxRep + 1);
auto const r = round();
if (r == Round::Up || (r == Round::Even && mantissa == kMidpoint))
{
++mantissa;
}
}
// The second step scales the final digit of the updated mantissa proportionally, converting
// from (kMaxRep, kMaxRepUp) to (0 to 9]. It then pushes that scaled digit onto the guard as
// if it was a digit that got removed, but doesn't actually remove it. This method should be
// future-proof in case the number of mantissa bits ever changes. (Though for integer values
// of the form 2^(2^x-1), the spread will always be the same.) Effects:
// * For round to nearest
// * if the updated mantissa is below the midpoint, it'll round "down" to kMaxRep
// * if above the midpoint, it'll round "up" to kMaxRepUp
// * it can never be exactly at the midpoint, because kMaxRepUp is always even, and
// kMaxRep is always odd, so don't worry about that case.
// * For round upward, will round up to kMaxRepUp for positive values, down to kMaxRep for
// negative.
// * For round downward, does the opposite of upward.
// * For round toward zero, always rounds down to kMaxRep.
auto const diff = mantissa - kMaxRep;
auto const digit = static_cast<unsigned>((diff * 10) / spread);
XRPL_ASSERT(
digit < 10u && digit != 5, "xrpl::Number::Guard::pushOverflow : valid overflow digit");
// Don't remove the digit from the mantissa, but add it to the guard as if it was.
push(digit);
}
}
// Returns:
// -1 if Guard is less than half
// 0 if Guard is exactly half
// 1 if Guard is greater than half
// Exact if Guard is _zero_, and appropriate amendments are enabled
// Down if Guard is less than half
// Even if Guard is exactly half
// Up if Guard is greater than half
Number::Guard::Round
Number::Guard::round() const noexcept
{
@@ -445,17 +509,23 @@ Number::Guard::round() const noexcept
template <UnsignedMantissa T>
void
Number::Guard::bringIntoRange(bool& negative, T& mantissa, int& exponent)
Number::Guard::bringIntoRange(bool& negative, T& mantissa, int& exponent) const
{
// Bring mantissa back into the minMantissa / maxMantissa range AFTER
// rounding
if (mantissa < minMantissa)
// rounding.
if (mantissa < minMantissa &&
(cuspRoundingFix < MantissaRange::CuspRoundingFix::Enabled330 || mantissa != 0))
{
mantissa *= 10;
--exponent;
}
if (exponent < kMinExponent)
// mantissa should never be 0, but if it _is_ assert, but fall back to making the result kZero.
if (exponent < kMinExponent ||
(cuspRoundingFix >= MantissaRange::CuspRoundingFix::Enabled330 && mantissa == 0))
{
// Engineers: If you hit this assert, you probably did something wrong in the operation
// leading up to the rounding work.
XRPL_ASSERT(mantissa != 0, "xrpl::Number::Guard::bringIntoRange : valid mantissa");
static constexpr Number kZero = Number{};
negative = kZero.negative_;
@@ -468,7 +538,9 @@ template <UnsignedMantissa T>
void
Number::Guard::doRoundUp(bool& negative, T& mantissa, int& exponent, std::string location)
{
auto r = round();
pushOverflow(mantissa);
auto const r = round();
if (r == Round::Up || (r == Round::Even && (mantissa & 1) == 1))
{
auto const safeToIncrement = [this](auto const& mantissa) {
@@ -485,18 +557,29 @@ Number::Guard::doRoundUp(bool& negative, T& mantissa, int& exponent, std::string
}
else
{
// Incrementing the mantissa will require dividing, which will require rounding. So
// _don't_ increment the mantissa. Instead, divide and round recursively. It should
// be impossible to recurse more than once, because once the mantissa is divided by
// 10, it will be _well_ under maxMantissa and kMaxRep, so adding 1 will have no
// chance of bringing it back over.
doDropDigit(mantissa, exponent);
XRPL_ASSERT_PARTS(
safeToIncrement(mantissa),
"xrpl::Number::Guard::doRoundUp",
"can't recurse more than once");
doRoundUp(negative, mantissa, exponent, location);
return;
if (cuspRoundingFix >= MantissaRange::CuspRoundingFix::Enabled330 &&
mantissa > kMaxRep && mantissa < kMaxRepUp)
{
// When rounding up a value in between kMaxRep, and kMaxRepUp, round to
// kMaxRepUp. Note that the decision for this rounding is dominated by the
// results of pushOverflow.
mantissa = kMaxRepUp;
}
else
{
// Incrementing the mantissa will require dividing, which will require rounding.
// So _don't_ increment the mantissa. Instead, divide and round recursively. It
// should be impossible to recurse more than once, because once the mantissa is
// divided by 10, it will be _well_ under maxMantissa and kMaxRep, so adding 1
// will have no chance of bringing it back over.
doDropDigit(mantissa, exponent);
XRPL_ASSERT_PARTS(
safeToIncrement(mantissa),
"xrpl::Number::Guard::doRoundUp",
"can't recurse more than once");
doRoundUp(negative, mantissa, exponent, location);
return;
}
}
}
else
@@ -514,6 +597,14 @@ Number::Guard::doRoundUp(bool& negative, T& mantissa, int& exponent, std::string
}
}
}
else if (
cuspRoundingFix >= MantissaRange::CuspRoundingFix::Enabled330 && mantissa > kMaxRep &&
mantissa < kMaxRepUp)
{
// When rounding down a value in between kMaxRep, and kMaxRepUp, round to kMaxRep.
// Note that the decision for this rounding is dominated by the results of pushOverflow.
mantissa = kMaxRep;
}
bringIntoRange(negative, mantissa, exponent);
if (exponent > kMaxExponent)
Throw<std::overflow_error>(std::string(location));
@@ -521,8 +612,10 @@ Number::Guard::doRoundUp(bool& negative, T& mantissa, int& exponent, std::string
template <UnsignedMantissa T>
void
Number::Guard::doRoundDown(bool& negative, T& mantissa, int& exponent)
Number::Guard::doRoundDown(bool& negative, T& mantissa, int& exponent) const
{
// Do not pushOverflow here.
auto r = round();
if (cuspRoundingFix >= MantissaRange::CuspRoundingFix::Enabled330)
{
@@ -557,6 +650,8 @@ Number::Guard::doRoundDown(bool& negative, T& mantissa, int& exponent)
void
Number::Guard::doRound(rep& drops, std::string location) const
{
// Do not pushOverflow here.
auto r = round();
if (r == Round::Up || (r == Round::Even && (drops & 1) == 1))
{
@@ -573,6 +668,8 @@ Number::Guard::doRound(rep& drops, std::string location) const
}
++drops;
}
XRPL_ASSERT(drops >= 0, "xrpl::Number::Guard::doRound : positive magnitude");
if (isNegative())
drops = -drops;
}
@@ -622,7 +719,9 @@ doNormalize(
{
static constexpr auto kMinExponent = Number::kMinExponent;
static constexpr auto kMaxExponent = Number::kMaxExponent;
static constexpr auto kMaxRep = Number::kMaxRep;
auto const repLimit = cuspRoundingFix >= MantissaRange::CuspRoundingFix::Enabled330
? Number::kMaxRepUp
: Number::kMaxRep;
using Guard = Number::Guard;
@@ -672,17 +771,17 @@ doNormalize(
// 9,900,000,000,000,123,450 or 9,900,000,000,000,123,460.
// mantissa() will return mantissa / 10, and exponent() will return
// exponent + 1.
if (m > kMaxRep)
if (m > repLimit)
{
if (exponent >= kMaxExponent)
throw std::overflow_error("Number::normalize 1.5");
g.doDropDigit(m, exponent);
}
// Before modification, m should be within the min/max range. After
// modification, it must be less than kMaxRep. In other words, the original
// value should have been no more than kMaxRep * 10.
// (kMaxRep * 10 > maxMantissa)
XRPL_ASSERT_PARTS(m <= kMaxRep, "xrpl::doNormalize", "intermediate mantissa fits in int64");
// modification, it must be less than repLimit. In other words, the original
// value should have been no more than repLimit * 10.
// (repLimit * 10 > maxMantissa)
XRPL_ASSERT_PARTS(m <= repLimit, "xrpl::doNormalize", "intermediate mantissa fits in limit");
mantissa = m;
g.doRoundUp(negative, mantissa, exponent, "Number::normalize 2");
@@ -814,6 +913,9 @@ Number::operator+=(Number const& y)
auto const& maxMantissa = g.maxMantissa;
auto const cuspRoundingFix = g.cuspRoundingFix;
auto const repLimit =
cuspRoundingFix >= MantissaRange::CuspRoundingFix::Enabled330 ? kMaxRepUp : kMaxRep;
// Bring the exponents of both values into agreement, so the mantissas are on the same scale
// and can be added directly together.
@@ -898,7 +1000,7 @@ Number::operator+=(Number const& y)
}
else
{
if (xm > maxMantissa || xm > kMaxRep)
if (xm > maxMantissa || xm > repLimit)
{
g.doDropDigit(xm, xe);
}
@@ -942,7 +1044,7 @@ Number::operator+=(Number const& y)
{
// Grow xm/xe and pull digits out of the Guard until it's back in the
// minMantissa/maxMantissa range.
while (xm < minMantissa && xm * 10 <= kMaxRep)
while (xm < minMantissa && xm * 10 <= repLimit)
{
xm *= 10;
xm -= g.pop();
@@ -1016,8 +1118,10 @@ Number::operator*=(Number const& y)
g.setNegative();
auto const& maxMantissa = g.maxMantissa;
auto const repLimit =
g.cuspRoundingFix >= MantissaRange::CuspRoundingFix::Enabled330 ? kMaxRepUp : kMaxRep;
while (zm > maxMantissa || zm > kMaxRep)
while (zm > maxMantissa || zm > repLimit)
{
g.doDropDigit(zm, ze);
}
@@ -1282,8 +1386,11 @@ to_string(Number const& amount)
}
std::string ret = negative ? "-" : "";
ret.append(std::to_string(mantissa));
ret.append(1, 'e');
ret.append(std::to_string(exponent));
if (exponent != 0)
{
ret.append(1, 'e');
ret.append(std::to_string(exponent));
}
return ret;
}

View File

@@ -76,24 +76,6 @@ getInverse()
return &kTab[0];
}
/**
* Returns max chars needed to encode a base64 string
*/
constexpr std::size_t
encodedSize(std::size_t n)
{
return 4 * ((n + 2) / 3);
}
/**
* Returns max bytes needed to decode a base64 string
*/
constexpr std::size_t
decodedSize(std::size_t n)
{
return ((n / 4) * 3) + 2;
}
/**
* Encode a series of octets as a padded, base64 string.
*

View File

@@ -127,15 +127,11 @@ Workers::deleteWorkers(beast::LockFreeStack<Worker>& stack)
{
Worker const* const worker = stack.popFront();
if (worker != nullptr)
{
// This call blocks until the thread orderly exits
delete worker;
}
else
{
if (worker == nullptr)
break;
}
// This call blocks until the thread orderly exits
delete worker;
}
}

View File

@@ -314,8 +314,7 @@ Value::~Value()
case ValueType::Array:
case ValueType::Object:
if (value_.mapVal != nullptr)
delete value_.mapVal;
delete value_.mapVal;
break;
// LCOV_EXCL_START

View File

@@ -28,6 +28,7 @@
#include <xrpl/protocol/UintTypes.h>
#include <xrpl/protocol/XRPAmount.h>
#include <algorithm>
#include <cstdint>
#include <initializer_list>
#include <limits>
@@ -80,13 +81,9 @@ isAnyFrozen(
return true;
}
for (auto const& account : accounts)
{
if (isVaultPseudoAccountFrozen(view, account, mptIssue, depth))
return true;
}
return false;
return std::ranges::any_of(accounts, [&](auto const& account) {
return isVaultPseudoAccountFrozen(view, account, mptIssue, depth);
});
}
Rate

View File

@@ -28,6 +28,7 @@
#include <xrpl/protocol/UintTypes.h>
#include <xrpl/protocol/XRPAmount.h>
#include <algorithm>
#include <cstdint>
#include <initializer_list>
#include <limits>
@@ -105,12 +106,9 @@ isAnyFrozen(
std::initializer_list<AccountID> const& accounts,
Issue const& issue)
{
for (auto const& account : accounts)
{
if (isFrozen(view, account, issue.currency, issue.account))
return true;
}
return false;
return std::ranges::any_of(accounts, [&](auto const& account) {
return isFrozen(view, account, issue.currency, issue.account);
});
}
bool

View File

@@ -13,6 +13,7 @@
#include <xrpl/nodestore/Scheduler.h>
#include <xrpl/nodestore/Types.h>
#include <atomic>
#include <cstdint>
#include <exception>
#include <functional>
@@ -52,6 +53,7 @@ DatabaseRotatingImp::rotate(
// callback finishes. Only then will the archive directory be
// deleted.
std::shared_ptr<NodeStore::Backend> oldArchiveBackend;
std::uint64_t copyForwards = 0;
{
std::scoped_lock const lock(mutex_);
@@ -62,11 +64,28 @@ DatabaseRotatingImp::rotate(
newArchiveBackendName = archiveBackend_->getName();
writableBackend_ = std::move(newBackend);
copyForwards = copyForwardCount_.exchange(0, std::memory_order_relaxed);
}
if (copyForwards > 0)
{
JLOG(j_.warn()) << "Rotating: copied forward " << copyForwards
<< " archive-served reads into the writable backend "
"during the rotation window";
}
f(newWritableBackendName, newArchiveBackendName);
}
void
DatabaseRotatingImp::setRotationInFlight(bool inFlight)
{
rotationInFlight_.store(inFlight, std::memory_order_release);
JLOG(j_.debug()) << "Rotating: copy-forward on archive reads "
<< (inFlight ? "enabled" : "disabled");
}
std::string
DatabaseRotatingImp::getName() const
{
@@ -177,9 +196,18 @@ DatabaseRotatingImp::fetchNodeObject(
writable = writableBackend_;
}
// Update writable backend with data from the archive backend
if (duplicate)
// Update writable backend with data from the archive backend.
// While a rotation is in flight, ordinary (duplicate == false)
// reads served by the archive are copied forward too: the
// archive is about to be deleted, and a body canonicalized
// into the cache after the freshen getKeys() snapshot would
// otherwise survive only in RAM once the archive is dropped.
if (duplicate || rotationInFlight_.load(std::memory_order_acquire))
{
if (!duplicate)
copyForwardCount_.fetch_add(1, std::memory_order_relaxed);
writable->store(nodeObject);
}
}
}

View File

@@ -23,7 +23,7 @@ namespace {
//------------------------------------------------------------------------------
// clang-format off
// NOLINTNEXTLINE(readability-identifier-naming)
char const* const versionString = "3.3.0-b1"
char const* const versionString = "3.3.0"
// clang-format on
;

View File

@@ -17,6 +17,7 @@
#include <xrpl/protocol/SeqProxy.h>
#include <xrpl/protocol/UintTypes.h>
#include <xrpl/protocol/digest.h>
#include <xrpl/protocol/jss.h>
#include <xrpl/protocol/nftPageMask.h>
#include <boost/endian/conversion.hpp>
@@ -32,6 +33,23 @@
namespace xrpl {
// This list should include all of the keylet functions that take a single
// AccountID parameter. Declared in Indexes.h; defined here so the header need
// not include jss.h.
std::array<KeyletDesc<AccountID const&>, 6> const kDirectAccountKeylets{
{{.function = &keylet::account, .expectedLEName = jss::AccountRoot, .includeInTests = false},
{.function = &keylet::ownerDir, .expectedLEName = jss::DirectoryNode, .includeInTests = true},
{.function = &keylet::signerList, .expectedLEName = jss::SignerList, .includeInTests = true},
// It's normally impossible to create an item at nftpage_min, but
// test it anyway, since the invariant checks for it.
{.function = &keylet::nftokenPageMin,
.expectedLEName = jss::NFTokenPage,
.includeInTests = true},
{.function = &keylet::nftokenPageMax,
.expectedLEName = jss::NFTokenPage,
.includeInTests = true},
{.function = &keylet::did, .expectedLEName = jss::DID, .includeInTests = true}}};
/**
* Type-specific prefix for calculating ledger indices.
*

View File

@@ -45,7 +45,7 @@ setCurrentTransactionRules(std::optional<Rules> r)
// amendments must also be added to useRulesGuards.
bool const enableLargeNumbers =
!r || (r->enabled(featureSingleAssetVault) || r->enabled(featureLendingProtocol));
// If enableLargeNumbers is true, then useRulesGuard must also return true.
// If enableLargeNumbers is true, then useRulesGuards must also return true.
// However, the reverse is not true. Other amendments can cause the rules guard to be used,
// even though large numbers are _not_ used.
XRPL_ASSERT(

View File

@@ -255,8 +255,47 @@ numberFromJson(SField const& field, json::Value const& value)
Throw<std::runtime_error>("not a number");
}
return STNumber{
field, Number{parts.negative, parts.mantissa, parts.exponent, Number::Normalized{}}};
Number const num{parts.negative, parts.mantissa, parts.exponent, Number::Normalized{}};
// Canonicalize "parts" and "num" with each other by getting rid of trailing 0s until either the
// exponents match, or there are no more 0s. If the two results don't match exactly, then the
// value has been rounded one way or another, and should not be used, because it may lead to an
// unexpected result. canonicalizeParts is not to be confused with Number::canonicalize, because
// they have completely different goals.
auto canonicalizeParts = [](NumberParts p, int otherExponent) {
if (p.mantissa == 0)
return NumberParts{};
while (p.exponent < otherExponent && p.mantissa % 10 == 0)
{
p.mantissa /= 10;
++p.exponent;
}
return p;
};
auto const numberMantissa = num.mantissa();
auto const numberExponent = num.exponent();
auto const canonicalParts = canonicalizeParts(parts, numberExponent);
auto const canonicalNum = canonicalizeParts(
NumberParts{
.mantissa = Number::externalToInternal(numberMantissa),
.exponent = numberExponent,
.negative = numberMantissa < 0,
},
canonicalParts.exponent);
if (canonicalParts.mantissa != canonicalNum.mantissa ||
canonicalParts.exponent != canonicalNum.exponent ||
canonicalParts.negative != canonicalNum.negative)
{
Throw<std::runtime_error>("number cannot be represented");
}
return STNumber{field, num};
}
} // namespace xrpl

View File

@@ -56,10 +56,15 @@ STObject::STObject(SOTemplate const& type, SField const& name) : STBase(name)
set(type);
}
STObject::STObject(SOTemplate const& type, SerialIter& sit, SField const& name) : STBase(name)
STObject::STObject(
SOTemplate const& type,
SerialIter& sit,
SField const& name,
bool requireCanonicalOrder)
: STBase(name)
{
v_.reserve(type.size());
set(sit);
set(sit, 0, requireCanonicalOrder);
applyTemplate(type); // May throw
}
@@ -208,12 +213,13 @@ STObject::applyTemplateFromSField(SField const& sField)
// return true = terminated with end-of-object
bool
STObject::set(SerialIter& sit, int depth)
STObject::set(SerialIter& sit, int depth, bool requireCanonicalOrder)
{
bool reachedEndOfObject = false;
v_.clear();
std::optional<int> prevFieldCode;
// Consume data in the pipe until we run out or reach the end
while (!sit.empty())
{
@@ -238,7 +244,6 @@ STObject::set(SerialIter& sit, int depth)
}
auto const& fn = SField::getField(type, field);
if (fn.isInvalid())
{
JLOG(debugLog().error())
@@ -246,6 +251,13 @@ STObject::set(SerialIter& sit, int depth)
Throw<std::runtime_error>("Unknown field");
}
if (requireCanonicalOrder && prevFieldCode.has_value() && fn.fieldCodeMem <= *prevFieldCode)
{
JLOG(debugLog().error()) << "Fields in object are not in canonical order";
Throw<std::runtime_error>("Fields in object are not in canonical order");
}
prevFieldCode = fn.fieldCodeMem;
// Unflatten the field
v_.emplace_back(sit, fn, depth + 1);
@@ -633,20 +645,6 @@ STObject::getAccountID(SField const& field) const
return getFieldByValue<STAccount>(field);
}
AccountID
STObject::getInitiator() const
{
// If sfDelegate is present, the delegate account is the initiator
// note: if a delegate is specified, its authorization to act on behalf of the account is
// enforced in `Transactor::invokeCheckPermission`
// cryptographic signature validity is checked separately (e.g., in `Transactor::checkSign`)
if (isFieldPresent(sfDelegate))
return getAccountID(sfDelegate);
// Default initiator
return getAccountID(sfAccount);
}
Blob
STObject::getFieldVL(SField const& field) const
{
@@ -710,7 +708,7 @@ STObject::getFieldNumber(SField const& field) const
void
STObject::set(std::unique_ptr<STBase> v)
{
set(std::move(*v.get()));
set(std::move(*v));
}
void

View File

@@ -13,6 +13,7 @@
#include <xrpl/protocol/UintTypes.h>
#include <xrpl/protocol/jss.h>
#include <algorithm>
#include <cstddef>
#include <stdexcept>
#include <utility>
@@ -50,6 +51,12 @@ STPathElement::getHash(STPathElement const& element)
return (hashAccount ^ hashCurrency ^ hashIssuer);
}
[[nodiscard]] size_t
STPathElement::getHash() const
{
return STPathElement::getHash(*this);
}
STPathSet::STPathSet(SerialIter& sit, SField const& name) : STBase(name)
{
std::vector<STPathElement> path;
@@ -125,21 +132,15 @@ STPathSet::move(std::size_t n, void* buf)
bool
STPathSet::assembleAdd(STPath const& base, STPathElement const& tail)
{ // assemble base+tail and add it to the set if it's not a duplicate
value_.push_back(base);
STPath combined = base;
combined.pushBack(tail);
auto it = value_.rbegin();
STPath& newPath = *it;
newPath.pushBack(tail);
while (++it != value_.rend())
if (!seenHashes_.insert(combined).second)
{
if (*it == newPath)
{
value_.pop_back();
return false;
}
return false;
}
value_.push_back(std::move(combined));
return true;
}
@@ -159,13 +160,10 @@ STPathSet::isDefault() const
bool
STPath::hasSeen(AccountID const& account, PathAsset const& asset, AccountID const& issuer) const
{
for (auto& p : path_)
{
if (p.getAccountID() == account && p.getPathAsset() == asset && p.getIssuerID() == issuer)
return true;
}
return false;
return std::ranges::any_of(path_, [&](auto& p) {
return p.getAccountID() == account && p.getPathAsset() == asset &&
p.getIssuerID() == issuer;
});
}
json::Value

View File

@@ -72,7 +72,7 @@ STTx::STTx(STObject&& object)
{
applyTemplate(getTxFormat(txType_)->getSOTemplate()); // may throw
tid_ = getHash(HashPrefix::TransactionId);
buildBatchTxnIds();
buildBatchTxns();
}
STTx::STTx(SerialIter& sit) : STObject(sfTransaction)
@@ -89,7 +89,7 @@ STTx::STTx(SerialIter& sit) : STObject(sfTransaction)
applyTemplate(getTxFormat(txType_)->getSOTemplate()); // May throw
tid_ = getHash(HashPrefix::TransactionId);
buildBatchTxnIds();
buildBatchTxns();
}
STTx::STTx(TxType type, std::function<void(STObject&)> assembler) : STObject(sfTransaction)
@@ -110,7 +110,7 @@ STTx::STTx(TxType type, std::function<void(STObject&)> assembler) : STObject(sfT
logicError("Transaction type was mutated during assembly");
tid_ = getHash(HashPrefix::TransactionId);
buildBatchTxnIds();
buildBatchTxns();
}
STBase*
@@ -279,12 +279,9 @@ STTx::checkSign(Rules const& rules) const
return std::unexpected("Sponsor: " + ret.error());
}
// Verify the batch signer signatures here too, so they are cached with the
// rest of signature checking (checkValidity / SF_SIGGOOD) and stay out of
// the transaction engine. Gated on a batch (batchTxnIds_ seated) that
// actually carries signers; a batch whose inners are all from the outer
// account has no sfBatchSigners and needs no signer crypto.
if (batchTxnIds_ && isFieldPresent(sfBatchSigners))
// Verify batch signer signatures here so they are cached with the rest
// of signature checking.
if (isFieldPresent(sfBatchSigners))
{
if (auto const ret = checkBatchSign(rules); !ret)
return ret;
@@ -307,11 +304,28 @@ STTx::checkBatchSign(Rules const& rules) const
if (!isFieldPresent(sfBatchSigners))
return std::unexpected("Missing BatchSigners field."); // LCOV_EXCL_LINE
STArray const& signers{getFieldArray(sfBatchSigners)};
// Bound signature verification to the protocol cap. This runs in
// checkValidity (via checkSign) at relay / submit time, BEFORE preflight
// and passesLocalChecks enforce the cap. Without this guard a malicious
// peer could put an oversized sfBatchSigners array in a 1 MB blob and
// force one signature verification per entry before any of those checks
// (or the fee charge) runs.
if (signers.size() > kMaxBatchSigners)
return std::unexpected("BatchSigners array exceeds max entries.");
// Defensive.
if (!batchTxns_)
{
// LCOV_EXCL_START
UNREACHABLE("STTx::checkBatchSign : batch transactions not built");
return std::unexpected("Missing inner transactions.");
// LCOV_EXCL_STOP
}
auto const txIds = getBatchTransactionIDs();
for (auto const& signer : signers)
{
Blob const& signingPubKey = signer.getFieldVL(sfSigningPubKey);
auto const result = signingPubKey.empty() ? checkBatchMultiSign(signer, rules)
: checkBatchSingleSign(signer);
auto const result = signingPubKey.empty() ? checkBatchMultiSign(signer, rules, txIds)
: checkBatchSingleSign(signer, txIds);
if (!result)
return result;
@@ -441,12 +455,11 @@ STTx::checkSingleSign(STObject const& sigObject) const
}
std::expected<void, std::string>
STTx::checkBatchSingleSign(STObject const& batchSigner) const
STTx::checkBatchSingleSign(STObject const& batchSigner, std::vector<uint256> const& txIds) const
{
XRPL_ASSERT(getTxnType() == ttBATCH, "STTx::checkBatchSingleSign : batch transaction");
Serializer msg;
serializeBatch(
msg, getAccountID(sfAccount), getSeqValue(), getFlags(), getBatchTransactionIDs());
serializeBatch(msg, getAccountID(sfAccount), getSeqValue(), getFlags(), txIds);
finishMultiSigningData(batchSigner.getAccountID(sfAccount), msg);
return singleSignHelper(batchSigner, msg.slice());
}
@@ -529,7 +542,10 @@ multiSignHelper(
}
std::expected<void, std::string>
STTx::checkBatchMultiSign(STObject const& batchSigner, Rules const& rules) const
STTx::checkBatchMultiSign(
STObject const& batchSigner,
Rules const& rules,
std::vector<uint256> const& txIds) const
{
XRPL_ASSERT(getTxnType() == ttBATCH, "STTx::checkBatchMultiSign : batch transaction");
// We can ease the computational load inside the loop a bit by
@@ -537,8 +553,7 @@ STTx::checkBatchMultiSign(STObject const& batchSigner, Rules const& rules) const
// with the stuff that stays constant from signature to signature.
auto const batchSignerAccount = batchSigner.getAccountID(sfAccount);
Serializer dataStart;
serializeBatch(
dataStart, getAccountID(sfAccount), getSeqValue(), getFlags(), getBatchTransactionIDs());
serializeBatch(dataStart, getAccountID(sfAccount), getSeqValue(), getFlags(), txIds);
dataStart.addBitString(batchSignerAccount);
return multiSignHelper(
batchSigner,
@@ -577,38 +592,79 @@ STTx::checkMultiSign(Rules const& rules, STObject const& sigObject) const
}
void
STTx::buildBatchTxnIds()
STTx::buildBatchTxns()
{
// Precondition: the template must have been applied first, so the fields
// (including sfRawTransactions) are canonical before the inner txns are
// hashed. The constructors call this immediately after applying the
// template; isFree() being false confirms a template is set.
XRPL_ASSERT(!isFree(), "STTx::buildBatchTxnIds : template applied");
if (getTxnType() != ttBATCH || !isFieldPresent(sfRawTransactions))
XRPL_ASSERT(!isFree(), "STTx::buildBatchTxns : template applied");
if (getTxnType() != ttBATCH)
return;
// A Batch always seats its inner transactions here, so every downstream
// consumer can rely on them. sfRawTransactions is required by the format
// (applyTemplate rejects a Batch without it); this guards a future change
// that made it optional.
if (!isFieldPresent(sfRawTransactions))
{
// LCOV_EXCL_START
UNREACHABLE("STTx::buildBatchTxns : missing RawTransactions");
Throw<std::runtime_error>("Batch has no RawTransactions.");
// LCOV_EXCL_STOP
}
auto const& raw = getFieldArray(sfRawTransactions);
if (raw.size() > kMaxBatchTxCount)
Throw<std::runtime_error>("Batch has too many inner transactions.");
// Seated for any batch with raw transactions. The count is validated in
// preflight and at the relay boundary, so build every id here; this keeps
// the invariant batchTxnIds_->size() == rawTransactions.size().
auto& ids = batchTxnIds_.emplace();
ids.reserve(raw.size());
// Build and validate each inner as an STTx once. A malformed inner throws;
// a nested batch is rejected before building it (a batch cannot contain a
// batch, and building one would recurse).
auto& txns = batchTxns_.emplace();
txns.reserve(raw.size());
for (STObject const& rb : raw)
ids.push_back(rb.getHash(HashPrefix::TransactionId));
{
if (rb.getFieldU16(sfTransactionType) == ttBATCH)
Throw<std::runtime_error>("Batch inner transaction cannot be a Batch.");
txns.push_back(std::make_shared<STTx const>(STObject{rb}));
}
}
std::vector<uint256> const&
std::vector<uint256>
STTx::getBatchTransactionIDs() const
{
XRPL_ASSERT(getTxnType() == ttBATCH, "STTx::getBatchTransactionIDs : batch transaction");
auto const& txns = getBatchTransactions();
std::vector<uint256> ids;
ids.reserve(txns.size());
for (auto const& stx : txns)
ids.push_back(stx->getTransactionID());
return ids;
}
std::vector<std::shared_ptr<STTx const>> const&
STTx::getBatchTransactions() const
{
XRPL_ASSERT(getTxnType() == ttBATCH, "STTx::getBatchTransactions : batch transaction");
XRPL_ASSERT(batchTxns_.has_value(), "STTx::getBatchTransactions : batch transactions built");
XRPL_ASSERT(
batchTxnIds_.has_value(), "STTx::getBatchTransactionIDs : batch transaction IDs built");
XRPL_ASSERT(
batchTxnIds_->size() == getFieldArray(sfRawTransactions).size(),
"STTx::getBatchTransactionIDs : batch transaction IDs size mismatch");
// NOLINTNEXTLINE(bugprone-unchecked-optional-access): guarded by assert above
return *batchTxnIds_;
batchTxns_->size() == getFieldArray(sfRawTransactions).size(),
"STTx::getBatchTransactions : batch transactions size mismatch");
return *batchTxns_;
}
AccountID
STTx::getInitiator() const
{
// If sfDelegate is present, the delegate account is the initiator
// note: if a delegate is specified, its authorization to act on behalf of the account is
// enforced in `Transactor::invokeCheckPermission`
// cryptographic signature validity is checked separately (e.g., in `Transactor::checkSign`)
if (isFieldPresent(sfDelegate))
return getAccountID(sfDelegate);
// Default initiator
return getAccountID(sfAccount);
}
AccountID
@@ -754,86 +810,65 @@ invalidMPTAmountInTx(STObject const& tx)
}
static bool
isBatchRawTransactionOkay(STObject const& st, std::string& reason)
isBatchRawTransactionOkay(STTx const& tx, std::string& reason)
{
if (!st.isFieldPresent(sfRawTransactions))
XRPL_ASSERT(
tx.getTxnType() == ttBATCH || !tx.isFieldPresent(sfRawTransactions),
"xrpl::isBatchRawTransactionOkay : raw transactions only on batch");
if (tx.getTxnType() != ttBATCH)
return true;
// sfRawTransactions only appears on a Batch. passesLocalChecks runs on
// unverified user and peer input, so reject (rather than assert) a non-batch
// transaction that carries it.
if (st.getFieldU16(sfTransactionType) != ttBATCH)
if (!tx.isFieldPresent(sfRawTransactions))
{
reason = "Only Batch transactions may contain raw transactions.";
// LCOV_EXCL_START
reason = "Batch transactions must contain raw transactions.";
return false;
// LCOV_EXCL_STOP
}
if (tx.isFieldPresent(sfBatchSigners) &&
tx.getFieldArray(sfBatchSigners).size() > kMaxBatchSigners)
{
reason = "BatchSigners array exceeds max entries.";
return false;
}
if (st.isFieldPresent(sfBatchSigners) &&
st.getFieldArray(sfBatchSigners).size() > kMaxBatchSigners)
// Inner structure (type, template, no nesting, count) is validated when the
// batch STTx is constructed; here we only run each inner's local checks.
for (auto const& inner : tx.getBatchTransactions())
{
reason = "Batch Signers array exceeds max entries.";
return false;
}
auto const& rawTxns = st.getFieldArray(sfRawTransactions);
if (rawTxns.size() > kMaxBatchTxCount)
{
reason = "Raw Transactions array exceeds max entries.";
return false;
}
for (STObject raw : rawTxns)
{
try
{
auto const tt = safeCast<TxType>(raw.getFieldU16(sfTransactionType));
if (tt == ttBATCH)
{
reason = "Raw Transactions may not contain batch transactions.";
return false;
}
raw.applyTemplate(getTxFormat(tt)->getSOTemplate());
// passesLocalChecks recurses back into isBatchRawTransactionOkay,
// but an inner can never be a batch (rejected above), so the
// recursion terminates at depth 1.
if (!passesLocalChecks(raw, reason))
return false;
}
catch (std::exception const& e)
{
reason = e.what();
if (!passesLocalChecks(*inner, reason))
return false;
}
}
return true;
}
bool
passesLocalChecks(STObject const& st, std::string& reason)
passesLocalChecks(STTx const& tx, std::string& reason)
{
if (!isMemoOkay(st, reason))
if (!isMemoOkay(tx, reason))
return false;
if (!isAccountFieldOkay(st))
if (!isAccountFieldOkay(tx))
{
reason = "An account field is invalid.";
return false;
}
if (isPseudoTx(st))
if (isPseudoTx(tx))
{
reason = "Cannot submit pseudo transactions.";
return false;
}
if (invalidMPTAmountInTx(st))
if (invalidMPTAmountInTx(tx))
{
reason = "Amount can not be MPT.";
return false;
}
if (!isBatchRawTransactionOkay(st, reason))
if (!isBatchRawTransactionOkay(tx, reason))
return false;
return true;

View File

@@ -64,7 +64,7 @@ to_string(Currency const& currency)
bool
toCurrency(Currency& currency, std::string const& code)
{
if (code.empty() || (code.compare(systemCurrencyCode()) == 0))
if (code.empty() || code == systemCurrencyCode())
{
currency = beast::kZero;
return true;

View File

@@ -17,7 +17,7 @@
#include <string>
#include <utility>
#include <vector>
#if defined(__clang__)
#ifdef __clang__
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wdeprecated"
#endif
@@ -342,6 +342,6 @@ makeCheckpointer(
} // namespace xrpl
#if defined(__clang__)
#ifdef __clang__
#pragma clang diagnostic pop
#endif

View File

@@ -9,6 +9,7 @@ Charge const kFeeRequestNoReply(10, "unsatisfiable request");
Charge const kFeeInvalidSignature(2000, "invalid signature");
Charge const kFeeUselessData(150, "useless data");
Charge const kFeeInvalidData(400, "invalid data");
Charge const kFeeMalformedData(2000, "malformed data");
Charge const kFeeMalformedRpc(100, "malformed RPC");
Charge const kFeeReferenceRpc(20, "reference RPC");

View File

@@ -7,10 +7,12 @@
#include <xrpl/protocol/Book.h>
#include <xrpl/resource/Consumer.h>
#include <cstddef>
#include <cstdint>
#include <exception>
#include <memory>
#include <mutex>
#include <utility>
namespace xrpl {
@@ -64,6 +66,9 @@ InfoSub::InfoSub(Source& source, Consumer consumer)
InfoSub::~InfoSub()
{
// Stream unsubscribes are O(1): each erases this connection's single seq_
// from one stream map, so they are cheap enough to run inline on the
// disconnect thread.
// Each Source teardown call below acquires a server-side lock and
// can throw. Wrap each independent call so partial failure does not
// skip the remaining teardown steps.
@@ -79,29 +84,48 @@ InfoSub::~InfoSub()
safeUnsub(seq_, [&] { source_.unsubPeerStatus(seq_); }, j);
safeUnsub(seq_, [&] { source_.unsubConsensus(seq_); }, j);
// Use the internal unsubscribe so that it won't call
// back to us and modify its own parameter
if (!realTimeSubscriptions_.empty())
{
safeUnsub(
seq_, [&] { source_.unsubAccountInternal(seq_, realTimeSubscriptions_, true); }, j);
}
if (!normalSubscriptions_.empty())
{
safeUnsub(
seq_, [&] { source_.unsubAccountInternal(seq_, normalSubscriptions_, false); }, j);
}
for (auto const& account : accountHistorySubscriptions_)
{
safeUnsub(seq_, [&] { source_.unsubAccountHistoryInternal(seq_, account, false); }, j);
}
// Book subscriptions are torn down inline here, keyed on seq_, rather than
// through the chunked account cleanup below. The book set is not capped, so
// it can be large; but each unsubBookInternal takes bookLock_ for a single
// O(1) erase and releases it, so even a large set never holds a lock across
// the whole loop - a competing book publish can interleave between erases.
// The disconnect thread still does O(N) brief acquisitions. Use the internal
// variant so it does not write back to bookSubscriptions_ on this
// partially-destroyed object.
for (auto const& book : bookSubscriptions_)
{
safeUnsub(seq_, [&] { source_.unsubBookInternal(seq_, book); }, j);
}
// Hand the account sets off (by move) to the Source for a chunked,
// off-thread teardown keyed on seq_, instead of erasing them inline here.
// This keeps the destructor from holding the account lock across a large
// erase loop. The job never references this object, which is being
// destroyed.
//
// Moving the sets without holding lock_ is safe: the destructor runs only
// when the last shared_ptr to this InfoSub is released, so by the
// shared_ptr contract no other thread holds a reference. Subscription maps
// store weak_ptrs, so a concurrent publisher must weak_ptr::lock() first;
// that succeeds only while a strong reference exists, which cannot overlap
// with destruction. No other thread can observe the moved-from sets.
//
// Wrapped like the steps above: scheduleAccountCleanup enqueues a JobQueue
// task, which allocates and locks and so can throw. A throw out of this
// noexcept destructor would terminate the process. Skipping the cleanup on
// throw is harmless: the account/rt maps hold weak_ptrs that the next
// publish prunes once this InfoSub is gone, and any history paging job
// self-terminates when its weak sink can no longer be locked.
safeUnsub(
seq_,
[&] {
source_.scheduleAccountCleanup(
seq_,
std::move(realTimeSubscriptions_),
std::move(normalSubscriptions_),
std::move(accountHistorySubscriptions_));
},
j);
}
Resource::Consumer&
@@ -121,6 +145,53 @@ InfoSub::onSendEmpty()
{
}
std::size_t
InfoSub::totalSubscriptionCount() const
{
// Hold lock_ for the whole read so the three sets cannot be mutated
// mid-count by a concurrent (un)subscribe on this connection.
std::scoped_lock const sl(lock_);
// Combined tally the per-connection cap is enforced against.
return normalSubscriptions_.size() + realTimeSubscriptions_.size() +
accountHistorySubscriptions_.size();
}
bool
InfoSub::tryReserveAccountSubscriptions(
hash_set<AccountID> const& proposedAccounts,
hash_set<AccountID> const& normalAccounts,
std::size_t cap)
{
// One lock hold covers the count, the check and the insert.
std::scoped_lock const sl(lock_);
// Entries not already tracked; re-subscribing held accounts is not charged.
auto const countNew = [](hash_set<AccountID> const& requested,
hash_set<AccountID> const& existing) {
std::size_t fresh = 0;
for (auto const& account : requested)
{
if (!existing.contains(account))
++fresh;
}
return fresh;
};
std::size_t const additional = countNew(proposedAccounts, realTimeSubscriptions_) +
countNew(normalAccounts, normalSubscriptions_);
std::size_t const current = normalSubscriptions_.size() + realTimeSubscriptions_.size() +
accountHistorySubscriptions_.size();
if (exceedsSubscriptionCap(current, additional, cap))
return false;
realTimeSubscriptions_.insert(proposedAccounts.begin(), proposedAccounts.end());
normalSubscriptions_.insert(normalAccounts.begin(), normalAccounts.end());
return true;
}
void
InfoSub::insertSubAccountInfo(AccountID const& account, bool rt)
{
@@ -165,6 +236,13 @@ InfoSub::deleteSubAccountHistory(AccountID const& account)
accountHistorySubscriptions_.erase(account);
}
bool
InfoSub::hasAccountHistorySubscription(AccountID const& account) const
{
std::scoped_lock const sl(lock_);
return accountHistorySubscriptions_.contains(account);
}
void
InfoSub::insertBookSubscription(Book const& book)
{

Some files were not shown because too many files have changed in this diff Show More