Compare commits

...

53 Commits

Author SHA1 Message Date
Ayaz Salikhov
3c43f4614f release: Bump version to 3.2.0 2026-06-15 19:46:56 -04:00
dependabot[bot]
6b63f0ff61 ci: [DEPENDABOT] bump codecov/codecov-action from 6.0.1 to 7.0.0 (#7426)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-15 19:46:49 -04:00
Bart
0ac8e6cf1e release: Bump version to 3.2.0-rc6 2026-06-15 22:24:03 +01:00
Vito Tumas
ed5f13481a fix: Disable transaction invariants 2026-06-15 22:24:03 +01:00
Vito Tumas
781ef175c9 perf: Dispatch "hasInvalidAmount()" on type tag instead of dynamic_cast 2026-06-15 22:24:03 +01:00
Ed Hennis
e5785c4fcb fix: Fix Number comparison operator 2026-06-15 22:24:02 +01:00
Michael Legleux
96d0563ea6 fix: Adjust xrpld systemd service 2026-06-15 22:24:02 +01:00
Bart
61dae6f792 release: Bump version to 3.2.0-rc5 2026-06-15 22:24:02 +01:00
yinyiqian1
fded06652a fix: Add zero NFT Offer ID check for NFTokenCancelOffer 2026-06-15 22:24:02 +01:00
Valentin Balaschenko
e833e8884d refactor: Revert "Explicitly trim the heap after cache sweeps (#6022)" 2026-06-15 22:24:02 +01:00
Michael Legleux
8e3eabc398 refactor: Remove auto-update script and update RPM version
* refactor: Update RPM version scheme; remove auto-update script; service hardening

- **RPM version scheme**: pre-releases now use `~` in the `Version` field instead of the `0.<release>.<suffix>` `Release`-field hack. Matches Debian's `~` convention, so RPM and DEB version strings are symmetric. Requires rpm ≥ 4.10 (RHEL 9 ships 4.17).

  Before/after for a pre-release build:
  ```
  # before
  xrpld-3.2.0-0.1.rc3+202606011647.d4cb68d5.el9.x86_64.rpm

  # after (symmetric with DEB)
  xrpld-3.2.0~rc2+202606010139.7679a310-1.el9.x86_64.rpm
  xrpld_3.2.0~rc2+202606010139.7679a310-1_amd64.deb
  ```
- **Auto-update removed**: `update-xrpld`, `update-xrpld.service`, and `update-xrpld.timer` deleted. The `50-xrpld.preset` `disable` line for the timer is dropped too.
- **Service hardening** (two new `[Service]` directives in `xrpld.service`):
  - `CapabilityBoundingSet=CAP_NET_BIND_SERVICE` — drops every Linux capability except `CAP_NET_BIND_SERVICE`, capping the privilege ceiling to least-privilege while still letting operators bind ports <1024 (e.g. WS/HTTPS on 443).
  - `SystemCallArchitectures=native` — restricts the service to the native syscall ABI, blocking alternate-ABI (32-bit/x32) syscalls used to evade seccomp filtering.

- [ ] Build RPM from a pre-release version (e.g. `3.2.0-b1`) and confirm `rpm -qi` shows `Version: 3.2.0~b1`, `Release: 1`
- [ ] Confirm `3.2.0~b1` sorts before `3.2.0` via `rpmvercmp`
- [ ] Install package and confirm no `update-xrpld*` units appear in `systemctl list-unit-files`
- [ ] Confirm `systemctl show xrpld` reflects the new `CapabilityBoundingSet` and `SystemCallArchitectures`

* fix: Track tmpfiles-created directories in RPM %files as %ghost
2026-06-15 22:24:02 +01:00
Sergey Kuznetsov
47b06ecd17 refactor: Use rocksdb includes only when it is available 2026-06-15 22:23:54 +01:00
Bart
5a25c9188b release: Bump version to 3.2.0-rc4 2026-06-15 22:23:53 +01:00
Bart
82ee5b7556 refactor: Handle int and uint API versions separately 2026-06-15 22:23:38 +01:00
Pratik Mankawde
f98c251011 refactor: Improve tracking of book (un)subscriptions 2026-06-15 22:23:38 +01:00
Sergey Kuznetsov
e29dc474b3 refactor: Improve payment channel closing and returned error codes 2026-06-15 22:23:28 +01:00
Pratik Mankawde
2728e11809 fix: Set request size limits and differential pricing for get-object-by-hash calls 2026-06-15 22:23:28 +01:00
Jingchen
9650fe8a6e refactor: Use explicit types to help compiler 2026-06-15 22:22:53 +01:00
Bart
7e15621e7b release: Bump version to 3.2.0-rc3 (#7371)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-05-31 22:55:18 +00:00
Vito Tumas
99431d7833 fix: Pin overpayment principal reduction to exact on-grid value (#7360) 2026-05-31 22:54:23 +00:00
Ed Hennis
47365f4220 fix: Improve upward rounding edge cases for Number::operator/= (#7328)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Vito Tumas <5780819+Tapanito@users.noreply.github.com>
2026-05-31 00:23:29 +00:00
Bart
1599c1a672 refactor: Revert "perf: Remove unnecessary caches (#5439)" (#7359)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-05-30 18:48:59 +00:00
yinyiqian1
763dd503be fix: Add zero domainID check for permissionedDomain (#7362) 2026-05-30 00:16:25 +00:00
Bart
2f3558c610 ci: Run PR title and description checks on staging and release branches (#7331)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-05-28 14:57:29 +00:00
Ayaz Salikhov
f9551ac5ca style: Run shfmt on workflows, actions and markdown bash code (#7333) 2026-05-27 19:24:18 +00:00
Bart
1acc42313c release: Bump version to 3.2.0-rc2 (#7348) 2026-05-27 15:11:38 -04:00
Bart
396d772a15 refactor: Enable support for fixCleanup3_2_0 amendment (#7347) 2026-05-27 19:10:33 +00:00
Ayaz Salikhov
1438bf1c67 release: Bump version to 3.2.0-rc1 (#7335) 2026-05-27 13:20:57 -04:00
Ed Hennis
7da643d864 fix: Fix a rounding error at the Number::maxRep cusp (#7051)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Vito Tumas <5780819+Tapanito@users.noreply.github.com>
2026-05-27 15:19:20 +00:00
Ayaz Salikhov
1162371def ci: Only push docker images in XRPLF/rippled (#7330) 2026-05-26 20:03:04 +00:00
dependabot[bot]
2a0feca46b ci: [DEPENDABOT] bump docker/setup-buildx-action from 4.0.0 to 4.1.0 (#7322)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-26 19:36:32 +00:00
dependabot[bot]
108a4c8217 ci: [DEPENDABOT] bump codecov/codecov-action from 6.0.0 to 6.0.1 (#7321)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-26 19:36:21 +00:00
dependabot[bot]
4584b01bde ci: [DEPENDABOT] bump docker/build-push-action from 7.1.0 to 7.2.0 (#7320)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-26 19:36:13 +00:00
dependabot[bot]
7c59786565 ci: [DEPENDABOT] bump docker/metadata-action from 6.0.0 to 6.1.0 (#7319)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-26 19:36:00 +00:00
dependabot[bot]
9623e67b76 ci: [DEPENDABOT] bump docker/login-action from 4.1.0 to 4.2.0 (#7318)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-26 19:35:52 +00:00
Andrzej Budzanowski
85af406a0f fix: Update clang-tidy to include src/tests directory header check (#7307) 2026-05-26 19:35:32 +00:00
Ayaz Salikhov
ac33fb32a7 chore: Pin Python packages for codegen using uv (#7329) 2026-05-26 18:35:38 +00:00
Ayaz Salikhov
23d0812827 style: Use shfmt instead of bashate (#7326) 2026-05-26 18:28:23 +00:00
Vito Tumas
49567e7283 fix: Fix edge-case where vault-depositor may get stuck (#7139) 2026-05-26 18:18:40 +00:00
Vito Tumas
633ef4706f fix: Fix VaultInvariant and VaultDeposit precision bugs at IOU scale boundaries (#7272)
Co-authored-by: Bart <bthomee@users.noreply.github.com>
2026-05-26 16:32:44 +00:00
Ayaz Salikhov
49cb3f45a4 ci: Add clang to nix images (#7308)
Co-authored-by: semgrep-companion-app[bot] <218312740+semgrep-companion-app[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-26 15:45:33 +00:00
Vito Tumas
22a21b175e fix: Include management-fee delta in doOverpayment assertion (#7039) 2026-05-26 14:01:52 +00:00
Pratik Mankawde
e9d885bd9b fix: Fix clang-tidy pre-commit hook to locate compile_commands.json from repo root (#7325)
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-26 13:50:18 +00:00
Jingchen
a911f9089e fix: Use consistent scale for debtTotal (#7093)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-24 20:44:29 +00:00
Peter Chen
e34c2667d7 fix: Skip deleted book directories and non-root modifications in ValidBookDirectory invariant (#7312) 2026-05-24 20:37:16 +00:00
Valentin Balaschenko
30de556224 fix: Address review feedback on FD/handle guarding (#5823 follow-up) (#7310) 2026-05-23 14:48:48 +00:00
Gregory Tsipenyuk
dcd2ff0b5f fix: Fix non-canonical MPT amount (#7117)
Co-authored-by: xrplf-ai-reviewer[bot] <266832837+xrplf-ai-reviewer[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-23 06:40:26 +00:00
Bart
dfb9b8ed9a release: Bump version to 3.2.0-b7 (#7316)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-05-22 19:32:12 +00:00
Jingchen
179e73594a fix: Check if the MPT first loss cover can be sent to the broker before deleting the broker (#7125)
Co-authored-by: xrplf-ai-reviewer[bot] <266832837+xrplf-ai-reviewer[bot]@users.noreply.github.com>
2026-05-22 11:58:48 +00:00
Michael Legleux
15dd653e4b fix: Fix RPM prerelease ordering and start xrpld on DEB install (#7313) 2026-05-22 11:30:45 +00:00
Michael Legleux
a37afe13ff ci: Re-enable full nproc for Linux (#7315) 2026-05-22 11:30:37 +00:00
Gregory Tsipenyuk
3547a9335f fix: Add assorted MPT/DEX fixes (#7040)
Co-authored-by: xrplf-ai-reviewer[bot] <266832837+xrplf-ai-reviewer[bot]@users.noreply.github.com>
Co-authored-by: Shawn Xie <35279399+shawnxie999@users.noreply.github.com>
2026-05-21 18:29:53 +00:00
Bart
1a98182e23 refactor: Remove dead fetchBatch code (#7309)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-05-21 17:52:41 +00:00
173 changed files with 9418 additions and 2382 deletions

View File

@@ -153,6 +153,7 @@ Checks: "-*,
readability-use-std-min-max
"
# ---
# bugprone-narrowing-conversions, # this will break a lot of code but we should enable it in the future because it can eliminate a lot of bugs
# readability-inconsistent-declaration-parameter-name, # in this codebase this check will break a lot of arg names
# readability-static-accessed-through-instance, # this check is probably unnecessary. it makes the code less readable
# ---
@@ -199,6 +200,6 @@ CheckOptions:
readability-identifier-naming.PublicMemberSuffix: ""
readability-identifier-naming.GlobalFunctionIgnoredRegexp: "^(to_string|hash_append|tuple_hash)$"
HeaderFilterRegex: '^.*/(test|xrpl|xrpld)/.*\.(h|hpp|ipp)$'
HeaderFilterRegex: '^.*/(tests?|xrpl|xrpld)/.*\.(h|hpp|ipp)$'
ExcludeHeaderFilterRegex: '^.*/protocol_autogen/.*\.(h|hpp)$'
WarningsAsErrors: "*"

View File

@@ -37,12 +37,12 @@ runs:
run: |
echo 'Installing dependencies.'
conan install \
--profile ci \
--build="${BUILD_OPTION}" \
--options:host='&:tests=True' \
--options:host='&:xrpld=True' \
--settings:all build_type="${BUILD_TYPE}" \
--conf:all tools.build:jobs=${BUILD_NPROC} \
--conf:all tools.build:verbosity="${LOG_VERBOSITY}" \
--conf:all tools.compilation:verbosity="${LOG_VERBOSITY}" \
.
--profile ci \
--build="${BUILD_OPTION}" \
--options:host='&:tests=True' \
--options:host='&:xrpld=True' \
--settings:all build_type="${BUILD_TYPE}" \
--conf:all tools.build:jobs=${BUILD_NPROC} \
--conf:all tools.build:verbosity="${LOG_VERBOSITY}" \
--conf:all tools.compilation:verbosity="${LOG_VERBOSITY}" \
.

View File

@@ -15,7 +15,7 @@ runs:
shell: bash
env:
VERSION: ${{ github.ref_name }}
run: echo "VERSION=${VERSION}" >> "${GITHUB_ENV}"
run: echo "VERSION=${VERSION}" >>"${GITHUB_ENV}"
# When a tag is not pushed, then the version (e.g. 1.2.3-b0) is extracted
# from the BuildInfo.cpp file and the shortened commit hash appended to it.
@@ -28,17 +28,17 @@ runs:
echo 'Extracting version from BuildInfo.cpp.'
VERSION="$(cat src/libxrpl/protocol/BuildInfo.cpp | grep "versionString =" | awk -F '"' '{print $2}')"
if [[ -z "${VERSION}" ]]; then
echo 'Unable to extract version from BuildInfo.cpp.'
exit 1
echo 'Unable to extract version from BuildInfo.cpp.'
exit 1
fi
echo 'Appending shortened commit hash to version.'
SHA='${{ github.sha }}'
VERSION="${VERSION}+${SHA:0:7}"
echo "VERSION=${VERSION}" >> "${GITHUB_ENV}"
echo "VERSION=${VERSION}" >>"${GITHUB_ENV}"
- name: Output version
id: version
shell: bash
run: echo "version=${VERSION}" >> "${GITHUB_OUTPUT}"
run: echo "version=${VERSION}" >>"${GITHUB_OUTPUT}"

403
.github/scripts/format-inline-bash.py vendored Executable file
View File

@@ -0,0 +1,403 @@
#!/usr/bin/env python3
"""
Format embedded shell snippets using the shfmt hook configured in
.pre-commit-config.yaml.
Two shapes are recognised:
* YAML workflow/action files: literal block-scalar runs (`run: |`) and
single-line runs (`run: some command`). A single-line run is upgraded to
a `run: |` block scalar if shfmt's output spans multiple lines.
* Markdown files: ``` ```bash ``` fenced code blocks.
Any block that shfmt cannot parse is skipped with a warning on stderr, so
the file is left untouched and surrounding blocks still get formatted.
For each occurrence the body is dedented, written to a temp .sh file,
formatted via `pre-commit run shfmt --files <temp>` (falling back to
`prek`), then re-indented and written back in place.
When invoked without arguments, every .yml/.yaml under .github/ plus every
.md file in the repo is scanned. When invoked with file arguments (the
pre-commit case), only those files are processed.
"""
from __future__ import annotations
import re
import shutil
import subprocess
import sys
import tempfile
from dataclasses import dataclass
from pathlib import Path
from typing import Union
REPO = Path(__file__).resolve().parents[2]
_HOOK_RUNNER = next((cmd for cmd in ("pre-commit", "prek") if shutil.which(cmd)), None)
if _HOOK_RUNNER is None:
sys.exit("error: neither `pre-commit` nor `prek` found on PATH")
RUN_BLOCK_RE = re.compile(r"^(?P<prefix>[ \t]*(?:- )?)run:[ \t]*\|[+-]?[ \t]*$")
RUN_INLINE_RE = re.compile(
r"^(?P<prefix>[ \t]*(?:- )?)run:[ \t]+" r"(?P<value>(?!\|[+-]?[ \t]*$)\S.*?)[ \t]*$"
)
MD_BASH_OPEN_RE = re.compile(r"^(?P<indent>[ ]{0,3})`{3}bash[ \t]*$")
MD_FENCE_CLOSE_RE = re.compile(r"^[ ]{0,3}`{3,}[ \t]*$")
@dataclass(frozen=True)
class BlockRun:
"""A `run: |` block scalar; `body_start:body_end` slices into `lines`."""
body_start: int
body_end: int
body_indent: int
@dataclass(frozen=True)
class InlineRun:
"""A single-line `run: value` at `line_idx`."""
line_idx: int
prefix: str
value: str
@dataclass(frozen=True)
class MdBashBlock:
"""A markdown ``` ```bash ``` fenced code block.
`body_start:body_end` slices into the file's lines; `open_line_idx`
points at the opening fence line.
"""
open_line_idx: int
body_start: int
body_end: int
body_indent: int
RunItem = Union[BlockRun, InlineRun]
def _scan_block_body(
lines: list[str], body_start: int, run_col: int
) -> tuple[int | None, int]:
"""Locate the body of a `run: |` block scalar starting at `body_start`.
Returns `(body_indent, scan_end)`. `scan_end` is the line index where the
outer scanner should resume. `body_indent` is `None` when no body is
present (the scalar is empty, or the next non-blank line has indent
`<= run_col`).
"""
body_indent: int | None = None
scan_end = len(lines)
for idx in range(body_start, len(lines)):
line = lines[idx]
if line.strip() == "":
continue
indent = len(line) - len(line.lstrip(" "))
if body_indent is None:
if indent > run_col:
body_indent = indent
else:
scan_end = idx
break
elif indent < body_indent:
scan_end = idx
break
if body_indent is not None:
while scan_end > body_start and lines[scan_end - 1].strip() == "":
scan_end -= 1
if scan_end <= body_start:
body_indent = None
return body_indent, scan_end
def find_run_blocks(lines: list[str]) -> list[RunItem]:
"""Return run items in document order."""
items: list[RunItem] = []
line_idx = 0
while line_idx < len(lines):
line = lines[line_idx]
if block_match := RUN_BLOCK_RE.match(line):
run_col = len(block_match.group("prefix"))
body_start = line_idx + 1
body_indent, scan_end = _scan_block_body(lines, body_start, run_col)
if body_indent is not None:
items.append(
BlockRun(
body_start=body_start,
body_end=scan_end,
body_indent=body_indent,
)
)
line_idx = scan_end
continue
if inline_match := RUN_INLINE_RE.match(line):
items.append(
InlineRun(
line_idx=line_idx,
prefix=inline_match.group("prefix"),
value=inline_match.group("value"),
)
)
line_idx += 1
return items
def find_md_bash_blocks(lines: list[str]) -> list[MdBashBlock]:
"""Return ``` ```bash ``` fenced code blocks in document order."""
blocks: list[MdBashBlock] = []
line_idx = 0
while line_idx < len(lines):
open_match = MD_BASH_OPEN_RE.match(lines[line_idx])
if not open_match:
line_idx += 1
continue
body_start = line_idx + 1
close_idx = next(
(
j
for j in range(body_start, len(lines))
if MD_FENCE_CLOSE_RE.match(lines[j])
),
None,
)
if close_idx is None:
line_idx = body_start
continue
body = lines[body_start:close_idx]
non_blank = [b for b in body if b.strip()]
body_indent = (
min(len(b) - len(b.lstrip(" ")) for b in non_blank)
if non_blank
else len(open_match.group("indent"))
)
blocks.append(
MdBashBlock(
open_line_idx=line_idx,
body_start=body_start,
body_end=close_idx,
body_indent=body_indent,
)
)
line_idx = close_idx + 1
return blocks
def dedent(lines: list[str], n: int) -> list[str]:
pad = " " * n
return [
(
""
if line.strip() == ""
else (line[n:] if line.startswith(pad) else line.lstrip(" "))
)
for line in lines
]
def reindent(lines: list[str], n: int) -> list[str]:
pad = " " * n
return [pad + line if line else "" for line in lines]
_SHFMT_ERR_RE = re.compile(r"\.sh:\d+:\d+:\s")
_GHA_EXPR_RE = re.compile(r"\$\{\{.*?\}\}", re.DOTALL)
_GHA_PLACEHOLDER_RE = re.compile(r"__GHA_EXPR_(\d+)__")
def _encode_gha_exprs(text: str) -> tuple[str, list[str]]:
"""Replace `${{ ... }}` expressions with bash-safe placeholder identifiers."""
exprs: list[str] = []
def repl(match: re.Match[str]) -> str:
exprs.append(match.group(0))
return f"__GHA_EXPR_{len(exprs) - 1}__"
return _GHA_EXPR_RE.sub(repl, text), exprs
def _decode_gha_exprs(text: str, exprs: list[str]) -> str:
"""Restore `${{ ... }}` expressions from placeholder identifiers."""
return _GHA_PLACEHOLDER_RE.sub(lambda m: exprs[int(m.group(1))], text)
def shfmt_via_hook(tmp_path: Path) -> tuple[bool, str]:
# `${{ ... }}` is not valid shell, so swap it for a placeholder identifier
# that shfmt can parse, then restore it after formatting.
encoded, exprs = _encode_gha_exprs(tmp_path.read_text())
if exprs:
tmp_path.write_text(encoded)
res = subprocess.run(
[_HOOK_RUNNER, "run", "shfmt", "--files", str(tmp_path)],
cwd=REPO,
capture_output=True,
text=True,
)
output = res.stdout + res.stderr
# shfmt emits parse errors as "<path>:<line>:<col>: <message>".
parse_err = bool(_SHFMT_ERR_RE.search(output))
# A non-zero exit that is neither a parse error nor pre-commit's "I had
# to modify files" signal means the hook itself failed to run (missing
# binary, install failure, bad config, ...). Surface that loudly rather
# than silently treating it as a no-op.
if (
res.returncode != 0
and not parse_err
and "files were modified by this hook" not in output
):
sys.exit(
f"error: `{_HOOK_RUNNER} run shfmt` failed with exit {res.returncode}:\n{output}"
)
if exprs and not parse_err:
tmp_path.write_text(_decode_gha_exprs(tmp_path.read_text(), exprs))
return not parse_err, output
def _skip(path: Path, where: int, kind: str, output: str) -> None:
print(
f" shfmt could not parse {kind} at {path}:{where + 1} — skipped",
file=sys.stderr,
)
print(f" {output.strip()}", file=sys.stderr)
def process_yaml_file(path: Path, tmp_path: Path) -> int:
text = path.read_text()
had_nl = text.endswith("\n")
lines = text.split("\n")
if had_nl:
lines = lines[:-1]
items = find_run_blocks(lines)
if not items:
return 0
changed = 0
# Process in reverse so earlier indices remain valid as we splice.
for item in reversed(items):
if isinstance(item, BlockRun):
body = lines[item.body_start : item.body_end]
tmp_path.write_text("\n".join(dedent(body, item.body_indent)) + "\n")
ok, output = shfmt_via_hook(tmp_path)
if not ok:
_skip(path, item.body_start, "block", output)
continue
formatted = tmp_path.read_text().rstrip("\n")
new_body = reindent(formatted.split("\n"), item.body_indent)
if new_body != body:
lines[item.body_start : item.body_end] = new_body
changed += 1
else:
tmp_path.write_text(item.value + "\n")
ok, output = shfmt_via_hook(tmp_path)
if not ok:
_skip(path, item.line_idx, "inline run", output)
continue
formatted = tmp_path.read_text().rstrip("\n")
if formatted == item.value:
continue
formatted_lines = formatted.split("\n")
if len(formatted_lines) == 1:
lines[item.line_idx] = f"{item.prefix}run: {formatted}"
else:
body_indent = len(item.prefix) + 2
lines[item.line_idx : item.line_idx + 1] = [
f"{item.prefix}run: |",
*reindent(formatted_lines, body_indent),
]
changed += 1
new_text = "\n".join(lines) + ("\n" if had_nl else "")
if new_text != text:
path.write_text(new_text)
return changed
def process_md_file(path: Path, tmp_path: Path) -> int:
text = path.read_text()
had_nl = text.endswith("\n")
lines = text.split("\n")
if had_nl:
lines = lines[:-1]
blocks = find_md_bash_blocks(lines)
if not blocks:
return 0
changed = 0
for block in reversed(blocks):
body = lines[block.body_start : block.body_end]
tmp_path.write_text("\n".join(dedent(body, block.body_indent)) + "\n")
ok, output = shfmt_via_hook(tmp_path)
if not ok:
_skip(path, block.open_line_idx, "```bash block", output)
continue
formatted = tmp_path.read_text().rstrip("\n")
formatted_lines = formatted.split("\n") if formatted else []
new_body = reindent(formatted_lines, block.body_indent)
if new_body != body:
lines[block.body_start : block.body_end] = new_body
changed += 1
new_text = "\n".join(lines) + ("\n" if had_nl else "")
if new_text != text:
path.write_text(new_text)
return changed
def process_file(path: Path, tmp_path: Path) -> int:
if path.suffix in (".yml", ".yaml"):
return process_yaml_file(path, tmp_path)
if path.suffix == ".md":
return process_md_file(path, tmp_path)
return 0
def gather_files(argv: list[str]) -> list[Path]:
"""Return YAML workflow/action files and markdown files that we should
process — either the paths in `argv` or, when `argv` is empty, every
such file in the repo (skipping `external/`)."""
if argv:
candidates: list[Path] = [
(REPO / a).resolve() if not Path(a).is_absolute() else Path(a) for a in argv
]
else:
gh = REPO / ".github"
candidates = [
*gh.rglob("*.yml"),
*gh.rglob("*.yaml"),
*(
p
for p in REPO.rglob("*.md")
if "external" not in p.relative_to(REPO).parts
),
]
return sorted(
p
for p in candidates
if p.exists()
and (
(p.suffix in (".yml", ".yaml") and ".github" in p.parts)
or p.suffix == ".md"
)
)
def main(argv: list[str]) -> int:
files = gather_files(argv)
if not files:
return 0
with tempfile.TemporaryDirectory(prefix="format-inline-bash-") as tmpdir:
tmp_path = Path(tmpdir) / "shfmt.sh"
total = 0
for f in files:
n = process_file(f, tmp_path)
if n:
print(f"{f.relative_to(REPO)}: reformatted {n} block(s)")
total += n
return 1 if total else 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))

0
.github/scripts/levelization/generate.py vendored Normal file → Executable file
View File

View File

@@ -12,7 +12,6 @@ libxrpl.ledger > xrpl.json
libxrpl.ledger > xrpl.ledger
libxrpl.ledger > xrpl.nodestore
libxrpl.ledger > xrpl.protocol
libxrpl.ledger > xrpl.server
libxrpl.ledger > xrpl.shamap
libxrpl.net > xrpl.basics
libxrpl.net > xrpl.net
@@ -206,7 +205,6 @@ xrpl.core > xrpl.protocol
xrpl.json > xrpl.basics
xrpl.ledger > xrpl.basics
xrpl.ledger > xrpl.protocol
xrpl.ledger > xrpl.server
xrpl.ledger > xrpl.shamap
xrpl.net > xrpl.basics
xrpl.nodestore > xrpl.basics

View File

@@ -6,7 +6,7 @@ set -e
# On MacOS, ensure that GNU sed is installed and available as `gsed`.
SED_COMMAND=sed
if [[ "${OSTYPE}" == 'darwin'* ]]; then
if ! command -v gsed &> /dev/null; then
if ! command -v gsed &>/dev/null; then
echo "Error: gsed is not installed. Please install it using 'brew install gnu-sed'."
exit 1
fi

View File

@@ -8,12 +8,12 @@ set -e
SED_COMMAND=sed
HEAD_COMMAND=head
if [[ "${OSTYPE}" == 'darwin'* ]]; then
if ! command -v gsed &> /dev/null; then
if ! command -v gsed &>/dev/null; then
echo "Error: gsed is not installed. Please install it using 'brew install gnu-sed'."
exit 1
fi
SED_COMMAND=gsed
if ! command -v ghead &> /dev/null; then
if ! command -v ghead &>/dev/null; then
echo "Error: ghead is not installed. Please install it using 'brew install coreutils'."
exit 1
fi
@@ -74,10 +74,10 @@ if grep -q '"xrpld"' cmake/XrplCore.cmake; then
# The script has been rerun, so just restore the name of the binary.
${SED_COMMAND} -i 's/"xrpld"/"rippled"/' cmake/XrplCore.cmake
elif ! grep -q '"rippled"' cmake/XrplCore.cmake; then
${HEAD_COMMAND} -n -1 cmake/XrplCore.cmake > cmake.tmp
echo ' # For the time being, we will keep the name of the binary as it was.' >> cmake.tmp
echo ' set_target_properties(xrpld PROPERTIES OUTPUT_NAME "rippled")' >> cmake.tmp
tail -1 cmake/XrplCore.cmake >> cmake.tmp
${HEAD_COMMAND} -n -1 cmake/XrplCore.cmake >cmake.tmp
echo ' # For the time being, we will keep the name of the binary as it was.' >>cmake.tmp
echo ' set_target_properties(xrpld PROPERTIES OUTPUT_NAME "rippled")' >>cmake.tmp
tail -1 cmake/XrplCore.cmake >>cmake.tmp
mv cmake.tmp cmake/XrplCore.cmake
fi

View File

@@ -6,7 +6,7 @@ set -e
# On MacOS, ensure that GNU sed is installed and available as `gsed`.
SED_COMMAND=sed
if [[ "${OSTYPE}" == 'darwin'* ]]; then
if ! command -v gsed &> /dev/null; then
if ! command -v gsed &>/dev/null; then
echo "Error: gsed is not installed. Please install it using 'brew install gnu-sed'."
exit 1
fi

View File

@@ -6,7 +6,7 @@ set -e
# On MacOS, ensure that GNU sed is installed and available as `gsed`.
SED_COMMAND=sed
if [[ "${OSTYPE}" == 'darwin'* ]]; then
if ! command -v gsed &> /dev/null; then
if ! command -v gsed &>/dev/null; then
echo "Error: gsed is not installed. Please install it using 'brew install gnu-sed'."
exit 1
fi
@@ -62,37 +62,37 @@ done
# restoring the verbiage that is already present in LICENSE.md. Ensure that if
# the script is run multiple times, duplicate notices are not added.
if ! grep -q 'Raw Material Software' include/xrpl/beast/core/CurrentThreadName.h; then
echo -e "// Portions of this file are from JUCE (http://www.juce.com).\n// Copyright (c) 2013 - Raw Material Software Ltd.\n// Please visit http://www.juce.com\n\n$(cat include/xrpl/beast/core/CurrentThreadName.h)" > include/xrpl/beast/core/CurrentThreadName.h
echo -e "// Portions of this file are from JUCE (http://www.juce.com).\n// Copyright (c) 2013 - Raw Material Software Ltd.\n// Please visit http://www.juce.com\n\n$(cat include/xrpl/beast/core/CurrentThreadName.h)" >include/xrpl/beast/core/CurrentThreadName.h
fi
if ! grep -q 'Dev Null' src/test/app/NetworkID_test.cpp; then
echo -e "// Copyright (c) 2020 Dev Null Productions\n\n$(cat src/test/app/NetworkID_test.cpp)" > src/test/app/NetworkID_test.cpp
echo -e "// Copyright (c) 2020 Dev Null Productions\n\n$(cat src/test/app/NetworkID_test.cpp)" >src/test/app/NetworkID_test.cpp
fi
if ! grep -q 'Dev Null' src/test/app/tx/apply_test.cpp; then
echo -e "// Copyright (c) 2020 Dev Null Productions\n\n$(cat src/test/app/tx/apply_test.cpp)" > src/test/app/tx/apply_test.cpp
echo -e "// Copyright (c) 2020 Dev Null Productions\n\n$(cat src/test/app/tx/apply_test.cpp)" >src/test/app/tx/apply_test.cpp
fi
if ! grep -q 'Dev Null' src/test/rpc/ManifestRPC_test.cpp; then
echo -e "// Copyright (c) 2020 Dev Null Productions\n\n$(cat src/test/rpc/ManifestRPC_test.cpp)" > src/test/rpc/ManifestRPC_test.cpp
echo -e "// Copyright (c) 2020 Dev Null Productions\n\n$(cat src/test/rpc/ManifestRPC_test.cpp)" >src/test/rpc/ManifestRPC_test.cpp
fi
if ! grep -q 'Dev Null' src/test/rpc/ValidatorInfo_test.cpp; then
echo -e "// Copyright (c) 2020 Dev Null Productions\n\n$(cat src/test/rpc/ValidatorInfo_test.cpp)" > src/test/rpc/ValidatorInfo_test.cpp
echo -e "// Copyright (c) 2020 Dev Null Productions\n\n$(cat src/test/rpc/ValidatorInfo_test.cpp)" >src/test/rpc/ValidatorInfo_test.cpp
fi
if ! grep -q 'Dev Null' src/xrpld/rpc/handlers/server_info/Manifest.cpp; then
echo -e "// Copyright (c) 2019 Dev Null Productions\n\n$(cat src/xrpld/rpc/handlers/server_info/Manifest.cpp)" > src/xrpld/rpc/handlers/server_info/Manifest.cpp
echo -e "// Copyright (c) 2019 Dev Null Productions\n\n$(cat src/xrpld/rpc/handlers/server_info/Manifest.cpp)" >src/xrpld/rpc/handlers/server_info/Manifest.cpp
fi
if ! grep -q 'Dev Null' src/xrpld/rpc/handlers/admin/status/ValidatorInfo.cpp; then
echo -e "// Copyright (c) 2019 Dev Null Productions\n\n$(cat src/xrpld/rpc/handlers/admin/status/ValidatorInfo.cpp)" > src/xrpld/rpc/handlers/admin/status/ValidatorInfo.cpp
echo -e "// Copyright (c) 2019 Dev Null Productions\n\n$(cat src/xrpld/rpc/handlers/admin/status/ValidatorInfo.cpp)" >src/xrpld/rpc/handlers/admin/status/ValidatorInfo.cpp
fi
if ! grep -q 'Bougalis' include/xrpl/basics/SlabAllocator.h; then
echo -e "// Copyright (c) 2022, Nikolaos D. Bougalis <nikb@bougalis.net>\n\n$(cat include/xrpl/basics/SlabAllocator.h)" > include/xrpl/basics/SlabAllocator.h # cspell: ignore Nikolaos Bougalis nikb
echo -e "// Copyright (c) 2022, Nikolaos D. Bougalis <nikb@bougalis.net>\n\n$(cat include/xrpl/basics/SlabAllocator.h)" >include/xrpl/basics/SlabAllocator.h # cspell: ignore Nikolaos Bougalis nikb
fi
if ! grep -q 'Bougalis' include/xrpl/basics/spinlock.h; then
echo -e "// Copyright (c) 2022, Nikolaos D. Bougalis <nikb@bougalis.net>\n\n$(cat include/xrpl/basics/spinlock.h)" > include/xrpl/basics/spinlock.h # cspell: ignore Nikolaos Bougalis nikb
echo -e "// Copyright (c) 2022, Nikolaos D. Bougalis <nikb@bougalis.net>\n\n$(cat include/xrpl/basics/spinlock.h)" >include/xrpl/basics/spinlock.h # cspell: ignore Nikolaos Bougalis nikb
fi
if ! grep -q 'Bougalis' include/xrpl/basics/tagged_integer.h; then
echo -e "// Copyright (c) 2014, Nikolaos D. Bougalis <nikb@bougalis.net>\n\n$(cat include/xrpl/basics/tagged_integer.h)" > include/xrpl/basics/tagged_integer.h # cspell: ignore Nikolaos Bougalis nikb
echo -e "// Copyright (c) 2014, Nikolaos D. Bougalis <nikb@bougalis.net>\n\n$(cat include/xrpl/basics/tagged_integer.h)" >include/xrpl/basics/tagged_integer.h # cspell: ignore Nikolaos Bougalis nikb
fi
if ! grep -q 'Ritchford' include/xrpl/beast/utility/Zero.h; then
echo -e "// Copyright (c) 2014, Tom Ritchford <tom@swirly.com>\n\n$(cat include/xrpl/beast/utility/Zero.h)" > include/xrpl/beast/utility/Zero.h # cspell: ignore Ritchford
echo -e "// Copyright (c) 2014, Tom Ritchford <tom@swirly.com>\n\n$(cat include/xrpl/beast/utility/Zero.h)" >include/xrpl/beast/utility/Zero.h # cspell: ignore Ritchford
fi
# Restore newlines and tabs in string literals in the affected file.

View File

@@ -6,7 +6,7 @@ set -e
# On MacOS, ensure that GNU sed is installed and available as `gsed`.
SED_COMMAND=sed
if [[ "${OSTYPE}" == 'darwin'* ]]; then
if ! command -v gsed &> /dev/null; then
if ! command -v gsed &>/dev/null; then
echo "Error: gsed is not installed. Please install it using 'brew install gnu-sed'."
exit 1
fi

View File

@@ -6,7 +6,7 @@ set -e
# On MacOS, ensure that GNU sed is installed and available as `gsed`.
SED_COMMAND=sed
if [[ "${OSTYPE}" == 'darwin'* ]]; then
if ! command -v gsed &> /dev/null; then
if ! command -v gsed &>/dev/null; then
echo "Error: gsed is not installed. Please install it using 'brew install gnu-sed'."
exit 1
fi

View File

@@ -6,7 +6,7 @@ set -e
# On MacOS, ensure that GNU sed is installed and available as `gsed`.
SED_COMMAND=sed
if [[ "${OSTYPE}" == 'darwin'* ]]; then
if ! command -v gsed &> /dev/null; then
if ! command -v gsed &>/dev/null; then
echo "Error: gsed is not installed. Please install it using 'brew install gnu-sed'."
exit 1
fi

View File

@@ -6,14 +6,16 @@ on:
- develop
paths:
- ".github/workflows/build-nix-image.yml"
- "docker/nix.Dockerfile"
- ".github/workflows/reusable-build-docker-image.yml"
- "docker/**"
- "flake.nix"
- "flake.lock"
- "nix/**"
pull_request:
paths:
- ".github/workflows/build-nix-image.yml"
- "docker/nix.Dockerfile"
- ".github/workflows/reusable-build-docker-image.yml"
- "docker/**"
- "flake.nix"
- "flake.lock"
- "nix/**"
@@ -27,75 +29,81 @@ defaults:
run:
shell: bash
env:
UBUNTU_VERSION: "20.04"
RHEL_VERSION: "9"
DEBIAN_VERSION: "bookworm"
jobs:
build:
name: Build and push Nix image (${{ matrix.distro }})
name: Build ${{ matrix.distro.name }} (${{ matrix.target.platform }})
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
# The base images are the oldest supported version of each distro
# that we want to build images for.
distro:
- name: nixos
base_image: nixos/nix:latest
- name: ubuntu
base_image: ubuntu:20.04
- name: rhel
base_image: registry.access.redhat.com/ubi9/ubi:latest
- name: debian
base_image: debian:bookworm
target:
- platform: linux/amd64
runner: ubuntu-latest
- platform: linux/arm64
runner: ubuntu-24.04-arm
uses: ./.github/workflows/reusable-build-docker-image.yml
with:
image_name: ghcr.io/xrplf/xrpld/nix-${{ matrix.distro.name }}
dockerfile: docker/nix.Dockerfile
base_image: ${{ matrix.distro.base_image }}
platform: ${{ matrix.target.platform }}
runner: ${{ matrix.target.runner }}
push: ${{ github.repository == 'XRPLF/rippled' && github.event_name == 'push' }}
merge:
name: Merge ${{ matrix.distro }} manifest
needs: build
if: ${{ github.repository == 'XRPLF/rippled' && github.event_name == 'push' }}
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
include:
- distro: nixos
- distro: ubuntu
- distro: rhel
- distro: debian
distro: [nixos, ubuntu, rhel, debian]
env:
IMAGE_NAME: ghcr.io/xrplf/xrpld/nix-${{ matrix.distro }}
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Determine base image
id: vars
run: |
case "${{ matrix.distro }}" in
nixos)
echo "base_image=nixos/nix:latest" >> $GITHUB_OUTPUT
;;
ubuntu)
echo "base_image=ubuntu:${UBUNTU_VERSION}" >> $GITHUB_OUTPUT
;;
rhel)
echo "base_image=registry.access.redhat.com/ubi${RHEL_VERSION}/ubi:latest" >> $GITHUB_OUTPUT
;;
debian)
echo "base_image=debian:${DEBIAN_VERSION}" >> $GITHUB_OUTPUT
;;
esac
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
- name: Docker metadata
id: meta
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
with:
images: ${{ env.IMAGE_NAME }}
tags: |
type=sha,prefix=sha-,format=short
type=raw,value=latest
- name: Login to GitHub Container Registry
if: github.event_name == 'push'
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Docker metadata
id: meta
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
with:
images: ghcr.io/xrplf/ci/nix-${{ matrix.distro }}
tags: |
type=sha,prefix=sha-,format=short
type=raw,value=latest
- name: Create multi-arch manifests
run: |
for tag in $(jq -cr '.tags[]' <<<"$DOCKER_METADATA_OUTPUT_JSON"); do
docker buildx imagetools create -t "$tag" "${tag}-amd64" "${tag}-arm64"
done
- name: Build and push
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
file: docker/nix.Dockerfile
platforms: linux/amd64
push: ${{ github.event_name == 'push' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-args: BASE_IMAGE=${{ steps.vars.outputs.base_image }}
- name: Inspect image
run: |
docker buildx imagetools inspect "${IMAGE_NAME}:${{ steps.meta.outputs.version }}"

View File

@@ -5,8 +5,17 @@ on:
types:
- checks_requested
pull_request:
types: [opened, edited, reopened, synchronize, ready_for_review]
branches: [develop]
types:
- opened
- edited
- reopened
- synchronize
- ready_for_review
branches:
- develop
- "release-*"
- "release/*"
- "staging/*"
jobs:
check_description:
@@ -20,11 +29,11 @@ jobs:
env:
PR_BODY: ${{ github.event.pull_request.body }}
if: ${{ github.event_name == 'pull_request' }}
run: printenv PR_BODY > pr_body.md
run: printenv PR_BODY >pr_body.md
- name: Check PR description differs from template
if: ${{ github.event_name == 'pull_request' }}
run: >
python .github/scripts/check-pr-description.py
--template-file .github/pull_request_template.md
--pr-body-file pr_body.md
run: |
python .github/scripts/check-pr-description.py \
--template-file .github/pull_request_template.md \
--pr-body-file pr_body.md

View File

@@ -5,10 +5,19 @@ on:
types:
- checks_requested
pull_request:
types: [opened, edited, reopened, synchronize, ready_for_review]
branches: [develop]
types:
- opened
- edited
- reopened
- synchronize
- ready_for_review
branches:
- develop
- "release-*"
- "release/*"
- "staging/*"
jobs:
check_title:
if: ${{ github.event.pull_request.draft != true }}
uses: XRPLF/actions/.github/workflows/check-pr-title.yml@291206777251b4d493641b5afbdf7c23009d2988
uses: XRPLF/actions/.github/workflows/check-pr-title.yml@cba1f0891650baf1a9c88624dc2d72573be2eb81

View File

@@ -98,7 +98,7 @@ jobs:
READY: ${{ contains(github.event.pull_request.labels.*.name, 'Ready to merge') }}
MERGE: ${{ github.event_name == 'merge_group' }}
run: |
echo "go=${{ (env.DRAFT != 'true' && env.READY == 'true') || env.FILES == 'true' || env.MERGE == 'true' }}" >> "${GITHUB_OUTPUT}"
echo "go=${{ (env.DRAFT != 'true' && env.READY == 'true') || env.FILES == 'true' || env.MERGE == 'true' }}" >>"${GITHUB_OUTPUT}"
cat "${GITHUB_OUTPUT}"
outputs:
go: ${{ steps.go.outputs.go == 'true' }}
@@ -168,9 +168,9 @@ jobs:
PR_URL: ${{ github.event.pull_request.html_url }}
run: |
gh api --method POST -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" \
/repos/xrplf/clio/dispatches -f "event_type=check_libxrpl" \
-F "client_payload[ref]=${{ needs.upload-recipe.outputs.recipe_ref }}" \
-F "client_payload[pr_url]=${PR_URL}"
/repos/xrplf/clio/dispatches -f "event_type=check_libxrpl" \
-F "client_payload[ref]=${{ needs.upload-recipe.outputs.recipe_ref }}" \
-F "client_payload[pr_url]=${PR_URL}"
passed:
if: failure() || cancelled()

View File

@@ -14,7 +14,7 @@ on:
jobs:
# Call the workflow in the XRPLF/actions repo that runs the pre-commit hooks.
run-hooks:
uses: XRPLF/actions/.github/workflows/pre-commit.yml@5e942d61bf32f7557a7c159cfac4712a687b3e3a
uses: XRPLF/actions/.github/workflows/pre-commit.yml@cba1f0891650baf1a9c88624dc2d72573be2eb81
with:
runs_on: ubuntu-latest
container: '{ "image": "ghcr.io/xrplf/ci/tools-rippled-pre-commit:sha-41ec7c1" }'

View File

@@ -0,0 +1,89 @@
# Build a single-platform Docker image. On push, the image is pushed to
# GHCR with arch-suffixed tags (e.g. `:latest-amd64`, `:sha-abc-amd64`)
# so the calling workflow can stitch per-arch builds into a multi-arch
# manifest without needing to pass digests around.
name: Reusable build Docker image (single platform)
on:
workflow_call:
inputs:
image_name:
description: "Full image name without tag (e.g. 'ghcr.io/xrplf/xrpld/nix-ubuntu')"
required: true
type: string
dockerfile:
description: "Path to the Dockerfile, relative to the repository root"
required: true
type: string
base_image:
description: "Value passed to the Dockerfile as the BASE_IMAGE build arg"
required: true
type: string
platform:
description: "Docker platform string, e.g. linux/amd64"
required: true
type: string
runner:
description: "GitHub Actions runner label to build on"
required: true
type: string
push:
description: "Whether to push the image to GHCR"
required: true
type: boolean
defaults:
run:
shell: bash
jobs:
build:
name: Build (${{ inputs.platform }})
runs-on: ${{ inputs.runner }}
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Determine arch
id: vars
env:
PLATFORM: ${{ inputs.platform }}
run: |
echo "arch=${PLATFORM##*/}" >>$GITHUB_OUTPUT
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
- name: Login to GitHub Container Registry
if: inputs.push
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Docker metadata
id: meta
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
with:
images: ${{ inputs.image_name }}
tags: |
type=sha,prefix=sha-,format=short
type=raw,value=latest
flavor: |
suffix=-${{ steps.vars.outputs.arch }},onlatest=true
- name: Build and push
uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
with:
context: .
file: ${{ inputs.dockerfile }}
platforms: ${{ inputs.platform }}
push: ${{ inputs.push }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-args: BASE_IMAGE=${{ inputs.base_image }}

View File

@@ -113,7 +113,7 @@ jobs:
- name: Set ccache log file
if: ${{ inputs.ccache_enabled && runner.debug == '1' }}
run: echo "CCACHE_LOGFILE=${{ runner.temp }}/ccache.log" >> "${GITHUB_ENV}"
run: echo "CCACHE_LOGFILE=${{ runner.temp }}/ccache.log" >>"${GITHUB_ENV}"
- name: Print build environment
uses: XRPLF/actions/print-build-env@59dec886e4afb05a1724443af08baccbc045b574
@@ -146,11 +146,11 @@ jobs:
CMAKE_ARGS: ${{ inputs.cmake_args }}
run: |
cmake \
-G '${{ runner.os == 'Windows' && 'Visual Studio 17 2022' || 'Ninja' }}' \
-DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake \
-DCMAKE_BUILD_TYPE="${BUILD_TYPE}" \
${CMAKE_ARGS} \
..
-G '${{ runner.os == 'Windows' && 'Visual Studio 17 2022' || 'Ninja' }}' \
-DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake \
-DCMAKE_BUILD_TYPE="${BUILD_TYPE}" \
${CMAKE_ARGS} \
..
- name: Check protocol autogen files are up-to-date
working-directory: ${{ env.BUILD_DIR }}
@@ -172,32 +172,32 @@ jobs:
cmake --build . --target code_gen
DIFF=$(git -C .. status --porcelain -- include/xrpl/protocol_autogen src/tests/libxrpl/protocol_autogen)
if [ -n "${DIFF}" ]; then
echo "::error::Generated protocol files are out of date"
git -C .. diff -- include/xrpl/protocol_autogen src/tests/libxrpl/protocol_autogen
echo "${MESSAGE}"
exit 1
echo "::error::Generated protocol files are out of date"
git -C .. diff -- include/xrpl/protocol_autogen src/tests/libxrpl/protocol_autogen
echo "${MESSAGE}"
exit 1
fi
- name: Build the binary
working-directory: ${{ env.BUILD_DIR }}
env:
BUILD_NPROC: ${{ runner.os == 'Linux' && '16' || steps.nproc.outputs.nproc }}
BUILD_NPROC: ${{ steps.nproc.outputs.nproc }}
BUILD_TYPE: ${{ inputs.build_type }}
CMAKE_TARGET: ${{ inputs.cmake_target }}
run: |
cmake \
--build . \
--config "${BUILD_TYPE}" \
--parallel "${BUILD_NPROC}" \
--target "${CMAKE_TARGET}"
--build . \
--config "${BUILD_TYPE}" \
--parallel "${BUILD_NPROC}" \
--target "${CMAKE_TARGET}"
- name: Show ccache statistics
if: ${{ inputs.ccache_enabled }}
run: |
ccache --show-stats -vv
if [ '${{ runner.debug }}' = '1' ]; then
cat "${CCACHE_LOGFILE}"
curl ${CCACHE_REMOTE_STORAGE%|*}/status || true
cat "${CCACHE_LOGFILE}"
curl ${CCACHE_REMOTE_STORAGE%|*}/status || true
fi
- name: Upload the binary (Linux)
@@ -214,7 +214,7 @@ jobs:
working-directory: ${{ env.BUILD_DIR }}
run: |
set -o pipefail
./xrpld --definitions | python3 -m json.tool > server_definitions.json
./xrpld --definitions | python3 -m json.tool >server_definitions.json
- name: Upload server definitions
if: ${{ github.event.repository.visibility == 'public' && inputs.config_name == 'debian-bookworm-gcc-13-amd64-release' }}
@@ -231,10 +231,10 @@ jobs:
run: |
ldd ./xrpld
if [ "$(ldd ./xrpld | grep -E '(libstdc\+\+|libgcc)' | wc -l)" -eq 0 ]; then
echo 'The binary is statically linked.'
echo 'The binary is statically linked.'
else
echo 'The binary is dynamically linked.'
exit 1
echo 'The binary is dynamically linked.'
exit 1
fi
- name: Verify presence of instrumentation (Linux)
@@ -250,12 +250,12 @@ jobs:
run: |
ASAN_OPTS="include=${GITHUB_WORKSPACE}/sanitizers/suppressions/runtime-asan-options.txt:suppressions=${GITHUB_WORKSPACE}/sanitizers/suppressions/asan.supp"
if [[ "${CONFIG_NAME}" == *gcc* ]]; then
ASAN_OPTS="${ASAN_OPTS}:alloc_dealloc_mismatch=0"
ASAN_OPTS="${ASAN_OPTS}:alloc_dealloc_mismatch=0"
fi
echo "ASAN_OPTIONS=${ASAN_OPTS}" >> ${GITHUB_ENV}
echo "TSAN_OPTIONS=include=${GITHUB_WORKSPACE}/sanitizers/suppressions/runtime-tsan-options.txt:suppressions=${GITHUB_WORKSPACE}/sanitizers/suppressions/tsan.supp" >> ${GITHUB_ENV}
echo "UBSAN_OPTIONS=include=${GITHUB_WORKSPACE}/sanitizers/suppressions/runtime-ubsan-options.txt:suppressions=${GITHUB_WORKSPACE}/sanitizers/suppressions/ubsan.supp" >> ${GITHUB_ENV}
echo "LSAN_OPTIONS=include=${GITHUB_WORKSPACE}/sanitizers/suppressions/runtime-lsan-options.txt:suppressions=${GITHUB_WORKSPACE}/sanitizers/suppressions/lsan.supp" >> ${GITHUB_ENV}
echo "ASAN_OPTIONS=${ASAN_OPTS}" >>${GITHUB_ENV}
echo "TSAN_OPTIONS=include=${GITHUB_WORKSPACE}/sanitizers/suppressions/runtime-tsan-options.txt:suppressions=${GITHUB_WORKSPACE}/sanitizers/suppressions/tsan.supp" >>${GITHUB_ENV}
echo "UBSAN_OPTIONS=include=${GITHUB_WORKSPACE}/sanitizers/suppressions/runtime-ubsan-options.txt:suppressions=${GITHUB_WORKSPACE}/sanitizers/suppressions/ubsan.supp" >>${GITHUB_ENV}
echo "LSAN_OPTIONS=include=${GITHUB_WORKSPACE}/sanitizers/suppressions/runtime-lsan-options.txt:suppressions=${GITHUB_WORKSPACE}/sanitizers/suppressions/lsan.supp" >>${GITHUB_ENV}
- name: Run the separate tests
if: ${{ !inputs.build_only }}
@@ -266,9 +266,9 @@ jobs:
PARALLELISM: ${{ runner.os == 'Windows' && '1' || steps.nproc.outputs.nproc }}
run: |
ctest \
--output-on-failure \
-C "${BUILD_TYPE}" \
-j "${PARALLELISM}"
--output-on-failure \
-C "${BUILD_TYPE}" \
-j "${PARALLELISM}"
- name: Run the embedded tests
if: ${{ !inputs.build_only }}
@@ -278,7 +278,7 @@ jobs:
run: |
set -o pipefail
# Coverage builds are slower due to instrumentation; use fewer parallel jobs to avoid flakiness
[ "$COVERAGE_ENABLED" = "true" ] && BUILD_NPROC=$(( BUILD_NPROC - 2 ))
[ "$COVERAGE_ENABLED" = "true" ] && BUILD_NPROC=$((BUILD_NPROC - 2))
./xrpld --unittest --unittest-jobs "${BUILD_NPROC}" 2>&1 | tee unittest.log
- name: Show test failure summary
@@ -287,19 +287,19 @@ jobs:
WORKING_DIR: ${{ runner.os == 'Windows' && format('{0}\{1}', env.BUILD_DIR, inputs.build_type) || env.BUILD_DIR }}
run: |
if [ ! -d "${WORKING_DIR}" ]; then
echo "Working directory '${WORKING_DIR}' does not exist."
exit 0
echo "Working directory '${WORKING_DIR}' does not exist."
exit 0
fi
cd "${WORKING_DIR}"
if [ ! -f unittest.log ]; then
echo "unittest.log not found; embedded tests may not have run."
exit 0
echo "unittest.log not found; embedded tests may not have run."
exit 0
fi
if ! grep -E "failed" unittest.log; then
echo "Log present but no failure lines found in unittest.log."
echo "Log present but no failure lines found in unittest.log."
fi
- name: Debug failure (Linux)
if: ${{ failure() && runner.os == 'Linux' && !inputs.build_only }}
@@ -317,14 +317,14 @@ jobs:
BUILD_TYPE: ${{ inputs.build_type }}
run: |
cmake \
--build . \
--config "${BUILD_TYPE}" \
--parallel "${BUILD_NPROC}" \
--target coverage
--build . \
--config "${BUILD_TYPE}" \
--parallel "${BUILD_NPROC}" \
--target coverage
- name: Upload coverage report
if: ${{ github.repository == 'XRPLF/rippled' && !inputs.build_only && env.COVERAGE_ENABLED == 'true' }}
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
disable_search: true
disable_telem: true

View File

@@ -38,9 +38,9 @@ jobs:
run: |
DIFF=$(git status --porcelain)
if [ -n "${DIFF}" ]; then
# Print the differences to give the contributor a hint about what to
# expect when running levelization on their own machine.
git diff
echo "${MESSAGE}"
exit 1
# Print the differences to give the contributor a hint about what to
# expect when running levelization on their own machine.
git diff
echo "${MESSAGE}"
exit 1
fi

View File

@@ -48,9 +48,9 @@ jobs:
run: |
DIFF=$(git status --porcelain)
if [ -n "${DIFF}" ]; then
# Print the differences to give the contributor a hint about what to
# expect when running the renaming scripts on their own machine.
git diff
echo "${MESSAGE}"
exit 1
# Print the differences to give the contributor a hint about what to
# expect when running the renaming scripts on their own machine.
git diff
echo "${MESSAGE}"
exit 1
fi

View File

@@ -70,13 +70,13 @@ jobs:
working-directory: ${{ env.BUILD_DIR }}
run: |
cmake \
-G 'Ninja' \
-DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake \
-DCMAKE_BUILD_TYPE="${BUILD_TYPE}" \
-Dtests=ON \
-Dwerr=ON \
-Dxrpld=ON \
..
-G 'Ninja' \
-DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake \
-DCMAKE_BUILD_TYPE="${BUILD_TYPE}" \
-Dtests=ON \
-Dwerr=ON \
-Dxrpld=ON \
..
# clang-tidy needs headers generated from proto files
- name: Build libxrpl.libpb
@@ -133,7 +133,7 @@ jobs:
- name: Write issue header
if: ${{ steps.run_clang_tidy.outcome != 'success' }}
run: |
cat > "${ISSUE_FILE}" <<EOF
cat >"${ISSUE_FILE}" <<EOF
## Clang-tidy Check Failed
### Clang-tidy Output:
@@ -144,30 +144,30 @@ jobs:
if: ${{ steps.run_clang_tidy.outcome != 'success' }}
run: |
if [ -f "${OUTPUT_FILE}" ]; then
# Extract lines containing 'error:', 'warning:', or 'note:'
grep -E '(error:|warning:|note:)' "${OUTPUT_FILE}" > filtered-output.txt || true
# Extract lines containing 'error:', 'warning:', or 'note:'
grep -E '(error:|warning:|note:)' "${OUTPUT_FILE}" >filtered-output.txt || true
# If filtered output is empty, use original (might be a different error format)
if [ ! -s filtered-output.txt ]; then
cp "${OUTPUT_FILE}" filtered-output.txt
fi
# If filtered output is empty, use original (might be a different error format)
if [ ! -s filtered-output.txt ]; then
cp "${OUTPUT_FILE}" filtered-output.txt
fi
# Truncate if too large
head -c 60000 filtered-output.txt >> "${ISSUE_FILE}"
if [ "$(wc -c < filtered-output.txt)" -gt 60000 ]; then
echo "" >> "${ISSUE_FILE}"
echo "... (output truncated, see artifacts for full output)" >> "${ISSUE_FILE}"
fi
# Truncate if too large
head -c 60000 filtered-output.txt >>"${ISSUE_FILE}"
if [ "$(wc -c <filtered-output.txt)" -gt 60000 ]; then
echo "" >>"${ISSUE_FILE}"
echo "... (output truncated, see artifacts for full output)" >>"${ISSUE_FILE}"
fi
rm filtered-output.txt
rm filtered-output.txt
else
echo "No output file found" >> "${ISSUE_FILE}"
echo "No output file found" >>"${ISSUE_FILE}"
fi
- name: Append issue footer
if: ${{ steps.run_clang_tidy.outcome != 'success' }}
run: |
cat >> "${ISSUE_FILE}" <<EOF
cat >>"${ISSUE_FILE}" <<EOF
\`\`\`
---
@@ -176,7 +176,7 @@ jobs:
- name: Create issue
if: ${{ steps.run_clang_tidy.outcome != 'success' && inputs.create_issue_on_failure }}
uses: XRPLF/actions/create-issue@36d450d12d301e8410c1b7936e5de70c291cbe36
uses: XRPLF/actions/create-issue@2b8bc36af85b88bca0dd7bfac2e2dc05f94ad712
with:
title: "Clang-tidy check failed"
body_file: ${{ env.ISSUE_FILE }}

View File

@@ -39,7 +39,7 @@ jobs:
id: generate
working-directory: .github/scripts/strategy-matrix
run: |
./generate.py --packaging --config=linux.json >> "${GITHUB_OUTPUT}"
./generate.py --packaging --config=linux.json >>"${GITHUB_OUTPUT}"
generate-version:
runs-on: ubuntu-latest

View File

@@ -42,4 +42,4 @@ jobs:
env:
GENERATE_CONFIG: ${{ inputs.os != '' && format('--config={0}.json', inputs.os) || '' }}
GENERATE_OPTION: ${{ inputs.strategy_matrix == 'all' && '--all' || '' }}
run: ./generate.py ${GENERATE_OPTION} ${GENERATE_CONFIG} >> "${GITHUB_OUTPUT}"
run: ./generate.py ${GENERATE_OPTION} ${GENERATE_CONFIG} >>"${GITHUB_OUTPUT}"

View File

@@ -37,37 +37,50 @@ repos:
exclude: ^include/xrpl/protocol_autogen/(transactions|ledger_entries)/
- repo: https://github.com/pre-commit/mirrors-clang-format
rev: cd481d7b0bfb5c7b3090c21846317f9a8262e891 # frozen: v22.1.0
rev: dd18dad857d6133e90bbe478f4f2f22ec0030269 # frozen: v22.1.5
hooks:
- id: clang-format
args: [--style=file]
"types_or": [c++, c, proto]
exclude: ^include/xrpl/protocol_autogen/(transactions|ledger_entries)/
- repo: https://github.com/BlankSpruce/gersemi
rev: 0.26.0
- repo: https://github.com/BlankSpruce/gersemi-pre-commit
rev: faadd6a9d852369ca94f4d15b2404c967ba8cb01 # frozen: 0.27.6
hooks:
- id: gersemi
- repo: https://github.com/rbubley/mirrors-prettier
rev: c2bc67fe8f8f549cc489e00ba8b45aa18ee713b1 # frozen: v3.8.1
rev: 515f543f5718ebfd6ce22e16708bb32c68ff96e1 # frozen: v3.8.3
hooks:
- id: prettier
args: [--end-of-line=auto]
- repo: https://github.com/psf/black-pre-commit-mirror
rev: ea488cebbfd88a5f50b8bd95d5c829d0bb76feb8 # frozen: 26.1.0
rev: 4160603246a6b365d4a2af661c6d71b0a0f50478 # frozen: 26.5.1
hooks:
- id: black
- repo: https://github.com/openstack/bashate
rev: 5798d24d571676fc407e81df574c1ef57b520f23 # frozen: 2.1.1
- repo: https://github.com/scop/pre-commit-shfmt
rev: 05c1426671b9237fb5e1444dd63aa5731bec0dfb # frozen: v3.13.1-1
hooks:
- id: bashate
args: ["--ignore=E006"]
- id: shfmt
args: [--write, --indent=4, --case-indent=true]
- repo: local
hooks:
- id: format-inline-bash-workflows
name: "format `run:` blocks in workflows/actions"
entry: ./.github/scripts/format-inline-bash.py
language: python
files: ^\.github/(workflows|actions)/.*\.ya?ml$
- id: format-inline-bash-markdown
name: "format ```bash blocks in markdown"
entry: ./.github/scripts/format-inline-bash.py
language: python
files: \.md$
- repo: https://github.com/streetsidesoftware/cspell-cli
rev: a42085ade523f591dca134379a595e7859986445 # frozen: v9.7.0
rev: 4643f154907327ee0a2c7038f0296e0dd77d9776 # frozen: v10.0.0
hooks:
- id: cspell # Spell check changed files
exclude: |

View File

@@ -151,8 +151,8 @@ git init
git remote add origin git@github.com:XRPLF/conan-center-index.git
git sparse-checkout init
for recipe in "${recipes[@]}"; do
echo "Checking out recipe '${recipe}'..."
git sparse-checkout add recipes/${recipe}
echo "Checking out recipe '${recipe}'..."
git sparse-checkout add recipes/${recipe}
done
git fetch origin master
git checkout master
@@ -180,7 +180,7 @@ the new recipe will be automatically pulled from the official Conan Center.
If you see an error similar to the following after running `conan profile show`:
```bash
```text
ERROR: Invalid setting '17' is not a valid 'settings.compiler.version' value.
Possible values are ['5.0', '5.1', '6.0', '6.1', '7.0', '7.3', '8.0', '8.1',
'9.0', '9.1', '10.0', '11.0', '12.0', '13', '13.0', '13.1', '14', '14.0', '15',

View File

@@ -1,8 +1,8 @@
#!/bin/bash
if [[ $# -ne 1 || "$1" == "--help" || "$1" == "-h" ]]; then
name=$( basename $0 )
cat <<- USAGE
name=$(basename $0)
cat <<-USAGE
Usage: $name <username>
Where <username> is the Github username of the upstream repo. e.g. XRPLF
@@ -14,7 +14,7 @@ fi
shift
user="$1"
# Get the origin URL. Expect it be an SSH-style URL
origin=$( git remote get-url origin )
origin=$(git remote get-url origin)
if [[ "${origin}" == "" ]]; then
echo Invalid origin remote >&2
exit 1
@@ -22,11 +22,11 @@ fi
# echo "Origin: ${origin}"
# Parse the origin
ifs_orig="${IFS}"
IFS=':' read remote originpath <<< "${origin}"
IFS=':' read remote originpath <<<"${origin}"
# echo "Remote: ${remote}, Originpath: ${originpath}"
IFS='@' read sshuser server <<< "${remote}"
IFS='@' read sshuser server <<<"${remote}"
# echo "SSHUser: ${sshuser}, Server: ${server}"
IFS='/' read originuser repo <<< "${originpath}"
IFS='/' read originuser repo <<<"${originpath}"
# echo "Originuser: ${originuser}, Repo: ${repo}"
if [[ "${sshuser}" == "" || "${server}" == "" || "${originuser}" == "" || "${repo}" == "" ]]; then
echo "Can't parse origin URL: ${origin}" >&2
@@ -35,9 +35,9 @@ fi
upstream="https://${server}/${user}/${repo}"
upstreampush="${remote}:${user}/${repo}"
upstreamgroup="upstream upstream-push"
current=$( git remote get-url upstream 2>/dev/null )
currentpush=$( git remote get-url upstream-push 2>/dev/null )
currentgroup=$( git config remotes.upstreams )
current=$(git remote get-url upstream 2>/dev/null)
currentpush=$(git remote get-url upstream-push 2>/dev/null)
currentgroup=$(git config remotes.upstreams)
if [[ "${current}" == "${upstream}" ]]; then
echo "Upstream already set up correctly. Skip"
elif [[ -n "${current}" && "${current}" != "${upstream}" && "${current}" != "${upstreampush}" ]]; then
@@ -45,9 +45,9 @@ elif [[ -n "${current}" && "${current}" != "${upstream}" && "${current}" != "${u
else
if [[ "${current}" == "${upstreampush}" ]]; then
echo "Upstream set to dangerous push URL. Update."
_run git remote rename upstream upstream-push || \
_run git remote remove upstream
currentpush=$( git remote get-url upstream-push 2>/dev/null )
_run git remote rename upstream upstream-push ||
_run git remote remove upstream
currentpush=$(git remote get-url upstream-push 2>/dev/null)
fi
_run git remote add upstream "${upstream}"
fi

View File

@@ -1,8 +1,8 @@
#!/bin/bash
if [[ $# -lt 3 || "$1" == "--help" || "$1" = "-h" ]]; then
name=$( basename $0 )
cat <<- USAGE
name=$(basename $0)
cat <<-USAGE
Usage: $name workbranch base/branch user/branch [user/branch [...]]
* workbranch will be created locally from base/branch
@@ -16,7 +16,7 @@ fi
work="$1"
shift
branches=( $( echo "${@}" | sed "s/:/\//" ) )
branches=($(echo "${@}" | sed "s/:/\//"))
base="${branches[0]}"
unset branches[0]
@@ -24,10 +24,10 @@ set -e
users=()
for b in "${branches[@]}"; do
users+=( $( echo $b | cut -d/ -f1 ) )
users+=($(echo $b | cut -d/ -f1))
done
users=( $( printf '%s\n' "${users[@]}" | sort -u ) )
users=($(printf '%s\n' "${users[@]}" | sort -u))
git fetch --multiple upstreams "${users[@]}"
git checkout -B "$work" --no-track "$base"
@@ -40,7 +40,7 @@ done
# Make sure the commits look right
git log --show-signature "$base..HEAD"
parts=( $( echo $base | sed "s/\// /" ) )
parts=($(echo $base | sed "s/\// /"))
repo="${parts[0]}"
b="${parts[1]}"
push=$repo
@@ -50,7 +50,7 @@ fi
if [[ "$repo" == "upstream" ]]; then
repo="upstreams"
fi
cat << PUSH
cat <<PUSH
-------------------------------------------------------------------
This script will not push. Verify everything is correct, then push

View File

@@ -1,8 +1,8 @@
#!/bin/bash
if [[ $# -ne 3 || "$1" == "--help" || "$1" = "-h" ]]; then
name=$( basename $0 )
cat <<- USAGE
name=$(basename $0)
cat <<-USAGE
Usage: $name workbranch base/branch version
* workbranch will be created locally from base/branch. If it exists,
@@ -16,7 +16,7 @@ fi
work="$1"
shift
base=$( echo "$1" | sed "s/:/\//" )
base=$(echo "$1" | sed "s/:/\//")
shift
version=$1
@@ -28,16 +28,16 @@ git fetch upstreams
git checkout -B "${work}" --no-track "${base}"
push=$( git rev-parse --abbrev-ref --symbolic-full-name '@{push}' \
2>/dev/null ) || true
push=$(git rev-parse --abbrev-ref --symbolic-full-name '@{push}' \
2>/dev/null) || true
if [[ "${push}" != "" ]]; then
echo "Warning: ${push} may already exist."
fi
build=$( find -name BuildInfo.cpp )
sed 's/\(^.*versionString =\).*$/\1 "'${version}'"/' ${build} > version.cpp && \
diff "${build}" version.cpp && exit 1 || \
mv -vi version.cpp ${build}
build=$(find -name BuildInfo.cpp)
sed 's/\(^.*versionString =\).*$/\1 "'${version}'"/' ${build} >version.cpp &&
diff "${build}" version.cpp && exit 1 ||
mv -vi version.cpp ${build}
git diff
@@ -47,7 +47,7 @@ git commit -S -m "Set version to ${version}"
git log --oneline --first-parent ${base}^..
cat << PUSH
cat <<PUSH
-------------------------------------------------------------------
This script will not push. Verify everything is correct, then push

View File

@@ -168,7 +168,13 @@ def main():
if not os.environ.get("TIDY"):
return 0
repo_root = Path(__file__).parent.parent
repo_root = Path(
subprocess.check_output(
["git", "rev-parse", "--show-toplevel"],
cwd=Path(__file__).parent,
text=True,
).strip()
)
files = staged_files(repo_root)
if not files:
return 0

View File

@@ -953,6 +953,21 @@
#
# Optional keys for NuDB and RocksDB:
#
# cache_size Size of cache for database records. Default is 16384.
# Setting this value to 0 will use the default value.
#
# cache_age Length of time in minutes to keep database records
# cached. Default is 5 minutes. Setting this value to
# 0 will use the default value.
#
# Note: if cache_size or cache_age is not specified,
# default values will be used for the unspecified
# parameter.
#
# Note: the cache will not be created if online_delete
# is specified, because the rotating NodeStore does
# not use this cache).
#
# fast_load Boolean. If set, load the last persisted ledger
# from disk upon process start before syncing to
# the network. This is likely to improve performance

View File

@@ -0,0 +1,13 @@
# Python dependencies for XRP Ledger code generation scripts
#
# These packages are required to run the code generation scripts that
# parse macro files and generate C++ wrapper classes.
# C preprocessor for Python - used to preprocess macro files
pcpp>=1.30
# Parser combinator library - used to parse the macro DSL
pyparsing>=3.0.0
# Template engine - used to generate C++ code from templates
Mako>=1.2.2

View File

@@ -1,13 +1,105 @@
# Python dependencies for XRP Ledger code generation scripts
#
# These packages are required to run the code generation scripts that
# parse macro files and generate C++ wrapper classes.
# C preprocessor for Python - used to preprocess macro files
pcpp>=1.30
# Parser combinator library - used to parse the macro DSL
pyparsing>=3.0.0
# Template engine - used to generate C++ code from templates
Mako>=1.2.2
# This file was autogenerated by uv via the following command:
# uv pip compile requirements.in --generate-hashes --output-file requirements.txt
mako==1.3.12 \
--hash=sha256:8f61569480282dbf557145ce441e4ba888be453c30989f879f0d652e39f53ea9 \
--hash=sha256:9f778e93289bd410bb35daadeb4fc66d95a746f0b75777b942088b7fd7af550a
# via -r requirements.in
markupsafe==3.0.3 \
--hash=sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f \
--hash=sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a \
--hash=sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf \
--hash=sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19 \
--hash=sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf \
--hash=sha256:0f4b68347f8c5eab4a13419215bdfd7f8c9b19f2b25520968adfad23eb0ce60c \
--hash=sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175 \
--hash=sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219 \
--hash=sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb \
--hash=sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6 \
--hash=sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab \
--hash=sha256:15d939a21d546304880945ca1ecb8a039db6b4dc49b2c5a400387cdae6a62e26 \
--hash=sha256:177b5253b2834fe3678cb4a5f0059808258584c559193998be2601324fdeafb1 \
--hash=sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce \
--hash=sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218 \
--hash=sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634 \
--hash=sha256:1ba88449deb3de88bd40044603fafffb7bc2b055d626a330323a9ed736661695 \
--hash=sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad \
--hash=sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73 \
--hash=sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c \
--hash=sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe \
--hash=sha256:2a15a08b17dd94c53a1da0438822d70ebcd13f8c3a95abe3a9ef9f11a94830aa \
--hash=sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559 \
--hash=sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa \
--hash=sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37 \
--hash=sha256:3537e01efc9d4dccdf77221fb1cb3b8e1a38d5428920e0657ce299b20324d758 \
--hash=sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f \
--hash=sha256:38664109c14ffc9e7437e86b4dceb442b0096dfe3541d7864d9cbe1da4cf36c8 \
--hash=sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d \
--hash=sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c \
--hash=sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97 \
--hash=sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a \
--hash=sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19 \
--hash=sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9 \
--hash=sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9 \
--hash=sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc \
--hash=sha256:591ae9f2a647529ca990bc681daebdd52c8791ff06c2bfa05b65163e28102ef2 \
--hash=sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4 \
--hash=sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354 \
--hash=sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50 \
--hash=sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698 \
--hash=sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9 \
--hash=sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b \
--hash=sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc \
--hash=sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115 \
--hash=sha256:7c3fb7d25180895632e5d3148dbdc29ea38ccb7fd210aa27acbd1201a1902c6e \
--hash=sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485 \
--hash=sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f \
--hash=sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12 \
--hash=sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025 \
--hash=sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009 \
--hash=sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d \
--hash=sha256:949b8d66bc381ee8b007cd945914c721d9aba8e27f71959d750a46f7c282b20b \
--hash=sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a \
--hash=sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5 \
--hash=sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f \
--hash=sha256:a320721ab5a1aba0a233739394eb907f8c8da5c98c9181d1161e77a0c8e36f2d \
--hash=sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1 \
--hash=sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287 \
--hash=sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6 \
--hash=sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f \
--hash=sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581 \
--hash=sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed \
--hash=sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b \
--hash=sha256:c0c0b3ade1c0b13b936d7970b1d37a57acde9199dc2aecc4c336773e1d86049c \
--hash=sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026 \
--hash=sha256:c4ffb7ebf07cfe8931028e3e4c85f0357459a3f9f9490886198848f4fa002ec8 \
--hash=sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676 \
--hash=sha256:d2ee202e79d8ed691ceebae8e0486bd9a2cd4794cec4824e1c99b6f5009502f6 \
--hash=sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e \
--hash=sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d \
--hash=sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d \
--hash=sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01 \
--hash=sha256:df2449253ef108a379b8b5d6b43f4b1a8e81a061d6537becd5582fba5f9196d7 \
--hash=sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419 \
--hash=sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795 \
--hash=sha256:e2103a929dfa2fcaf9bb4e7c091983a49c9ac3b19c9061b6d5427dd7d14d81a1 \
--hash=sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5 \
--hash=sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d \
--hash=sha256:e8fc20152abba6b83724d7ff268c249fa196d8259ff481f3b1476383f8f24e42 \
--hash=sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe \
--hash=sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda \
--hash=sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e \
--hash=sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737 \
--hash=sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523 \
--hash=sha256:f42d0984e947b8adf7dd6dde396e720934d12c506ce84eea8476409563607591 \
--hash=sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc \
--hash=sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a \
--hash=sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50
# via mako
pcpp==1.30 \
--hash=sha256:05fe08292b6da57f385001c891a87f40d6aa7f46787b03e8ba326d20a3297c6e \
--hash=sha256:5af9fbce55f136d7931ae915fae03c34030a3b36c496e72d9636cedc8e2543a1
# via -r requirements.in
pyparsing==3.3.2 \
--hash=sha256:850ba148bd908d7e2411587e247a1e4f0327839c40e2e5e6d05a007ecc69911d \
--hash=sha256:c777f4d763f140633dcb6d8a3eda953bf7a214dc4eff598413c070bcdc117cbc
# via -r requirements.in

View File

@@ -93,6 +93,7 @@ words:
- daria
- dcmake
- dearmor
- dedented
- deleteme
- demultiplexer
- deserializaton
@@ -199,11 +200,13 @@ words:
- nonxrp
- noreplace
- noripple
- nostdinc
- notifempty
- nudb
- nullptr
- nunl
- Nyffenegger
- onlatest
- ostr
- pargs
- partitioner
@@ -254,6 +257,7 @@ words:
- sfields
- shamap
- shamapitem
- shfmt
- shlibs
- sidechain
- SIGGOOD
@@ -298,6 +302,7 @@ words:
- unauthorizing
- unergonomic
- unfetched
- unfindable
- unflatten
- unfund
- unimpair

48
docker/check-sanitizers.sh Executable file
View File

@@ -0,0 +1,48 @@
#!/bin/bash
# Sanity-check that the sanitizer runtimes shipped with g++/clang++ work
# end-to-end against the system loader: compile each example with both
# compilers, run it, and confirm the expected diagnostic is emitted.
set -eo pipefail
cpp_files_dir="${1:?usage: $0 <cpp_files_dir>}"
case "$(uname -m)" in
x86_64) loader=/lib64/ld-linux-x86-64.so.2 ;;
aarch64) loader=/lib/ld-linux-aarch64.so.1 ;;
*)
echo "Unsupported arch: $(uname -m)" >&2
exit 1
;;
esac
declare -A sanitize=(
[asan]="-fsanitize=address"
[tsan]="-fsanitize=thread"
[ubsan]="-fsanitize=undefined"
)
declare -A expect=(
[asan]="heap-use-after-free"
[tsan]="data race"
[ubsan]="signed integer overflow"
)
for compiler in g++ clang++; do
for name in asan tsan ubsan; do
bin="/tmp/${name}-${compiler}"
echo "=== Build ${name} with ${compiler} ==="
"$compiler" -std=c++20 -O1 -g ${sanitize[$name]} \
-Wl,--dynamic-linker=$loader \
"${cpp_files_dir}/${name}.cpp" -o "$bin"
echo "=== Run ${name}-${compiler} ==="
output=$("$bin" 2>&1) || true
echo "$output"
echo "$output" | grep -q "${expect[$name]}" ||
{
echo "expected '${expect[$name]}' from $bin"
exit 1
}
rm -f "$bin"
done
done

28
docker/cpp_files/asan.cpp Normal file
View File

@@ -0,0 +1,28 @@
#include <atomic>
#include <cstddef>
#include <iostream>
#if defined(__clang__) || defined(__GNUC__)
__attribute__((noinline))
#elif defined(_MSC_VER)
__declspec(noinline)
#endif
int
read_after_free(volatile int* array, std::size_t index)
{
std::atomic_signal_fence(std::memory_order_seq_cst);
int value = array[index];
std::atomic_signal_fence(std::memory_order_seq_cst);
return value;
}
int
main()
{
int* array = new int[5]{10, 20, 30, 40, 50};
delete[] array;
std::cout << "Value at index 2: " << read_after_free(array, 2) << std::endl;
return 0;
}

26
docker/cpp_files/tsan.cpp Normal file
View File

@@ -0,0 +1,26 @@
#include <iostream>
#include <thread>
static int kCounter = 0;
void
increment()
{
for (int i = 0; i < 100'000; ++i)
{
++kCounter;
}
}
int
main()
{
std::thread t1(increment);
std::thread t2(increment);
t1.join();
t2.join();
std::cout << "Final counter value: " << kCounter << std::endl;
return 0;
}

View File

@@ -0,0 +1,13 @@
#include <iostream>
#include <limits>
int
main()
{
int maxInt = std::numeric_limits<int>::max();
int volatile one = 1;
std::cout << "Current max: " << maxInt << std::endl;
int overflowed = maxInt + one;
std::cout << "Overflowed result: " << overflowed << std::endl;
return 0;
}

View File

@@ -45,8 +45,30 @@ COPY --from=builder /tmp/build/result /nix/ci-env
ENV PATH="/nix/ci-env/bin:$PATH"
# Externally-built dynamically-linked ELF binaries hard-code the loader path
# (e.g. /lib64/ld-linux-x86-64.so.2) in their PT_INTERP header. Copy the
# loader from the Nix store to that path when the base image doesn't already
# provide one (i.e. on nixos/nix).
RUN <<EOF
case "$(uname -m)" in
x86_64) target=/lib64/ld-linux-x86-64.so.2 ;;
aarch64) target=/lib/ld-linux-aarch64.so.1 ;;
*) echo "Unsupported arch: $(uname -m)" >&2; exit 1 ;;
esac
if [ ! -e "$target" ]; then
# Use the loader from the same glibc that gcc links libc against, so
# ld-linux and libc/libpthread share GLIBC_PRIVATE symbols at runtime.
src="$(dirname "$(gcc -print-file-name=libc.so.6)")/$(basename "$target")"
[ -e "$src" ] || { echo "ld-linux not found at $src" >&2; exit 1; }
mkdir -p "$(dirname "$target")"
cp "$src" "$target"
fi
EOF
RUN <<EOF
ccache --version
clang --version
clang++ --version
clang-format --version
cmake --version
conan --version
@@ -64,3 +86,10 @@ python3 --version
run-clang-tidy --help
vim --version
EOF
# Sanity-check that the sanitizer runtimes shipped with g++/clang++ work
# end-to-end against the system loader.
COPY docker/cpp_files/ /tmp/cpp_files/
COPY docker/check-sanitizers.sh /tmp/check-sanitizers.sh
RUN grep -qi ubuntu /etc/os-release 2>/dev/null && /tmp/check-sanitizers.sh /tmp/cpp_files || true

4
flake.lock generated
View File

@@ -15,7 +15,7 @@
"type": "indirect"
}
},
"nixpkgs-glibc231": {
"nixpkgs-custom-glibc": {
"flake": false,
"locked": {
"lastModified": 1593520194,
@@ -35,7 +35,7 @@
"root": {
"inputs": {
"nixpkgs": "nixpkgs",
"nixpkgs-glibc231": "nixpkgs-glibc231"
"nixpkgs-custom-glibc": "nixpkgs-custom-glibc"
}
}
},

View File

@@ -6,16 +6,16 @@
# version — matches the system libc on Ubuntu 20.04 LTS. Imported
# manually (flake = false) because this revision predates nixpkgs'
# own flake.nix.
nixpkgs-glibc231 = {
nixpkgs-custom-glibc = {
url = "github:NixOS/nixpkgs/9cd98386a38891d1074fc18036b842dc4416f562";
flake = false;
};
};
outputs =
{ nixpkgs, nixpkgs-glibc231, ... }:
{ nixpkgs, nixpkgs-custom-glibc, ... }:
let
forEachSystem = import ./nix/utils.nix { inherit nixpkgs nixpkgs-glibc231; };
forEachSystem = import ./nix/utils.nix { inherit nixpkgs nixpkgs-custom-glibc; };
in
{
devShells = forEachSystem (import ./nix/devshell.nix);

View File

@@ -2,12 +2,16 @@
#include <xrpl/beast/utility/instrumentation.h>
#include <array>
#include <cstdint>
#include <functional>
#include <limits>
#include <optional>
#include <ostream>
#include <set>
#include <stdexcept>
#include <string>
#include <unordered_map>
namespace xrpl {
@@ -38,17 +42,58 @@ isPowerOfTen(T value)
return logTen(value).has_value();
}
namespace detail {
/** Builds a table of the powers of 10
*
* This function is marked consteval, so it can only be run in
* a constexpr context. This assures that it is and can only be run at
* compile time. Doing it at runtime would be pretty wasteful and
* inefficient.
*/
constexpr std::size_t kInt64Digits = 20;
consteval std::array<std::uint64_t, kInt64Digits>
buildPowersOfTen()
{
std::array<std::uint64_t, kInt64Digits> result{};
std::uint64_t power = 1;
std::size_t exponent = 0;
// end the loop early so it doesn't overflow;
for (; exponent < result.size() - 1; ++exponent, power *= 10)
{
result[exponent] = power;
if (power > std::numeric_limits<std::uint64_t>::max() / 10)
throw std::logic_error("Power of 10 table is too big");
}
result[exponent] = power;
if (power < std::numeric_limits<std::uint64_t>::max() / 10)
throw std::logic_error("Power of 10 table is not big enough for the uint64_t type");
return result;
}
} // namespace detail
constexpr std::array<std::uint64_t, detail::kInt64Digits> kPowerOfTen = detail::buildPowersOfTen();
static_assert(kPowerOfTen[0] == 1);
static_assert(kPowerOfTen[1] == 10);
static_assert(kPowerOfTen[10] == 10'000'000'000);
static_assert(
isPowerOfTen(kPowerOfTen.back()) && *logTen(kPowerOfTen.back()) == detail::kInt64Digits - 1);
/** MantissaRange defines a range for the mantissa of a normalized Number.
*
* The mantissa is in the range [min, max], where
* * min is a power of 10, and
* * max = min * 10 - 1.
*
* The mantissa_scale enum indicates whether the range is "small" or "large".
* This intentionally restricts the number of MantissaRanges that can be
* instantiated to two: one for each scale.
* The MantissaScale enum indicates properties of the range: size, and some behavioral
* options. This intentionally restricts the number of unique MantissaRanges that can
* be instantiated: one for each scale.
*
* The "small" scale is based on the behavior of STAmount for IOUs. It has a min
* The "Small" scale is based on the behavior of STAmount for IOUs. It has a min
* value of 10^15, and a max value of 10^16-1. This was sufficient for
* uses before Lending Protocol was implemented, mostly related to AMM.
*
@@ -59,46 +104,100 @@ isPowerOfTen(T value)
* STNumber field type, and for internal calculations. That necessitated the
* "large" scale.
*
* The "large" scale is intended to represent all values that can be represented
* The "Large" scales are intended to represent all values that can be represented
* by an STAmount - IOUs, XRP, and MPTs. It has a min value of 10^18, and a max
* value of 10^19-1.
* value of 10^19-1. "LargeLegacy" is like "Large", but preserves
* a rounding error when a computation results in a mantissa of
* Number::kMaxRep that needs to be rounded up, but rounds down
* instead. It will maintain consistent behavior until the fixCleanup3_2_0
* amendment is enabled.
*
* Note that if the mentioned amendments are eventually retired, this class
* should be left in place, but the "small" scale option should be removed. This
* should be left in place, but the "Small" scale option should be removed. This
* will allow for future expansion beyond 64-bits if it is ever needed.
*/
struct MantissaRange
struct MantissaRange final
{
using rep = std::uint64_t;
enum class MantissaScale { Small, Large };
explicit constexpr MantissaRange(MantissaScale scale)
: min(getMin(scale)), log(logTen(min).value_or(-1)), scale(scale)
enum class MantissaScale {
Small,
// LargeLegacy can be removed when fixCleanup3_2_0 is retired
LargeLegacy,
Large,
};
// This entire enum can be removed when fixCleanup3_2_0 is retired
enum class CuspRoundingFix : bool {
Disabled = false,
Enabled = true,
};
explicit constexpr MantissaRange(MantissaScale sc) : scale(sc)
{
}
rep min;
rep max{(min * 10) - 1};
int log;
MantissaScale scale;
MantissaScale const scale;
int const log{getExponent(scale)};
rep const min{getMin(scale, log)};
rep const max{(min * 10) - 1};
CuspRoundingFix const cuspRoundingFixEnabled{isCuspFixEnabled(scale)};
static MantissaRange const&
getMantissaRange(MantissaScale scale);
static std::set<MantissaScale> const&
getAllScales();
private:
static constexpr rep
getMin(MantissaScale scale)
static constexpr int
getExponent(MantissaScale scale)
{
switch (scale)
{
case MantissaScale::Small:
return 1'000'000'000'000'000ULL;
return 15;
case MantissaScale::LargeLegacy:
case MantissaScale::Large:
return 1'000'000'000'000'000'000ULL;
return 18;
// LCOV_EXCL_START
default:
// Since this can never be called outside a non-constexpr
// context, this throw assures that the build fails if an
// If called in a constexpr context, this throw assures that the build fails if an
// invalid scale is used.
throw std::runtime_error("Unknown mantissa scale");
// LCOV_EXCL_STOP
}
}
// Keep this function for future use with different ways to compute
// the ranges.
static constexpr rep
getMin(MantissaScale scale, int exponent)
{
if (exponent < 0 || exponent >= kPowerOfTen.size())
throw std::runtime_error("Invalid exponent"); // LCOV_EXCL_LINE
return kPowerOfTen[exponent];
}
static constexpr CuspRoundingFix
isCuspFixEnabled(MantissaScale scale)
{
switch (scale)
{
case MantissaScale::Small:
case MantissaScale::LargeLegacy:
return CuspRoundingFix::Disabled;
case MantissaScale::Large:
return CuspRoundingFix::Enabled;
default:
// If called in a constexpr context, this throw assures that the build fails if an
// invalid scale is used.
throw std::runtime_error("Unknown mantissa scale"); // LCOV_EXCL_LINE
}
}
static std::unordered_map<MantissaScale, MantissaRange> const&
getRanges();
};
// Like std::integral, but only 64-bit integral types.
@@ -203,7 +302,7 @@ concept Integral64 = std::is_same_v<T, std::int64_t> || std::is_same_v<T, std::u
* amendments are enabled to determine which result to expect.
*
*/
class Number
class Number final
{
using rep = std::int64_t;
using internalrep = MantissaRange::rep;
@@ -309,33 +408,40 @@ public:
}
friend constexpr bool
operator<(Number const& x, Number const& y) noexcept
operator<(Number const& l, Number const& r) noexcept
{
bool const lneg = l.negative_;
bool const rneg = r.negative_;
// If the two amounts have different signs (zero is treated as positive)
// then the comparison is true iff the left is negative.
bool const lneg = x.negative_;
bool const rneg = y.negative_;
if (lneg != rneg)
return lneg;
// Both have same sign and the left is zero: the right must be
// greater than 0.
if (x.mantissa_ == 0)
return y.mantissa_ > 0;
// Both have same sign and the left is zero: both must be non-negative.
// If the right is greater than 0, then it is larger, so the comparison is true.
if (l.mantissa_ == 0)
return r.mantissa_ > 0;
// Both have same sign, the right is zero and the left is non-zero.
if (y.mantissa_ == 0)
// Both have same sign, the right is zero and the left is non-zero, so the left must be
// positive, and thus is larger, so the comparison is false.
if (r.mantissa_ == 0)
return false;
// Both have the same sign, compare by exponents:
if (x.exponent_ > y.exponent_)
if (l.exponent_ > r.exponent_)
return lneg;
if (x.exponent_ < y.exponent_)
if (l.exponent_ < r.exponent_)
return !lneg;
// If equal exponents, compare mantissas
return x.mantissa_ < y.mantissa_;
// If equal signs and exponents, compare mantissas.
if (lneg)
{
// If negative, the operator is reversed.
return l.mantissa_ > r.mantissa_;
}
return l.mantissa_ < r.mantissa_;
}
/** Return the sign of the amount */
@@ -424,49 +530,28 @@ public:
return kRange.get().log;
}
/// oneSmall is needed because the ranges are private
static constexpr Number
oneSmall();
/// oneLarge is needed because the ranges are private
static constexpr Number
oneLarge();
// And one is needed because it needs to choose between oneSmall and
// oneLarge based on the current range
static Number
one();
template <Integral64 T>
template <
auto MinMantissa,
auto MaxMantissa,
Integral64 T = std::decay_t<decltype(MinMantissa)>>
[[nodiscard]]
std::pair<T, int>
normalizeToRange(T minMantissa, T maxMantissa) const;
normalizeToRange() const;
private:
static thread_local RoundingMode mode;
// The available ranges for mantissa
static constexpr MantissaRange kSmallRange{MantissaRange::MantissaScale::Small};
static_assert(isPowerOfTen(kSmallRange.min));
static_assert(kSmallRange.min == 1'000'000'000'000'000LL);
static_assert(kSmallRange.max == 9'999'999'999'999'999LL);
static_assert(kSmallRange.log == 15);
static_assert(kSmallRange.min < kMaxRep);
static_assert(kSmallRange.max < kMaxRep);
static constexpr MantissaRange kLargeRange{MantissaRange::MantissaScale::Large};
static_assert(isPowerOfTen(kLargeRange.min));
static_assert(kLargeRange.min == 1'000'000'000'000'000'000ULL);
static_assert(kLargeRange.max == internalrep(9'999'999'999'999'999'999ULL));
static_assert(kLargeRange.log == 18);
static_assert(kLargeRange.min < kMaxRep);
static_assert(kLargeRange.max > kMaxRep);
// The range for the mantissa when normalized.
// Use reference_wrapper to avoid making copies, and prevent accidentally
// changing the values inside the range.
static thread_local std::reference_wrapper<MantissaRange const> kRange;
void
normalize();
normalize(MantissaRange const& range);
/** Normalize Number components to an arbitrary range.
*
@@ -481,7 +566,8 @@ private:
T& mantissa,
int& exponent,
internalrep const& minMantissa,
internalrep const& maxMantissa);
internalrep const& maxMantissa,
MantissaRange::CuspRoundingFix cuspRoundingFixEnabled);
template <class T>
friend void
@@ -490,7 +576,9 @@ private:
T& mantissa,
int& exponent,
MantissaRange::rep const& minMantissa,
MantissaRange::rep const& maxMantissa);
MantissaRange::rep const& maxMantissa,
MantissaRange::CuspRoundingFix cuspRoundingFixEnabled,
bool dropped);
[[nodiscard]] bool
isnormal() const noexcept;
@@ -526,7 +614,7 @@ static constexpr Number kNumZero{};
inline Number::Number(bool negative, internalrep mantissa, int exponent, Normalized)
: Number(negative, mantissa, exponent, Unchecked{})
{
normalize();
normalize(kRange);
}
inline Number::Number(internalrep mantissa, int exponent, Normalized)
@@ -696,10 +784,21 @@ Number::isnormal() const noexcept
kMinExponent <= exponent_ && exponent_ <= kMaxExponent);
}
template <Integral64 T>
template <auto MinMantissa, auto MaxMantissa, Integral64 T>
std::pair<T, int>
Number::normalizeToRange(T minMantissa, T maxMantissa) const
Number::normalizeToRange() const
{
static_assert(std::is_same_v<T, std::uint64_t> || std::is_same_v<T, std::int64_t>);
static_assert(std::is_same_v<T, std::decay_t<decltype(MinMantissa)>>);
static_assert(std::is_same_v<T, std::decay_t<decltype(MaxMantissa)>>);
auto constexpr kMIN = static_cast<T>(MinMantissa);
auto constexpr kMAX = static_cast<T>(MaxMantissa);
static_assert(kMIN > 0);
static_assert(kMIN % 10 == 0);
static_assert(isPowerOfTen(kMIN));
static_assert(kMAX % 10 == 9);
static_assert((kMAX + 1) / 10 == kMIN);
bool negative = negative_;
internalrep mantissa = mantissa_;
int exponent = exponent_;
@@ -711,7 +810,10 @@ Number::normalizeToRange(T minMantissa, T maxMantissa) const
"xrpl::Number::normalizeToRange",
"Number is non-negative for unsigned range.");
}
Number::normalize(negative, mantissa, exponent, minMantissa, maxMantissa);
// Don't need to worry about the cuspRounding fix because rounding up will never take the
// mantissa over maxMantissa with a ones digit value other than 0. 0 can safely be truncated.
Number::normalize(
negative, mantissa, exponent, kMIN, kMAX, MantissaRange::CuspRoundingFix::Disabled);
auto const sign = negative ? -1 : 1;
return std::make_pair(static_cast<T>(sign * mantissa), exponent);
@@ -763,6 +865,8 @@ to_string(MantissaRange::MantissaScale const& scale)
{
case MantissaRange::MantissaScale::Small:
return "small";
case MantissaRange::MantissaScale::LargeLegacy:
return "largeLegacy";
case MantissaRange::MantissaScale::Large:
return "large";
default:

View File

@@ -1,29 +0,0 @@
#pragma once
#if XRPL_ROCKSDB_AVAILABLE
// #include <rocksdb2/port/port_posix.h>
#include <rocksdb/cache.h>
#include <rocksdb/compaction_filter.h>
#include <rocksdb/comparator.h>
#include <rocksdb/convenience.h>
#include <rocksdb/db.h>
#include <rocksdb/env.h>
#include <rocksdb/filter_policy.h>
#include <rocksdb/flush_block_policy.h>
#include <rocksdb/iterator.h>
#include <rocksdb/memtablerep.h>
#include <rocksdb/merge_operator.h>
#include <rocksdb/options.h>
#include <rocksdb/perf_context.h>
#include <rocksdb/slice.h>
#include <rocksdb/slice_transform.h>
#include <rocksdb/statistics.h>
#include <rocksdb/status.h>
#include <rocksdb/table.h>
#include <rocksdb/table_properties.h>
#include <rocksdb/transaction_log.h>
#include <rocksdb/types.h>
#include <rocksdb/universal_compaction.h>
#include <rocksdb/write_batch.h>
#endif

View File

@@ -1,49 +0,0 @@
#pragma once
#include <xrpl/protocol/MultiApiJson.h>
#include <xrpl/server/InfoSub.h>
#include <memory>
#include <mutex>
namespace xrpl {
/** Listen to public/subscribe messages from a book. */
class BookListeners
{
public:
using pointer = std::shared_ptr<BookListeners>;
BookListeners() = default;
/** Add a new subscription for this book
*/
void
addSubscriber(InfoSub::ref sub);
/** Stop publishing to a subscriber
*/
void
removeSubscriber(std::uint64_t sub);
/** Publish a transaction to subscribers
Publish a transaction to clients subscribed to changes on this book.
Uses havePublished to prevent sending duplicate transactions to clients
that have subscribed to multiple books.
@param jvObj JSON transaction data to publish
@param havePublished InfoSub sequence numbers that have already
published this transaction.
*/
void
publish(MultiApiJson const& jvObj, hash_set<std::uint64_t>& havePublished);
private:
std::recursive_mutex lock_;
hash_map<std::uint64_t, InfoSub::wptr> listeners_;
};
} // namespace xrpl

View File

@@ -1,11 +1,11 @@
#pragma once
#include <xrpl/basics/UnorderedContainers.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/ledger/AcceptedLedgerTx.h>
#include <xrpl/ledger/BookListeners.h>
#include <xrpl/ledger/ReadView.h>
#include <xrpl/protocol/Asset.h>
#include <xrpl/protocol/Book.h>
#include <xrpl/protocol/MultiApiJson.h>
#include <xrpl/protocol/UintTypes.h>
#include <memory>
@@ -77,34 +77,24 @@ public:
*/
virtual bool
isBookToXRP(Asset const& asset, std::optional<Domain> const& domain = std::nullopt) = 0;
/**
* Process a transaction for order book tracking.
* @param ledger The ledger the transaction was applied to
* @param alTx The transaction to process
* @param jvObj The JSON object of the transaction
*/
virtual void
processTxn(
std::shared_ptr<ReadView const> const& ledger,
AcceptedLedgerTx const& alTx,
MultiApiJson const& jvObj) = 0;
/**
* Get the book listeners for a book.
* @param book The book to get the listeners for
* @return The book listeners for the book
*/
virtual BookListeners::pointer
getBookListeners(Book const&) = 0;
/**
* Create a new book listeners for a book.
* @param book The book to create the listeners for
* @return The new book listeners for the book
*/
virtual BookListeners::pointer
makeBookListeners(Book const&) = 0;
};
/** Extract the set of books affected by a transaction.
*
* Walks the transaction's metadata nodes and collects every order book
* whose offers were created, modified, or deleted. Used by NetworkOPs to
* fan transaction notifications out to book subscribers.
*
* @param alTx The accepted ledger transaction to inspect.
* @param j Journal used to log per-node parsing failures. Inspecting an
* offer node can throw if a required field is missing; in that
* case the bad node is skipped and a warn-level message is
* emitted via @p j. Other affected books in the same transaction
* are still returned.
* @return The set of books whose offers were created, modified, or
* deleted. May be empty for non-offer transactions.
*/
hash_set<Book>
affectedBooks(AcceptedLedgerTx const& alTx, beast::Journal const& j);
} // namespace xrpl

View File

@@ -203,6 +203,21 @@ getAssetsTotalScale(SLE::const_ref vaultSle)
return scale(vaultSle->at(sfAssetsTotal), vaultSle->at(sfAsset));
}
// Compute the minimum required broker cover, rounded consistently.
// DebtTotal is a broker-level aggregate maintained at vault scale, so the
// rounding must also use vault scale — never an individual loan's scale.
inline Number
minimumBrokerCover(Number const& debtTotal, TenthBips32 coverRateMinimum, SLE::const_ref vaultSle)
{
XRPL_ASSERT(
vaultSle && vaultSle->getType() == ltVAULT, "xrpl::minimumBrokerCover : valid Vault sle");
NumberRoundModeGuard const mg(Number::RoundingMode::Upward);
return roundToAsset(
vaultSle->at(sfAsset),
tenthBipsOfValue(debtTotal, coverRateMinimum),
getAssetsTotalScale(vaultSle));
}
TER
checkLoanGuards(
Asset const& vaultAsset,
@@ -446,6 +461,7 @@ loanAccruedInterest(
ExtendedPaymentComponents
computeOverpaymentComponents(
Rules const& rules,
Asset const& asset,
int32_t const loanScale,
Number const& overpayment,

View File

@@ -171,6 +171,15 @@ canTransfer(
[[nodiscard]] TER
canTrade(ReadView const& view, Asset const& asset, std::uint8_t depth = 0);
/** Convenience to combine canTrade/Transfer. Returns tesSUCCESS if Asset is Issue.
*/
[[nodiscard]] TER
canMPTTradeAndTransfer(
ReadView const& v,
Asset const& asset,
AccountID const& from,
AccountID const& to);
//------------------------------------------------------------------------------
//
// Empty holding operations (MPT-specific)
@@ -277,17 +286,4 @@ issuerFundsToSelfIssue(ReadView const& view, MPTIssue const& issue);
void
issuerSelfDebitHookMPT(ApplyView& view, MPTIssue const& issue, std::uint64_t amount);
//------------------------------------------------------------------------------
//
// MPT DEX
//
//------------------------------------------------------------------------------
/* Return true if a transaction is allowed for the specified MPT/account. The
* function checks MPTokenIssuance and MPToken objects flags to determine if the
* transaction is allowed.
*/
TER
checkMPTTxAllowed(ReadView const& v, TxType tx, Asset const& asset, AccountID const& accountID);
} // namespace xrpl

View File

@@ -5,8 +5,21 @@
#include <xrpl/protocol/TER.h>
#include <xrpl/protocol/UintTypes.h>
#include <cstdint>
#include <memory>
#include <optional>
namespace xrpl {
/** Close a payment channel and return its remaining funds to the channel owner.
*
* @param slep The SLE for the PayChannel object to close.
* @param view The apply view in which ledger state modifications are made.
* @param key The ledger key identifying the PayChannel entry.
* @param j Journal used for fatal-level diagnostic messages.
* @return tesSUCCESS on success; tefBAD_LEDGER if a directory removal
* fails; tefINTERNAL if the source account SLE cannot be found.
*/
TER
closeChannel(
std::shared_ptr<SLE> const& slep,
@@ -14,4 +27,27 @@ closeChannel(
uint256 const& key,
beast::Journal j);
/** Add two uint32_t values with saturation at UINT32_MAX.
*
* @param rules The current ledger rules used to check amendment status.
* @param lhs Left-hand operand.
* @param rhs Right-hand operand.
* @return @p lhs + @p rhs, saturated at UINT32_MAX when the amendment
* is active.
*/
uint32_t
saturatingAdd(Rules const& rules, uint32_t const lhs, uint32_t const rhs);
/** Determine whether a payment channel time field represents an expired time.
*
* @param view The apply view providing the parent close time and rules.
* @param timeField The optional expiry timestamp (seconds since the XRP
* Ledger epoch). If empty, the function returns false.
* @return @c true if @p timeField is set and the indicated time is
* in the past relative to the view's parent close time;
* @c false otherwise.
*/
bool
isChannelExpired(ApplyView const& view, std::optional<std::uint32_t> timeField);
} // namespace xrpl

View File

@@ -63,9 +63,15 @@ enum class AuthType { StrongAuth, WeakAuth, Legacy };
[[nodiscard]] bool
isGlobalFrozen(ReadView const& view, Asset const& asset);
[[nodiscard]] TER
checkGlobalFrozen(ReadView const& view, Asset const& asset);
[[nodiscard]] bool
isIndividualFrozen(ReadView const& view, AccountID const& account, Asset const& asset);
[[nodiscard]] TER
checkIndividualFrozen(ReadView const& view, AccountID const& account, Asset const& asset);
/**
* isFrozen check is recursive for MPT shares in a vault, descending to
* assets in the vault, up to maxAssetCheckDepth recursion depth. This is

View File

@@ -1,5 +1,7 @@
#pragma once
#include <xrpl/ledger/ReadView.h>
#include <xrpl/protocol/AccountID.h>
#include <xrpl/protocol/STAmount.h>
#include <xrpl/protocol/STLedgerEntry.h>
@@ -43,6 +45,14 @@ sharesToAssetsDeposit(
/** Controls whether to truncate shares instead of rounding. */
enum class TruncateShares : bool { No = false, Yes = true };
/** Controls whether the withdraw conversion helpers
(assetsToSharesWithdraw and sharesToAssetsWithdraw) subtract
sfLossUnrealized from sfAssetsTotal before computing the exchange rate.
The default (No) applies the standard discounted rate; Yes is used when
the redeemer is the sole remaining shareholder.
*/
enum class WaiveUnrealizedLoss : bool { No = false, Yes = true };
/** From the perspective of a vault, return the number of shares to demand from
the depositor when they ask to withdraw a fixed amount of assets. Since
shares are MPT this number is integral, and it will be rounded to nearest
@@ -52,6 +62,8 @@ enum class TruncateShares : bool { No = false, Yes = true };
@param issuance The MPTokenIssuance SLE for the vault's shares.
@param assets The amount of assets to convert.
@param truncate Whether to truncate instead of rounding.
@param waive Whether to waive the unrealized-loss discount when computing
the exchange rate.
@return The number of shares, or nullopt on error.
*/
@@ -60,7 +72,8 @@ assetsToSharesWithdraw(
std::shared_ptr<SLE const> const& vault,
std::shared_ptr<SLE const> const& issuance,
STAmount const& assets,
TruncateShares truncate = TruncateShares::No);
TruncateShares truncate = TruncateShares::No,
WaiveUnrealizedLoss waive = WaiveUnrealizedLoss::No);
/** From the perspective of a vault, return the number of assets to give the
depositor when they redeem a fixed amount of shares. Note, since shares are
@@ -69,6 +82,8 @@ assetsToSharesWithdraw(
@param vault The vault SLE.
@param issuance The MPTokenIssuance SLE for the vault's shares.
@param shares The amount of shares to convert.
@param waive Whether to waive (i.e. not subtract) the vault's unrealized
loss when computing the exchange rate.
@return The number of assets, or nullopt on error.
*/
@@ -76,6 +91,22 @@ assetsToSharesWithdraw(
sharesToAssetsWithdraw(
std::shared_ptr<SLE const> const& vault,
std::shared_ptr<SLE const> const& issuance,
STAmount const& shares);
STAmount const& shares,
WaiveUnrealizedLoss waive = WaiveUnrealizedLoss::No);
/** Returns true iff `account` holds all of the vault's outstanding shares —
i.e. is the sole remaining shareholder. Returns false if the account
holds no shares or fewer than the total outstanding.
@param view The ledger view.
@param account The candidate sole shareholder.
@param issuance The MPTokenIssuance SLE for the vault's shares; provides
both the share MPTID and the outstanding-amount total.
*/
[[nodiscard]] bool
isSoleShareholder(
ReadView const& view,
AccountID const& account,
std::shared_ptr<SLE const> const& issuance);
} // namespace xrpl

View File

@@ -83,10 +83,6 @@ public:
virtual Status
fetch(uint256 const& hash, std::shared_ptr<NodeObject>* pObject) = 0;
/** Fetch a batch synchronously. */
virtual std::pair<std::vector<std::shared_ptr<NodeObject>>, Status>
fetchBatch(std::vector<uint256> const& hashes) = 0;
/** Store a single object.
Depending on the implementation this may happen immediately
or deferred using a scheduled task.

View File

@@ -131,6 +131,10 @@ public:
std::uint32_t ledgerSeq,
std::function<void(std::shared_ptr<NodeObject> const&)>&& callback);
/** Remove expired entries from the positive and negative caches. */
virtual void
sweep() = 0;
/** Gather statistics pertaining to read and write activities.
*
* @param obj Json object reference into which to place counters.

View File

@@ -22,6 +22,32 @@ public:
beast::Journal j)
: Database(scheduler, readThreads, config, j), backend_(std::move(backend))
{
std::optional<int> cacheSize, cacheAge;
if (config.exists("cache_size"))
{
cacheSize = get<int>(config, "cache_size");
if (cacheSize.value() < 0)
Throw<std::runtime_error>("Specified negative value for cache_size");
}
if (config.exists("cache_age"))
{
cacheAge = get<int>(config, "cache_age");
if (cacheAge.value() < 0)
Throw<std::runtime_error>("Specified negative value for cache_age");
}
if (cacheSize.has_value() || cacheAge.has_value())
{
cache_ = std::make_shared<TaggedCache<uint256, NodeObject>>(
"DatabaseNodeImp",
cacheSize.value_or(0),
std::chrono::minutes(cacheAge.value_or(0)),
stopwatch(),
j);
}
XRPL_ASSERT(
backend_,
"xrpl::NodeStore::DatabaseNodeImp::DatabaseNodeImp : non-null "
@@ -67,16 +93,19 @@ public:
backend_->sync();
}
std::vector<std::shared_ptr<NodeObject>>
fetchBatch(std::vector<uint256> const& hashes);
void
asyncFetch(
uint256 const& hash,
std::uint32_t ledgerSeq,
std::function<void(std::shared_ptr<NodeObject> const&)>&& callback) override;
void
sweep() override;
private:
// Cache for database objects. This cache is not always initialized. Check
// for null before using.
std::shared_ptr<TaggedCache<uint256, NodeObject>> cache_;
// Persistent key/value storage
std::shared_ptr<Backend> backend_;

View File

@@ -55,6 +55,9 @@ public:
void
sync() override;
void
sweep() override;
private:
std::shared_ptr<Backend> writableBackend_;
std::shared_ptr<Backend> archiveBackend_;

View File

@@ -102,25 +102,32 @@ getAPIVersionNumber(json::Value const& jv, bool betaEnabled)
json::Value const maxVersion(
betaEnabled ? RPC::kApiBetaVersion : RPC::kApiMaximumSupportedVersion);
if (jv.isObject())
if (!jv.isObject() || !jv.isMember(jss::api_version))
return RPC::kApiVersionIfUnspecified;
try
{
if (jv.isMember(jss::api_version))
auto const& rawVersion = jv[jss::api_version];
switch (rawVersion.type())
{
auto const specifiedVersion = jv[jss::api_version];
if (!specifiedVersion.isInt() && !specifiedVersion.isUInt())
{
return RPC::kApiInvalidVersion;
case json::ValueType::Int:
if (rawVersion.asInt() < 0)
return RPC::kApiInvalidVersion;
[[fallthrough]];
case json::ValueType::UInt: {
auto const apiVersion = rawVersion.asUInt();
if (apiVersion < kMinVersion || apiVersion > maxVersion)
return RPC::kApiInvalidVersion;
return apiVersion;
}
auto const specifiedVersionInt = specifiedVersion.asInt();
if (specifiedVersionInt < kMinVersion || specifiedVersionInt > maxVersion)
{
default:
return RPC::kApiInvalidVersion;
}
return specifiedVersionInt;
}
}
return RPC::kApiVersionIfUnspecified;
catch (...)
{
return RPC::kApiInvalidVersion;
}
}
} // namespace RPC

View File

@@ -122,4 +122,17 @@ private:
std::optional<Rules> saved_;
};
class NumberSO;
class NumberMantissaScaleGuard;
bool
useRulesGuards(Rules const& rules);
void
createGuards(
Rules const& rules,
std::optional<NumberSO>& stNumberSO,
std::optional<CurrentTransactionRulesGuard>& rulesGuard,
std::optional<NumberMantissaScaleGuard>& mantissaScaleGuard);
} // namespace xrpl

View File

@@ -3,11 +3,13 @@
#include <xrpl/basics/CountedObject.h>
#include <xrpl/basics/LocalValue.h>
#include <xrpl/basics/Number.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/protocol/Asset.h>
#include <xrpl/protocol/IOUAmount.h>
#include <xrpl/protocol/Issue.h>
#include <xrpl/protocol/MPTAmount.h>
#include <xrpl/protocol/Protocol.h>
#include <xrpl/protocol/SField.h>
#include <xrpl/protocol/STBase.h>
#include <xrpl/protocol/Serializer.h>
@@ -187,7 +189,6 @@ public:
/**
* Checks if this amount evaluates to zero when constrained to a specific
* accounting scale.
*
* For XRP and MPT `roundToScale` is a no-op, returns true only when the amount itself is zero.
* The `scale` argument is ignored in that case.
* For IOU, the amount is rounded to the given scale using Number::RoundingMode::ToNearest mode
@@ -558,7 +559,7 @@ STAmount::fromNumber(A const& a, Number const& number)
return STAmount{asset, intValue, 0, negative};
}
auto const [mantissa, exponent] = working.normalizeToRange(kMinValue, kMaxValue);
auto const [mantissa, exponent] = working.normalizeToRange<kMinValue, kMaxValue>();
return STAmount{asset, mantissa, exponent, negative};
}
@@ -593,12 +594,25 @@ STAmount::value() const noexcept
return *this;
}
inline bool
[[nodiscard]] inline bool
isLegalNet(STAmount const& value)
{
return !value.native() || (value.mantissa() <= STAmount::kMaxNativeN);
}
[[nodiscard]] inline bool
isLegalMPT(STAmount const& value)
{
return !value.holds<MPTIssue>() ||
(!value.negative() && value.exponent() == 0 && value.mantissa() <= kMaxMpTokenAmount);
}
/* Check recursively if an object has invalid MPTAmount or XRPAmount in STAmount field.
* Calls isLegalNet() and isLegalMPT().
*/
[[nodiscard]] bool
hasInvalidAmount(STBase const& field, beast::Journal j);
//------------------------------------------------------------------------------
//
// Operators

View File

@@ -15,7 +15,7 @@
// Add new amendments to the top of this list.
// Keep it sorted in reverse chronological order.
XRPL_FIX (Cleanup3_2_0, Supported::No, VoteBehavior::DefaultNo)
XRPL_FIX (Cleanup3_2_0, Supported::Yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(MPTokensV2, Supported::No, VoteBehavior::DefaultNo)
XRPL_FIX (Cleanup3_1_3, Supported::Yes, VoteBehavior::DefaultYes)
XRPL_FIX (BatchInnerSigs, Supported::No, VoteBehavior::DefaultNo)

View File

@@ -15,8 +15,8 @@ Generation requires a one-time setup step to create a virtual environment
and install Python dependencies, followed by running the generation target:
```bash
cmake --build . --target setup_code_gen # create venv and install dependencies (once)
cmake --build . --target code_gen # generate code
cmake --build . --target setup_code_gen # create venv and install dependencies (once)
cmake --build . --target code_gen # generate code
```
By default, `CODEGEN_VENV_DIR` points to `.venv` in the project root. The

View File

@@ -1,6 +1,7 @@
#pragma once
#include <xrpl/basics/CountedObject.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/json/json_value.h>
#include <xrpl/protocol/Book.h>
#include <xrpl/protocol/ErrorCodes.h>
@@ -26,6 +27,19 @@ public:
};
/** Manages a client's subscription to data feeds.
*
* An InfoSub holds a non-owning reference to its `Source` (typically the
* process-wide `NetworkOPsImp`). The destructor reaches back into the
* `Source` to remove this subscriber from every server-side subscription
* map.
*
* @note Lifetime contract: every `InfoSub` instance MUST be destroyed
* before the backing `Source`. NetworkOPsImp shutdown drops all
* subscriber strong refs before its own teardown to satisfy this.
* @note Thread-safety: per-instance state is guarded by `lock_`. The
* destructor reads tracking sets without taking `lock_` because
* the strong-pointer ref-count is zero at destruction time, so
* no other thread can be calling the public mutators.
*/
class InfoSub : public CountedObject<InfoSub>
{
@@ -117,8 +131,43 @@ public:
virtual bool
subBook(ref ispListener, Book const&) = 0;
/**
* Remove a book subscription for a live subscriber.
*
* Clears the book from the subscriber's own tracking set
* (InfoSub::bookSubscriptions_) and then removes the server-side
* entry from subBook_. Call this from RPC unsubscribe handlers.
*
* @param ispListener The subscriber requesting removal.
* @param book The order book to unsubscribe from.
* @return true if the entry was present and removed, false if the
* subscriber was not subscribed to @p book.
*
* @note Thread-safety: acquires subLock_ internally.
* @note Do NOT call from ~InfoSub(). Use unsubBookInternal instead
* to avoid a redundant write-back to bookSubscriptions_ on a
* partially-destroyed object.
*/
virtual bool
unsubBook(std::uint64_t uListener, Book const&) = 0;
unsubBook(ref ispListener, Book const&) = 0;
/**
* Remove a book subscription during InfoSub teardown.
*
* Removes only the server-side entry from subBook_. Does NOT touch
* InfoSub::bookSubscriptions_ because the InfoSub is being destroyed.
* Called by ~InfoSub() for each book in bookSubscriptions_.
*
* @param uListener The sequence number of the subscriber being torn down.
* @param book The order book entry to remove.
* @return true if the entry was present and removed, false otherwise
* (e.g., already removed by a concurrent RPC unsubscribe).
*
* @note Thread-safety: acquires subLock_ internally.
*/
virtual bool
unsubBookInternal(std::uint64_t uListener, Book const&) = 0;
virtual bool
subTransactions(ref ispListener) = 0;
@@ -158,6 +207,13 @@ public:
addRpcSub(std::string const& strUrl, ref rspEntry) = 0;
virtual bool
tryRemoveRpcSub(std::string const& strUrl) = 0;
/** Journal used by InfoSub for diagnostics that occur after the
* owning subsystem (e.g. application-level Logs) is the only
* surviving sink — primarily destructor-time cleanup failures.
*/
[[nodiscard]] virtual beast::Journal const&
journal() const = 0;
};
public:
@@ -184,6 +240,31 @@ public:
void
deleteSubAccountInfo(AccountID const& account, bool rt);
/** Record that this subscriber is following @p book.
*
* Called by NetworkOPsImp::subBook so that ~InfoSub() can issue a
* matching unsubBook for every book this subscriber is tracking,
* keeping per-subscriber state symmetric with the server-side map.
*
* @param book The order book this subscriber has just subscribed to.
* @note Idempotent: re-inserting an already-tracked book is a no-op.
* @note Thread-safe: takes InfoSub::lock_.
*/
void
insertBookSubscription(Book const& book);
/** Stop tracking @p book for this subscriber.
*
* Called by the unsubscribe RPC handler so that the book is not
* re-unsubscribed by ~InfoSub(). Pairs with insertBookSubscription.
*
* @param book The order book to forget.
* @note No-op if @p book was not previously inserted.
* @note Thread-safe: takes InfoSub::lock_.
*/
void
deleteBookSubscription(Book const& book);
// return false if already subscribed to this account
bool
insertSubAccountHistory(AccountID const& account);
@@ -217,6 +298,7 @@ private:
std::shared_ptr<InfoSubRequest> request_;
std::uint64_t seq_;
hash_set<AccountID> accountHistorySubscriptions_;
hash_set<Book> bookSubscriptions_;
unsigned int apiVersion_ = 0;
static int

View File

@@ -249,6 +249,19 @@ public:
virtual void
stateAccounting(json::Value& obj) = 0;
/** Total number of (book, subscriber) entries currently tracked.
*
* Counts every weak_ptr stored across every book in subBook_, NOT the
* number of distinct subscribers and NOT the number of distinct
* books: a single subscriber following N books contributes N entries.
*
* @note Diagnostic accessor; intended for tests and operator visibility
* into per-book subscription state. The returned value is a
* snapshot under the subscription lock.
*/
virtual std::size_t
getBookSubscribersCount() = 0;
};
} // namespace xrpl

View File

@@ -23,7 +23,6 @@
#include <sys/resource.h>
#include <dirent.h>
#include <unistd.h>
#endif
#include <algorithm>
@@ -99,7 +98,10 @@ private:
static constexpr std::chrono::milliseconds kMaxAcceptDelay{2000};
std::chrono::milliseconds acceptDelay_{kInitialAcceptDelay};
boost::asio::steady_timer backoffTimer_;
static constexpr double kFreeFdThreshold = 0.70;
static constexpr std::uint64_t kMaxUsedFdPercent = 70;
static constexpr std::chrono::milliseconds kFdSampleInterval{250};
clock_type::time_point fdSampleAt_;
bool cachedThrottle_{false};
struct FDStats
{
@@ -280,6 +282,7 @@ Door<Handler>::Door(
, acceptor_(ioContext)
, strand_(boost::asio::make_strand(ioContext))
, backoffTimer_(ioContext)
, fdSampleAt_(clock_type::now() - kFdSampleInterval)
{
reOpen();
}
@@ -338,11 +341,11 @@ Door<Handler>::doAccept(boost::asio::yield_context doYield)
{
if (shouldThrottleForFds())
{
JLOG(j_.warn()) << "Throttling do_accept for " << acceptDelay_.count() << "ms.";
backoffTimer_.expires_after(acceptDelay_);
boost::system::error_code tec;
backoffTimer_.async_wait(doYield[tec]);
acceptDelay_ = std::min(acceptDelay_ * 2, kMaxAcceptDelay);
JLOG(j_.warn()) << "Throttling do_accept for " << acceptDelay_.count() << "ms.";
continue;
}
@@ -359,8 +362,11 @@ Door<Handler>::doAccept(boost::asio::yield_context doYield)
if (ec == boost::asio::error::no_descriptors ||
ec == boost::asio::error::no_buffer_space)
{
JLOG(j_.warn()) << "accept: Too many open files. Pausing for "
<< acceptDelay_.count() << "ms.";
char const* const cause = (ec == boost::asio::error::no_descriptors)
? "too many open files"
: "kernel buffer space exhausted";
JLOG(j_.warn()) << "accept: " << cause << ". Pausing for " << acceptDelay_.count()
<< "ms.";
backoffTimer_.expires_after(acceptDelay_);
boost::system::error_code tec;
@@ -428,14 +434,15 @@ Door<Handler>::shouldThrottleForFds()
#if BOOST_OS_WINDOWS
return false;
#else
auto const stats = queryFdStats();
if (!stats || stats->limit == 0)
return false;
auto const now = clock_type::now();
if (now - fdSampleAt_ < kFdSampleInterval)
return cachedThrottle_;
auto const& s = *stats;
auto const free = (s.limit > s.used) ? (s.limit - s.used) : 0ull;
double const freeRatio = static_cast<double>(free) / static_cast<double>(s.limit);
return freeRatio < kFreeFdThreshold;
fdSampleAt_ = now;
auto const stats = queryFdStats();
cachedThrottle_ =
stats && stats->limit > 0 && stats->used * 100 > stats->limit * kMaxUsedFdPercent;
return cachedThrottle_;
#endif
}

View File

@@ -398,6 +398,15 @@ private:
static NotTEC
preflight2(PreflightContext const& ctx);
/** Universal validations
- Valid MPTAmount and XRPAmount
Do not try to call preflightUniversal from preflight() in derived classes. See
the description of invokePreflight for details.
*/
static NotTEC
preflightUniversal(PreflightContext const& ctx);
/** Check transaction-specific invariants only.
*
* Walks every modified ledger entry via visitInvariantEntry, then
@@ -463,6 +472,9 @@ Transactor::invokePreflight(PreflightContext const& ctx)
if (auto const ret = preflight1(ctx, T::getFlagsMask(ctx)))
return ret;
if (auto const ret = preflightUniversal(ctx))
return ret;
if (auto const ret = T::preflight(ctx))
return ret;

View File

@@ -373,6 +373,21 @@ public:
finalize(STTx const&, TER const, XRPAmount const, ReadView const&, beast::Journal const&);
};
/** Verify that MPT/XRP STAmounts are canonical in any ledger entries left after the
* transaction applies.
*/
class ValidAmounts
{
std::vector<std::shared_ptr<SLE const>> afterEntries_;
public:
void
visitEntry(bool, std::shared_ptr<SLE const> const&, std::shared_ptr<SLE const> const&);
[[nodiscard]] bool
finalize(STTx const&, TER const, XRPAmount const, ReadView const&, beast::Journal const&) const;
};
// additional invariant checks can be declared above and then added to this
// tuple
using InvariantChecks = std::tuple<
@@ -401,7 +416,9 @@ using InvariantChecks = std::tuple<
ValidLoanBroker,
ValidLoan,
ValidVault,
ValidMPTPayment>;
ValidMPTPayment,
ValidAmounts,
ValidMPTTransfer>;
/**
* @brief get a tuple of all invariant checks

View File

@@ -71,4 +71,42 @@ public:
finalize(STTx const&, TER const, XRPAmount const, ReadView const&, beast::Journal const&);
};
class ValidMPTTransfer
{
struct Value
{
std::optional<std::uint64_t> amtBefore;
std::optional<std::uint64_t> amtAfter;
};
// MPTID: {holder: Value}
hash_map<uint192, hash_map<AccountID, Value>> amount_;
// Deleted MPToken
// MPToken key: true if MPTAuthorized is set
hash_map<uint256, bool> deletedAuthorized_;
public:
void
visitEntry(bool, std::shared_ptr<SLE const> const&, std::shared_ptr<SLE const> const&);
bool
finalize(STTx const&, TER const, XRPAmount const, ReadView const&, beast::Journal const&);
private:
/**
* @brief Check whether a holder is authorized to send or receive an MPToken.
*
* Deleted MPToken SLEs are no longer present in the view by the time
* finalize() runs, so their authorization state is captured during
* visitEntry() and stored in deletedAuthorized_. For deleted MPTokens,
* returns true if reqAuth is false or lsfMPTAuthorized was set at deletion.
* For existing MPTokens, returns the result of requireAuth()
*/
[[nodiscard]] bool
isAuthorized(
ReadView const& view,
MPTID const& mptid,
AccountID const& holder,
bool requireAuth) const;
};
} // namespace xrpl

View File

@@ -4,9 +4,11 @@
#include <xrpl/basics/base_uint.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/ledger/ReadView.h>
#include <xrpl/protocol/AccountID.h>
#include <xrpl/protocol/MPTIssue.h>
#include <xrpl/protocol/STTx.h>
#include <xrpl/protocol/TER.h>
#include <xrpl/protocol/XRPAmount.h>
#include <optional>
#include <unordered_map>
@@ -79,16 +81,83 @@ private:
std::vector<Shares> beforeMPTs_;
std::unordered_map<uint256, DeltaInfo> deltas_;
/**
* @brief Compute the minimum STAmount scale for rounding invariant
* calculations.
*
* Post-amendment (@c fixCleanup3_2_0) this is simply the posterior
* @c assetsTotal scale. Pre-amendment it is the coarsest scale across
* @p vaultDelta and both asset-field deltas.
*
* @param vaultDelta Delta of the vault's asset balance for this transaction.
* @param rules Active ledger rules (used to check the amendment).
* @returns The minimum scale to apply when rounding vault-related amounts.
*/
[[nodiscard]] std::int32_t
computeVaultMinScale(DeltaInfo const& vaultDelta, Rules const& rules) const;
/**
* @brief Return the vault-asset balance-change delta for an account.
*
* Looks up the ledger-entry delta recorded during @c visitEntry for the
* account entry (XRP), trust line (IOU), or MPToken (MPT) that corresponds
* to the vault asset held by @p id.
*
* @param id Account whose asset delta is requested.
* @returns The delta, or @c std::nullopt if the entry was not touched.
*/
[[nodiscard]] std::optional<DeltaInfo>
deltaAssets(AccountID const& id) const;
/**
* @brief Return the vault-asset delta for the transaction's sending
* account, adjusted for the fee.
*
* Calls @c deltaAssets for @c tx[sfAccount] and, for non-delegated XRP
* transactions, adds the consumed fee back so the invariant sees the net
* asset movement rather than the fee-reduced balance change.
*
* @param tx The transaction being applied.
* @param fee Fee charged by this transaction.
* @returns The fee-adjusted delta, or @c std::nullopt if the net delta is
* zero or the account entry was not touched.
*/
[[nodiscard]] std::optional<DeltaInfo>
deltaAssetsTxAccount(STTx const& tx, XRPAmount fee) const;
/**
* @brief Return the vault-share balance-change delta for an account.
*
* For the vault's pseudo-account the @c MPTokenIssuance outstanding-amount
* delta is returned; for all other accounts the @c MPToken delta is
* returned.
*
* @param id Account whose share delta is requested.
* @returns The delta, or @c std::nullopt if the entry was not touched.
*/
[[nodiscard]] std::optional<DeltaInfo>
deltaShares(AccountID const& id) const;
/**
* @brief Check whether a vault holds no assets.
*
* @param vault Snapshot of the vault to test.
* @returns @c true when both @c assetsAvailable and @c assetsTotal are
* zero.
*/
[[nodiscard]] static bool
isVaultEmpty(Vault const& vault);
public:
// Compute the coarsest scale required to represent all numbers
[[nodiscard]] static std::int32_t
computeCoarsestScale(std::vector<DeltaInfo> const& numbers);
void
visitEntry(bool, std::shared_ptr<SLE const> const&, std::shared_ptr<SLE const> const&);
bool
finalize(STTx const&, TER const, XRPAmount const, ReadView const&, beast::Journal const&);
// Compute the coarsest scale required to represent all numbers
[[nodiscard]] static std::int32_t
computeCoarsestScale(std::vector<DeltaInfo> const& numbers);
};
} // namespace xrpl

View File

@@ -16,6 +16,9 @@ public:
static TxConsequences
makeTxConsequences(PreflightContext const& ctx);
static bool
checkExtraFeatures(PreflightContext const& ctx);
static NotTEC
preflight(PreflightContext const& ctx);

View File

@@ -1,39 +1,102 @@
{
pkgs,
glibc231,
customGlibc,
...
}:
let
inherit (import ./packages.nix { inherit pkgs; }) commonPackages;
inherit (pkgs) lib;
# binutils wrapped to emit binaries that reference glibc 2.31 (dynamic
# linker path, library search path, RPATH).
binutils231 = pkgs.wrapBintoolsWith {
# Underlying compiler toolchains to wrap. Bump these in one place to
# roll the whole environment forward.
customGccPackage = pkgs.gcc15;
customLlvmPackages = pkgs.llvmPackages_22;
customClangMajor = lib.versions.major (lib.getVersion customLlvmPackages.clang-unwrapped);
# binutils wrapped to emit binaries that reference the custom glibc
# (dynamic linker path, library search path, RPATH).
customBinutils = pkgs.wrapBintoolsWith {
bintools = pkgs.binutils-unwrapped;
libc = glibc231;
libc = customGlibc;
};
# Rebuild gcc 15 (specifically libstdc++ / libgcc_s) against glibc 2.31.
# The override swaps gcc15.cc's bootstrap stdenv for one that uses the
# existing gcc 15 binary but links against glibc 2.31, so the resulting
# compiler ships runtime libraries that only reference symbols available
# in glibc 2.31.
gcc15CcWithGlibc231 = pkgs.gcc15.cc.override {
# Rebuild gcc (specifically libstdc++ / libgcc_s) against the custom
# glibc. The override swaps gcc.cc's bootstrap stdenv for one that uses
# the existing gcc binary but links against the custom glibc, so the
# resulting compiler ships runtime libraries that only reference symbols
# available in that glibc.
customGccCc = customGccPackage.cc.override {
stdenv = pkgs.stdenvAdapters.overrideCC pkgs.stdenv (
pkgs.wrapCCWith {
cc = pkgs.gcc15.cc;
libc = glibc231;
bintools = binutils231;
cc = customGccPackage.cc;
libc = customGlibc;
bintools = customBinutils;
}
);
};
# cc-wrapper around the rebuilt compiler, pointing at glibc 2.31 headers
# and libraries. This is what we actually expose to users.
gcc15WithGlibc231 = pkgs.wrapCCWith {
cc = gcc15CcWithGlibc231;
libc = glibc231;
bintools = binutils231;
# cc-wrapper around the rebuilt compiler, pointing at the custom glibc
# headers and libraries. This is what we actually expose to users.
customGcc = pkgs.wrapCCWith {
cc = customGccCc;
libc = customGlibc;
bintools = customBinutils;
};
# stdenv built around the rebuilt gcc / custom glibc. Used to rebuild
# compiler-rt below so its sanitizer runtimes see the custom glibc
# headers.
customStdenv = pkgs.stdenvAdapters.overrideCC pkgs.stdenv customGcc;
# Rebuild compiler-rt against the custom glibc so the sanitizer runtimes
# don't use glibc symbols (or sysconf constants like _SC_SIGSTKSZ) that
# only exist in newer glibc versions. scudo is dropped because its CMake
# includes CheckAtomic with -nostdinc++ in CMAKE_REQUIRED_FLAGS, which
# makes std::atomic unfindable in our stdenv; we don't use scudo (only
# asan/ubsan/tsan etc.).
customCompilerRt =
(customLlvmPackages.compiler-rt.override {
stdenv = customStdenv;
}).overrideAttrs
(old: {
postPatch = (old.postPatch or "") + ''
substituteInPlace lib/CMakeLists.txt \
--replace-quiet 'add_subdirectory(scudo/standalone)' \
'# scudo/standalone disabled in xrpld ci-env'
'';
});
# cc-wrapper around clang, pointing at the custom glibc headers and
# libraries. Reuses the rebuilt gcc for libstdc++ / libgcc_s so that
# C++ binaries produced by clang also only reference symbols available
# in the custom glibc. compiler-rt is wired into a resource-root so
# sanitizer runtimes (libclang_rt.*.a) are found at link time; this
# mirrors what nixpkgs does internally when building llvmPackages.clang.
customClang = pkgs.wrapCCWith {
cc = customLlvmPackages.clang-unwrapped;
libc = customGlibc;
bintools = customBinutils;
gccForLibs = customGccCc;
extraPackages = [ customCompilerRt ];
extraBuildCommands = ''
rsrc="$out/resource-root"
mkdir "$rsrc"
ln -s "${customLlvmPackages.clang-unwrapped.lib}/lib/clang/${customClangMajor}/include" "$rsrc/include"
ln -s "${customCompilerRt.out}/lib" "$rsrc/lib"
ln -s "${customCompilerRt.out}/share" "$rsrc/share" || true
echo "-resource-dir=$rsrc" >> $out/nix-support/cc-cflags
'';
};
# Strip the generic cc/c++/cpp symlinks from the clang wrapper so it can
# coexist with the gcc wrapper in buildEnv. gcc remains the default
# compiler (cc/c++/cpp); clang is invoked explicitly as clang/clang++.
customClangForCiEnv = pkgs.symlinkJoin {
name = "clang-wrapper-custom-for-ci-env";
paths = [ customClang ];
postBuild = ''
rm -f $out/bin/cc $out/bin/c++ $out/bin/cpp
'';
};
in
@@ -41,8 +104,9 @@ in
default = pkgs.buildEnv {
name = "xrpld-ci-env";
paths = commonPackages ++ [
gcc15WithGlibc231
binutils231
customGcc
customClangForCiEnv
customBinutils
];
pathsToLink = [
"/bin"

View File

@@ -17,6 +17,7 @@ in
llvmPackages_22.clang-tools
mold
ninja
patchelf
perl # needed for openssl
pkg-config
pre-commit

View File

@@ -1,4 +1,4 @@
{ nixpkgs, nixpkgs-glibc231 }:
{ nixpkgs, nixpkgs-custom-glibc }:
function:
nixpkgs.lib.genAttrs
[
@@ -12,10 +12,10 @@ nixpkgs.lib.genAttrs
function {
pkgs = import nixpkgs { inherit system; };
# glibc 2.31 — matches the system libc on Ubuntu 20.04 LTS. Sourced
# from the nixpkgs snapshot pinned via the `nixpkgs-glibc231` flake
# input, so the build uses the compiler from that snapshot
# from the nixpkgs snapshot pinned via the `nixpkgs-custom-glibc`
# flake input, so the build uses the compiler from that snapshot
# (gcc 9.3.0) along with the matching patches, configure flags, and
# hardening defaults.
glibc231 = (import nixpkgs-glibc231 { inherit system; }).glibc;
customGlibc = (import nixpkgs-custom-glibc { inherit system; }).glibc;
}
)

View File

@@ -15,7 +15,6 @@ package/
xrpld.sysusers sysusers.d config (used by both RPM and DEB)
xrpld.tmpfiles tmpfiles.d config (used by both RPM and DEB)
xrpld.logrotate logrotate config (installed to /etc/logrotate.d/xrpld)
update-xrpld auto-update script (installed to /usr/libexec/xrpld/, run by update-xrpld.timer)
```
## Prerequisites
@@ -74,10 +73,10 @@ VERSION=2.4.0-local
PKG_RELEASE=1
docker run --rm \
-v "$(pwd):/src" \
-w /src \
"$IMAGE" \
./package/build_pkg.sh --pkg-version "$VERSION" --pkg-release "$PKG_RELEASE"
-v "$(pwd):/src" \
-w /src \
"$IMAGE" \
./package/build_pkg.sh --pkg-version "$VERSION" --pkg-release "$PKG_RELEASE"
# Output:
# build/debbuild/*.deb (DEB + dbgsym .ddeb)
@@ -92,12 +91,12 @@ needed, but the host toolchain replaces the pinned CI image:
```bash
cmake \
-Dxrpld=ON \
-Dxrpld_version=2.4.0-local \
-Dtests=OFF \
..
-Dxrpld=ON \
-Dxrpld_version=2.4.0-local \
-Dtests=OFF \
..
cmake --build . --target package # deb on Debian/Ubuntu, rpm on RHEL
cmake --build . --target package # deb on Debian/Ubuntu, rpm on RHEL
```
The `cmake/XrplPackaging.cmake` module defines the target only if at least one

View File

@@ -36,12 +36,35 @@ SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-}"
while [[ $# -gt 0 ]]; do
case "$1" in
--src-dir) need_arg "$@"; SRC_DIR="$2"; shift 2 ;;
--build-dir) need_arg "$@"; BUILD_DIR="$2"; shift 2 ;;
--pkg-version) need_arg "$@"; PKG_VERSION="$2"; shift 2 ;;
--pkg-release) need_arg "$@"; PKG_RELEASE="$2"; shift 2 ;;
--source-date-epoch) need_arg "$@"; SOURCE_DATE_EPOCH="$2"; shift 2 ;;
-h|--help) usage; exit 0 ;;
--src-dir)
need_arg "$@"
SRC_DIR="$2"
shift 2
;;
--build-dir)
need_arg "$@"
BUILD_DIR="$2"
shift 2
;;
--pkg-version)
need_arg "$@"
PKG_VERSION="$2"
shift 2
;;
--pkg-release)
need_arg "$@"
PKG_RELEASE="$2"
shift 2
;;
--source-date-epoch)
need_arg "$@"
SOURCE_DATE_EPOCH="$2"
shift 2
;;
-h | --help)
usage
exit 0
;;
*)
echo "Unknown argument: $1" >&2
usage >&2
@@ -91,10 +114,11 @@ VER_BASE="${VERSION%%-*}"
VER_SUFFIX="${VERSION#*-}"
[[ "${VER_SUFFIX}" == "${VERSION}" ]] && VER_SUFFIX=""
# Reject multi-segment suffixes (e.g. "beta-1", "rc1-15-gabc123"). The RPM
# Release field forbids '-', and the convention here is single-token suffixes
# like b1 or rc2. Fail early with a clear message rather than letting either
# rpmbuild blow up or silently mangling dashes into dots.
# Reject multi-segment suffixes (e.g. "beta-1", "rc1-15-gabc123"). Neither an
# RPM Version nor a Debian upstream version may contain '-' (it's the NVR /
# version-revision separator), and the convention here is single-token
# suffixes like b1 or rc2. Fail early with a clear message rather than letting
# the package tooling blow up or silently mangle dashes.
if [[ "${VER_SUFFIX}" == *-* ]]; then
echo "build_pkg.sh: multi-segment pre-release in VERSION='${VERSION}' (suffix '${VER_SUFFIX}')." >&2
echo "Use single-token suffixes like 3.2.0-b1 or 3.2.0-rc2." >&2
@@ -109,20 +133,17 @@ stage_common() {
local dest="$1"
mkdir -p "${dest}"
cp "${BUILD_DIR}/xrpld" "${dest}/xrpld"
cp "${SRC_DIR}/cfg/xrpld-example.cfg" "${dest}/xrpld.cfg"
cp "${SRC_DIR}/cfg/validators-example.txt" "${dest}/validators.txt"
cp "${SRC_DIR}/LICENSE.md" "${dest}/LICENSE.md"
cp "${SRC_DIR}/README.md" "${dest}/README.md"
cp "${BUILD_DIR}/xrpld" "${dest}/xrpld"
cp "${SRC_DIR}/cfg/xrpld-example.cfg" "${dest}/xrpld.cfg"
cp "${SRC_DIR}/cfg/validators-example.txt" "${dest}/validators.txt"
cp "${SRC_DIR}/LICENSE.md" "${dest}/LICENSE.md"
cp "${SRC_DIR}/README.md" "${dest}/README.md"
cp "${SHARED}/xrpld.service" "${dest}/xrpld.service"
cp "${SHARED}/xrpld.sysusers" "${dest}/xrpld.sysusers"
cp "${SHARED}/xrpld.tmpfiles" "${dest}/xrpld.tmpfiles"
cp "${SHARED}/xrpld.logrotate" "${dest}/xrpld.logrotate"
cp "${SHARED}/update-xrpld" "${dest}/update-xrpld"
cp "${SHARED}/update-xrpld.service" "${dest}/update-xrpld.service"
cp "${SHARED}/update-xrpld.timer" "${dest}/update-xrpld.timer"
cp "${SHARED}/50-xrpld.preset" "${dest}/50-xrpld.preset"
cp "${SHARED}/xrpld.service" "${dest}/xrpld.service"
cp "${SHARED}/xrpld.sysusers" "${dest}/xrpld.sysusers"
cp "${SHARED}/xrpld.tmpfiles" "${dest}/xrpld.tmpfiles"
cp "${SHARED}/xrpld.logrotate" "${dest}/xrpld.logrotate"
cp "${SHARED}/50-xrpld.preset" "${dest}/50-xrpld.preset"
}
build_rpm() {
@@ -133,16 +154,18 @@ build_rpm() {
cp "${SRC_DIR}/package/rpm/xrpld.spec" "${topdir}/SPECS/xrpld.spec"
stage_common "${topdir}/SOURCES"
# RPM Version can't contain '-'. A pre-release goes in Release with a
# leading "0." so 3.2.0-b1 sorts before the final 3.2.0-<pkg_release>.
local rpm_release="${PKG_RELEASE}"
[[ -n "${VER_SUFFIX}" ]] && rpm_release="0.${VER_SUFFIX}.${PKG_RELEASE}"
# Pre-releases use the modern rpm '~' convention (rpm >= 4.10): the suffix
# goes in Version (e.g. 3.2.0~b1), which rpmvercmp sorts *before* the final
# 3.2.0 — identical semantics to Debian's '~'. Release is just the package
# release number. This replaces the older "0.<release>.<suffix>" Release
# hack and keeps the RPM and DEB version strings symmetric.
local rpm_version="${VER_BASE}${VER_SUFFIX:+~${VER_SUFFIX}}"
set -x
rpmbuild -bb \
--define "_topdir ${topdir}" \
--define "xrpld_version ${VER_BASE}" \
--define "xrpld_release ${rpm_release}" \
--define "xrpld_version ${rpm_version}" \
--define "xrpld_release ${PKG_RELEASE}" \
"${topdir}/SPECS/xrpld.spec"
}
@@ -154,13 +177,10 @@ build_deb() {
stage_common "${staging}"
cp -r "${DEBIAN_DIR}" "${staging}/debian"
# Debhelper auto-discovers these only from debian/.
cp "${staging}/xrpld.service" "${staging}/debian/xrpld.service"
cp "${staging}/xrpld.sysusers" "${staging}/debian/xrpld.sysusers"
cp "${staging}/xrpld.tmpfiles" "${staging}/debian/xrpld.tmpfiles"
cp "${staging}/xrpld.logrotate" "${staging}/debian/xrpld.logrotate"
cp "${staging}/update-xrpld.service" "${staging}/debian/xrpld.update-xrpld.service"
cp "${staging}/update-xrpld.timer" "${staging}/debian/xrpld.update-xrpld.timer"
cp "${staging}/xrpld.service" "${staging}/debian/xrpld.service"
cp "${staging}/xrpld.sysusers" "${staging}/debian/xrpld.sysusers"
cp "${staging}/xrpld.tmpfiles" "${staging}/debian/xrpld.tmpfiles"
cp "${staging}/xrpld.logrotate" "${staging}/debian/xrpld.logrotate"
# Debian '~' marks a pre-release; 3.2.0~b1 sorts before 3.2.0.
local deb_full_version="${VER_BASE}${VER_SUFFIX:+~${VER_SUFFIX}}-${PKG_RELEASE}"
@@ -171,12 +191,12 @@ build_deb() {
# b<N>, rc<N> -> unstable (pre-release)
local deb_distribution
case "${VER_SUFFIX}" in
"") deb_distribution="stable" ;;
b0) deb_distribution="develop" ;;
*) deb_distribution="unstable" ;;
"") deb_distribution="stable" ;;
b0) deb_distribution="develop" ;;
*) deb_distribution="unstable" ;;
esac
cat > "${staging}/debian/changelog" <<EOF
cat >"${staging}/debian/changelog" <<EOF
xrpld (${deb_full_version}) ${deb_distribution}; urgency=medium
* Release ${VERSION}.
@@ -186,7 +206,7 @@ EOF
chmod +x "${staging}/debian/rules"
set -x
( cd "${staging}" && dpkg-buildpackage -b --no-sign -d )
(cd "${staging}" && dpkg-buildpackage -b --no-sign -d)
}
"build_${pkg_type}"

View File

@@ -9,8 +9,7 @@ override_dh_auto_configure override_dh_auto_build override_dh_auto_test:
@:
override_dh_installsystemd:
dh_installsystemd --no-start xrpld.service
dh_installsystemd --name=update-xrpld --no-start update-xrpld.service update-xrpld.timer
dh_installsystemd --no-stop-on-upgrade xrpld.service
execute_before_dh_installtmpfiles:
dh_installsysusers
@@ -21,7 +20,6 @@ override_dh_install:
install -D -m 0755 xrpld debian/xrpld/usr/bin/xrpld
install -D -m 0644 xrpld.cfg debian/xrpld/etc/xrpld/xrpld.cfg
install -D -m 0644 validators.txt debian/xrpld/etc/xrpld/validators.txt
install -D -m 0755 update-xrpld debian/xrpld/usr/libexec/xrpld/update-xrpld
override_dh_dwz:
@:

View File

@@ -1,2 +1 @@
README.md
LICENSE.md

View File

@@ -35,8 +35,6 @@ install -Dm0644 %{_sourcedir}/validators.txt %{buildroot}%{_sysconfdir}/%{
# systemd units, sysusers, tmpfiles, preset
install -Dm0644 %{_sourcedir}/xrpld.service %{buildroot}%{_unitdir}/xrpld.service
install -Dm0644 %{_sourcedir}/update-xrpld.service %{buildroot}%{_unitdir}/update-xrpld.service
install -Dm0644 %{_sourcedir}/update-xrpld.timer %{buildroot}%{_unitdir}/update-xrpld.timer
install -Dm0644 %{_sourcedir}/xrpld.sysusers %{buildroot}%{_sysusersdir}/xrpld.conf
install -Dm0644 %{_sourcedir}/xrpld.tmpfiles %{buildroot}%{_tmpfilesdir}/xrpld.conf
install -Dm0644 %{_sourcedir}/50-xrpld.preset %{buildroot}%{_presetdir}/50-xrpld.preset
@@ -44,9 +42,6 @@ install -Dm0644 %{_sourcedir}/50-xrpld.preset %{buildroot}%{_presetdir}/50-
# Logrotate config
install -Dm0644 %{_sourcedir}/xrpld.logrotate %{buildroot}%{_sysconfdir}/logrotate.d/%{name}
# Update helper
install -Dm0755 %{_sourcedir}/update-xrpld %{buildroot}%{_libexecdir}/%{name}/update-xrpld
# Docs
install -Dm0644 %{_sourcedir}/LICENSE.md %{buildroot}%{_docdir}/%{name}/LICENSE.md
install -Dm0644 %{_sourcedir}/README.md %{buildroot}%{_docdir}/%{name}/README.md
@@ -61,10 +56,10 @@ ln -s %{_bindir}/%{name} %{buildroot}/usr/local/bin/rippled
%post
systemd-tmpfiles --create %{_tmpfilesdir}/xrpld.conf || :
%systemd_post xrpld.service update-xrpld.timer
%systemd_post xrpld.service
%preun
%systemd_preun xrpld.service update-xrpld.timer
%systemd_preun xrpld.service
%postun
%systemd_postun_with_restart xrpld.service
@@ -74,7 +69,6 @@ systemd-tmpfiles --create %{_tmpfilesdir}/xrpld.conf || :
%doc %{_docdir}/%{name}/README.md
%dir %{_sysconfdir}/%{name}
%dir %{_libexecdir}/%{name}
%{_bindir}/%{name}
@@ -82,18 +76,13 @@ systemd-tmpfiles --create %{_tmpfilesdir}/xrpld.conf || :
%config(noreplace) %{_sysconfdir}/%{name}/validators.txt
%config(noreplace) %{_sysconfdir}/logrotate.d/%{name}
%{_libexecdir}/%{name}/update-xrpld
%{_unitdir}/xrpld.service
%{_unitdir}/update-xrpld.service
%{_unitdir}/update-xrpld.timer
%{_presetdir}/50-xrpld.preset
%{_sysusersdir}/xrpld.conf
%{_tmpfilesdir}/xrpld.conf
%ghost %dir /var/lib/%{name}
%ghost %dir /var/log/%{name}
%ghost %dir /var/lib/xrpld
%ghost %dir /var/log/xrpld
# Legacy compatibility for pre-FHS package layouts.
# TODO: remove after rippled fully deprecated.

View File

@@ -1,4 +1,2 @@
# /usr/lib/systemd/system-preset/50-xrpld.preset
enable xrpld.service
# Don't enable automatic updates
disable update-xrpld.timer

View File

@@ -1,152 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# Optional: also write logs to a legacy file in addition to journald.
# By default, this script logs to systemd/journald, viewable via:
# journalctl -t update-xrpld
#
# Uncomment the line below if you need a flat file for compatibility with
# external tooling, manual inspection, or environments where journald logs
# are not persisted or easily accessible.
#
# Note: This duplicates all output (stdout/stderr) to both journald and the file.
# It is generally not needed on modern systems and may cause log file growth
# if left enabled long-term.
#
# Requires /var/log/xrpld/ to exist and be writable by the service (root).
#
# exec > >(tee -a /var/log/xrpld/update.log) 2>&1
PATH=/usr/sbin:/usr/bin:/sbin:/bin
PKG_NAME=${PKG_NAME:-xrpld}
log() {
# If running under systemd/journald, let it handle timestamps.
if [[ -n "${JOURNAL_STREAM:-}" ]]; then
printf '%s\n' "$*"
else
printf '%s %s\n' "$(date -u +'%Y-%m-%dT%H:%M:%SZ')" "$*"
fi
}
require_root() {
if [[ ${EUID:-$(id -u)} -ne 0 ]]; then
log "RESULT: failed reason=not-root"
exit 1
fi
}
get_installed_version() {
if command -v dpkg-query >/dev/null 2>&1; then
dpkg-query -W -f='${Version}' "$PKG_NAME" 2>/dev/null || printf 'unknown'
elif command -v rpm >/dev/null 2>&1; then
rpm -q --qf '%{VERSION}-%{RELEASE}' "$PKG_NAME" 2>/dev/null || printf 'unknown'
else
printf 'unknown'
fi
}
trap 'log "RESULT: failed reason=script-error exit_code=$?"' ERR
apt_can_update() {
apt-get update -qq
apt-get -s --only-upgrade install "$PKG_NAME" 2>/dev/null | grep -q "^Inst ${PKG_NAME}\b"
}
apt_apply_update() {
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq \
-o Dpkg::Options::="--force-confdef" \
-o Dpkg::Options::="--force-confold" \
"$PKG_NAME"
}
get_rpm_pm() {
if command -v dnf >/dev/null 2>&1; then
printf 'dnf\n'
elif command -v yum >/dev/null 2>&1; then
printf 'yum\n'
else
return 1
fi
}
rpm_refresh_metadata() {
local pm=$1
if [[ "$pm" == "dnf" ]]; then
dnf makecache --refresh -q >/dev/null
else
yum clean expire-cache -q >/dev/null
fi
}
rpm_can_update() {
local pm=$1
rpm_refresh_metadata "$pm"
local rc=0
set +e
"$pm" check-update -q "$PKG_NAME" >/dev/null 2>&1
rc=$?
set -e
if [[ $rc -eq 100 ]]; then
return 0
elif [[ $rc -eq 0 ]]; then
return 1
else
log "$pm check-update failed with exit code ${rc}."
exit 1
fi
}
rpm_apply_update() {
local pm=$1
"$pm" update -y "$PKG_NAME"
}
restart_service() {
# Preserve the operator's prior service state: if xrpld was intentionally
# stopped before the update, don't bring it back up just because the
# auto-update timer fired.
if systemctl is-active --quiet "${PKG_NAME}.service"; then
systemctl restart "${PKG_NAME}.service"
log "${PKG_NAME} service restarted successfully."
else
log "${PKG_NAME} service was not running; skipping restart to preserve prior state."
fi
}
main() {
require_root
if command -v apt-get >/dev/null 2>&1; then
log "Checking for ${PKG_NAME} updates via apt"
if apt_can_update; then
log "Update available; installing."
apt_apply_update
restart_service
log "RESULT: updated ${PKG_NAME}=$(get_installed_version)"
else
log "RESULT: no-update ${PKG_NAME}=$(get_installed_version)"
fi
return
fi
local rpm_pm=""
if rpm_pm="$(get_rpm_pm)"; then
log "Checking for ${PKG_NAME} updates via ${rpm_pm}"
if rpm_can_update "$rpm_pm"; then
log "Update available; installing"
rpm_apply_update "$rpm_pm"
restart_service
log "RESULT: updated ${PKG_NAME}=$(get_installed_version)"
else
log "RESULT: no-update ${PKG_NAME}=$(get_installed_version)"
fi
return
fi
log "RESULT: failed reason=no-package-manager"
exit 1
}
main "$@"

View File

@@ -1,16 +0,0 @@
[Unit]
Description=Check for and install xrpld package updates
Documentation=man:systemd.service(5)
Wants=network-online.target
After=network-online.target
ConditionPathExists=/usr/libexec/xrpld/update-xrpld
ConditionPathExists=/usr/bin/xrpld
[Service]
Type=oneshot
ExecStart=/usr/bin/flock -n /run/lock/xrpld-update.lock /usr/libexec/xrpld/update-xrpld
StandardOutput=journal
StandardError=journal
SyslogIdentifier=update-xrpld
TimeoutStartSec=30min
PrivateTmp=true

View File

@@ -1,10 +0,0 @@
[Unit]
Description=Daily xrpld update check
[Timer]
OnCalendar=*-*-* 00:00:00
RandomizedDelaySec=24h
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -2,14 +2,15 @@
Description=XRP Ledger Daemon
After=network-online.target
Wants=network-online.target
StartLimitIntervalSec=300
StartLimitIntervalSec=5min
StartLimitBurst=5
[Service]
Type=simple
ExecStart=/usr/bin/xrpld --net --silent --conf /etc/xrpld/xrpld.cfg
Restart=always
Restart=on-failure
RestartSec=5s
TimeoutStopSec=5min
NoNewPrivileges=true
ProtectSystem=full
ProtectHome=true
@@ -17,6 +18,11 @@ PrivateTmp=true
User=xrpld
Group=xrpld
LimitNOFILE=65536
SystemCallArchitectures=native
# Uncomment both lines to allow xrpld to bind to privileged ports (<1024)
#CapabilityBoundingSet=CAP_NET_BIND_SERVICE
#AmbientCapabilities=CAP_NET_BIND_SERVICE
[Install]
WantedBy=multi-user.target

View File

@@ -10,9 +10,11 @@
#include <iterator>
#include <limits>
#include <numeric>
#include <set>
#include <stdexcept>
#include <string>
#include <type_traits>
#include <unordered_map>
#include <utility>
#ifdef _MSC_VER
@@ -28,7 +30,76 @@ using int128_t = __int128_t;
namespace xrpl {
thread_local Number::RoundingMode Number::mode = Number::RoundingMode::ToNearest;
thread_local std::reference_wrapper<MantissaRange const> Number::kRange = kLargeRange;
thread_local std::reference_wrapper<MantissaRange const> Number::kRange =
MantissaRange::getMantissaRange(MantissaRange::MantissaScale::Large);
std::set<MantissaRange::MantissaScale> const&
MantissaRange::getAllScales()
{
static std::set<MantissaRange::MantissaScale> const kScales = {
MantissaRange::MantissaScale::Small,
MantissaRange::MantissaScale::LargeLegacy,
MantissaRange::MantissaScale::Large,
};
return kScales;
}
std::unordered_map<MantissaRange::MantissaScale, MantissaRange> const&
MantissaRange::getRanges()
{
static auto const kMap = []() {
std::unordered_map<MantissaScale, MantissaRange> map;
for (auto const scale : getAllScales())
{
map.emplace(scale, scale);
}
// Use these constexpr declarations to do static_asserts to verify the MantissaRanges are
// created correctly, but nothing else.
{
[[maybe_unused]]
constexpr static MantissaRange kRange{MantissaRange::MantissaScale::Small};
static_assert(isPowerOfTen(kRange.min));
static_assert(kRange.min == 1'000'000'000'000'000LL);
static_assert(kRange.max == 9'999'999'999'999'999LL);
static_assert(kRange.log == 15);
static_assert(kRange.min < Number::kMaxRep);
static_assert(kRange.max < Number::kMaxRep);
static_assert(kRange.cuspRoundingFixEnabled == CuspRoundingFix::Disabled);
}
{
[[maybe_unused]]
constexpr static MantissaRange kRange{MantissaRange::MantissaScale::LargeLegacy};
static_assert(isPowerOfTen(kRange.min));
static_assert(kRange.min == 1'000'000'000'000'000'000ULL);
static_assert(kRange.max == rep(9'999'999'999'999'999'999ULL));
static_assert(kRange.log == 18);
static_assert(kRange.min < Number::kMaxRep);
static_assert(kRange.max > Number::kMaxRep);
static_assert(kRange.cuspRoundingFixEnabled == CuspRoundingFix::Disabled);
}
{
[[maybe_unused]]
constexpr static MantissaRange kRange{MantissaRange::MantissaScale::Large};
static_assert(isPowerOfTen(kRange.min));
static_assert(kRange.min == 1'000'000'000'000'000'000ULL);
static_assert(kRange.max == rep(9'999'999'999'999'999'999ULL));
static_assert(kRange.log == 18);
static_assert(kRange.min < Number::kMaxRep);
static_assert(kRange.max > Number::kMaxRep);
static_assert(kRange.cuspRoundingFixEnabled == CuspRoundingFix::Enabled);
}
return map;
}();
return kMap;
}
MantissaRange const&
MantissaRange::getMantissaRange(MantissaScale scale)
{
return getRanges().at(scale);
}
Number::RoundingMode
Number::getround()
@@ -51,10 +122,37 @@ Number::getMantissaScale()
void
Number::setMantissaScale(MantissaRange::MantissaScale scale)
{
if (scale != MantissaRange::MantissaScale::Small &&
scale != MantissaRange::MantissaScale::Large)
if (!MantissaRange::getAllScales().contains(scale))
logicError("Unknown mantissa scale");
kRange = scale == MantissaRange::MantissaScale::Small ? kSmallRange : kLargeRange;
kRange = MantissaRange::getMantissaRange(scale);
}
// Optimization equivalent to:
// auto r = static_cast<unsigned>(u % 10);
// u /= 10;
// return r;
// Derived from Hacker's Delight Second Edition Chapter 10
// by Henry S. Warren, Jr.
static inline unsigned
divu10(uint128_t& u)
{
// q = u * 0.75
auto q = (u >> 1) + (u >> 2);
// iterate towards q = u * 0.8
q += q >> 4;
q += q >> 8;
q += q >> 16;
q += q >> 32;
q += q >> 64;
// q /= 8 approximately == u / 10
q >>= 3;
// r = u - q * 10 approximately == u % 10
auto r = static_cast<unsigned>(u - ((q << 3) + (q << 1)));
// correction c is 1 if r >= 10 else 0
auto c = (r + 6) >> 4;
u = q + c;
r -= c * 10;
return r;
}
// Guard
@@ -80,6 +178,10 @@ public:
setPositive() noexcept;
void
setNegative() noexcept;
// Should only be called by doNormalize, and then only for division
// operations with remainders.
void
setDropped() noexcept;
[[nodiscard]] bool
isNegative() const noexcept;
@@ -92,6 +194,18 @@ public:
unsigned
pop() noexcept;
/** Drop a digit from the mantissa, and increment the exponent, storing the dropped digit in
* this Guard.
*
* Substitute for:
push(mantissa % 10);
mantissa /= 10;
++exponent;
*/
template <class T>
void
doDropDigit(T& mantissa, int& exponent) noexcept;
// Indicate round direction: 1 is up, -1 is down, 0 is even
// This enables the client to round towards nearest, and on
// tie, round towards even.
@@ -107,6 +221,7 @@ public:
int& exponent,
internalrep const& minMantissa,
internalrep const& maxMantissa,
MantissaRange::CuspRoundingFix cuspRoundingFixEnabled,
std::string location);
// Modify the result to the correctly rounded value
@@ -139,6 +254,12 @@ Number::Guard::setNegative() noexcept
sbit_ = 1;
}
inline void
Number::Guard::setDropped() noexcept
{
xbit_ = 1;
}
inline bool
Number::Guard::isNegative() const noexcept
{
@@ -168,6 +289,27 @@ Number::Guard::pop() noexcept
return d;
}
template <class T>
void
Number::Guard::doDropDigit(T& mantissa, int& exponent) noexcept
{
push(mantissa % 10);
mantissa /= 10;
++exponent;
}
// Use the divu10 optimization for uint128s
template <>
void
Number::Guard::doDropDigit<uint128_t>(uint128_t& mantissa, int& exponent) noexcept
{
// The following is optimization for:
// push(static_cast<unsigned>(mantissa % 10));
// mantissa /= 10;
push(divu10(mantissa));
++exponent;
}
// Returns:
// -1 if Guard is less than half
// 0 if Guard is exactly half
@@ -242,18 +384,60 @@ Number::Guard::doRoundUp(
int& exponent,
internalrep const& minMantissa,
internalrep const& maxMantissa,
MantissaRange::CuspRoundingFix cuspRoundingFixEnabled,
std::string location)
{
auto r = round();
if (r == 1 || (r == 0 && (mantissa & 1) == 1))
{
++mantissa;
// Ensure mantissa after incrementing fits within both the
// min/maxMantissa range and is a valid "rep".
if (mantissa > maxMantissa || mantissa > kMaxRep)
auto const safeToIncrement = [&maxMantissa](auto const& mantissa) {
return mantissa < maxMantissa && mantissa < kMaxRep;
};
if (cuspRoundingFixEnabled == MantissaRange::CuspRoundingFix::Enabled)
{
mantissa /= 10;
++exponent;
// Ensure mantissa after incrementing fits within both the
// min/maxMantissa range and is a valid "rep".
if (safeToIncrement(mantissa))
{
// Nothing unusual here, just increment the mantissa
++mantissa;
}
else
{
// Incrementing the mantissa will require dividing, which will require rounding. So
// _don't_ increment the mantissa. Instead, divide and round recursively. It should
// be impossible to recurse more than once, because once the mantissa is divided by
// 10, it will be _well_ under maxMantissa and kMaxRep, so adding 1 will have no
// chance of bringing it back over.
doDropDigit(mantissa, exponent);
XRPL_ASSERT_PARTS(
safeToIncrement(mantissa),
"xrpl::Number::Guard::doRoundUp",
"can't recurse more than once");
doRoundUp(
negative,
mantissa,
exponent,
minMantissa,
maxMantissa,
cuspRoundingFixEnabled,
location);
return;
}
}
else
{
// Need to preserve the incorrect behavior until the fix amendment can be retired,
// because otherwise would risk an unplanned ledger fork.
++mantissa;
// Ensure mantissa after incrementing fits within both the
// min/maxMantissa range and is a valid "rep".
if (mantissa > maxMantissa || mantissa > kMaxRep)
{
// Don't use doDropDigit here
mantissa /= 10;
++exponent;
}
}
}
bringIntoRange(negative, mantissa, exponent, minMantissa);
@@ -293,9 +477,9 @@ Number::Guard::doRound(rep& drops, std::string location) const
{
static_assert(sizeof(internalrep) == sizeof(rep));
// This should be impossible, because it's impossible to represent
// "maxRep + 0.6" in Number, regardless of the scale. There aren't
// enough digits available. You'd either get a mantissa of "maxRep"
// or "(maxRep + 1) / 10", neither of which will round up when
// "kMaxRep + 0.6" in Number, regardless of the scale. There aren't
// enough digits available. You'd either get a mantissa of "kMaxRep"
// or "(kMaxRep + 1) / 10", neither of which will round up when
// converting to rep, though the latter might overflow _before_
// rounding.
Throw<std::overflow_error>(std::string(location)); // LCOV_EXCL_LINE
@@ -331,33 +515,13 @@ Number::externalToInternal(rep mantissa)
return static_cast<internalrep>(-temp);
}
constexpr Number
Number::oneSmall()
{
return Number{false, Number::kSmallRange.min, -Number::kSmallRange.log, Number::Unchecked{}};
};
constexpr Number kOneSml = Number::oneSmall();
constexpr Number
Number::oneLarge()
{
return Number{false, Number::kLargeRange.min, -Number::kLargeRange.log, Number::Unchecked{}};
};
constexpr Number kOneLrg = Number::oneLarge();
Number
Number::one()
{
if (&kRange.get() == &kSmallRange)
return kOneSml;
XRPL_ASSERT(&kRange.get() == &kLargeRange, "Number::one() : valid range");
return kOneLrg;
auto const& range = kRange.get();
return Number{false, range.min, -range.log, Number::Unchecked{}};
}
// Use the member names in this static function for now so the diff is cleaner
// TODO: Rename the function parameters to get rid of the "_" suffix
template <class T>
void
doNormalize(
@@ -365,7 +529,9 @@ doNormalize(
T& mantissa,
int& exponent,
MantissaRange::rep const& minMantissa,
MantissaRange::rep const& maxMantissa)
MantissaRange::rep const& maxMantissa,
MantissaRange::CuspRoundingFix cuspRoundingFixEnabled,
bool dropped)
{
static constexpr auto kMinExponent = Number::kMinExponent;
static constexpr auto kMaxExponent = Number::kMaxExponent;
@@ -390,13 +556,13 @@ doNormalize(
Guard g;
if (negative)
g.setNegative();
if (dropped)
g.setDropped();
while (m > maxMantissa)
{
if (exponent >= kMaxExponent)
throw std::overflow_error("Number::normalize 1");
g.push(m % 10);
m /= 10;
++exponent;
g.doDropDigit(m, exponent);
}
if ((exponent < kMinExponent) || (m < minMantissa))
{
@@ -407,7 +573,7 @@ doNormalize(
}
// When using the largeRange, "m" needs fit within an int64, even if
// the final mantissa_ is going to end up larger to fit within the
// the final mantissa is going to end up larger to fit within the
// MantissaRange. Cut it down here so that the rounding will be done while
// it's smaller.
//
@@ -415,26 +581,31 @@ doNormalize(
// so "m" will be modified to 990,000,000,000,012,345. Then that value
// will be rounded to 990,000,000,000,012,345 or
// 990,000,000,000,012,346, depending on the rounding mode. Finally,
// mantissa_ will be "m*10" so it fits within the range, and end up as
// mantissa will be "m*10" so it fits within the range, and end up as
// 9,900,000,000,000,123,450 or 9,900,000,000,000,123,460.
// mantissa() will return mantissa_ / 10, and exponent() will return
// exponent_ + 1.
// mantissa() will return mantissa / 10, and exponent() will return
// exponent + 1.
if (m > kMaxRep)
{
if (exponent >= kMaxExponent)
throw std::overflow_error("Number::normalize 1.5");
g.push(m % 10);
m /= 10;
++exponent;
g.doDropDigit(m, exponent);
}
// Before modification, m should be within the min/max range. After
// modification, it must be less than maxRep. In other words, the original
// value should have been no more than maxRep * 10.
// (maxRep * 10 > maxMantissa)
// modification, it must be less than kMaxRep. In other words, the original
// value should have been no more than kMaxRep * 10.
// (kMaxRep * 10 > maxMantissa)
XRPL_ASSERT_PARTS(m <= kMaxRep, "xrpl::doNormalize", "intermediate mantissa fits in int64");
mantissa = m;
g.doRoundUp(negative, mantissa, exponent, minMantissa, maxMantissa, "Number::normalize 2");
g.doRoundUp(
negative,
mantissa,
exponent,
minMantissa,
maxMantissa,
cuspRoundingFixEnabled,
"Number::normalize 2");
XRPL_ASSERT_PARTS(
mantissa >= minMantissa && mantissa <= maxMantissa,
"xrpl::doNormalize",
@@ -448,9 +619,15 @@ Number::normalize<uint128_t>(
uint128_t& mantissa,
int& exponent,
internalrep const& minMantissa,
internalrep const& maxMantissa)
internalrep const& maxMantissa,
MantissaRange::CuspRoundingFix cuspRoundingFixEnabled)
{
doNormalize(negative, mantissa, exponent, minMantissa, maxMantissa);
// Not used by every compiler version, and thus not necessarily
// counted by coverage build
// LCOV_EXCL_START
doNormalize(
negative, mantissa, exponent, minMantissa, maxMantissa, cuspRoundingFixEnabled, false);
// LCOV_EXCL_STOP
}
template <>
@@ -460,9 +637,15 @@ Number::normalize<unsigned long long>(
unsigned long long& mantissa,
int& exponent,
internalrep const& minMantissa,
internalrep const& maxMantissa)
internalrep const& maxMantissa,
MantissaRange::CuspRoundingFix cuspRoundingFixEnabled)
{
doNormalize(negative, mantissa, exponent, minMantissa, maxMantissa);
// Not used by every compiler version, and thus not necessarily
// counted by coverage build
// LCOV_EXCL_START
doNormalize(
negative, mantissa, exponent, minMantissa, maxMantissa, cuspRoundingFixEnabled, false);
// LCOV_EXCL_STOP
}
template <>
@@ -472,16 +655,17 @@ Number::normalize<unsigned long>(
unsigned long& mantissa,
int& exponent,
internalrep const& minMantissa,
internalrep const& maxMantissa)
internalrep const& maxMantissa,
MantissaRange::CuspRoundingFix cuspRoundingFixEnabled)
{
doNormalize(negative, mantissa, exponent, minMantissa, maxMantissa);
doNormalize(
negative, mantissa, exponent, minMantissa, maxMantissa, cuspRoundingFixEnabled, false);
}
void
Number::normalize()
Number::normalize(MantissaRange const& range)
{
auto const& range = kRange.get();
normalize(negative_, mantissa_, exponent_, range.min, range.max);
normalize(negative_, mantissa_, exponent_, range.min, range.max, range.cuspRoundingFixEnabled);
}
// Copy the number, but set a new exponent. Because the mantissa doesn't change,
@@ -542,9 +726,7 @@ Number::operator+=(Number const& y)
g.setNegative();
do
{
g.push(xm % 10);
xm /= 10;
++xe;
g.doDropDigit(xm, xe);
} while (xe < ye);
}
else if (xe > ye)
@@ -553,26 +735,30 @@ Number::operator+=(Number const& y)
g.setNegative();
do
{
g.push(ym % 10);
ym /= 10;
++ye;
g.doDropDigit(ym, ye);
} while (xe > ye);
}
auto const& range = kRange.get();
auto const& minMantissa = range.min;
auto const& maxMantissa = range.max;
auto const cuspRoundingFixEnabled = range.cuspRoundingFixEnabled;
if (xn == yn)
{
xm += ym;
if (xm > maxMantissa || xm > kMaxRep)
{
g.push(xm % 10);
xm /= 10;
++xe;
g.doDropDigit(xm, xe);
}
g.doRoundUp(xn, xm, xe, minMantissa, maxMantissa, "Number::addition overflow");
g.doRoundUp(
xn,
xm,
xe,
minMantissa,
maxMantissa,
cuspRoundingFixEnabled,
"Number::addition overflow");
}
else
{
@@ -598,38 +784,10 @@ Number::operator+=(Number const& y)
negative_ = xn;
mantissa_ = static_cast<internalrep>(xm);
exponent_ = xe;
normalize();
normalize(range);
return *this;
}
// Optimization equivalent to:
// auto r = static_cast<unsigned>(u % 10);
// u /= 10;
// return r;
// Derived from Hacker's Delight Second Edition Chapter 10
// by Henry S. Warren, Jr.
static inline unsigned
divu10(uint128_t& u)
{
// q = u * 0.75
auto q = (u >> 1) + (u >> 2);
// iterate towards q = u * 0.8
q += q >> 4;
q += q >> 8;
q += q >> 16;
q += q >> 32;
q += q >> 64;
// q /= 8 approximately == u / 10
q >>= 3;
// r = u - q * 10 approximately == u % 10
auto r = static_cast<unsigned>(u - ((q << 3) + (q << 1)));
// correction c is 1 if r >= 10 else 0
auto c = (r + 6) >> 4;
u = q + c;
r -= c * 10;
return r;
}
Number&
Number::operator*=(Number const& y)
{
@@ -667,15 +825,13 @@ Number::operator*=(Number const& y)
auto const& range = kRange.get();
auto const& minMantissa = range.min;
auto const& maxMantissa = range.max;
auto const cuspRoundingFixEnabled = range.cuspRoundingFixEnabled;
while (zm > maxMantissa || zm > kMaxRep)
{
// The following is optimization for:
// g.push(static_cast<unsigned>(zm % 10));
// zm /= 10;
g.push(divu10(zm));
++ze;
g.doDropDigit(zm, ze);
}
xm = static_cast<internalrep>(zm);
xe = ze;
g.doRoundUp(
@@ -684,12 +840,13 @@ Number::operator*=(Number const& y)
xe,
minMantissa,
maxMantissa,
cuspRoundingFixEnabled,
"Number::multiplication overflow : exponent is " + std::to_string(xe));
negative_ = zn;
mantissa_ = xm;
exponent_ = xe;
normalize();
normalize(range);
return *this;
}
@@ -703,7 +860,9 @@ Number::operator/=(Number const& y)
return *this;
// n* = numerator
// d* = denominator
// *p = negative (positive?)
// z* = result (quotient)
// *p = negative (p for positive, even though the value means not
// positive?)
// *s = sign
// *m = mantissa
// *e = exponent
@@ -715,72 +874,155 @@ Number::operator/=(Number const& y)
bool const dp = y.negative_;
int const ds = (dp ? -1 : 1);
auto dm = y.mantissa_;
auto de = y.exponent_;
// Create the denominator as 128-bit unsigned, since that's what we
// need to work with.
uint128_t const dm = static_cast<uint128_t>(y.mantissa_);
auto const de = y.exponent_;
auto const& range = kRange.get();
auto const& minMantissa = range.min;
auto const& maxMantissa = range.max;
auto const cuspRoundingFixEnabled = range.cuspRoundingFixEnabled;
// Shift by 10^17 gives greatest precision while not overflowing
// uint128_t or the cast back to int64_t
// TODO: Can/should this be made bigger for largeRange?
// log(2^128,10) ~ 38.5
// largeRange.log = 18, fits in 10^19
// f can be up to 10^(38-19) = 10^19 safely
static_assert(kSmallRange.log == 15);
static_assert(kLargeRange.log == 18);
bool const small = Number::getMantissaScale() == MantissaRange::MantissaScale::Small;
uint128_t const f = small ? 100'000'000'000'000'000 : 10'000'000'000'000'000'000ULL;
XRPL_ASSERT_PARTS(f >= minMantissa * 10, "Number::operator/=", "factor expected size");
// Division operates on two large integers (16-digit for small
// mantissas, 19-digit for large) using integer math. If the values
// were just divided directly, the result would be only ever be one
// digit or zero - not very useful.
// e.g. 9'876'543'210'987'654 / 1'234'567'890'123'456 = 8
// 1'234'567'890'123'456 / 9'876'543'210'987'654 = 0
// Introduce a power-of-ten multiplication factor for the numerator
// which will ensure the result has a meaningful number of digits.
//
// Consider numbers with a 2-digit mantissa:
// * Assume both numbers have an exponent of 0, using "ToNearest" rounding
// * 23 / 67 = 0
// * Use a factor of 10^4
// * 230'000 / 67 = 3432 with an exponent of -4
// * The normalized result will be 34, exponent -2, or 0.34
//
// The most extreme results are 10/99 and 99/10
// * 100'000 / 99 = 1'010e-4 = 10e-2 or 0.10
// * 990'000 / 10 = 99'000e-4 = 99e-1 or 9.9
//
// Note that the computations give 2 or 3 digits after the
// decimal point to determine which way to round for most scenarios.
//
// For small mantissas (where the MantissaRange.log == 15), shifting by 10^17 gives sufficient
// precision while not overflowing uint128_t or the cast back to int64_t. (This is legacy
// behavior, which must not be changed.)
//
// For large mantissas (where the MantissaRange.log == 18), a shift by 10^20 would be optimal
// for most scenarios. However, larger mantissa values would overflow 2^128.
//
// * log(2^128,10) ~ 38.5
// * largeRange.log = 18, fits in 10^19
// * The expanded numerator must fit in 10^38
// * f not be more than 10^(38-19) = 10^19 safely
//
// So, we do the division into stages:
//
// Stage 1: Use the same factor of 10^17, for the initial division. This
// will frequently not result in a whole number quotient.
//
// Stage 2: If there is a remainder from the first step, repeat the
// process with a "correction" factor of 10^5. Shift the
// result of Stage 1 over by 5 places, and add the second result to it.
// This is equivalent to if we had used an initial factor of 10^22,
// a couple digits more than we actually need.
//
// Stage 3: If there is still a remainder, and the CuspRoundingFix
// is enabled, pass a flag indicating such to doNormalize. The Guard
// in doNormalize will treat that flag as if non-zero digits had
// been dropped from the mantissa when shrinking it into range.
// This is only relevant when rounding away from zero (Upward for
// positive numbers, Downward for negative), or if the "regular"
// remainder is exactly 0.5 for "ToNearest". This will give the
// rounding the most accurate result possible, as if infinite
// precision was used in the initial calculation.
// unsigned denominator
auto const dmu = static_cast<uint128_t>(dm);
// correctionFactor can be anything between 10 and f, depending on how much
// extra precision we want to only use for rounding with the
// largeRange. Three digits seems like plenty, and is more than
// the smallRange uses.
uint128_t const correctionFactor = 1'000;
// Stage 1: Do the initial division with a factor of 10^17.
auto constexpr factorExponent = 17;
uint128_t constexpr f = kPowerOfTen[factorExponent];
auto const numerator = uint128_t(nm) * f;
auto zm = numerator / dmu;
auto ze = ne - de - (small ? 17 : 19);
bool zn = (ns * ds) < 0;
if (!small)
auto zm = numerator / dm;
auto ze = ne - de - factorExponent;
bool zp = (ns * ds) < 0;
// dropped is used in the same way as Guard::xbit_. In the case of
// division, it indicates if there's any remainder left over after
// we have been as precise as reasonable. If there is, it would be as
// if we were using infinite precision math, and a non-zero digit
// had been shifted off the end of the result when normalizing.
bool dropped = false;
if (range.scale != MantissaRange::MantissaScale::Small)
{
// Virtually multiply numerator by correctionFactor. Since that would
// overflow in the existing uint128_t, we'll do that part separately.
// Stage 2
//
// If there is a remainder, treat it as a secondary numerator.
// Multiply by correctionFactor separately from stage 1.
// The math for this would work for small mantissas, but we need to
// preserve existing behavior.
// preserve legacy behavior.
//
// Consider:
// ((numerator * correctionFactor) / dmu) / correctionFactor
// = ((numerator / dmu) * correctionFactor) / correctionFactor)
// ((numerator * correctionFactor) / dm) / correctionFactor
// = ((numerator / dm) * correctionFactor) / correctionFactor)
//
// But that assumes infinite precision. With integer math, this is
// equivalent to
//
// = ((numerator / dmu * correctionFactor)
// + ((numerator % dmu) * correctionFactor) / dmu) / correctionFactor
// = ((numerator / dm * correctionFactor)
// + ((numerator % dm) * correctionFactor) / dm) / correctionFactor
// = ((zm * correctionFactor)
// + (remainder * correctionFactor) / dm) / correctionFactor
//
// We have already set `mantissa_ = numerator / dmu`. Now we
// compute `remainder = numerator % dmu`, and if it is
// nonzero, we do the rest of the arithmetic. If it's zero, we can skip
// it.
auto const remainder = (numerator % dmu);
// The trick is that multiplication by correctionFactor is done on the mantissa, but
// division by correctionFactor is done by modifying the exponent, so no precision is lost
// until we normalize.
//
// If remainder is zero, we can skip this stage entirely because
// the first stage gave an exact answer.
auto constexpr correctionExponent = 5;
uint128_t constexpr correctionFactor = kPowerOfTen[correctionExponent];
static_assert(factorExponent + correctionExponent == 22);
auto const remainder = (numerator % dm);
if (remainder != 0)
{
zm *= correctionFactor;
auto const correction = remainder * correctionFactor / dmu;
zm += correction;
// divide by 1000 by moving the exponent, so we don't lose the
// integer value we just computed
ze -= 3;
auto const partialNumerator = remainder * correctionFactor;
auto const correction = partialNumerator / dm;
// If the correction is zero, we do not have to make any
// modifications to z*, because it will not have any
// effect on the final result. (We'd be adding a bunch of
// zeros to the end of zm that would just be removed in
// normalize.) However, if that is the case, then Stage 3 is
// even more important for accuracy.
if (correction != 0)
{
zm *= correctionFactor;
// divide by the correctionFactor by moving the exponent, so we don't lose the
// integer value we just computed
ze -= correctionExponent;
zm += correction;
}
// Stage 3: If there's still anything left, and the cusp
// rounding fix is enabled, flag if there is still
// a remainder from stage 2.
bool const useTrailingRemainder =
cuspRoundingFixEnabled == MantissaRange::CuspRoundingFix::Enabled;
if (useTrailingRemainder)
{
dropped = partialNumerator % dm != 0;
}
}
}
normalize(zn, zm, ze, minMantissa, maxMantissa);
negative_ = zn;
doNormalize(zp, zm, ze, minMantissa, maxMantissa, cuspRoundingFixEnabled, dropped);
negative_ = zp;
mantissa_ = static_cast<internalrep>(zm);
exponent_ = ze;
XRPL_ASSERT_PARTS(isnormal(), "xrpl::Number::operator/=", "result is normalized");
@@ -801,10 +1043,9 @@ operator rep() const
g.setNegative();
drops = -drops;
}
for (; offset < 0; ++offset)
while (offset < 0)
{
g.push(drops % 10);
drops /= 10;
g.doDropDigit(drops, offset);
}
for (; offset > 0; --offset)
{
@@ -831,7 +1072,7 @@ Number::truncate() const noexcept
}
// We are guaranteed that normalize() will never throw an exception
// because exponent is either negative or zero at this point.
ret.normalize();
ret.normalize(kRange);
return ret;
}

View File

@@ -1,55 +0,0 @@
#include <xrpl/ledger/BookListeners.h>
#include <xrpl/basics/UnorderedContainers.h>
#include <xrpl/json/json_value.h>
#include <xrpl/protocol/MultiApiJson.h>
#include <xrpl/server/InfoSub.h>
#include <cstdint>
#include <mutex>
namespace xrpl {
void
BookListeners::addSubscriber(InfoSub::ref sub)
{
std::scoped_lock const sl(lock_);
listeners_[sub->getSeq()] = sub;
}
void
BookListeners::removeSubscriber(std::uint64_t seq)
{
std::scoped_lock const sl(lock_);
listeners_.erase(seq);
}
void
BookListeners::publish(MultiApiJson const& jvObj, hash_set<std::uint64_t>& havePublished)
{
std::scoped_lock const sl(lock_);
auto it = listeners_.cbegin();
while (it != listeners_.cend())
{
InfoSub::pointer p = it->second.lock();
if (p)
{
// Only publish jvObj if this is the first occurrence
if (havePublished.emplace(p->getSeq()).second)
{
jvObj.visit(
p->getApiVersion(), //
[&](json::Value const& jv) { p->send(jv, true); });
}
++it;
}
else
{
it = listeners_.erase(it);
}
}
}
} // namespace xrpl

View File

@@ -559,15 +559,34 @@ tryOverpayment(
<< ", new total value: " << newLoanProperties.loanState.valueOutstanding
<< ", first payment principal: " << newLoanProperties.firstPaymentPrincipal;
// Calculate what the new loan state should be with the new periodic payment
// including rounding errors
auto const newTheoreticalState = computeTheoreticalLoanState(
rules,
newLoanProperties.periodicPayment,
periodicRate,
paymentRemaining,
managementFeeRate) +
errors;
// Calculate what the new loan state should be with the new periodic payment,
// including the preserved rounding errors.
auto const newTheoreticalState = [&]() {
auto const state = computeTheoreticalLoanState(
rules,
newLoanProperties.periodicPayment,
periodicRate,
paymentRemaining,
managementFeeRate) +
errors;
if (!rules.enabled(fixCleanup3_2_0))
return state;
// The new principal is known exactly: it is reduced by the overpayment's
// principal portion. computeTheoreticalLoanState instead derives the
// principal -- and, from it, the management fee and interest -- via a
// lossy (P * factor) / factor round-trip. Pin the principal to the exact
// value and re-derive the management fee from the exact interest gross
// (value - principal), so the intermediate state is fully consistent with
// the exact principal rather than the one-scale-unit-high round-trip.
Number const principal =
roundedOldState.principalOutstanding - overpaymentComponents.trackedPrincipalDelta;
Number const managementFee =
tenthBipsOfValue(state.valueOutstanding - principal, managementFeeRate);
return constructLoanState(state.valueOutstanding, principal, managementFee);
}();
JLOG(j.debug()) << "new theoretical value: " << newTheoreticalState.valueOutstanding
<< ", principal: " << newTheoreticalState.principalOutstanding
@@ -762,16 +781,6 @@ doOverpayment(
// The proxies still hold the original (pre-overpayment) values, which
// allows us to compute deltas and verify they match what we expect
// from the overpaymentComponents and loanPaymentParts.
XRPL_ASSERT_PARTS(
overpaymentComponents.trackedPrincipalDelta ==
principalOutstandingProxy - newRoundedLoanState.principalOutstanding,
"xrpl::detail::doOverpayment",
"principal change agrees");
// I'm not 100% sure the following asserts are correct. If in doubt, and
// everything else works, remove any that cause trouble.
JLOG(j.debug()) << "valueChange: " << loanPaymentParts.valueChange
<< ", totalValue before: " << *totalValueOutstandingProxy
<< ", totalValue after: " << newRoundedLoanState.valueOutstanding
@@ -783,18 +792,50 @@ doOverpayment(
<< overpaymentComponents.trackedPrincipalDelta -
(totalValueOutstandingProxy - newRoundedLoanState.valueOutstanding);
XRPL_ASSERT_PARTS(
loanPaymentParts.valueChange ==
newRoundedLoanState.valueOutstanding -
(totalValueOutstandingProxy - overpaymentComponents.trackedPrincipalDelta) +
overpaymentComponents.trackedInterestPart(),
"xrpl::detail::doOverpayment",
"interest paid agrees");
// The three assertions below are invariants that only hold once
// fixCleanup3_2_0 pins the new principal to the exact reduction
// (oldPrincipal - trackedPrincipalDelta). Before the amendment, the lossy
// (P * factor) / factor round-trip can leave the new principal one
// scale-unit high, so these equalities do not hold on the pre-amendment
// code path and must be gated to match the fix they verify.
if (rules.enabled(fixCleanup3_2_0))
{
// The valueChange returned by tryOverpayment satisfies
// valueChange = (newInterestDue - oldInterestDue) + untrackedInterest.
// Using the loan-state identity v = p + i + m and the adjacent
// `principal change agrees` assertion (dp = oldP - newP), this
// rearranges into three independently-computable terms:
//
// 1. TVO change beyond what principal repayment alone explains:
// newTVO - (oldTVO - dp)
// 2. Management fee released by re-amortization (positive when
// mfee decreased; zero when managementFeeRate == 0):
// oldMfee - newMfee
// 3. The overpayment's penalty interest part (= untrackedInterest
// for the overpayment path; see computeOverpaymentComponents):
// trackedInterestPart()
[[maybe_unused]] Number const tvoChange = newRoundedLoanState.valueOutstanding -
(totalValueOutstandingProxy - overpaymentComponents.trackedPrincipalDelta);
[[maybe_unused]] Number const managementFeeReleased =
managementFeeOutstandingProxy - newRoundedLoanState.managementFeeDue;
[[maybe_unused]] Number const interestPart = overpaymentComponents.trackedInterestPart();
XRPL_ASSERT_PARTS(
overpaymentComponents.trackedPrincipalDelta == loanPaymentParts.principalPaid,
"xrpl::detail::doOverpayment",
"principal payment matches");
XRPL_ASSERT_PARTS(
overpaymentComponents.trackedPrincipalDelta ==
principalOutstandingProxy - newRoundedLoanState.principalOutstanding,
"xrpl::detail::doOverpayment",
"principal change agrees");
XRPL_ASSERT_PARTS(
loanPaymentParts.valueChange == tvoChange + managementFeeReleased + interestPart,
"xrpl::detail::doOverpayment",
"interest paid agrees");
XRPL_ASSERT_PARTS(
overpaymentComponents.trackedPrincipalDelta == loanPaymentParts.principalPaid,
"xrpl::detail::doOverpayment",
"principal payment matches");
}
// All validations passed, so update the proxy objects (which will
// modify the actual Loan ledger object)
@@ -1130,11 +1171,13 @@ computePaymentComponents(
// Cap each component to never exceed what's actually outstanding
deltas.principal = std::min(deltas.principal, currentLedgerState.principalOutstanding);
XRPL_ASSERT_PARTS(
deltas.interest <= currentLedgerState.interestDue,
"xrpl::detail::computePaymentComponents",
"interest due delta not greater than outstanding");
if (fixCleanup320Enabled)
{
XRPL_ASSERT_PARTS(
deltas.interest <= currentLedgerState.interestDue,
"xrpl::detail::computePaymentComponents",
"interest due delta not greater than outstanding");
}
// Cap interest to both the outstanding amount AND what's left of the
// periodic payment after principal is paid
deltas.interest = std::min(
@@ -1275,6 +1318,7 @@ computePaymentComponents(
*/
ExtendedPaymentComponents
computeOverpaymentComponents(
Rules const& rules,
Asset const& asset,
int32_t const loanScale,
Number const& overpayment,
@@ -1282,10 +1326,13 @@ computeOverpaymentComponents(
TenthBips32 const overpaymentFeeRate,
TenthBips16 const managementFeeRate)
{
XRPL_ASSERT(
overpayment > 0 && isRounded(asset, overpayment, loanScale),
"xrpl::detail::computeOverpaymentComponents : valid overpayment "
"amount");
if (rules.enabled(fixCleanup3_2_0))
{
XRPL_ASSERT(
overpayment > 0 && isRounded(asset, overpayment, loanScale),
"xrpl::detail::computeOverpaymentComponents : valid overpayment "
"amount");
}
// First, deduct the fixed overpayment fee from the total amount.
// This reduces the effective payment that will be applied to the loan.
@@ -2027,51 +2074,63 @@ loanMakePayment(
// It shouldn't be possible for the overpayment to be greater than
// totalValueOutstanding, because that would have been processed as
// another normal payment. But cap it just in case.
Number const overpayment = std::min(roundedAmount - totalPaid, *totalValueOutstandingProxy);
Number const overpaymentRaw =
std::min(roundedAmount - totalPaid, *totalValueOutstandingProxy);
detail::ExtendedPaymentComponents const overpaymentComponents =
detail::computeOverpaymentComponents(
asset,
loanScale,
overpayment,
overpaymentInterestRate,
overpaymentFeeRate,
managementFeeRate);
bool const fixEnabled = view.rules().enabled(fixCleanup3_2_0);
Number const overpayment = fixEnabled
? roundToAsset(asset, overpaymentRaw, loanScale, Number::RoundingMode::Downward)
: overpaymentRaw;
// Don't process an overpayment if the whole amount (or more!)
// gets eaten by fees and interest.
if (overpaymentComponents.trackedPrincipalDelta > 0)
// Post-amendment, the rounded overpayment can be zero; pre-amendment
// it's always positive given the surrounding guards.
if (!fixEnabled || overpayment > 0)
{
XRPL_ASSERT_PARTS(
overpaymentComponents.untrackedInterest >= beast::kZero,
"xrpl::loanMakePayment",
"overpayment penalty did not reduce value of loan");
// Can't just use `periodicPayment` here, because it might
// change
auto periodicPaymentProxy = loan->at(sfPeriodicPayment);
if (auto const overResult = detail::doOverpayment(
detail::ExtendedPaymentComponents const overpaymentComponents =
detail::computeOverpaymentComponents(
view.rules(),
asset,
loanScale,
overpaymentComponents,
totalValueOutstandingProxy,
principalOutstandingProxy,
managementFeeOutstandingProxy,
periodicPaymentProxy,
periodicRate,
paymentRemainingProxy,
managementFeeRate,
j))
overpayment,
overpaymentInterestRate,
overpaymentFeeRate,
managementFeeRate);
// Don't process an overpayment if the whole amount (or more!)
// gets eaten by fees and interest.
if (overpaymentComponents.trackedPrincipalDelta > 0)
{
totalParts += *overResult;
}
else if (overResult.error())
{
// error() will be the TER returned if a payment is not
// made. It will only evaluate to true if it's unsuccessful.
// Otherwise, tesSUCCESS means nothing was done, so
// continue.
return Unexpected(overResult.error());
XRPL_ASSERT_PARTS(
overpaymentComponents.untrackedInterest >= beast::kZero,
"xrpl::loanMakePayment",
"overpayment penalty did not reduce value of loan");
// Can't just use `periodicPayment` here, because it might
// change
auto periodicPaymentProxy = loan->at(sfPeriodicPayment);
if (auto const overResult = detail::doOverpayment(
view.rules(),
asset,
loanScale,
overpaymentComponents,
totalValueOutstandingProxy,
principalOutstandingProxy,
managementFeeOutstandingProxy,
periodicPaymentProxy,
periodicRate,
paymentRemainingProxy,
managementFeeRate,
j))
{
totalParts += *overResult;
}
else if (overResult.error())
{
// error() will be the TER returned if a payment is not
// made. It will only evaluate to true if it's unsuccessful.
// Otherwise, tesSUCCESS means nothing was done, so
// continue.
return Unexpected(overResult.error());
}
}
}
}

View File

@@ -25,7 +25,6 @@
#include <xrpl/protocol/STLedgerEntry.h>
#include <xrpl/protocol/TER.h>
#include <xrpl/protocol/TxFlags.h>
#include <xrpl/protocol/TxFormats.h>
#include <xrpl/protocol/UintTypes.h>
#include <xrpl/protocol/XRPAmount.h>
@@ -383,10 +382,11 @@ requireAuth(
// belong to someone who is explicitly authorized e.g. a vault owner.
}
if (featureSAVEnabled)
bool const featureMPTV2Enabled = view.rules().enabled(featureMPTokensV2);
if (featureSAVEnabled || featureMPTV2Enabled)
{
// Implicitly authorize Vault and LoanBroker pseudo-accounts
if (isPseudoAccount(view, account, {&sfVaultID, &sfLoanBrokerID}))
// Implicitly authorize Vault, LoanBroker, and AMM pseudo-accounts
if (isPseudoAccount(view, account, {&sfVaultID, &sfLoanBrokerID, &sfAMMID}))
return tesSUCCESS;
}
@@ -618,6 +618,22 @@ canTrade(ReadView const& view, Asset const& asset, std::uint8_t depth)
});
}
TER
canMPTTradeAndTransfer(
ReadView const& view,
Asset const& asset,
AccountID const& from,
AccountID const& to)
{
if (!asset.holds<MPTIssue>())
return tesSUCCESS;
if (auto const ter = canTrade(view, asset); !isTesSuccess(ter))
return ter;
return canTransfer(view, asset, from, to);
}
TER
lockEscrowMPT(ApplyView& view, AccountID const& sender, STAmount const& amount, beast::Journal j)
{
@@ -985,96 +1001,4 @@ issuerSelfDebitHookMPT(ApplyView& view, MPTIssue const& issue, std::uint64_t amo
view.issuerSelfDebitHookMPT(issue, amount, available);
}
static TER
checkMPTAllowed(
ReadView const& view,
TxType txType,
Asset const& asset,
AccountID const& accountID,
std::uint8_t depth = 0)
{
if (!asset.holds<MPTIssue>())
return tesSUCCESS;
auto const& issuanceID = asset.get<MPTIssue>().getMptID();
auto const validTx = txType == ttAMM_CREATE || txType == ttAMM_DEPOSIT ||
txType == ttAMM_WITHDRAW || txType == ttOFFER_CREATE || txType == ttCHECK_CREATE ||
txType == ttCHECK_CASH || txType == ttPAYMENT;
XRPL_ASSERT(validTx, "xrpl::checkMPTAllowed : all MPT tx or DEX");
if (!validTx)
return tefINTERNAL; // LCOV_EXCL_LINE
auto const& issuer = asset.getIssuer();
if (!view.exists(keylet::account(issuer)))
return tecNO_ISSUER; // LCOV_EXCL_LINE
auto const issuanceKey = keylet::mptIssuance(issuanceID);
auto const issuanceSle = view.read(issuanceKey);
if (!issuanceSle)
return tecOBJECT_NOT_FOUND; // LCOV_EXCL_LINE
if (issuanceSle->isFlag(lsfMPTLocked))
return tecLOCKED; // LCOV_EXCL_LINE
// Offer crossing and Payment
if (!issuanceSle->isFlag(lsfMPTCanTrade))
return tecNO_PERMISSION;
if (accountID != issuer)
{
if (!issuanceSle->isFlag(lsfMPTCanTransfer))
return tecNO_PERMISSION;
auto const mptSle = view.read(keylet::mptoken(issuanceKey.key, accountID));
// Allow to succeed since some tx create MPToken if it doesn't exist.
// Tx's have their own check for missing MPToken.
if (!mptSle)
return tesSUCCESS;
if (mptSle->isFlag(lsfMPTLocked))
return tecLOCKED;
// Post-fixCleanup3_2_0: vault shares inherit the underlying
// asset's checks here too. Without this, a share could be
// placed on the AMM, in an Offer, or in a Check even after
// the issuer has restricted the underlying. Mirrors the
// canTransfer / canTrade inheritance for path-find-adjacent
// operations that don't go through canTransfer directly.
if (view.rules().enabled(fixCleanup3_2_0) &&
issuanceSle->isFieldPresent(sfReferenceHolding))
{
// Defensive depth bound on the inheritance recursion.
// Reachable only post-fixCleanup3_2_0 and unreachable in
// practice (vault-of-vault-shares forbidden at VaultCreate).
if (depth >= kMaxAssetCheckDepth)
{
// LCOV_EXCL_START
UNREACHABLE("xrpl::MPTokenHelpers::checkMPTAllowed : reached asset check depth");
return tecINTERNAL;
// LCOV_EXCL_STOP
}
auto const sleHolding =
view.read(keylet::unchecked(issuanceSle->getFieldH256(sfReferenceHolding)));
if (!sleHolding)
return tefINTERNAL; // LCOV_EXCL_LINE
return checkMPTAllowed(
view, txType, assetOfHolding(*issuanceSle, *sleHolding), accountID, depth + 1);
}
}
return tesSUCCESS;
}
TER
checkMPTTxAllowed(
ReadView const& view,
TxType txType,
Asset const& asset,
AccountID const& accountID)
{
// use isDEXAllowed for payment/offer crossing
XRPL_ASSERT(txType != ttPAYMENT, "xrpl::checkMPTTxAllowed : not payment");
return checkMPTAllowed(view, txType, asset, accountID);
}
} // namespace xrpl

View File

@@ -5,14 +5,20 @@
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/ledger/ApplyView.h>
#include <xrpl/ledger/View.h>
#include <xrpl/ledger/helpers/AccountRootHelpers.h>
#include <xrpl/protocol/AccountID.h>
#include <xrpl/protocol/Feature.h>
#include <xrpl/protocol/Indexes.h>
#include <xrpl/protocol/SField.h>
#include <xrpl/protocol/STLedgerEntry.h>
#include <xrpl/protocol/TER.h>
#include <algorithm>
#include <cstdint>
#include <limits>
#include <memory>
#include <optional>
namespace xrpl {
@@ -65,4 +71,28 @@ closeChannel(
return tesSUCCESS;
}
uint32_t
saturatingAdd(Rules const& rules, uint32_t const lhs, uint32_t const rhs)
{
if (rules.enabled(fixCleanup3_2_0))
{
static constexpr auto kUint32Max =
static_cast<uint64_t>(std::numeric_limits<uint32_t>::max());
uint64_t const saturatedResult = std::min(uint64_t{lhs} + rhs, kUint32Max);
return static_cast<uint32_t>(saturatedResult);
}
return lhs + rhs;
}
bool
isChannelExpired(ApplyView const& view, std::optional<uint32_t> timeField)
{
if (!timeField)
return false;
if (view.rules().enabled(fixCleanup3_2_0))
return after(view.header().parentCloseTime, *timeField);
return view.header().parentCloseTime.time_since_epoch().count() >= *timeField;
}
} // namespace xrpl

View File

@@ -3,6 +3,8 @@
#include <xrpl/basics/Log.h>
#include <xrpl/basics/base_uint.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/beast/utility/Zero.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/ledger/ReadView.h>
#include <xrpl/ledger/helpers/CredentialHelpers.h>
#include <xrpl/protocol/AccountID.h>
@@ -19,6 +21,16 @@ namespace xrpl::permissioned_dex {
bool
accountInDomain(ReadView const& view, AccountID const& account, Domain const& domainID)
{
// Avoid constructing a zero-key PermissionedDomain keylet.
// keylet::permissionedDomain(uint256) uses the DomainID as the ledger key.
if (view.rules().enabled(fixCleanup3_2_0) && domainID == beast::kZero)
{
// LCOV_EXCL_START
UNREACHABLE("xrpl::permissioned_dex::accountInDomain : domainID is zero");
return false;
// LCOV_EXCL_STOP
}
auto const sleDomain = view.read(keylet::permissionedDomain(domainID));
if (!sleDomain)
return false;

View File

@@ -28,6 +28,7 @@
#include <cstdint>
#include <initializer_list>
#include <limits>
#include <string>
#include <variant>
@@ -55,6 +56,14 @@ isGlobalFrozen(ReadView const& view, Asset const& asset)
[&](MPTIssue const& issue) { return isGlobalFrozen(view, issue); });
}
TER
checkGlobalFrozen(ReadView const& view, Asset const& asset)
{
if (isGlobalFrozen(view, asset))
return asset.holds<MPTIssue>() ? tecLOCKED : tecFROZEN;
return tesSUCCESS;
}
bool
isIndividualFrozen(ReadView const& view, AccountID const& account, Asset const& asset)
{
@@ -62,6 +71,14 @@ isIndividualFrozen(ReadView const& view, AccountID const& account, Asset const&
[&](auto const& issue) { return isIndividualFrozen(view, account, issue); }, asset.value());
}
TER
checkIndividualFrozen(ReadView const& view, AccountID const& account, Asset const& asset)
{
if (isIndividualFrozen(view, account, asset))
return asset.holds<MPTIssue>() ? tecLOCKED : tecFROZEN;
return tesSUCCESS;
}
bool
isFrozen(ReadView const& view, AccountID const& account, Asset const& asset, std::uint8_t depth)
{
@@ -1105,6 +1122,13 @@ directSendNoFeeMPT(
auto const mptokenID = keylet::mptoken(mptID.key, uReceiverID);
if (auto sle = view.peek(mptokenID))
{
if (view.rules().enabled(featureMPTokensV2))
{
if ((*sle)[sfMPTAmount] > (std::numeric_limits<std::uint64_t>::max() - amt))
{
return tecINTERNAL; // LCOV_EXCL_LINE
}
}
view.creditHookMPT(uSenderID, uReceiverID, saAmount, (*sle)[sfMPTAmount], available);
(*sle)[sfMPTAmount] += amt;
view.update(sle);

View File

@@ -2,11 +2,16 @@
#include <xrpl/basics/Number.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/ledger/ReadView.h>
#include <xrpl/protocol/AccountID.h>
#include <xrpl/protocol/Indexes.h>
#include <xrpl/protocol/LedgerFormats.h>
#include <xrpl/protocol/SField.h>
#include <xrpl/protocol/STAmount.h>
#include <xrpl/protocol/STLedgerEntry.h>
#include <xrpl/protocol/STNumber.h> // IWYU pragma: keep
#include <cstdint>
#include <memory>
#include <optional>
@@ -70,7 +75,8 @@ assetsToSharesWithdraw(
std::shared_ptr<SLE const> const& vault,
std::shared_ptr<SLE const> const& issuance,
STAmount const& assets,
TruncateShares truncate)
TruncateShares truncate,
WaiveUnrealizedLoss waive)
{
XRPL_ASSERT(!assets.negative(), "xrpl::assetsToSharesWithdraw : non-negative assets");
XRPL_ASSERT(
@@ -80,7 +86,8 @@ assetsToSharesWithdraw(
return std::nullopt; // LCOV_EXCL_LINE
Number assetTotal = vault->at(sfAssetsTotal);
assetTotal -= vault->at(sfLossUnrealized);
if (waive == WaiveUnrealizedLoss::No)
assetTotal -= vault->at(sfLossUnrealized);
STAmount shares{vault->at(sfShareMPTID)};
if (assetTotal == 0)
return shares;
@@ -96,7 +103,8 @@ assetsToSharesWithdraw(
sharesToAssetsWithdraw(
std::shared_ptr<SLE const> const& vault,
std::shared_ptr<SLE const> const& issuance,
STAmount const& shares)
STAmount const& shares,
WaiveUnrealizedLoss waive)
{
XRPL_ASSERT(!shares.negative(), "xrpl::sharesToAssetsWithdraw : non-negative shares");
XRPL_ASSERT(
@@ -106,7 +114,8 @@ sharesToAssetsWithdraw(
return std::nullopt; // LCOV_EXCL_LINE
Number assetTotal = vault->at(sfAssetsTotal);
assetTotal -= vault->at(sfLossUnrealized);
if (waive == WaiveUnrealizedLoss::No)
assetTotal -= vault->at(sfLossUnrealized);
STAmount assets{vault->at(sfAsset)};
if (assetTotal == 0)
return assets;
@@ -115,4 +124,24 @@ sharesToAssetsWithdraw(
return assets;
}
[[nodiscard]] bool
isSoleShareholder(ReadView const& view, AccountID const& account, SLE::const_ref issuance)
{
XRPL_ASSERT(
issuance && issuance->getType() == ltMPTOKEN_ISSUANCE,
"xrpl::isSoleShareholder : valid issuance SLE");
std::uint64_t const outstanding = issuance->at(sfOutstandingAmount);
if (outstanding == 0)
return false;
auto const shareMPTID =
makeMptID(issuance->getFieldU32(sfSequence), issuance->getAccountID(sfIssuer));
auto const sleToken = view.read(keylet::mptoken(shareMPTID, account));
if (!sleToken)
return false; // LCOV_EXCL_LINE
return sleToken->getFieldU64(sfMPTAmount) == outstanding;
}
} // namespace xrpl

View File

@@ -4,21 +4,16 @@
#include <xrpl/basics/Log.h>
#include <xrpl/basics/base_uint.h>
#include <xrpl/basics/contract.h>
#include <xrpl/basics/strHex.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/nodestore/Database.h>
#include <xrpl/nodestore/NodeObject.h>
#include <xrpl/nodestore/Scheduler.h>
#include <xrpl/nodestore/Types.h>
#include <chrono>
#include <cstddef>
#include <cstdint>
#include <exception>
#include <functional>
#include <memory>
#include <utility>
#include <vector>
namespace xrpl::NodeStore {
@@ -29,6 +24,13 @@ DatabaseNodeImp::store(NodeObjectType type, Blob&& data, uint256 const& hash, st
auto obj = NodeObject::createObject(type, std::move(data), hash);
backend_->store(obj);
if (cache_)
{
// After the store, replace a negative cache entry if there is one
cache_->canonicalize(hash, obj, [](std::shared_ptr<NodeObject> const& n) {
return n->getType() == NodeObjectType::Dummy;
});
}
}
void
@@ -37,9 +39,25 @@ DatabaseNodeImp::asyncFetch(
std::uint32_t ledgerSeq,
std::function<void(std::shared_ptr<NodeObject> const&)>&& callback)
{
if (cache_)
{
std::shared_ptr<NodeObject> const obj = cache_->fetch(hash);
if (obj)
{
callback(obj->getType() == NodeObjectType::Dummy ? nullptr : obj);
return;
}
}
Database::asyncFetch(hash, ledgerSeq, std::move(callback));
}
void
DatabaseNodeImp::sweep()
{
if (cache_)
cache_->sweep();
}
std::shared_ptr<NodeObject>
DatabaseNodeImp::fetchNodeObject(
uint256 const& hash,
@@ -47,32 +65,58 @@ DatabaseNodeImp::fetchNodeObject(
FetchReport& fetchReport,
bool duplicate)
{
std::shared_ptr<NodeObject> nodeObject = nullptr;
Status status = Status::Ok;
std::shared_ptr<NodeObject> nodeObject = cache_ ? cache_->fetch(hash) : nullptr;
if (!nodeObject)
{
JLOG(j_.trace()) << "fetchNodeObject " << hash << ": record not "
<< (cache_ ? "cached" : "found");
try
{
status = backend_->fetch(hash, &nodeObject);
}
catch (std::exception const& e)
{
JLOG(j_.fatal()) << "fetchNodeObject " << hash
<< ": Exception fetching from backend: " << e.what();
rethrow();
}
Status status = Status::Ok;
try
{
status = backend_->fetch(hash, &nodeObject);
}
catch (std::exception const& e)
{
JLOG(j_.fatal()) << "fetchNodeObject " << hash
<< ": Exception fetching from backend: " << e.what();
rethrow();
}
switch (status)
switch (status)
{
case Status::Ok:
if (cache_)
{
if (nodeObject)
{
cache_->canonicalizeReplaceClient(hash, nodeObject);
}
else
{
auto notFound = NodeObject::createObject(NodeObjectType::Dummy, {}, hash);
cache_->canonicalizeReplaceClient(hash, notFound);
if (notFound->getType() != NodeObjectType::Dummy)
nodeObject = notFound;
}
}
break;
case Status::NotFound:
break;
case Status::DataCorrupt:
JLOG(j_.fatal()) << "fetchNodeObject " << hash << ": nodestore data is corrupted";
break;
default:
JLOG(j_.warn()) << "fetchNodeObject " << hash << ": backend returns unknown result "
<< static_cast<int>(status);
break;
}
}
else
{
case Status::Ok:
case Status::NotFound:
break;
case Status::DataCorrupt:
JLOG(j_.fatal()) << "fetchNodeObject " << hash << ": nodestore data is corrupted";
break;
default:
JLOG(j_.warn()) << "fetchNodeObject " << hash << ": backend returns unknown result "
<< static_cast<int>(status);
break;
JLOG(j_.trace()) << "fetchNodeObject " << hash << ": record found in cache";
if (nodeObject->getType() == NodeObjectType::Dummy)
nodeObject.reset();
}
if (nodeObject)
@@ -81,33 +125,4 @@ DatabaseNodeImp::fetchNodeObject(
return nodeObject;
}
std::vector<std::shared_ptr<NodeObject>>
DatabaseNodeImp::fetchBatch(std::vector<uint256> const& hashes)
{
using namespace std::chrono;
auto const before = steady_clock::now();
// Get the node objects that match the hashes from the backend. To protect
// against the backends returning fewer or more results than expected, the
// container is resized to the number of hashes.
auto results = backend_->fetchBatch(hashes).first;
XRPL_ASSERT(
results.size() == hashes.size() || results.empty(),
"number of output objects either matches number of input hashes or is empty");
results.resize(hashes.size());
for (size_t i = 0; i < results.size(); ++i)
{
if (!results[i])
{
JLOG(j_.error()) << "fetchBatch - "
<< "record not found in db. hash = " << strHex(hashes[i]);
}
}
auto fetchDurationUs =
std::chrono::duration_cast<std::chrono::microseconds>(steady_clock::now() - before).count();
updateFetchMetrics(hashes.size(), 0, fetchDurationUs);
return results;
}
} // namespace xrpl::NodeStore

View File

@@ -113,6 +113,12 @@ DatabaseRotatingImp::store(NodeObjectType type, Blob&& data, uint256 const& hash
storeStats(1, nObj->getData().size());
}
void
DatabaseRotatingImp::sweep()
{
// Nothing to do.
}
std::shared_ptr<NodeObject>
DatabaseRotatingImp::fetchNodeObject(
uint256 const& hash,

View File

@@ -44,8 +44,10 @@ ManagerImp::missingBackend()
// the Factory classes is an undefined behaviour.
void
registerNuDBFactory(Manager& manager);
#if XRPL_ROCKSDB_AVAILABLE
void
registerRocksDBFactory(Manager& manager);
#endif
void
registerNullFactory(Manager& manager);
void
@@ -54,7 +56,9 @@ registerMemoryFactory(Manager& manager);
ManagerImp::ManagerImp()
{
registerNuDBFactory(*this);
#if XRPL_ROCKSDB_AVAILABLE
registerRocksDBFactory(*this);
#endif
registerNullFactory(*this);
registerMemoryFactory(*this);
}

View File

@@ -22,7 +22,6 @@
#include <string>
#include <tuple>
#include <utility>
#include <vector>
namespace xrpl::NodeStore {
@@ -146,28 +145,6 @@ public:
return Status::Ok;
}
std::pair<std::vector<std::shared_ptr<NodeObject>>, Status>
fetchBatch(std::vector<uint256> const& hashes) override
{
std::vector<std::shared_ptr<NodeObject>> results;
results.reserve(hashes.size());
for (auto const& h : hashes)
{
std::shared_ptr<NodeObject> nObj;
Status const status = fetch(h, &nObj);
if (status != Status::Ok)
{
results.push_back({});
}
else
{
results.push_back(nObj);
}
}
return {results, Status::Ok};
}
void
store(std::shared_ptr<NodeObject> const& object) override
{

View File

@@ -42,7 +42,6 @@
#include <stdexcept>
#include <string>
#include <utility>
#include <vector>
namespace xrpl::NodeStore {
@@ -232,28 +231,6 @@ public:
return status;
}
std::pair<std::vector<std::shared_ptr<NodeObject>>, Status>
fetchBatch(std::vector<uint256> const& hashes) override
{
std::vector<std::shared_ptr<NodeObject>> results;
results.reserve(hashes.size());
for (auto const& h : hashes)
{
std::shared_ptr<NodeObject> nObj;
Status const status = fetch(h, &nObj);
if (status != Status::Ok)
{
results.push_back({});
}
else
{
results.push_back(nObj);
}
}
return {results, Status::Ok};
}
void
doInsert(std::shared_ptr<NodeObject> const& no)
{

View File

@@ -12,8 +12,6 @@
#include <functional>
#include <memory>
#include <string>
#include <utility>
#include <vector>
namespace xrpl::NodeStore {
@@ -52,12 +50,6 @@ public:
return Status::NotFound;
}
std::pair<std::vector<std::shared_ptr<NodeObject>>, Status>
fetchBatch(std::vector<uint256> const& hashes) override
{
return {};
}
void
store(std::shared_ptr<NodeObject> const& object) override
{

View File

@@ -1,12 +1,22 @@
#if XRPL_ROCKSDB_AVAILABLE
#include <xrpl/basics/BasicConfig.h>
#include <xrpl/basics/ByteUtilities.h>
#include <xrpl/basics/Log.h>
#include <xrpl/basics/base_uint.h>
#include <xrpl/basics/contract.h>
#include <xrpl/basics/safe_cast.h>
#include <xrpl/beast/core/CurrentThreadName.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/nodestore/Backend.h>
#include <xrpl/nodestore/Factory.h>
#include <xrpl/nodestore/Manager.h>
#include <xrpl/nodestore/NodeObject.h>
#include <xrpl/nodestore/Scheduler.h>
#include <xrpl/nodestore/Types.h>
#include <xrpl/nodestore/detail/BatchWriter.h>
#include <xrpl/nodestore/detail/DecodedBlob.h>
#include <xrpl/nodestore/detail/EncodedBlob.h>
#include <boost/filesystem/operations.hpp>
#include <boost/filesystem/path.hpp>
@@ -24,27 +34,13 @@
#include <rocksdb/table.h>
#include <rocksdb/write_batch.h>
#include <atomic>
#include <bit>
#include <cstddef>
#include <functional>
#include <memory>
#include <stdexcept>
#include <string>
#include <utility>
#include <vector>
#if XRPL_ROCKSDB_AVAILABLE
#include <xrpl/basics/ByteUtilities.h>
#include <xrpl/basics/contract.h>
#include <xrpl/basics/safe_cast.h>
#include <xrpl/beast/core/CurrentThreadName.h>
#include <xrpl/nodestore/Factory.h>
#include <xrpl/nodestore/Manager.h>
#include <xrpl/nodestore/detail/BatchWriter.h>
#include <xrpl/nodestore/detail/DecodedBlob.h>
#include <xrpl/nodestore/detail/EncodedBlob.h>
#include <atomic>
#include <memory>
namespace xrpl::NodeStore {
@@ -330,28 +326,6 @@ public:
return status;
}
std::pair<std::vector<std::shared_ptr<NodeObject>>, Status>
fetchBatch(std::vector<uint256> const& hashes) override
{
std::vector<std::shared_ptr<NodeObject>> results;
results.reserve(hashes.size());
for (auto const& h : hashes)
{
std::shared_ptr<NodeObject> nObj;
Status const status = fetch(h, &nObj);
if (status != Status::Ok)
{
results.push_back({});
}
else
{
results.push_back(nObj);
}
}
return {results, Status::Ok};
}
void
store(std::shared_ptr<NodeObject> const& object) override
{

View File

@@ -23,7 +23,7 @@ namespace {
//------------------------------------------------------------------------------
// clang-format off
// NOLINTNEXTLINE(readability-identifier-naming)
char const* const versionString = "3.2.0-b6"
char const* const versionString = "3.2.0"
// clang-format on
;

View File

@@ -56,7 +56,7 @@ IOUAmount::fromNumber(Number const& number)
// to normalize, which calls fromNumber
IOUAmount result{};
std::tie(result.mantissa_, result.exponent_) =
number.normalizeToRange(kMinMantissa, kMaxMantissa);
number.normalizeToRange<kMinMantissa, kMaxMantissa>();
return result;
}

View File

@@ -7,6 +7,7 @@
#include <xrpl/beast/hash/uhash.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/protocol/Feature.h>
#include <xrpl/protocol/IOUAmount.h>
#include <xrpl/protocol/STVector256.h>
#include <memory>
@@ -38,15 +39,68 @@ setCurrentTransactionRules(std::optional<Rules> r)
// Make global changes associated with the rules before the value is moved.
// Push the appropriate setting, instead of having the class pull every time
// the value is needed. That could get expensive fast.
bool const enableLargeNumbers =
// If any new conditions with new amendments are added, those amendments must also be added to
// useRulesGuards.
bool const enableVaultNumbers =
!r || (r->enabled(featureSingleAssetVault) || r->enabled(featureLendingProtocol));
Number::setMantissaScale(
enableLargeNumbers ? MantissaRange::MantissaScale::Large
: MantissaRange::MantissaScale::Small);
bool const enableCuspRoundingFix = !r || r->enabled(fixCleanup3_2_0);
XRPL_ASSERT(
!r || useRulesGuards(*r) == (enableCuspRoundingFix || enableVaultNumbers),
"setCurrentTransactionRules : rule decisions match");
// Declare the range this way to keep clang-tidy from complaining
auto const range = [enableCuspRoundingFix, enableVaultNumbers]() {
if (enableVaultNumbers)
{
if (enableCuspRoundingFix)
{
return MantissaRange::MantissaScale::Large;
}
return MantissaRange::MantissaScale::LargeLegacy;
}
return MantissaRange::MantissaScale::Small;
}();
Number::setMantissaScale(range);
*getCurrentTransactionRulesRef() = std::move(r);
}
bool
useRulesGuards(Rules const& rules)
{
// The list of amendments used here - to decide whether to create a RulesGuard - must be a
// superset of the list used to figure out which mantissa scale to use in
// setCurrentTransactionRules. Additional amendments can be added if desired.
//
// As soon as any one of these amendments is retired, this whole function can be removed, along
// with createGuards, and any other callers, and the first set of guards can be created directly
// at the call site, without using optional.
return rules.enabled(fixCleanup3_2_0) || rules.enabled(featureSingleAssetVault) ||
rules.enabled(featureLendingProtocol);
}
void
createGuards(
Rules const& rules,
std::optional<NumberSO>& stNumberSO,
std::optional<CurrentTransactionRulesGuard>& rulesGuard,
std::optional<NumberMantissaScaleGuard>& mantissaScaleGuard)
{
if (useRulesGuards(rules))
{
// raii classes for the current ledger rules.
// fixUniversalNumber predates the rulesGuard and should be replaced.
stNumberSO.emplace(rules.enabled(fixUniversalNumber));
rulesGuard.emplace(rules);
}
else
{
// Without those features enabled, always use the old number rules.
mantissaScaleGuard.emplace(MantissaRange::MantissaScale::Small);
}
}
class Rules::Impl
{
private:

View File

@@ -19,8 +19,10 @@
#include <xrpl/protocol/Protocol.h>
#include <xrpl/protocol/Rules.h>
#include <xrpl/protocol/SField.h>
#include <xrpl/protocol/STArray.h>
#include <xrpl/protocol/STBase.h>
#include <xrpl/protocol/STNumber.h>
#include <xrpl/protocol/STObject.h>
#include <xrpl/protocol/Serializer.h>
#include <xrpl/protocol/SystemParameters.h>
#include <xrpl/protocol/UintTypes.h>
@@ -1222,6 +1224,68 @@ operator-(STAmount const& value)
STAmount::Unchecked{});
}
static bool
hasInvalidAmount(STBase const& field, int depth, beast::Journal j);
static bool
hasInvalidAmount(STObject const& object, int depth, beast::Journal j)
{
return std::ranges::any_of(
object, [&](STBase const& field) { return hasInvalidAmount(field, depth, j); });
}
static bool
hasInvalidAmount(STArray const& array, int depth, beast::Journal j)
{
return std::ranges::any_of(
array, [&](STObject const& object) { return hasInvalidAmount(object, depth, j); });
}
static bool
hasInvalidAmount(STBase const& field, int depth, beast::Journal j)
{
if (depth > 10)
{
JLOG(j.error()) << "hasInvalidAmount: depth exceeds 10";
return true;
}
// Dispatch on the serialized type tag rather than RTTI: this is on the invariant-checking path
// and a dynamic_cast chain over every field of every modified entry is measurably expensive.
// The object-like tags below all denote STObject subclasses (STLedgerEntry, STTx), so the
// downcast is sound; nested fields are only ever plain STI_OBJECT / STI_ARRAY containers.
// safeDowncast keeps a dynamic_cast validity assert in debug builds while compiling to
// static_cast in release.
switch (field.getSType())
{
case STI_AMOUNT: {
auto const& amount = safeDowncast<STAmount const&>(field);
return !isLegalMPT(amount) || !isLegalNet(amount);
}
case STI_OBJECT:
case STI_LEDGERENTRY:
case STI_TRANSACTION:
return hasInvalidAmount(safeDowncast<STObject const&>(field), depth + 1, j);
case STI_ARRAY:
return hasInvalidAmount(safeDowncast<STArray const&>(field), depth + 1, j);
default: {
XRPL_ASSERT(
dynamic_cast<STObject const*>(&field) == nullptr,
"xrpl::hasInvalidAmount : valid object type");
return false;
}
}
}
bool
hasInvalidAmount(STBase const& field, beast::Journal j)
{
return hasInvalidAmount(field, 0, j);
}
//------------------------------------------------------------------------------
//
// Arithmetic

Some files were not shown because too many files have changed in this diff Show More