Merge upstream/ripple/smart-escrow into xrplf/smart-contracts

Upstream rewrote the wasm VM and host-function system (Rust `crates/`
bridged via cxx), collapsed `transactions.macro` onto `TxSettings`, and
moved invariant running from `ApplyContext` to `Transactor`. This merge
resolves those conflicts and the breaks that carried no conflict marker.

Conflict resolutions of note:
- transactions.macro: took upstream's 5-argument `TxSettings` form and
  re-expressed our `emitable` column as `TxSettings::emittance`, a new
  scoped `Emittance` enum in TxSettings.h. `Emitable.cpp` and the
  transaction code generator read the new member.
- sfields.macro: upstream claimed UINT32 75-80, so `sfParameterFlag`
  moves from 80 to 86. The amendment is not live, so no wire break.
- HostFunc.h: took upstream's version, which drops `floatRoot`, and
  re-added the 15 contract virtuals. `setDataNestedObjectField`'s
  parameters are renamed to `(account, key, nestedKey, value)` to match
  the implementation; the wire order is unchanged.
- WasmCommon.h: `SubmitTxnFailure` (-21) and `InvalidState` (-22) join
  upstream's enum and the Rust `host_errors!` table. `Success` is gone;
  the helpers that compared against it now return `expected<void, ...>`.
- Transactor.cpp: kept the emitted-transaction pass, now using
  `checkInvariants(result, fee, InvariantScope::ProtocolOnly)`.

Breaks with no conflict marker:
- `Emitable.cpp` used the 8-argument TRANSACTION macro.
- `STTx::getSeqValue` is gone; use `getSeqProxy().value()`.
- `NetworkOPsImp::subLock_` is now `streamLock_`, held with `scoped_lock`.
- The `LedgerEntryHelpers` namespace is now `ledger_entry_helpers`.
- `keylet::vault` takes a `SeqProxy`. The three contract `ledger_entry`
  parsers were copy-pasted from the vault one and returned vault keylets;
  they now build contract, contract source and contract data keylets from
  their own fields. `contract_hash` is added to jss.

The 15 contract host functions still register through the deleted C++
wrapper layer, so contract bytecode does not run yet. Porting them to the
Rust-declared ABI is the next commit.
This commit is contained in:
Mayukha Vadari
2026-09-15 15:37:08 -04:00
932 changed files with 84511 additions and 47992 deletions

39
.github/actions/cargo-cache/action.yml vendored Normal file
View File

@@ -0,0 +1,39 @@
name: Use cargo artifacts cache
description: >
Cache the cargo build artifacts with rust-cache. Never caches ~/.cargo/bin:
when saving the cache, rust-cache deletes all binaries that were already
present there, which on persistent self-hosted runners wipes the tools
installed by prepare-runner. Harmless on ephemeral runners, but kept
consistent everywhere.
inputs:
workspaces:
description: "Workspaces to cache, as 'workspace -> target' lines."
required: false
default: crates
key:
description: "Additional part of the cache key."
required: false
default: ""
cache-directories:
description: "Additional non-workspace directories to cache."
required: false
default: ""
save-if:
description: >
Condition for saving the cache after the job. Defaults to save only from develop branch
required: false
default: ${{ github.ref == 'refs/heads/develop' }}
runs:
using: composite
steps:
- name: Use cargo artifacts cache
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
cache-bin: "false"
cache-directories: ${{ inputs.cache-directories }}
key: ${{ inputs.key }}
save-if: ${{ inputs.save-if }}
workspaces: ${{ inputs.workspaces }}

View File

@@ -1,44 +0,0 @@
name: Generate build version number
description: "Generate build version number."
outputs:
version:
description: "The generated build version number."
value: ${{ steps.version.outputs.version }}
runs:
using: composite
steps:
# When a tag is pushed, the version is used as-is.
- name: Generate version for tag event
if: ${{ startsWith(github.ref, 'refs/tags/') }}
shell: bash
env:
VERSION: ${{ github.ref_name }}
run: echo "VERSION=${VERSION}" >>"${GITHUB_ENV}"
# When a tag is not pushed, then the version (e.g. 1.2.3-b0) is extracted
# from the BuildInfo.cpp file and the shortened commit hash appended to it.
# We use a plus sign instead of a hyphen because Conan recipe versions do
# not support two hyphens.
- name: Generate version for non-tag event
if: ${{ !startsWith(github.ref, 'refs/tags/') }}
shell: bash
run: |
echo 'Extracting version from BuildInfo.cpp.'
VERSION="$(cat src/libxrpl/protocol/BuildInfo.cpp | grep "versionString =" | awk -F '"' '{print $2}')"
if [[ -z "${VERSION}" ]]; then
echo 'Unable to extract version from BuildInfo.cpp.'
exit 1
fi
echo 'Appending shortened commit hash to version.'
SHA='${{ github.sha }}'
VERSION="${VERSION}+${SHA:0:7}"
echo "VERSION=${VERSION}" >>"${GITHUB_ENV}"
- name: Output version
id: version
shell: bash
run: echo "version=${VERSION}" >>"${GITHUB_OUTPUT}"

44
.github/actions/release-info/action.yml vendored Normal file
View File

@@ -0,0 +1,44 @@
name: Release info
description: "Derive the version, release channel and package release number for this build."
outputs:
version:
description: "The build version number."
value: ${{ steps.version.outputs.version }}
channel:
description: "The release channel this build belongs to."
value: ${{ steps.release_info.outputs.channel }}
pkg_release:
description: "The package release number: 1 for a tag, <run number>.<commit date>git<short commit hash> otherwise."
value: ${{ steps.release_info.outputs.pkg_release }}
runs:
using: composite
steps:
# A tag names its own version. Anything else takes it from BuildInfo.cpp and
# appends the commit hash as build metadata, joined with a plus sign because a
# Conan version cannot contain two hyphens.
- name: Determine version
id: version
shell: bash
env:
IS_TAG: ${{ startsWith(github.ref, 'refs/tags/') }}
REF_NAME: ${{ github.ref_name }}
SHA: ${{ github.sha }}
run: |
if [[ "${IS_TAG}" == "true" ]]; then
version="${REF_NAME}"
else
version="$(awk -F'"' '/versionString =/ { print $2 }' src/libxrpl/protocol/BuildInfo.cpp)"
if [[ -z "${version}" ]]; then
echo "Unable to read versionString from BuildInfo.cpp." >&2
exit 1
fi
version="${version}+${SHA:0:7}"
fi
echo "version=${version}" | tee -a "${GITHUB_OUTPUT}"
- name: Determine release channel and package release
id: release_info
uses: XRPLF/actions/release-info@7cc0e4a8d9d0b838f92c48d312856b190341bbba

View File

@@ -0,0 +1,70 @@
name: Setup Nix environment
description: "Build the flake's CI environment and put its tools on PATH."
# The environment from nix/ci-env.nix, the same one the Linux CI images bake in
# (see nix/docker). Exported onto PATH rather than entered with `nix develop`:
# the composite actions below run plain `bash` and would escape a dev shell.
runs:
using: composite
steps:
- name: Build the CI environment
id: build
shell: bash
env:
# --out-link doubles as a GC root for the length of the job.
OUT_LINK: ${{ runner.temp }}/xrpld-ci-env
run: |
# --extra-experimental-features: flakes may not be on in the runner's nix.conf.
nix --extra-experimental-features "nix-command flakes" \
build .#default --out-link "${OUT_LINK}" --print-build-logs
echo "path=$(readlink -f "${OUT_LINK}")" >>"${GITHUB_OUTPUT}"
- name: Export the environment
shell: bash
env:
ENV_PATH: ${{ steps.build.outputs.path }}
run: |
echo "${ENV_PATH}/bin" >>"${GITHUB_PATH}"
# Already KEY=VALUE per line. See `darwinEnv` in nix/ci-env.nix.
ENV_FILE="${ENV_PATH}/share/xrpld-ci-env/env"
if [ -f "${ENV_FILE}" ]; then
cat "${ENV_FILE}" >>"${GITHUB_ENV}"
fi
# XrplSanity.cmake otherwise rejects a Nix compiler as one that leaked.
echo "XRPL_DEVSHELL=ci-env" >>"${GITHUB_ENV}"
# Unlike the Linux nix images, macOS needs no SSL_CERT_FILE: it has its
# own trust store, and pinning would break TLS to hosts relying on it.
# In RUNNER_TEMP, which the runner empties per job, like the `.conan2`
# prepare-runner hands the system toolchain - but under its own name:
# that Conan is a different version, and the two would migrate each
# other's cache.
echo "CONAN_HOME=${RUNNER_TEMP}/.conan2-nix" >>"${GITHUB_ENV}"
# Config, profiles and remote, exactly as the dev shell sets them up on
# entry; the `setup-conan` action is skipped for this toolchain.
- name: Setup Conan
shell: bash
run: ./conan/init.sh
# `Check tools` runs later but swallows failures; a bad export would just
# build with the system toolchain.
- name: Verify the toolchain resolves into the Nix store
shell: bash
run: |
for tool in clang clang++ cmake ninja conan; do
path="$(command -v "${tool}" || true)"
echo "${tool} -> ${path:-<not found>}"
case "${path}" in
/nix/store/*) ;;
*)
echo "::error::${tool} does not resolve into the Nix store"
exit 1
;;
esac
done

View File

@@ -4,7 +4,8 @@ updates:
directories:
- /
- .github/actions/build-deps/
- .github/actions/generate-version/
- .github/actions/cargo-cache/
- .github/actions/release-info/
- .github/actions/set-compiler-env/
- .github/actions/setup-conan/
schedule:
@@ -19,3 +20,19 @@ updates:
github-actions:
patterns:
- "*"
- package-ecosystem: cargo
directory: /crates
schedule:
interval: weekly
day: monday
time: "04:00"
timezone: Etc/GMT
commit-message:
prefix: "chore: [DEPENDABOT] "
target-branch: develop
open-pull-requests-limit: 10
groups:
rust-dependencies:
patterns:
- "*"

View File

@@ -18,7 +18,7 @@ If too broad, please consider splitting into multiple PRs.
If there is a relevant task or issue, please link it here.
-->
### Context of Change
## Context of Change
<!--
Please include the context of a change.
@@ -29,7 +29,7 @@ If a refactor, how is this better than the previous implementation?
If there is a spec or design document for this feature, please link it here.
-->
### API Impact
## API Impact
<!--
Please check [x] relevant options, delete irrelevant ones.

View File

@@ -1,6 +1,9 @@
benchmarks.libxrpl > xrpl.basics
benchmarks.libxrpl > xrpl.config
benchmarks.libxrpl > xrpl.nodestore
benchmarks.libxrpl > xrpl.protocol
benchmarks.libxrpl > xrpl.protocol_autogen
benchmarks.libxrpl > xrpl.tx
libxrpl.basics > xrpl.basics
libxrpl.conditions > xrpl.basics
libxrpl.conditions > xrpl.conditions

View File

@@ -49,7 +49,7 @@ ${SED_COMMAND} -i -E 's@ripple/xrpld@XRPLF/rippled@g' BUILD.md
${SED_COMMAND} -i -E 's@XRPLF/xrpld@XRPLF/rippled@g' BUILD.md
${SED_COMMAND} -i -E 's@xrpld \(`xrpld`\)@xrpld@g' BUILD.md
${SED_COMMAND} -i -E 's@XRPLF/xrpld@XRPLF/rippled@g' CONTRIBUTING.md
${SED_COMMAND} -i -E 's@XRPLF/xrpld@XRPLF/rippled@g' docs/build/install.md
${SED_COMMAND} -i -E 's@XRPLF/xrpld@XRPLF/rippled@g' docs/install.md
popd
echo "Processing complete."

View File

@@ -77,8 +77,8 @@ ${SED_COMMAND} -i 's/Ripple integrators/XRPL developers/' README.md
${SED_COMMAND} -i 's/sanitizer-configuration-for-rippled/sanitizer-configuration-for-xrpld/' docs/build/sanitizers.md
${SED_COMMAND} -i 's/rippled/xrpld/g' .github/scripts/levelization/README.md
${SED_COMMAND} -i 's/rippled/xrpld/g' .github/scripts/strategy-matrix/generate.py
${SED_COMMAND} -i 's@/rippled@/xrpld@g' docs/build/install.md
${SED_COMMAND} -i 's@github.com/XRPLF/xrpld@github.com/XRPLF/rippled@g' docs/build/install.md
${SED_COMMAND} -i 's@/rippled@/xrpld@g' docs/install.md
${SED_COMMAND} -i 's@github.com/XRPLF/xrpld@github.com/XRPLF/rippled@g' docs/install.md
${SED_COMMAND} -i 's/rippled/xrpld/g' docs/Doxyfile
${SED_COMMAND} -i 's/ripple_basics/basics/' include/xrpl/basics/CountedObject.h
${SED_COMMAND} -i 's/<ripple/<xrpl/' include/xrpl/protocol/AccountID.h

View File

@@ -7,7 +7,12 @@ from pathlib import Path
THIS_DIR = Path(__file__).parent.resolve()
_BASE_CMAKE_ARGS = ["-Dtests=ON", "-Dwerr=ON", "-Dxrpld=ON", "-Dwextra=ON"]
_BASE_CMAKE_ARGS = [
"-Dtests=ON",
"-Dwerr=ON",
"-Dxrpld=ON",
"-Dwextra=ON",
]
# Maps sanitizer names (as used in cmake) to short config-name suffixes.
_SANITIZER_SUFFIX: dict[str, str] = {
@@ -17,6 +22,19 @@ _SANITIZER_SUFFIX: dict[str, str] = {
}
def config_name(
distro: str,
compiler: str,
build_type: str,
arch: str,
suffix: str = "",
sanitizer: str = "",
) -> str:
"""Name a config. Its artifacts are named after it, so packaging reuses this."""
parts = [s for s in [suffix, _SANITIZER_SUFFIX.get(sanitizer, "")] if s]
return "-".join([f"{distro}-{compiler}-{build_type.lower()}-{arch}", *parts])
def get_cmake_args(build_type: str, extra_args: str) -> str:
"""Get the full list of CMake arguments for a config."""
args = _BASE_CMAKE_ARGS.copy()
@@ -31,17 +49,27 @@ def get_cmake_args(build_type: str, extra_args: str) -> str:
# Every config must declare 'minimal'. Minimal configs form the reduced matrix
# built for pull requests by default; the full matrix adds the rest. Packaging
# configs declare it too, but packaging is gated in the workflow, not by it.
# built for pull requests by default; the full matrix adds the rest.
#
# Configs may also opt into 'benchmark' to smoke-run the benchmarks. Note that
# the flag applies to every entry a config expands into, so only set it on
# configs that expand to a single combination.
# Configs may also opt into 'benchmark' to smoke-run the benchmarks, or carry a
# 'package' map to be packaged as well. Note that either applies to every entry
# a config expands into, so only set them on configs that expand to a single
# combination.
@dataclasses.dataclass
class PackageConfig:
"""The 'package' map of a config whose binaries are also packaged."""
type: str # "deb" or "rpm"; has to match what the image provides
# The packaging container image: a vanilla distro image, not the nix image
# the config itself builds in.
image: str
@dataclasses.dataclass
class LinuxConfig:
"""One entry in linux.json's 'configs' or 'package_configs' arrays."""
"""One entry in a linux.json 'configs' array."""
compiler: list[str]
build_type: list[str]
@@ -51,7 +79,11 @@ class LinuxConfig:
sanitizers: list[str] = dataclasses.field(default_factory=list)
suffix: str = ""
extra_cmake_args: str = ""
image: str = "" # only used by package_configs entries
package: PackageConfig | None = None # set to also package this config
def __post_init__(self) -> None:
if isinstance(self.package, dict):
self.package = PackageConfig(**self.package)
@dataclasses.dataclass
@@ -60,22 +92,16 @@ class LinuxFile:
image_tag: str
configs: dict[str, list[LinuxConfig]] # distro → configs
package_configs: dict[str, list[LinuxConfig]] # distro → packaging configs
@classmethod
def load(cls, path: Path) -> "LinuxFile":
data = json.loads(path.read_text())
def parse(section: dict) -> dict[str, list[LinuxConfig]]:
return {
distro: [LinuxConfig(**c) for c in cfgs]
for distro, cfgs in section.items()
}
return cls(
image_tag=data["image_tag"],
configs=parse(data["configs"]),
package_configs=parse(data.get("package_configs", {})),
configs={
distro: [LinuxConfig(**c) for c in cfgs]
for distro, cfgs in data["configs"].items()
},
)
@@ -88,6 +114,9 @@ class PlatformConfig:
build_only: bool = False # if true, skip tests (e.g. macos/Windows Debug)
benchmark: bool = False # if true, smoke-run the benchmarks after testing
extra_cmake_args: str = ""
# "" is the runner's system compiler, "nix" the flake's CI environment.
# macOS only: Linux always builds in a Nix image, Windows has no Nix.
toolchain: str = ""
def __post_init__(self) -> None:
if isinstance(self.build_type, str):
@@ -137,6 +166,7 @@ class MatrixEntry:
sanitizers: str
image: str = "" # container image; empty for macOS/Windows (runs natively)
compiler: str = "" # compiler name ("gcc" or "clang"); empty for macOS/Windows
toolchain: str = "" # "nix" for the flake's CI environment; see PlatformConfig
@dataclasses.dataclass
@@ -146,7 +176,7 @@ class PackagingEntry:
xrpld_artifact_name: str
validator_keys_artifact_name: str
image: str
distro: str # e.g. "debian" or "rhel"; drives package-format-specific steps
package_type: str # "deb" or "rpm"; drives the format-specific steps
# ---------------------------------------------------------------------------
@@ -187,13 +217,9 @@ def expand_linux_matrix(linux: LinuxFile, minimal: bool) -> list[MatrixEntry]:
effective_sanitizers,
effective_archs.items(),
):
name = f"{distro}-{compiler}-{build_type.lower()}-{arch}"
suffix_parts = [
s for s in [cfg.suffix, _SANITIZER_SUFFIX.get(sanitizer, "")] if s
]
if suffix_parts:
name += "-" + "-".join(suffix_parts)
name = config_name(
distro, compiler, build_type, arch, cfg.suffix, sanitizer
)
entries.append(
MatrixEntry(
config_name=name,
@@ -213,27 +239,33 @@ def expand_linux_matrix(linux: LinuxFile, minimal: bool) -> list[MatrixEntry]:
def expand_linux_packaging(linux: LinuxFile) -> list[PackagingEntry]:
"""Generate the packaging matrix from a LinuxFile's package_configs section.
"""Generate the packaging matrix from the configs that carry a 'package' map.
Packaging uses vanilla distro images (debian:bookworm, ubi9, …) instead of
the nix-based build images, because deb/rpm tooling (debhelper, rpm-build)
is taken from the distro's archive rather than from nixpkgs. Each config
entry carries its own 'image'.
Packaging consumes the binaries that config's build job uploaded, so the
artifact names come from the same config name, and a packaged config is one
that passes -Dvalidator_keys=ON.
The artifact names must match what the build job uploads: one artifact per
binary, each named after the build config.
Packaging itself runs in vanilla distro images (debian:trixie, almalinux:10)
instead of the nix-based build images, because deb/rpm tooling (debhelper,
rpm-build) is taken from the distro's archive rather than from nixpkgs.
"""
entries = []
for distro, configs in linux.package_configs.items():
for distro, configs in linux.configs.items():
for cfg in configs:
for compiler, build_type in itertools.product(cfg.compiler, cfg.build_type):
config_name = f"{distro}-{compiler}-{build_type.lower()}-amd64"
if cfg.package is None:
continue
for compiler, build_type, arch in itertools.product(
cfg.compiler, cfg.build_type, cfg.arch
):
# The packaging workflow hardcodes an amd64 runner.
assert arch == "amd64", f"cannot package {distro} on {arch}"
name = config_name(distro, compiler, build_type, arch, cfg.suffix)
entries.append(
PackagingEntry(
xrpld_artifact_name=f"xrpld-{config_name}",
validator_keys_artifact_name=f"validator-keys-{config_name}",
image=cfg.image,
distro=distro,
xrpld_artifact_name=f"xrpld-{name}",
validator_keys_artifact_name=f"validator-keys-{name}",
image=cfg.package.image,
package_type=cfg.package.type,
)
)
@@ -253,9 +285,12 @@ def expand_platform_matrix(pf: PlatformFile, minimal: bool) -> list[MatrixEntry]
if minimal and not cfg.minimal:
continue
for build_type in cfg.build_type:
name = f"{platform_name}-{arch}-{build_type.lower()}"
if cfg.toolchain:
name += f"-{cfg.toolchain}"
entries.append(
MatrixEntry(
config_name=f"{platform_name}-{arch}-{build_type.lower()}",
config_name=name,
cmake_args=get_cmake_args(build_type, cfg.extra_cmake_args),
cmake_target="install" if is_windows else "all",
build_only=cfg.build_only,
@@ -263,6 +298,7 @@ def expand_platform_matrix(pf: PlatformFile, minimal: bool) -> list[MatrixEntry]
build_type=build_type,
architecture=Architecture(platform=pf.platform, runner=pf.runner),
sanitizers="",
toolchain=cfg.toolchain,
)
)
return entries

View File

@@ -1,5 +1,5 @@
{
"image_tag": "sha-fecfc0c",
"image_tag": "sha-060957e",
"configs": {
"ubuntu": [
{
@@ -71,7 +71,11 @@
"build_type": ["Release"],
"arch": ["amd64"],
"minimal": false,
"extra_cmake_args": "-Dvalidator_keys=ON"
"extra_cmake_args": "-Dvalidator_keys=ON",
"package": {
"type": "deb",
"image": "ghcr.io/xrplf/xrpld/packaging-debian:sha-49cdc10"
}
}
],
@@ -81,28 +85,11 @@
"build_type": ["Release"],
"arch": ["amd64"],
"minimal": false,
"extra_cmake_args": "-Dvalidator_keys=ON"
}
]
},
"package_configs": {
"debian": [
{
"compiler": ["gcc"],
"build_type": ["Release"],
"arch": ["amd64"],
"minimal": false,
"image": "ghcr.io/xrplf/xrpld/packaging-debian:sha-577d745"
}
],
"rhel": [
{
"compiler": ["gcc"],
"build_type": ["Release"],
"arch": ["amd64"],
"minimal": false,
"image": "ghcr.io/xrplf/xrpld/packaging-rhel:sha-577d745"
"extra_cmake_args": "-Dvalidator_keys=ON",
"package": {
"type": "rpm",
"image": "ghcr.io/xrplf/xrpld/packaging-rhel:sha-49cdc10"
}
}
]
}

View File

@@ -12,6 +12,19 @@
"extra_cmake_args": "-DCMAKE_POLICY_VERSION_MINIMUM=3.5",
"build_only": true,
"minimal": false
},
{
"build_type": "Release",
"extra_cmake_args": "-DCMAKE_POLICY_VERSION_MINIMUM=3.5",
"toolchain": "nix",
"minimal": false
},
{
"build_type": "Debug",
"extra_cmake_args": "-DCMAKE_POLICY_VERSION_MINIMUM=3.5",
"toolchain": "nix",
"build_only": true,
"minimal": false
}
]
}

View File

@@ -5,14 +5,13 @@ on:
branches:
- develop
paths:
- ".github/workflows/build-nix-images.yml"
- "flake.nix"
- "flake.lock"
- "rust-toolchain.toml"
- "nix/**"
- "!nix/docker/README.md"
- "!nix/devshell.nix"
- "bin/check-tools.sh"
- "!nix/check-tools/**"
- "bin/default-loader-path.sh"
- "bin/install-sanitizer-libs.sh"
pull_request:
@@ -24,6 +23,7 @@ on:
- "nix/**"
- "!nix/docker/README.md"
- "!nix/devshell.nix"
- "!nix/check-tools/**"
- "bin/check-tools.sh"
- "bin/default-loader-path.sh"
- "bin/install-sanitizer-libs.sh"
@@ -58,7 +58,7 @@ jobs:
base_image: debian:bookworm
- name: rhel
base_image: registry.access.redhat.com/ubi9/ubi:latest
uses: XRPLF/actions/.github/workflows/build-multiarch-image.yml@9e7e4e80af9e684c116b38369add8eea64451f32
uses: XRPLF/actions/.github/workflows/build-multiarch-image.yml@034e87065fcd0100214cf0672923bd38d193cf78
with:
image_name: xrpld/nix-${{ matrix.distro.name }}
dockerfile: nix/docker/Dockerfile

View File

@@ -6,13 +6,13 @@ on:
- develop
paths:
- ".github/workflows/build-packaging-images.yml"
- "package/Dockerfile"
- "package/install-packaging-tools.sh"
- "bin/install-packaging-tools.sh"
- "package/docker/**"
pull_request:
paths:
- ".github/workflows/build-packaging-images.yml"
- "package/Dockerfile"
- "package/install-packaging-tools.sh"
- "bin/install-packaging-tools.sh"
- "package/docker/**"
workflow_dispatch:
concurrency:
@@ -33,14 +33,17 @@ jobs:
strategy:
fail-fast: false
matrix:
# Newest of each distro: these images only wrap pre-built binaries, so
# they set no floor for consumers. build_pkg.py pins the RPM dist tag.
distro:
- name: debian
base_image: debian:bookworm
base_image: debian:trixie
# AlmaLinux rather than UBI, which does not ship rpm-sign.
- name: rhel
base_image: registry.access.redhat.com/ubi9/ubi:latest
uses: XRPLF/actions/.github/workflows/build-multiarch-image.yml@9e7e4e80af9e684c116b38369add8eea64451f32
base_image: almalinux:10
uses: XRPLF/actions/.github/workflows/build-multiarch-image.yml@034e87065fcd0100214cf0672923bd38d193cf78
with:
image_name: xrpld/packaging-${{ matrix.distro.name }}
dockerfile: package/Dockerfile
dockerfile: package/docker/Dockerfile
base_image: ${{ matrix.distro.base_image }}
push: ${{ github.event_name == 'push' }}

View File

@@ -30,7 +30,7 @@ jobs:
permissions:
contents: read
packages: write
uses: XRPLF/actions/.github/workflows/build-multiarch-image.yml@9e7e4e80af9e684c116b38369add8eea64451f32
uses: XRPLF/actions/.github/workflows/build-multiarch-image.yml@034e87065fcd0100214cf0672923bd38d193cf78
with:
image_name: xrpld/pre-commit
dockerfile: bin/pre-commit/Dockerfile

80
.github/workflows/cargo-audit.yml vendored Normal file
View File

@@ -0,0 +1,80 @@
name: Cargo audit
on:
schedule:
# 06:32 UTC every Monday.
- cron: "32 6 * * 1"
push:
branches:
- "develop"
- "release/*"
paths:
- "crates/**/Cargo.toml"
- "crates/Cargo.lock"
- ".github/workflows/cargo-audit.yml"
pull_request:
paths:
- "crates/**/Cargo.toml"
- "crates/Cargo.lock"
- ".github/workflows/cargo-audit.yml"
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
defaults:
run:
shell: bash
working-directory: crates
permissions:
contents: read
jobs:
audit:
runs-on: ubuntu-latest
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-060957e
permissions:
contents: read
# Needed to open an issue on scheduled failures.
issues: write
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Run cargo audit
id: audit
continue-on-error: true
run: |
set -o pipefail
cargo audit | tee /tmp/cargo-audit.txt
- name: Prepare issue body
if: ${{ steps.audit.outcome != 'success' && github.event_name == 'schedule' }}
run: |
{
echo "## \`cargo audit\` found advisories"
echo
echo '```'
cat /tmp/cargo-audit.txt
echo '```'
echo
echo "---"
echo "*This issue was automatically created by the cargo-audit workflow.*"
} >/tmp/cargo-audit-issue.md
- name: Create issue
if: ${{ steps.audit.outcome != 'success' && github.event_name == 'schedule' }}
uses: XRPLF/actions/create-issue@2b8bc36af85b88bca0dd7bfac2e2dc05f94ad712
with:
title: "cargo audit found vulnerabilities"
body_file: /tmp/cargo-audit-issue.md
labels: "Bug,Security"
- name: Fail if advisories were found
if: ${{ steps.audit.outcome != 'success' }}
run: |
echo "cargo audit found advisories!"
cat /tmp/cargo-audit.txt
exit 1

View File

@@ -79,7 +79,7 @@ jobs:
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Prepare runner
uses: XRPLF/actions/prepare-runner@c00c22ada3bd6bcda48fcb0d62fbbab49fec8a0f
uses: XRPLF/actions/prepare-runner@b3e255d74d785d053e4903da8ac90983cd7d9e82
with:
enable_ccache: false

View File

@@ -77,24 +77,28 @@ jobs:
# Keep the paths below in sync with those in `on-trigger.yml`.
.github/actions/build-deps/**
.github/actions/generate-version/**
.github/actions/release-info/**
.github/actions/setup-conan/**
.github/actions/setup-nix-env/**
.github/scripts/strategy-matrix/**
.github/workflows/reusable-build-test-config.yml
.github/workflows/reusable-build-test.yml
.github/workflows/reusable-check-autogen.yml
.github/workflows/reusable-clang-tidy.yml
.github/workflows/reusable-package.yml
.github/workflows/reusable-rust.yml
.github/workflows/reusable-strategy-matrix.yml
.github/workflows/reusable-test.yml
.github/workflows/reusable-upload-recipe.yml
.clang-tidy
.codecov.yml
bin/check-nix-store-refs.sh
bin/check-tools.sh
bin/default-loader-path.sh
cfg/**
cmake/**
conan/**
crates/**
external/**
include/**
src/**
@@ -102,6 +106,9 @@ jobs:
CMakeLists.txt
conanfile.py
conan.lock
flake.lock
flake.nix
nix/**
LICENSE.md
package/**
README.md
@@ -168,6 +175,13 @@ jobs:
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
rust:
needs: should-run
if: ${{ needs.should-run.outputs.go == 'true' }}
uses: ./.github/workflows/reusable-rust.yml
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
package:
needs: [should-run, build-test]
# Packaging consumes the debian/rhel release binaries, which are only built
@@ -211,6 +225,7 @@ jobs:
- check-rename
- clang-tidy
- build-test
- rust
- package
- upload-recipe
- notify-clio

View File

@@ -1,5 +1,9 @@
# This workflow uploads the libxrpl recipe to the Conan remote and builds
# release packages when a versioned tag is pushed.
# When a versioned tag is pushed, this workflow:
#
# - uploads the libxrpl recipe to the Conan remote
# - builds and tests the release binaries
# - builds the DEB and RPM packages
# - publishes those packages to the XRPLF package repositories
name: Tag
on:
@@ -24,7 +28,7 @@ jobs:
remote_password: ${{ secrets.NEXUS_REMOTE_PASSWORD }}
build-test:
if: ${{ github.repository == 'XRPLF/rippled' }}
if: ${{ github.repository_owner == 'XRPLF' }}
uses: ./.github/workflows/reusable-build-test.yml
strategy:
fail-fast: true
@@ -37,6 +41,12 @@ jobs:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
package:
if: ${{ github.repository == 'XRPLF/rippled' }}
if: ${{ github.repository_owner == 'XRPLF' }}
needs: build-test
uses: ./.github/workflows/reusable-package.yml
with:
publish: true
secrets:
remote_username: ${{ secrets.NEXUS_REMOTE_USERNAME }}
remote_password: ${{ secrets.NEXUS_REMOTE_PASSWORD }}
signing_key: ${{ secrets.NEXUS_PACKAGES_PRIVATE_KEY }}

View File

@@ -15,24 +15,28 @@ on:
# Keep the paths below in sync with those in `on-pr.yml`.
- ".github/actions/build-deps/**"
- ".github/actions/generate-version/**"
- ".github/actions/release-info/**"
- ".github/actions/setup-conan/**"
- ".github/actions/setup-nix-env/**"
- ".github/scripts/strategy-matrix/**"
- ".github/workflows/reusable-build-test-config.yml"
- ".github/workflows/reusable-build-test.yml"
- ".github/workflows/reusable-check-autogen.yml"
- ".github/workflows/reusable-clang-tidy.yml"
- ".github/workflows/reusable-package.yml"
- ".github/workflows/reusable-rust.yml"
- ".github/workflows/reusable-strategy-matrix.yml"
- ".github/workflows/reusable-test.yml"
- ".github/workflows/reusable-upload-recipe.yml"
- ".clang-tidy"
- ".codecov.yml"
- "bin/check-nix-store-refs.sh"
- "bin/check-tools.sh"
- "bin/default-loader-path.sh"
- "cfg/**"
- "cmake/**"
- "conan/**"
- "crates/**"
- "external/**"
- "include/**"
- "src/**"
@@ -40,6 +44,9 @@ on:
- "CMakeLists.txt"
- "conanfile.py"
- "conan.lock"
- "flake.lock"
- "flake.nix"
- "nix/**"
- "LICENSE.md"
- "package/**"
- "README.md"
@@ -96,6 +103,11 @@ jobs:
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
rust:
uses: ./.github/workflows/reusable-rust.yml
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
upload-recipe:
needs: build-test
# Only run when pushing to the develop branch.
@@ -108,3 +120,11 @@ jobs:
package:
needs: build-test
uses: ./.github/workflows/reusable-package.yml
with:
# Packages are built on every trigger; only develop pushes in XRPLF/rippled
# publish them, matching upload-recipe above.
publish: ${{ github.repository == 'XRPLF/rippled' && github.event_name == 'push' && github.ref == 'refs/heads/develop' }}
secrets:
remote_username: ${{ secrets.NEXUS_REMOTE_USERNAME }}
remote_password: ${{ secrets.NEXUS_REMOTE_PASSWORD }}
signing_key: ${{ secrets.NEXUS_PACKAGES_PRIVATE_KEY }}

View File

@@ -14,7 +14,7 @@ on:
jobs:
# Call the workflow in the XRPLF/actions repo that runs the pre-commit hooks.
run-hooks:
uses: XRPLF/actions/.github/workflows/pre-commit.yml@3ba08d6ddf114092891d48491fc2e26c3ba15552
uses: XRPLF/actions/.github/workflows/pre-commit.yml@279ec358f4a1be4088be3e024b07916fa97c75b6
with:
runs_on: ubuntu-latest
container: '{ "image": "ghcr.io/xrplf/xrpld/pre-commit:sha-f56b79f" }'
container: '{ "image": "ghcr.io/xrplf/xrpld/pre-commit:sha-473fe44" }'

View File

@@ -41,13 +41,13 @@ env:
jobs:
build:
runs-on: ubuntu-latest
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-fecfc0c
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-060957e
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Prepare runner
uses: XRPLF/actions/prepare-runner@c00c22ada3bd6bcda48fcb0d62fbbab49fec8a0f
uses: XRPLF/actions/prepare-runner@b3e255d74d785d053e4903da8ac90983cd7d9e82
with:
enable_ccache: false
@@ -91,4 +91,4 @@ jobs:
steps:
- name: Deploy to GitHub Pages
id: deploy
uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0
uses: actions/deploy-pages@368f82528645a54fb793d4d04e342629a3f51346 # v5.0.1

View File

@@ -69,6 +69,12 @@ on:
type: string
default: ""
toolchain:
description: 'Where the toolchain comes from ("nix" to build the flake CI environment on the runner, empty for the system one). macOS only: Linux always builds in a Nix image, and Nix has no Windows support.'
required: false
type: string
default: ""
secrets:
CODECOV_TOKEN:
description: "The Codecov token to use for uploading coverage reports."
@@ -111,6 +117,9 @@ jobs:
VOIDSTAR_ENABLED: ${{ contains(inputs.cmake_args, '-Dvoidstar=ON') }}
VALIDATOR_KEYS_ENABLED: ${{ contains(inputs.cmake_args, '-Dvalidator_keys=ON') }}
SANITIZERS_ENABLED: ${{ inputs.sanitizers != '' }}
# The binaries reusable-package.yml consumes. A private repository skips
# them except on a tag push, which is what produces its release packages.
PACKAGING_ARTIFACTS_ENABLED: ${{ github.event.repository.visibility == 'public' || startsWith(github.ref, 'refs/tags/') }}
steps:
- name: Cleanup workspace (macOS and Windows)
if: ${{ runner.os == 'macOS' || runner.os == 'Windows' }}
@@ -120,10 +129,15 @@ jobs:
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Prepare runner
uses: XRPLF/actions/prepare-runner@c00c22ada3bd6bcda48fcb0d62fbbab49fec8a0f
uses: XRPLF/actions/prepare-runner@b3e255d74d785d053e4903da8ac90983cd7d9e82
with:
enable_ccache: ${{ inputs.ccache_enabled }}
# Before any step that uses a build tool, composite actions included.
- name: Setup Nix environment
if: ${{ inputs.toolchain == 'nix' }}
uses: ./.github/actions/setup-nix-env
- name: Set ccache log file
if: ${{ inputs.ccache_enabled && runner.debug == '1' }}
run: echo "CCACHE_LOGFILE=${{ runner.temp }}/ccache.log" >>"${GITHUB_ENV}"
@@ -148,7 +162,21 @@ jobs:
with:
compiler: ${{ inputs.compiler }}
- name: Use cargo artifacts cache
uses: ./.github/actions/cargo-cache
with:
cache-directories: ${{ env.BUILD_DIR }}/corrosion
key: ${{ inputs.config_name }}
# two workspaces here because build artifacts are located in 2 places:
# - crates/target when cargo is called directly
# - build/cargo when cargo is called by cmake
workspaces: |
crates
crates -> ${{ runner.os == 'Windows' && format('../{0}/x64/{1}/cargo', env.BUILD_DIR, inputs.build_type) || format('../{0}/cargo', env.BUILD_DIR) }}
# `setup-nix-env` already did this for the Nix toolchain.
- name: Setup Conan
if: ${{ inputs.toolchain != 'nix' }}
env:
SANITIZERS: ${{ inputs.sanitizers }}
uses: ./.github/actions/setup-conan
@@ -212,6 +240,24 @@ jobs:
--target "${CMAKE_TARGET}" \
2>&1 | tee "${GITHUB_WORKSPACE}/build.log"
# Nothing may reference the store, so whole trees are checked - the Conan
# cache included, since what it holds is what gets uploaded and reused.
- name: Check the build output for Nix store references (Nix toolchain)
if: ${{ inputs.toolchain == 'nix' }}
run: ./bin/check-nix-store-refs.sh "${BUILD_DIR}"
- name: Check the Conan cache for Nix store references (Nix toolchain)
if: ${{ inputs.toolchain == 'nix' }}
run: ./bin/check-nix-store-refs.sh "${CONAN_HOME}"
# Only what PatchNixBinary.cmake retargets: the toolchain in the Linux
# images always references the store. Same condition it uses.
- name: Check for Nix store references (Linux)
if: ${{ runner.os == 'Linux' && env.SANITIZERS_ENABLED == 'false' }}
run: |
./bin/check-nix-store-refs.sh "${BUILD_DIR}/xrpld"
./bin/check-nix-store-refs.sh "${BUILD_DIR}/xrpl_tests"
- name: Show ccache statistics
if: ${{ inputs.ccache_enabled }}
run: |
@@ -222,7 +268,7 @@ jobs:
fi
- name: Upload the binary (Linux)
if: ${{ github.event.repository.visibility == 'public' && runner.os == 'Linux' }}
if: ${{ env.PACKAGING_ARTIFACTS_ENABLED == 'true' && runner.os == 'Linux' }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: xrpld-${{ inputs.config_name }}
@@ -236,7 +282,7 @@ jobs:
run: ./validator-keys --unittest
- name: Upload the validator-keys binary
if: ${{ github.event.repository.visibility == 'public' && env.VALIDATOR_KEYS_ENABLED == 'true' }}
if: ${{ env.PACKAGING_ARTIFACTS_ENABLED == 'true' && env.VALIDATOR_KEYS_ENABLED == 'true' }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: validator-keys-${{ inputs.config_name }}
@@ -323,6 +369,14 @@ jobs:
LD_PRELOAD="$PRELOAD" ./xrpld --unittest --unittest-jobs "${BUILD_NPROC}" 2>&1 | tee "${GITHUB_WORKSPACE}/unittest.log"
- name: Run Rust tests
if: ${{ !inputs.build_only }}
working-directory: crates
# `xrpl-wasm-vm-ffi` is left out on Windows: its tests link as an executable, and
# MSVC - unlike the Unix linkers - will not dead-strip the never-called cxx wrappers
# whose C++ shims only the CMake build defines. The other runners cover these tests.
run: cargo nextest run --workspace --all-features --locked --no-tests=warn ${{ runner.os == 'Windows' && '--exclude xrpl-wasm-vm-ffi' || '' }}
# Smoke-run every benchmark module with a single repetition to confirm the
# benchmarks still build and execute. This is a correctness check, not a
# performance measurement, so there is nothing to gain from repeating it
@@ -394,6 +448,7 @@ jobs:
disable_telem: true
fail_ci_if_error: true
files: ${{ env.BUILD_DIR }}/coverage.xml
flags: cpp
plugins: noop
token: ${{ secrets.CODECOV_TOKEN }}
verbose: true

View File

@@ -51,5 +51,6 @@ jobs:
config_name: ${{ matrix.config_name }}
sanitizers: ${{ matrix.sanitizers }}
compiler: ${{ matrix.compiler || '' }}
toolchain: ${{ matrix.toolchain || '' }}
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

View File

@@ -27,14 +27,14 @@ jobs:
determine-files:
permissions:
contents: read
uses: XRPLF/actions/.github/workflows/determine-tidy-files.yml@d041ac9f1fa9f07a4ba335eb4c1c82233fb3fef6
uses: XRPLF/actions/.github/workflows/determine-tidy-files.yml@70145243b905dc3e040a61d39c00e178cfb96f71
run-clang-tidy:
name: Run clang tidy
needs: [determine-files]
if: ${{ needs.determine-files.outputs.cpp_changed_files != '' || needs.determine-files.outputs.need_full_run == 'true' }}
runs-on: ["self-hosted", "Linux", "X64", "heavy"]
container: "ghcr.io/xrplf/xrpld/nix-debian:sha-fecfc0c"
container: "ghcr.io/xrplf/xrpld/nix-debian:sha-060957e"
permissions:
contents: read
issues: write
@@ -43,7 +43,7 @@ jobs:
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Prepare runner
uses: XRPLF/actions/prepare-runner@c00c22ada3bd6bcda48fcb0d62fbbab49fec8a0f
uses: XRPLF/actions/prepare-runner@b3e255d74d785d053e4903da8ac90983cd7d9e82
with:
enable_ccache: false
@@ -59,6 +59,12 @@ jobs:
with:
compiler: ${{ env.COMPILER }}
- name: Use cargo artifacts cache
uses: ./.github/actions/cargo-cache
with:
cache-directories: ${{ env.BUILD_DIR }}/corrosion
workspaces: crates -> ../${{ env.BUILD_DIR }}/cargo
- name: Setup Conan
uses: ./.github/actions/setup-conan
@@ -82,11 +88,10 @@ jobs:
-Dverify_headers=ON \
..
# clang-tidy needs headers generated from proto files
- name: Build libxrpl.libpb
- name: Build clang-tidy prerequisites
working-directory: ${{ env.BUILD_DIR }}
run: |
ninja -j ${{ steps.nproc.outputs.nproc }} xrpl.libpb
ninja -j ${{ steps.nproc.outputs.nproc }} tidy_prerequisites
- name: Run clang tidy
id: run_clang_tidy

View File

@@ -1,17 +1,40 @@
# Build Linux packages (DEB and RPM) from pre-built binary artifacts (xrpld and
# validator-keys). Discovers which configurations to package from linux.json
# (configs in "package_configs") and fans out one job per distro. Only
# linux/amd64 is supported; the runner is hardcoded in the job below.
# Build, verify and publish Linux packages from the pre-built xrpld and
# validator-keys artifacts, in three stages:
#
# - 'package' builds and signs one format per config that carries a "package"
# map in linux.json; that map names the container image and the format
# - 'test-install' installs what was built on a range of distros and runs the
# binaries there, so a package that cannot be installed never reaches Nexus
# - 'publish' uploads with the image's publish_pkg.py, doing a --dry-run
# unless 'publish: true'
#
# Only linux/amd64 is supported; the runner is hardcoded in the jobs below.
name: Package
on:
workflow_call:
inputs:
pkg_release:
description: "Package release number. Increment when repackaging the same executable."
publish:
description: "Whether to publish the packages after building them."
required: false
type: boolean
default: false
nexus_url:
description: "The base URL of the Nexus instance hosting the deb and rpm repositories."
required: false
type: string
default: "1"
default: https://packages.xrplf.org
secrets:
remote_username:
description: "The username of a Nexus account with write access to the repositories."
required: false
remote_password:
description: "The password or token for that Nexus account."
required: false
signing_key:
description: "Armoured PGP private key used to sign the RPMs. Required when publishing."
required: false
defaults:
run:
@@ -19,6 +42,7 @@ defaults:
env:
BUILD_DIR: build
PACKAGE_DIR: packages
jobs:
generate-matrix:
@@ -41,7 +65,7 @@ jobs:
package:
needs: [generate-matrix]
if: ${{ github.event.repository.visibility == 'public' }}
if: ${{ github.event.repository.visibility == 'public' || startsWith(github.ref, 'refs/tags/') }}
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.generate-matrix.outputs.matrix) }}
@@ -50,12 +74,17 @@ jobs:
contents: read
runs-on: ["self-hosted", "Linux", "X64", "heavy"]
container: ${{ matrix.image }}
timeout-minutes: 30
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Prepare runner
uses: XRPLF/actions/prepare-runner@b3e255d74d785d053e4903da8ac90983cd7d9e82
with:
enable_ccache: false
- name: Download pre-built xrpld binary
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
@@ -71,17 +100,210 @@ jobs:
- name: Make binaries executable
run: chmod +x "${BUILD_DIR}/xrpld" "${BUILD_DIR}/validator-keys"
- name: Determine release info
id: release_info
uses: ./.github/actions/release-info
- name: Build package
env:
PKG_RELEASE: ${{ inputs.pkg_release }}
run: ./package/build_pkg.sh
PACKAGE_TYPE: ${{ matrix.package_type }}
PKG_RELEASE: ${{ steps.release_info.outputs.pkg_release }}
CHANNEL: ${{ steps.release_info.outputs.channel }}
run: |
./package/build_pkg.py \
--package-type "${PACKAGE_TYPE}" \
--build-dir "${BUILD_DIR}" \
--pkg-release "${PKG_RELEASE}" \
--channel "${CHANNEL}"
# Before the upload, so the artifact, the tested package and the published
# package are the same bytes.
- name: Sign RPM
if: ${{ inputs.publish && matrix.package_type == 'rpm' }}
env:
PKG_SIGNING_KEY: ${{ secrets.signing_key }}
run: ./package/sign_rpm.py --package-dir "${BUILD_DIR}"
# Split from the debug symbols, which are an order of magnitude larger, so
# that test-install downloads only what it installs.
- name: Upload package artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ matrix.xrpld_artifact_name }}-pkg
path: |
${{ env.BUILD_DIR }}/debbuild/*.deb
${{ env.BUILD_DIR }}/debbuild/*.ddeb
${{ env.BUILD_DIR }}/rpmbuild/RPMS/**/*.rpm
${{ env.BUILD_DIR }}/debbuild/xrpld_*.deb
${{ env.BUILD_DIR }}/rpmbuild/RPMS/**/xrpld-[0-9]*.rpm
if-no-files-found: error
- name: Upload debug symbol artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ matrix.xrpld_artifact_name }}-pkg-debug
path: |
${{ env.BUILD_DIR }}/debbuild/xrpld-dbgsym_*.deb
${{ env.BUILD_DIR }}/debbuild/xrpld-dbgsym_*.ddeb
${{ env.BUILD_DIR }}/rpmbuild/RPMS/**/xrpld-debuginfo-*.rpm
if-no-files-found: error
# Every distro family the packages target, oldest release first, so both ends
# of the dependency range they declare are exercised.
test-install:
needs: [package]
strategy:
fail-fast: false
matrix:
include:
- package_type: deb
image: debian:11
- package_type: deb
image: debian:12
- package_type: deb
image: debian:13
- package_type: deb
image: ubuntu:20.04
- package_type: deb
image: ubuntu:22.04
- package_type: deb
image: ubuntu:24.04
- package_type: deb
image: ubuntu:26.04
- package_type: rpm
image: almalinux:9
- package_type: rpm
image: almalinux:10
- package_type: rpm
image: rockylinux/rockylinux:9
- package_type: rpm
image: rockylinux/rockylinux:10
- package_type: rpm
image: registry.access.redhat.com/ubi9/ubi
- package_type: rpm
image: registry.access.redhat.com/ubi10/ubi
name: "install ${{ matrix.package_type }} on ${{ matrix.image }}"
permissions:
contents: read
runs-on: ubuntu-latest
container: ${{ matrix.image }}
timeout-minutes: 5
steps:
# Both formats land in one directory; the step below picks its own by
# extension, so this stays independent of the artifact names.
- name: Download package artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: "*-pkg"
merge-multiple: true
path: ${{ env.PACKAGE_DIR }}
- name: Find the package
id: find
env:
PACKAGE_TYPE: ${{ matrix.package_type }}
run: |
package="$(find "${PACKAGE_DIR}" -type f -name "*.${PACKAGE_TYPE}" -print -quit)"
test -n "${package}" || {
echo "no .${PACKAGE_TYPE} found in ${PACKAGE_DIR}" >&2
exit 1
}
echo "package=${package}" >>"${GITHUB_OUTPUT}"
# Debian 11 went end-of-life on 2026-08-31
# (https://www.debian.org/News/2026/20260831) and its packages are
# already partly gone from deb.debian.org, so switch to the
# snapshot.debian.org entries the image ships commented out in its
# sources.list: they are pinned to the snapshot the image was built
# from, so they serve every version it needs and never go away.
# Snapshots keep their original, long-passed Valid-Until, hence the
# disabled check; the retries absorb snapshot.debian.org's throttling.
- name: Switch Debian 11 to snapshot.debian.org
if: ${{ matrix.image == 'debian:11' }}
run: |
sed -i 's|^deb |# deb |; s|^# deb http://snapshot|deb http://snapshot|' /etc/apt/sources.list
printf '%s\n' \
'Acquire::Check-Valid-Until "false";' \
'Acquire::Retries "3";' \
>/etc/apt/apt.conf.d/99snapshot
- name: Install the DEB
if: ${{ matrix.package_type == 'deb' }}
env:
DEBIAN_FRONTEND: noninteractive
PACKAGE: ${{ steps.find.outputs.package }}
run: |
# Stock Debian and Ubuntu images carry no package lists, so apt has
# nothing to resolve the systemd dependency from until it fetches them.
apt-get update -qq
apt-get install -y "./${PACKAGE}"
- name: Install the RPM
if: ${{ matrix.package_type == 'rpm' }}
env:
PACKAGE: ${{ steps.find.outputs.package }}
run: dnf install -y "./${PACKAGE}"
- name: Run xrpld
run: xrpld --version
- name: Run validator-keys
run: validator-keys --version
- name: Run rippled, the legacy compatibility symlink
run: rippled --version
- name: Check the service account
run: id xrpld
- name: Check the state directory
run: test -d /var/lib/xrpld
- name: Check the log directory
run: test -d /var/log/xrpld
publish:
needs: [generate-matrix, package, test-install]
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.generate-matrix.outputs.matrix) }}
name: "publish ${{ matrix.xrpld_artifact_name }}"
permissions:
contents: read
runs-on: ["self-hosted", "Linux", "X64", "heavy"]
container: ${{ matrix.image }}
timeout-minutes: 30
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Prepare runner
uses: XRPLF/actions/prepare-runner@b3e255d74d785d053e4903da8ac90983cd7d9e82
with:
enable_ccache: false
# Both artifacts, so the debug symbols are published alongside the package.
- name: Download package artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: ${{ matrix.xrpld_artifact_name }}-pkg*
merge-multiple: true
path: ${{ env.PACKAGE_DIR }}
- name: Determine release info
id: release_info
uses: ./.github/actions/release-info
- name: Publish package
env:
CHANNEL: ${{ steps.release_info.outputs.channel }}
DRY_RUN_OPTION: ${{ !inputs.publish && '--dry-run' || '' }}
NEXUS_URL: ${{ inputs.nexus_url }}
NEXUS_USERNAME: ${{ inputs.publish && secrets.remote_username || '' }}
NEXUS_PASSWORD: ${{ inputs.publish && secrets.remote_password || '' }}
run: |
publish_pkg.py \
--channel "${CHANNEL}" \
--package-dir "${PACKAGE_DIR}" \
--nexus-url "${NEXUS_URL}" \
${DRY_RUN_OPTION}

84
.github/workflows/reusable-rust.yml vendored Normal file
View File

@@ -0,0 +1,84 @@
# Clippy, coverage and documentation for the Rust crates in crates/. Each runs
# as an independent job on a GitHub-hosted runner, but inside the same container
# image used to build the crates in the C++/Corrosion path, so the toolchain
# (and therefore the lints and the cargo cache) matches what production builds
# use. Coverage is the exception: it needs the nightly rustc that honours
# #[coverage(off)], which the image carries alongside the pinned stable.
#
# Rust unit tests are deliberately NOT run here. They run as part of the C++
# build (reusable-build-test-config.yml), which already compiles the crates on a
# self-hosted runner, so there is no need to provision a toolchain again.
name: Rust
on:
workflow_call:
secrets:
CODECOV_TOKEN:
description: "The Codecov token to use for uploading coverage reports."
required: true
defaults:
run:
shell: bash
working-directory: crates
permissions:
contents: read
jobs:
clippy:
runs-on: ubuntu-latest
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-060957e
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Use cargo artifacts cache
uses: ./.github/actions/cargo-cache
- name: Run clippy
run: cargo clippy --workspace --all-targets --all-features --locked -- -D warnings
coverage:
runs-on: ubuntu-latest
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-060957e
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Use the nightly Rust toolchain
run: rust-nightly path >>"${GITHUB_PATH}"
- name: Use cargo artifacts cache
uses: ./.github/actions/cargo-cache
- name: Generate coverage report
run: cargo llvm-cov nextest --workspace --all-features --locked --no-tests=warn --lcov --output-path lcov.info
- name: Upload coverage report
if: ${{ github.repository == 'XRPLF/rippled' }}
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
disable_search: true
disable_telem: true
fail_ci_if_error: true
files: crates/lcov.info
flags: rust
plugins: noop
token: ${{ secrets.CODECOV_TOKEN }}
verbose: true
doc:
runs-on: ubuntu-latest
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-060957e
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Use cargo artifacts cache
uses: ./.github/actions/cargo-cache
- name: Build documentation
env:
RUSTDOCFLAGS: "-D warnings"
run: cargo doc --workspace --no-deps --all-features --locked

View File

@@ -40,7 +40,7 @@ defaults:
jobs:
upload:
runs-on: ubuntu-latest
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-fecfc0c
container: ghcr.io/xrplf/xrpld/nix-ubuntu:sha-060957e
env:
REMOTE_NAME: ${{ inputs.remote_name }}
CONAN_LOGIN_USERNAME_XRPLF: ${{ secrets.remote_username }}
@@ -49,9 +49,14 @@ jobs:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Generate build version number
id: version
uses: ./.github/actions/generate-version
- name: Prepare runner
uses: XRPLF/actions/prepare-runner@b3e255d74d785d053e4903da8ac90983cd7d9e82
with:
enable_ccache: false
- name: Determine release info
id: release_info
uses: ./.github/actions/release-info
- name: Set up Conan
uses: ./.github/actions/setup-conan
@@ -64,8 +69,8 @@ jobs:
- name: Upload Conan recipe (version)
run: |
conan export . --version=${{ steps.version.outputs.version }}
conan upload --confirm --check --remote="${REMOTE_NAME}" xrpl/${{ steps.version.outputs.version }}
conan export . --version=${{ steps.release_info.outputs.version }}
conan upload --confirm --check --remote="${REMOTE_NAME}" xrpl/${{ steps.release_info.outputs.version }}
# When this workflow is triggered by a push event, it will always be when merging into the
# 'develop' branch, see on-trigger.yml.
@@ -92,4 +97,4 @@ jobs:
conan upload --confirm --check --remote="${REMOTE_NAME}" xrpl/release
outputs:
ref: xrpl/${{ steps.version.outputs.version }}
ref: xrpl/${{ steps.release_info.outputs.version }}

View File

@@ -68,10 +68,15 @@ jobs:
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Prepare runner
uses: XRPLF/actions/prepare-runner@c00c22ada3bd6bcda48fcb0d62fbbab49fec8a0f
uses: XRPLF/actions/prepare-runner@b3e255d74d785d053e4903da8ac90983cd7d9e82
with:
enable_ccache: false
# Before any step that uses a build tool, composite actions included.
- name: Setup Nix environment
if: ${{ matrix.toolchain == 'nix' }}
uses: ./.github/actions/setup-nix-env
- name: Print build environment
uses: XRPLF/actions/print-build-env@59dec886e4afb05a1724443af08baccbc045b574
@@ -87,7 +92,9 @@ jobs:
with:
compiler: ${{ matrix.compiler }}
# `setup-nix-env` already did this for the Nix toolchain.
- name: Setup Conan
if: ${{ matrix.toolchain != 'nix' }}
env:
SANITIZERS: ${{ matrix.sanitizers }}
uses: ./.github/actions/setup-conan
@@ -106,6 +113,10 @@ jobs:
log_verbosity: ${{ runner.os == 'Windows' && 'quiet' || 'verbose' }}
sanitizers: ${{ matrix.sanitizers }}
- name: Check the Conan cache for Nix store references (Nix toolchain)
if: ${{ matrix.toolchain == 'nix' }}
run: ./bin/check-nix-store-refs.sh "${CONAN_HOME}"
- name: Log into Conan remote
if: ${{ github.repository == 'XRPLF/rippled' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') }}
run: conan remote login "${CONAN_REMOTE_NAME}" "${{ secrets.NEXUS_REMOTE_USERNAME }}" --password "${{ secrets.NEXUS_REMOTE_PASSWORD }}"