rippled/package/README.md

# Linux Packaging

This directory contains all files needed to build RPM and Debian packages for
`xrpld`. The packages also ship the `validator-keys` tool, so packaging requires
a build configured with `-Dvalidator_keys=ON`.

## Directory layout

```
package/
  build_pkg.py        Staging and build script (called by the CMake `package` target and CI)
  sign_rpm.py         Signs the built RPMs (called by CI when publishing)
  docker/
    Dockerfile          Packaging image, built by `build-packaging-images.yml`; installs its tooling with `bin/install-packaging-tools.sh`
    publish_pkg.py      Uploads built packages to the XRPLF Nexus repositories (called by CI, and shipped in that image)
  rpm/
    xrpld.spec      RPM spec
  debian/           Debian control files (control.in, lintian-overrides.in, rules, copyright, docs, links, source/format).
                    The `.in` files are templates rendered by `build_pkg.py`; `docs` and `links` are staged under the package name
  shared/
    xrpld.service       systemd unit file (used by both RPM and DEB)
    xrpld.sysusers      sysusers.d config (used by both RPM and DEB)
    xrpld.tmpfiles      tmpfiles.d config (used by both RPM and DEB)
    xrpld.logrotate     logrotate config (installed to /etc/logrotate.d/xrpld)
```

## Prerequisites

Packaging is declared on the build configs themselves, in
[`.github/scripts/strategy-matrix/linux.json`](../.github/scripts/strategy-matrix/linux.json):
a config that is also packaged carries a `package` map, so its binaries and its
packaging job cannot drift apart. Today only `linux/amd64` is emitted. The map
pins the full container image in `image` — edit that field to move to a new
image and both CI and local builds pick it up — and names the format that image
builds in `type`, which CI passes to `build_pkg.py` as `--package-type`; the two
have to stay in step. An optional `variant` names a flavour of the package (see
[Package variants](#package-variants)), and CI passes it as `--variant`.

| Package type | Image (`configs.<distro>[].package.image` in `linux.json`) | Tools required                                                 |
| ------------ | ---------------------------------------------------------- | -------------------------------------------------------------- |
| RPM          | `ghcr.io/xrplf/xrpld/packaging-rhel:sha-<sha>`             | `rpmbuild`, `rpmsign`                                          |
| DEB          | `ghcr.io/xrplf/xrpld/packaging-debian:sha-<sha>`           | `dpkg-buildpackage`, debhelper with compat level 13, `lintian` |

To print the full packaging matrix (artifact names, images and package names)
for the current `linux.json`:

```bash
./.github/scripts/strategy-matrix/generate.py --packaging
```

## Package variants

A config whose binaries are not the plain release build cannot be packaged as
`xrpld`: both would carry the same name and version, so whichever published last
would win. It is packaged as a **variant** instead — `variant: "assert"` in its
`package` map, which CI passes to `build_pkg.py` as `--variant assert`,
producing `xrpld-assert`. What the build option itself does is a build concern,
not a packaging one; see the options table in [`BUILD.md`](../BUILD.md).

A variant ships the same paths as `xrpld` — `/usr/bin/xrpld`, `/etc/xrpld`,
`xrpld.service`, `/etc/logrotate.d/xrpld` — differing only in the per-package
documentation directory, so it declares itself a stand-in for the plain package
rather than something installable next to it: `Conflicts`, `Replaces` and a
versioned `Provides: xrpld` on Debian, `Conflicts` and `Provides` on RPM.
Neither format declares `Obsoletes`, so `apt upgrade` and `dnf upgrade` keep an
installed flavour on its own flavour, and switching is always explicit:

```bash
apt-get install xrpld-assert # apt removes the plain package itself
dnf swap xrpld xrpld-VARIANT # 'dnf install' alone stops at the conflict
```

Only the DEB packages carry a variant today — `xrpld-assert` comes from the
`debian` config alone, there being no call for an assert build on RHEL-based
distributions — but the RPM side works the same way if one is added.

A switch is a removal plus an installation rather than an upgrade, so unlike a
version upgrade it stops the service: Debian's scriptlets start it again, while
on RPM the operator runs `systemctl start xrpld`. Configuration survives either
way, being conffiles on Debian and `%config(noreplace)` on RPM.

`dnf` installs the replacement before erasing the old flavour, whose `%preun`
would leave `xrpld.service` disabled, so `%postun` re-applies the preset when
the unit file outlives the erase — which, since rpm keeps a file another
installed package owns, happens only during a swap. The cost is that a
deliberate `systemctl disable` is not carried across an RPM switch.

The alternative is an `xrpld-common` package owning the unit, the sysusers and
tmpfiles snippets and the configuration, required by both flavours at an exact
version: nothing is erased mid-swap, so no scriptlet has to detect one. It is
not worth it for a single variant — it moves files out of the production
package, and a sanitizer flavour would likely need its own unit anyway, putting
the lifecycle back where it is now.

Adding a variant is the flavour in `VARIANTS` in `build_pkg.py`, which is the
list `--variant` accepts, plus a config in `linux.json` with the CMake arguments
and a `package` map naming it, for one format or for both: `generate.py
--packaging` emits the package names per format, and the `test-install-deb` and
`test-install-rpm` jobs install what their own format produced.

Operators switch between the flavours as described in
[`docs/install.md`](../docs/install.md#optional-the-assert-enabled-build).

## Building packages

### Via CI

Caller workflows (`on-pr.yml`, `on-tag.yml`, `on-trigger.yml`) call
`reusable-package.yml`, which runs in three stages:

1. `package` fans out one job per config carrying a `package` map, building and
   signing in that config's container, and uploading `<config>-pkg` alongside
   `<config>-pkg-debug` for the much larger debug symbols.
2. `test-install-deb` and `test-install-rpm` call
   [`reusable-package-test-install.yml`](../.github/workflows/reusable-package-test-install.yml)
   with their format's package names and distro images, installing each package
   in the container of every distro that format targets and running the binaries
   there, so one that cannot be installed never reaches Nexus.
3. `publish` uploads both artifacts, or lists what it would upload.

The packaging script derives the package version from the downloaded binary's
`xrpld --version` output; no CMake configure or build step is needed inside the
packaging job.

The binaries come from the `debian` and `rhel` build configs themselves — the
ones carrying the `package` map — which pass `-Dvalidator_keys=ON` so that the
build job produces `validator-keys` next to `xrpld` and uploads it as the
`validator-keys-<config name>` artifact. The packaging matrix names both
artifacts (`xrpld_artifact_name` and `validator_keys_artifact_name`) after that
same config, so a packaged config must keep `-Dvalidator_keys=ON`. Those configs
are not `minimal`, so `on-pr.yml` only packages once a PR runs the full matrix.

`validator-keys` is fetched from an exact commit pinned in
[`cmake/XrplValidatorKeys.cmake`](../cmake/XrplValidatorKeys.cmake), so a given
`xrpld` version always packages the same tool; bump that commit deliberately.

### Locally (mirrors CI)

With `xrpld` and `validator-keys` binaries already built at `build/xrpld` and
`build/validator-keys`, run the packaging step inside the same container CI uses.
The image tag is derived from `linux.json` so you don't need to hardcode a SHA.

```bash
# From the repo root. Each distro's container image is the `package.image` field
# of its config in linux.json. Example for the rpm-producing image (use
# .configs.debian[0].package.image and --package-type deb for the other one):
IMAGE=$(jq -r '.configs.rhel[0].package.image' .github/scripts/strategy-matrix/linux.json)

PKG_RELEASE=1

docker run --rm \
    -v "$(pwd):/src" \
    -w /src \
    "${IMAGE}" \
    ./package/build_pkg.py \
    --package-type rpm \
    --pkg-release "${PKG_RELEASE}" \
    --channel UNRELEASED

# Output (the deb image writes build/debbuild/*.deb instead):
#   build/rpmbuild/RPMS/x86_64/*.rpm
```

Add `--variant assert` to package binaries built with `-Dassert=ON`; the package
is then named `xrpld-assert`.

### Via CMake (host-side target)

If you run CMake configure on a host that has `rpmbuild` or `dpkg-buildpackage`
installed natively, you can use the CMake target directly — no container
needed, but the host toolchain replaces the pinned CI image:

```bash
cmake \
    -Dxrpld=ON \
    -Dvalidator_keys=ON \
    -Dpkg_release=1 \
    -Dtests=OFF \
    ..

cmake --build . --target package # deb on Debian/Ubuntu, rpm on RHEL
```

The `cmake/XrplPackaging.cmake` module defines the `package` target only if at
least one of `rpmbuild` / `dpkg-buildpackage` is present and both the `xrpld` and
`validator-keys` targets exist (`-Dxrpld=ON -Dvalidator_keys=ON`); the target
builds both binaries before packaging, passing `--package-type deb` when
`dpkg-buildpackage` is present and `rpm` otherwise, and `--channel UNRELEASED`.
The packaging script installs to FHS-standard paths (`/usr/bin`, `/etc/xrpld`,
etc.) regardless of `CMAKE_INSTALL_PREFIX`.

The package version is not a CMake input on this path: `build_pkg.py` derives it
from the just-built `xrpld` binary's `xrpld --version` output. The package
release defaults to 1 and is overridable with `-Dpkg_release=N`.

`-Dassert=ON` passes `--variant assert`, so such a build packages as
`xrpld-assert` without anything else being asked for.

## Publishing packages

Packages are published to the XRPLF repositories on Sonatype Nexus through
`https://packages-upload.xrplf.org`. Reads go through
`https://packages.xrplf.org`, which Cloudflare proxies to cache them and which
rejects request bodies over 100 MB, so uploads use the DNS-only host instead.
The `release-info` action decides the channel from the event, and
`publish_pkg.py` maps that channel to its repositories:

| Event                    | Version           | Channel   | DEB repository | RPM upload repository |
| ------------------------ | ----------------- | --------- | -------------- | --------------------- |
| tag                      | `X.Y.Z`           | `stable`  | `deb-stable`   | `rpm-stable-hosted`   |
| tag                      | `X.Y.Z-rcN`       | `rc`      | `deb-rc`       | `rpm-rc-hosted`       |
| tag                      | `X.Y.Z-bN`        | `beta`    | `deb-beta`     | `rpm-beta-hosted`     |
| push to `develop`        | `xrpld --version` | `develop` | `deb-develop`  | `rpm-develop-hosted`  |
| tag, non-public codebase | _any_             | `private` | `deb-private`  | `rpm-private-hosted`  |

A variant is published to the same channel under its own name, so
`xrpld-assert` never overwrites `xrpld`.

Only a tag names a channel — do not extend that to `develop`, where
`BuildInfo.cpp`'s `versionString` moves through `-bN`, `-rcN` and even the final
version during a release cycle, which would send develop builds into `stable`.
Versions sort in row order, so moving to a more mature channel never downgrades.

The action decides the package release number on the same split: a tag's version
is unique, so its packages are release 1, while develop repeats the same version
and takes `<run number>.<commit date>git<commit hash>`, e.g.
`857.20260826gitb6a8995` — the leading run number keeps each push superseding
the last, and the date and hash say which commit a package on
`packages.xrplf.org` came from. Both reach the packaging scripts as arguments,
so neither script derives anything itself.

Publishing is its own job, gated behind the install tests, uploading from the same
image that built the packages with the `publish_pkg.py` shipped in it — the
same copy other repositories run. Without `publish: true` the job is a
`--dry-run`, listing the uploads it would make without needing credentials, so
any run that builds packages also exercises the upload routing. `on-trigger.yml`
passes `publish: true` for develop pushes in `XRPLF/rippled` and `on-tag.yml`
for tags in any `XRPLF` repository, both authenticating with the
`NEXUS_REMOTE_USERNAME` / `NEXUS_REMOTE_PASSWORD` secrets already used for the
Conan remote; `on-pr.yml` never publishes.

Nexus owns the repository metadata; nothing here indexes anything. Worth knowing:

- Each apt-hosted repository needs a distribution (ours use `any`) and a PGP
  signing keypair configured in Nexus, which rejects one created without a
  keypair. Nexus signs the apt metadata with it, never the packages.
- yum-hosted repositories cannot be signed by Nexus, so each `rpm-<channel>-hosted`
  repository sits behind a `rpm-<channel>` yum group repository whose metadata
  Nexus signs. Uploads go to the hosted repository; clients point at the group
  and verify the metadata with `repo_gpgcheck=1`. Nexus never signs the RPMs
  themselves, so `sign_rpm.py` signs them before they are uploaded, and clients
  verify them with `gpgcheck=1`.
- yum metadata is rebuilt asynchronously, so a successful publish is not
  immediately installable.
- Each job uploads only what it built, and uploads are not transactional, so a
  failure can leave one format published alone. Re-running is safe: both the apt
  POST and the yum PUT replace an existing asset.
- The `develop` repositories gain a package per push, so they need a cleanup
  policy to stay bounded; tagged channels publish each version once.

### Publishing from other repositories

`publish_pkg.py` knows nothing about `xrpld`, so the packaging image
installs it at `/usr/local/bin/publish_pkg.py` for other XRPLF repositories that
build their packages elsewhere.

## How `build_pkg.py` works

`build_pkg.py` derives the `xrpld` software version from
`${BUILD_DIR}/xrpld --version` in both package formats.

The binary's version is already SemVer-validated by `BuildInfo`.
`build_pkg.py` converts pre-release versions such as `3.2.0-b1` or
`3.2.0-rc1` from `-` to `~` for package metadata so pre-releases sort before
the final release. If that normalized package version still contains `-`,
packaging fails because RPM forbids `-` in `Version`, and Debian uses `-` as
the upstream/revision separator.

> [!NOTE]
> Debug and sanitizer builds are not packaged yet.

`pkg_version` is the normalized package metadata version derived inside
`build_pkg.py` from the binary-reported `xrpld` version (`-` pre-release
separator converted to `~`). It is not a separate user input.

`PKG_RELEASE` is a different value: the package release iteration for that
`xrpld` version. RPM receives the normalized `pkg_version` and `PKG_RELEASE` as
the `pkg_version` and `pkg_release` macros for its `Version` and `Release`
values; DEB writes them as `${pkg_version}-${PKG_RELEASE}` in
`debian/changelog`.

With `PKG_RELEASE=1`, the package metadata becomes:

| Input version      | RPM version/release          | Debian version       |
| ------------------ | ---------------------------- | -------------------- |
| `3.2.0`            | `3.2.0-1%{?dist}`            | `3.2.0-1`            |
| `3.2.0-b0+abc1234` | `3.2.0~b0+abc1234-1%{?dist}` | `3.2.0~b0+abc1234-1` |
| `3.2.0-b1`         | `3.2.0~b1-1%{?dist}`         | `3.2.0~b1-1`         |
| `3.2.0-rc1`        | `3.2.0~rc1-1%{?dist}`        | `3.2.0~rc1-1`        |

`build_pkg.py` defines `dist` as `.el9` rather than letting rpmbuild take it
from the build host, so the RHEL image can track a newer release without
changing what the packages claim to target.

The Debian changelog entry carries the channel passed as `--channel`, which
only accepts the channels in the table above plus `UNRELEASED`, the Debian
convention for a build that targets no channel at all — what local and CMake
builds pass, since nothing publishes them. An unsupported pre-release, and
build metadata on a final release such as `3.2.0+abc123`, are both rejected.

The RPM path intentionally uses `~` in `Version`, matching the Debian
pre-release ordering convention, so RPM filenames/NVRs begin with forms like
`xrpld-3.2.0~b1-...` and `xrpld-3.2.0~rc1-...` instead of encoding
pre-releases with an older `0.<release>.<suffix>` RPM `Release` value.

`--variant` is the flavour of the package, empty by default and accepting only
the flavours in `VARIANTS`; see [Package variants](#package-variants). The RPM
path passes it to the spec as the `pkg_variant` macro, which suffixes `Name` and
adds the `Conflicts`/`Provides` pair. Debian control files have no conditionals, so the DEB path renders
`debian/control.in` and `debian/lintian-overrides.in` instead, substituting
`@PKG@` with the package name and `@VARIANT_FIELDS@` with the
`Conflicts`/`Replaces`/`Provides` block, empty for the plain package; a token
with no value fails the build rather than reaching dpkg. The files debhelper
keys by package name (`docs`, `links`, and the units) are staged under that same
name. The paths inside the package are unchanged either way, so `debian/rules`
reads its package name from `dh_listpackages` and names the unit, sysusers,
tmpfiles and logrotate files with `--name xrpld`.

The package format is `--package-type`, either `deb` or `rpm`. It is required,
so a job never silently builds the wrong format for the image it runs in; the
matching build tool still has to be on PATH.

Every input is a named argument, and every argument but `--build-dir` and
`--pkg-release` is required. The repository root is not an argument
at all: the script reads it from its own location. Only secrets stay in the
environment, so they never reach the process list -- `PKG_SIGNING_KEY` for
`sign_rpm.py`, and `NEXUS_USERNAME` / `NEXUS_PASSWORD` for `publish_pkg.py`.

Signing is not part of this script. `sign_rpm.py` does it in a separate CI step
that only runs when publishing, so a published RPM is always signed and a local
build never needs a key.

It resolves the build directory to an absolute path, then calls
`stage_common()` to copy the `xrpld` and `validator-keys` binaries, config files,
and shared support files into the staging area, and invokes the platform build
tool. Both binaries must be present in the build directory and must run in the
packaging environment; a missing or non-runnable one fails early. That runtime
check is what catches a binary still linked against the Nix store's ELF loader (see
`patch_nix_binary` in `cmake/PatchNixBinary.cmake`).

### RPM

1. Creates the standard `rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}` tree inside the build directory.
2. Copies `xrpld.spec` and all shared source files (binaries, configs, service files) into `SOURCES/`.
3. Runs `rpmbuild -bb`, passing the normalized package metadata version as the
   `pkg_version` RPM macro and `PKG_RELEASE` as the `pkg_release` RPM macro.
   The spec uses manual `install` commands to place files, disables `dwz`, and
   generates debuginfo packages.
4. Output: `rpmbuild/RPMS/x86_64/xrpld-*.rpm`

RPM upgrades intentionally do not restart a running `xrpld` service. The spec
uses `%systemd_postun`, matching Debian's `dh_installsystemd
--no-stop-on-upgrade` behavior; operators pick up the new binary on the next
service restart.

### DEB

1. Creates a staging source tree at `debbuild/source/` inside the build directory.
2. Stages the binaries, configs, `README.md`, `LICENSE.md`, and
   `validator-keys-LICENSE`.
3. Stages `package/debian/` into `debbuild/source/debian/`: the `.in` templates
   are rendered, and the files debhelper keys by package name (`docs`, `links`,
   `lintian-overrides`) are staged under the name being built.
4. Copies shared service/sysusers/tmpfiles/logrotate into `debian/` as
   `<package>.xrpld.*`, which `dh_installsystemd`, `dh_installsysusers`,
   `dh_installtmpfiles` and `dh_installlogrotate` read because `debian/rules`
   passes them `--name xrpld`.
5. Generates a minimal `debian/changelog` using `${pkg_version}-${PKG_RELEASE}`,
   where `pkg_version` is derived from the binary-reported `xrpld` version.
6. Runs `dpkg-buildpackage -b --no-sign -d` (`-d` skips the build-dependency check, since the binary is already built). `debian/rules` uses manual `install` commands.

   It also rewrites the `libc6` bound to `LIBC_MIN` in `debian/rules`, the glibc
   the Nix toolchain builds against. `dpkg-shlibdeps` would otherwise derive it
   from the build host's symbols file — on trixie that yields `libc6 (>= 2.34)`
   because of `sysconf`, locking out distros the binaries run on. A check fails
   the build if either binary outgrows `LIBC_MIN`.

7. Output: `debbuild/*.deb`, the binary package and the `-dbgsym` package.
   Debian gives dbgsym packages a `.deb` extension; only Ubuntu uses `.ddeb`.

## Post-build verification

```bash
# DEB (one invocation per package: the dbgsym package is a .deb too)
for deb in debbuild/*.deb; do dpkg-deb -c "${deb}"; done | grep -E 'systemd|sysusers|tmpfiles'
lintian -I debbuild/*.deb

# RPM
rpm -qlp rpmbuild/RPMS/x86_64/*.rpm
```

`lintian` still reports `embedded-library zlib`, `no-manual-page` and
`initial-upload-closes-no-bugs`; only the `/usr/local` tags are overridden.

## Reproducibility

Both formats build reproducibly as they are: the same binaries at the same
commit give byte-identical packages on a rebuild, and nothing has to be
exported by hand.

`build_pkg.py` sets `SOURCE_DATE_EPOCH` from the latest git commit time.
`dpkg-buildpackage` honours it on its own; the RPM spec sets three macros:

- `%clamp_mtime_to_source_date_epoch` — file modification times, from
  `SOURCE_DATE_EPOCH`.
- `%use_source_date_epoch_as_buildtime` — the `BUILDTIME` header, from the
  same.
- `%_buildhost` — pinned, so the builder's hostname stays out of the header.