xahaud/BUILD.md

⚠️ WARNING ⚠️
These instructions assume you have a C++ development environment ready with Git, Python, Conan, CMake, and a C++ compiler. For help setting one up on Linux, macOS, or Windows, see this guide.

These instructions also assume a basic familiarity with Conan and CMake. If you are unfamiliar with Conan, you can read our crash course or the official [Getting Started][3] walkthrough.

Branches

For a stable release, choose the master branch or one of the tagged releases.

git checkout master

For the latest release candidate, choose the release branch.

git checkout release

For the latest set of untested features, or to contribute, choose the develop branch.

git checkout develop

Minimum Requirements

See System Requirements.

Building rippled generally requires git, Python, Conan, CMake, and a C++ compiler. Some guidance on setting up such a C++ development environment can be found here.

• Python 3.7
• Conan 2.x
• CMake 3.16

rippled is written in the C++20 dialect and includes the <concepts> header. The [minimum compiler versions][2] required are:

Compiler	Version
GCC	11
Clang	13
Apple Clang	13.1.6
MSVC	19.23

Linux

The Ubuntu operating system has received the highest level of quality assurance, testing, and support.

Here are sample instructions for setting up a C++ development environment on Linux.

Mac

Many rippled engineers use macOS for development.

Here are sample instructions for setting up a C++ development environment on macOS.

Windows

Windows is not recommended for production use at this time.

• Additionally, 32-bit Windows development is not supported.

Steps

Set Up Conan

After you have a C++ development environment ready with Git, Python, Conan, CMake, and a C++ compiler, you may need to set up your Conan profile.

These instructions assume a basic familiarity with Conan and CMake.

If you are unfamiliar with Conan, then please read this crash course or the official [Getting Started][3] walkthrough.

You'll need at least one Conan profile:

conan profile detect --force

Update the compiler settings:

For Conan 2, you can edit the profile directly at ~/.conan2/profiles/default, or use the Conan CLI. Ensure C++20 is set:

conan profile show

Look for compiler.cppstd=20 in the output. If it's not set, edit the profile:

# Edit ~/.conan2/profiles/default and ensure these settings exist:
[settings]
compiler.cppstd=20

Configure Conan (1.x only) to use recipe revisions:

conan config set general.revisions_enabled=1

Linux developers will commonly have a default Conan profile that compiles with GCC and links with libstdc++. If you are linking with libstdc++ (see profile setting compiler.libcxx), then you will need to choose the libstdc++11 ABI:

# In ~/.conan2/profiles/default, ensure:
[settings]
compiler.libcxx=libstdc++11

Ensure inter-operability between boost::string_view and std::string_view types:

conan profile update 'conf.tools.build:cxxflags+=["-DBOOST_BEAST_USE_STD_STRING_VIEW"]' default
conan profile update 'env.CXXFLAGS="-DBOOST_BEAST_USE_STD_STRING_VIEW"' default

If you have other flags in the conf.tools.build or env.CXXFLAGS sections, make sure to retain the existing flags and append the new ones. You can check them with:

conan profile show default

Windows developers may need to use the x64 native build tools. An easy way to do that is to run the shortcut "x64 Native Tools Command Prompt" for the version of Visual Studio that you have installed.

Windows developers must also build rippled and its dependencies for the x64 architecture:

# In ~/.conan2/profiles/default, ensure:
[settings]
arch=x86_64

3. (Optional) If you have multiple compilers installed on your platform, make sure that Conan and CMake select the one you want to use. This setting will set the correct variables (CMAKE_<LANG>_COMPILER) in the generated CMake toolchain file.

   # In ~/.conan2/profiles/default, add under [conf] section:
   [conf]
   tools.build:compiler_executables={"c": "<path>", "cpp": "<path>"}
   

   For setting environment variables for dependencies:

   # In ~/.conan2/profiles/default, add under [buildenv] section:
   [buildenv]
   CC=<path>
   CXX=<path>
   

4. Export our Conan recipe for Snappy. It doesn't explicitly link the C++ standard library, which allows you to statically link it with GCC, if you want.

   conan export external/snappy --version 1.1.10 --user xahaud --channel stable
   

Export our Conan recipe for RocksDB. It does not override paths to dependencies when building with Visual Studio.

conan export external/soci --version 4.0.3 --user xahaud --channel stable

6. Export our Conan recipe for WasmEdge.

   conan export external/wasmedge --version 0.11.2 --user xahaud --channel stable
   

Build and Test

1. Create a build directory and move into it.

   mkdir .build
   cd .build
   

   You can use any directory name. Conan treats your working directory as an install folder and generates files with implementation details. You don't need to worry about these files, but make sure to change your working directory to your build directory before calling Conan.

   Note: You can specify a directory for the installation files by adding the install-folder or -if option to every conan install command in the next step.

2. Use conan to generate CMake files for every configuration you want to build:

   conan install .. --output-folder . --build missing --settings build_type=Release
   conan install .. --output-folder . --build missing --settings build_type=Debug
   

   To build Debug, in the next step, be sure to set -DCMAKE_BUILD_TYPE=Debug

   For a single-configuration generator, e.g. Unix Makefiles or Ninja, you only need to run this command once. For a multi-configuration generator, e.g. Visual Studio, you may want to run it more than once.

   Each of these commands should also have a different build_type setting. A second command with the same build_type setting will overwrite the files generated by the first. You can pass the build type on the command line with --settings build_type=$BUILD_TYPE or in the profile itself, under the section [settings] with the key build_type.

   If you are using a Microsoft Visual C++ compiler, then you will need to ensure consistency between the build_type setting and the compiler.runtime setting.

   When build_type is Release, compiler.runtime should be MT.

   When build_type is Debug, compiler.runtime should be MTd.

   conan install .. --output-folder . --build missing --settings build_type=Release --settings compiler.runtime=MT
   conan install .. --output-folder . --build missing --settings build_type=Debug --settings compiler.runtime=MTd
   

3. Configure CMake and pass the toolchain file generated by Conan, located at $OUTPUT_FOLDER/build/generators/conan_toolchain.cmake.

   Single-config generators:

   Pass the CMake variable CMAKE_BUILD_TYPE and make sure it matches the one of the build_type settings you chose in the previous step.

   For example, to build Debug, in the next command, replace "Release" with "Debug"

   cmake -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=Release -Dxrpld=ON -Dtests=ON ..
   

   Multi-config generators:

   cmake -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake -Dxrpld=ON -Dtests=ON  ..
   

   Note: You can pass build options for rippled in this step.

4. Build rippled.

   For a single-configuration generator, it will build whatever configuration you passed for CMAKE_BUILD_TYPE. For a multi-configuration generator, you must pass the option --config to select the build configuration.

   Single-config generators:

   cmake --build .
   

   Multi-config generators:

   cmake --build . --config Release
   cmake --build . --config Debug
   

5. Test rippled.

   Single-config generators:

   ./rippled --unittest
   

   Multi-config generators:

   ./Release/rippled --unittest
   ./Debug/rippled --unittest
   

   The location of rippled in your build directory depends on your CMake generator. Pass --help to see the rest of the command line options.

Coverage report

The coverage report is intended for developers using compilers GCC or Clang (including Apple Clang). It is generated by the build target coverage, which is only enabled when the coverage option is set, e.g. with --options coverage=True in conan or -Dcoverage=ON variable in cmake

Prerequisites for the coverage report:

• gcovr tool (can be installed e.g. with pip)
• gcov for GCC (installed with the compiler by default) or
• llvm-cov for Clang (installed with the compiler by default)
• Debug build type

A coverage report is created when the following steps are completed, in order:

1. rippled binary built with instrumentation data, enabled by the coverage option mentioned above
2. completed run of unit tests, which populates coverage capture data
3. completed run of the gcovr tool (which internally invokes either gcov or llvm-cov) to assemble both instrumentation data and the coverage capture data into a coverage report

The above steps are automated into a single target coverage. The instrumented rippled binary can also be used for regular development or testing work, at the cost of extra disk space utilization and a small performance hit (to store coverage capture). In case of a spurious failure of unit tests, it is possible to re-run the coverage target without rebuilding the rippled binary (since it is simply a dependency of the coverage report target). It is also possible to select only specific tests for the purpose of the coverage report, by setting the coverage_test variable in cmake

The default coverage report format is html-details, but the user can override it to any of the formats listed in cmake/CodeCoverage.cmake by setting the coverage_format variable in cmake. It is also possible to generate more than one format at a time by setting the coverage_extra_args variable in cmake. The specific command line used to run the gcovr tool will be displayed if the CODE_COVERAGE_VERBOSE variable is set.

By default, the code coverage tool runs parallel unit tests with --unittest-jobs set to the number of available CPU cores. This may cause spurious test errors on Apple. Developers can override the number of unit test jobs with the coverage_test_parallelism variable in cmake.

Example use with some cmake variables set:

cd .build
conan install .. --output-folder . --build missing --settings build_type=Debug
cmake -DCMAKE_BUILD_TYPE=Debug -Dcoverage=ON -Dxrpld=ON -Dtests=ON -Dcoverage_test_parallelism=2 -Dcoverage_format=html-details -Dcoverage_extra_args="--json coverage.json" -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake ..
cmake --build . --target coverage

After the coverage target is completed, the generated coverage report will be stored inside the build directory, as either of:

• file named coverage.extension , with a suitable extension for the report format, or
• directory named coverage, with the index.html and other files inside, for the html-details or html-nested report formats.

Options

Option	Default Value	Description
assert	OFF	Enable assertions.
coverage	OFF	Prepare the coverage report.
san	N/A	Enable a sanitizer with Clang. Choices are thread and address.
tests	OFF	Build tests.
unity	ON	Configure a unity build.
xrpld	OFF	Build the xrpld (rippled) application, and not just the libxrpl library.

Unity builds may be faster for the first build (at the cost of much more memory) since they concatenate sources into fewer translation units. Non-unity builds may be faster for incremental builds, and can be helpful for detecting #include omissions.

Troubleshooting

Conan

If you have trouble building dependencies after changing Conan settings, try removing the Conan cache.

For Conan 2:

rm -rf ~/.conan2/p

Or clear the entire Conan 2 cache:

conan cache clean "*"

macOS compilation with Apple Clang 17+

If you're on macOS with Apple Clang 17 or newer, you need to add a compiler flag to work around a compilation error in gRPC dependencies.

Edit ~/.conan2/profiles/default and add under the [conf] section:

[conf]
tools.build:cxxflags=["-Wno-missing-template-arg-list-after-template-kw"]

call to 'async_teardown' is ambiguous

If you are compiling with an early version of Clang 16, then you might hit a regression when compiling C++20 that manifests as an error in a Boost header. You can workaround it by adding this preprocessor definition:

conan profile update 'env.CXXFLAGS="-DBOOST_ASIO_DISABLE_CONCEPTS"' default
conan profile update 'conf.tools.build:cxxflags+=["-DBOOST_ASIO_DISABLE_CONCEPTS"]' default

recompile with -fPIC

If you get a linker error suggesting that you recompile Boost with position-independent code, such as:

/usr/bin/ld.gold: error: /home/username/.conan/data/boost/1.77.0/_/_/package/.../lib/libboost_container.a(alloc_lib.o):
  requires unsupported dynamic reloc 11; recompile with -fPIC

Conan most likely downloaded a bad binary distribution of the dependency. This seems to be a bug in Conan just for Boost 1.77.0 compiled with GCC for Linux. The solution is to build the dependency locally by passing --build boost when calling conan install.

/usr/bin/ld.gold: error: /home/username/.conan/data/boost/1.77.0/_/_/package/dc8aedd23a0f0a773a5fcdcfe1ae3e89c4205978/lib/libboost_container.a(alloc_lib.o): requires unsupported dynamic reloc 11; recompile with -fPIC

Add a Dependency

If you want to experiment with a new package, follow these steps:

1. Search for the package on Conan Center.
2. Modify conanfile.py:
   • Add a version of the package to the requires property.
   • Change any default options for the package by adding them to the default_options property (with syntax '$package:$option': $value).
3. Modify CMakeLists.txt:
   • Add a call to find_package($package REQUIRED).
   • Link a library from the package to the target ripple_libs (search for the existing call to target_link_libraries(ripple_libs INTERFACE ...)).
4. Start coding! Don't forget to include whatever headers you need from the package.

A crash course in CMake and Conan

To better understand how to use Conan, we should first understand why we use Conan, and to understand that, we need to understand how we use CMake.

CMake

Technically, you don't need CMake to build this project. You could manually compile every translation unit into an object file, using the right compiler options, and then manually link all those objects together, using the right linker options. However, that is very tedious and error-prone, which is why we lean on tools like CMake.

We have written CMake configuration files (CMakeLists.txt and friends) for this project so that CMake can be used to correctly compile and link all of the translation units in it. Or rather, CMake will generate files for a separate build system (e.g. Make, Ninja, Visual Studio, Xcode, etc.) that compile and link all of the translation units. Even then, CMake has parameters, some of which are platform-specific. In CMake's parlance, parameters are specially-named variables like CMAKE_BUILD_TYPE or CMAKE_MSVC_RUNTIME_LIBRARY. Parameters include:

• what build system to generate files for
• where to find the compiler and linker
• where to find dependencies, e.g. libraries and headers
• how to link dependencies, e.g. any special compiler or linker flags that need to be used with them, including preprocessor definitions
• how to compile translation units, e.g. with optimizations, debug symbols, position-independent code, etc.
• on Windows, which runtime library to link with

For some of these parameters, like the build system and compiler, CMake goes through a complicated search process to choose default values. For others, like the dependencies, we had written in the CMake configuration files of this project our own complicated process to choose defaults. For most developers, things "just worked"... until they didn't, and then you were left trying to debug one of these complicated processes, instead of choosing and manually passing the parameter values yourself.

You can pass every parameter to CMake on the command line, but writing out these parameters every time we want to configure CMake is a pain. Most humans prefer to put them into a configuration file, once, that CMake can read every time it is configured. For CMake, that file is a toolchain file.

Conan

These next few paragraphs on Conan are going to read much like the ones above for CMake.

Technically, you don't need Conan to build this project. You could manually download, configure, build, and install all of the dependencies yourself, and then pass all of the parameters necessary for CMake to link to those dependencies. To guarantee ABI compatibility, you must be sure to use the same set of compiler and linker options for all dependencies and this project. However, that is very tedious and error-prone, which is why we lean on tools like Conan.

We have written a Conan configuration file (conanfile.py) so that Conan can be used to correctly download, configure, build, and install all of the dependencies for this project, using a single set of compiler and linker options for all of them. It generates files that contain almost all of the parameters that CMake expects. Those files include:

• A single toolchain file.
• For every dependency, a CMake package configuration file, package version file, and for every build type, a package targets file. Together, these files implement version checking and define IMPORTED targets for the dependencies.

The toolchain file itself amends the search path (CMAKE_PREFIX_PATH) so that find_package() will discover the generated package configuration files.

Nearly all we must do to properly configure CMake is pass the toolchain file. What CMake parameters are left out? You'll still need to pick a build system generator, and if you choose a single-configuration generator, you'll need to pass the CMAKE_BUILD_TYPE, which should match the build_type setting you gave to Conan.

Even then, Conan has parameters, some of which are platform-specific. In Conan's parlance, parameters are either settings or options. Settings are shared by all packages, e.g. the build type. Options are specific to a given package, e.g. whether to build and link OpenSSL as a shared library.

For settings, Conan goes through a complicated search process to choose defaults. For options, each package recipe defines its own defaults.

You can pass every parameter to Conan on the command line, but it is more convenient to put them in a profile. All we must do to properly configure Conan is edit and pass the profile.