xahaud/BUILD.md

> These instructions assume you have a C++ development environment ready
> with Git, Python, Conan, CMake, and a C++ compiler. For help setting one up
> on Linux, macOS, or Windows, see [our guide](./docs/build/environment.md).
>
> These instructions also assume a basic familiarity with Conan and CMake.
> If you are unfamiliar with Conan,
> you can read our [crash course](./docs/build/conan.md)
> or the official [Getting Started][3] walkthrough.

## Branches

For a stable release, choose the `master` branch or one of the [tagged
releases](https://github.com/ripple/rippled/releases).

```
git checkout master
```

For the latest release candidate, choose the `release` branch.

```
git checkout release
```

For the latest set of untested features, or to contribute, choose the `develop`
branch.

```
git checkout develop
```


## Minimum Requirements

- [Python 3.7](https://www.python.org/downloads/)
- [Conan 2.x](https://conan.io/downloads)
- [CMake 3.16](https://cmake.org/download/)

`rippled` is written in the C++20 dialect and includes the `<concepts>` header.
The [minimum compiler versions][2] required are:

| Compiler    | Version |
|-------------|---------|
| GCC         | 10      |
| Clang       | 13      |
| Apple Clang | 13.1.6  |
| MSVC        | 19.23   |

We don't recommend Windows for `rippled` production at this time. As of
January 2023, Ubuntu has the highest level of quality assurance, testing,
and support.

Windows developers should use Visual Studio 2019. `rippled` isn't
compatible with [Boost](https://www.boost.org/) 1.78 or 1.79, and Conan
can't build earlier Boost versions.

**Note:** 32-bit Windows development isn't supported.


## Steps


### Set Up Conan

1. (Optional) If you've never used Conan, use autodetect to set up a default profile.

   ```
   conan profile detect --force
   ```

2. Update the compiler settings.

   For Conan 2, you can edit the profile directly at `~/.conan2/profiles/default`,
   or use the Conan CLI. Ensure C++20 is set:

   ```
   conan profile show
   ```

   Look for `compiler.cppstd=20` in the output. If it's not set, edit the profile:

   ```
   # Edit ~/.conan2/profiles/default and ensure these settings exist:
   [settings]
   compiler.cppstd=20
   ```

   Linux developers will commonly have a default Conan [profile][] that compiles
   with GCC and links with libstdc++.
   If you are linking with libstdc++ (see profile setting `compiler.libcxx`),
   then you will need to choose the `libstdc++11` ABI.

   ```
   # In ~/.conan2/profiles/default, ensure:
   [settings]
   compiler.libcxx=libstdc++11
   ```

   On Windows, you should use the x64 native build tools.
   An easy way to do that is to run the shortcut "x64 Native Tools Command
   Prompt" for the version of Visual Studio that you have installed.

   Windows developers must also build `rippled` and its dependencies for the x64
   architecture.

   ```
   # In ~/.conan2/profiles/default, ensure:
   [settings]
   arch=x86_64
   ```

3. (Optional) If you have multiple compilers installed on your platform,
   make sure that Conan and CMake select the one you want to use.
   This setting will set the correct variables (`CMAKE_<LANG>_COMPILER`)
   in the generated CMake toolchain file.

   ```
   # In ~/.conan2/profiles/default, add under [conf] section:
   [conf]
   tools.build:compiler_executables={"c": "<path>", "cpp": "<path>"}
   ```

   For setting environment variables for dependencies:

   ```
   # In ~/.conan2/profiles/default, add under [buildenv] section:
   [buildenv]
   CC=<path>
   CXX=<path>
   ```

4. Export our [Conan recipe for Snappy](./external/snappy).
   It doesn't explicitly link the C++ standard library,
   which allows you to statically link it with GCC, if you want.

   ```
   conan export external/snappy --version 1.1.10 --user xahaud --channel stable
   ```

5. Export our [Conan recipe for SOCI](./external/soci).
   It patches their CMake to correctly import its dependencies.

   ```
   conan export external/soci --version 4.0.3 --user xahaud --channel stable
   ```

6. Export our [Conan recipe for WasmEdge](./external/wasmedge).

   ```
   conan export external/wasmedge --version 0.11.2 --user xahaud --channel stable
   ```

### Build and Test

1. Create a build directory and move into it.

   ```
   mkdir .build
   cd .build
   ```

   You can use any directory name. Conan treats your working directory as an
   install folder and generates files with implementation details.
   You don't need to worry about these files, but make sure to change
   your working directory to your build directory before calling Conan.

   **Note:** You can specify a directory for the installation files by adding
   the `install-folder` or `-if` option to every `conan install` command
   in the next step.

2. Generate CMake files for every configuration you want to build. 

    ```
    conan install .. --output-folder . --build missing --settings build_type=Release
    conan install .. --output-folder . --build missing --settings build_type=Debug
    ```

    For a single-configuration generator, e.g. `Unix Makefiles` or `Ninja`,
    you only need to run this command once.
    For a multi-configuration generator, e.g. `Visual Studio`, you may want to
    run it more than once.

    Each of these commands should also have a different `build_type` setting.
    A second command with the same `build_type` setting will overwrite the files
    generated by the first. You can pass the build type on the command line with
    `--settings build_type=$BUILD_TYPE` or in the profile itself,
    under the section `[settings]` with the key `build_type`.
    
    If you are using a Microsoft Visual C++ compiler,
    then you will need to ensure consistency between the `build_type` setting
    and the `compiler.runtime` setting.
    
    When `build_type` is `Release`, `compiler.runtime` should be `MT`.
    
    When `build_type` is `Debug`, `compiler.runtime` should be `MTd`.

    ```
    conan install .. --output-folder . --build missing --settings build_type=Release --settings compiler.runtime=MT
    conan install .. --output-folder . --build missing --settings build_type=Debug --settings compiler.runtime=MTd
    ```

3. Configure CMake and pass the toolchain file generated by Conan, located at
   `$OUTPUT_FOLDER/build/generators/conan_toolchain.cmake`.

    Single-config generators:
    
    ```
    cmake -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=Release ..
    ```

    Pass the CMake variable [`CMAKE_BUILD_TYPE`][build_type]
    and make sure it matches the `build_type` setting you chose in the previous
    step.

    Multi-config gnerators:

    ```
    cmake -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake ..
    ```

    **Note:** You can pass build options for `rippled` in this step.

4. Build `rippled`.

   For a single-configuration generator, it will build whatever configuration
   you passed for `CMAKE_BUILD_TYPE`. For a multi-configuration generator,
   you must pass the option `--config` to select the build configuration. 

   Single-config generators:

   ```
   cmake --build .
   ```

   Multi-config generators:
    
   ```
   cmake --build . --config Release
   cmake --build . --config Debug
   ```

5. Test rippled.

   Single-config generators:

   ```
   ./rippled --unittest
   ```

   Multi-config generators:

   ```
   ./Release/rippled --unittest
   ./Debug/rippled --unittest
   ```

   The location of `rippled` in your build directory depends on your CMake
   generator. Pass `--help` to see the rest of the command line options.


## Options

| Option | Default Value | Description |
| --- | ---| ---|
| `assert` | OFF | Enable assertions.
| `reporting` | OFF | Build the reporting mode feature. |
| `tests` | ON | Build tests. |
| `unity` | ON | Configure a unity build. |
| `san` | N/A | Enable a sanitizer with Clang. Choices are `thread` and `address`. |

[Unity builds][5] may be faster for the first build
(at the cost of much more memory) since they concatenate sources into fewer
translation units. Non-unity builds may be faster for incremental builds,
and can be helpful for detecting `#include` omissions.


## Troubleshooting


### Conan

If you have trouble building dependencies after changing Conan settings,
try removing the Conan cache.

For Conan 2:
```
rm -rf ~/.conan2/p
```

Or clear the entire Conan 2 cache:
```
conan cache clean "*"
```


### macOS compilation with Apple Clang 17+

If you're on macOS with Apple Clang 17 or newer, you need to add a compiler flag to work around a compilation error in gRPC dependencies.

Edit `~/.conan2/profiles/default` and add under the `[conf]` section:

```
[conf]
tools.build:cxxflags=["-Wno-missing-template-arg-list-after-template-kw"]
```


### recompile with -fPIC

If you get a linker error suggesting that you recompile Boost with
position-independent code, such as:

```
/usr/bin/ld.gold: error: /home/username/.conan/data/boost/1.77.0/_/_/package/.../lib/libboost_container.a(alloc_lib.o):
  requires unsupported dynamic reloc 11; recompile with -fPIC
```

Conan most likely downloaded a bad binary distribution of the dependency.
This seems to be a [bug][1] in Conan just for Boost 1.77.0 compiled with GCC
for Linux. The solution is to build the dependency locally by passing
`--build boost` when calling `conan install`.

```
/usr/bin/ld.gold: error: /home/username/.conan/data/boost/1.77.0/_/_/package/dc8aedd23a0f0a773a5fcdcfe1ae3e89c4205978/lib/libboost_container.a(alloc_lib.o): requires unsupported dynamic reloc 11; recompile with -fPIC
```


## Add a Dependency

If you want to experiment with a new package, follow these steps:

1. Search for the package on [Conan Center](https://conan.io/center/).
2. Modify [`conanfile.py`](./conanfile.py):
    - Add a version of the package to the `requires` property.
    - Change any default options for the package by adding them to the
    `default_options` property (with syntax `'$package:$option': $value`).
3. Modify [`CMakeLists.txt`](./CMakeLists.txt):
    - Add a call to `find_package($package REQUIRED)`.
    - Link a library from the package to the target `ripple_libs`
    (search for the existing call to `target_link_libraries(ripple_libs INTERFACE ...)`).
4. Start coding! Don't forget to include whatever headers you need from the package.


## A crash course in CMake and Conan

To better understand how to use Conan,
we should first understand _why_ we use Conan,
and to understand that,
we need to understand how we use CMake.


### CMake

Technically, you don't need CMake to build this project.
You could manually compile every translation unit into an object file,
using the right compiler options,
and then manually link all those objects together,
using the right linker options.
However, that is very tedious and error-prone,
which is why we lean on tools like CMake.

We have written CMake configuration files
([`CMakeLists.txt`](./CMakeLists.txt) and friends)
for this project so that CMake can be used to correctly compile and link
all of the translation units in it.
Or rather, CMake will generate files for a separate build system
(e.g. Make, Ninja, Visual Studio, Xcode, etc.)
that compile and link all of the translation units.
Even then, CMake has parameters, some of which are platform-specific.
In CMake's parlance, parameters are specially-named **variables** like
[`CMAKE_BUILD_TYPE`][build_type] or
[`CMAKE_MSVC_RUNTIME_LIBRARY`][runtime].
Parameters include:

- what build system to generate files for
- where to find the compiler and linker
- where to find dependencies, e.g. libraries and headers
- how to link dependencies, e.g. any special compiler or linker flags that
    need to be used with them, including preprocessor definitions
- how to compile translation units, e.g. with optimizations, debug symbols,
    position-independent code, etc.
- on Windows, which runtime library to link with

For some of these parameters, like the build system and compiler,
CMake goes through a complicated search process to choose default values.
For others, like the dependencies,
_we_ had written in the CMake configuration files of this project
our own complicated process to choose defaults.
For most developers, things "just worked"... until they didn't, and then
you were left trying to debug one of these complicated processes, instead of
choosing and manually passing the parameter values yourself.

You can pass every parameter to CMake on the command line,
but writing out these parameters every time we want to configure CMake is
a pain.
Most humans prefer to put them into a configuration file, once, that
CMake can read every time it is configured.
For CMake, that file is a [toolchain file][toolchain].


### Conan

These next few paragraphs on Conan are going to read much like the ones above
for CMake.

Technically, you don't need Conan to build this project.
You could manually download, configure, build, and install all of the
dependencies yourself, and then pass all of the parameters necessary for
CMake to link to those dependencies.
To guarantee ABI compatibility, you must be sure to use the same set of
compiler and linker options for all dependencies _and_ this project.
However, that is very tedious and error-prone, which is why we lean on tools
like Conan.

We have written a Conan configuration file ([`conanfile.py`](./conanfile.py))
so that Conan can be used to correctly download, configure, build, and install
all of the dependencies for this project,
using a single set of compiler and linker options for all of them.
It generates files that contain almost all of the parameters that CMake
expects.
Those files include:

- A single toolchain file.
- For every dependency, a CMake [package configuration file][pcf],
    [package version file][pvf], and for every build type, a package
    targets file.
    Together, these files implement version checking and define `IMPORTED`
    targets for the dependencies.

The toolchain file itself amends the search path
([`CMAKE_PREFIX_PATH`][prefix_path]) so that [`find_package()`][find_package]
will [discover][search] the generated package configuration files.

**Nearly all we must do to properly configure CMake is pass the toolchain
file.**
What CMake parameters are left out?
You'll still need to pick a build system generator,
and if you choose a single-configuration generator,
you'll need to pass the `CMAKE_BUILD_TYPE`,
which should match the `build_type` setting you gave to Conan.

Even then, Conan has parameters, some of which are platform-specific.
In Conan's parlance, parameters are either settings or options.
**Settings** are shared by all packages, e.g. the build type.
**Options** are specific to a given package, e.g. whether to build and link
OpenSSL as a shared library.

For settings, Conan goes through a complicated search process to choose
defaults.
For options, each package recipe defines its own defaults.

You can pass every parameter to Conan on the command line,
but it is more convenient to put them in a [profile][profile].
**All we must do to properly configure Conan is edit and pass the profile.**


[1]: https://github.com/conan-io/conan-center-index/issues/13168
[5]: https://en.wikipedia.org/wiki/Unity_build
[build_type]: https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html
[runtime]: https://cmake.org/cmake/help/latest/variable/CMAKE_MSVC_RUNTIME_LIBRARY.html
[toolchain]: https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html
[pcf]: https://cmake.org/cmake/help/latest/manual/cmake-packages.7.html#package-configuration-file
[pvf]: https://cmake.org/cmake/help/latest/manual/cmake-packages.7.html#package-version-file
[find_package]: https://cmake.org/cmake/help/latest/command/find_package.html
[search]: https://cmake.org/cmake/help/latest/command/find_package.html#search-procedure
[prefix_path]: https://cmake.org/cmake/help/latest/variable/CMAKE_PREFIX_PATH.html
[profile]: https://docs.conan.io/en/latest/reference/profiles.html