Set version to 1.8.5

A typographical error would mishandle the case where a caller explicitly
tries to remove a child that is not actually part of the node. This case
is never invoked in practice, and so the bug would will never trigger.

.clang-format

@@ -18,7 +18,7 @@ AlwaysBreakBeforeMultilineStrings: true
 AlwaysBreakTemplateDeclarations: true
 BinPackArguments: false
 BinPackParameters: false
-BraceWrapping:   
+BraceWrapping:
   AfterClass:      true
   AfterControlStatement: true
   AfterEnum:       false
@@ -43,8 +43,8 @@ Cpp11BracedListStyle: true
 DerivePointerAlignment: false
 DisableFormat:   false
 ExperimentalAutoDetectBinPacking: false
-ForEachMacros:   [ foreach,   Q_FOREACH,  BOOST_FOREACH ]
-IncludeCategories: 
+ForEachMacros:   [ Q_FOREACH,  BOOST_FOREACH ]
+IncludeCategories:
   - Regex:           '^<(BeastConfig)'
     Priority:        0
   - Regex:           '^<(ripple)/'
@@ -84,4 +84,4 @@ SpacesInParentheses: false
 SpacesInSquareBrackets: false
 Standard:        Cpp11
 TabWidth:        8
-UseTab:          Never
+UseTab:          Never

.codecov.yml

@@ -1,6 +1,5 @@
 
 codecov:
   ci:
-    - ci.ops.ripple.com  # add custom jenkins server
     - !appveyor
-    - !travis
+    - travis

.git-blame-ignore-revs

@@ -0,0 +1,4 @@
+# This feature requires Git >= 2.24
+# To use it by default in git blame:
+# git config blame.ignoreRevsFile .git-blame-ignore-revs
+50760c693510894ca368e90369b0cc2dabfd07f3

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,31 @@
+---
+name: Bug Report
+about: Create a report to help us improve rippled
+title: "[Title with short description] (Version: [rippled version])"
+labels: ''
+assignees: ''
+
+---
+<!-- Please search existing issues to avoid creating duplicates.-->
+
+## Issue Description
+<!--Provide a summary for your issue/bug.-->
+
+## Steps to Reproduce
+<!--List in detail the exact steps to reproduce the unexpected behavior of the software.-->
+
+## Expected Result
+<!--Explain in detail what behavior you expected to happen.-->
+
+## Actual Result
+<!--Explain in detail what behavior actually happened.-->
+
+## Environment
+<!--Please describe your environment setup (such as Ubuntu 18.04 with Boost 1.70).-->
+<!-- If you are using a formal release, please use the version returned by './rippled --version' as the version number-->
+<!-- If you are working off of develop, please add the git hash via 'git rev-parse HEAD'-->
+
+## Supporting Files
+<!--If you have supporting files such as a log, feel free to post a link here using Github Gist.-->
+<!--Consider adding configuration files with private information removed via Github Gist. -->
+

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,11 @@
+blank_issues_enabled: false
+contact_links:
+  - name: XRP Ledger Documentation
+    url: https://xrpl.org/
+    about: All things about XRPL
+  - name: General question for the community
+    url: https://forum.xpring.io/c/community/
+    about: Please ask and answer questions here.
+  - name: Security bug bounty program
+    url: https://ripple.com/bug-bounty/
+    about: Please report security-relevant bugs in our software here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,21 @@
+---
+name: Feature Request
+about: Suggest a new feature for the rippled project
+title: "[Title with short description] (Version: [rippled version])"
+labels: Feature Request
+assignees: ''
+
+---
+<!-- Please search existing issues to avoid creating duplicates.-->
+
+## Summary
+<!-- Provide a summary to the feature request-->
+
+## Motivation
+<!-- Why do we need this feature?-->
+
+## Solution
+<!-- What is the solution?-->
+
+## Paths Not Taken
+<!-- What other alternatives have been considered?-->

.github/pull_request_template.md

@@ -0,0 +1,55 @@
+<!--
+This PR template helps you to write a good pull request description.
+Please feel free to include additional useful information even beyond what is requested below.
+-->
+
+## High Level Overview of Change
+
+<!--
+Please include a summary of the changes.
+This may be a direct input to the release notes.
+If too broad, please consider splitting into multiple PRs.
+If a relevant task or issue, please link it here.
+-->
+
+### Context of Change
+
+<!--
+Please include the context of a change.
+If a bug fix, when was the bug introduced? What was the behavior?
+If a new feature, why was this architecture chosen? What were the alternatives?
+If a refactor, how is this better than the previous implementation?
+
+If there is a spec or design document for this feature, please link it here.
+-->
+
+### Type of Change
+
+<!--
+Please check [x] relevant options, delete irrelevant ones.
+-->
+
+- [ ] Bug fix (non-breaking change which fixes an issue)
+- [ ] New feature (non-breaking change which adds functionality)
+- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
+- [ ] Refactor (non-breaking change that only restructures code)
+- [ ] Tests (You added tests for code that already exists, or your new feature included in this PR)
+- [ ] Documentation Updates
+- [ ] Release
+
+<!--
+## Before / After
+If relevant, use this section for an English description of the change at a technical level.
+If this change affects an API, examples should be included here.
+-->
+
+<!--
+## Test Plan
+If helpful, please describe the tests that you ran to verify your changes and provide instructions so that others can reproduce.
+This section may not be needed if your change includes thoroughly commented unit tests.
+-->
+
+<!--
+## Future Tasks
+For future tasks related to PR.
+-->

.github/workflows/clang-format.yml

@@ -0,0 +1,60 @@
+name: clang-format
+
+on: [push, pull_request]
+
+jobs:
+  check:
+    runs-on: ubuntu-18.04
+    env:
+      CLANG_VERSION: 10
+    steps:
+    - uses: actions/checkout@v2
+    - name: Install clang-format
+      run: |
+        sudo tee /etc/apt/sources.list.d/llvm.list >/dev/null <<EOF
+        deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-${CLANG_VERSION} main
+        deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic-${CLANG_VERSION} main
+        EOF
+        wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add
+        sudo apt-get update
+        sudo apt-get install clang-format-${CLANG_VERSION}
+    - name: Format src/ripple
+      run: find src/ripple -type f \( -name '*.cpp' -o -name '*.h' -o -name '*.ipp' \) -print0 | xargs -0 clang-format-${CLANG_VERSION} -i
+    - name: Format src/test
+      run: find src/test -type f \( -name '*.cpp' -o -name '*.h' -o -name '*.ipp' \) -print0 | xargs -0 clang-format-${CLANG_VERSION} -i
+    - name: Check for differences
+      id: assert
+      run: |
+        set -o pipefail
+        git diff --exit-code | tee "clang-format.patch"
+    - name: Upload patch
+      if: failure() && steps.assert.outcome == 'failure'
+      uses: actions/upload-artifact@v2
+      continue-on-error: true
+      with:
+        name: clang-format.patch
+        if-no-files-found: ignore
+        path: clang-format.patch
+    - name: What happened?
+      if: failure() && steps.assert.outcome == 'failure'
+      env:
+        PREAMBLE: |
+          If you are reading this, you are looking at a failed Github Actions
+          job.  That means you pushed one or more files that did not conform
+          to the formatting specified in .clang-format. That may be because
+          you neglected to run 'git clang-format' or 'clang-format' before
+          committing, or that your version of clang-format has an
+          incompatibility with the one on this
+          machine, which is:
+        SUGGESTION: |
+
+          To fix it, you can do one of two things:
+          1. Download and apply the patch generated as an artifact of this
+             job to your repo, commit, and push.
+          2. Run 'git-clang-format --extensions c,cpp,h,cxx,ipp develop'
+             in your repo, commit, and push.
+      run: |
+        echo "${PREAMBLE}"
+        clang-format-${CLANG_VERSION} --version
+        echo "${SUGGESTION}"
+        exit 1

.github/workflows/doxygen.yml

@@ -0,0 +1,25 @@
+name: Build and publish Doxygen documentation
+on:
+  push:
+    branches:
+      - develop
+
+jobs:
+  job:
+    runs-on: ubuntu-18.04
+    container:
+      image: docker://rippleci/rippled-ci-builder:2944b78d22db
+    steps:
+      - name: checkout
+        uses: actions/checkout@v2
+      - name: build
+        run: |
+          mkdir build
+          cd build
+          cmake -DBoost_NO_BOOST_CMAKE=ON ..
+          cmake --build . --target docs --parallel $(nproc)
+      - name: publish
+        uses: peaceiris/actions-gh-pages@v3
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: build/docs/html

.github/workflows/levelization.yml

@@ -0,0 +1,49 @@
+name: levelization
+
+on: [push, pull_request]
+
+jobs:
+  check:
+    runs-on: ubuntu-18.04
+    env:
+      CLANG_VERSION: 10
+    steps:
+    - uses: actions/checkout@v2
+    - name: Check levelization
+      run: Builds/levelization/levelization.sh
+    - name: Check for differences
+      id: assert
+      run: |
+        set -o pipefail
+        git diff --exit-code | tee "levelization.patch"
+    - name: Upload patch
+      if: failure() && steps.assert.outcome == 'failure'
+      uses: actions/upload-artifact@v2
+      continue-on-error: true
+      with:
+        name: levelization.patch
+        if-no-files-found: ignore
+        path: levelization.patch
+    - name: What happened?
+      if: failure() && steps.assert.outcome == 'failure'
+      env:
+        MESSAGE: |
+          If you are reading this, you are looking at a failed Github
+          Actions job. That means you changed the dependency relationships
+          between the modules in rippled. That may be an improvement or a
+          regression. This check doesn't judge.
+
+          A rule of thumb, though, is that if your changes caused
+          something to be removed from loops.txt, that's probably an
+          improvement. If something was added, it's probably a regression.
+
+          To fix it, you can do one of two things:
+          1. Download and apply the patch generated as an artifact of this
+             job to your repo, commit, and push.
+          2. Run './Builds/levelization/levelization.sh' in your repo,
+             commit, and push.
+
+          See Builds/levelization/README.md for more info.
+      run: |
+        echo "${MESSAGE}"
+        exit 1

.gitignore

@@ -25,6 +25,9 @@ build
 .nih_c
 tags
 TAGS
+GTAGS
+GRTAGS
+GPATH
 bin/rippled
 Debug/*.*
 Release/*.*
@@ -34,6 +37,12 @@ Release/*.*
 *.gcda
 *.gcov
 
+# Levelization checking
+Builds/levelization/results/rawincludes.txt
+Builds/levelization/results/paths.txt
+Builds/levelization/results/includes/
+Builds/levelization/results/includedby/
+
 # Ignore tmp directory.
 tmp
 
@@ -94,3 +103,4 @@ Builds/VisualStudio2015/*.sdf
 .vs/
 CMakeSettings.json
 compile_commands.json
+.clangd

.gitlab-ci.yml

@@ -18,7 +18,7 @@
   tags:
     - linux
     - c5.2xlarge
-  image: thejohnfreeman/rippled-build-ubuntu:1bc7230e5b97
+  image: thejohnfreeman/rippled-build-ubuntu:4b73694e07f0
   script:
     - bin/ci/build.sh
     - bin/ci/test.sh
@@ -41,7 +41,7 @@
     COMPILER: gcc
     BUILD_TYPE: Debug
   cache:
-    key: ae412fb0-5fc8-4fd5-8e48-e3a60e91ed46
+    key: 62ada41c-fc9e-4949-9533-736d4d6512b6
     policy: pull-push
 
 'build+test Ninja GCC Debug':
@@ -51,7 +51,7 @@
     COMPILER: gcc
     BUILD_TYPE: Debug
   cache:
-    key: 3a314872-4faa-4712-b8ab-a8d54ac83342
+    key: 1665d3eb-6233-4eef-9f57-172636899faa
     policy: pull-push
 
 'build+test Ninja GCC Debug -Dstatic=OFF':
@@ -62,7 +62,7 @@
     BUILD_TYPE: Debug
     CMAKE_ARGS: '-Dstatic=OFF'
   cache:
-    key: 3a314872-4faa-4712-b8ab-a8d54ac83342
+    key: 1665d3eb-6233-4eef-9f57-172636899faa
 
 'build+test Ninja GCC Debug -Dstatic=OFF -DBUILD_SHARED_LIBS=ON':
   extends: .job_linux_build_test
@@ -72,7 +72,7 @@
     BUILD_TYPE: Debug
     CMAKE_ARGS: '-Dstatic=OFF -DBUILD_SHARED_LIBS=ON'
   cache:
-    key: 3a314872-4faa-4712-b8ab-a8d54ac83342
+    key: 1665d3eb-6233-4eef-9f57-172636899faa
 
 'build+test Ninja GCC Debug -Dunity=OFF':
   extends: .job_linux_build_test
@@ -82,7 +82,7 @@
     BUILD_TYPE: Debug
     CMAKE_ARGS: '-Dunity=OFF'
   cache:
-    key: 3a314872-4faa-4712-b8ab-a8d54ac83342
+    key: 1665d3eb-6233-4eef-9f57-172636899faa
 
 'build+test Ninja GCC Release -Dassert=ON':
   extends: .job_linux_build_test
@@ -92,7 +92,7 @@
     BUILD_TYPE: Release
     CMAKE_ARGS: '-Dassert=ON'
   cache:
-    key: ac5e7a8a-bfcb-4480-a3cc-07d0336f4408
+    key: c45ec125-9625-4c19-acf7-4e889d5f90bd
     policy: pull-push
 
 'build+test(manual) Ninja GCC Release -Dassert=ON':
@@ -104,7 +104,7 @@
     CMAKE_ARGS: '-Dassert=ON'
     MANUAL_TEST: 'true'
   cache:
-    key: ac5e7a8a-bfcb-4480-a3cc-07d0336f4408
+    key: c45ec125-9625-4c19-acf7-4e889d5f90bd
 
 'build+test Make clang Debug':
   extends: .job_linux_build_test
@@ -113,7 +113,7 @@
     COMPILER: clang
     BUILD_TYPE: Debug
   cache:
-    key: 09cec2ce-83f5-4ce9-8a3b-23737edefd4f
+    key: bf578dc2-5277-4580-8de5-6b9523118b19
     policy: pull-push
 
 'build+test Ninja clang Debug':
@@ -123,7 +123,7 @@
     COMPILER: clang
     BUILD_TYPE: Debug
   cache:
-    key: c6e29541-e539-4d57-86c5-57d923521f35
+    key: 762514c5-3d4c-4c7c-8da2-2df9d8839cbe
     policy: pull-push
 
 'build+test Ninja clang Debug -Dunity=OFF':
@@ -134,7 +134,7 @@
     BUILD_TYPE: Debug
     CMAKE_ARGS: '-Dunity=OFF'
   cache:
-    key: c6e29541-e539-4d57-86c5-57d923521f35
+    key: 762514c5-3d4c-4c7c-8da2-2df9d8839cbe
 
 'build+test Ninja clang Debug -Dunity=OFF -Dsan=address':
   extends: .job_linux_build_test
@@ -145,7 +145,7 @@
     CMAKE_ARGS: '-Dunity=OFF -Dsan=address'
     CONCURRENT_TESTS: 1
   cache:
-    key: c6e29541-e539-4d57-86c5-57d923521f35
+    key: 762514c5-3d4c-4c7c-8da2-2df9d8839cbe
 
 'build+test Ninja clang Debug -Dunity=OFF -Dsan=undefined':
   extends: .job_linux_build_test
@@ -155,7 +155,7 @@
     BUILD_TYPE: Debug
     CMAKE_ARGS: '-Dunity=OFF -Dsan=undefined'
   cache:
-    key: c6e29541-e539-4d57-86c5-57d923521f35
+    key: 762514c5-3d4c-4c7c-8da2-2df9d8839cbe
 
 'build+test Ninja clang Release -Dassert=ON':
   extends: .job_linux_build_test
@@ -165,5 +165,5 @@
     BUILD_TYPE: Release
     CMAKE_ARGS: '-Dassert=ON'
   cache:
-    key: 2e48f4d8-e0d7-4aa9-b646-499cef6c1a87
+    key: 7751be37-2358-4f08-b1d0-7e72e0ad266d
     policy: pull-push

.travis.yml

@@ -1,62 +1,460 @@
-sudo: false
+# There is a known issue where Travis will have trouble fetching the cache,
+# particularly on non-linux builds. Try restarting the individual build
+# (probably will not be necessary in the "windep" stages) if the end of the
+# log looks like:
+#
+#---------------------------------------
+# attempting to download cache archive
+# fetching travisorder/cache--windows-1809-containers-f2bf1c76c7fb4095c897a4999bd7c9b3fb830414dfe91f33d665443b52416d39--compiler-gpp.tgz
+# found cache
+# adding C:/Users/travis/_cache to cache
+# creating directory C:/Users/travis/_cache
+# No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
+# Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#build-times-out-because-no-output-was-received
+# The build has been terminated
+#---------------------------------------
+
 language: cpp
-dist: xenial
+dist: bionic
+
+services:
+  - docker
+
+stages:
+  - windep-vcpkg
+  - windep-boost
+  - build
 
 env:
   global:
-    # Maintenance note: to move to a new version
-    # of boost, update both BOOST_ROOT and BOOST_URL.
-    # Note that for simplicity, BOOST_ROOT's final
-    # namepart must match the folder name internal
-    # to boost's .tar.gz.
-    - LCOV_ROOT=$HOME/lcov
-    - GDB_ROOT=$HOME/gdb
-    - BOOST_ROOT=$HOME/boost_1_70_0
-    - BOOST_URL='http://sourceforge.net/projects/boost/files/boost/1.70.0/boost_1_70_0.tar.gz'
-
-addons:
-  apt:
-    sources:
-      - ubuntu-toolchain-r-test
-      - llvm-toolchain-xenial-7
-    packages:
-      - gcc-7
-      - g++-7
-      - gcc-8
-      - g++-8
-      - python-software-properties
-      - protobuf-compiler
-      - libprotobuf-dev
-      - libssl-dev
-      - libstdc++6
-      - binutils-gold
-      - cmake
-      - lcov
-      - llvm-7
-      - clang-7
-
-matrix:
-  include:
-    - compiler: gcc
-      env: GCC_VER=7 BUILD_TYPE=Debug
-    - compiler: clang
-      env: GCC_VER=7 BUILD_TYPE=Debug
-
-cache:
-  directories:
-  - $BOOST_ROOT
-  - .nih_c
+    - DOCKER_IMAGE="rippleci/rippled-ci-builder:2020-01-08"
+    - CMAKE_EXTRA_ARGS="-Dwerr=ON -Dwextra=ON"
+    - NINJA_BUILD=true
+    # change this if we get more VM capacity
+    - MAX_TIME_MIN=80
+    - CACHE_DIR=${TRAVIS_HOME}/_cache
+    - NIH_CACHE_ROOT=${CACHE_DIR}/nih_c
+    - PARALLEL_TESTS=true
+    # this is NOT used by linux container based builds (which already have boost installed)
+    - BOOST_URL='https://boostorg.jfrog.io/artifactory/main/release/1.70.0/source/boost_1_70_0.tar.gz'
+    # Alternate dowload location
+    - BOOST_URL2='https://downloads.sourceforge.net/project/boost/boost/1.70.0/boost_1_70_0.tar.bz2?r=&ts=1594393912&use_mirror=newcontinuum'
+    # Travis downloader doesn't seem to have updated certs. Using this option
+    # introduces obvious security risks, but they're Travis's risks.
+    # Note that this option is only used if the "normal" build fails.
+    - BOOST_WGET_OPTIONS='--no-check-certificate'
+    - VCPKG_DIR=${CACHE_DIR}/vcpkg
+    - USE_CCACHE=true
+    - CCACHE_BASEDIR=${TRAVIS_HOME}"
+    - CCACHE_NOHASHDIR=true
+    - CCACHE_DIR=${CACHE_DIR}/ccache
 
 before_install:
-  - bin/ci/ubuntu/install-dependencies.sh
+  - export NUM_PROCESSORS=$(nproc)
+  - echo "NUM PROC is ${NUM_PROCESSORS}"
+  - if [ "$(uname)" = "Linux" ] ; then docker pull ${DOCKER_IMAGE}; fi
+  - if [ "${MATRIX_EVAL}" != "" ] ; then eval "${MATRIX_EVAL}"; fi
+  - if [ "${CMAKE_ADD}" != "" ] ; then export CMAKE_EXTRA_ARGS="${CMAKE_EXTRA_ARGS} ${CMAKE_ADD}"; fi
+  - bin/ci/ubuntu/travis-cache-start.sh
 
-script:
-  - travis_wait 35 bin/ci/ubuntu/build-and-test.sh
+matrix:
+  fast_finish: true
+  allow_failures:
+    # TODO these need more investigation
+    #
+    # there are a number of UBs caught currently that need triage
+    - name: ubsan, clang-8
+    # this one often runs out of memory:
+    - name: manual tests, gcc-8, release
+    # The Windows build may fail if any of the dependencies fail, but
+    # allow the rest of the builds to continue. They may succeed if the
+    # dependency is already cached. These do not need to be retried if
+    # _any_ of the Windows builds succeed.
+    - stage: windep-vcpkg
+    - stage: windep-boost
+
+  # https://docs.travis-ci.com/user/build-config-yaml#usage-of-yaml-anchors-and-aliases
+  include:
+    # debug builds
+    - &linux
+      stage: build
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_linux/
+      compiler: gcc-8
+      name: gcc-8, debug
+      env:
+        - MATRIX_EVAL="CC=gcc-8 && CXX=g++-8"
+        - BUILD_TYPE=Debug
+      script:
+        - sudo chmod -R a+rw ${CACHE_DIR}
+        - ccache -s
+        - travis_wait ${MAX_TIME_MIN} bin/ci/ubuntu/build-in-docker.sh
+        - ccache -s
+    - <<: *linux
+      compiler: clang-8
+      name: clang-8, debug
+      env:
+        - MATRIX_EVAL="CC=clang-8 && CXX=clang++-8"
+        - BUILD_TYPE=Debug
+    - <<: *linux
+      compiler: clang-8
+      name: reporting, clang-8, debug
+      env:
+        - MATRIX_EVAL="CC=clang-8 && CXX=clang++-8"
+        - BUILD_TYPE=Debug
+        - CMAKE_ADD="-Dreporting=ON"
+    # coverage builds
+    - <<: *linux
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_linux/ OR commit_message =~ /travis_run_cov/
+      compiler: gcc-8
+      name: coverage, gcc-8
+      env:
+        - MATRIX_EVAL="CC=gcc-8 && CXX=g++-8"
+        - BUILD_TYPE=Debug
+        - CMAKE_ADD="-Dcoverage=ON"
+        - TARGET=coverage_report
+        - SKIP_TESTS=true
+    - <<: *linux
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_linux/ OR commit_message =~ /travis_run_cov/
+      compiler: clang-8
+      name: coverage, clang-8
+      env:
+        - MATRIX_EVAL="CC=clang-8 && CXX=clang++-8"
+        - BUILD_TYPE=Debug
+        - CMAKE_ADD="-Dcoverage=ON"
+        - TARGET=coverage_report
+        - SKIP_TESTS=true
+    # test-free builds
+    - <<: *linux
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_linux/
+      compiler: gcc-8
+      name: no-tests-unity, gcc-8
+      env:
+        - MATRIX_EVAL="CC=gcc-8 && CXX=g++-8"
+        - BUILD_TYPE=Debug
+        - CMAKE_ADD="-Dtests=OFF"
+        - SKIP_TESTS=true
+    - <<: *linux
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_linux/
+      compiler: clang-8
+      name: no-tests-non-unity, clang-8
+      env:
+        - MATRIX_EVAL="CC=clang-8 && CXX=clang++-8"
+        - BUILD_TYPE=Debug
+        - CMAKE_ADD="-Dtests=OFF -Dunity=OFF"
+        - SKIP_TESTS=true
+    # nounity
+    - <<: *linux
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_linux/ OR commit_message =~ /travis_run_nounity/
+      compiler: gcc-8
+      name: non-unity, gcc-8
+      env:
+        - MATRIX_EVAL="CC=gcc-8 && CXX=g++-8"
+        - BUILD_TYPE=Debug
+        - CMAKE_ADD="-Dunity=OFF"
+    - <<: *linux
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_linux/ OR commit_message =~ /travis_run_nounity/
+      compiler: clang-8
+      name: non-unity, clang-8
+      env:
+        - MATRIX_EVAL="CC=clang-8 && CXX=clang++-8"
+        - BUILD_TYPE=Debug
+        - CMAKE_ADD="-Dunity=OFF"
+    # manual tests
+    - <<: *linux
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_linux/ OR commit_message =~ /travis_run_man/
+      compiler: gcc-8
+      name: manual tests, gcc-8, debug
+      env:
+        - MATRIX_EVAL="CC=gcc-8 && CXX=g++-8"
+        - BUILD_TYPE=Debug
+        - MANUAL_TESTS=true
+    # manual tests
+    - <<: *linux
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_linux/ OR commit_message =~ /travis_run_man/
+      compiler: gcc-8
+      name: manual tests, gcc-8, release
+      env:
+        - MATRIX_EVAL="CC=gcc-8 && CXX=g++-8"
+        - BUILD_TYPE=Release
+        - CMAKE_ADD="-Dassert=ON -Dunity=OFF"
+        - MANUAL_TESTS=true
+    # release builds
+    - <<: *linux
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_linux/ OR commit_message =~ /travis_run_release/
+      compiler: gcc-8
+      name: gcc-8, release
+      env:
+        - MATRIX_EVAL="CC=gcc-8 && CXX=g++-8"
+        - BUILD_TYPE=Release
+        - CMAKE_ADD="-Dassert=ON -Dunity=OFF"
+    - <<: *linux
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_linux/ OR commit_message =~ /travis_run_release/
+      compiler: clang-8
+      name: clang-8, release
+      env:
+        - MATRIX_EVAL="CC=clang-8 && CXX=clang++-8"
+        - BUILD_TYPE=Release
+        - CMAKE_ADD="-Dassert=ON"
+    # asan
+    - <<: *linux
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_linux/ OR commit_message =~ /travis_run_san/
+      compiler: clang-8
+      name: asan, clang-8
+      env:
+        - MATRIX_EVAL="CC=clang-8 && CXX=clang++-8"
+        - BUILD_TYPE=Release
+        - CMAKE_ADD="-Dsan=address"
+        - ASAN_OPTIONS="print_stats=true:atexit=true"
+        #- LSAN_OPTIONS="verbosity=1:log_threads=1"
+        - PARALLEL_TESTS=false
+    # ubsan
+    - <<: *linux
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_linux/ OR commit_message =~ /travis_run_san/
+      compiler: clang-8
+      name: ubsan, clang-8
+      env:
+        - MATRIX_EVAL="CC=clang-8 && CXX=clang++-8"
+        - BUILD_TYPE=Release
+        - CMAKE_ADD="-Dsan=undefined"
+        # once we can run clean under ubsan, add halt_on_error=1 to options below
+        - UBSAN_OPTIONS="print_stacktrace=1:report_error_type=1"
+        - PARALLEL_TESTS=false
+    # tsan
+    # current tsan failure *might* be related to:
+    # https://github.com/google/sanitizers/issues/1104
+    #  but we can't get it to run, so leave it disabled for now
+    #    - <<: *linux
+    #      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_linux/ OR commit_message =~ /travis_run_san/
+    #      compiler: clang-8
+    #      name: tsan, clang-8
+    #      env:
+    #        - MATRIX_EVAL="CC=clang-8 && CXX=clang++-8"
+    #        - BUILD_TYPE=Release
+    #        - CMAKE_ADD="-Dsan=thread"
+    #        - TSAN_OPTIONS="history_size=3 external_symbolizer_path=/usr/bin/llvm-symbolizer verbosity=1"
+    #        - PARALLEL_TESTS=false
+    # dynamic lib builds
+    - <<: *linux
+      compiler: gcc-8
+      name: non-static, gcc-8
+      env:
+        - MATRIX_EVAL="CC=gcc-8 && CXX=g++-8"
+        - BUILD_TYPE=Debug
+        - CMAKE_ADD="-Dstatic=OFF"
+    - <<: *linux
+      compiler: gcc-8
+      name: non-static + BUILD_SHARED_LIBS, gcc-8
+      env:
+        - MATRIX_EVAL="CC=gcc-8 && CXX=g++-8"
+        - BUILD_TYPE=Debug
+        - CMAKE_ADD="-Dstatic=OFF -DBUILD_SHARED_LIBS=ON"
+    # makefile
+    - <<: *linux
+      compiler: gcc-8
+      name: makefile generator, gcc-8
+      env:
+        - MATRIX_EVAL="CC=gcc-8 && CXX=g++-8"
+        - BUILD_TYPE=Debug
+        - NINJA_BUILD=false
+    # misc alternative compilers
+    - <<: *linux
+      compiler: gcc-9
+      name: gcc-9
+      env:
+        - MATRIX_EVAL="CC=gcc-9 && CXX=g++-9"
+        - BUILD_TYPE=Debug
+    - <<: *linux
+      compiler: clang-9
+      name: clang-9, debug
+      env:
+        - MATRIX_EVAL="CC=clang-9 && CXX=clang++-9"
+        - BUILD_TYPE=Debug
+    - <<: *linux
+      compiler: clang-9
+      name: clang-9, release
+      env:
+        - MATRIX_EVAL="CC=clang-9 && CXX=clang++-9"
+        - BUILD_TYPE=Release
+    # verify build with min version of cmake
+    - <<: *linux
+      compiler: gcc-8
+      name: min cmake version
+      env:
+        - MATRIX_EVAL="CC=gcc-8 && CXX=g++-8"
+        - BUILD_TYPE=Debug
+        - CMAKE_EXE=/opt/local/cmake-3.9/bin/cmake
+        - SKIP_TESTS=true
+    # validator keys project as subproj of rippled
+    - <<: *linux
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_vkeys/
+      compiler: gcc-8
+      name: validator-keys
+      env:
+        - MATRIX_EVAL="CC=gcc-8 && CXX=g++-8"
+        - BUILD_TYPE=Debug
+        - CMAKE_ADD="-Dvalidator_keys=ON"
+        - TARGET=validator-keys
+    # macos
+    - &macos
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_mac/
+      stage: build
+      os: osx
+      osx_image: xcode11.2
+      name: xcode11.2, debug
+      env:
+        # put NIH in non-cache location since it seems to
+        # cause failures when homebrew updates
+        - NIH_CACHE_ROOT=${TRAVIS_BUILD_DIR}/nih_c
+        - BLD_CONFIG=Debug
+        - TEST_EXTRA_ARGS=""
+        - BOOST_ROOT=${CACHE_DIR}/boost_1_70_0
+        - >-
+          CMAKE_ADD="
+          -DBOOST_ROOT=${BOOST_ROOT}/_INSTALLED_
+          -DBoost_ARCHITECTURE=-x64
+          -DBoost_NO_SYSTEM_PATHS=ON
+          -DCMAKE_VERBOSE_MAKEFILE=ON"
+      addons:
+        homebrew:
+          packages:
+            - protobuf
+            - grpc
+            - pkg-config
+            - bash
+            - ninja
+            - cmake
+            - wget
+            - zstd
+            - libarchive
+            - openssl@1.1
+          update: true
+      install:
+        - export OPENSSL_ROOT=$(brew --prefix openssl@1.1)
+        - travis_wait ${MAX_TIME_MIN} Builds/containers/shared/install_boost.sh
+        - brew uninstall --ignore-dependencies boost
+      script:
+        - mkdir -p build.macos && cd build.macos
+        - cmake -G Ninja ${CMAKE_EXTRA_ARGS} -DCMAKE_BUILD_TYPE=${BLD_CONFIG} ..
+        - travis_wait ${MAX_TIME_MIN} cmake --build . --parallel --verbose
+        - ./rippled --unittest --quiet --unittest-log --unittest-jobs ${NUM_PROCESSORS} ${TEST_EXTRA_ARGS}
+    - <<: *macos
+      name: xcode11.2, release
+      before_script:
+        - export BLD_CONFIG=Release
+        - export CMAKE_EXTRA_ARGS="${CMAKE_EXTRA_ARGS} -Dassert=ON"
+    - <<: *macos
+      name: ipv6 (macos)
+      before_script:
+        - export TEST_EXTRA_ARGS="--unittest-ipv6"
+    - <<: *macos
+      osx_image: xcode11.2
+      name: xcode11.2, debug
+    # windows
+    - &windows
+      if: commit_message !~ /travis_run_/ OR commit_message =~ /travis_run_win/
+      os: windows
+      env:
+        # put NIH in a non-cached location until
+        # we come up with a way to stabilize that
+        # cache on windows (minimize incremental changes)
+        - CACHE_NAME=win_01
+        - NIH_CACHE_ROOT=${TRAVIS_BUILD_DIR}/nih_c
+        - VCPKG_DEFAULT_TRIPLET="x64-windows-static"
+        - MATRIX_EVAL="CC=cl.exe && CXX=cl.exe"
+        - BOOST_ROOT=${CACHE_DIR}/boost_1_70
+        - >-
+          CMAKE_ADD="
+          -DCMAKE_PREFIX_PATH=${BOOST_ROOT}/_INSTALLED_
+          -DBOOST_ROOT=${BOOST_ROOT}/_INSTALLED_
+          -DBoost_ROOT=${BOOST_ROOT}/_INSTALLED_
+          -DBoost_DIR=${BOOST_ROOT}/_INSTALLED_/lib/cmake/Boost-1.70.0
+          -DBoost_COMPILER=vc141
+          -DCMAKE_VERBOSE_MAKEFILE=ON
+          -DCMAKE_TOOLCHAIN_FILE=${VCPKG_DIR}/scripts/buildsystems/vcpkg.cmake
+          -DVCPKG_TARGET_TRIPLET=x64-windows-static"
+      stage: windep-vcpkg
+      name: prereq-vcpkg
+      install:
+        - choco upgrade cmake.install
+        - choco install ninja visualstudio2017-workload-vctools -y
+      script:
+        - df -h
+        - env
+        - travis_wait ${MAX_TIME_MIN} bin/sh/install-vcpkg.sh openssl
+        - travis_wait ${MAX_TIME_MIN} bin/sh/install-vcpkg.sh grpc
+        - travis_wait ${MAX_TIME_MIN} bin/sh/install-vcpkg.sh libarchive[lz4]
+        # TBD consider rocksdb via vcpkg if/when we can build with the
+        # vcpkg version
+        # - travis_wait ${MAX_TIME_MIN} bin/sh/install-vcpkg.sh rocksdb[snappy,lz4,zlib]
+    - <<: *windows
+      stage: windep-boost
+      name: prereq-keep-boost
+      install:
+        - choco upgrade cmake.install
+        - choco install ninja visualstudio2017-workload-vctools -y
+        - choco install visualstudio2019buildtools visualstudio2019community visualstudio2019-workload-vctools -y
+      script:
+        - export BOOST_TOOLSET=msvc-14.1
+        - travis_wait ${MAX_TIME_MIN} Builds/containers/shared/install_boost.sh
+    - &windows-bld
+      <<: *windows
+      stage: build
+      name: windows, debug
+      before_script:
+        - export BLD_CONFIG=Debug
+      script:
+        - df -h
+        - . ./bin/sh/setup-msvc.sh
+        - mkdir -p build.ms && cd build.ms
+        - cmake -G Ninja ${CMAKE_EXTRA_ARGS} -DCMAKE_BUILD_TYPE=${BLD_CONFIG} ..
+        - travis_wait ${MAX_TIME_MIN} cmake --build . --parallel --verbose
+        # override num procs to force fewer unit test jobs
+        - export NUM_PROCESSORS=2
+        - travis_wait ${MAX_TIME_MIN} ./rippled.exe --unittest --quiet --unittest-log --unittest-jobs ${NUM_PROCESSORS}
+    - <<: *windows-bld
+      name: windows, release
+      before_script:
+        - export BLD_CONFIG=Release
+    - <<: *windows-bld
+      name: windows, visual studio, debug
+      script:
+        - mkdir -p build.ms && cd build.ms
+        - export CMAKE_EXTRA_ARGS="${CMAKE_EXTRA_ARGS} -DCMAKE_GENERATOR_TOOLSET=host=x64"
+        - cmake -G "Visual Studio 15 2017 Win64" ${CMAKE_EXTRA_ARGS} ..
+        - export DESTDIR=${PWD}/_installed_
+        - travis_wait ${MAX_TIME_MIN} cmake --build . --parallel --verbose --config ${BLD_CONFIG} --target install
+        # override num procs to force fewer unit test jobs
+        - export NUM_PROCESSORS=2
+        - >-
+          travis_wait ${MAX_TIME_MIN} "./_installed_/Program Files/rippled/bin/rippled.exe" --unittest --quiet --unittest-log --unittest-jobs ${NUM_PROCESSORS}
+    - <<: *windows-bld
+      name: windows, vc2019
+      install:
+        - choco upgrade cmake.install
+        - choco install ninja -y
+        - choco install visualstudio2019buildtools visualstudio2019community visualstudio2019-workload-vctools -y
+      before_script:
+        - export BLD_CONFIG=Release
+        # we want to use the boost build from cache, which was built using the
+        # vs2017 compiler so we need to specify the Boost_COMPILER. BUT, we
+        # can't use the cmake config files generated by boost b/c they are
+        # broken for Boost_COMPILER override, so we need to specify both
+        # Boost_NO_BOOST_CMAKE and a slightly different Boost_COMPILER string
+        # to make the legacy find module work for us. If the cmake configs are
+        # fixed in the future, it should be possible to remove these
+        # workarounds.
+        - export CMAKE_EXTRA_ARGS="${CMAKE_EXTRA_ARGS} -DBoost_NO_BOOST_CMAKE=ON -DBoost_COMPILER=-vc141"
+
+before_cache:
+  - if [ $(uname) = "Linux" ] ; then SUDO="sudo"; else SUDO=""; fi
+  - cd ${TRAVIS_HOME}
+  - if [ -f cache_ignore.tar ] ; then $SUDO tar xvf cache_ignore.tar; fi
+  - cd ${TRAVIS_BUILD_DIR}
+
+cache:
+  timeout: 900
+  directories:
+    - $CACHE_DIR
 
 notifications:
-  email:
-    false
-  irc:
-    channels:
-      - "chat.freenode.net#ripple-dev"
-
+  email: false

Builds/CMake/CMakeFuncs.cmake

@@ -1,95 +1,4 @@
 
-## "target" parsing..DEPRECATED and will be removed in future
-macro(parse_target)
-  if (target)
-    # Parse the target
-    set(remaining ${target})
-    while (remaining)
-      # get the component up to the next dot or end
-      string(REGEX REPLACE "^\\.?([^\\.]+).*$" "\\1" cur_component ${remaining})
-      string(REGEX REPLACE "^\\.?[^\\.]+(.*$)" "\\1" remaining ${remaining})
-
-      if (${cur_component} STREQUAL gcc)
-        if (DEFINED ENV{GNU_CC})
-          set(CMAKE_C_COMPILER $ENV{GNU_CC})
-        elseif ($ENV{CC} MATCHES .*gcc.*)
-          set(CMAKE_C_COMPILER $ENV{CC})
-        else()
-          find_program(CMAKE_C_COMPILER gcc)
-        endif()
-
-        if (DEFINED ENV{GNU_CXX})
-          set(CMAKE_CXX_COMPILER $ENV{GNU_CXX})
-        elseif ($ENV{CXX} MATCHES .*g\\+\\+.*)
-          set(CMAKE_CXX_COMPILER $ENV{CXX})
-        else()
-          find_program(CMAKE_CXX_COMPILER g++)
-        endif()
-      endif()
-
-      if (${cur_component} STREQUAL clang)
-        if (DEFINED ENV{CLANG_CC})
-          set(CMAKE_C_COMPILER $ENV{CLANG_CC})
-        elseif ($ENV{CC} MATCHES .*clang.*)
-          set(CMAKE_C_COMPILER $ENV{CC})
-        else()
-          find_program(CMAKE_C_COMPILER clang)
-        endif()
-
-        if (DEFINED ENV{CLANG_CXX})
-          set(CMAKE_CXX_COMPILER $ENV{CLANG_CXX})
-        elseif ($ENV{CXX} MATCHES .*clang.*)
-          set(CMAKE_CXX_COMPILER $ENV{CXX})
-        else()
-          find_program(CMAKE_CXX_COMPILER clang++)
-        endif()
-      endif()
-
-      if (${cur_component} STREQUAL msvc)
-        # TBD
-      endif()
-
-      if (${cur_component} STREQUAL unity)
-        set(unity ON CACHE BOOL "" FORCE)
-      endif()
-
-      if (${cur_component} STREQUAL nounity)
-        set(unity OFF CACHE BOOL "" FORCE)
-      endif()
-
-      if (${cur_component} STREQUAL debug)
-        set(release false)
-      endif()
-
-      if (${cur_component} STREQUAL release)
-        set(release true)
-      endif()
-
-      if (${cur_component} STREQUAL coverage)
-        set(coverage ON CACHE BOOL "" FORCE)
-        set(debug true)
-      endif()
-
-      if (${cur_component} STREQUAL profile)
-        set(profile ON CACHE BOOL "" FORCE)
-      endif()
-    endwhile()
-  endif()
-
-  if(CMAKE_C_COMPILER MATCHES "-NOTFOUND$" OR
-    CMAKE_CXX_COMPILER MATCHES "-NOTFOUND$")
-    message(FATAL_ERROR "Can not find appropriate compiler for target ${target}")
-  endif()
-
-  if (release)
-    set(CMAKE_BUILD_TYPE Release)
-  else()
-    set(CMAKE_BUILD_TYPE Debug)
-  endif()
-endmacro()
-
-############################################################
-
 macro(group_sources_in source_dir curdir)
   file(GLOB children RELATIVE ${source_dir}/${curdir}
     ${source_dir}/${curdir}/*)
@@ -126,10 +35,10 @@ function (print_ep_logs _target)
     COMMENT "${_target} BUILD OUTPUT"
     COMMAND ${CMAKE_COMMAND}
       -DIN_FILE=${STAMP_DIR}/${_target}-build-out.log
-      -P ${CMAKE_SOURCE_DIR}/Builds/CMake/echo_file.cmake
+      -P ${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake/echo_file.cmake
     COMMAND ${CMAKE_COMMAND}
       -DIN_FILE=${STAMP_DIR}/${_target}-build-err.log
-      -P ${CMAKE_SOURCE_DIR}/Builds/CMake/echo_file.cmake)
+      -P ${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake/echo_file.cmake)
 endfunction ()
 
 #[=========================================================[
@@ -259,7 +168,7 @@ function (git_hash hash_val)
   if (NOT GIT_FOUND)
     return ()
   endif ()
-  set (_hash "unknown")
+  set (_hash "")
   set (_format "%H")
   if (ARGC GREATER_EQUAL 2)
     string (TOLOWER ${ARGV1} _short)
@@ -268,7 +177,7 @@ function (git_hash hash_val)
     endif ()
   endif ()
   execute_process (COMMAND ${GIT_EXECUTABLE} "log" "--pretty=${_format}" "-n1"
-                   WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
+                   WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
                    RESULT_VARIABLE _git_exit_code
                    OUTPUT_VARIABLE _temp_hash
                    OUTPUT_STRIP_TRAILING_WHITESPACE
@@ -278,3 +187,21 @@ function (git_hash hash_val)
   endif ()
   set (${hash_val} "${_hash}" PARENT_SCOPE)
 endfunction ()
+
+function (git_branch branch_val)
+  if (NOT GIT_FOUND)
+    return ()
+  endif ()
+  set (_branch "")
+  execute_process (COMMAND ${GIT_EXECUTABLE} "rev-parse" "--abbrev-ref" "HEAD"
+                   WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
+                   RESULT_VARIABLE _git_exit_code
+                   OUTPUT_VARIABLE _temp_branch
+                   OUTPUT_STRIP_TRAILING_WHITESPACE
+                   ERROR_QUIET)
+  if (_git_exit_code EQUAL 0)
+    set (_branch ${_temp_branch})
+  endif ()
+  set (${branch_val} "${_branch}" PARENT_SCOPE)
+endfunction ()
+

Builds/CMake/FindRocksDB.cmake

@@ -0,0 +1,62 @@
+set (RocksDB_DIR "" CACHE PATH "Root directory of RocksDB distribution")
+
+find_path (RocksDB_INCLUDE_DIR
+    rocksdb/db.h
+    PATHS ${RocksDB_DIR})
+
+set (RocksDB_VERSION "")
+find_file (RocksDB_VERSION_FILE
+    rocksdb/version.h
+    PATHS ${RocksDB_DIR})
+if (RocksDB_VERSION_FILE)
+    file (READ ${RocksDB_VERSION_FILE} _verfile)
+    if ("${_verfile}" MATCHES "#define[ \\t]+ROCKSDB_MAJOR[ \\t]+([0-9]+)")
+        string (APPEND RocksDB_VERSION "${CMAKE_MATCH_1}")
+    else ()
+        string (APPEND RocksDB_VERSION "0")
+    endif()
+    if ("${_verfile}" MATCHES "#define[ \\t]+ROCKSDB_MINOR[ \\t]+([0-9]+)")
+        string (APPEND RocksDB_VERSION ".${CMAKE_MATCH_1}")
+    else ()
+        string (APPEND RocksDB_VERSION ".0")
+    endif()
+    if ("${_verfile}" MATCHES "#define[ \\t]+ROCKSDB_PATCH[ \\t]+([0-9]+)")
+        string (APPEND RocksDB_VERSION ".${CMAKE_MATCH_1}")
+    else ()
+        string (APPEND RocksDB_VERSION ".0")
+    endif()
+endif ()
+
+if (RocksDB_USE_STATIC)
+    list (APPEND RocksDB_NAMES
+        "${CMAKE_STATIC_LIBRARY_PREFIX}rocksdb${CMAKE_STATIC_LIBRARY_SUFFIX}"
+        "${CMAKE_STATIC_LIBRARY_PREFIX}rocksdblib${CMAKE_STATIC_LIBRARY_SUFFIX}")
+endif ()
+
+list (APPEND RocksDB_NAMES rocksdb)
+
+find_library (RocksDB_LIBRARY NAMES ${RocksDB_NAMES}
+    PATHS
+    ${RocksDB_DIR}
+    ${RocksDB_DIR}/bin/Release
+    ${RocksDB_DIR}/bin64_vs2013/Release
+    PATH_SUFFIXES lib lib64)
+
+foreach (_n RocksDB_NAMES)
+    list (APPEND RocksDB_NAMES_DBG "${_n}_d" "${_n}d")
+endforeach ()
+find_library (RocksDB_LIBRARY_DEBUG NAMES ${RocksDB_NAMES_DBG}
+    PATHS
+    ${RocksDB_DIR}
+    ${RocksDB_DIR}/bin/Debug
+    ${RocksDB_DIR}/bin64_vs2013/Debug
+    PATH_SUFFIXES lib lib64)
+
+include (FindPackageHandleStandardArgs)
+find_package_handle_standard_args (RocksDB
+    REQUIRED_VARS RocksDB_LIBRARY RocksDB_INCLUDE_DIR
+    VERSION_VAR RocksDB_VERSION)
+
+mark_as_advanced (RocksDB_INCLUDE_DIR RocksDB_LIBRARY)
+set (RocksDB_INCLUDE_DIRS ${RocksDB_INCLUDE_DIR})
+set (RocksDB_LIBRARIES ${RocksDB_LIBRARY})

Builds/CMake/RippleConfig.cmake

@@ -12,16 +12,16 @@ if (static OR MSVC)
 else ()
   set (Boost_USE_STATIC_RUNTIME OFF)
 endif ()
-find_dependency (Boost 1.67
+find_dependency (Boost 1.70
   COMPONENTS
     chrono
+    container
     context
     coroutine
     date_time
     filesystem
     program_options
     regex
-    serialization
     system
     thread)
 #[=========================================================[
@@ -45,8 +45,9 @@ if (static OR APPLE OR MSVC)
   set (OPENSSL_USE_STATIC_LIBS ON)
 endif ()
 set (OPENSSL_MSVC_STATIC_RT ON)
-find_dependency (OpenSSL 1.0.2 REQUIRED)
+find_dependency (OpenSSL 1.1.1 REQUIRED)
 find_dependency (ZLIB)
+find_dependency (date)
 if (TARGET ZLIB::ZLIB)
   set_target_properties(OpenSSL::Crypto PROPERTIES
     INTERFACE_LINK_LIBRARIES ZLIB::ZLIB)

Builds/CMake/RippledCompiler.cmake

@@ -0,0 +1,179 @@
+#[===================================================================[
+   setup project-wide compiler settings
+#]===================================================================]
+
+#[=========================================================[
+   TODO some/most of these common settings belong in a
+   toolchain file, especially the ABI-impacting ones
+#]=========================================================]
+add_library (common INTERFACE)
+add_library (Ripple::common ALIAS common)
+# add a single global dependency on this interface lib
+link_libraries (Ripple::common)
+set_target_properties (common
+  PROPERTIES INTERFACE_POSITION_INDEPENDENT_CODE ON)
+set(CMAKE_CXX_EXTENSIONS OFF)
+target_compile_features (common INTERFACE cxx_std_17)
+target_compile_definitions (common
+  INTERFACE
+    $<$<CONFIG:Debug>:DEBUG _DEBUG>
+    $<$<AND:$<BOOL:${profile}>,$<NOT:$<BOOL:${assert}>>>:NDEBUG>)
+    # ^^^^ NOTE: CMAKE release builds already have NDEBUG
+    # defined, so no need to add it explicitly except for
+    # this special case of (profile ON) and (assert OFF)
+    # -- presumably this is because we don't want profile
+    # builds asserting unless asserts were specifically
+    # requested
+
+if (MSVC)
+  # remove existing exception flag since we set it to -EHa
+  string (REGEX REPLACE "[-/]EH[a-z]+" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
+
+  foreach (var_
+      CMAKE_C_FLAGS_DEBUG CMAKE_C_FLAGS_RELEASE
+      CMAKE_CXX_FLAGS_DEBUG CMAKE_CXX_FLAGS_RELEASE)
+
+    # also remove dynamic runtime
+    string (REGEX REPLACE "[-/]MD[d]*" " " ${var_} "${${var_}}")
+
+    # /ZI (Edit & Continue debugging information) is incompatible with Gy-
+    string (REPLACE "/ZI" "/Zi" ${var_} "${${var_}}")
+
+    # omit debug info completely under CI (not needed)
+    if (is_ci)
+      string (REPLACE "/Zi" " " ${var_} "${${var_}}")
+    endif ()
+  endforeach ()
+
+  target_compile_options (common
+    INTERFACE
+      -bigobj            # Increase object file max size
+      -fp:precise        # Floating point behavior
+      -Gd                # __cdecl calling convention
+      -Gm-               # Minimal rebuild: disabled
+      -Gy-               # Function level linking: disabled
+      -MP                # Multiprocessor compilation
+      -openmp-           # pragma omp: disabled
+      -errorReport:none  # No error reporting to Internet
+      -nologo            # Suppress login banner
+      -wd4018            # Disable signed/unsigned comparison warnings
+      -wd4244            # Disable float to int possible loss of data warnings
+      -wd4267            # Disable size_t to T possible loss of data warnings
+      -wd4800            # Disable C4800(int to bool performance)
+      -wd4503            # Decorated name length exceeded, name was truncated
+      $<$<COMPILE_LANGUAGE:CXX>:
+        -EHa
+        -GR
+      >
+      $<$<CONFIG:Release>:-Ox>
+      $<$<AND:$<COMPILE_LANGUAGE:CXX>,$<CONFIG:Debug>>:
+        -GS
+        -Zc:forScope
+      >
+      # static runtime
+      $<$<CONFIG:Debug>:-MTd>
+      $<$<NOT:$<CONFIG:Debug>>:-MT>
+      $<$<BOOL:${werr}>:-WX>
+      )
+  target_compile_definitions (common
+    INTERFACE
+      _WIN32_WINNT=0x6000
+      _SCL_SECURE_NO_WARNINGS
+      _CRT_SECURE_NO_WARNINGS
+      WIN32_CONSOLE
+      WIN32_LEAN_AND_MEAN
+      NOMINMAX
+      # TODO: Resolve these warnings, don't just silence them
+      _SILENCE_ALL_CXX17_DEPRECATION_WARNINGS
+      $<$<AND:$<COMPILE_LANGUAGE:CXX>,$<CONFIG:Debug>>:_CRTDBG_MAP_ALLOC>)
+  target_link_libraries (common
+    INTERFACE
+      -errorreport:none
+      -machine:X64)
+else ()
+  # HACK : because these need to come first, before any warning demotion
+  string (APPEND CMAKE_CXX_FLAGS " -Wall -Wdeprecated")
+  if (wextra)
+    string (APPEND CMAKE_CXX_FLAGS " -Wextra -Wno-unused-parameter")
+  endif ()
+  # not MSVC
+  target_compile_options (common
+    INTERFACE
+      $<$<BOOL:${werr}>:-Werror>
+      $<$<COMPILE_LANGUAGE:CXX>:
+        -frtti
+        -Wnon-virtual-dtor
+      >
+      -Wno-sign-compare
+      -Wno-char-subscripts
+      -Wno-format
+      -Wno-unused-local-typedefs
+      $<$<BOOL:${is_gcc}>:
+        -Wno-unused-but-set-variable
+        -Wno-deprecated
+      >
+      $<$<NOT:$<CONFIG:Debug>>:-fno-strict-aliasing>
+      # tweak gcc optimization for debug
+      $<$<AND:$<BOOL:${is_gcc}>,$<CONFIG:Debug>>:-O0>
+      # Add debug symbols to release config
+      $<$<CONFIG:Release>:-g>)
+  target_link_libraries (common
+    INTERFACE
+      -rdynamic
+      # link to static libc/c++ iff:
+      #   * static option set and
+      #   * NOT APPLE (AppleClang does not support static libc/c++) and
+      #   * NOT san (sanitizers typically don't work with static libc/c++)
+      $<$<AND:$<BOOL:${static}>,$<NOT:$<BOOL:${APPLE}>>,$<NOT:$<BOOL:${san}>>>:-static-libstdc++>)
+endif ()
+
+if (use_gold AND is_gcc)
+  # use gold linker if available
+  execute_process (
+    COMMAND ${CMAKE_CXX_COMPILER} -fuse-ld=gold -Wl,--version
+    ERROR_QUIET OUTPUT_VARIABLE LD_VERSION)
+    #[=========================================================[
+       NOTE: THE gold linker inserts -rpath as DT_RUNPATH by
+       default intead of DT_RPATH, so you might have slightly
+       unexpected runtime ld behavior if you were expecting
+       DT_RPATH.  Specify --disable-new-dtags to gold if you do
+       not want the default DT_RUNPATH behavior. This rpath
+       treatment as well as static/dynamic selection means that
+       gold does not currently have ideal default behavior when
+       we are using jemalloc. Thus for simplicity we don't use
+       it when jemalloc is requested. An alternative to
+       disabling would be to figure out all the settings
+       required to make gold play nicely with jemalloc.
+    #]=========================================================]
+  if (("${LD_VERSION}" MATCHES "GNU gold") AND (NOT jemalloc))
+    target_link_libraries (common
+      INTERFACE
+        -fuse-ld=gold
+        -Wl,--no-as-needed
+        #[=========================================================[
+           see https://bugs.launchpad.net/ubuntu/+source/eglibc/+bug/1253638/comments/5
+           DT_RUNPATH does not work great for transitive
+           dependencies (of which boost has a few) - so just
+           switch to DT_RPATH if doing dynamic linking with gold
+        #]=========================================================]
+        $<$<NOT:$<BOOL:${static}>>:-Wl,--disable-new-dtags>)
+  endif ()
+  unset (LD_VERSION)
+endif ()
+
+if (use_lld)
+  # use lld linker if available
+  execute_process (
+    COMMAND ${CMAKE_CXX_COMPILER} -fuse-ld=lld -Wl,--version
+    ERROR_QUIET OUTPUT_VARIABLE LD_VERSION)
+  if ("${LD_VERSION}" MATCHES "LLD")
+    target_link_libraries (common INTERFACE -fuse-ld=lld)
+  endif ()
+  unset (LD_VERSION)
+endif()
+
+if (assert)
+  foreach (var_ CMAKE_C_FLAGS_RELEASE CMAKE_CXX_FLAGS_RELEASE)
+    STRING (REGEX REPLACE "[-/]DNDEBUG" "" ${var_} "${${var_}}")
+  endforeach ()
+endif ()

Builds/CMake/RippledCov.cmake

@@ -0,0 +1,98 @@
+#[===================================================================[
+   coverage report target
+#]===================================================================]
+
+if (coverage)
+  if (is_clang)
+    if (APPLE)
+      execute_process (COMMAND xcrun -f llvm-profdata
+        OUTPUT_VARIABLE LLVM_PROFDATA
+        OUTPUT_STRIP_TRAILING_WHITESPACE)
+    else ()
+      find_program (LLVM_PROFDATA llvm-profdata)
+    endif ()
+    if (NOT LLVM_PROFDATA)
+      message (WARNING "unable to find llvm-profdata - skipping coverage_report target")
+    endif ()
+
+    if (APPLE)
+      execute_process (COMMAND xcrun -f llvm-cov
+        OUTPUT_VARIABLE LLVM_COV
+        OUTPUT_STRIP_TRAILING_WHITESPACE)
+    else ()
+      find_program (LLVM_COV llvm-cov)
+    endif ()
+    if (NOT LLVM_COV)
+      message (WARNING "unable to find llvm-cov - skipping coverage_report target")
+    endif ()
+
+    set (extract_pattern "")
+    if (coverage_core_only)
+      set (extract_pattern "${CMAKE_CURRENT_SOURCE_DIR}/src/ripple/")
+    endif ()
+
+    if (LLVM_COV AND LLVM_PROFDATA)
+      add_custom_target (coverage_report
+        USES_TERMINAL
+        COMMAND ${CMAKE_COMMAND} -E echo "Generating coverage - results will be in ${CMAKE_BINARY_DIR}/coverage/index.html."
+        COMMAND ${CMAKE_COMMAND} -E echo "Running rippled tests."
+        COMMAND rippled --unittest$<$<BOOL:${coverage_test}>:=${coverage_test}> --quiet --unittest-log
+        COMMAND ${LLVM_PROFDATA}
+          merge -sparse default.profraw -o rip.profdata
+        COMMAND ${CMAKE_COMMAND} -E echo "Summary of coverage:"
+        COMMAND ${LLVM_COV}
+          report -instr-profile=rip.profdata
+          $<TARGET_FILE:rippled> ${extract_pattern}
+        # generate html report
+        COMMAND ${LLVM_COV}
+          show -format=html -output-dir=${CMAKE_BINARY_DIR}/coverage
+          -instr-profile=rip.profdata
+          $<TARGET_FILE:rippled> ${extract_pattern}
+        BYPRODUCTS coverage/index.html)
+    endif ()
+  elseif (is_gcc)
+    find_program (LCOV lcov)
+    if (NOT LCOV)
+      message (WARNING "unable to find lcov - skipping coverage_report target")
+    endif ()
+
+    find_program (GENHTML genhtml)
+    if (NOT GENHTML)
+      message (WARNING "unable to find genhtml - skipping coverage_report target")
+    endif ()
+
+    set (extract_pattern "*")
+    if (coverage_core_only)
+      set (extract_pattern "*/src/ripple/*")
+    endif ()
+
+    if (LCOV AND GENHTML)
+      add_custom_target (coverage_report
+        USES_TERMINAL
+        COMMAND ${CMAKE_COMMAND} -E echo "Generating coverage- results will be in ${CMAKE_BINARY_DIR}/coverage/index.html."
+        # create baseline info file
+        COMMAND ${LCOV}
+          --no-external -d "${CMAKE_CURRENT_SOURCE_DIR}" -c -d . -i -o baseline.info
+          | grep -v "ignoring data for external file"
+        # run tests
+        COMMAND ${CMAKE_COMMAND} -E echo "Running rippled tests for coverage report."
+        COMMAND rippled --unittest$<$<BOOL:${coverage_test}>:=${coverage_test}> --quiet --unittest-log
+        # Create test coverage data file
+        COMMAND ${LCOV}
+          --no-external -d "${CMAKE_CURRENT_SOURCE_DIR}" -c -d . -o tests.info
+          | grep -v "ignoring data for external file"
+        # Combine baseline and test coverage data
+        COMMAND ${LCOV}
+          -a baseline.info -a tests.info -o lcov-all.info
+        # extract our files
+        COMMAND ${LCOV}
+          -e lcov-all.info "${extract_pattern}" -o lcov.info
+        COMMAND ${CMAKE_COMMAND} -E echo "Summary of coverage:"
+        COMMAND ${LCOV} --summary lcov.info
+        # generate HTML report
+        COMMAND ${GENHTML}
+          -o ${CMAKE_BINARY_DIR}/coverage lcov.info
+        BYPRODUCTS coverage/index.html)
+    endif ()
+  endif ()
+endif ()

Builds/CMake/RippledDocs.cmake

@@ -0,0 +1,79 @@
+#[===================================================================[
+   docs target (optional)
+#]===================================================================]
+if (tests)
+  find_package (Doxygen)
+  if (NOT TARGET Doxygen::doxygen)
+    message (STATUS "doxygen executable not found -- skipping docs target")
+    return ()
+  endif ()
+  
+  set (doxygen_output_directory "${CMAKE_BINARY_DIR}/docs")
+  set (doxygen_include_path "${CMAKE_CURRENT_SOURCE_DIR}/src")
+  set (doxygen_index_file "${doxygen_output_directory}/html/index.html")
+  set (doxyfile "${CMAKE_CURRENT_SOURCE_DIR}/docs/Doxyfile")
+  
+  file (GLOB_RECURSE doxygen_input
+    docs/*.md
+    src/ripple/*.h
+    src/ripple/*.cpp
+    src/ripple/*.md
+    src/test/*.h
+    src/test/*.md
+    Builds/*/README.md)
+  list (APPEND doxygen_input
+    README.md
+    RELEASENOTES.md
+    src/README.md)
+  set (dependencies "${doxygen_input}" "${doxyfile}")
+  
+  function (verbose_find_path variable name)
+    # find_path sets a CACHE variable, so don't try using a "local" variable.
+    find_path (${variable} "${name}" ${ARGN})
+    if (NOT ${variable})
+      message (WARNING "could not find ${name}")
+    else ()
+      message (STATUS "found ${name}: ${${variable}}/${name}")
+    endif ()
+  endfunction ()
+  
+  verbose_find_path (doxygen_plantuml_jar_path plantuml.jar PATH_SUFFIXES share/plantuml)
+  verbose_find_path (doxygen_dot_path dot)
+  
+  # https://en.cppreference.com/w/Cppreference:Archives
+  # https://stackoverflow.com/questions/60822559/how-to-move-a-file-download-from-configure-step-to-build-step
+  set (download_script "${CMAKE_BINARY_DIR}/docs/download-cppreference.cmake")
+  file (WRITE
+    "${download_script}"
+    "file (DOWNLOAD \
+      http://upload.cppreference.com/mwiki/images/b/b2/html_book_20190607.zip \
+      ${CMAKE_BINARY_DIR}/docs/cppreference.zip \
+      EXPECTED_HASH MD5=82b3a612d7d35a83e3cb1195a63689ab \
+    )\n \
+    execute_process ( \
+      COMMAND \"${CMAKE_COMMAND}\" -E tar -xf cppreference.zip \
+    )\n"
+  )
+  set (tagfile "${CMAKE_BINARY_DIR}/docs/cppreference-doxygen-web.tag.xml")
+  add_custom_command (
+    OUTPUT "${tagfile}"
+    COMMAND "${CMAKE_COMMAND}" -P "${download_script}"
+    WORKING_DIRECTORY "${CMAKE_BINARY_DIR}/docs"
+  )
+  set (doxygen_tagfiles "${tagfile}=http://en.cppreference.com/w/")
+  
+  add_custom_command (
+    OUTPUT "${doxygen_index_file}"
+    COMMAND "${CMAKE_COMMAND}" -E env
+      "DOXYGEN_OUTPUT_DIRECTORY=${doxygen_output_directory}"
+      "DOXYGEN_INCLUDE_PATH=${doxygen_include_path}"
+      "DOXYGEN_TAGFILES=${doxygen_tagfiles}"
+      "DOXYGEN_PLANTUML_JAR_PATH=${doxygen_plantuml_jar_path}"
+      "DOXYGEN_DOT_PATH=${doxygen_dot_path}"
+      "${DOXYGEN_EXECUTABLE}" "${doxyfile}"
+    WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}"
+    DEPENDS "${dependencies}" "${tagfile}")
+  add_custom_target (docs
+    DEPENDS "${doxygen_index_file}"
+    SOURCES "${dependencies}")
+endif ()

Builds/CMake/RippledInstall.cmake

@@ -0,0 +1,62 @@
+#[===================================================================[
+   install stuff
+#]===================================================================]
+
+install (
+  TARGETS
+    ed25519-donna
+    common
+    opts
+    ripple_syslibs
+    ripple_boost
+    xrpl_core
+  EXPORT RippleExports
+  LIBRARY DESTINATION lib
+  ARCHIVE DESTINATION lib
+  RUNTIME DESTINATION bin
+  INCLUDES DESTINATION include)
+
+if(${INSTALL_SECP256K1})
+install (
+  TARGETS
+    secp256k1
+  EXPORT RippleExports
+  LIBRARY DESTINATION lib
+  ARCHIVE DESTINATION lib
+  RUNTIME DESTINATION bin
+  INCLUDES DESTINATION include)
+endif()
+
+install (EXPORT RippleExports
+  FILE RippleTargets.cmake
+  NAMESPACE Ripple::
+  DESTINATION lib/cmake/ripple)
+include (CMakePackageConfigHelpers)
+write_basic_package_version_file (
+  RippleConfigVersion.cmake
+  VERSION ${rippled_version}
+  COMPATIBILITY SameMajorVersion)
+
+if (is_root_project)
+  install (TARGETS rippled RUNTIME DESTINATION bin)
+  set_target_properties(rippled PROPERTIES INSTALL_RPATH_USE_LINK_PATH ON)
+  install (
+    FILES
+      ${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake/RippleConfig.cmake
+      ${CMAKE_CURRENT_BINARY_DIR}/RippleConfigVersion.cmake
+    DESTINATION lib/cmake/ripple)
+  # sample configs should not overwrite existing files
+  # install if-not-exists workaround as suggested by
+  # https://cmake.org/Bug/view.php?id=12646
+  install(CODE "
+    macro (copy_if_not_exists SRC DEST NEWNAME)
+      if (NOT EXISTS \"\$ENV{DESTDIR}\${CMAKE_INSTALL_PREFIX}/\${DEST}/\${NEWNAME}\")
+        file (INSTALL FILE_PERMISSIONS OWNER_READ OWNER_WRITE DESTINATION \"\${CMAKE_INSTALL_PREFIX}/\${DEST}\" FILES \"\${SRC}\" RENAME \"\${NEWNAME}\")
+      else ()
+        message (\"-- Skipping : \$ENV{DESTDIR}\${CMAKE_INSTALL_PREFIX}/\${DEST}/\${NEWNAME}\")
+      endif ()
+    endmacro()
+    copy_if_not_exists(\"${CMAKE_CURRENT_SOURCE_DIR}/cfg/rippled-example.cfg\" etc rippled.cfg)
+    copy_if_not_exists(\"${CMAKE_CURRENT_SOURCE_DIR}/cfg/validators-example.txt\" etc validators.txt)
+  ")
+endif ()

Builds/CMake/RippledInterface.cmake

@@ -0,0 +1,103 @@
+#[===================================================================[
+   rippled compile options/settings via an interface library
+#]===================================================================]
+
+add_library (opts INTERFACE)
+add_library (Ripple::opts ALIAS opts)
+target_compile_definitions (opts
+  INTERFACE
+    BOOST_ASIO_DISABLE_HANDLER_TYPE_REQUIREMENTS
+    $<$<BOOL:${boost_show_deprecated}>:
+      BOOST_ASIO_NO_DEPRECATED
+      BOOST_FILESYSTEM_NO_DEPRECATED
+    >
+    $<$<NOT:$<BOOL:${boost_show_deprecated}>>:
+      BOOST_COROUTINES_NO_DEPRECATION_WARNING
+      BOOST_BEAST_ALLOW_DEPRECATED
+      BOOST_FILESYSTEM_DEPRECATED
+    >
+    $<$<BOOL:${beast_no_unit_test_inline}>:BEAST_NO_UNIT_TEST_INLINE=1>
+    $<$<BOOL:${beast_disable_autolink}>:BEAST_DONT_AUTOLINK_TO_WIN32_LIBRARIES=1>
+    $<$<BOOL:${single_io_service_thread}>:RIPPLE_SINGLE_IO_SERVICE_THREAD=1>)
+target_compile_options (opts
+  INTERFACE
+    $<$<AND:$<BOOL:${is_gcc}>,$<COMPILE_LANGUAGE:CXX>>:-Wsuggest-override>
+    $<$<BOOL:${perf}>:-fno-omit-frame-pointer>
+    $<$<AND:$<BOOL:${is_gcc}>,$<BOOL:${coverage}>>:-fprofile-arcs -ftest-coverage>
+    $<$<AND:$<BOOL:${is_clang}>,$<BOOL:${coverage}>>:-fprofile-instr-generate -fcoverage-mapping>
+    $<$<BOOL:${profile}>:-pg>
+    $<$<AND:$<BOOL:${is_gcc}>,$<BOOL:${profile}>>:-p>)
+
+target_link_libraries (opts
+  INTERFACE
+    $<$<AND:$<BOOL:${is_gcc}>,$<BOOL:${coverage}>>:-fprofile-arcs -ftest-coverage>
+    $<$<AND:$<BOOL:${is_clang}>,$<BOOL:${coverage}>>:-fprofile-instr-generate -fcoverage-mapping>
+    $<$<BOOL:${profile}>:-pg>
+    $<$<AND:$<BOOL:${is_gcc}>,$<BOOL:${profile}>>:-p>)
+
+if (jemalloc)
+  if (static)
+    set(JEMALLOC_USE_STATIC ON CACHE BOOL "" FORCE)
+  endif ()
+  find_package (jemalloc REQUIRED)
+  target_compile_definitions (opts INTERFACE PROFILE_JEMALLOC)
+  target_include_directories (opts SYSTEM INTERFACE ${JEMALLOC_INCLUDE_DIRS})
+  target_link_libraries (opts INTERFACE ${JEMALLOC_LIBRARIES})
+  get_filename_component (JEMALLOC_LIB_PATH ${JEMALLOC_LIBRARIES} DIRECTORY)
+  ## TODO see if we can use the BUILD_RPATH target property (is it transitive?)
+  set (CMAKE_BUILD_RPATH ${CMAKE_BUILD_RPATH} ${JEMALLOC_LIB_PATH})
+endif ()
+
+if (san)
+  target_compile_options (opts
+    INTERFACE
+      # sanitizers recommend minimum of -O1 for reasonable performance
+      $<$<CONFIG:Debug>:-O1>
+      ${SAN_FLAG}
+      -fno-omit-frame-pointer)
+  target_compile_definitions (opts
+    INTERFACE
+      $<$<STREQUAL:${san},address>:SANITIZER=ASAN>
+      $<$<STREQUAL:${san},thread>:SANITIZER=TSAN>
+      $<$<STREQUAL:${san},memory>:SANITIZER=MSAN>
+      $<$<STREQUAL:${san},undefined>:SANITIZER=UBSAN>)
+  target_link_libraries (opts INTERFACE ${SAN_FLAG} ${SAN_LIB})
+endif ()
+
+#[===================================================================[
+   rippled transitive library deps via an interface library
+#]===================================================================]
+
+add_library (ripple_syslibs INTERFACE)
+add_library (Ripple::syslibs ALIAS ripple_syslibs)
+target_link_libraries (ripple_syslibs
+  INTERFACE
+    $<$<BOOL:${MSVC}>:
+      legacy_stdio_definitions.lib
+      Shlwapi
+      kernel32
+      user32
+      gdi32
+      winspool
+      comdlg32
+      advapi32
+      shell32
+      ole32
+      oleaut32
+      uuid
+      odbc32
+      odbccp32
+      crypt32
+    >
+    $<$<NOT:$<BOOL:${MSVC}>>:dl>
+    $<$<NOT:$<OR:$<BOOL:${MSVC}>,$<BOOL:${APPLE}>>>:rt>)
+
+if (NOT MSVC)
+  set (THREADS_PREFER_PTHREAD_FLAG ON)
+  find_package (Threads)
+  target_link_libraries (ripple_syslibs INTERFACE Threads::Threads)
+endif ()
+
+add_library (ripple_libs INTERFACE)
+add_library (Ripple::libs ALIAS ripple_libs)
+target_link_libraries (ripple_libs INTERFACE Ripple::syslibs)

Builds/CMake/RippledMultiConfig.cmake

@@ -0,0 +1,39 @@
+#[===================================================================[
+   multiconfig misc
+#]===================================================================]
+
+if (is_multiconfig)
+  # This code finds all source files in the src subdirectory for inclusion
+  # in the IDE file tree as non-compiled sources. Since this file list will
+  # have some overlap with files we have already added to our targets to
+  # be compiled, we explicitly remove any of these target source files from
+  # this list.
+  file (GLOB_RECURSE all_sources RELATIVE ${CMAKE_CURRENT_SOURCE_DIR}
+    CONFIGURE_DEPENDS
+    src/*.* Builds/*.md docs/*.md src/*.md Builds/*.cmake)
+  file(GLOB md_files RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} CONFIGURE_DEPENDS
+    *.md)
+  LIST(APPEND all_sources ${md_files})
+  foreach (_target secp256k1 ed25519-donna pbufs xrpl_core rippled)
+    get_target_property (_type ${_target} TYPE)
+    if(_type STREQUAL "INTERFACE_LIBRARY")
+      continue()
+    endif()
+    get_target_property (_src ${_target} SOURCES)
+    list (REMOVE_ITEM all_sources ${_src})
+  endforeach ()
+  target_sources (rippled PRIVATE ${all_sources})
+  set_property (
+    SOURCE ${all_sources}
+    APPEND
+    PROPERTY HEADER_FILE_ONLY true)
+  if (MSVC)
+    set_property(
+      DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
+      PROPERTY VS_STARTUP_PROJECT rippled)
+  endif ()
+
+  group_sources(src)
+  group_sources(docs)
+  group_sources(Builds)
+endif ()

Builds/CMake/RippledNIH.cmake

@@ -0,0 +1,33 @@
+#[===================================================================[
+   NIH prefix path..this is where we will download
+   and build any ExternalProjects, and they will hopefully
+   survive across build directory deletion (manual cleans)
+#]===================================================================]
+
+string (REGEX REPLACE "[ \\/%]+" "_" gen_for_path ${CMAKE_GENERATOR})
+string (TOLOWER ${gen_for_path} gen_for_path)
+# HACK: trying to shorten paths for windows CI (which hits 260 MAXPATH easily)
+# @see:  https://issues.jenkins-ci.org/browse/JENKINS-38706?focusedCommentId=339847
+string (REPLACE "visual_studio" "vs" gen_for_path ${gen_for_path})
+if (NOT DEFINED NIH_CACHE_ROOT)
+  if (DEFINED ENV{NIH_CACHE_ROOT})
+    set (NIH_CACHE_ROOT $ENV{NIH_CACHE_ROOT})
+  else ()
+    set (NIH_CACHE_ROOT "${CMAKE_CURRENT_SOURCE_DIR}/.nih_c")
+  endif ()
+endif ()
+set (nih_cache_path
+  "${NIH_CACHE_ROOT}/${gen_for_path}/${CMAKE_CXX_COMPILER_ID}_${CMAKE_CXX_COMPILER_VERSION}")
+if (NOT is_multiconfig)
+  set (nih_cache_path "${nih_cache_path}/${CMAKE_BUILD_TYPE}")
+endif ()
+file(TO_CMAKE_PATH "${nih_cache_path}" nih_cache_path)
+message (STATUS "NIH-EP cache path: ${nih_cache_path}")
+## two convenience variables:
+set (ep_lib_prefix ${CMAKE_STATIC_LIBRARY_PREFIX})
+set (ep_lib_suffix ${CMAKE_STATIC_LIBRARY_SUFFIX})
+
+# this is a setting for FetchContent and needs to be
+# a cache variable
+# https://cmake.org/cmake/help/latest/module/FetchContent.html#populating-the-content
+set (FETCHCONTENT_BASE_DIR ${nih_cache_path} CACHE STRING "" FORCE)

Builds/CMake/RippledRelease.cmake

@@ -0,0 +1,190 @@
+#[===================================================================[
+   package/container targets - (optional)
+#]===================================================================]
+
+if (is_root_project)
+  if (NOT DOCKER)
+    find_program (DOCKER docker)
+  endif ()
+
+  if (DOCKER)
+    # if no container label is provided, use current git hash
+    git_hash (commit_hash)
+    if (NOT container_label)
+      set (container_label ${commit_hash})
+    endif ()
+    message (STATUS "using [${container_label}] as build container tag...")
+
+    file (MAKE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/packages)
+    file (MAKE_DIRECTORY ${NIH_CACHE_ROOT}/pkgbuild)
+    if (is_linux)
+      execute_process (COMMAND id -u
+        OUTPUT_VARIABLE DOCKER_USER_ID
+        OUTPUT_STRIP_TRAILING_WHITESPACE)
+      message (STATUS "docker local user id: ${DOCKER_USER_ID}")
+      execute_process (COMMAND id -g
+        OUTPUT_VARIABLE DOCKER_GROUP_ID
+        OUTPUT_STRIP_TRAILING_WHITESPACE)
+      message (STATUS "docker local group id: ${DOCKER_GROUP_ID}")
+    endif ()
+    if (DOCKER_USER_ID AND DOCKER_GROUP_ID)
+      set(map_user TRUE)
+    endif ()
+    #[===================================================================[
+        rpm
+    #]===================================================================]
+    add_custom_target (rpm_container
+      docker build
+        --pull
+        --build-arg GIT_COMMIT=${commit_hash}
+        -t rippled-rpm-builder:${container_label}
+        $<$<BOOL:${rpm_cache_from}>:--cache-from=${rpm_cache_from}>
+        -f centos-builder/Dockerfile .
+      WORKING_DIRECTORY  ${CMAKE_CURRENT_SOURCE_DIR}/Builds/containers
+      VERBATIM
+      USES_TERMINAL
+      COMMAND_EXPAND_LISTS
+      SOURCES
+        Builds/containers/centos-builder/Dockerfile
+        Builds/containers/centos-builder/centos_setup.sh
+        Builds/containers/centos-builder/extras.sh
+        Builds/containers/shared/build_deps.sh
+        Builds/containers/shared/rippled.service
+        Builds/containers/shared/update_sources.sh
+        Builds/containers/shared/update-rippled.sh
+        Builds/containers/packaging/rpm/rippled.spec
+        Builds/containers/packaging/rpm/build_rpm.sh
+        bin/getRippledInfo
+    )
+    exclude_from_default (rpm_container)
+    add_custom_target (rpm
+      docker run
+        -e NIH_CACHE_ROOT=/opt/rippled_bld/pkg/.nih_c
+        -v ${NIH_CACHE_ROOT}/pkgbuild:/opt/rippled_bld/pkg/.nih_c
+        -v ${CMAKE_CURRENT_SOURCE_DIR}:/opt/rippled_bld/pkg/rippled
+        -v ${CMAKE_CURRENT_BINARY_DIR}/packages:/opt/rippled_bld/pkg/out
+        "$<$<BOOL:${map_user}>:--volume=/etc/passwd:/etc/passwd;--volume=/etc/group:/etc/group;--user=${DOCKER_USER_ID}:${DOCKER_GROUP_ID}>"
+        -t rippled-rpm-builder:${container_label}
+        /bin/bash -c "cp -fpu rippled/Builds/containers/packaging/rpm/build_rpm.sh . && ./build_rpm.sh"
+      VERBATIM
+      USES_TERMINAL
+      COMMAND_EXPAND_LISTS
+      SOURCES
+        Builds/containers/packaging/rpm/rippled.spec
+    )
+    exclude_from_default (rpm)
+    if (NOT have_package_container)
+      add_dependencies(rpm rpm_container)
+    endif ()
+    #[===================================================================[
+        dpkg
+    #]===================================================================]
+    # currently use ubuntu 16.04 as a base b/c it has one of
+    # the lower versions of libc among ubuntu and debian releases.
+    # we could change this in the future and build with some other deb
+    # based system.
+    add_custom_target (dpkg_container
+      docker build
+        --pull
+        --build-arg DIST_TAG=16.04
+        --build-arg GIT_COMMIT=${commit_hash}
+        -t rippled-dpkg-builder:${container_label}
+        $<$<BOOL:${dpkg_cache_from}>:--cache-from=${dpkg_cache_from}>
+        -f ubuntu-builder/Dockerfile .
+      WORKING_DIRECTORY  ${CMAKE_CURRENT_SOURCE_DIR}/Builds/containers
+      VERBATIM
+      USES_TERMINAL
+      COMMAND_EXPAND_LISTS
+      SOURCES
+        Builds/containers/ubuntu-builder/Dockerfile
+        Builds/containers/ubuntu-builder/ubuntu_setup.sh
+        Builds/containers/shared/build_deps.sh
+        Builds/containers/shared/rippled.service
+        Builds/containers/shared/update_sources.sh
+        Builds/containers/shared/update-rippled.sh
+        Builds/containers/packaging/dpkg/build_dpkg.sh
+        Builds/containers/packaging/dpkg/debian/README.Debian
+        Builds/containers/packaging/dpkg/debian/conffiles
+        Builds/containers/packaging/dpkg/debian/control
+        Builds/containers/packaging/dpkg/debian/copyright
+        Builds/containers/packaging/dpkg/debian/dirs
+        Builds/containers/packaging/dpkg/debian/docs
+        Builds/containers/packaging/dpkg/debian/rippled-dev.install
+        Builds/containers/packaging/dpkg/debian/rippled.install
+        Builds/containers/packaging/dpkg/debian/rippled.links
+        Builds/containers/packaging/dpkg/debian/rippled.postinst
+        Builds/containers/packaging/dpkg/debian/rippled.postrm
+        Builds/containers/packaging/dpkg/debian/rippled.preinst
+        Builds/containers/packaging/dpkg/debian/rippled.prerm
+        Builds/containers/packaging/dpkg/debian/rules
+        bin/getRippledInfo
+    )
+    exclude_from_default (dpkg_container)
+    add_custom_target (dpkg
+      docker run
+        -e NIH_CACHE_ROOT=/opt/rippled_bld/pkg/.nih_c
+        -v ${NIH_CACHE_ROOT}/pkgbuild:/opt/rippled_bld/pkg/.nih_c
+        -v ${CMAKE_CURRENT_SOURCE_DIR}:/opt/rippled_bld/pkg/rippled
+        -v ${CMAKE_CURRENT_BINARY_DIR}/packages:/opt/rippled_bld/pkg/out
+        "$<$<BOOL:${map_user}>:--volume=/etc/passwd:/etc/passwd;--volume=/etc/group:/etc/group;--user=${DOCKER_USER_ID}:${DOCKER_GROUP_ID}>"
+        -t rippled-dpkg-builder:${container_label}
+        /bin/bash -c "cp -fpu rippled/Builds/containers/packaging/dpkg/build_dpkg.sh . && ./build_dpkg.sh"
+      VERBATIM
+      USES_TERMINAL
+      COMMAND_EXPAND_LISTS
+      SOURCES
+        Builds/containers/packaging/dpkg/debian/control
+    )
+    exclude_from_default (dpkg)
+    if (NOT have_package_container)
+      add_dependencies(dpkg dpkg_container)
+    endif ()
+    #[===================================================================[
+        ci container
+    #]===================================================================]
+    # now use the same ubuntu image for our travis-ci docker images,
+    # but we use a newer distro (18.04 vs 16.04).
+    #
+    # the following steps assume the github pkg repo, but it's possible to
+    # adapt these for other docker hub repositories.
+    #
+    # steps for publishing a new CI image when you make changes:
+    #
+    #   mkdir bld.ci && cd bld.ci && cmake -Dpackages_only=ON -Dcontainer_label=CI_LATEST
+    #   cmake --build . --target ci_container --verbose
+    #   docker tag rippled-ci-builder:CI_LATEST <HUB REPO PATH>/rippled-ci-builder:YYYY-MM-DD
+    #      (NOTE: change YYYY-MM-DD to match current date, or use a different
+    #             tag/version scheme if you prefer)
+    #   docker push <HUB REPO PATH>/rippled-ci-builder:YYYY-MM-DD
+    #      (NOTE: <HUB REPO PATH> is probably your user or org name if using
+    #             docker hub, or it might be something like
+    #             docker.pkg.github.com/ripple/rippled if using the github pkg
+    #             registry. for any registry, you will need to be logged-in via
+    #             docker and have push access.)
+    #
+    # ...then change the DOCKER_IMAGE line in .travis.yml :
+    #     - DOCKER_IMAGE="<HUB REPO PATH>/rippled-ci-builder:YYYY-MM-DD"
+    add_custom_target (ci_container
+      docker build
+        --pull
+        --build-arg DIST_TAG=18.04
+        --build-arg GIT_COMMIT=${commit_hash}
+        --build-arg CI_USE=true
+        -t rippled-ci-builder:${container_label}
+        $<$<BOOL:${ci_cache_from}>:--cache-from=${ci_cache_from}>
+        -f ubuntu-builder/Dockerfile .
+      WORKING_DIRECTORY  ${CMAKE_CURRENT_SOURCE_DIR}/Builds/containers
+      VERBATIM
+      USES_TERMINAL
+      COMMAND_EXPAND_LISTS
+      SOURCES
+        Builds/containers/ubuntu-builder/Dockerfile
+        Builds/containers/ubuntu-builder/ubuntu_setup.sh
+        Builds/containers/shared/build_deps.sh
+    )
+    exclude_from_default (ci_container)
+  else ()
+    message (STATUS "docker NOT found -- won't be able to build containers for packaging")
+  endif ()
+endif ()
+

Builds/CMake/RippledSanity.cmake

@@ -0,0 +1,88 @@
+#[===================================================================[
+   convenience variables and sanity checks
+#]===================================================================]
+
+if (NOT ep_procs)
+  ProcessorCount(ep_procs)
+  if (ep_procs GREATER 1)
+    # never use more than half of cores for EP builds
+    math (EXPR ep_procs "${ep_procs} / 2")
+    message (STATUS "Using ${ep_procs} cores for ExternalProject builds.")
+  endif ()
+endif ()
+get_property (is_multiconfig GLOBAL PROPERTY GENERATOR_IS_MULTI_CONFIG)
+if (is_multiconfig STREQUAL "NOTFOUND")
+  if (${CMAKE_GENERATOR} STREQUAL "Xcode" OR ${CMAKE_GENERATOR} MATCHES "^Visual Studio")
+    set (is_multiconfig TRUE)
+  endif ()
+endif ()
+
+set (CMAKE_CONFIGURATION_TYPES "Debug;Release" CACHE STRING "" FORCE)
+if (NOT is_multiconfig)
+  if (NOT CMAKE_BUILD_TYPE)
+    message (STATUS "Build type not specified - defaulting to Release")
+    set (CMAKE_BUILD_TYPE Release CACHE STRING "build type" FORCE)
+  elseif (NOT (CMAKE_BUILD_TYPE STREQUAL Debug OR CMAKE_BUILD_TYPE STREQUAL Release))
+    # for simplicity, these are the only two config types we care about. Limiting
+    # the build types simplifies dealing with external project builds especially
+    message (FATAL_ERROR " *** Only Debug or Release build types are currently supported ***")
+  endif ()
+endif ()
+
+get_directory_property(has_parent PARENT_DIRECTORY)
+if (has_parent)
+  set (is_root_project OFF)
+else ()
+  set (is_root_project ON)
+endif ()
+
+if ("${CMAKE_CXX_COMPILER_ID}" MATCHES ".*Clang") # both Clang and AppleClang
+  set (is_clang TRUE)
+  if ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "Clang" AND
+         CMAKE_CXX_COMPILER_VERSION VERSION_LESS 7.0)
+    message (FATAL_ERROR "This project requires clang 7 or later")
+  endif ()
+  # TODO min AppleClang version check ?
+elseif ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU")
+  set (is_gcc TRUE)
+  if (CMAKE_CXX_COMPILER_VERSION VERSION_LESS 7.0)
+    message (FATAL_ERROR "This project requires GCC 7 or later")
+  endif ()
+endif ()
+if (CMAKE_GENERATOR STREQUAL "Xcode")
+  set (is_xcode TRUE)
+endif ()
+
+if (CMAKE_SYSTEM_NAME STREQUAL "Linux")
+  set (is_linux TRUE)
+else ()
+  set (is_linux FALSE)
+endif ()
+
+if ("$ENV{CI}" STREQUAL "true" OR "$ENV{CONTINUOUS_INTEGRATION}" STREQUAL "true")
+  set (is_ci TRUE)
+else ()
+  set (is_ci FALSE)
+endif ()
+
+# check for in-source build and fail
+if ("${CMAKE_CURRENT_SOURCE_DIR}" STREQUAL "${CMAKE_BINARY_DIR}")
+  message (FATAL_ERROR "Builds (in-source) are not allowed in "
+    "${CMAKE_CURRENT_SOURCE_DIR}. Please remove CMakeCache.txt and the CMakeFiles "
+    "directory from ${CMAKE_CURRENT_SOURCE_DIR} and try building in a separate directory.")
+endif ()
+
+if ("${CMAKE_GENERATOR}" MATCHES "Visual Studio" AND
+    NOT ("${CMAKE_GENERATOR}" MATCHES .*Win64.*))
+  message (FATAL_ERROR
+    "Visual Studio 32-bit build is not supported. Use -G\"${CMAKE_GENERATOR} Win64\"")
+endif ()
+
+if (NOT CMAKE_SIZEOF_VOID_P EQUAL 8)
+  message (FATAL_ERROR "Rippled requires a 64 bit target architecture.\n"
+    "The most likely cause of this warning is trying to build rippled with a 32-bit OS.")
+endif ()
+
+if (APPLE AND NOT HOMEBREW)
+  find_program (HOMEBREW brew)
+endif ()

Builds/CMake/RippledSettings.cmake

@@ -0,0 +1,127 @@
+#[===================================================================[
+   declare user options/settings
+#]===================================================================]
+
+option (assert "Enables asserts, even in release builds" OFF)
+
+option (reporting "Build rippled with reporting mode enabled" OFF)
+
+option (tests "Build tests" ON)
+
+option (unity "Creates a build using UNITY support in cmake. This is the default" ON)
+if (unity)
+  if (CMAKE_VERSION VERSION_LESS 3.16)
+    message (WARNING "unity option only supported for with cmake 3.16+ (please upgrade)")
+    set (unity OFF CACHE BOOL "unity only available for cmake 3.16+" FORCE)
+  else ()
+    if (NOT is_ci)
+      set (CMAKE_UNITY_BUILD_BATCH_SIZE 15 CACHE STRING "")
+    endif ()
+  endif ()
+endif ()
+if (is_gcc OR is_clang)
+  option (coverage "Generates coverage info." OFF)
+  option (profile "Add profiling flags" OFF)
+  set (coverage_test "" CACHE STRING
+    "On gcc & clang, the specific unit test(s) to run for coverage. Default is all tests.")
+  if (coverage_test AND NOT coverage)
+    set (coverage ON CACHE BOOL "gcc/clang only" FORCE)
+  endif ()
+  option (coverage_core_only
+    "Include only src/ripple files when generating coverage report. \
+    Set to OFF to include all sources in coverage report."
+    ON)
+  option (wextra "compile with extra gcc/clang warnings enabled" ON)
+else ()
+  set (profile OFF CACHE BOOL "gcc/clang only" FORCE)
+  set (coverage OFF CACHE BOOL "gcc/clang only" FORCE)
+  set (wextra OFF CACHE BOOL "gcc/clang only" FORCE)
+endif ()
+if (is_linux)
+  option (BUILD_SHARED_LIBS "build shared ripple libraries" OFF)
+  option (static "link protobuf, openssl, libc++, and boost statically" ON)
+  option (perf "Enables flags that assist with perf recording" OFF)
+  option (use_gold "enables detection of gold (binutils) linker" ON)
+else ()
+  # we are not ready to allow shared-libs on windows because it would require
+  # export declarations. On macos it's more feasible, but static openssl
+  # produces odd linker errors, thus we disable shared lib builds for now.
+  set (BUILD_SHARED_LIBS OFF CACHE BOOL "build shared ripple libraries - OFF for win/macos" FORCE)
+  set (static ON CACHE BOOL "static link, linux only. ON for WIN/macos" FORCE)
+  set (perf OFF CACHE BOOL "perf flags, linux only" FORCE)
+  set (use_gold OFF CACHE BOOL "gold linker, linux only" FORCE)
+endif ()
+if (is_clang)
+  option (use_lld "enables detection of lld linker" ON)
+else ()
+  set (use_lld OFF CACHE BOOL "try lld linker, clang only" FORCE)
+endif ()
+option (jemalloc "Enables jemalloc for heap profiling" OFF)
+option (werr "treat warnings as errors" OFF)
+option (local_protobuf
+  "Force a local build of protobuf instead of looking for an installed version." OFF)
+option (local_grpc
+  "Force a local build of gRPC instead of looking for an installed version." OFF)
+
+# this one is a string and therefore can't be an option
+set (san "" CACHE STRING "On gcc & clang, add sanitizer instrumentation")
+set_property (CACHE san PROPERTY STRINGS ";undefined;memory;address;thread")
+if (san)
+  string (TOLOWER ${san} san)
+  set (SAN_FLAG "-fsanitize=${san}")
+  set (SAN_LIB "")
+  if (is_gcc)
+    if (san STREQUAL "address")
+      set (SAN_LIB "asan")
+    elseif (san STREQUAL "thread")
+      set (SAN_LIB "tsan")
+    elseif (san STREQUAL "memory")
+      set (SAN_LIB "msan")
+    elseif (san STREQUAL "undefined")
+      set (SAN_LIB "ubsan")
+    endif ()
+  endif ()
+  set (_saved_CRL ${CMAKE_REQUIRED_LIBRARIES})
+  set (CMAKE_REQUIRED_LIBRARIES "${SAN_FLAG};${SAN_LIB}")
+  check_cxx_compiler_flag (${SAN_FLAG} COMPILER_SUPPORTS_SAN)
+  set (CMAKE_REQUIRED_LIBRARIES ${_saved_CRL})
+  if (NOT COMPILER_SUPPORTS_SAN)
+    message (FATAL_ERROR "${san} sanitizer does not seem to be supported by your compiler")
+  endif ()
+endif ()
+set (container_label "" CACHE STRING "tag to use for package building containers")
+option (packages_only
+  "ONLY generate package building targets. This is special use-case and almost \
+   certainly not what you want. Use with caution as you won't be able to build \
+   any compiled targets locally." OFF)
+option (have_package_container
+  "Sometimes you already have the tagged container you want to use for package \
+   building and you don't want docker to rebuild it. This flag will detach the \
+   dependency of the package build from the container build. It's an advanced \
+   use case and most likely you should not be touching this flag." OFF)
+
+# the remaining options are obscure and rarely used
+option (beast_no_unit_test_inline
+  "Prevents unit test definitions from being inserted into global table"
+  OFF)
+option (single_io_service_thread
+  "Restricts the number of threads calling io_service::run to one. \
+  This can be useful when debugging."
+  OFF)
+option (boost_show_deprecated
+  "Allow boost to fail on deprecated usage. Only useful if you're trying\
+  to find deprecated calls."
+  OFF)
+option (beast_hashers
+  "Use local implementations for sha/ripemd hashes (experimental, not recommended)"
+  OFF)
+
+if (WIN32)
+  option (beast_disable_autolink "Disables autolinking of system libraries on WIN32" OFF)
+else ()
+  set (beast_disable_autolink OFF CACHE BOOL "WIN32 only" FORCE)
+endif ()
+if (coverage)
+  message (STATUS "coverage build requested - forcing Debug build")
+  set (CMAKE_BUILD_TYPE Debug CACHE STRING "build type" FORCE)
+endif ()

Builds/CMake/RippledValidatorKeys.cmake

@@ -0,0 +1,24 @@
+option (validator_keys "Enables building of validator-keys-tool as a separate target (imported via FetchContent)" OFF)
+
+if (validator_keys AND CMAKE_VERSION VERSION_GREATER_EQUAL 3.11)
+  git_branch (current_branch)
+  # default to tracking VK develop branch unless we are on master/release
+  if (NOT (current_branch STREQUAL "master" OR current_branch STREQUAL "release"))
+    set (current_branch "develop")
+  endif ()
+  message (STATUS "tracking ValidatorKeys branch: ${current_branch}")
+
+  FetchContent_Declare (
+    validator_keys_src
+    GIT_REPOSITORY https://github.com/ripple/validator-keys-tool.git
+    GIT_TAG        "${current_branch}"
+  )
+  FetchContent_GetProperties (validator_keys_src)
+  if (NOT validator_keys_src_POPULATED)
+    message (STATUS "Pausing to download ValidatorKeys...")
+    FetchContent_Populate (validator_keys_src)
+  endif ()
+  add_subdirectory (${validator_keys_src_SOURCE_DIR} ${CMAKE_BINARY_DIR}/validator-keys)
+endif ()
+
+

Builds/CMake/RippledVersion.cmake

@@ -0,0 +1,15 @@
+#[===================================================================[
+   read version from source
+#]===================================================================]
+
+file (STRINGS src/ripple/protocol/impl/BuildInfo.cpp BUILD_INFO)
+foreach (line_ ${BUILD_INFO})
+  if (line_ MATCHES "versionString[ ]*=[ ]*\"(.+)\"")
+    set (rippled_version ${CMAKE_MATCH_1})
+  endif ()
+endforeach ()
+if (rippled_version)
+  message (STATUS "rippled version: ${rippled_version}")
+else ()
+  message (FATAL_ERROR "unable to determine rippled version")
+endif ()

Builds/CMake/deps/Boost.cmake

@@ -0,0 +1,96 @@
+#[===================================================================[
+   NIH dep: boost
+#]===================================================================]
+
+if((NOT DEFINED BOOST_ROOT) AND(DEFINED ENV{BOOST_ROOT}))
+  set(BOOST_ROOT $ENV{BOOST_ROOT})
+endif()
+file(TO_CMAKE_PATH "${BOOST_ROOT}" BOOST_ROOT)
+if(WIN32 OR CYGWIN)
+  # Workaround for MSVC having two boost versions - x86 and x64 on same PC in stage folders
+  if(DEFINED BOOST_ROOT)
+    if(IS_DIRECTORY ${BOOST_ROOT}/stage64/lib)
+      set(BOOST_LIBRARYDIR ${BOOST_ROOT}/stage64/lib)
+    elseif(IS_DIRECTORY ${BOOST_ROOT}/stage/lib)
+      set(BOOST_LIBRARYDIR ${BOOST_ROOT}/stage/lib)
+    elseif(IS_DIRECTORY ${BOOST_ROOT}/lib)
+      set(BOOST_LIBRARYDIR ${BOOST_ROOT}/lib)
+    else()
+      message(WARNING "Did not find expected boost library dir. "
+        "Defaulting to ${BOOST_ROOT}")
+      set(BOOST_LIBRARYDIR ${BOOST_ROOT})
+    endif()
+  endif()
+endif()
+message(STATUS "BOOST_ROOT: ${BOOST_ROOT}")
+message(STATUS "BOOST_LIBRARYDIR: ${BOOST_LIBRARYDIR}")
+
+# uncomment the following as needed to debug FindBoost issues:
+#set(Boost_DEBUG ON)
+
+#[=========================================================[
+   boost dynamic libraries don't trivially support @rpath
+   linking right now (cmake's default), so just force
+   static linking for macos, or if requested on linux by flag
+#]=========================================================]
+if(static)
+  set(Boost_USE_STATIC_LIBS ON)
+endif()
+set(Boost_USE_MULTITHREADED ON)
+if(static AND NOT APPLE)
+  set(Boost_USE_STATIC_RUNTIME ON)
+else()
+  set(Boost_USE_STATIC_RUNTIME OFF)
+endif()
+# TBD:
+# Boost_USE_DEBUG_RUNTIME:  When ON, uses Boost libraries linked against the
+find_package(Boost 1.70 REQUIRED
+  COMPONENTS
+    chrono
+    container
+    context
+    coroutine
+    date_time
+    filesystem
+    program_options
+    regex
+    system
+    thread)
+
+add_library(ripple_boost INTERFACE)
+add_library(Ripple::boost ALIAS ripple_boost)
+if(is_xcode)
+  target_include_directories(ripple_boost BEFORE INTERFACE ${Boost_INCLUDE_DIRS})
+  target_compile_options(ripple_boost INTERFACE --system-header-prefix="boost/")
+else()
+  target_include_directories(ripple_boost SYSTEM BEFORE INTERFACE ${Boost_INCLUDE_DIRS})
+endif()
+
+target_link_libraries(ripple_boost
+  INTERFACE
+    Boost::boost
+    Boost::chrono
+    Boost::container
+    Boost::coroutine
+    Boost::date_time
+    Boost::filesystem
+    Boost::program_options
+    Boost::regex
+    Boost::system
+    Boost::thread)
+if(Boost_COMPILER)
+  target_link_libraries(ripple_boost INTERFACE Boost::disable_autolinking)
+endif()
+if(san AND is_clang)
+  # TODO: gcc does not support -fsanitize-blacklist...can we do something else
+  # for gcc ?
+  if(NOT Boost_INCLUDE_DIRS AND TARGET Boost::headers)
+    get_target_property(Boost_INCLUDE_DIRS Boost::headers INTERFACE_INCLUDE_DIRECTORIES)
+  endif()
+  message(STATUS "Adding [${Boost_INCLUDE_DIRS}] to sanitizer blacklist")
+  file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/san_bl.txt "src:${Boost_INCLUDE_DIRS}/*")
+  target_compile_options(opts
+    INTERFACE
+      # ignore boost headers for sanitizing
+      -fsanitize-blacklist=${CMAKE_CURRENT_BINARY_DIR}/san_bl.txt)
+endif()

Builds/CMake/deps/Ed25519-donna.cmake

@@ -0,0 +1,28 @@
+#[===================================================================[
+   NIH dep: ed25519-donna
+#]===================================================================]
+
+add_library (ed25519-donna STATIC
+  src/ed25519-donna/ed25519.c)
+target_include_directories (ed25519-donna
+  PUBLIC
+    $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/src>
+    $<INSTALL_INTERFACE:include>
+  PRIVATE
+    ${CMAKE_CURRENT_SOURCE_DIR}/src/ed25519-donna)
+#[=========================================================[
+   NOTE for macos:
+   https://github.com/floodyberry/ed25519-donna/issues/29
+   our source for ed25519-donna-portable.h has been
+   patched to workaround this.
+#]=========================================================]
+target_link_libraries (ed25519-donna PUBLIC OpenSSL::SSL)
+add_library (NIH::ed25519-donna ALIAS ed25519-donna)
+target_link_libraries (ripple_libs INTERFACE NIH::ed25519-donna)
+#[===========================[
+   headers installation
+#]===========================]
+install (
+  FILES
+    src/ed25519-donna/ed25519.h
+  DESTINATION include/ed25519-donna)

Builds/CMake/deps/Findlibarchive_pc.cmake

@@ -0,0 +1,22 @@
+find_package (PkgConfig REQUIRED)
+pkg_search_module (libarchive_PC QUIET libarchive>=3.4.3)
+
+if(static)
+  set(LIBARCHIVE_LIB libarchive.a)
+else()
+  set(LIBARCHIVE_LIB archive)
+endif()
+
+find_library (archive
+  NAMES ${LIBARCHIVE_LIB}
+  HINTS
+    ${libarchive_PC_LIBDIR}
+    ${libarchive_PC_LIBRARY_DIRS}
+  NO_DEFAULT_PATH)
+
+find_path (LIBARCHIVE_INCLUDE_DIR
+  NAMES archive.h
+  HINTS
+    ${libarchive_PC_INCLUDEDIR}
+    ${libarchive_PC_INCLUDEDIRS}
+  NO_DEFAULT_PATH)

Builds/CMake/deps/Findlz4.cmake

@@ -0,0 +1,24 @@
+find_package (PkgConfig)
+if (PKG_CONFIG_FOUND)
+  pkg_search_module (lz4_PC QUIET liblz4>=1.9)
+endif ()
+
+if(static)
+  set(LZ4_LIB liblz4.a)
+else()
+  set(LZ4_LIB lz4.so)
+endif()
+
+find_library (lz4
+  NAMES ${LZ4_LIB}
+  HINTS
+    ${lz4_PC_LIBDIR}
+    ${lz4_PC_LIBRARY_DIRS}
+  NO_DEFAULT_PATH)
+
+find_path (LZ4_INCLUDE_DIR
+  NAMES lz4.h
+  HINTS
+    ${lz4_PC_INCLUDEDIR}
+    ${lz4_PC_INCLUDEDIRS}
+  NO_DEFAULT_PATH)

Builds/CMake/deps/Findsecp256k1.cmake

@@ -0,0 +1,24 @@
+find_package (PkgConfig)
+if (PKG_CONFIG_FOUND)
+  pkg_search_module (secp256k1_PC QUIET libsecp256k1)
+endif ()
+
+if(static)
+  set(SECP256K1_LIB libsecp256k1.a)
+else()
+  set(SECP256K1_LIB secp256k1)
+endif()
+
+find_library(secp256k1
+  NAMES ${SECP256K1_LIB}
+  HINTS
+    ${secp256k1_PC_LIBDIR}
+    ${secp256k1_PC_LIBRARY_PATHS}
+  NO_DEFAULT_PATH)
+
+find_path (SECP256K1_INCLUDE_DIR
+  NAMES secp256k1.h
+  HINTS
+    ${secp256k1_PC_INCLUDEDIR}
+    ${secp256k1_PC_INCLUDEDIRS}
+  NO_DEFAULT_PATH)

Builds/CMake/deps/Findsnappy.cmake

@@ -0,0 +1,24 @@
+find_package (PkgConfig)
+if (PKG_CONFIG_FOUND)
+  pkg_search_module (snappy_PC QUIET snappy>=1.1.7)
+endif ()
+
+if(static)
+  set(SNAPPY_LIB libsnappy.a)
+else()
+  set(SNAPPY_LIB libsnappy.so)
+endif()
+
+find_library (snappy
+  NAMES ${SNAPPY_LIB}
+  HINTS
+    ${snappy_PC_LIBDIR}
+    ${snappy_PC_LIBRARY_DIRS}
+  NO_DEFAULT_PATH)
+
+find_path (SNAPPY_INCLUDE_DIR
+  NAMES snappy.h
+  HINTS
+    ${snappy_PC_INCLUDEDIR}
+    ${snappy_PC_INCLUDEDIRS}
+  NO_DEFAULT_PATH)

Builds/CMake/deps/Findsoci.cmake

@@ -0,0 +1,17 @@
+find_package (PkgConfig)
+if (PKG_CONFIG_FOUND)
+  # TBD - currently no soci pkgconfig
+  #pkg_search_module (soci_PC QUIET libsoci_core>=3.2)
+endif ()
+
+if(static)
+  set(SOCI_LIB libsoci.a)
+else()
+  set(SOCI_LIB libsoci_core.so)
+endif()
+
+find_library (soci
+  NAMES ${SOCI_LIB})
+
+find_path (SOCI_INCLUDE_DIR
+  NAMES soci/soci.h)

Builds/CMake/deps/Findsqlite.cmake

@@ -0,0 +1,24 @@
+find_package (PkgConfig)
+if (PKG_CONFIG_FOUND)
+  pkg_search_module (sqlite_PC QUIET sqlite3>=3.26.0)
+endif ()
+
+if(static)
+  set(SQLITE_LIB libsqlite3.a)
+else()
+  set(SQLITE_LIB sqlite3.so)
+endif()
+
+find_library (sqlite3
+  NAMES ${SQLITE_LIB}
+  HINTS
+    ${sqlite_PC_LIBDIR}
+    ${sqlite_PC_LIBRARY_DIRS}
+  NO_DEFAULT_PATH)
+
+find_path (SQLITE_INCLUDE_DIR
+  NAMES sqlite3.h
+  HINTS
+    ${sqlite_PC_INCLUDEDIR}
+    ${sqlite_PC_INCLUDEDIRS}
+  NO_DEFAULT_PATH)

Builds/CMake/deps/Libarchive.cmake

@@ -0,0 +1,163 @@
+#[===================================================================[
+   NIH dep: libarchive
+#]===================================================================]
+
+option (local_libarchive "use local build of libarchive." OFF)
+add_library (archive_lib UNKNOWN IMPORTED GLOBAL)
+
+if (NOT local_libarchive)
+  if (NOT WIN32)
+    find_package(libarchive_pc REQUIRED)
+  endif ()
+  if (archive)
+    message (STATUS "Found libarchive using pkg-config. Using ${archive}.")
+    set_target_properties (archive_lib PROPERTIES
+      IMPORTED_LOCATION_DEBUG
+        ${archive}
+      IMPORTED_LOCATION_RELEASE
+        ${archive}
+      INTERFACE_INCLUDE_DIRECTORIES
+        ${LIBARCHIVE_INCLUDE_DIR})
+      # pkg-config can return extra info for static lib linking
+      # this is probably needed/useful generally, but apply
+      # to APPLE for now (mostly for homebrew)
+      if (APPLE AND static AND libarchive_PC_STATIC_LIBRARIES)
+        message(STATUS "NOTE: libarchive static libs: ${libarchive_PC_STATIC_LIBRARIES}")
+        # also, APPLE seems to need iconv...maybe linux does too (TBD)
+        target_link_libraries (archive_lib
+          INTERFACE iconv ${libarchive_PC_STATIC_LIBRARIES})
+      endif ()
+  else ()
+    ## now try searching using the minimal find module that cmake provides
+    find_package(LibArchive 3.4.3 QUIET)
+    if (LibArchive_FOUND)
+      if (static)
+        # find module doesn't find static libs currently, so we re-search
+        get_filename_component(_loc ${LibArchive_LIBRARY} DIRECTORY)
+        find_library(_la_static
+          NAMES libarchive.a archive_static.lib archive.lib
+          PATHS ${_loc})
+        if (_la_static)
+          set (_la_lib ${_la_static})
+        else ()
+          message (WARNING "unable to find libarchive static lib - switching to local build")
+          set (local_libarchive ON CACHE BOOL "" FORCE)
+        endif ()
+      else ()
+        set (_la_lib ${LibArchive_LIBRARY})
+      endif ()
+      if (NOT local_libarchive)
+        message (STATUS "Found libarchive using module/config. Using ${_la_lib}.")
+        set_target_properties (archive_lib PROPERTIES
+          IMPORTED_LOCATION_DEBUG
+            ${_la_lib}
+          IMPORTED_LOCATION_RELEASE
+            ${_la_lib}
+          INTERFACE_INCLUDE_DIRECTORIES
+            ${LibArchive_INCLUDE_DIRS})
+      endif ()
+    else ()
+      set (local_libarchive ON CACHE BOOL "" FORCE)
+    endif ()
+  endif ()
+endif()
+
+if (local_libarchive)
+  set (lib_post "")
+  if (MSVC)
+    set (lib_post "_static")
+  endif ()
+  ExternalProject_Add (libarchive
+    PREFIX ${nih_cache_path}
+    GIT_REPOSITORY https://github.com/libarchive/libarchive.git
+    GIT_TAG v3.4.3
+    CMAKE_ARGS
+      # passing the compiler seems to be needed for windows CI, sadly
+      -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
+      -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
+      $<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
+      -DCMAKE_DEBUG_POSTFIX=_d
+      $<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
+      -DENABLE_LZ4=ON
+      -ULZ4_*
+      -DLZ4_INCLUDE_DIR=$<JOIN:$<TARGET_PROPERTY:lz4_lib,INTERFACE_INCLUDE_DIRECTORIES>,::>
+      # because we are building a static lib, this lz4 library doesn't
+      # actually matter since you can't generally link static libs to other static
+      # libs. The include files are needed, but the library itself is not (until
+      # we link our application, at which point we use the lz4 we built above).
+      # nonetheless, we need to provide a library to libarchive else it will
+      # NOT include lz4 support when configuring
+      -DLZ4_LIBRARY=$<IF:$<CONFIG:Debug>,$<TARGET_PROPERTY:lz4_lib,IMPORTED_LOCATION_DEBUG>,$<TARGET_PROPERTY:lz4_lib,IMPORTED_LOCATION_RELEASE>>
+      -DENABLE_WERROR=OFF
+      -DENABLE_TAR=OFF
+      -DENABLE_TAR_SHARED=OFF
+      -DENABLE_INSTALL=ON
+      -DENABLE_NETTLE=OFF
+      -DENABLE_OPENSSL=OFF
+      -DENABLE_LZO=OFF
+      -DENABLE_LZMA=OFF
+      -DENABLE_ZLIB=OFF
+      -DENABLE_BZip2=OFF
+      -DENABLE_LIBXML2=OFF
+      -DENABLE_EXPAT=OFF
+      -DENABLE_PCREPOSIX=OFF
+      -DENABLE_LibGCC=OFF
+      -DENABLE_CNG=OFF
+      -DENABLE_CPIO=OFF
+      -DENABLE_CPIO_SHARED=OFF
+      -DENABLE_CAT=OFF
+      -DENABLE_CAT_SHARED=OFF
+      -DENABLE_XATTR=OFF
+      -DENABLE_ACL=OFF
+      -DENABLE_ICONV=OFF
+      -DENABLE_TEST=OFF
+      -DENABLE_COVERAGE=OFF
+      $<$<BOOL:${MSVC}>:
+        "-DCMAKE_C_FLAGS=-GR -Gd -fp:precise -FS -MP"
+        "-DCMAKE_C_FLAGS_DEBUG=-MTd"
+        "-DCMAKE_C_FLAGS_RELEASE=-MT"
+      >
+    LIST_SEPARATOR ::
+    LOG_BUILD ON
+    LOG_CONFIGURE ON
+    BUILD_COMMAND
+      ${CMAKE_COMMAND}
+      --build .
+      --config $<CONFIG>
+      --target archive_static
+      $<$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.12>:--parallel ${ep_procs}>
+      $<$<BOOL:${is_multiconfig}>:
+        COMMAND
+          ${CMAKE_COMMAND} -E copy
+            <BINARY_DIR>/libarchive/$<CONFIG>/${ep_lib_prefix}archive${lib_post}$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
+            <BINARY_DIR>/libarchive
+        >
+    TEST_COMMAND ""
+    INSTALL_COMMAND ""
+    DEPENDS lz4_lib
+    BUILD_BYPRODUCTS
+      <BINARY_DIR>/libarchive/${ep_lib_prefix}archive${lib_post}${ep_lib_suffix}
+      <BINARY_DIR>/libarchive/${ep_lib_prefix}archive${lib_post}_d${ep_lib_suffix}
+  )
+  ExternalProject_Get_Property (libarchive BINARY_DIR)
+  ExternalProject_Get_Property (libarchive SOURCE_DIR)
+  if (CMAKE_VERBOSE_MAKEFILE)
+    print_ep_logs (libarchive)
+  endif ()
+  file (MAKE_DIRECTORY ${SOURCE_DIR}/libarchive)
+  set_target_properties (archive_lib PROPERTIES
+    IMPORTED_LOCATION_DEBUG
+      ${BINARY_DIR}/libarchive/${ep_lib_prefix}archive${lib_post}_d${ep_lib_suffix}
+    IMPORTED_LOCATION_RELEASE
+      ${BINARY_DIR}/libarchive/${ep_lib_prefix}archive${lib_post}${ep_lib_suffix}
+    INTERFACE_INCLUDE_DIRECTORIES
+      ${SOURCE_DIR}/libarchive
+    INTERFACE_COMPILE_DEFINITIONS
+      LIBARCHIVE_STATIC)
+endif()
+
+add_dependencies (archive_lib libarchive)
+target_link_libraries (archive_lib INTERFACE lz4_lib)
+target_link_libraries (ripple_libs INTERFACE archive_lib)
+exclude_if_included (libarchive)
+exclude_if_included (archive_lib)

Builds/CMake/deps/Lz4.cmake

@@ -0,0 +1,79 @@
+#[===================================================================[
+   NIH dep: lz4
+#]===================================================================]
+
+add_library (lz4_lib STATIC IMPORTED GLOBAL)
+
+if (NOT WIN32)
+  find_package(lz4)
+endif()
+
+if(lz4)
+  set_target_properties (lz4_lib PROPERTIES
+    IMPORTED_LOCATION_DEBUG
+      ${lz4}
+    IMPORTED_LOCATION_RELEASE
+      ${lz4}
+    INTERFACE_INCLUDE_DIRECTORIES
+      ${LZ4_INCLUDE_DIR})
+
+else()
+  ExternalProject_Add (lz4
+    PREFIX ${nih_cache_path}
+    GIT_REPOSITORY https://github.com/lz4/lz4.git
+    GIT_TAG v1.9.2
+    SOURCE_SUBDIR contrib/cmake_unofficial
+    CMAKE_ARGS
+      -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
+      -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
+      $<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
+      -DCMAKE_DEBUG_POSTFIX=_d
+      $<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
+      -DBUILD_STATIC_LIBS=ON
+      -DBUILD_SHARED_LIBS=OFF
+      $<$<BOOL:${MSVC}>:
+        "-DCMAKE_C_FLAGS=-GR -Gd -fp:precise -FS -MP"
+        "-DCMAKE_C_FLAGS_DEBUG=-MTd"
+        "-DCMAKE_C_FLAGS_RELEASE=-MT"
+      >
+    LOG_BUILD ON
+    LOG_CONFIGURE ON
+    BUILD_COMMAND
+      ${CMAKE_COMMAND}
+      --build .
+      --config $<CONFIG>
+      --target lz4_static
+      $<$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.12>:--parallel ${ep_procs}>
+      $<$<BOOL:${is_multiconfig}>:
+        COMMAND
+          ${CMAKE_COMMAND} -E copy
+          <BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}lz4$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
+          <BINARY_DIR>
+        >
+    TEST_COMMAND ""
+    INSTALL_COMMAND ""
+    BUILD_BYPRODUCTS
+      <BINARY_DIR>/${ep_lib_prefix}lz4${ep_lib_suffix}
+      <BINARY_DIR>/${ep_lib_prefix}lz4_d${ep_lib_suffix}
+  )
+  ExternalProject_Get_Property (lz4 BINARY_DIR)
+  ExternalProject_Get_Property (lz4 SOURCE_DIR)
+
+  file (MAKE_DIRECTORY ${SOURCE_DIR}/lz4)
+  set_target_properties (lz4_lib PROPERTIES
+    IMPORTED_LOCATION_DEBUG
+      ${BINARY_DIR}/${ep_lib_prefix}lz4_d${ep_lib_suffix}
+    IMPORTED_LOCATION_RELEASE
+      ${BINARY_DIR}/${ep_lib_prefix}lz4${ep_lib_suffix}
+    INTERFACE_INCLUDE_DIRECTORIES
+      ${SOURCE_DIR}/lib)
+
+  if (CMAKE_VERBOSE_MAKEFILE)
+    print_ep_logs (lz4)
+  endif ()
+  add_dependencies (lz4_lib lz4)
+  target_link_libraries (ripple_libs INTERFACE lz4_lib)
+  exclude_if_included (lz4)
+endif()
+
+exclude_if_included (lz4_lib)

Builds/CMake/deps/Nudb.cmake

@@ -0,0 +1,47 @@
+#[===================================================================[
+   NIH dep: nudb
+
+   NuDB is header-only, thus is an INTERFACE lib in CMake.
+   TODO: move the library definition into NuDB repo and add
+   proper targets and export/install
+#]===================================================================]
+
+if (is_root_project) # NuDB not needed in the case of xrpl_core inclusion build
+  add_library (nudb INTERFACE)
+  if (CMAKE_VERSION VERSION_GREATER_EQUAL 3.11)
+    FetchContent_Declare(
+      nudb_src
+      GIT_REPOSITORY https://github.com/CPPAlliance/NuDB.git
+      GIT_TAG        2.0.5
+    )
+    FetchContent_GetProperties(nudb_src)
+    if(NOT nudb_src_POPULATED)
+      message (STATUS "Pausing to download NuDB...")
+      FetchContent_Populate(nudb_src)
+    endif()
+  else ()
+    ExternalProject_Add (nudb_src
+      PREFIX ${nih_cache_path}
+      GIT_REPOSITORY https://github.com/CPPAlliance/NuDB.git
+      GIT_TAG 2.0.5
+      CONFIGURE_COMMAND ""
+      BUILD_COMMAND ""
+      TEST_COMMAND ""
+      INSTALL_COMMAND ""
+    )
+    ExternalProject_Get_Property (nudb_src SOURCE_DIR)
+    set (nudb_src_SOURCE_DIR "${SOURCE_DIR}")
+    file (MAKE_DIRECTORY ${nudb_src_SOURCE_DIR}/include)
+    add_dependencies (nudb nudb_src)
+  endif ()
+
+  file(TO_CMAKE_PATH "${nudb_src_SOURCE_DIR}" nudb_src_SOURCE_DIR)
+# specify as system includes so as to avoid warnings
+  target_include_directories (nudb SYSTEM INTERFACE ${nudb_src_SOURCE_DIR}/include)
+  target_link_libraries (nudb
+    INTERFACE
+      Boost::thread
+      Boost::system)
+  add_library (NIH::nudb ALIAS nudb)
+  target_link_libraries (ripple_libs INTERFACE NIH::nudb)
+endif ()

Builds/CMake/deps/OpenSSL.cmake

@@ -0,0 +1,48 @@
+#[===================================================================[
+   NIH dep: openssl
+#]===================================================================]
+
+#[===============================================[
+  OPENSSL_ROOT_DIR is the only variable that
+  FindOpenSSL honors for locating, so convert any
+  OPENSSL_ROOT vars to this
+#]===============================================]
+if (NOT DEFINED OPENSSL_ROOT_DIR)
+  if (DEFINED ENV{OPENSSL_ROOT})
+    set (OPENSSL_ROOT_DIR $ENV{OPENSSL_ROOT})
+  elseif (HOMEBREW)
+    execute_process (COMMAND ${HOMEBREW} --prefix openssl
+      OUTPUT_VARIABLE OPENSSL_ROOT_DIR
+      OUTPUT_STRIP_TRAILING_WHITESPACE)
+  endif ()
+  file (TO_CMAKE_PATH "${OPENSSL_ROOT_DIR}" OPENSSL_ROOT_DIR)
+endif ()
+
+if (static)
+  set (OPENSSL_USE_STATIC_LIBS ON)
+endif ()
+set (OPENSSL_MSVC_STATIC_RT ON)
+find_package (OpenSSL 1.1.1 REQUIRED)
+target_link_libraries (ripple_libs
+  INTERFACE
+    OpenSSL::SSL
+    OpenSSL::Crypto)
+# disable SSLv2...this can also be done when building/configuring OpenSSL
+set_target_properties(OpenSSL::SSL PROPERTIES
+    INTERFACE_COMPILE_DEFINITIONS OPENSSL_NO_SSL2)
+#[=========================================================[
+   https://gitlab.kitware.com/cmake/cmake/issues/16885
+   depending on how openssl is built, it might depend
+   on zlib. In fact, the openssl find package should
+   figure this out for us, but it does not currently...
+   so let's add zlib ourselves to the lib list
+   TODO: investigate linking to static zlib for static
+   build option
+#]=========================================================]
+find_package (ZLIB)
+set (has_zlib FALSE)
+if (TARGET ZLIB::ZLIB)
+  set_target_properties(OpenSSL::Crypto PROPERTIES
+    INTERFACE_LINK_LIBRARIES ZLIB::ZLIB)
+  set (has_zlib TRUE)
+endif ()

Builds/CMake/deps/Postgres.cmake

@@ -0,0 +1,70 @@
+if(reporting)
+    find_package(PostgreSQL)
+    if(NOT PostgreSQL_FOUND)
+        message("find_package did not find postgres")
+        find_library(postgres NAMES pq libpq libpq-dev pq-dev postgresql-devel)
+        find_path(libpq-fe NAMES libpq-fe.h PATH_SUFFIXES postgresql pgsql include)
+
+        if(NOT libpq-fe_FOUND OR NOT postgres_FOUND)
+            message("No system installed Postgres found. Will build")
+            add_library(postgres SHARED IMPORTED GLOBAL)
+            add_library(pgport SHARED IMPORTED GLOBAL)
+            add_library(pgcommon SHARED IMPORTED GLOBAL)
+            ExternalProject_Add(postgres_src
+                PREFIX ${nih_cache_path}
+                GIT_REPOSITORY https://github.com/postgres/postgres.git
+                GIT_TAG master
+                CONFIGURE_COMMAND ./configure --without-readline > /dev/null
+                BUILD_COMMAND ${CMAKE_COMMAND} -E env --unset=MAKELEVEL make
+                UPDATE_COMMAND ""
+                BUILD_IN_SOURCE 1
+                INSTALL_COMMAND ""
+                BUILD_BYPRODUCTS
+                    <BINARY_DIR>/src/interfaces/libpq/${ep_lib_prefix}pq.a
+                    <BINARY_DIR>/src/common/${ep_lib_prefix}pgcommon.a
+                    <BINARY_DIR>/src/port/${ep_lib_prefix}pgport.a
+                LOG_BUILD TRUE
+                )
+            ExternalProject_Get_Property (postgres_src SOURCE_DIR)
+            ExternalProject_Get_Property (postgres_src BINARY_DIR)
+
+            set (postgres_src_SOURCE_DIR "${SOURCE_DIR}")
+            file (MAKE_DIRECTORY ${postgres_src_SOURCE_DIR})
+            list(APPEND INCLUDE_DIRS
+                ${SOURCE_DIR}/src/include
+                ${SOURCE_DIR}/src/interfaces/libpq
+                )
+            set_target_properties(postgres PROPERTIES
+                IMPORTED_LOCATION
+                    ${BINARY_DIR}/src/interfaces/libpq/${ep_lib_prefix}pq.a
+                INTERFACE_INCLUDE_DIRECTORIES
+                    "${INCLUDE_DIRS}"
+                )
+            set_target_properties(pgcommon PROPERTIES
+                IMPORTED_LOCATION
+                    ${BINARY_DIR}/src/common/${ep_lib_prefix}pgcommon.a
+                INTERFACE_INCLUDE_DIRECTORIES
+                    "${INCLUDE_DIRS}"
+                )
+            set_target_properties(pgport PROPERTIES
+                IMPORTED_LOCATION
+                    ${BINARY_DIR}/src/port/${ep_lib_prefix}pgport.a
+                INTERFACE_INCLUDE_DIRECTORIES
+                    "${INCLUDE_DIRS}"
+                )
+            add_dependencies(postgres postgres_src)
+            add_dependencies(pgcommon postgres_src)
+            add_dependencies(pgport postgres_src)
+            file(TO_CMAKE_PATH "${postgres_src_SOURCE_DIR}" postgres_src_SOURCE_DIR)
+            target_link_libraries(ripple_libs INTERFACE postgres pgcommon pgport)
+        else()
+            message("Found system installed Postgres via find_libary")
+            target_include_directories(ripple_libs INTERFACE ${libpq-fe})
+            target_link_libraries(ripple_libs INTERFACE ${postgres})
+        endif()
+    else()
+        message("Found system installed Postgres via find_package")
+        target_include_directories(ripple_libs INTERFACE ${PostgreSQL_INCLUDE_DIRS})
+        target_link_libraries(ripple_libs INTERFACE ${PostgreSQL_LIBRARIES})
+    endif()
+endif()

Builds/CMake/deps/Protobuf.cmake

@@ -0,0 +1,155 @@
+#[===================================================================[
+   import protobuf (lib and compiler) and create a lib
+   from our proto message definitions. If the system protobuf
+   is not found, fallback on EP to download and build a version
+   from official source.
+#]===================================================================]
+
+if (static)
+  set (Protobuf_USE_STATIC_LIBS ON)
+endif ()
+find_package (Protobuf 3.8)
+if (is_multiconfig)
+    set(protobuf_protoc_lib ${Protobuf_PROTOC_LIBRARIES})
+else ()
+    string(TOUPPER ${CMAKE_BUILD_TYPE} upper_cmake_build_type)
+    set(protobuf_protoc_lib ${Protobuf_PROTOC_LIBRARY_${upper_cmake_build_type}})
+endif ()
+if (local_protobuf OR NOT (Protobuf_FOUND AND Protobuf_PROTOC_EXECUTABLE AND protobuf_protoc_lib))
+  include (GNUInstallDirs)
+  message (STATUS "using local protobuf build.")
+  set(protobuf_reqs Protobuf_PROTOC_EXECUTABLE protobuf_protoc_lib)
+  foreach(lib ${protobuf_reqs})
+    if(NOT ${lib})
+      message(STATUS "Couldn't find ${lib}")
+    endif()
+  endforeach()
+  if (WIN32)
+    # protobuf prepends lib even on windows
+    set (pbuf_lib_pre "lib")
+  else ()
+    set (pbuf_lib_pre ${ep_lib_prefix})
+  endif ()
+  # for the external project build of protobuf, we currently ignore the
+  # static option and always build static libs here. This is consistent
+  # with our other EP builds. Dynamic libs in an EP would add complexity
+  # because we'd need to get them into the runtime path, and probably
+  # install them.
+  ExternalProject_Add (protobuf_src
+    PREFIX ${nih_cache_path}
+    GIT_REPOSITORY https://github.com/protocolbuffers/protobuf.git
+    GIT_TAG v3.8.0
+    SOURCE_SUBDIR cmake
+    CMAKE_ARGS
+      -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
+      -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
+      -DCMAKE_INSTALL_PREFIX=<BINARY_DIR>/_installed_
+      -Dprotobuf_BUILD_TESTS=OFF
+      -Dprotobuf_BUILD_EXAMPLES=OFF
+      -Dprotobuf_BUILD_PROTOC_BINARIES=ON
+      -Dprotobuf_MSVC_STATIC_RUNTIME=ON
+      -DBUILD_SHARED_LIBS=OFF
+      -Dprotobuf_BUILD_SHARED_LIBS=OFF
+      -DCMAKE_DEBUG_POSTFIX=_d
+      -Dprotobuf_DEBUG_POSTFIX=_d
+      -Dprotobuf_WITH_ZLIB=$<IF:$<BOOL:${has_zlib}>,ON,OFF>
+      $<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
+      $<$<BOOL:${unity}>:-DCMAKE_UNITY_BUILD=ON}>
+      $<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
+      $<$<BOOL:${MSVC}>:
+	"-DCMAKE_CXX_FLAGS=-GR -Gd -fp:precise -FS -EHa -MP"
+      >
+    LOG_BUILD ON
+    LOG_CONFIGURE ON
+    BUILD_COMMAND
+      ${CMAKE_COMMAND}
+      --build .
+      --config $<CONFIG>
+      $<$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.12>:--parallel ${ep_procs}>
+    TEST_COMMAND ""
+    INSTALL_COMMAND
+      ${CMAKE_COMMAND} -E env --unset=DESTDIR ${CMAKE_COMMAND} --build . --config $<CONFIG> --target install
+    BUILD_BYPRODUCTS
+      <BINARY_DIR>/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protobuf${ep_lib_suffix}
+      <BINARY_DIR>/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protobuf_d${ep_lib_suffix}
+      <BINARY_DIR>/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protoc${ep_lib_suffix}
+      <BINARY_DIR>/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protoc_d${ep_lib_suffix}
+      <BINARY_DIR>/_installed_/bin/protoc${CMAKE_EXECUTABLE_SUFFIX}
+  )
+  ExternalProject_Get_Property (protobuf_src BINARY_DIR)
+  ExternalProject_Get_Property (protobuf_src SOURCE_DIR)
+  if (CMAKE_VERBOSE_MAKEFILE)
+    print_ep_logs (protobuf_src)
+  endif ()
+  exclude_if_included (protobuf_src)
+
+  if (NOT TARGET protobuf::libprotobuf)
+    add_library (protobuf::libprotobuf STATIC IMPORTED GLOBAL)
+  endif ()
+  file (MAKE_DIRECTORY ${BINARY_DIR}/_installed_/include)
+  set_target_properties (protobuf::libprotobuf PROPERTIES
+    IMPORTED_LOCATION_DEBUG
+      ${BINARY_DIR}/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protobuf_d${ep_lib_suffix}
+    IMPORTED_LOCATION_RELEASE
+      ${BINARY_DIR}/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protobuf${ep_lib_suffix}
+    INTERFACE_INCLUDE_DIRECTORIES
+      ${BINARY_DIR}/_installed_/include)
+  add_dependencies (protobuf::libprotobuf protobuf_src)
+  exclude_if_included (protobuf::libprotobuf)
+
+  if (NOT TARGET protobuf::libprotoc)
+    add_library (protobuf::libprotoc STATIC IMPORTED GLOBAL)
+  endif ()
+  set_target_properties (protobuf::libprotoc PROPERTIES
+    IMPORTED_LOCATION_DEBUG
+      ${BINARY_DIR}/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protoc_d${ep_lib_suffix}
+    IMPORTED_LOCATION_RELEASE
+      ${BINARY_DIR}/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protoc${ep_lib_suffix}
+    INTERFACE_INCLUDE_DIRECTORIES
+      ${BINARY_DIR}/_installed_/include)
+  add_dependencies (protobuf::libprotoc protobuf_src)
+  exclude_if_included (protobuf::libprotoc)
+
+  if (NOT TARGET protobuf::protoc)
+    add_executable (protobuf::protoc IMPORTED)
+    exclude_if_included (protobuf::protoc)
+  endif ()
+  set_target_properties (protobuf::protoc PROPERTIES
+    IMPORTED_LOCATION "${BINARY_DIR}/_installed_/bin/protoc${CMAKE_EXECUTABLE_SUFFIX}")
+  add_dependencies (protobuf::protoc protobuf_src)
+else ()
+  if (NOT TARGET protobuf::protoc)
+    if (EXISTS "${Protobuf_PROTOC_EXECUTABLE}")
+      add_executable (protobuf::protoc IMPORTED)
+      set_target_properties (protobuf::protoc PROPERTIES
+        IMPORTED_LOCATION "${Protobuf_PROTOC_EXECUTABLE}")
+    else ()
+      message (FATAL_ERROR "Protobuf import failed")
+    endif ()
+  endif ()
+endif ()
+
+file (MAKE_DIRECTORY ${CMAKE_BINARY_DIR}/proto_gen)
+set (save_CBD ${CMAKE_CURRENT_BINARY_DIR})
+set (CMAKE_CURRENT_BINARY_DIR ${CMAKE_BINARY_DIR}/proto_gen)
+protobuf_generate_cpp (
+  PROTO_SRCS
+  PROTO_HDRS
+  src/ripple/proto/ripple.proto)
+set (CMAKE_CURRENT_BINARY_DIR ${save_CBD})
+
+add_library (pbufs STATIC ${PROTO_SRCS} ${PROTO_HDRS})
+
+target_include_directories (pbufs PRIVATE src)
+target_include_directories (pbufs
+  SYSTEM PUBLIC ${CMAKE_BINARY_DIR}/proto_gen)
+target_link_libraries (pbufs protobuf::libprotobuf)
+target_compile_options (pbufs
+  PUBLIC
+    $<$<BOOL:${is_xcode}>:
+      --system-header-prefix="google/protobuf"
+      -Wno-deprecated-dynamic-exception-spec
+    >)
+add_library (Ripple::pbufs ALIAS pbufs)
+target_link_libraries (ripple_libs INTERFACE Ripple::pbufs)
+exclude_if_included (pbufs)

Builds/CMake/deps/Rocksdb.cmake

@@ -0,0 +1,177 @@
+#[===================================================================[
+   NIH dep: rocksdb
+#]===================================================================]
+
+add_library (rocksdb_lib UNKNOWN IMPORTED GLOBAL)
+set_target_properties (rocksdb_lib
+  PROPERTIES INTERFACE_COMPILE_DEFINITIONS RIPPLE_ROCKSDB_AVAILABLE=1)
+
+option (local_rocksdb "use local build of rocksdb." OFF)
+if (NOT local_rocksdb)
+  find_package (RocksDB 6.27 QUIET CONFIG)
+  if (TARGET RocksDB::rocksdb)
+    message (STATUS "Found RocksDB using config.")
+    get_target_property (_rockslib_l RocksDB::rocksdb IMPORTED_LOCATION_DEBUG)
+    if (_rockslib_l)
+      set_target_properties (rocksdb_lib PROPERTIES IMPORTED_LOCATION_DEBUG ${_rockslib_l})
+    endif ()
+    get_target_property (_rockslib_l RocksDB::rocksdb IMPORTED_LOCATION_RELEASE)
+    if (_rockslib_l)
+      set_target_properties (rocksdb_lib PROPERTIES IMPORTED_LOCATION_RELEASE ${_rockslib_l})
+    endif ()
+    get_target_property (_rockslib_l RocksDB::rocksdb IMPORTED_LOCATION)
+    if (_rockslib_l)
+      set_target_properties (rocksdb_lib PROPERTIES IMPORTED_LOCATION ${_rockslib_l})
+    endif ()
+    get_target_property (_rockslib_i RocksDB::rocksdb INTERFACE_INCLUDE_DIRECTORIES)
+    if (_rockslib_i)
+      set_target_properties (rocksdb_lib PROPERTIES INTERFACE_INCLUDE_DIRECTORIES ${_rockslib_i})
+    endif ()
+    target_link_libraries (ripple_libs INTERFACE RocksDB::rocksdb)
+  else ()
+    # using a find module with rocksdb is difficult because
+    # you have no idea how it was configured (transitive dependencies).
+    # the code below will generally find rocksdb using the module, but
+    # will then result in linker errors for static linkage since the
+    # transitive dependencies are unknown. force local build here for now, but leave the code as
+    # a placeholder for future investigation.
+    if (static)
+      set (local_rocksdb ON CACHE BOOL "" FORCE)
+      # TBD if there is some way to extract transitive deps..then:
+      #set (RocksDB_USE_STATIC ON)
+    else ()
+      find_package (RocksDB 6.27 MODULE)
+      if (ROCKSDB_FOUND)
+        if (RocksDB_LIBRARY_DEBUG)
+          set_target_properties (rocksdb_lib PROPERTIES IMPORTED_LOCATION_DEBUG ${RocksDB_LIBRARY_DEBUG})
+        endif ()
+        set_target_properties (rocksdb_lib PROPERTIES IMPORTED_LOCATION_RELEASE ${RocksDB_LIBRARIES})
+        set_target_properties (rocksdb_lib PROPERTIES IMPORTED_LOCATION ${RocksDB_LIBRARIES})
+        set_target_properties (rocksdb_lib PROPERTIES INTERFACE_INCLUDE_DIRECTORIES ${RocksDB_INCLUDE_DIRS})
+      else ()
+        set (local_rocksdb ON CACHE BOOL "" FORCE)
+      endif ()
+    endif ()
+  endif ()
+endif ()
+
+if (local_rocksdb)
+  message (STATUS "Using local build of RocksDB.")
+  ExternalProject_Add (rocksdb
+    PREFIX ${nih_cache_path}
+    GIT_REPOSITORY https://github.com/facebook/rocksdb.git
+    GIT_TAG v6.27.3
+    PATCH_COMMAND
+      # only used by windows build
+      ${CMAKE_COMMAND} -E copy_if_different
+      ${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake/rocks_thirdparty.inc
+      <SOURCE_DIR>/thirdparty.inc
+    COMMAND
+      # fixup their build version file to keep the values
+      # from changing always
+      ${CMAKE_COMMAND} -E copy_if_different
+      ${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake/rocksdb_build_version.cc.in
+      <SOURCE_DIR>/util/build_version.cc.in
+    CMAKE_ARGS
+      -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
+      -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
+      $<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
+      $<$<BOOL:${unity}>:-DCMAKE_UNITY_BUILD=ON}>
+      -DCMAKE_DEBUG_POSTFIX=_d
+      $<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
+      -DBUILD_SHARED_LIBS=OFF
+      -DCMAKE_POSITION_INDEPENDENT_CODE=ON
+      -DWITH_JEMALLOC=$<IF:$<BOOL:${jemalloc}>,ON,OFF>
+      -DWITH_SNAPPY=ON
+      -DWITH_LZ4=ON
+      -DWITH_ZLIB=OFF
+      -DUSE_RTTI=ON
+      -DWITH_ZSTD=OFF
+      -DWITH_GFLAGS=OFF
+      -DWITH_BZ2=OFF
+      -ULZ4_*
+      -Ulz4_*
+      -Dlz4_INCLUDE_DIRS=$<JOIN:$<TARGET_PROPERTY:lz4_lib,INTERFACE_INCLUDE_DIRECTORIES>,::>
+      -Dlz4_LIBRARIES=$<IF:$<CONFIG:Debug>,$<TARGET_PROPERTY:lz4_lib,IMPORTED_LOCATION_DEBUG>,$<TARGET_PROPERTY:lz4_lib,IMPORTED_LOCATION_RELEASE>>
+      -Dlz4_FOUND=ON
+      -USNAPPY_*
+      -Usnappy_*
+      -USnappy_*
+      -Dsnappy_INCLUDE_DIRS=$<JOIN:$<TARGET_PROPERTY:snappy_lib,INTERFACE_INCLUDE_DIRECTORIES>,::>
+      -Dsnappy_LIBRARIES=$<IF:$<CONFIG:Debug>,$<TARGET_PROPERTY:snappy_lib,IMPORTED_LOCATION_DEBUG>,$<TARGET_PROPERTY:snappy_lib,IMPORTED_LOCATION_RELEASE>>
+      -Dsnappy_FOUND=ON
+      -DSnappy_INCLUDE_DIRS=$<JOIN:$<TARGET_PROPERTY:snappy_lib,INTERFACE_INCLUDE_DIRECTORIES>,::>
+      -DSnappy_LIBRARIES=$<IF:$<CONFIG:Debug>,$<TARGET_PROPERTY:snappy_lib,IMPORTED_LOCATION_DEBUG>,$<TARGET_PROPERTY:snappy_lib,IMPORTED_LOCATION_RELEASE>>
+      -DSnappy_FOUND=ON
+      -DWITH_MD_LIBRARY=OFF
+      -DWITH_RUNTIME_DEBUG=$<IF:$<CONFIG:Debug>,ON,OFF>
+      -DFAIL_ON_WARNINGS=OFF
+      -DWITH_ASAN=OFF
+      -DWITH_TSAN=OFF
+      -DWITH_UBSAN=OFF
+      -DWITH_NUMA=OFF
+      -DWITH_TBB=OFF
+      -DWITH_WINDOWS_UTF8_FILENAMES=OFF
+      -DWITH_XPRESS=OFF
+      -DPORTABLE=ON
+      -DFORCE_SSE42=OFF
+      -DDISABLE_STALL_NOTIF=OFF
+      -DOPTDBG=ON
+      -DROCKSDB_LITE=OFF
+      -DWITH_FALLOCATE=ON
+      -DWITH_LIBRADOS=OFF
+      -DWITH_JNI=OFF
+      -DROCKSDB_INSTALL_ON_WINDOWS=OFF
+      -DWITH_TESTS=OFF
+      -DWITH_TOOLS=OFF
+      $<$<BOOL:${MSVC}>:
+        "-DCMAKE_CXX_FLAGS=-GR -Gd -fp:precise -FS -MP /DNDEBUG"
+      >
+      $<$<NOT:$<BOOL:${MSVC}>>:
+        "-DCMAKE_CXX_FLAGS=-DNDEBUG"
+      >
+    LOG_BUILD ON
+    LOG_CONFIGURE ON
+    BUILD_COMMAND
+      ${CMAKE_COMMAND}
+      --build .
+      --config $<CONFIG>
+      $<$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.12>:--parallel ${ep_procs}>
+      $<$<BOOL:${is_multiconfig}>:
+        COMMAND
+          ${CMAKE_COMMAND} -E copy
+          <BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}rocksdb$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
+          <BINARY_DIR>
+        >
+    LIST_SEPARATOR ::
+    TEST_COMMAND ""
+    INSTALL_COMMAND ""
+    DEPENDS snappy_lib lz4_lib
+    BUILD_BYPRODUCTS
+      <BINARY_DIR>/${ep_lib_prefix}rocksdb${ep_lib_suffix}
+      <BINARY_DIR>/${ep_lib_prefix}rocksdb_d${ep_lib_suffix}
+  )
+  ExternalProject_Get_Property (rocksdb BINARY_DIR)
+  ExternalProject_Get_Property (rocksdb SOURCE_DIR)
+  if (CMAKE_VERBOSE_MAKEFILE)
+    print_ep_logs (rocksdb)
+  endif ()
+  file (MAKE_DIRECTORY ${SOURCE_DIR}/include)
+  set_target_properties (rocksdb_lib PROPERTIES
+    IMPORTED_LOCATION_DEBUG
+      ${BINARY_DIR}/${ep_lib_prefix}rocksdb_d${ep_lib_suffix}
+    IMPORTED_LOCATION_RELEASE
+      ${BINARY_DIR}/${ep_lib_prefix}rocksdb${ep_lib_suffix}
+    INTERFACE_INCLUDE_DIRECTORIES
+      ${SOURCE_DIR}/include)
+  add_dependencies (rocksdb_lib rocksdb)
+  exclude_if_included (rocksdb)
+endif ()
+
+target_link_libraries (rocksdb_lib
+  INTERFACE
+    snappy_lib
+    lz4_lib
+    $<$<BOOL:${MSVC}>:rpcrt4>)
+exclude_if_included (rocksdb_lib)
+target_link_libraries (ripple_libs INTERFACE rocksdb_lib)

Builds/CMake/deps/Secp256k1.cmake

@@ -0,0 +1,58 @@
+#[===================================================================[
+   NIH dep: secp256k1
+#]===================================================================]
+
+add_library (secp256k1_lib STATIC IMPORTED GLOBAL)
+
+if (NOT WIN32)
+  find_package(secp256k1)
+endif()
+
+if(secp256k1)
+  set_target_properties (secp256k1_lib PROPERTIES
+    IMPORTED_LOCATION_DEBUG
+      ${secp256k1}
+    IMPORTED_LOCATION_RELEASE
+      ${secp256k1}
+    INTERFACE_INCLUDE_DIRECTORIES
+      ${SECP256K1_INCLUDE_DIR})
+
+  add_library (secp256k1 ALIAS secp256k1_lib)
+  add_library (NIH::secp256k1 ALIAS secp256k1_lib)
+
+else()
+  set(INSTALL_SECP256K1 true)
+
+  add_library (secp256k1 STATIC
+    src/secp256k1/src/secp256k1.c)
+  target_compile_definitions (secp256k1
+    PRIVATE
+      USE_NUM_NONE
+      USE_FIELD_10X26
+      USE_FIELD_INV_BUILTIN
+      USE_SCALAR_8X32
+      USE_SCALAR_INV_BUILTIN)
+  target_include_directories (secp256k1
+    PUBLIC
+      $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/src>
+      $<INSTALL_INTERFACE:include>
+    PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src/secp256k1)
+  target_compile_options (secp256k1
+    PRIVATE
+      $<$<BOOL:${MSVC}>:-wd4319>
+      $<$<NOT:$<BOOL:${MSVC}>>:
+        -Wno-deprecated-declarations
+        -Wno-unused-function
+      >
+      $<$<BOOL:${is_gcc}>:-Wno-nonnull-compare>)
+  target_link_libraries (ripple_libs INTERFACE NIH::secp256k1)
+#[===========================[
+     headers installation
+#]===========================]
+  install (
+    FILES
+      src/secp256k1/include/secp256k1.h
+  DESTINATION include/secp256k1/include)
+
+  add_library (NIH::secp256k1 ALIAS secp256k1)
+endif()

Builds/CMake/deps/Snappy.cmake

@@ -0,0 +1,77 @@
+#[===================================================================[
+   NIH dep: snappy
+#]===================================================================]
+
+add_library (snappy_lib STATIC IMPORTED GLOBAL)
+
+if (NOT WIN32)
+  find_package(snappy)
+endif()
+
+if(snappy)
+  set_target_properties (snappy_lib PROPERTIES
+    IMPORTED_LOCATION_DEBUG
+      ${snappy}
+    IMPORTED_LOCATION_RELEASE
+      ${snappy}
+    INTERFACE_INCLUDE_DIRECTORIES
+    ${SNAPPY_INCLUDE_DIR})
+
+else()
+  ExternalProject_Add (snappy
+    PREFIX ${nih_cache_path}
+    GIT_REPOSITORY https://github.com/google/snappy.git
+    GIT_TAG 1.1.7
+    CMAKE_ARGS
+      -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
+      -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
+      $<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
+      -DCMAKE_DEBUG_POSTFIX=_d
+      $<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
+      -DBUILD_SHARED_LIBS=OFF
+      -DCMAKE_POSITION_INDEPENDENT_CODE=ON
+      -DSNAPPY_BUILD_TESTS=OFF
+      $<$<BOOL:${MSVC}>:
+        "-DCMAKE_CXX_FLAGS=-GR -Gd -fp:precise -FS -EHa -MP"
+        "-DCMAKE_CXX_FLAGS_DEBUG=-MTd"
+        "-DCMAKE_CXX_FLAGS_RELEASE=-MT"
+      >
+    LOG_BUILD ON
+    LOG_CONFIGURE ON
+    BUILD_COMMAND
+      ${CMAKE_COMMAND}
+      --build .
+      --config $<CONFIG>
+      $<$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.12>:--parallel ${ep_procs}>
+      $<$<BOOL:${is_multiconfig}>:
+        COMMAND
+          ${CMAKE_COMMAND} -E copy
+          <BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}snappy$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
+          <BINARY_DIR>
+        >
+    TEST_COMMAND ""
+    INSTALL_COMMAND
+      ${CMAKE_COMMAND} -E copy_if_different <BINARY_DIR>/config.h <BINARY_DIR>/snappy-stubs-public.h <SOURCE_DIR>
+    BUILD_BYPRODUCTS
+      <BINARY_DIR>/${ep_lib_prefix}snappy${ep_lib_suffix}
+      <BINARY_DIR>/${ep_lib_prefix}snappy_d${ep_lib_suffix}
+  )
+  ExternalProject_Get_Property (snappy BINARY_DIR)
+  ExternalProject_Get_Property (snappy SOURCE_DIR)
+  if (CMAKE_VERBOSE_MAKEFILE)
+    print_ep_logs (snappy)
+  endif ()
+  file (MAKE_DIRECTORY ${SOURCE_DIR}/snappy)
+  set_target_properties (snappy_lib PROPERTIES
+    IMPORTED_LOCATION_DEBUG
+      ${BINARY_DIR}/${ep_lib_prefix}snappy_d${ep_lib_suffix}
+    IMPORTED_LOCATION_RELEASE
+      ${BINARY_DIR}/${ep_lib_prefix}snappy${ep_lib_suffix}
+    INTERFACE_INCLUDE_DIRECTORIES
+      ${SOURCE_DIR})
+endif()
+
+add_dependencies (snappy_lib snappy)
+target_link_libraries (ripple_libs INTERFACE snappy_lib)
+exclude_if_included (snappy)
+exclude_if_included (snappy_lib)

Builds/CMake/deps/Soci.cmake

@@ -0,0 +1,164 @@
+#[===================================================================[
+   NIH dep: soci
+#]===================================================================]
+
+foreach (_comp core empty sqlite3)
+  add_library ("soci_${_comp}" STATIC IMPORTED GLOBAL)
+endforeach ()
+
+if (NOT WIN32)
+  find_package(soci)
+endif()
+
+if (soci)
+  foreach (_comp core empty sqlite3)
+    set_target_properties ("soci_${_comp}" PROPERTIES
+      IMPORTED_LOCATION_DEBUG
+        ${soci}
+      IMPORTED_LOCATION_RELEASE
+        ${soci}
+      INTERFACE_INCLUDE_DIRECTORIES
+        ${SOCI_INCLUDE_DIR})
+  endforeach ()
+
+else()
+  set (soci_lib_pre ${ep_lib_prefix})
+  set (soci_lib_post "")
+  if (WIN32)
+    # for some reason soci on windows still prepends lib (non-standard)
+    set (soci_lib_pre lib)
+    # this version in the name might change if/when we change versions of soci
+    set (soci_lib_post "_4_0")
+  endif ()
+  get_target_property (_boost_incs Boost::date_time INTERFACE_INCLUDE_DIRECTORIES)
+  get_target_property (_boost_dt Boost::date_time IMPORTED_LOCATION)
+  if (NOT _boost_dt)
+    get_target_property (_boost_dt Boost::date_time IMPORTED_LOCATION_RELEASE)
+  endif ()
+  if (NOT _boost_dt)
+    get_target_property (_boost_dt Boost::date_time IMPORTED_LOCATION_DEBUG)
+  endif ()
+
+  ExternalProject_Add (soci
+    PREFIX ${nih_cache_path}
+    GIT_REPOSITORY https://github.com/SOCI/soci.git
+    GIT_TAG 04e1870294918d20761736743bb6136314c42dd5
+    # We had an issue with soci integer range checking for boost::optional
+    # and needed to remove the exception that SOCI throws in this case.
+    # This is *probably* a bug in SOCI, but has never been investigated more
+    # nor reported to the maintainers.
+    # This cmake script comments out the lines in question.
+    # This patch process is likely fragile and should be reviewed carefully
+    # whenever we update the GIT_TAG above.
+    PATCH_COMMAND
+      ${CMAKE_COMMAND} -P ${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake/soci_patch.cmake
+    CMAKE_ARGS
+      -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
+      -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
+      $<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
+      $<$<BOOL:${CMAKE_TOOLCHAIN_FILE}>:-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE}>
+      $<$<BOOL:${VCPKG_TARGET_TRIPLET}>:-DVCPKG_TARGET_TRIPLET=${VCPKG_TARGET_TRIPLET}>
+      $<$<BOOL:${unity}>:-DCMAKE_UNITY_BUILD=ON}>
+      -DCMAKE_PREFIX_PATH=${CMAKE_BINARY_DIR}/sqlite3
+      -DCMAKE_MODULE_PATH=${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake
+      -DCMAKE_INCLUDE_PATH=$<JOIN:$<TARGET_PROPERTY:sqlite,INTERFACE_INCLUDE_DIRECTORIES>,::>
+      -DCMAKE_LIBRARY_PATH=${sqlite_BINARY_DIR}
+      -DCMAKE_DEBUG_POSTFIX=_d
+      $<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
+      -DSOCI_CXX_C11=ON
+      -DSOCI_STATIC=ON
+      -DSOCI_LIBDIR=lib
+      -DSOCI_SHARED=OFF
+      -DSOCI_TESTS=OFF
+      # hacks to workaround the fact that soci doesn't currently use
+      # boost imported targets in its cmake. If they switch to
+      # proper imported targets, this next line can be removed
+      # (as well as the get_property above that sets _boost_incs)
+      -DBoost_INCLUDE_DIRS=$<JOIN:${_boost_incs},::>
+      -DBoost_INCLUDE_DIR=$<JOIN:${_boost_incs},::>
+      -DBOOST_ROOT=${BOOST_ROOT}
+      -DWITH_BOOST=ON
+      -DBoost_FOUND=ON
+      -DBoost_NO_BOOST_CMAKE=ON
+      -DBoost_DATE_TIME_FOUND=ON
+      -DSOCI_HAVE_BOOST=ON
+      -DSOCI_HAVE_BOOST_DATE_TIME=ON
+      -DBoost_DATE_TIME_LIBRARY=${_boost_dt}
+      -DSOCI_DB2=OFF
+      -DSOCI_FIREBIRD=OFF
+      -DSOCI_MYSQL=OFF
+      -DSOCI_ODBC=OFF
+      -DSOCI_ORACLE=OFF
+      -DSOCI_POSTGRESQL=OFF
+      -DSOCI_SQLITE3=ON
+      -DSQLITE3_INCLUDE_DIR=$<JOIN:$<TARGET_PROPERTY:sqlite,INTERFACE_INCLUDE_DIRECTORIES>,::>
+      -DSQLITE3_LIBRARY=$<IF:$<CONFIG:Debug>,$<TARGET_PROPERTY:sqlite,IMPORTED_LOCATION_DEBUG>,$<TARGET_PROPERTY:sqlite,IMPORTED_LOCATION_RELEASE>>
+      $<$<BOOL:${APPLE}>:-DCMAKE_FIND_FRAMEWORK=LAST>
+      $<$<BOOL:${MSVC}>:
+        "-DCMAKE_CXX_FLAGS=-GR -Gd -fp:precise -FS -EHa -MP"
+        "-DCMAKE_CXX_FLAGS_DEBUG=-MTd"
+        "-DCMAKE_CXX_FLAGS_RELEASE=-MT"
+      >
+      $<$<NOT:$<BOOL:${MSVC}>>:
+        "-DCMAKE_CXX_FLAGS=-Wno-deprecated-declarations"
+      >
+      # SEE: https://github.com/SOCI/soci/issues/640
+      $<$<AND:$<BOOL:${is_gcc}>,$<VERSION_GREATER_EQUAL:${CMAKE_CXX_COMPILER_VERSION},8>>:
+        "-DCMAKE_CXX_FLAGS=-Wno-deprecated-declarations -Wno-error=format-overflow -Wno-format-overflow -Wno-error=format-truncation"
+      >
+    LIST_SEPARATOR ::
+    LOG_BUILD ON
+    LOG_CONFIGURE ON
+    BUILD_COMMAND
+      ${CMAKE_COMMAND}
+      --build .
+      --config $<CONFIG>
+      $<$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.12>:--parallel ${ep_procs}>
+      $<$<BOOL:${is_multiconfig}>:
+        COMMAND
+          ${CMAKE_COMMAND} -E copy
+          <BINARY_DIR>/lib/$<CONFIG>/${soci_lib_pre}soci_core${soci_lib_post}$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
+          <BINARY_DIR>/lib/$<CONFIG>/${soci_lib_pre}soci_empty${soci_lib_post}$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
+          <BINARY_DIR>/lib/$<CONFIG>/${soci_lib_pre}soci_sqlite3${soci_lib_post}$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
+          <BINARY_DIR>/lib
+        >
+    TEST_COMMAND ""
+    INSTALL_COMMAND ""
+    DEPENDS sqlite
+    BUILD_BYPRODUCTS
+      <BINARY_DIR>/lib/${soci_lib_pre}soci_core${soci_lib_post}${ep_lib_suffix}
+      <BINARY_DIR>/lib/${soci_lib_pre}soci_core${soci_lib_post}_d${ep_lib_suffix}
+      <BINARY_DIR>/lib/${soci_lib_pre}soci_empty${soci_lib_post}${ep_lib_suffix}
+      <BINARY_DIR>/lib/${soci_lib_pre}soci_empty${soci_lib_post}_d${ep_lib_suffix}
+      <BINARY_DIR>/lib/${soci_lib_pre}soci_sqlite3${soci_lib_post}${ep_lib_suffix}
+      <BINARY_DIR>/lib/${soci_lib_pre}soci_sqlite3${soci_lib_post}_d${ep_lib_suffix}
+  )
+  ExternalProject_Get_Property (soci BINARY_DIR)
+  ExternalProject_Get_Property (soci SOURCE_DIR)
+  if (CMAKE_VERBOSE_MAKEFILE)
+    print_ep_logs (soci)
+  endif ()
+  file (MAKE_DIRECTORY ${SOURCE_DIR}/include)
+  file (MAKE_DIRECTORY ${BINARY_DIR}/include)
+  foreach (_comp core empty sqlite3)
+    set_target_properties ("soci_${_comp}" PROPERTIES
+      IMPORTED_LOCATION_DEBUG
+        ${BINARY_DIR}/lib/${soci_lib_pre}soci_${_comp}${soci_lib_post}_d${ep_lib_suffix}
+      IMPORTED_LOCATION_RELEASE
+        ${BINARY_DIR}/lib/${soci_lib_pre}soci_${_comp}${soci_lib_post}${ep_lib_suffix}
+      INTERFACE_INCLUDE_DIRECTORIES
+        "${SOURCE_DIR}/include;${BINARY_DIR}/include")
+    add_dependencies ("soci_${_comp}" soci) # something has to depend on the ExternalProject to trigger it
+    target_link_libraries (ripple_libs INTERFACE "soci_${_comp}")
+    if (NOT _comp STREQUAL "core")
+      target_link_libraries ("soci_${_comp}" INTERFACE soci_core)
+    endif ()
+  endforeach ()
+endif()
+
+foreach (_comp core empty sqlite3)
+  exclude_if_included ("soci_${_comp}")
+endforeach ()
+
+
+exclude_if_included (soci)

Builds/CMake/deps/Sqlite.cmake

@@ -0,0 +1,93 @@
+#[===================================================================[
+   NIH dep: sqlite
+#]===================================================================]
+
+add_library (sqlite STATIC IMPORTED GLOBAL)
+
+if (NOT WIN32)
+  find_package(sqlite)
+endif()
+
+
+if(sqlite3)
+  set_target_properties (sqlite PROPERTIES
+    IMPORTED_LOCATION_DEBUG
+      ${sqlite3}
+    IMPORTED_LOCATION_RELEASE
+      ${sqlite3}
+    INTERFACE_INCLUDE_DIRECTORIES
+    ${SQLITE_INCLUDE_DIR})
+
+else()
+  ExternalProject_Add (sqlite3
+    PREFIX ${nih_cache_path}
+    # sqlite doesn't use git, but it provides versioned tarballs
+    URL https://www.sqlite.org/2018/sqlite-amalgamation-3260000.zip
+      http://www.sqlite.org/2018/sqlite-amalgamation-3260000.zip
+      https://www2.sqlite.org/2018/sqlite-amalgamation-3260000.zip
+      http://www2.sqlite.org/2018/sqlite-amalgamation-3260000.zip
+    # ^^^ version is apparent in the URL:  3260000 => 3.26.0
+    URL_HASH SHA256=de5dcab133aa339a4cf9e97c40aa6062570086d6085d8f9ad7bc6ddf8a52096e
+    # Don't need to worry about MITM attacks too much because the download
+    # is checked against a strong hash
+    TLS_VERIFY false
+    # we wrote a very simple CMake file to build sqlite
+    # so that's what we copy here so that we can build with
+    # CMake. sqlite doesn't generally provided a build system
+    # for the single amalgamation source file.
+    PATCH_COMMAND
+      ${CMAKE_COMMAND} -E copy_if_different
+      ${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake/CMake_sqlite3.txt
+      <SOURCE_DIR>/CMakeLists.txt
+    CMAKE_ARGS
+      -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
+      -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
+      $<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
+      -DCMAKE_DEBUG_POSTFIX=_d
+      $<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
+      $<$<BOOL:${MSVC}>:
+        "-DCMAKE_C_FLAGS=-GR -Gd -fp:precise -FS -MP"
+        "-DCMAKE_C_FLAGS_DEBUG=-MTd"
+        "-DCMAKE_C_FLAGS_RELEASE=-MT"
+      >
+    LOG_BUILD ON
+    LOG_CONFIGURE ON
+    BUILD_COMMAND
+      ${CMAKE_COMMAND}
+      --build .
+      --config $<CONFIG>
+      $<$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.12>:--parallel ${ep_procs}>
+      $<$<BOOL:${is_multiconfig}>:
+        COMMAND
+          ${CMAKE_COMMAND} -E copy
+            <BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}sqlite3$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
+            <BINARY_DIR>
+        >
+    TEST_COMMAND ""
+    INSTALL_COMMAND ""
+    BUILD_BYPRODUCTS
+      <BINARY_DIR>/${ep_lib_prefix}sqlite3${ep_lib_suffix}
+      <BINARY_DIR>/${ep_lib_prefix}sqlite3_d${ep_lib_suffix}
+  )
+  ExternalProject_Get_Property (sqlite3 BINARY_DIR)
+  ExternalProject_Get_Property (sqlite3 SOURCE_DIR)
+  if (CMAKE_VERBOSE_MAKEFILE)
+    print_ep_logs (sqlite3)
+  endif ()
+
+  set_target_properties (sqlite PROPERTIES
+    IMPORTED_LOCATION_DEBUG
+      ${BINARY_DIR}/${ep_lib_prefix}sqlite3_d${ep_lib_suffix}
+    IMPORTED_LOCATION_RELEASE
+      ${BINARY_DIR}/${ep_lib_prefix}sqlite3${ep_lib_suffix}
+    INTERFACE_INCLUDE_DIRECTORIES
+      ${SOURCE_DIR})
+
+  add_dependencies (sqlite sqlite3)
+  exclude_if_included (sqlite3)
+endif()
+
+target_link_libraries (sqlite INTERFACE $<$<NOT:$<BOOL:${MSVC}>>:dl>)
+target_link_libraries (ripple_libs INTERFACE sqlite)
+exclude_if_included (sqlite)
+set(sqlite_BINARY_DIR ${BINARY_DIR})

Builds/CMake/deps/cassandra.cmake

@@ -0,0 +1,165 @@
+if(reporting)
+    find_library(cassandra NAMES cassandra)
+    if(NOT cassandra)
+
+        message("System installed Cassandra cpp driver not found. Will build")
+
+        find_library(zlib NAMES zlib1g-dev zlib-devel zlib z)
+        if(NOT zlib)
+            message("zlib not found. will build")
+            add_library(zlib STATIC IMPORTED GLOBAL)
+            ExternalProject_Add(zlib_src
+                PREFIX ${nih_cache_path}
+                GIT_REPOSITORY https://github.com/madler/zlib.git
+                GIT_TAG master
+                INSTALL_COMMAND ""
+                BUILD_BYPRODUCTS <BINARY_DIR>/${ep_lib_prefix}z.a
+                LOG_BUILD TRUE
+                LOG_CONFIGURE TRUE
+                )
+
+
+            ExternalProject_Get_Property (zlib_src SOURCE_DIR)
+            ExternalProject_Get_Property (zlib_src BINARY_DIR)
+            set (zlib_src_SOURCE_DIR "${SOURCE_DIR}")
+            file (MAKE_DIRECTORY ${zlib_src_SOURCE_DIR}/include)
+
+            set_target_properties (zlib PROPERTIES
+                IMPORTED_LOCATION
+                ${BINARY_DIR}/${ep_lib_prefix}z.a
+                INTERFACE_INCLUDE_DIRECTORIES
+                ${SOURCE_DIR}/include)
+            add_dependencies(zlib zlib_src)
+
+            file(TO_CMAKE_PATH "${zlib_src_SOURCE_DIR}" zlib_src_SOURCE_DIR)
+        endif()
+
+
+
+
+        find_library(krb5 NAMES krb5-dev libkrb5-dev)
+
+        if(NOT krb5)
+            message("krb5 not found. will build")
+            add_library(krb5 STATIC IMPORTED GLOBAL)
+            ExternalProject_Add(krb5_src
+                PREFIX ${nih_cache_path}
+                GIT_REPOSITORY https://github.com/krb5/krb5.git
+                GIT_TAG master
+                UPDATE_COMMAND ""
+                CONFIGURE_COMMAND autoreconf src && CFLAGS=-fcommon ./src/configure --enable-static --disable-shared > /dev/null
+                BUILD_IN_SOURCE 1
+                BUILD_COMMAND make
+                INSTALL_COMMAND ""
+                BUILD_BYPRODUCTS <SOURCE_DIR>/lib/${ep_lib_prefix}krb5.a
+                LOG_BUILD TRUE
+                )
+
+            ExternalProject_Get_Property (krb5_src SOURCE_DIR)
+            ExternalProject_Get_Property (krb5_src BINARY_DIR)
+            set (krb5_src_SOURCE_DIR "${SOURCE_DIR}")
+            file (MAKE_DIRECTORY ${krb5_src_SOURCE_DIR}/include)
+
+            set_target_properties (krb5 PROPERTIES
+                IMPORTED_LOCATION
+                ${BINARY_DIR}/lib/${ep_lib_prefix}krb5.a
+                INTERFACE_INCLUDE_DIRECTORIES
+                ${SOURCE_DIR}/include)
+            add_dependencies(krb5 krb5_src)
+
+            file(TO_CMAKE_PATH "${krb5_src_SOURCE_DIR}" krb5_src_SOURCE_DIR)
+        endif()
+
+
+        find_library(libuv1 NAMES uv1 libuv1 liubuv1-dev libuv1:amd64)
+
+
+        if(NOT libuv1)
+            message("libuv1 not found, will build")
+            add_library(libuv1 STATIC IMPORTED GLOBAL)
+            ExternalProject_Add(libuv_src
+                PREFIX ${nih_cache_path}
+                GIT_REPOSITORY https://github.com/libuv/libuv.git
+                GIT_TAG v1.x
+                INSTALL_COMMAND ""
+                BUILD_BYPRODUCTS <BINARY_DIR>/${ep_lib_prefix}uv_a.a
+                LOG_BUILD TRUE
+                LOG_CONFIGURE TRUE
+                )
+
+            ExternalProject_Get_Property (libuv_src SOURCE_DIR)
+            ExternalProject_Get_Property (libuv_src BINARY_DIR)
+            set (libuv_src_SOURCE_DIR "${SOURCE_DIR}")
+            file (MAKE_DIRECTORY ${libuv_src_SOURCE_DIR}/include)
+
+            set_target_properties (libuv1 PROPERTIES
+                IMPORTED_LOCATION
+                ${BINARY_DIR}/${ep_lib_prefix}uv_a.a
+                INTERFACE_INCLUDE_DIRECTORIES
+                ${SOURCE_DIR}/include)
+            add_dependencies(libuv1 libuv_src)
+
+            file(TO_CMAKE_PATH "${libuv_src_SOURCE_DIR}" libuv_src_SOURCE_DIR)
+        endif()
+
+        add_library (cassandra STATIC IMPORTED GLOBAL)
+        ExternalProject_Add(cassandra_src
+            PREFIX ${nih_cache_path}
+            GIT_REPOSITORY https://github.com/datastax/cpp-driver.git
+            GIT_TAG master
+            CMAKE_ARGS
+            -DLIBUV_ROOT_DIR=${BINARY_DIR}
+            -DLIBUV_LIBARY=${BINARY_DIR}/libuv_a.a
+            -DLIBUV_INCLUDE_DIR=${SOURCE_DIR}/include
+            -DCASS_BUILD_STATIC=ON
+            INSTALL_COMMAND ""
+            BUILD_BYPRODUCTS <BINARY_DIR>/${ep_lib_prefix}cassandra_static.a
+            LOG_BUILD TRUE
+            LOG_CONFIGURE TRUE
+            )
+
+        ExternalProject_Get_Property (cassandra_src SOURCE_DIR)
+        ExternalProject_Get_Property (cassandra_src BINARY_DIR)
+        set (cassandra_src_SOURCE_DIR "${SOURCE_DIR}")
+        file (MAKE_DIRECTORY ${cassandra_src_SOURCE_DIR}/include)
+
+        set_target_properties (cassandra PROPERTIES
+            IMPORTED_LOCATION
+            ${BINARY_DIR}/${ep_lib_prefix}cassandra_static.a
+            INTERFACE_INCLUDE_DIRECTORIES
+            ${SOURCE_DIR}/include)
+        add_dependencies(cassandra cassandra_src)
+
+        if(NOT libuv1)
+            ExternalProject_Add_StepDependencies(cassandra_src build libuv1)
+            target_link_libraries(cassandra INTERFACE libuv1)
+        else()
+            target_link_libraries(cassandra INTERFACE ${libuv1})
+        endif()
+        if(NOT krb5)
+
+            ExternalProject_Add_StepDependencies(cassandra_src build krb5)
+            target_link_libraries(cassandra INTERFACE krb5)
+        else()
+            target_link_libraries(cassandra INTERFACE ${krb5})
+        endif()
+
+        if(NOT zlib)
+            ExternalProject_Add_StepDependencies(cassandra_src build zlib)
+            target_link_libraries(cassandra INTERFACE zlib)
+        else()
+            target_link_libraries(cassandra INTERFACE ${zlib})
+        endif()
+
+        file(TO_CMAKE_PATH "${cassandra_src_SOURCE_DIR}" cassandra_src_SOURCE_DIR)
+        target_link_libraries(ripple_libs INTERFACE cassandra)
+    else()
+        message("Found system installed cassandra cpp driver")
+
+        find_path(cassandra_includes NAMES cassandra.h REQUIRED)
+        target_link_libraries (ripple_libs INTERFACE ${cassandra})
+        target_include_directories(ripple_libs INTERFACE ${cassandra_includes})
+    endif()
+
+    exclude_if_included (cassandra)
+endif()

Builds/CMake/deps/date.cmake

@@ -0,0 +1,49 @@
+#[===================================================================[
+   NIH dep: date
+
+   the main library is header-only, thus is an INTERFACE lib in CMake.
+
+   NOTE: this has been accepted into c++20 so can likely be replaced
+   when we update to that standard
+#]===================================================================]
+
+find_package (date QUIET)
+if (NOT TARGET date::date)
+  if (CMAKE_VERSION VERSION_GREATER_EQUAL 3.14)
+    FetchContent_Declare(
+      hh_date_src
+      GIT_REPOSITORY https://github.com/HowardHinnant/date.git
+      GIT_TAG        fc4cf092f9674f2670fb9177edcdee870399b829
+    )
+    FetchContent_MakeAvailable(hh_date_src)
+  else ()
+    ExternalProject_Add (hh_date_src
+      PREFIX ${nih_cache_path}
+      GIT_REPOSITORY https://github.com/HowardHinnant/date.git
+      GIT_TAG        fc4cf092f9674f2670fb9177edcdee870399b829
+      CONFIGURE_COMMAND ""
+      BUILD_COMMAND ""
+      TEST_COMMAND ""
+      INSTALL_COMMAND ""
+    )
+    ExternalProject_Get_Property (hh_date_src SOURCE_DIR)
+    set (hh_date_src_SOURCE_DIR "${SOURCE_DIR}")
+    file (MAKE_DIRECTORY ${hh_date_src_SOURCE_DIR}/include)
+    add_library (date_interface INTERFACE)
+    add_library (date::date ALIAS date_interface)
+    add_dependencies (date_interface hh_date_src)
+    file (TO_CMAKE_PATH "${hh_date_src_SOURCE_DIR}" hh_date_src_SOURCE_DIR)
+    target_include_directories (date_interface
+      SYSTEM INTERFACE
+          $<BUILD_INTERFACE:${hh_date_src_SOURCE_DIR}/include>
+          $<INSTALL_INTERFACE:include>)
+    install (
+      FILES
+        ${hh_date_src_SOURCE_DIR}/include/date/date.h
+      DESTINATION include/date)
+    install (TARGETS date_interface
+      EXPORT RippleExports
+      INCLUDES DESTINATION include)
+  endif ()
+endif ()
+

Builds/CMake/deps/gRPC.cmake

@@ -0,0 +1,364 @@
+
+# currently linking to unsecure versions...if we switch, we'll
+# need to add ssl as a link dependency to the grpc targets
+option (use_secure_grpc "use TLS version of grpc libs." OFF)
+if (use_secure_grpc)
+  set (grpc_suffix "")
+else ()
+  set (grpc_suffix "_unsecure")
+endif ()
+
+find_package (gRPC 1.23 CONFIG QUIET)
+if (TARGET gRPC::gpr AND NOT local_grpc)
+  get_target_property (_grpc_l gRPC::gpr IMPORTED_LOCATION_DEBUG)
+  if (NOT _grpc_l)
+    get_target_property (_grpc_l gRPC::gpr IMPORTED_LOCATION_RELEASE)
+  endif ()
+  if (NOT _grpc_l)
+    get_target_property (_grpc_l gRPC::gpr IMPORTED_LOCATION)
+  endif ()
+  message (STATUS "Found cmake config for gRPC. Using ${_grpc_l}.")
+else ()
+  find_package (PkgConfig QUIET)
+  if (PKG_CONFIG_FOUND)
+    pkg_check_modules (grpc QUIET "grpc${grpc_suffix}>=1.25" "grpc++${grpc_suffix}" gpr)
+  endif ()
+
+  if (grpc_FOUND)
+    message (STATUS "Found gRPC using pkg-config. Using ${grpc_gpr_PREFIX}.")
+  endif ()
+
+  add_executable (gRPC::grpc_cpp_plugin IMPORTED)
+  exclude_if_included (gRPC::grpc_cpp_plugin)
+
+  if (grpc_FOUND AND NOT local_grpc)
+    # use installed grpc (via pkg-config)
+    macro (add_imported_grpc libname_)
+      if (static)
+        set (_search "${CMAKE_STATIC_LIBRARY_PREFIX}${libname_}${CMAKE_STATIC_LIBRARY_SUFFIX}")
+      else ()
+        set (_search "${CMAKE_SHARED_LIBRARY_PREFIX}${libname_}${CMAKE_SHARED_LIBRARY_SUFFIX}")
+      endif()
+      find_library(_found_${libname_}
+        NAMES ${_search}
+        HINTS ${grpc_LIBRARY_DIRS})
+      if (_found_${libname_})
+        message (STATUS "importing ${libname_} as ${_found_${libname_}}")
+      else ()
+        message (FATAL_ERROR "using pkg-config for grpc, can't find ${_search}")
+      endif ()
+      add_library ("gRPC::${libname_}" STATIC IMPORTED GLOBAL)
+      set_target_properties ("gRPC::${libname_}" PROPERTIES IMPORTED_LOCATION ${_found_${libname_}})
+      if (grpc_INCLUDE_DIRS)
+        set_target_properties ("gRPC::${libname_}" PROPERTIES INTERFACE_INCLUDE_DIRECTORIES ${grpc_INCLUDE_DIRS})
+      endif ()
+      target_link_libraries (ripple_libs INTERFACE "gRPC::${libname_}")
+      exclude_if_included ("gRPC::${libname_}")
+    endmacro ()
+
+    set_target_properties (gRPC::grpc_cpp_plugin PROPERTIES
+        IMPORTED_LOCATION "${grpc_gpr_PREFIX}/bin/grpc_cpp_plugin${CMAKE_EXECUTABLE_SUFFIX}")
+
+    pkg_check_modules (cares QUIET libcares)
+    if (cares_FOUND)
+      if (static)
+        set (_search "${CMAKE_STATIC_LIBRARY_PREFIX}cares${CMAKE_STATIC_LIBRARY_SUFFIX}")
+        set (_prefix cares_STATIC)
+        set (_static STATIC)
+      else ()
+        set (_search "${CMAKE_SHARED_LIBRARY_PREFIX}cares${CMAKE_SHARED_LIBRARY_SUFFIX}")
+        set (_prefix cares)
+        set (_static)
+      endif()
+      find_library(_location NAMES ${_search} HINTS ${cares_LIBRARY_DIRS})
+      if (NOT _location)
+        message (FATAL_ERROR "using pkg-config for grpc, can't find c-ares")
+      endif ()
+      add_library (c-ares::cares ${_static} IMPORTED GLOBAL)
+      set_target_properties (c-ares::cares PROPERTIES
+        IMPORTED_LOCATION ${_location}
+        INTERFACE_INCLUDE_DIRECTORIES "${${_prefix}_INCLUDE_DIRS}"
+        INTERFACE_LINK_OPTIONS "${${_prefix}_LDFLAGS}"
+      )
+      exclude_if_included (c-ares::cares)
+    else ()
+      message (FATAL_ERROR "using pkg-config for grpc, can't find c-ares")
+    endif ()
+  else ()
+    #[===========================[
+       c-ares (grpc requires)
+    #]===========================]
+    ExternalProject_Add (c-ares_src
+      PREFIX ${nih_cache_path}
+      GIT_REPOSITORY https://github.com/c-ares/c-ares.git
+      GIT_TAG cares-1_15_0
+      CMAKE_ARGS
+        -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
+        $<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
+        -DCMAKE_DEBUG_POSTFIX=_d
+        $<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
+        -DCMAKE_INSTALL_PREFIX=<BINARY_DIR>/_installed_
+        -DCARES_SHARED=OFF
+        -DCARES_STATIC=ON
+        -DCARES_STATIC_PIC=ON
+        -DCARES_INSTALL=ON
+        -DCARES_MSVC_STATIC_RUNTIME=ON
+        $<$<BOOL:${MSVC}>:
+          "-DCMAKE_C_FLAGS=-GR -Gd -fp:precise -FS -MP"
+        >
+      LOG_BUILD ON
+      LOG_CONFIGURE ON
+      BUILD_COMMAND
+        ${CMAKE_COMMAND}
+          --build .
+          --config $<CONFIG>
+          $<$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.12>:--parallel ${ep_procs}>
+      TEST_COMMAND ""
+      INSTALL_COMMAND
+        ${CMAKE_COMMAND} -E env --unset=DESTDIR ${CMAKE_COMMAND} --build . --config $<CONFIG> --target install
+      BUILD_BYPRODUCTS
+        <BINARY_DIR>/_installed_/lib/${ep_lib_prefix}cares${ep_lib_suffix}
+        <BINARY_DIR>/_installed_/lib/${ep_lib_prefix}cares_d${ep_lib_suffix}
+    )
+    exclude_if_included (c-ares_src)
+    ExternalProject_Get_Property (c-ares_src BINARY_DIR)
+    set (cares_binary_dir "${BINARY_DIR}")
+
+    add_library (c-ares::cares STATIC IMPORTED GLOBAL)
+    file (MAKE_DIRECTORY ${BINARY_DIR}/_installed_/include)
+    set_target_properties (c-ares::cares PROPERTIES
+      IMPORTED_LOCATION_DEBUG
+        ${BINARY_DIR}/_installed_/lib/${ep_lib_prefix}cares_d${ep_lib_suffix}
+      IMPORTED_LOCATION_RELEASE
+        ${BINARY_DIR}/_installed_/lib/${ep_lib_prefix}cares${ep_lib_suffix}
+      INTERFACE_INCLUDE_DIRECTORIES
+        ${BINARY_DIR}/_installed_/include)
+    add_dependencies (c-ares::cares c-ares_src)
+    exclude_if_included (c-ares::cares)
+
+    if (NOT has_zlib)
+      #[===========================[
+         zlib (grpc requires)
+      #]===========================]
+      if (MSVC)
+        set (zlib_debug_postfix "d") # zlib cmake sets this internally for MSVC, so we really don't have a choice
+        set (zlib_base "zlibstatic")
+      else ()
+        set (zlib_debug_postfix "_d")
+        set (zlib_base "z")
+      endif ()
+      ExternalProject_Add (zlib_src
+        PREFIX ${nih_cache_path}
+        GIT_REPOSITORY https://github.com/madler/zlib.git
+        GIT_TAG v1.2.11
+        CMAKE_ARGS
+          -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
+          $<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
+          -DCMAKE_DEBUG_POSTFIX=${zlib_debug_postfix}
+          $<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
+          -DCMAKE_INSTALL_PREFIX=<BINARY_DIR>/_installed_
+          -DBUILD_SHARED_LIBS=OFF
+          $<$<BOOL:${MSVC}>:
+            "-DCMAKE_C_FLAGS=-GR -Gd -fp:precise -FS -MP"
+            "-DCMAKE_C_FLAGS_DEBUG=-MTd"
+            "-DCMAKE_C_FLAGS_RELEASE=-MT"
+          >
+        LOG_BUILD ON
+        LOG_CONFIGURE ON
+        BUILD_COMMAND
+          ${CMAKE_COMMAND}
+            --build .
+            --config $<CONFIG>
+            $<$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.12>:--parallel ${ep_procs}>
+        TEST_COMMAND ""
+        INSTALL_COMMAND
+          ${CMAKE_COMMAND} -E env --unset=DESTDIR ${CMAKE_COMMAND} --build . --config $<CONFIG> --target install
+        BUILD_BYPRODUCTS
+          <BINARY_DIR>/_installed_/lib/${ep_lib_prefix}${zlib_base}${ep_lib_suffix}
+          <BINARY_DIR>/_installed_/lib/${ep_lib_prefix}${zlib_base}${zlib_debug_postfix}${ep_lib_suffix}
+      )
+      exclude_if_included (zlib_src)
+      ExternalProject_Get_Property (zlib_src BINARY_DIR)
+      set (zlib_binary_dir "${BINARY_DIR}")
+
+      add_library (ZLIB::ZLIB STATIC IMPORTED GLOBAL)
+      file (MAKE_DIRECTORY ${BINARY_DIR}/_installed_/include)
+      set_target_properties (ZLIB::ZLIB PROPERTIES
+        IMPORTED_LOCATION_DEBUG
+          ${BINARY_DIR}/_installed_/lib/${ep_lib_prefix}${zlib_base}${zlib_debug_postfix}${ep_lib_suffix}
+        IMPORTED_LOCATION_RELEASE
+          ${BINARY_DIR}/_installed_/lib/${ep_lib_prefix}${zlib_base}${ep_lib_suffix}
+        INTERFACE_INCLUDE_DIRECTORIES
+          ${BINARY_DIR}/_installed_/include)
+      add_dependencies (ZLIB::ZLIB zlib_src)
+      exclude_if_included (ZLIB::ZLIB)
+    endif ()
+
+    #[===========================[
+       grpc
+    #]===========================]
+    ExternalProject_Add (grpc_src
+      PREFIX ${nih_cache_path}
+      GIT_REPOSITORY https://github.com/grpc/grpc.git
+      GIT_TAG v1.25.0
+      CMAKE_ARGS
+        -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
+        -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
+        $<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
+        $<$<BOOL:${CMAKE_TOOLCHAIN_FILE}>:-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE}>
+        $<$<BOOL:${VCPKG_TARGET_TRIPLET}>:-DVCPKG_TARGET_TRIPLET=${VCPKG_TARGET_TRIPLET}>
+        $<$<BOOL:${unity}>:-DCMAKE_UNITY_BUILD=ON}>
+        -DCMAKE_DEBUG_POSTFIX=_d
+        $<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
+        -DgRPC_BUILD_TESTS=OFF
+        -DgRPC_BENCHMARK_PROVIDER=""
+        -DgRPC_BUILD_CSHARP_EXT=OFF
+        -DgRPC_MSVC_STATIC_RUNTIME=ON
+        -DgRPC_INSTALL=OFF
+        -DgRPC_CARES_PROVIDER=package
+        -Dc-ares_DIR=${cares_binary_dir}/_installed_/lib/cmake/c-ares
+        -DgRPC_SSL_PROVIDER=package
+        -DOPENSSL_ROOT_DIR=${OPENSSL_ROOT_DIR}
+        -DgRPC_PROTOBUF_PROVIDER=package
+        -DProtobuf_USE_STATIC_LIBS=$<IF:$<AND:$<BOOL:${Protobuf_FOUND}>,$<NOT:$<BOOL:${static}>>>,OFF,ON>
+        -DProtobuf_INCLUDE_DIR=$<JOIN:$<TARGET_PROPERTY:protobuf::libprotobuf,INTERFACE_INCLUDE_DIRECTORIES>,:_:>
+        -DProtobuf_LIBRARY=$<IF:$<CONFIG:Debug>,$<TARGET_PROPERTY:protobuf::libprotobuf,IMPORTED_LOCATION_DEBUG>,$<TARGET_PROPERTY:protobuf::libprotobuf,IMPORTED_LOCATION_RELEASE>>
+        -DProtobuf_PROTOC_LIBRARY=$<IF:$<CONFIG:Debug>,$<TARGET_PROPERTY:protobuf::libprotoc,IMPORTED_LOCATION_DEBUG>,$<TARGET_PROPERTY:protobuf::libprotoc,IMPORTED_LOCATION_RELEASE>>
+        -DProtobuf_PROTOC_EXECUTABLE=$<TARGET_PROPERTY:protobuf::protoc,IMPORTED_LOCATION>
+        -DgRPC_ZLIB_PROVIDER=package
+        $<$<NOT:$<BOOL:${has_zlib}>>:-DZLIB_ROOT=${zlib_binary_dir}/_installed_>
+        $<$<BOOL:${MSVC}>:
+          "-DCMAKE_CXX_FLAGS=-GR -Gd -fp:precise -FS -EHa -MP"
+          "-DCMAKE_C_FLAGS=-GR -Gd -fp:precise -FS -MP"
+        >
+      LOG_BUILD ON
+      LOG_CONFIGURE ON
+      BUILD_COMMAND
+        ${CMAKE_COMMAND}
+        --build .
+        --config $<CONFIG>
+        $<$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.12>:--parallel ${ep_procs}>
+        $<$<BOOL:${is_multiconfig}>:
+          COMMAND
+            ${CMAKE_COMMAND} -E copy
+            <BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}grpc${grpc_suffix}$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
+            <BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}grpc++${grpc_suffix}$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
+            <BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}address_sorting$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
+            <BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}gpr$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
+            <BINARY_DIR>/$<CONFIG>/grpc_cpp_plugin${CMAKE_EXECUTABLE_SUFFIX}
+            <BINARY_DIR>
+          >
+      LIST_SEPARATOR :_:
+      TEST_COMMAND ""
+      INSTALL_COMMAND ""
+      DEPENDS c-ares_src
+      BUILD_BYPRODUCTS
+        <BINARY_DIR>/${ep_lib_prefix}grpc${grpc_suffix}${ep_lib_suffix}
+        <BINARY_DIR>/${ep_lib_prefix}grpc${grpc_suffix}_d${ep_lib_suffix}
+        <BINARY_DIR>/${ep_lib_prefix}grpc++${grpc_suffix}${ep_lib_suffix}
+        <BINARY_DIR>/${ep_lib_prefix}grpc++${grpc_suffix}_d${ep_lib_suffix}
+        <BINARY_DIR>/${ep_lib_prefix}address_sorting${ep_lib_suffix}
+        <BINARY_DIR>/${ep_lib_prefix}address_sorting_d${ep_lib_suffix}
+        <BINARY_DIR>/${ep_lib_prefix}gpr${ep_lib_suffix}
+        <BINARY_DIR>/${ep_lib_prefix}gpr_d${ep_lib_suffix}
+        <BINARY_DIR>/grpc_cpp_plugin${CMAKE_EXECUTABLE_SUFFIX}
+    )
+    if (TARGET protobuf_src)
+      ExternalProject_Add_StepDependencies(grpc_src build protobuf_src)
+    endif ()
+    exclude_if_included (grpc_src)
+    ExternalProject_Get_Property (grpc_src BINARY_DIR)
+    ExternalProject_Get_Property (grpc_src SOURCE_DIR)
+    set (grpc_binary_dir "${BINARY_DIR}")
+    set (grpc_source_dir "${SOURCE_DIR}")
+    if (CMAKE_VERBOSE_MAKEFILE)
+      print_ep_logs (grpc_src)
+    endif ()
+    file (MAKE_DIRECTORY ${SOURCE_DIR}/include)
+
+    macro (add_imported_grpc libname_)
+      add_library ("gRPC::${libname_}" STATIC IMPORTED GLOBAL)
+      set_target_properties ("gRPC::${libname_}" PROPERTIES
+        IMPORTED_LOCATION_DEBUG
+          ${grpc_binary_dir}/${ep_lib_prefix}${libname_}_d${ep_lib_suffix}
+        IMPORTED_LOCATION_RELEASE
+          ${grpc_binary_dir}/${ep_lib_prefix}${libname_}${ep_lib_suffix}
+        INTERFACE_INCLUDE_DIRECTORIES
+          ${grpc_source_dir}/include)
+      add_dependencies ("gRPC::${libname_}" grpc_src)
+      target_link_libraries (ripple_libs INTERFACE "gRPC::${libname_}")
+      exclude_if_included ("gRPC::${libname_}")
+    endmacro ()
+
+    set_target_properties (gRPC::grpc_cpp_plugin PROPERTIES
+        IMPORTED_LOCATION "${grpc_binary_dir}/grpc_cpp_plugin${CMAKE_EXECUTABLE_SUFFIX}")
+    add_dependencies (gRPC::grpc_cpp_plugin grpc_src)
+  endif ()
+
+  add_imported_grpc (gpr)
+  add_imported_grpc ("grpc${grpc_suffix}")
+  add_imported_grpc ("grpc++${grpc_suffix}")
+  add_imported_grpc (address_sorting)
+
+  target_link_libraries ("gRPC::grpc${grpc_suffix}" INTERFACE c-ares::cares gRPC::gpr gRPC::address_sorting ZLIB::ZLIB)
+  target_link_libraries ("gRPC::grpc++${grpc_suffix}" INTERFACE "gRPC::grpc${grpc_suffix}" gRPC::gpr)
+endif ()
+
+#[=================================[
+   generate protobuf sources for
+   grpc defs and bundle into a
+   static lib
+#]=================================]
+set (GRPC_GEN_DIR "${CMAKE_BINARY_DIR}/proto_gen_grpc")
+file (MAKE_DIRECTORY ${GRPC_GEN_DIR})
+set (GRPC_PROTO_SRCS)
+set (GRPC_PROTO_HDRS)
+set (GRPC_PROTO_ROOT "${CMAKE_CURRENT_SOURCE_DIR}/src/ripple/proto/org")
+file(GLOB_RECURSE GRPC_DEFINITION_FILES LIST_DIRECTORIES false "${GRPC_PROTO_ROOT}/*.proto")
+foreach(file ${GRPC_DEFINITION_FILES})
+  get_filename_component(_abs_file ${file} ABSOLUTE)
+  get_filename_component(_abs_dir ${_abs_file} DIRECTORY)
+  get_filename_component(_basename ${file} NAME_WE)
+  get_filename_component(_proto_inc ${GRPC_PROTO_ROOT} DIRECTORY) # updir one level
+  file(RELATIVE_PATH _rel_root_file ${_proto_inc} ${_abs_file})
+  get_filename_component(_rel_root_dir ${_rel_root_file} DIRECTORY)
+  file(RELATIVE_PATH _rel_dir ${CMAKE_CURRENT_SOURCE_DIR} ${_abs_dir})
+
+  set (src_1 "${GRPC_GEN_DIR}/${_rel_root_dir}/${_basename}.grpc.pb.cc")
+  set (src_2 "${GRPC_GEN_DIR}/${_rel_root_dir}/${_basename}.pb.cc")
+  set (hdr_1 "${GRPC_GEN_DIR}/${_rel_root_dir}/${_basename}.grpc.pb.h")
+  set (hdr_2 "${GRPC_GEN_DIR}/${_rel_root_dir}/${_basename}.pb.h")
+  add_custom_command(
+    OUTPUT ${src_1} ${src_2} ${hdr_1} ${hdr_2}
+    COMMAND protobuf::protoc
+    ARGS --grpc_out=${GRPC_GEN_DIR}
+         --cpp_out=${GRPC_GEN_DIR}
+         --plugin=protoc-gen-grpc=$<TARGET_FILE:gRPC::grpc_cpp_plugin>
+         -I ${_proto_inc} -I ${_rel_dir}
+         ${_abs_file}
+    DEPENDS ${_abs_file} protobuf::protoc gRPC::grpc_cpp_plugin
+    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
+    COMMENT "Running gRPC C++ protocol buffer compiler on ${file}"
+    VERBATIM)
+    set_source_files_properties(${src_1} ${src_2} ${hdr_1} ${hdr_2} PROPERTIES GENERATED TRUE)
+    list(APPEND GRPC_PROTO_SRCS ${src_1} ${src_2})
+    list(APPEND GRPC_PROTO_HDRS ${hdr_1} ${hdr_2})
+endforeach()
+
+add_library (grpc_pbufs STATIC ${GRPC_PROTO_SRCS} ${GRPC_PROTO_HDRS})
+#target_include_directories (grpc_pbufs PRIVATE src)
+target_include_directories (grpc_pbufs SYSTEM PUBLIC ${GRPC_GEN_DIR})
+target_link_libraries (grpc_pbufs protobuf::libprotobuf "gRPC::grpc++${grpc_suffix}")
+target_compile_options (grpc_pbufs
+  PRIVATE
+    $<$<BOOL:${MSVC}>:-wd4065>
+    $<$<NOT:$<BOOL:${MSVC}>>:-Wno-deprecated-declarations>
+  PUBLIC
+    $<$<BOOL:${MSVC}>:-wd4996>
+    $<$<BOOL:${is_xcode}>:
+      --system-header-prefix="google/protobuf"
+      -Wno-deprecated-dynamic-exception-spec
+    >)
+add_library (Ripple::grpc_pbufs ALIAS grpc_pbufs)
+target_link_libraries (ripple_libs INTERFACE Ripple::grpc_pbufs)
+exclude_if_included (grpc_pbufs)

Builds/CMake/echo_file.cmake

@@ -5,11 +5,13 @@
   IN_FILE variable.
 #]=========================================================]
 
-file (READ ${IN_FILE} contents)
-## only print files that actually have some text in them
-if (contents MATCHES "[a-z0-9A-Z]+")
-  execute_process(
-    COMMAND
-      ${CMAKE_COMMAND} -E echo "${contents}")
+if (EXISTS ${IN_FILE})
+  file (READ ${IN_FILE} contents)
+  ## only print files that actually have some text in them
+  if (contents MATCHES "[a-z0-9A-Z]+")
+    execute_process(
+      COMMAND
+        ${CMAKE_COMMAND} -E echo "${contents}")
+  endif ()
 endif ()
 

Builds/CMake/rocks_thirdparty.inc

@@ -1,16 +1,15 @@
 set (THIRDPARTY_LIBS "")
+
 if(WITH_SNAPPY)
-  find_package(snappy REQUIRED)
   add_definitions(-DSNAPPY)
-  include_directories(${SNAPPY_INCLUDE_DIR})
-  list(APPEND THIRDPARTY_LIBS ${SNAPPY_LIBRARIES})
+  include_directories(${snappy_INCLUDE_DIRS})
+  set (THIRDPARTY_LIBS ${THIRDPARTY_LIBS} ${snappy_LIBRARIES})
 endif()
 
 if(WITH_LZ4)
-  find_package(lz4 REQUIRED)
   add_definitions(-DLZ4)
-  include_directories(${LZ4_INCLUDE_DIR})
-  list(APPEND THIRDPARTY_LIBS ${LZ4_LIBRARIES})
+  include_directories(${lz4_INCLUDE_DIRS})
+  set (THIRDPARTY_LIBS ${THIRDPARTY_LIBS} ${lz4_LIBRARIES})
 endif()
 
 

Builds/CMake/rocksdb_build_version.cc.in

@@ -1,4 +1,71 @@
-#include "build_version.h"
-const char* rocksdb_build_git_sha = "rocksdb_build_git_sha: N/A";
-const char* rocksdb_build_git_date = "rocksdb_build_git_date: N/A";
-const char* rocksdb_build_compile_date = "N/A";
+// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
+
+#include <memory>
+
+#include "rocksdb/version.h"
+#include "util/string_util.h"
+
+// The build script may replace these values with real values based
+// on whether or not GIT is available and the platform settings
+static const std::string rocksdb_build_git_sha  = "rocksdb_build_git_sha:@GIT_SHA@";
+static const std::string rocksdb_build_git_tag = "rocksdb_build_git_tag:@GIT_TAG@";
+#define HAS_GIT_CHANGES @GIT_MOD@
+#if HAS_GIT_CHANGES == 0
+// If HAS_GIT_CHANGES is 0, the GIT date is used.
+// Use the time the branch/tag was last modified
+static const std::string rocksdb_build_date = "rocksdb_build_date:@GIT_DATE@";
+#else
+// If HAS_GIT_CHANGES is > 0, the branch/tag has modifications.
+// Use the time the build was created.
+static const std::string rocksdb_build_date = "rocksdb_build_date:@BUILD_DATE@";
+#endif
+
+namespace ROCKSDB_NAMESPACE {
+static void AddProperty(std::unordered_map<std::string, std::string> *props, const std::string& name) {
+  size_t colon = name.find(":");
+  if (colon != std::string::npos && colon > 0 && colon < name.length() - 1) {
+    // If we found a "@:", then this property was a build-time substitution that failed.  Skip it
+    size_t at = name.find("@", colon);
+    if (at != colon + 1) {
+      // Everything before the colon is the name, after is the value
+      (*props)[name.substr(0, colon)] = name.substr(colon + 1);
+    }
+  }
+}
+  
+static std::unordered_map<std::string, std::string>* LoadPropertiesSet() {
+  auto * properties = new std::unordered_map<std::string, std::string>();
+  AddProperty(properties, rocksdb_build_git_sha);
+  AddProperty(properties, rocksdb_build_git_tag);
+  AddProperty(properties, rocksdb_build_date);
+  return properties;
+}
+
+const std::unordered_map<std::string, std::string>& GetRocksBuildProperties() {
+  static std::unique_ptr<std::unordered_map<std::string, std::string>> props(LoadPropertiesSet());
+  return *props;
+}
+
+std::string GetRocksVersionAsString(bool with_patch) {
+  std::string version = ToString(ROCKSDB_MAJOR) + "." + ToString(ROCKSDB_MINOR);
+  if (with_patch) {
+    return version + "." + ToString(ROCKSDB_PATCH);
+  } else {
+    return version;
+  }
+}
+  
+std::string GetRocksBuildInfoAsString(const std::string& program, bool verbose) {
+  std::string info = program + " (RocksDB) " + GetRocksVersionAsString(true);
+  if (verbose) {
+    for (const auto& it : GetRocksBuildProperties()) {
+      info.append("\n    ");
+      info.append(it.first);
+      info.append(": ");
+      info.append(it.second);
+    }
+  }
+  return info;
+}
+} // namespace ROCKSDB_NAMESPACE
+

Builds/CMake/soci_patch.cmake

@@ -1,13 +1,39 @@
 # This patches unsigned-types.h in the soci official sources
 # so as to remove type range check exceptions that cause
 # us trouble when using boost::optional to select int values
+
+# Some versions of CMake erroneously patch external projects on every build.
+# If the patch makes no changes, skip it. This workaround can be
+# removed once we stop supporting vulnerable versions of CMake.
+# https://gitlab.kitware.com/cmake/cmake/-/issues/21086
 file (STRINGS include/soci/unsigned-types.h sourcecode)
+# Delete the .patched file if it exists, so it doesn't end up duplicated.
+# Trying to remove a file that does not exist is not a problem.
+file (REMOVE include/soci/unsigned-types.h.patched)
 foreach (line_ ${sourcecode})
   if (line_ MATCHES "^[ \\t]+throw[ ]+soci_error[ ]*\\([ ]*\"Value outside of allowed.+$")
     set (line_ "//${CMAKE_MATCH_0}")
   endif ()
   file (APPEND include/soci/unsigned-types.h.patched "${line_}\n")
 endforeach ()
+execute_process( COMMAND ${CMAKE_COMMAND} -E compare_files
+                 include/soci/unsigned-types.h include/soci/unsigned-types.h.patched
+                 RESULT_VARIABLE compare_result
+)
+if( compare_result EQUAL 0)
+  message(DEBUG "The soci source and patch files are identical. Make no changes.")
+  file (REMOVE include/soci/unsigned-types.h.patched)
+  return()
+endif()
 file (RENAME include/soci/unsigned-types.h include/soci/unsigned-types.h.orig)
 file (RENAME include/soci/unsigned-types.h.patched include/soci/unsigned-types.h)
+# also fix Boost.cmake so that it just returns when we override the Boost_FOUND var
+file (APPEND cmake/dependencies/Boost.cmake.patched "if (Boost_FOUND)\n")
+file (APPEND cmake/dependencies/Boost.cmake.patched "  return ()\n")
+file (APPEND cmake/dependencies/Boost.cmake.patched "endif ()\n")
+file (STRINGS cmake/dependencies/Boost.cmake sourcecode)
+foreach (line_ ${sourcecode})
+  file (APPEND cmake/dependencies/Boost.cmake.patched "${line_}\n")
+endforeach ()
+file (RENAME cmake/dependencies/Boost.cmake.patched cmake/dependencies/Boost.cmake)
 

Builds/Test.py

@@ -272,9 +272,17 @@ def run_cmake(directory, cmake_dir, args):
         args += ( '-DCMAKE_BUILD_TYPE=Debug', )
     if re.search('release', cmake_dir):
         args += ( '-DCMAKE_BUILD_TYPE=Release', )
-    if re.search('gcc', cmake_dir):
+    m = re.search('gcc(-[^.]*)', cmake_dir)
+    if m:
+        args += ( '-DCMAKE_C_COMPILER=' + m.group(0),
+          '-DCMAKE_CXX_COMPILER=g++' + m.group(1), )
+    elif re.search('gcc', cmake_dir):
         args += ( '-DCMAKE_C_COMPILER=gcc', '-DCMAKE_CXX_COMPILER=g++', )
-    if re.search('clang', cmake_dir):
+    m = re.search('clang(-[^.]*)', cmake_dir)
+    if m:
+        args += ( '-DCMAKE_C_COMPILER=' + m.group(0),
+          '-DCMAKE_CXX_COMPILER=clang++' + m.group(1), )
+    elif re.search('clang', cmake_dir):
         args += ( '-DCMAKE_C_COMPILER=clang', '-DCMAKE_CXX_COMPILER=clang++', )
 
     args += ( os.path.join('..', '..', '..'), )

Builds/VisualStudio2017/README.md

@@ -16,7 +16,7 @@ need these software components
 |-----------|-----------------------|
 | [Visual Studio 2017](README.md#install-visual-studio-2017)| 15.5.4 |
 | [Git for Windows](README.md#install-git-for-windows)| 2.16.1 |
-| [OpenSSL Library](README.md#install-openssl) | 1.0.2n |
+| [OpenSSL Library](README.md#install-openssl) | 1.1.1L |
 | [Boost library](README.md#build-boost) | 1.70.0 |
 | [CMake for Windows](README.md#optional-install-cmake-for-windows)* | 3.12 |
 
@@ -50,17 +50,19 @@ Windows is mandatory for running the unit tests.
 
 ### Install OpenSSL
 
-[Download OpenSSL.](http://slproweb.com/products/Win32OpenSSL.html) There will
-four `Win64` bit variants available, you want the non-light `v1.0` line. As of
-this writing, you **should** select
+[Download the latest version of
+OpenSSL.](http://slproweb.com/products/Win32OpenSSL.html) There will
+several `Win64` bit variants available, you want the non-light
+`v1.1` line. As of this writing, you **should** select
 
-* Win64 OpenSSL v1.0.2n. 
+* Win64 OpenSSL v1.1.1L
 
 and should **not** select
 
-* Win64 OpenSSL v1.0.2n light
-* Win64 OpenSSL v1.1.0g
-* Win64 OpenSSL v1.1.0g light
+* Anything with "Win32" in the name
+* Anything with "light" in the name
+* Anything with "EXPERIMENTAL" in the name
+* Anything in the 3.0 line - rippled won't currently build with this version.
 
 Run the installer, and choose an appropriate location for your OpenSSL
 installation. In this guide we use `C:\lib\OpenSSL-Win64` as the destination
@@ -146,7 +148,7 @@ If you receive an error about not having the "correct access rights" make sure
 you have Github ssh keys, as described above.
 
 For a stable release, choose the `master` branch or one of the tagged releases
-listed on [rippled's GitHub page](https://github.com/ripple/rippled/releases). 
+listed on [rippled's GitHub page](https://github.com/ripple/rippled/releases).
 
 ```
 git checkout master
@@ -175,7 +177,7 @@ To begin, simply:
    cloned rippled folder.
 2. Right-click on `CMakeLists.txt` in the **Solution Explorer - Folder View** to
    generate a `CMakeSettings.json` file. A sample settings file is provided
-   [here](/Builds/VisualStudio2017/CMakeSettings-example.json). Customize the 
+   [here](/Builds/VisualStudio2017/CMakeSettings-example.json). Customize the
    settings for `BOOST_ROOT`, `OPENSSL_ROOT` to match the install paths if they
    differ from those in the file.
 4. Select either the `x64-Release` or `x64-Debug` configuration from the
@@ -214,14 +216,14 @@ execute the following commands within your `rippled` cloned repository:
 ```
 mkdir build\cmake
 cd build\cmake
-cmake ..\.. -G"Visual Studio 15 2017 Win64" -DBOOST_ROOT="C:\lib\boost_1_70_0" -DOPENSSL_ROOT="C:\lib\OpenSSL-Win64"
+cmake ..\.. -G"Visual Studio 15 2017 Win64" -DBOOST_ROOT="C:\lib\boost_1_70_0" -DOPENSSL_ROOT="C:\lib\OpenSSL-Win64" -DCMAKE_GENERATOR_TOOLSET=host=x64
 ```
 Now launch Visual Studio 2017 and select **File | Open | Project/Solution**.
 Navigate to the `build\cmake` folder created above and select the `rippled.sln`
 file. You can then choose whether to build the `Debug` or `Release` solution
 configuration.
 
-The executable will be in 
+The executable will be in
 ```
 .\build\cmake\Release\rippled.exe
 ```
@@ -233,21 +235,24 @@ These paths are relative to your cloned git repository.
 
 # Unity/No-Unity Builds
 
-The rippled build system defaults to using [unity source files](http://onqtam.com/programming/2018-07-07-unity-builds/)
-to improve build times. In some cases it might be desirable to disable the unity build and compile
-individual translation units. Here is how you can switch to a "no-unity" build configuration:
+The rippled build system defaults to using
+[unity source files](http://onqtam.com/programming/2018-07-07-unity-builds/)
+to improve build times. In some cases it might be desirable to disable the
+unity build and compile individual translation units. Here is how you can
+switch to a "no-unity" build configuration:
 
 ## Visual Studio Integrated CMake
- 
-Edit your `CmakeSettings.json` (described above) by adding `-Dunity=OFF` to the `cmakeCommandArgs` entry 
-for each build configuration.
+
+Edit your `CmakeSettings.json` (described above) by adding `-Dunity=OFF`
+to the `cmakeCommandArgs` entry for each build configuration.
 
 ## Standalone CMake Builds
 
-When running cmake to generate the Visual Studio project files, add `-Dunity=OFF` to the 
-command line options passed to cmake.
+When running cmake to generate the Visual Studio project files, add
+`-Dunity=OFF` to the command line options passed to cmake.
 
-**Note:** you will need to re-run the cmake configuration step anytime you want to switch between unity/no-unity builds.
+**Note:** you will need to re-run the cmake configuration step anytime you
+want to switch between unity/no-unity builds.
 
 # Unit Test (Recommended)
 

Builds/containers/centos-builder/Dockerfile

@@ -8,28 +8,32 @@ COPY centos-builder/centos_setup.sh /tmp/
 COPY shared/build_deps.sh /tmp/
 COPY shared/install_cmake.sh /tmp/
 COPY centos-builder/extras.sh /tmp/
+COPY shared/install_boost.sh /tmp/
 RUN chmod +x /tmp/centos_setup.sh && \
     chmod +x /tmp/build_deps.sh && \
+    chmod +x /tmp/install_boost.sh && \
     chmod +x /tmp/install_cmake.sh && \
     chmod +x /tmp/extras.sh
 RUN /tmp/centos_setup.sh
-RUN /tmp/install_cmake.sh
+
+RUN /tmp/install_cmake.sh 3.16.1 /opt/local/cmake-3.16
+RUN ln -s /opt/local/cmake-3.16 /opt/local/cmake
 ENV PATH="/opt/local/cmake/bin:$PATH"
-RUN source scl_source enable devtoolset-6 python27 && \
+# also install min supported cmake for testing
+RUN if [ "${CI_USE}" = true ] ; then /tmp/install_cmake.sh 3.9.0 /opt/local/cmake-3.9; fi
+
+RUN source scl_source enable devtoolset-7 python27 && \
      /tmp/build_deps.sh
+ENV BOOST_ROOT="/opt/local/boost/_INSTALLED_"
 ENV PLANTUML_JAR="/opt/plantuml/plantuml.jar"
-ENV BOOST_ROOT="/opt/local/boost"
 ENV OPENSSL_ROOT="/opt/local/openssl"
 ENV GDB_ROOT="/opt/local/gdb"
-RUN source scl_source enable devtoolset-6 python27 && \
+RUN source scl_source enable devtoolset-7 python27 && \
      /tmp/extras.sh
 
 # prep files for package building
 RUN mkdir -m 777 -p /opt/rippled_bld/pkg
 WORKDIR /opt/rippled_bld/pkg
-
-COPY packaging/rpm/rippled.spec ./
-COPY shared/update_sources.sh ./
 RUN mkdir -m 777 ./rpmbuild
 RUN mkdir -m 777 ./rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
 

Builds/containers/centos-builder/centos_setup.sh

@@ -7,16 +7,12 @@ yum -y upgrade
 yum -y update
 yum -y install epel-release centos-release-scl
 yum -y install \
-    wget curl time gcc-c++ time yum-utils \
+    wget curl time gcc-c++ time yum-utils autoconf automake pkgconfig libtool \
     libstdc++-static rpm-build gnupg which make cmake \
-    devtoolset-4 devtoolset-4-gdb devtoolset-4-libasan-devel devtoolset-4-libtsan-devel devtoolset-4-libubsan-devel \
-    devtoolset-6 devtoolset-6-gdb devtoolset-6-libasan-devel devtoolset-6-libtsan-devel devtoolset-6-libubsan-devel \
     devtoolset-7 devtoolset-7-gdb devtoolset-7-libasan-devel devtoolset-7-libtsan-devel devtoolset-7-libubsan-devel \
-    llvm-toolset-7 llvm-toolset-7-runtime llvm-toolset-7-build llvm-toolset-7-clang \
-    llvm-toolset-7-clang-analyzer llvm-toolset-7-clang-devel llvm-toolset-7-clang-libs \
-    llvm-toolset-7-clang-tools-extra llvm-toolset-7-compiler-rt llvm-toolset-7-lldb \
-    llvm-toolset-7-lldb-devel llvm-toolset-7-python-lldb \
-    flex flex-devel bison bison-devel \
+    devtoolset-8 devtoolset-8-gdb devtoolset-8-binutils devtoolset-8-libstdc++-devel \
+    devtoolset-8-libasan-devel devtoolset-8-libtsan-devel devtoolset-8-libubsan-devel devtoolset-8-liblsan-devel \
+    flex flex-devel bison bison-devel parallel \
     ncurses ncurses-devel ncurses-libs graphviz graphviz-devel \
     lzip p7zip bzip2 bzip2-devel lzma-sdk lzma-sdk-devel xz-devel \
     zlib zlib-devel zlib-static texinfo openssl openssl-static \
@@ -26,10 +22,16 @@ yum -y install \
     python-devel python27-python-devel rh-python35-python-devel \
     python27 rh-python35 \
     ninja-build git svn \
-    protobuf protobuf-static protobuf-c-devel \
-    protobuf-compiler protobuf-devel \
-    swig ccache perl-Digest-MD5 python2-pip
+    swig perl-Digest-MD5 python2-pip
 
-# TODO need permanent link
-yum -y install ftp://ftp.pbone.net/mirror/archive.fedoraproject.org/fedora-secondary/updates/26/i386/Packages/p/python2-six-1.10.0-9.fc26.noarch.rpm
+if [ "${CI_USE}" = true ] ; then
+    # TODO need permanent link
+    yum -y install ftp://ftp.pbone.net/mirror/archive.fedoraproject.org/fedora-secondary/updates/26/i386/Packages/p/python2-six-1.10.0-9.fc26.noarch.rpm
 
+    yum -y install \
+        llvm-toolset-7 llvm-toolset-7-runtime llvm-toolset-7-build llvm-toolset-7-clang \
+        llvm-toolset-7-clang-analyzer llvm-toolset-7-clang-devel llvm-toolset-7-clang-libs \
+        llvm-toolset-7-clang-tools-extra llvm-toolset-7-compiler-rt llvm-toolset-7-lldb \
+        llvm-toolset-7-lldb-devel llvm-toolset-7-python-lldb
+
+fi

Builds/containers/centos-builder/extras.sh

@@ -3,50 +3,31 @@ set -ex
 
 if [ "${CI_USE}" = true ] ; then
     cd /tmp
-    wget https://ftp.gnu.org/gnu/gdb/gdb-8.2.tar.xz
-    tar xf gdb-8.2.tar.xz
-    cd gdb-8.2
-    ./configure CFLAGS="-w -O2" CXXFLAGS="-std=gnu++11 -g -O2 -w" --prefix=/opt/local/gdb-8.2
+    wget https://ftp.gnu.org/gnu/gdb/gdb-8.3.1.tar.xz
+    tar xf gdb-8.3.1.tar.xz
+    cd gdb-8.3
+    ./configure CFLAGS="-w -O2" CXXFLAGS="-std=gnu++11 -g -O2 -w" --prefix=/opt/local/gdb-8.3
     make -j$(nproc)
     make install
-    ln -s /opt/local/gdb-8.2 /opt/local/gdb
+    ln -s /opt/local/gdb-8.3 /opt/local/gdb
     cd ..
-    rm -f gdb-8.2.tar.xz
-    rm -rf gdb-8.2
+    rm -f gdb-8.3.tar.xz
+    rm -rf gdb-8.3
 
     # clang from source
-    RELEASE=tags/RELEASE_701/final
-    INSTALL=/opt/llvm-7.0.1/
-    mkdir -p /tmp/clang-src
-    cd /tmp/clang-src
-    TOPDIR=`pwd`
-    svn co -q http://llvm.org/svn/llvm-project/llvm/${RELEASE} llvm
-    cd ${TOPDIR}/llvm/tools
-    svn co -q http://llvm.org/svn/llvm-project/cfe/${RELEASE} clang
-    cd ${TOPDIR}/llvm/tools/clang/tools
-    svn co -q http://llvm.org/svn/llvm-project/clang-tools-extra/${RELEASE} extra
-    cd ${TOPDIR}/llvm/tools
-    svn co -q http://llvm.org/svn/llvm-project/lld/${RELEASE} lld
-    cd ${TOPDIR}/llvm/tools
-    svn co -q http://llvm.org/svn/llvm-project/polly/${RELEASE} polly
-    cd ${TOPDIR}/llvm/projects
-    svn co -q http://llvm.org/svn/llvm-project/compiler-rt/${RELEASE} compiler-rt
-    cd ${TOPDIR}/llvm/projects
-    svn co -q http://llvm.org/svn/llvm-project/openmp/${RELEASE} openmp
-    cd ${TOPDIR}/llvm/projects
-    svn co -q http://llvm.org/svn/llvm-project/libcxx/${RELEASE} libcxx
-    svn co -q http://llvm.org/svn/llvm-project/libcxxabi/${RELEASE} libcxxabi
-    cd ${TOPDIR}/llvm/projects
-    ## config/build
-    cd ${TOPDIR}
+    cd /tmp
+    git clone https://github.com/llvm/llvm-project.git
+    cd llvm-project
+    git checkout llvmorg-9.0.0
+    INSTALL=/opt/llvm-9/
     mkdir mybuilddir && cd mybuilddir
+    # TODO figure out necessary options
     cmake ../llvm -G Ninja \
       -DCMAKE_BUILD_TYPE=Release \
+      -DLLVM_ENABLE_PROJECTS='clang;clang-tools-extra;libcxx;libcxxabi;lldb;compiler-rt;lld;polly' \
       -DCMAKE_INSTALL_PREFIX=${INSTALL} \
-      -DLLVM_LIBDIR_SUFFIX=64 \
-      -DLLVM_ENABLE_EH=ON \
-      -DLLVM_ENABLE_RTTI=ON
+      -DLLVM_LIBDIR_SUFFIX=64
     cmake --build . --parallel --target install
     cd /tmp
-    rm -rf clang-src
+    rm -rf llvm-project
 fi

Builds/containers/gitlab-ci/build_container.sh

@@ -9,24 +9,15 @@ else
     echo "invalid package type"
     exit 1
 fi
-if docker pull "${ARTIFACTORY_HUB}/${container_name}:${CI_COMMIT_SHA}"; then
-    echo "${pkgtype} container for ${CI_COMMIT_SHA} already exists" \
-        "- skipping container build!"
-    exit 0
-else
-    echo "no existing ${pkgtype} container for this branch - searching history."
-    for CID_PREV in $(git log --pretty=%H -n30) ; do
-        if docker pull "${ARTIFACTORY_HUB}/${container_name}:${CID_PREV}"; then
-            echo "found container for previous commit ${CID_PREV}" \
-                "- using as cache."
-            docker tag \
-               "${ARTIFACTORY_HUB}/${container_name}:${CID_PREV}" \
-               "${container_name}:${CID_PREV}"
-            CMAKE_EXTRA="-D${pkgtype}_cache_from=${container_name}:${CID_PREV}"
-            break
-        fi
-    done
+
+if docker pull "${ARTIFACTORY_HUB}/${container_name}:latest_${CI_COMMIT_REF_SLUG}"; then
+    echo "found container for latest - using as cache."
+    docker tag \
+       "${ARTIFACTORY_HUB}/${container_name}:latest_${CI_COMMIT_REF_SLUG}" \
+       "${container_name}:latest_${CI_COMMIT_REF_SLUG}"
+    CMAKE_EXTRA="-D${pkgtype}_cache_from=${container_name}:latest_${CI_COMMIT_REF_SLUG}"
 fi
+
 cmake --version
 test -d build && rm -rf build
 mkdir -p build/container && cd build/container
@@ -34,8 +25,4 @@ eval time \
     cmake -Dpackages_only=ON -DCMAKE_VERBOSE_MAKEFILE=ON ${CMAKE_EXTRA} \
     -G Ninja ../..
 time cmake --build . --target "${pkgtype}_container" -- -v
-docker tag \
-    "${container_name}:${CI_COMMIT_SHA}" \
-    "${ARTIFACTORY_HUB}/${container_name}:${CI_COMMIT_SHA}"
-time docker push "${ARTIFACTORY_HUB}/${container_name}:${CI_COMMIT_SHA}"
 

Builds/containers/gitlab-ci/build_package.sh

@@ -2,22 +2,26 @@
 set -ex
 pkgtype=$1
 if [ "${pkgtype}" = "rpm" ] ; then
-    container_name="${RPM_CONTAINER_NAME}"
+    container_name="${RPM_CONTAINER_FULLNAME}"
+    container_tag="${RPM_CONTAINER_TAG}"
 elif [ "${pkgtype}" = "dpkg" ] ; then
-    container_name="${DPKG_CONTAINER_NAME}"
+    container_name="${DPKG_CONTAINER_FULLNAME}"
+    container_tag="${DPKG_CONTAINER_TAG}"
 else
     echo "invalid package type"
     exit 1
 fi
-time docker pull "${ARTIFACTORY_HUB}/${container_name}:${CI_COMMIT_SHA}"
+time docker pull "${ARTIFACTORY_HUB}/${container_name}"
 docker tag \
-  "${ARTIFACTORY_HUB}/${container_name}:${CI_COMMIT_SHA}" \
-  "${container_name}:${CI_COMMIT_SHA}"
+  "${ARTIFACTORY_HUB}/${container_name}" \
+  "${container_name}"
 docker images
 test -d build && rm -rf build
 mkdir -p build/${pkgtype} && cd build/${pkgtype}
 time cmake \
-  -Dpackages_only=ON -Dhave_package_container=ON -DCMAKE_VERBOSE_MAKEFILE=ON \
+  -Dpackages_only=ON \
+  -Dcontainer_label="${container_tag}" \
+  -Dhave_package_container=ON \
+  -DCMAKE_VERBOSE_MAKEFILE=OFF \
   -G Ninja ../..
-time cmake --build . --target ${pkgtype} -- -v
-
+time cmake --build . --target ${pkgtype}

Builds/containers/gitlab-ci/docker_alpine_setup.sh

@@ -5,6 +5,7 @@ set -ex
 echo $(nproc)
 docker login -u rippled \
     -p ${ARTIFACTORY_DEPLOY_KEY_RIPPLED} ${ARTIFACTORY_HUB}
+apk add --update py-pip
 apk add \
     bash util-linux coreutils binutils grep \
     make ninja cmake build-base gcc g++ abuild git \

Builds/containers/gitlab-ci/pkgbuild.yml

@@ -8,19 +8,26 @@
 # NOTE: these are sensible defaults for Ripple pipelines. These
 # can be overridden by project or group variables as needed.
 variables:
+  # these containers are built manually using the rippled
+  # cmake build (container targets) and tagged/pushed so they
+  # can be used here
+  RPM_CONTAINER_TAG: "2020-02-10"
   RPM_CONTAINER_NAME: "rippled-rpm-builder"
+  RPM_CONTAINER_FULLNAME: "${RPM_CONTAINER_NAME}:${RPM_CONTAINER_TAG}"
+  DPKG_CONTAINER_TAG: "2020-02-10"
   DPKG_CONTAINER_NAME: "rippled-dpkg-builder"
+  DPKG_CONTAINER_FULLNAME: "${DPKG_CONTAINER_NAME}:${DPKG_CONTAINER_TAG}"
   ARTIFACTORY_HOST: "artifactory.ops.ripple.com"
   ARTIFACTORY_HUB: "${ARTIFACTORY_HOST}:6555"
-  GIT_SIGN_PUBKEYS_URL: "https://gitlab.ops.ripple.com/snippets/11/raw"
+  GIT_SIGN_PUBKEYS_URL: "https://gitlab.ops.ripple.com/xrpledger/rippled-packages/snippets/49/raw"
   PUBLIC_REPO_ROOT: "https://repos.ripple.com/repos"
   # also need to define this variable ONLY for the primary
   # build/publish pipeline on the mainline repo:
   #   IS_PRIMARY_REPO = "true"
 
 stages:
-  - build_containers
   - build_packages
+  - sign_packages
   - smoketest
   - verify_sig
   - tag_images
@@ -30,20 +37,23 @@ stages:
   - push_to_prod
   - verify_from_prod
   - get_final_hashes
+  - build_containers
 
 .dind_template: &dind_param
   before_script:
     - . ./Builds/containers/gitlab-ci/docker_alpine_setup.sh
   variables:
     docker_driver: overlay2
+    DOCKER_TLS_CERTDIR: ""
   image:
-    name: docker:latest
+    name: artifactory.ops.ripple.com/docker:latest
   services:
     # workaround for TLS issues - consider going back
     # back to unversioned `dind` when issues are resolved
-    - docker:18-dind
+    - name: artifactory.ops.ripple.com/docker:stable-dind
+      alias: docker
   tags:
-    - docker-4xlarge
+    - 4xlarge
 
 .only_primary_template: &only_primary
   only:
@@ -64,35 +74,6 @@ stages:
   script:
     - . ./Builds/containers/gitlab-ci/smoketest.sh repo
 
-#########################################################################
-##                                                                     ##
-##  stage: build_containers                                            ##
-##                                                                     ##
-##  build containers from docker definitions. These containers are     ##
-##  subsequently used to build the rpm and deb packages.               ##
-##                                                                     ##
-#########################################################################
-
-build_centos_container:
-  stage: build_containers
-  <<: *dind_param
-  cache:
-    key: containers
-    paths:
-      - .nih_c
-  script:
-    - . ./Builds/containers/gitlab-ci/build_container.sh rpm
-
-build_ubuntu_container:
-  stage: build_containers
-  <<: *dind_param
-  cache:
-    key: containers
-    paths:
-      - .nih_c
-  script:
-    - . ./Builds/containers/gitlab-ci/build_container.sh dpkg
-
 #########################################################################
 ##                                                                     ##
 ##  stage: build_packages                                              ##
@@ -103,34 +84,89 @@ build_ubuntu_container:
 
 rpm_build:
   stage: build_packages
-  dependencies:
-    - build_centos_container
   <<: *dind_param
   artifacts:
     paths:
       - build/rpm/packages/
-  cache:
-    key: rpm
-    paths:
-      - .nih_c/pkgbuild
   script:
     - . ./Builds/containers/gitlab-ci/build_package.sh rpm
 
 dpkg_build:
   stage: build_packages
-  dependencies:
-    - build_ubuntu_container
   <<: *dind_param
   artifacts:
     paths:
       - build/dpkg/packages/
-  cache:
-    key: dpkg
-    paths:
-      - .nih_c/pkgbuild
   script:
     - . ./Builds/containers/gitlab-ci/build_package.sh dpkg
 
+#########################################################################
+##                                                                     ##
+##  stage: sign_packages                                               ##
+##                                                                     ##
+##  build packages using containers from previous stage.               ##
+##                                                                     ##
+#########################################################################
+
+rpm_sign:
+  stage: sign_packages
+  dependencies:
+    - rpm_build
+  image:
+    name: artifactory.ops.ripple.com/centos:7
+  <<: *only_primary
+  before_script:
+  - |
+    # Make sure GnuPG is installed
+    yum -y install gnupg rpm-sign
+    # checking GPG signing support
+    if [ -n "$GPG_KEY_B64" ]; then
+      echo "$GPG_KEY_B64"| base64 -d | gpg --batch --no-tty --allow-secret-key-import --import -
+      unset GPG_KEY_B64
+      export GPG_PASSPHRASE=$(echo $GPG_KEY_PASS_B64 | base64 -di)
+      unset GPG_KEY_PASS_B64
+      export GPG_KEYID=$(gpg --with-colon --list-secret-keys | head -n1 | cut -d : -f 5)
+    else
+      echo -e "\033[0;31m****** GPG signing disabled ******\033[0m"
+      exit 1
+    fi
+  artifacts:
+    paths:
+      - build/rpm/packages/
+  script:
+    - ls -alh build/rpm/packages
+    - . ./Builds/containers/gitlab-ci/sign_package.sh rpm
+
+dpkg_sign:
+  stage: sign_packages
+  dependencies:
+    - dpkg_build
+  image:
+    name: artifactory.ops.ripple.com/ubuntu:18.04
+  <<: *only_primary
+  before_script:
+  - |
+    # make sure we have GnuPG
+    apt update
+    apt install -y gpg dpkg-sig
+    # checking GPG signing support
+    if [ -n "$GPG_KEY_B64" ]; then
+      echo "$GPG_KEY_B64"| base64 -d | gpg --batch --no-tty --allow-secret-key-import --import -
+      unset GPG_KEY_B64
+      export GPG_PASSPHRASE=$(echo $GPG_KEY_PASS_B64 | base64 -di)
+      unset GPG_KEY_PASS_B64
+      export GPG_KEYID=$(gpg --with-colon --list-secret-keys | head -n1 | cut -d : -f 5)
+    else
+      echo -e "\033[0;31m****** GPG signing disabled ******\033[0m"
+      exit 1
+    fi
+  artifacts:
+    paths:
+      - build/dpkg/packages/
+  script:
+    - ls -alh build/dpkg/packages
+    - . ./Builds/containers/gitlab-ci/sign_package.sh dpkg
+
 #########################################################################
 ##                                                                     ##
 ##  stage: smoketest                                                   ##
@@ -143,67 +179,96 @@ centos_7_smoketest:
   stage: smoketest
   dependencies:
     - rpm_build
+    - rpm_sign
   image:
-    name: centos:7
+    name: artifactory.ops.ripple.com/centos:7
   <<: *run_local_smoketest
 
-fedora_29_smoketest:
+# TODO: Remove "allow_failure" when tests fixed
+rocky_8_smoketest:
   stage: smoketest
   dependencies:
     - rpm_build
+    - rpm_sign
   image:
-    name: fedora:29
+    name: rockylinux/rockylinux:8
   <<: *run_local_smoketest
+  allow_failure: true
 
-fedora_28_smoketest:
+fedora_34_smoketest:
   stage: smoketest
   dependencies:
     - rpm_build
+    - rpm_sign
   image:
-    name: fedora:28
+    name: artifactory.ops.ripple.com/fedora:34
   <<: *run_local_smoketest
+  allow_failure: true
 
-fedora_27_smoketest:
+fedora_35_smoketest:
   stage: smoketest
   dependencies:
     - rpm_build
+    - rpm_sign
   image:
-    name: fedora:27
-  <<: *run_local_smoketest
-
-## this one is not LTS, but we
-## get some extra coverage by including it
-## consider dropping it when 20.04 is ready
-ubuntu_19_smoketest:
-  stage: smoketest
-  dependencies:
-    - dpkg_build
-  image:
-    name: ubuntu:19.04
+    name: artifactory.ops.ripple.com/fedora:35
   <<: *run_local_smoketest
+  allow_failure: true
 
 ubuntu_18_smoketest:
   stage: smoketest
   dependencies:
     - dpkg_build
+    - dpkg_sign
   image:
-    name: ubuntu:18.04
+    name: artifactory.ops.ripple.com/ubuntu:18.04
   <<: *run_local_smoketest
 
-ubuntu_16_smoketest:
+ubuntu_20_smoketest:
   stage: smoketest
   dependencies:
     - dpkg_build
+    - dpkg_sign
   image:
-    name: ubuntu:16.04
+    name: artifactory.ops.ripple.com/ubuntu:20.04
   <<: *run_local_smoketest
 
+# TODO: remove "allow_failure" when 22.04 released in 4/2022...
+ubuntu_22_smoketest:
+  stage: smoketest
+  dependencies:
+    - dpkg_build
+    - dpkg_sign
+  image:
+    name: artifactory.ops.ripple.com/ubuntu:22.04
+  <<: *run_local_smoketest
+  allow_failure: true
+
 debian_9_smoketest:
   stage: smoketest
   dependencies:
     - dpkg_build
+    - dpkg_sign
   image:
-    name: debian:9
+    name: artifactory.ops.ripple.com/debian:9
+  <<: *run_local_smoketest
+
+debian_10_smoketest:
+  stage: smoketest
+  dependencies:
+    - dpkg_build
+    - dpkg_sign
+  image:
+    name: artifactory.ops.ripple.com/debian:10
+  <<: *run_local_smoketest
+
+debian_11_smoketest:
+  stage: smoketest
+  dependencies:
+    - dpkg_build
+    - dpkg_sign
+  image:
+    name: artifactory.ops.ripple.com/debian:11
   <<: *run_local_smoketest
 
 #########################################################################
@@ -221,7 +286,7 @@ debian_9_smoketest:
 verify_head_signed:
   stage: verify_sig
   image:
-    name: ubuntu:latest
+    name: artifactory.ops.ripple.com/ubuntu:latest
   <<: *only_primary
   script:
     - . ./Builds/containers/gitlab-ci/verify_head_commit.sh
@@ -239,17 +304,19 @@ tag_bld_images:
   stage: tag_images
   variables:
     docker_driver: overlay2
+    DOCKER_TLS_CERTDIR: ""
   image:
-    name: docker:latest
+    name: artifactory.ops.ripple.com/docker:latest
   services:
     # workaround for TLS issues - consider going back
     # back to unversioned `dind` when issues are resolved
-    - docker:18-dind
+    - name: artifactory.ops.ripple.com/docker:stable-dind
+      alias: docker
   tags:
-    - docker-large
+    - large
   dependencies:
-    - rpm_build
-    - dpkg_build
+    - rpm_sign
+    - dpkg_sign
   <<: *only_primary
   script:
     - . ./Builds/containers/gitlab-ci/tag_docker_image.sh
@@ -269,13 +336,13 @@ push_test:
     DEB_REPO: "rippled-deb-test-mirror"
     RPM_REPO: "rippled-rpm-test-mirror"
   image:
-    name: alpine:latest
+    name: artifactory.ops.ripple.com/alpine:latest
   artifacts:
     paths:
       - files.info
   dependencies:
-    - rpm_build
-    - dpkg_build
+    - rpm_sign
+    - dpkg_sign
   <<: *only_primary
   script:
     - . ./Builds/containers/gitlab-ci/push_to_artifactory.sh "PUT" "."
@@ -294,56 +361,47 @@ centos_7_verify_repo_test:
   variables:
     RPM_REPO: "rippled-rpm-test-mirror"
   image:
-    name: centos:7
+    name: artifactory.ops.ripple.com/centos:7
   dependencies:
-    - rpm_build
+    - rpm_sign
   <<: *only_primary
   <<: *run_repo_smoketest
 
-fedora_29_verify_repo_test:
+rocky_8_verify_repo_test:
   stage: verify_from_test
   variables:
     RPM_REPO: "rippled-rpm-test-mirror"
   image:
-    name: fedora:29
+    name: rockylinux/rockylinux:8
   dependencies:
-    - rpm_build
+    - rpm_sign
   <<: *only_primary
   <<: *run_repo_smoketest
+  allow_failure: true
 
-fedora_28_verify_repo_test:
+fedora_34_verify_repo_test:
   stage: verify_from_test
   variables:
     RPM_REPO: "rippled-rpm-test-mirror"
   image:
-    name: fedora:28
+    name: artifactory.ops.ripple.com/fedora:34
   dependencies:
-    - rpm_build
+    - rpm_sign
   <<: *only_primary
   <<: *run_repo_smoketest
+  allow_failure: true
 
-fedora_27_verify_repo_test:
+fedora_35_verify_repo_test:
   stage: verify_from_test
   variables:
     RPM_REPO: "rippled-rpm-test-mirror"
   image:
-    name: fedora:27
+    name: artifactory.ops.ripple.com/fedora:35
   dependencies:
-    - rpm_build
-  <<: *only_primary
-  <<: *run_repo_smoketest
-
-ubuntu_19_verify_repo_test:
-  stage: verify_from_test
-  variables:
-    DISTRO: "disco"
-    DEB_REPO: "rippled-deb-test-mirror"
-  image:
-    name: ubuntu:19.04
-  dependencies:
-    - dpkg_build
+    - rpm_sign
   <<: *only_primary
   <<: *run_repo_smoketest
+  allow_failure: true
 
 ubuntu_18_verify_repo_test:
   stage: verify_from_test
@@ -351,33 +409,71 @@ ubuntu_18_verify_repo_test:
     DISTRO: "bionic"
     DEB_REPO: "rippled-deb-test-mirror"
   image:
-    name: ubuntu:18.04
+    name: artifactory.ops.ripple.com/ubuntu:18.04
   dependencies:
-    - dpkg_build
+    - dpkg_sign
   <<: *only_primary
   <<: *run_repo_smoketest
 
-ubuntu_16_verify_repo_test:
+ubuntu_20_verify_repo_test:
   stage: verify_from_test
   variables:
-    DISTRO: "xenial"
+    DISTRO: "focal"
     DEB_REPO: "rippled-deb-test-mirror"
   image:
-    name: ubuntu:16.04
+    name: artifactory.ops.ripple.com/ubuntu:20.04
   dependencies:
-    - dpkg_build
+    - dpkg_sign
   <<: *only_primary
   <<: *run_repo_smoketest
 
+# TODO: remove "allow_failure" when 22.04 released in 4/2022...
+ubuntu_22_verify_repo_test:
+  stage: verify_from_test
+  variables:
+    DISTRO: "jammy"
+    DEB_REPO: "rippled-deb-test-mirror"
+  image:
+    name: artifactory.ops.ripple.com/ubuntu:22.04
+  dependencies:
+    - dpkg_sign
+  <<: *only_primary
+  <<: *run_repo_smoketest
+  allow_failure: true
+
 debian_9_verify_repo_test:
   stage: verify_from_test
   variables:
     DISTRO: "stretch"
     DEB_REPO: "rippled-deb-test-mirror"
   image:
-    name: debian:9
+    name: artifactory.ops.ripple.com/debian:9
   dependencies:
-    - dpkg_build
+    - dpkg_sign
+  <<: *only_primary
+  <<: *run_repo_smoketest
+
+debian_10_verify_repo_test:
+  stage: verify_from_test
+  variables:
+    DISTRO: "buster"
+    DEB_REPO: "rippled-deb-test-mirror"
+  image:
+    name: artifactory.ops.ripple.com/debian:10
+  dependencies:
+    - dpkg_sign
+  <<: *only_primary
+  <<: *run_repo_smoketest
+
+debian_11_verify_repo_test:
+  stage: verify_from_test
+  variables:
+    DISTRO: "bullseye"
+    DEB_REPO: "rippled-deb-test-mirror"
+  image:
+    name: artifactory.ops.ripple.com/debian:11
+  dependencies:
+    - dpkg_sign
   <<: *only_primary
   <<: *run_repo_smoketest
 
@@ -393,7 +489,7 @@ debian_9_verify_repo_test:
 wait_before_push_prod:
   stage: wait_approval_prod
   image:
-    name: alpine:latest
+    name: artifactory.ops.ripple.com/alpine:latest
   <<: *only_primary
   script:
     - echo "proceeding to next stage"
@@ -414,14 +510,14 @@ push_prod:
     DEB_REPO: "rippled-deb"
     RPM_REPO: "rippled-rpm"
   image:
-    name: alpine:latest
+    name: artifactory.ops.ripple.com/alpine:latest
   stage: push_to_prod
   artifacts:
     paths:
       - files.info
   dependencies:
-    - rpm_build
-    - dpkg_build
+    - rpm_sign
+    - dpkg_sign
   <<: *only_primary
   script:
     - . ./Builds/containers/gitlab-ci/push_to_artifactory.sh "PUT" "."
@@ -440,56 +536,47 @@ centos_7_verify_repo_prod:
   variables:
     RPM_REPO: "rippled-rpm"
   image:
-    name: centos:7
+    name: artifactory.ops.ripple.com/centos:7
   dependencies:
-    - rpm_build
+    - rpm_sign
   <<: *only_primary
   <<: *run_repo_smoketest
 
-fedora_29_verify_repo_prod:
+rocky_8_verify_repo_test:
+  stage: verify_from_test
+  variables:
+    RPM_REPO: "rippled-rpm-test-mirror"
+  image:
+    name: rockylinux/rockylinux:8
+  dependencies:
+    - rpm_sign
+  <<: *only_primary
+  <<: *run_repo_smoketest
+  allow_failure: true
+
+fedora_34_verify_repo_prod:
   stage: verify_from_prod
   variables:
     RPM_REPO: "rippled-rpm"
   image:
-    name: fedora:29
+    name: artifactory.ops.ripple.com/fedora:34
   dependencies:
-    - rpm_build
+    - rpm_sign
   <<: *only_primary
   <<: *run_repo_smoketest
+  allow_failure: true
 
-fedora_28_verify_repo_prod:
+fedora_35_verify_repo_prod:
   stage: verify_from_prod
   variables:
     RPM_REPO: "rippled-rpm"
   image:
-    name: fedora:28
+    name: artifactory.ops.ripple.com/fedora:35
   dependencies:
-    - rpm_build
-  <<: *only_primary
-  <<: *run_repo_smoketest
-
-fedora_27_verify_repo_prod:
-  stage: verify_from_prod
-  variables:
-    RPM_REPO: "rippled-rpm"
-  image:
-    name: fedora:27
-  dependencies:
-    - rpm_build
-  <<: *only_primary
-  <<: *run_repo_smoketest
-
-ubuntu_19_verify_repo_prod:
-  stage: verify_from_prod
-  variables:
-    DISTRO: "disco"
-    DEB_REPO: "rippled-deb"
-  image:
-    name: ubuntu:19.04
-  dependencies:
-    - dpkg_build
+    - rpm_sign
   <<: *only_primary
   <<: *run_repo_smoketest
+  allow_failure: true
 
 ubuntu_18_verify_repo_prod:
   stage: verify_from_prod
@@ -497,33 +584,71 @@ ubuntu_18_verify_repo_prod:
     DISTRO: "bionic"
     DEB_REPO: "rippled-deb"
   image:
-    name: ubuntu:18.04
+    name: artifactory.ops.ripple.com/ubuntu:18.04
   dependencies:
-    - dpkg_build
+    - dpkg_sign
   <<: *only_primary
   <<: *run_repo_smoketest
 
-ubuntu_16_verify_repo_prod:
+ubuntu_20_verify_repo_prod:
   stage: verify_from_prod
   variables:
-    DISTRO: "xenial"
+    DISTRO: "focal"
     DEB_REPO: "rippled-deb"
   image:
-    name: ubuntu:16.04
+    name: artifactory.ops.ripple.com/ubuntu:20.04
   dependencies:
-    - dpkg_build
+    - dpkg_sign
   <<: *only_primary
   <<: *run_repo_smoketest
 
+# TODO: remove "allow_failure" when 22.04 released in 4/2022...
+ubuntu_22_verify_repo_prod:
+  stage: verify_from_prod
+  variables:
+    DISTRO: "jammy"
+    DEB_REPO: "rippled-deb"
+  image:
+    name: artifactory.ops.ripple.com/ubuntu:22.04
+  dependencies:
+    - dpkg_sign
+  <<: *only_primary
+  <<: *run_repo_smoketest
+  allow_failure: true
+
 debian_9_verify_repo_prod:
   stage: verify_from_prod
   variables:
     DISTRO: "stretch"
     DEB_REPO: "rippled-deb"
   image:
-    name: debian:9
+    name: artifactory.ops.ripple.com/debian:9
   dependencies:
-    - dpkg_build
+    - dpkg_sign
+  <<: *only_primary
+  <<: *run_repo_smoketest
+
+debian_10_verify_repo_prod:
+  stage: verify_from_prod
+  variables:
+    DISTRO: "buster"
+    DEB_REPO: "rippled-deb"
+  image:
+    name: artifactory.ops.ripple.com/debian:10
+  dependencies:
+    - dpkg_sign
+  <<: *only_primary
+  <<: *run_repo_smoketest
+
+debian_11_verify_repo_prod:
+  stage: verify_from_prod
+  variables:
+    DISTRO: "bullseye"
+    DEB_REPO: "rippled-deb"
+  image:
+    name: artifactory.ops.ripple.com/debian:11
+  dependencies:
+    - dpkg_sign
   <<: *only_primary
   <<: *run_repo_smoketest
 
@@ -541,15 +666,38 @@ get_prod_hashes:
     DEB_REPO: "rippled-deb"
     RPM_REPO: "rippled-rpm"
   image:
-    name: alpine:latest
+    name: artifactory.ops.ripple.com/alpine:latest
   stage: get_final_hashes
   artifacts:
     paths:
       - files.info
   dependencies:
-    - rpm_build
-    - dpkg_build
+    - rpm_sign
+    - dpkg_sign
   <<: *only_primary
   script:
     - . ./Builds/containers/gitlab-ci/push_to_artifactory.sh "GET" ".checksums"
 
+#########################################################################
+##                                                                     ##
+##  stage: build_containers                                            ##
+##                                                                     ##
+##  build containers from docker definitions. These containers are NOT ##
+##  used for the package build. This step is only used to ensure that  ##
+##  the package build targets and files are still working properly.    ##
+##                                                                     ##
+#########################################################################
+
+build_centos_container:
+  stage: build_containers
+  <<: *dind_param
+  script:
+    - . ./Builds/containers/gitlab-ci/build_container.sh rpm
+  allow_failure: true
+
+build_ubuntu_container:
+  stage: build_containers
+  <<: *dind_param
+  script:
+    - . ./Builds/containers/gitlab-ci/build_container.sh dpkg
+  allow_failure: true

Builds/containers/gitlab-ci/push_to_artifactory.sh

@@ -19,7 +19,7 @@ RIPPLED_DBG_PKG=$(ls rippled-dbgsym_*.deb)
 # TODO - where to upload src tgz?
 RIPPLED_SRC=$(ls rippled_*.orig.tar.gz)
 DEB_MATRIX=";deb.component=${COMPONENT};deb.architecture=amd64"
-for dist in stretch buster xenial bionic disco ; do
+for dist in stretch buster bullseye bionic focal jammy; do
     DEB_MATRIX="${DEB_MATRIX};deb.distribution=${dist}"
 done
 echo "{ \"debs\": {" > "${TOPDIR}/files.info"
@@ -88,4 +88,3 @@ JSON
 )
     curl ${SLACK_NOTIFY_URL} --data-urlencode "${CONTENT}"
 fi
-

Builds/containers/gitlab-ci/sign_package.sh

@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+set -eo pipefail
+
+sign_dpkg() {
+  if [ -n "${GPG_KEYID}" ]; then
+    dpkg-sig \
+      -g "--no-tty --digest-algo 'sha512' --passphrase '${GPG_PASSPHRASE}' --pinentry-mode=loopback" \
+			-k "${GPG_KEYID}" \
+			--sign builder \
+			"build/dpkg/packages/*.deb"
+	fi
+}
+
+sign_rpm() {
+  if [ -n "${GPG_KEYID}" ] ; then
+    find build/rpm/packages -name "*.rpm" -exec bash -c '
+	echo "yes" | setsid rpm \
+			--define "_gpg_name ${GPG_KEYID}" \
+			--define "_signature gpg" \
+			--define "__gpg_check_password_cmd /bin/true" \
+			--define "__gpg_sign_cmd %{__gpg} gpg --batch --no-armor --digest-algo 'sha512' --passphrase '${GPG_PASSPHRASE}' --no-secmem-warning -u '%{_gpg_name}' --sign --detach-sign --output %{__signature_filename} %{__plaintext_filename}" \
+			--addsign '{} \;
+	fi
+}
+
+case "${1}" in
+    dpkg)
+        sign_dpkg
+        ;;
+    rpm)
+        sign_rpm
+        ;;
+    *)
+        echo "Usage: ${0} (dpkg|rpm)"
+        ;;
+esac
+

Builds/containers/gitlab-ci/smoketest.sh

@@ -16,7 +16,7 @@ case ${ID} in
     ubuntu|debian)
         pkgtype="dpkg"
         ;;
-    fedora|centos|rhel|scientific)
+    fedora|centos|rhel|scientific|rocky)
         pkgtype="rpm"
         ;;
     *)
@@ -51,7 +51,7 @@ if [ "${pkgtype}" = "dpkg" ] ; then
     elif [ "${install_from}" = "local" ] ; then
         # cached pkg install
         updateWithRetry
-        apt-get -y install libprotobuf-dev libssl-dev
+        apt-get -y install libprotobuf-dev libprotoc-dev protobuf-compiler libssl-dev
         rm -f build/dpkg/packages/rippled-dbgsym*.*
         dpkg --no-debsig -i build/dpkg/packages/*.deb
     else
@@ -76,7 +76,12 @@ else
         yum -y install ${rpm_version_release}
     elif [ "${install_from}" = "local" ] ; then
         # cached pkg install
-        yum install -y yum-utils openssl-static zlib-static
+        pkgs=("yum-utils openssl-static zlib-static")
+        if [ "$ID" = "rocky" ]; then
+            sed -i 's/enabled=0/enabled=1/g' /etc/yum.repos.d/Rocky-PowerTools.repo
+            pkgs="${pkgs[@]/openssl-static}"
+        fi
+        yum install -y $pkgs
         rm -f build/rpm/packages/rippled-debug*.rpm
         rm -f build/rpm/packages/*.src.rpm
         rpm -i build/rpm/packages/*.rpm
@@ -95,5 +100,3 @@ fi
 # run unit tests
 /opt/ripple/bin/rippled --unittest --unittest-jobs $(nproc)
 /opt/ripple/bin/validator-keys --unittest
-
-

Builds/containers/gitlab-ci/tag_docker_image.sh

@@ -4,17 +4,19 @@ docker login -u rippled \
     -p ${ARTIFACTORY_DEPLOY_KEY_RIPPLED} "${ARTIFACTORY_HUB}"
 # this gives us rippled_version :
 source build/rpm/packages/build_vars
-docker pull "${ARTIFACTORY_HUB}/${RPM_CONTAINER_NAME}:${CI_COMMIT_SHA}"
-docker pull "${ARTIFACTORY_HUB}/${DPKG_CONTAINER_NAME}:${CI_COMMIT_SHA}"
+docker pull "${ARTIFACTORY_HUB}/${RPM_CONTAINER_FULLNAME}"
+docker pull "${ARTIFACTORY_HUB}/${DPKG_CONTAINER_FULLNAME}"
 # tag/push two labels...one using the current rippled version and one just using "latest"
 for label in ${rippled_version} latest ; do
     docker tag \
-        "${ARTIFACTORY_HUB}/${RPM_CONTAINER_NAME}:${CI_COMMIT_SHA}" \
+        "${ARTIFACTORY_HUB}/${RPM_CONTAINER_FULLNAME}" \
+        "${ARTIFACTORY_HUB}/${RPM_CONTAINER_NAME}:${label}_${CI_COMMIT_REF_SLUG}"
+    docker push \
         "${ARTIFACTORY_HUB}/${RPM_CONTAINER_NAME}:${label}_${CI_COMMIT_REF_SLUG}"
     docker tag \
-        "${ARTIFACTORY_HUB}/${DPKG_CONTAINER_NAME}:${CI_COMMIT_SHA}" \
+        "${ARTIFACTORY_HUB}/${DPKG_CONTAINER_FULLNAME}" \
+        "${ARTIFACTORY_HUB}/${DPKG_CONTAINER_NAME}:${label}_${CI_COMMIT_REF_SLUG}"
+    docker push \
         "${ARTIFACTORY_HUB}/${DPKG_CONTAINER_NAME}:${label}_${CI_COMMIT_REF_SLUG}"
-    docker push "${ARTIFACTORY_HUB}/${RPM_CONTAINER_NAME}"
-    docker push "${ARTIFACTORY_HUB}/${DPKG_CONTAINER_NAME}"
 done
 

Builds/containers/gitlab-ci/verify_head_commit.sh

@@ -1,6 +1,7 @@
 #!/usr/bin/env sh
 set -ex
 apt -y update
+DEBIAN_FRONTEND="noninteractive" apt-get -y install tzdata
 apt -y install software-properties-common curl git gnupg
 curl -sk -o rippled-pubkeys.txt "${GIT_SIGN_PUBKEYS_URL}"
 gpg --import rippled-pubkeys.txt
@@ -8,8 +9,8 @@ if git verify-commit HEAD; then
     echo "git commit signature check passed"
 else
     echo "git commit signature check failed"
-    git log -n 5 --color
-        --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an> [%G?]%Creset'
+    git log -n 5 --color \
+        --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an> [%G?]%Creset' \
         --abbrev-commit
     exit 1
 fi

Builds/containers/packaging/dpkg/build_dpkg.sh

@@ -1,6 +1,11 @@
 #!/usr/bin/env bash
 set -ex
 
+# make sure pkg source files are up to date with repo
+cd /opt/rippled_bld/pkg
+cp -fpru rippled/Builds/containers/packaging/dpkg/debian/. debian/
+cp -fpu rippled/Builds/containers/shared/rippled.service debian/
+cp -fpu rippled/Builds/containers/shared/update_sources.sh .
 source update_sources.sh
 
 # Build the dpkg
@@ -11,7 +16,7 @@ RIPPLED_DPKG_VERSION=$(echo "${RIPPLED_VERSION}" | sed 's!-!~!g')
 # version here (hardcoded to 1). Does it ever need to change?
 RIPPLED_DPKG_FULL_VERSION="${RIPPLED_DPKG_VERSION}-1"
 
-cd rippled
+cd /opt/rippled_bld/pkg/rippled
 if [[ -n $(git status --porcelain) ]]; then
     git status
     error "Unstaged changes in this repo - please commit first"
@@ -40,6 +45,8 @@ CHANGELOG
 # PATH must be preserved for our more modern cmake in /opt/local
 # TODO : consider allowing lintian to run in future ?
 export DH_BUILD_DDEBS=1
+export CC=gcc-8
+export CXX=g++-8
 debuild --no-lintian --preserve-envvar PATH --preserve-env -us -uc
 rc=$?; if [[ $rc != 0 ]]; then
     error "error building dpkg"

Builds/containers/packaging/dpkg/debian/conffiles

@@ -1,3 +1,3 @@
-opt/ripple/etc/rippled.cfg
-opt/ripple/etc/validators.txt
-etc/logrotate.d/rippled
+/opt/ripple/etc/rippled.cfg
+/opt/ripple/etc/validators.txt
+/etc/logrotate.d/rippled

Builds/containers/packaging/dpkg/debian/control

@@ -2,7 +2,7 @@ Source: rippled
 Section: misc
 Priority: extra
 Maintainer: Ripple Labs Inc. <support@ripple.com>
-Build-Depends: cmake, debhelper (>=9), libprotobuf-dev, libssl-dev, zlib1g-dev, dh-systemd, ninja-build
+Build-Depends: cmake, debhelper (>=9), zlib1g-dev, dh-systemd, ninja-build
 Standards-Version: 3.9.7
 Homepage: http://ripple.com/
 
@@ -17,5 +17,5 @@ Section: devel
 Recommends: rippled (= ${binary:Version})
 Architecture: any
 Multi-Arch: same
-Depends: ${misc:Depends}, ${shlibs:Depends}, libprotobuf-dev, libssl-dev
+Depends: ${misc:Depends}, ${shlibs:Depends}, libprotobuf-dev, libprotoc-dev, protobuf-compiler
 Description: development files for applications using xrpl core library (serialize + sign)

Builds/containers/packaging/dpkg/debian/docs

@@ -1,3 +1,3 @@
 README.md
-LICENSE
+LICENSE.md
 RELEASENOTES.md

Builds/containers/packaging/dpkg/debian/rippled.install

@@ -1,6 +1,7 @@
 opt/ripple/bin/rippled
 opt/ripple/bin/validator-keys
 opt/ripple/bin/update-rippled.sh
+opt/ripple/bin/getRippledInfo
 opt/ripple/etc/rippled.cfg
 opt/ripple/etc/validators.txt
 opt/ripple/etc/update-rippled-cron

Builds/containers/packaging/dpkg/debian/rules

@@ -11,6 +11,9 @@ export CXXFLAGS:=$(subst -Werror=format-security,,$(CXXFLAGS))
 %:
 	dh $@ --with systemd
 
+override_dh_systemd_start:
+	dh_systemd_start --no-restart-on-upgrade
+
 override_dh_auto_configure:
 	env
 	rm -rf bld
@@ -19,28 +22,22 @@ override_dh_auto_configure:
 	cmake .. -G Ninja \
 		-DCMAKE_INSTALL_PREFIX=/opt/ripple \
 		-DCMAKE_BUILD_TYPE=Release \
-		-Dstatic=true \
-		-Dlocal_protobuf=ON \
-		-DCMAKE_VERBOSE_MAKEFILE=ON
+		-Dstatic=ON \
+		-Dunity=OFF \
+		-Dvalidator_keys=ON \
+		-DCMAKE_VERBOSE_MAKEFILE=OFF
 
 override_dh_auto_build:
 	cd bld && \
-	cmake --build . --parallel -- -v
+	cmake --build . --target rippled --target validator-keys --parallel
 
 override_dh_auto_install:
-	cd bld && DESTDIR=../debian/tmp cmake --build . --target install -- -v
-	rm -rf bld_vl
-	mkdir -p bld_vl
-	cd bld_vl && \
-	cmake ../../validator-keys-tool -G Ninja \
-		-DCMAKE_BUILD_TYPE=Release \
-		-DCMAKE_PREFIX_PATH=${PWD}/debian/tmp/opt/ripple/ \
-		-Dstatic=true \
-		-DCMAKE_VERBOSE_MAKEFILE=ON && \
-	cmake --build . --parallel -- -v
-	install -D bld_vl/validator-keys debian/tmp/opt/ripple/bin/validator-keys
+	cd bld && DESTDIR=../debian/tmp cmake --build . --target install
+	install -D bld/validator-keys/validator-keys debian/tmp/opt/ripple/bin/validator-keys
 	install -D Builds/containers/shared/update-rippled.sh debian/tmp/opt/ripple/bin/update-rippled.sh
+	install -D bin/getRippledInfo debian/tmp/opt/ripple/bin/getRippledInfo
 	install -D Builds/containers/shared/update-rippled-cron debian/tmp/opt/ripple/etc/update-rippled-cron
 	install -D Builds/containers/shared/rippled-logrotate debian/tmp/etc/logrotate.d/rippled
+	rm -rf debian/tmp/opt/ripple/lib64/cmake/date
 	rm -rf bld
 	rm -rf bld_vl

Builds/containers/packaging/rpm/build_rpm.sh

@@ -1,6 +1,9 @@
 #!/usr/bin/env bash
 set -ex
 
+cd /opt/rippled_bld/pkg
+cp -fpu rippled/Builds/containers/packaging/rpm/rippled.spec .
+cp -fpu rippled/Builds/containers/shared/update_sources.sh .
 source update_sources.sh
 
 # Build the rpm
@@ -25,16 +28,16 @@ if [[ $RPM_PATCH ]]; then
     export RPM_PATCH
 fi
 
-cd rippled
+cd /opt/rippled_bld/pkg/rippled
 if [[ -n $(git status --porcelain) ]]; then
     git status
     error "Unstaged changes in this repo - please commit first"
 fi
 git archive --format tar.gz --prefix rippled/ -o ../rpmbuild/SOURCES/rippled.tar.gz HEAD
+# TODO include validator-keys sources
 cd ..
-tar -zc --exclude-vcs -f ./rpmbuild/SOURCES/validator-keys.tar.gz validator-keys-tool/
 
-source /opt/rh/devtoolset-7/enable
+source /opt/rh/devtoolset-8/enable
 
 rpmbuild --define "_topdir ${PWD}/rpmbuild" -ba rippled.spec
 rc=$?; if [[ $rc != 0 ]]; then

Builds/containers/packaging/rpm/rippled.spec

@@ -11,9 +11,8 @@ Summary:        rippled daemon
 License:        MIT
 URL:            http://ripple.com/
 Source0:        rippled.tar.gz
-Source1:        validator-keys.tar.gz
 
-BuildRequires:  protobuf-static openssl-static cmake zlib-static ninja-build
+BuildRequires:  cmake zlib-static ninja-build
 
 %description
 rippled
@@ -21,48 +20,38 @@ rippled
 %package devel
 Summary: Files for development of applications using xrpl core library
 Group: Development/Libraries
-Requires: openssl-static, zlib-static
+Requires: zlib-static
 
 %description devel
 core library for development of standalone applications that sign transactions.
 
 %prep
-%setup -c -n rippled -a 1
+%setup -c -n rippled
 
 %build
 cd rippled
 mkdir -p bld.release
 cd bld.release
-cmake .. -G Ninja -DCMAKE_INSTALL_PREFIX=%{_prefix} -DCMAKE_BUILD_TYPE=Release -Dstatic=true -DCMAKE_VERBOSE_MAKEFILE=ON -Dlocal_protobuf=ON
-# build VK
-cd ../../validator-keys-tool
-mkdir -p bld.release
-cd bld.release
-# Install a copy of the rippled artifacts into a local VK build dir so that it
-# can use them to build against (VK needs xrpl_core lib to build). We install
-# into a local build dir instead of buildroot because we want VK to have
-# relative paths embedded in debug info, otherwise check-buildroot (rpmbuild)
-# will complain
-mkdir xrpl_dir
-DESTDIR="%{_builddir}/validator-keys-tool/bld.release/xrpl_dir"  cmake --build ../../rippled/bld.release --target install -- -v
-cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=%{_builddir}/validator-keys-tool/bld.release/xrpl_dir/opt/ripple -Dstatic=true -DCMAKE_VERBOSE_MAKEFILE=ON
-cmake --build . --parallel -- -v
+cmake .. -G Ninja -DCMAKE_INSTALL_PREFIX=%{_prefix} -DCMAKE_BUILD_TYPE=Release -Dstatic=true -Dunity=OFF -DCMAKE_VERBOSE_MAKEFILE=OFF -Dvalidator_keys=ON
+cmake --build . --parallel --target rippled --target validator-keys
 
 %pre
 test -e /etc/pki/tls || { mkdir -p /etc/pki; ln -s /usr/lib/ssl /etc/pki/tls; }
 
 %install
 rm -rf $RPM_BUILD_ROOT
-DESTDIR=$RPM_BUILD_ROOT cmake --build rippled/bld.release --target install -- -v
+DESTDIR=$RPM_BUILD_ROOT cmake --build rippled/bld.release --target install
+rm -rf ${RPM_BUILD_ROOT}/%{_prefix}/lib64/cmake/date
 install -d ${RPM_BUILD_ROOT}/etc/opt/ripple
 install -d ${RPM_BUILD_ROOT}/usr/local/bin
 ln -s %{_prefix}/etc/rippled.cfg ${RPM_BUILD_ROOT}/etc/opt/ripple/rippled.cfg
 ln -s %{_prefix}/etc/validators.txt ${RPM_BUILD_ROOT}/etc/opt/ripple/validators.txt
 ln -s %{_prefix}/bin/rippled ${RPM_BUILD_ROOT}/usr/local/bin/rippled
-install -D validator-keys-tool/bld.release/validator-keys ${RPM_BUILD_ROOT}%{_bindir}/validator-keys
+install -D rippled/bld.release/validator-keys/validator-keys ${RPM_BUILD_ROOT}%{_bindir}/validator-keys
 install -D ./rippled/Builds/containers/shared/rippled.service ${RPM_BUILD_ROOT}/usr/lib/systemd/system/rippled.service
 install -D ./rippled/Builds/containers/packaging/rpm/50-rippled.preset ${RPM_BUILD_ROOT}/usr/lib/systemd/system-preset/50-rippled.preset
 install -D ./rippled/Builds/containers/shared/update-rippled.sh ${RPM_BUILD_ROOT}%{_bindir}/update-rippled.sh
+install -D ./rippled/bin/getRippledInfo ${RPM_BUILD_ROOT}%{_bindir}/getRippledInfo
 install -D ./rippled/Builds/containers/shared/update-rippled-cron ${RPM_BUILD_ROOT}%{_prefix}/etc/update-rippled-cron
 install -D ./rippled/Builds/containers/shared/rippled-logrotate ${RPM_BUILD_ROOT}/etc/logrotate.d/rippled
 install -d $RPM_BUILD_ROOT/var/log/rippled
@@ -87,10 +76,11 @@ chmod 644 /etc/logrotate.d/rippled
 chown -R root:$GROUP_NAME %{_prefix}/etc/update-rippled-cron
 
 %files
-%doc rippled/README.md rippled/LICENSE
+%doc rippled/README.md rippled/LICENSE.md
 %{_bindir}/rippled
 /usr/local/bin/rippled
 %{_bindir}/update-rippled.sh
+%{_bindir}/getRippledInfo
 %{_prefix}/etc/update-rippled-cron
 %{_bindir}/validator-keys
 %config(noreplace) %{_prefix}/etc/rippled.cfg
@@ -109,6 +99,9 @@ chown -R root:$GROUP_NAME %{_prefix}/etc/update-rippled-cron
 %{_prefix}/lib/cmake/ripple
 
 %changelog
+* Wed Aug 28 2019 Mike Ellery <mellery451@gmail.com>
+- Switch to subproject build for validator-keys
+
 * Wed May 15 2019 Mike Ellery <mellery451@gmail.com>
 - Make validator-keys use local rippled build for core lib
 
@@ -117,4 +110,3 @@ chown -R root:$GROUP_NAME %{_prefix}/etc/update-rippled-cron
 
 * Thu Jun 02 2016 Brandon Wilson <bwilson@ripple.com>
 - Install validators.txt
-

Builds/containers/shared/build_deps.sh

@@ -6,20 +6,15 @@ function build_boost()
     local boost_ver=$1
     local do_link=$2
     local boost_path=$(echo "${boost_ver}" | sed -e 's!\.!_!g')
-    cd /tmp
-    wget https://dl.bintray.com/boostorg/release/${boost_ver}/source/boost_${boost_path}.tar.bz2
     mkdir -p /opt/local
     cd /opt/local
-    tar xf /tmp/boost_${boost_path}.tar.bz2
+    BOOST_ROOT=/opt/local/boost_${boost_path}
+    BOOST_URL="https://boostorg.jfrog.io/artifactory/main/release/${boost_ver}/source/boost_${boost_path}.tar.gz"
+    BOOST_BUILD_ALL=true
+    . /tmp/install_boost.sh
     if [ "$do_link" = true ] ; then
         ln -s ./boost_${boost_path} boost
     fi
-    cd boost_${boost_path}
-    ./bootstrap.sh
-    ./b2 -j$(nproc)
-    ./b2 stage
-    cd ..
-    rm -f /tmp/boost_${boost_path}.tar.bz2
 }
 
 build_boost "1.70.0" true
@@ -27,52 +22,126 @@ build_boost "1.70.0" true
 # installed in opt, so won't be used
 # unless specified by OPENSSL_ROOT_DIR
 cd /tmp
-OPENSSL_VER=1.1.1
+OPENSSL_VER=1.1.1d
 wget https://www.openssl.org/source/openssl-${OPENSSL_VER}.tar.gz
 tar xf openssl-${OPENSSL_VER}.tar.gz
 cd openssl-${OPENSSL_VER}
 # NOTE: add -g to the end of the following line if we want debug symbols for openssl
 SSLDIR=$(openssl version -d | cut -d: -f2 | tr -d [:space:]\")
 ./config -fPIC --prefix=/opt/local/openssl --openssldir=${SSLDIR} zlib shared
-make -j$(nproc)
+make -j$(nproc) >> make_output.txt 2>&1
 make install
 cd ..
 rm -f openssl-${OPENSSL_VER}.tar.gz
 rm -rf openssl-${OPENSSL_VER}
-LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/opt/local/openssl/lib /opt/local/openssl/bin/openssl version -a
+LD_LIBRARY_PATH=${LD_LIBRARY_PATH:-}:/opt/local/openssl/lib /opt/local/openssl/bin/openssl version -a
+
+cd /tmp
+wget https://libarchive.org/downloads/libarchive-3.4.1.tar.gz
+tar xzf libarchive-3.4.1.tar.gz
+cd libarchive-3.4.1
+mkdir _bld && cd _bld
+cmake -DCMAKE_BUILD_TYPE=Release ..
+make -j$(nproc) >> make_output.txt 2>&1
+make install
+cd ../..
+rm -f libarchive-3.4.1.tar.gz
+rm -rf libarchive-3.4.1
+
+cd /tmp
+wget https://github.com/protocolbuffers/protobuf/releases/download/v3.10.1/protobuf-all-3.10.1.tar.gz
+tar xf protobuf-all-3.10.1.tar.gz
+cd protobuf-3.10.1
+./autogen.sh
+./configure
+make -j$(nproc) >> make_output.txt 2>&1
+make install
+ldconfig
+cd ..
+rm -f protobuf-all-3.10.1.tar.gz
+rm -rf protobuf-3.10.1
+
+cd /tmp
+wget https://c-ares.haxx.se/download/c-ares-1.15.0.tar.gz
+tar xf c-ares-1.15.0.tar.gz
+cd c-ares-1.15.0
+mkdir _bld && cd _bld
+cmake \
+  -DHAVE_LIBNSL=OFF \
+  -DCMAKE_BUILD_TYPE=Release \
+  -DCARES_STATIC=ON \
+  -DCARES_SHARED=OFF \
+  -DCARES_INSTALL=ON \
+  -DCARES_STATIC_PIC=ON \
+  -DCARES_BUILD_TOOLS=OFF \
+  -DCARES_BUILD_TESTS=OFF \
+  -DCARES_BUILD_CONTAINER_TESTS=OFF \
+  ..
+make -j$(nproc) >> make_output.txt 2>&1
+make install
+cd ../..
+rm -f c-ares-1.15.0.tar.gz
+rm -rf c-ares-1.15.0
+
+cd /tmp
+wget https://github.com/grpc/grpc/archive/v1.25.0.tar.gz
+tar xf v1.25.0.tar.gz
+cd grpc-1.25.0
+mkdir _bld && cd _bld
+cmake \
+  -DCMAKE_BUILD_TYPE=Release \
+  -DBUILD_SHARED_LIBS=OFF \
+  -DgRPC_ZLIB_PROVIDER=package \
+  -DgRPC_CARES_PROVIDER=package \
+  -DgRPC_SSL_PROVIDER=package \
+  -DgRPC_PROTOBUF_PROVIDER=package \
+  -DProtobuf_USE_STATIC_LIBS=ON \
+  ..
+make -j$(nproc) >> make_output.txt 2>&1
+make install
+cd ../..
+rm -f xf v1.25.0.tar.gz
+rm -rf grpc-1.25.0
 
 if [ "${CI_USE}" = true ] ; then
+
+    build_boost "1.71.0" false
+
     cd /tmp
-    wget https://github.com/doxygen/doxygen/archive/Release_1_8_14.tar.gz
-    tar xf Release_1_8_14.tar.gz
-    cd doxygen-Release_1_8_14
+    wget https://github.com/doxygen/doxygen/archive/Release_1_8_16.tar.gz
+    tar xf Release_1_8_16.tar.gz
+    cd doxygen-Release_1_8_16
     mkdir build
     cd build
     cmake -G "Unix Makefiles" ..
-    make -j$(nproc)
+    make -j$(nproc) >> make_output.txt 2>&1
     make install
     cd ../..
-    rm -f Release_1_8_14.tar.gz
-    rm -rf doxygen-Release_1_8_14
+    rm -f Release_1_8_16.tar.gz
+    rm -rf doxygen-Release_1_8_16
 
     mkdir -p /opt/plantuml
     wget -O /opt/plantuml/plantuml.jar https://downloads.sourceforge.net/project/plantuml/plantuml.jar
 
     cd /tmp
-    wget https://github.com/linux-test-project/lcov/releases/download/v1.13/lcov-1.13.tar.gz
-    tar xfz lcov-1.13.tar.gz
-    cd lcov-1.13
+    wget https://github.com/linux-test-project/lcov/releases/download/v1.14/lcov-1.14.tar.gz
+    tar xfz lcov-1.14.tar.gz
+    cd lcov-1.14
     make install PREFIX=/usr/local
     cd ..
-    rm -r lcov-1.13 lcov-1.13.tar.gz
+    rm -r lcov-1.14 lcov-1.14.tar.gz
+
+    cd /tmp
+    wget https://github.com/ccache/ccache/releases/download/v3.7.6/ccache-3.7.6.tar.gz
+    tar xf ccache-3.7.6.tar.gz
+    cd ccache-3.7.6
+    ./configure --prefix=/usr/local
+    make
+    make install
+    cd ..
+    rm -f ccache-3.7.6.tar.gz
+    rm -rf ccache-3.7.6
 
     pip install requests
     pip install https://github.com/codecov/codecov-python/archive/master.zip
-
-    set +e
-    mkdir -p /opt/local/nih_cache
-    mkdir -p /opt/jenkins
-    set -e
 fi
-
-

Builds/containers/shared/install_boost.sh

@@ -0,0 +1,93 @@
+#!/usr/bin/env bash
+# Assumptions:
+# 1) BOOST_ROOT and BOOST_URL are already defined,
+# and contain valid values. BOOST_URL2 may be defined
+# as a fallback. BOOST_WGET_OPTIONS may be defined with
+# retry options if the download(s) fail on the first try.
+# 2) The last namepart of BOOST_ROOT matches the
+# folder name internal to boost's .tar.gz
+# When testing you can force a boost build by clearing travis caches:
+# https://travis-ci.org/ripple/rippled/caches
+set -exu
+
+odir=$(pwd)
+: ${BOOST_TOOLSET:=msvc-14.1}
+
+if [[ -d "$BOOST_ROOT/lib" || -d "${BOOST_ROOT}/stage/lib" ]] ; then
+    echo "Using cached boost at $BOOST_ROOT"
+    exit
+fi
+
+#fetch/unpack:
+fn=$(basename -- "$BOOST_URL")
+ext="${fn##*.}"
+wopt="--quiet"
+wget ${wopt} $BOOST_URL -O /tmp/boost.tar.${ext} || \
+  ( [ -n "${BOOST_URL2}" ] && \
+    wget ${wopt} $BOOST_URL2 -O /tmp/boost.tar.${ext} ) || \
+  ( [ -n "${BOOST_WGET_OPTIONS}" ] &&
+    ( wget ${wopt} ${BOOST_WGET_OPTIONS} $BOOST_URL -O /tmp/boost.tar.${ext} || \
+      ( [ -n "${BOOST_URL2}" ] && \
+        wget ${wopt} ${BOOST_WGET_OPTIONS} $BOOST_URL2 -O /tmp/boost.tar.${ext} )
+    )
+  )
+cd $(dirname $BOOST_ROOT)
+rm -fr ${BOOST_ROOT}
+mkdir ${BOOST_ROOT}
+tar xf /tmp/boost.tar.${ext} -C ${BOOST_ROOT} --strip-components 1
+cd $BOOST_ROOT
+
+BLDARGS=()
+if [[ ${BOOST_BUILD_ALL:-false} == "true" ]]; then
+    # we never need boost-python...so even for ALL
+    # option we can skip it
+    BLDARGS+=(--without-python)
+else
+    BLDARGS+=(--with-chrono)
+    BLDARGS+=(--with-container)
+    BLDARGS+=(--with-context)
+    BLDARGS+=(--with-coroutine)
+    BLDARGS+=(--with-date_time)
+    BLDARGS+=(--with-filesystem)
+    BLDARGS+=(--with-program_options)
+    BLDARGS+=(--with-regex)
+    BLDARGS+=(--with-system)
+    BLDARGS+=(--with-atomic)
+    BLDARGS+=(--with-thread)
+fi
+BLDARGS+=(-j$((2*${NUM_PROCESSORS:-2})))
+BLDARGS+=(--prefix=${BOOST_ROOT}/_INSTALLED_)
+BLDARGS+=(-d0) # suppress messages/output
+
+if [[ -z ${COMSPEC:-} ]]; then
+    if [[ "$(uname)" == "Darwin" ]] ; then
+        BLDARGS+=(cxxflags="-std=c++14 -fvisibility=default")
+    else
+        BLDARGS+=(cxxflags="-std=c++14")
+        BLDARGS+=(runtime-link="static,shared")
+    fi
+    BLDARGS+=(--layout=tagged)
+    ./bootstrap.sh
+    ./b2 "${BLDARGS[@]}" stage
+    ./b2 "${BLDARGS[@]}" install
+else
+    BLDARGS+=(runtime-link="static,shared")
+    BLDARGS+=(--layout=versioned)
+    BLDARGS+=(--toolset="${BOOST_TOOLSET}")
+    BLDARGS+=(address-model=64)
+    BLDARGS+=(architecture=x86)
+    BLDARGS+=(link=static)
+    BLDARGS+=(threading=multi)
+    cmd /E:ON /D /S /C"bootstrap.bat"
+    ./b2.exe "${BLDARGS[@]}" stage
+    ./b2.exe "${BLDARGS[@]}" install
+fi
+
+if [[ ${CI:-false} == "true" ]]; then
+    # save some disk space...these are mostly
+    # obj files and don't need to be kept in CI contexts
+    rm -rf bin.v2
+fi
+
+cd $odir
+

Builds/containers/shared/install_cmake.sh

@@ -1,16 +1,34 @@
 #!/usr/bin/env bash
-set -ex
+set -e
 
-cd /tmp
-CM_INSTALLER=cmake-3.13.2-Linux-x86_64.sh
-CM_VER_DIR=/opt/local/cmake-3.13
-wget https://cmake.org/files/v3.13/$CM_INSTALLER
-chmod a+x $CM_INSTALLER
-mkdir -p $CM_VER_DIR
-ln -s $CM_VER_DIR /opt/local/cmake
-./$CM_INSTALLER --prefix=$CM_VER_DIR --exclude-subdir
-rm -f /tmp/$CM_INSTALLER
+IFS=. read cm_maj cm_min cm_rel <<<"$1"
+: ${cm_rel:-0}
+CMAKE_ROOT=${2:-"${HOME}/cmake"}
 
-export PATH="/opt/local/cmake/bin:${PATH}"
+function cmake_version ()
+{
+    if [[ -d ${CMAKE_ROOT} ]] ; then
+        local perms=$(test $(uname) = "Linux" && echo "/111" || echo "+111")
+        local installed=$(find ${CMAKE_ROOT} -perm ${perms} -type f -name cmake)
+        if [[ "${installed}" != "" ]] ; then
+            echo "$(${installed} --version | head -1)"
+        fi
+    fi
+}
+
+installed=$(cmake_version)
+if [[ "${installed}" != "" && ${installed} =~ ${cm_maj}.${cm_min}.${cm_rel} ]] ; then
+    echo "cmake already installed: ${installed}"
+    exit
+fi
+
+pkgname="cmake-${cm_maj}.${cm_min}.${cm_rel}-$(uname)-x86_64.tar.gz"
+tmppkg="/tmp/cmake.tar.gz"
+wget --quiet https://cmake.org/files/v${cm_maj}.${cm_min}/${pkgname} -O ${tmppkg}
+mkdir -p ${CMAKE_ROOT}
+cd ${CMAKE_ROOT}
+tar --strip-components 1 -xf ${tmppkg}
+rm -f ${tmppkg}
+echo "installed: $(cmake_version)"
 
 

Builds/containers/shared/rippled.service

@@ -1,12 +1,12 @@
 [Unit]
 Description=Ripple Daemon
+After=network-online.target
+Wants=network-online.target
 
 [Service]
 Type=simple
 ExecStart=/opt/ripple/bin/rippled --net --silent --conf /etc/opt/ripple/rippled.cfg
-# Default KillSignal can be used if/when rippled handles SIGTERM
-KillSignal=SIGINT
-Restart=no
+Restart=on-failure
 User=rippled
 Group=rippled
 LimitNOFILE=65536

Builds/containers/shared/update-rippled.sh

@@ -2,6 +2,12 @@
 
 # auto-update script for rippled daemon
 
+# Check for sudo/root permissions
+if [[ $(id -u) -ne 0 ]] ; then
+   echo "This update script must be run as root or sudo"
+   exit 1
+fi
+
 LOCKDIR=/tmp/rippleupdate.lock
 UPDATELOG=/var/log/rippled/update.log
 

Builds/containers/shared/update_sources.sh

@@ -8,13 +8,6 @@ function error {
 cd /opt/rippled_bld/pkg/rippled
 export RIPPLED_VERSION=$(egrep -i -o "\b(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-[0-9a-z\-]+(\.[0-9a-z\-]+)*)?(\+[0-9a-z\-]+(\.[0-9a-z\-]+)*)?\b" src/ripple/protocol/impl/BuildInfo.cpp)
 
-cd ..
-git clone https://github.com/ripple/validator-keys-tool.git
-cd validator-keys-tool
-git checkout origin/master
-git submodule update --init --recursive
-cd ..
-
 : ${PKG_OUTDIR:=/opt/rippled_bld/pkg/out}
 export PKG_OUTDIR
 if [ ! -d ${PKG_OUTDIR} ]; then

Builds/containers/ubuntu-builder/Dockerfile

@@ -1,32 +1,36 @@
-ARG DIST_TAG=16.04
-ARG GIT_COMMIT=unknown
-
+ARG DIST_TAG=18.04
 FROM ubuntu:$DIST_TAG
+ARG GIT_COMMIT=unknown
+ARG CI_USE=false
 LABEL git-commit=$GIT_COMMIT
 
 # install/setup prerequisites:
 COPY ubuntu-builder/ubuntu_setup.sh /tmp/
 COPY shared/build_deps.sh /tmp/
 COPY shared/install_cmake.sh /tmp/
+COPY shared/install_boost.sh /tmp/
 RUN chmod +x /tmp/ubuntu_setup.sh && \
     chmod +x /tmp/build_deps.sh && \
+    chmod +x /tmp/install_boost.sh && \
     chmod +x /tmp/install_cmake.sh
 RUN /tmp/ubuntu_setup.sh
-RUN /tmp/install_cmake.sh
+
+RUN /tmp/install_cmake.sh 3.16.1 /opt/local/cmake-3.16
+RUN ln -s /opt/local/cmake-3.16 /opt/local/cmake
 ENV PATH="/opt/local/cmake/bin:$PATH"
+# also install min supported cmake for testing
+RUN if [ "${CI_USE}" = true ] ; then /tmp/install_cmake.sh 3.9.0 /opt/local/cmake-3.9; fi
+
 RUN /tmp/build_deps.sh
 ENV PLANTUML_JAR="/opt/plantuml/plantuml.jar"
-ENV BOOST_ROOT="/opt/local/boost"
+ENV BOOST_ROOT="/opt/local/boost/_INSTALLED_"
 ENV OPENSSL_ROOT="/opt/local/openssl"
 
 # prep files for package building
-RUN mkdir -m 777 -p /opt/rippled_bld/pkg
+RUN mkdir -m 777 -p /opt/rippled_bld/pkg/debian
+RUN update-alternatives --set gcc /usr/bin/gcc-8
 WORKDIR /opt/rippled_bld/pkg
 
-COPY packaging/dpkg/debian /opt/rippled_bld/pkg/debian/
-COPY shared/update_sources.sh ./
-COPY shared/rippled.service /opt/rippled_bld/pkg/debian/
-
 COPY packaging/dpkg/build_dpkg.sh ./
 CMD ./build_dpkg.sh
 

Builds/containers/ubuntu-builder/ubuntu_setup.sh

@@ -24,29 +24,28 @@ apt-get -y upgrade
 if [[ ${VERSION_ID} =~ ^18\. ]] ; then
     apt-add-repository -y multiverse
     apt-add-repository -y universe
-elif [[ ${VERSION_ID} =~ ^16\. ]] ; then
-    add-apt-repository -y ppa:ubuntu-toolchain-r/test
 fi
+add-apt-repository -y ppa:ubuntu-toolchain-r/test
 apt-get -y clean
 apt-get -y update
 
-apt-get -y --fix-missing install  \
-    make cmake ninja-build ccache \
-    protobuf-compiler libprotobuf-dev openssl libssl-dev \
+apt-get -y --fix-missing install \
+    make cmake ninja-build autoconf automake libtool pkg-config libtool \
+    openssl libssl-dev \
     liblzma-dev libbz2-dev zlib1g-dev \
     libjemalloc-dev \
     python-pip \
     gdb gdbserver \
     libstdc++6 \
-    flex bison \
+    flex bison parallel \
     libicu-dev texinfo \
     java-common javacc \
-    gcc-7 g++-7 \
-    gcc-8 g++-8 \
     dpkg-dev debhelper devscripts fakeroot \
     debmake git-buildpackage dh-make gitpkg debsums gnupg \
-    dh-buildinfo dh-make dh-systemd
+    dh-buildinfo dh-make dh-systemd \
+    apt-transport-https
 
+apt-get -y install gcc-7 g++-7
 update-alternatives --install \
     /usr/bin/gcc gcc /usr/bin/gcc-7 40 \
     --slave /usr/bin/g++ g++ /usr/bin/g++-7 \
@@ -56,6 +55,8 @@ update-alternatives --install \
     --slave /usr/bin/gcov gcov /usr/bin/gcov-7 \
     --slave /usr/bin/gcov-tool gcov-tool /usr/bin/gcov-dump-7 \
     --slave /usr/bin/gcov-dump gcov-dump /usr/bin/gcov-tool-7
+
+apt-get -y install gcc-8 g++-8
 update-alternatives --install \
     /usr/bin/gcc gcc /usr/bin/gcc-8 20 \
     --slave /usr/bin/g++ g++ /usr/bin/g++-8 \
@@ -71,6 +72,30 @@ update-alternatives --install /usr/bin/cpp cpp /usr/bin/cpp-7 40
 update-alternatives --install /usr/bin/cpp cpp /usr/bin/cpp-8 20
 update-alternatives --auto cpp
 
+if [ "${CI_USE}" = true ] ; then
+    apt-get -y install gcc-6 g++-6
+    update-alternatives --install \
+        /usr/bin/gcc gcc /usr/bin/gcc-6 10 \
+        --slave /usr/bin/g++ g++ /usr/bin/g++-6 \
+        --slave /usr/bin/gcc-ar gcc-ar /usr/bin/gcc-ar-6 \
+        --slave /usr/bin/gcc-nm gcc-nm /usr/bin/gcc-nm-6 \
+        --slave /usr/bin/gcc-ranlib gcc-ranlib /usr/bin/gcc-ranlib-6 \
+        --slave /usr/bin/gcov gcov /usr/bin/gcov-6 \
+        --slave /usr/bin/gcov-tool gcov-tool /usr/bin/gcov-dump-6 \
+        --slave /usr/bin/gcov-dump gcov-dump /usr/bin/gcov-tool-6
+
+    apt-get -y install gcc-9 g++-9
+    update-alternatives --install \
+        /usr/bin/gcc gcc /usr/bin/gcc-9 15 \
+        --slave /usr/bin/g++ g++ /usr/bin/g++-9 \
+        --slave /usr/bin/gcc-ar gcc-ar /usr/bin/gcc-ar-9 \
+        --slave /usr/bin/gcc-nm gcc-nm /usr/bin/gcc-nm-9 \
+        --slave /usr/bin/gcc-ranlib gcc-ranlib /usr/bin/gcc-ranlib-9 \
+        --slave /usr/bin/gcov gcov /usr/bin/gcov-9 \
+        --slave /usr/bin/gcov-tool gcov-tool /usr/bin/gcov-dump-9 \
+        --slave /usr/bin/gcov-dump gcov-dump /usr/bin/gcov-tool-9
+fi
+
 if [[ ${VERSION_ID} =~ ^18\. ]] ; then
     apt-get -y install binutils
 elif [[ ${VERSION_ID} =~ ^16\. ]] ; then
@@ -82,39 +107,83 @@ if [[ ${VERSION_ID} =~ ^18\. ]] ; then
     cat << EOF > /etc/apt/sources.list.d/llvm.list
 deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic main
 deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic main
-deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-6.0 main
-deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic-6.0 main
 deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-7 main
 deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic-7 main
+deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-8 main
+deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic-8 main
+deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-9 main
+deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic-9 main
 EOF
 elif [[ ${VERSION_ID} =~ ^16\. ]] ; then
     cat << EOF > /etc/apt/sources.list.d/llvm.list
 deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial main
 deb-src http://apt.llvm.org/xenial/ llvm-toolchain-xenial main
-deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-6.0 main
-deb-src http://apt.llvm.org/xenial/ llvm-toolchain-xenial-6.0 main
 deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-7 main
 deb-src http://apt.llvm.org/xenial/ llvm-toolchain-xenial-7 main
+deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-8 main
+deb-src http://apt.llvm.org/xenial/ llvm-toolchain-xenial-8 main
+deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-9 main
+deb-src http://apt.llvm.org/xenial/ llvm-toolchain-xenial-9 main
 EOF
 fi
 apt-get -y update
 
-apt-get -y install  \
-    clang-7 libclang-common-7-dev libclang-7-dev libllvm7 lldb-7 llvm-7 \
-    llvm-7-dev llvm-7-runtime clang-format-7 python-clang-7 python-lldb-7 \
-    liblldb-7-dev lld-7 libfuzzer-7-dev libc++-7-dev
+apt-get -y install \
+    clang-7 libclang-common-7-dev libclang-7-dev libllvm7 llvm-7 \
+    llvm-7-dev llvm-7-runtime clang-format-7 python-clang-7 \
+    lld-7 libfuzzer-7-dev libc++-7-dev
 update-alternatives --install \
-  /usr/bin/clang clang /usr/bin/clang-7 40 \
-  --slave /usr/bin/clang++ clang++ /usr/bin/clang++-7 \
-  --slave /usr/bin/llvm-profdata llvm-profdata /usr/bin/llvm-profdata-7 \
-  --slave /usr/bin/asan-symbolize asan-symbolize /usr/bin/asan_symbolize-7 \
-  --slave /usr/bin/clang-format clang-format /usr/bin/clang-format-7 \
-  --slave /usr/bin/lldb lldb /usr/bin/lldb-7 \
-  --slave /usr/bin/lldb-server lldb-server /usr/bin/lldb-server-7 \
-  --slave /usr/bin/llvm-ar llvm-ar /usr/bin/llvm-ar-7 \
-  --slave /usr/bin/llvm-cov llvm-cov /usr/bin/llvm-cov-7 \
-  --slave /usr/bin/llvm-nm llvm-nm /usr/bin/llvm-nm-7
+    /usr/bin/clang clang /usr/bin/clang-7 40 \
+    --slave /usr/bin/clang++ clang++ /usr/bin/clang++-7 \
+    --slave /usr/bin/llvm-profdata llvm-profdata /usr/bin/llvm-profdata-7 \
+    --slave /usr/bin/asan-symbolize asan-symbolize /usr/bin/asan_symbolize-7 \
+    --slave /usr/bin/llvm-symbolizer llvm-symbolizer /usr/bin/llvm-symbolizer-7 \
+    --slave /usr/bin/clang-format clang-format /usr/bin/clang-format-7 \
+    --slave /usr/bin/llvm-ar llvm-ar /usr/bin/llvm-ar-7 \
+    --slave /usr/bin/llvm-cov llvm-cov /usr/bin/llvm-cov-7 \
+    --slave /usr/bin/llvm-nm llvm-nm /usr/bin/llvm-nm-7
+apt-get -y install \
+    clang-8 libclang-common-8-dev libclang-8-dev libllvm8 llvm-8 \
+    llvm-8-dev llvm-8-runtime clang-format-8 python-clang-8 \
+    lld-8 libfuzzer-8-dev libc++-8-dev
+update-alternatives --install \
+    /usr/bin/clang clang /usr/bin/clang-8 20 \
+    --slave /usr/bin/clang++ clang++ /usr/bin/clang++-8 \
+    --slave /usr/bin/llvm-profdata llvm-profdata /usr/bin/llvm-profdata-8 \
+    --slave /usr/bin/asan-symbolize asan-symbolize /usr/bin/asan_symbolize-8 \
+    --slave /usr/bin/llvm-symbolizer llvm-symbolizer /usr/bin/llvm-symbolizer-8 \
+    --slave /usr/bin/clang-format clang-format /usr/bin/clang-format-8 \
+    --slave /usr/bin/llvm-ar llvm-ar /usr/bin/llvm-ar-8 \
+    --slave /usr/bin/llvm-cov llvm-cov /usr/bin/llvm-cov-8 \
+    --slave /usr/bin/llvm-nm llvm-nm /usr/bin/llvm-nm-8
 update-alternatives --auto clang
 
+if [ "${CI_USE}" = true ] ; then
+    apt-get -y install \
+        clang-9 libclang-common-9-dev libclang-9-dev libllvm9 llvm-9 \
+        llvm-9-dev llvm-9-runtime clang-format-9 python-clang-9 \
+        lld-9 libfuzzer-9-dev libc++-9-dev
+    update-alternatives --install \
+        /usr/bin/clang clang /usr/bin/clang-9 20 \
+        --slave /usr/bin/clang++ clang++ /usr/bin/clang++-9 \
+        --slave /usr/bin/llvm-profdata llvm-profdata /usr/bin/llvm-profdata-9 \
+        --slave /usr/bin/asan-symbolize asan-symbolize /usr/bin/asan_symbolize-9 \
+        --slave /usr/bin/llvm-symbolizer llvm-symbolizer /usr/bin/llvm-symbolizer-9 \
+        --slave /usr/bin/clang-format clang-format /usr/bin/clang-format-9 \
+        --slave /usr/bin/llvm-ar llvm-ar /usr/bin/llvm-ar-9 \
+        --slave /usr/bin/llvm-cov llvm-cov /usr/bin/llvm-cov-9 \
+        --slave /usr/bin/llvm-nm llvm-nm /usr/bin/llvm-nm-9
+
+    # only install latest lldb
+    apt-get -y install lldb-9 python-lldb-9 liblldb-9-dev
+    update-alternatives --install \
+        /usr/bin/lldb lldb /usr/bin/lldb-9 50 \
+        --slave /usr/bin/lldb-server lldb-server /usr/bin/lldb-server-9 \
+        --slave /usr/bin/lldb-argdumper lldb-argdumper /usr/bin/lldb-argdumper-9 \
+        --slave /usr/bin/lldb-instr lldb-instr /usr/bin/lldb-instr-9 \
+        --slave /usr/bin/lldb-mi lldb-mi /usr/bin/lldb-mi-9
+    update-alternatives --auto clang
+fi
+
 apt-get -y autoremove
 

Builds/levelization/README.md

@@ -0,0 +1,113 @@
+# Levelization
+
+Levelization is the term used to describe efforts to prevent rippled from
+having or creating cyclic dependencies.
+
+rippled code is organized into directories under `src/rippled` (and
+`src/test`) representing modules. The modules are intended to be
+organized into "tiers" or "levels" such that a module from one level can
+only include code from lower levels. Additionally, a module
+in one level should never include code in an `impl` folder of any level
+other than it's own.
+
+Unfortunately, over time, enforcement of levelization has been
+inconsistent, so the current state of the code doesn't necessarily
+reflect these rules. Whenever possible, developers should refactor any
+levelization violations they find (by moving files or individual
+classes). At the very least, don't make things worse.
+
+The table below summarizes the _desired_ division of modules, based on the
+state of the rippled code when it was created. The levels are numbered from
+the bottom up with the lower level, lower numbered, more independent
+modules listed first, and the higher level, higher numbered modules with
+more dependencies listed later.
+
+**tl;dr:** The modules listed first are more independent than the modules
+listed later.
+
+| Level / Tier | Module(s)                                     |
+|--------------|-----------------------------------------------|
+| 01           | ripple/beast ripple/unity
+| 02           | ripple/basics
+| 03           | ripple/json ripple/crypto
+| 04           | ripple/protocol
+| 05           | ripple/core ripple/conditions ripple/consensus ripple/resource ripple/server
+| 06           | ripple/peerfinder ripple/ledger ripple/nodestore ripple/net
+| 07           | ripple/shamap ripple/overlay
+| 08           | ripple/app
+| 09           | ripple/rpc
+| 10           | test/jtx test/beast test/csf
+| 11           | test/unit_test
+| 12           | test/crypto test/conditions test/json test/resource test/shamap test/peerfinder test/basics test/overlay
+| 13           | test
+| 14           | test/net test/protocol test/ledger test/consensus test/core test/server test/nodestore
+| 15           | test/rpc test/app
+
+(Note that `test` levelization is *much* less important and *much* less
+strictly enforced than `ripple` levelization, other than the requirement
+that `test` code should *never* be included in `ripple` code.)
+
+## Validation
+
+The [levelization.sh](levelization.sh) script takes no parameters,
+reads no environment variables, and can be run from any directory,
+as long as it is in the expected location in the rippled repo.
+It can be run at any time from within a checked out repo, and will
+do an analysis of all the `#include`s in
+the rippled source. The only caveat is that it runs much slower
+under Windows than in Linux. It hasn't yet been tested under MacOS.
+It generates many files of [results](results):
+
+* `rawincludes.txt`: The raw dump of the `#includes`
+* `paths.txt`: A second dump grouping the source module
+  to the destination module, deduped, and with frequency counts.
+* `includes/`: A directory where each file represents a module and
+  contains a list of modules and counts that the module _includes_.
+* `includedby/`: Similar to `includes/`, but the other way around. Each
+  file represents a module and contains a list of modules and counts
+  that _include_ the module.
+* [`loops.txt`](results/loops.txt): A list of direct loops detected
+  between modules as they actually exist, as opposed to how they are
+  desired as described above. In a perfect repo, this file will be
+  empty.
+  This file is committed to the repo, and is used by the [levelization
+  Github workflow](../../.github/workflows/levelization.yml) to validate
+  that nothing changed.
+* [`ordering.txt`](results/ordering.txt): A list showing relationships
+  between modules where there are no loops as they actually exist, as
+  opposed to how they are desired as described above.
+  This file is committed to the repo, and is used by the [levelization
+  Github workflow](../../.github/workflows/levelization.yml) to validate
+  that nothing changed.
+* [`levelization.yml`](../../.github/workflows/levelization.yml)
+  Github Actions workflow to test that levelization loops haven't
+  changed.  Unfortunately, if changes are detected, it can't tell if
+  they are improvements or not, so if you have resolved any issues or
+  done anything else to improve levelization, run `levelization.sh`,
+  and commit the updated results.
+
+The  `loops.txt` and `ordering.txt` files relate the modules
+using comparison signs, which indicate the number of times each
+module is included in the other.
+
+* `A > B` means that A should probably be at a higher level than B,
+  because B is included in A significantly more than A is included in B.
+  These results can be included in both `loops.txt` and `ordering.txt`.
+  Because `ordering.txt`only includes relationships where B is not
+  included in A at all, it will only include these types of results.
+* `A ~= B` means that A and B are included in each other a different
+  number of times, but the values are so close that the script can't
+  definitively say that one should be above the other. These results
+  will only be included in `loops.txt`.
+* `A == B` means that A and B include each other the same number of
+  times, so the script has no clue which should be higher. These results
+  will only be included in `loops.txt`.
+
+The committed files hide the detailed values intentionally, to
+prevent false alarms and merging issues, and because it's easy to
+get those details locally.
+
+1. Run `levelization.sh`
+2. Grep the modules in `paths.txt`.
+   * For example, if a cycle is found `A ~= B`, simply `grep -w
+     A Builds/levelization/results/paths.txt | grep -w B`

Builds/levelization/levelization.sh

@@ -0,0 +1,127 @@
+#!/bin/bash
+
+# Usage: levelization.sh
+# This script takes no parameters, reads no environment variables,
+# and can be run from any directory, as long as it is in the expected
+# location in the repo.
+
+pushd $( dirname $0 )
+
+if [ -v PS1 ]
+then
+  # if the shell is interactive, clean up any flotsam before analyzing
+  git clean -ix
+fi
+
+rm -rfv results
+mkdir results
+includes="$( pwd )/results/rawincludes.txt"
+pushd ../..
+echo Raw includes:
+grep -r '#include.*/.*\.h' src/ripple/ src/test/ | \
+    grep -v boost | tee ${includes}
+popd
+pushd results
+
+oldifs=${IFS}
+IFS=:
+mkdir includes
+mkdir includedby
+echo Build levelization paths
+exec 3< ${includes} # open rawincludes.txt for input
+while read -r -u 3 file include
+do
+    level=$( echo ${file} | cut -d/ -f 2,3 )
+    # If the "level" indicates a file, cut off the filename
+    if [[ "${level##*.}" != "${level}" ]]
+    then
+        # Use the "toplevel" label as a workaround for `sort`
+        # inconsistencies between different utility versions
+        level="$( dirname ${level} )/toplevel"
+    fi
+    level=$( echo ${level} | tr '/' '.' )
+
+    includelevel=$( echo ${include} | sed 's/.*["<]//; s/[">].*//' | \
+        cut -d/ -f 1,2 )
+    if [[ "${includelevel##*.}" != "${includelevel}" ]]
+    then
+        # Use the "toplevel" label as a workaround for `sort`
+        # inconsistencies between different utility versions
+        includelevel="$( dirname ${includelevel} )/toplevel"
+    fi
+    includelevel=$( echo ${includelevel} | tr '/' '.' )
+
+    if [[ "$level" != "$includelevel" ]]
+    then
+        echo $level $includelevel | tee -a paths.txt
+    fi
+done
+echo Sort and dedup paths
+sort -ds paths.txt | uniq -c | tee sortedpaths.txt
+mv sortedpaths.txt paths.txt
+exec 3>&- #close fd 3
+IFS=${oldifs}
+unset oldifs
+
+echo Split into flat-file database
+exec 4<paths.txt # open paths.txt for input
+while read -r -u 4 count level include
+do
+    echo ${include} ${count} | tee -a includes/${level}
+    echo ${level} ${count} | tee -a includedby/${include}
+done
+exec 4>&- #close fd 4
+
+loops="$( pwd )/loops.txt"
+ordering="$( pwd )/ordering.txt"
+pushd includes
+echo Search for loops
+# Redirect stdout to a file
+exec 4>&1
+exec 1>"${loops}"
+for source in *
+do
+  if [[ -f "$source" ]]
+  then
+    exec 5<"${source}" # open for input
+    while read -r -u 5 include includefreq
+    do
+      if [[ -f $include ]]
+      then
+        if grep -q -w $source $include
+        then
+          if grep -q -w "Loop: $include $source" "${loops}"
+          then
+            continue
+          fi
+          sourcefreq=$( grep -w $source $include | cut -d\  -f2 )
+          echo "Loop: $source $include"
+          # If the counts are close, indicate that the two modules are
+          # on the same level, though they shouldn't be
+          if [[ $(( $includefreq - $sourcefreq )) -gt 3 ]]
+          then
+              echo -e "  $source > $include\n"
+          elif [[ $(( $sourcefreq - $includefreq )) -gt 3 ]]
+          then
+              echo -e "  $include > $source\n"
+          elif [[ $sourcefreq -eq $includefreq ]]
+          then
+              echo -e "  $include == $source\n"
+          else
+              echo -e "  $include ~= $source\n"
+          fi
+        else
+          echo "$source > $include" >> "${ordering}"
+        fi
+      fi
+    done
+    exec 5>&- #close fd 5
+  fi
+done
+exec 1>&4 #close fd 1
+exec 4>&- #close fd 4
+cat "${ordering}"
+cat "${loops}"
+popd
+popd
+popd

Builds/levelization/results/loops.txt

@@ -0,0 +1,57 @@
+Loop: ripple.app ripple.core
+  ripple.app > ripple.core
+
+Loop: ripple.app ripple.ledger
+  ripple.app > ripple.ledger
+
+Loop: ripple.app ripple.net
+  ripple.app > ripple.net
+
+Loop: ripple.app ripple.nodestore
+  ripple.app > ripple.nodestore
+
+Loop: ripple.app ripple.overlay
+  ripple.overlay ~= ripple.app
+
+Loop: ripple.app ripple.peerfinder
+  ripple.peerfinder ~= ripple.app
+
+Loop: ripple.app ripple.rpc
+  ripple.rpc > ripple.app
+
+Loop: ripple.app ripple.shamap
+  ripple.app > ripple.shamap
+
+Loop: ripple.basics ripple.core
+  ripple.core > ripple.basics
+
+Loop: ripple.basics ripple.json
+  ripple.json ~= ripple.basics
+
+Loop: ripple.basics ripple.nodestore
+  ripple.nodestore > ripple.basics
+
+Loop: ripple.basics ripple.protocol
+  ripple.protocol > ripple.basics
+
+Loop: ripple.basics ripple.rpc
+  ripple.rpc > ripple.basics
+
+Loop: ripple.core ripple.net
+  ripple.net > ripple.core
+
+Loop: ripple.net ripple.rpc
+  ripple.rpc > ripple.net
+
+Loop: ripple.nodestore ripple.overlay
+  ripple.overlay ~= ripple.nodestore
+
+Loop: ripple.overlay ripple.rpc
+  ripple.rpc ~= ripple.overlay
+
+Loop: test.jtx test.toplevel
+  test.toplevel > test.jtx
+
+Loop: test.jtx test.unit_test
+  test.unit_test == test.jtx
+

Builds/levelization/results/ordering.txt

@@ -0,0 +1,220 @@
+ripple.app > ripple.basics
+ripple.app > ripple.beast
+ripple.app > ripple.conditions
+ripple.app > ripple.consensus
+ripple.app > ripple.crypto
+ripple.app > ripple.json
+ripple.app > ripple.protocol
+ripple.app > ripple.resource
+ripple.app > test.unit_test
+ripple.basics > ripple.beast
+ripple.conditions > ripple.basics
+ripple.conditions > ripple.protocol
+ripple.consensus > ripple.basics
+ripple.consensus > ripple.beast
+ripple.consensus > ripple.json
+ripple.consensus > ripple.protocol
+ripple.core > ripple.beast
+ripple.core > ripple.json
+ripple.core > ripple.protocol
+ripple.crypto > ripple.basics
+ripple.json > ripple.beast
+ripple.ledger > ripple.basics
+ripple.ledger > ripple.beast
+ripple.ledger > ripple.core
+ripple.ledger > ripple.json
+ripple.ledger > ripple.protocol
+ripple.net > ripple.basics
+ripple.net > ripple.beast
+ripple.net > ripple.json
+ripple.net > ripple.protocol
+ripple.net > ripple.resource
+ripple.nodestore > ripple.beast
+ripple.nodestore > ripple.core
+ripple.nodestore > ripple.json
+ripple.nodestore > ripple.protocol
+ripple.nodestore > ripple.unity
+ripple.overlay > ripple.basics
+ripple.overlay > ripple.beast
+ripple.overlay > ripple.core
+ripple.overlay > ripple.json
+ripple.overlay > ripple.peerfinder
+ripple.overlay > ripple.protocol
+ripple.overlay > ripple.resource
+ripple.overlay > ripple.server
+ripple.peerfinder > ripple.basics
+ripple.peerfinder > ripple.beast
+ripple.peerfinder > ripple.core
+ripple.peerfinder > ripple.protocol
+ripple.protocol > ripple.beast
+ripple.protocol > ripple.crypto
+ripple.protocol > ripple.json
+ripple.resource > ripple.basics
+ripple.resource > ripple.beast
+ripple.resource > ripple.json
+ripple.resource > ripple.protocol
+ripple.rpc > ripple.beast
+ripple.rpc > ripple.core
+ripple.rpc > ripple.crypto
+ripple.rpc > ripple.json
+ripple.rpc > ripple.ledger
+ripple.rpc > ripple.nodestore
+ripple.rpc > ripple.protocol
+ripple.rpc > ripple.resource
+ripple.rpc > ripple.server
+ripple.rpc > ripple.shamap
+ripple.server > ripple.basics
+ripple.server > ripple.beast
+ripple.server > ripple.crypto
+ripple.server > ripple.json
+ripple.server > ripple.protocol
+ripple.shamap > ripple.basics
+ripple.shamap > ripple.beast
+ripple.shamap > ripple.crypto
+ripple.shamap > ripple.nodestore
+ripple.shamap > ripple.protocol
+test.app > ripple.app
+test.app > ripple.basics
+test.app > ripple.beast
+test.app > ripple.core
+test.app > ripple.json
+test.app > ripple.ledger
+test.app > ripple.overlay
+test.app > ripple.protocol
+test.app > ripple.resource
+test.app > ripple.rpc
+test.app > test.jtx
+test.app > test.rpc
+test.app > test.toplevel
+test.app > test.unit_test
+test.basics > ripple.basics
+test.basics > ripple.beast
+test.basics > ripple.json
+test.basics > ripple.protocol
+test.basics > ripple.rpc
+test.basics > test.jtx
+test.basics > test.unit_test
+test.beast > ripple.basics
+test.beast > ripple.beast
+test.conditions > ripple.basics
+test.conditions > ripple.beast
+test.conditions > ripple.conditions
+test.consensus > ripple.app
+test.consensus > ripple.basics
+test.consensus > ripple.beast
+test.consensus > ripple.consensus
+test.consensus > ripple.ledger
+test.consensus > ripple.rpc
+test.consensus > test.csf
+test.consensus > test.toplevel
+test.consensus > test.unit_test
+test.core > ripple.basics
+test.core > ripple.beast
+test.core > ripple.core
+test.core > ripple.crypto
+test.core > ripple.json
+test.core > ripple.server
+test.core > test.jtx
+test.core > test.toplevel
+test.core > test.unit_test
+test.csf > ripple.basics
+test.csf > ripple.beast
+test.csf > ripple.consensus
+test.csf > ripple.json
+test.csf > ripple.protocol
+test.json > ripple.beast
+test.json > ripple.json
+test.json > test.jtx
+test.jtx > ripple.app
+test.jtx > ripple.basics
+test.jtx > ripple.beast
+test.jtx > ripple.consensus
+test.jtx > ripple.core
+test.jtx > ripple.json
+test.jtx > ripple.ledger
+test.jtx > ripple.net
+test.jtx > ripple.protocol
+test.jtx > ripple.server
+test.ledger > ripple.app
+test.ledger > ripple.basics
+test.ledger > ripple.beast
+test.ledger > ripple.core
+test.ledger > ripple.ledger
+test.ledger > ripple.protocol
+test.ledger > test.jtx
+test.ledger > test.toplevel
+test.net > ripple.net
+test.net > test.jtx
+test.net > test.toplevel
+test.net > test.unit_test
+test.nodestore > ripple.app
+test.nodestore > ripple.basics
+test.nodestore > ripple.beast
+test.nodestore > ripple.core
+test.nodestore > ripple.nodestore
+test.nodestore > ripple.protocol
+test.nodestore > ripple.unity
+test.nodestore > test.jtx
+test.nodestore > test.toplevel
+test.nodestore > test.unit_test
+test.overlay > ripple.app
+test.overlay > ripple.basics
+test.overlay > ripple.beast
+test.overlay > ripple.core
+test.overlay > ripple.overlay
+test.overlay > ripple.peerfinder
+test.overlay > ripple.protocol
+test.overlay > ripple.shamap
+test.overlay > test.jtx
+test.overlay > test.unit_test
+test.peerfinder > ripple.basics
+test.peerfinder > ripple.beast
+test.peerfinder > ripple.core
+test.peerfinder > ripple.peerfinder
+test.peerfinder > ripple.protocol
+test.peerfinder > test.beast
+test.peerfinder > test.unit_test
+test.protocol > ripple.basics
+test.protocol > ripple.beast
+test.protocol > ripple.crypto
+test.protocol > ripple.json
+test.protocol > ripple.protocol
+test.protocol > test.toplevel
+test.resource > ripple.basics
+test.resource > ripple.beast
+test.resource > ripple.resource
+test.resource > test.unit_test
+test.rpc > ripple.app
+test.rpc > ripple.basics
+test.rpc > ripple.beast
+test.rpc > ripple.core
+test.rpc > ripple.json
+test.rpc > ripple.net
+test.rpc > ripple.nodestore
+test.rpc > ripple.overlay
+test.rpc > ripple.protocol
+test.rpc > ripple.resource
+test.rpc > ripple.rpc
+test.rpc > test.jtx
+test.rpc > test.nodestore
+test.rpc > test.toplevel
+test.server > ripple.app
+test.server > ripple.basics
+test.server > ripple.beast
+test.server > ripple.core
+test.server > ripple.json
+test.server > ripple.rpc
+test.server > ripple.server
+test.server > test.jtx
+test.server > test.toplevel
+test.server > test.unit_test
+test.shamap > ripple.basics
+test.shamap > ripple.beast
+test.shamap > ripple.nodestore
+test.shamap > ripple.protocol
+test.shamap > ripple.shamap
+test.shamap > test.unit_test
+test.toplevel > ripple.json
+test.toplevel > test.csf
+test.unit_test > ripple.basics
+test.unit_test > ripple.beast

Builds/linux/README.md

@@ -6,17 +6,25 @@ builds, including docker based builds for those distributions, please consult
 the [rippled-package-builder](https://github.com/ripple/rippled-package-builder)
 repository. 
 
-Development is regularly done on Ubuntu 16.04 or later. For non Ubuntu
-distributions, the steps below should work be installing the appropriate
-dependencies using that distribution's package management tools.
+Note: Ubuntu 16.04 users may need to update their compiler (see the dependencies
+section). For non Ubuntu distributions, the steps below should work be
+installing the appropriate dependencies using that distribution's package
+management tools.
 
 ## Dependencies
 
+gcc-8 or later is required.
+
 Use `apt-get` to install the dependencies provided by the distribution
 
 ```
 $ apt-get update
-$ apt-get install -y gcc g++ wget git cmake protobuf-compiler libprotobuf-dev libssl-dev
+$ apt-get install -y gcc g++ wget git cmake pkg-config libprotoc-dev protobuf-compiler libprotobuf-dev libssl-dev
+```
+
+To build the software in reporting mode, install these additional dependencies:
+```
+$ apt-get install -y autoconf flex bison
 ```
 
 Advanced users can choose to install newer versions of gcc, or the clang compiler.
@@ -28,9 +36,8 @@ protobuf will give errors.
 Boost 1.70 or later is required. We recommend downloading and compiling boost
 with the following process: After changing to the directory where
 you wish to download and compile boost, run
-
 ``` 
-$ wget https://dl.bintray.com/boostorg/release/1.70.0/source/boost_1_70_0.tar.gz
+$ wget https://boostorg.jfrog.io/artifactory/main/release/1.70.0/source/boost_1_70_0.tar.gz
 $ tar -xzf boost_1_70_0.tar.gz
 $ cd boost_1_70_0
 $ ./bootstrap.sh
@@ -91,7 +98,7 @@ export BOOST_ROOT=~/boost_1_70_0
 Alternatively, you can add `DBOOST_ROOT=~/boost_1_70_0` to the command line when
 invoking `cmake`.
 
-### Generate and Build
+### Generate Configuration
 
 All builds should be done in a separate directory from the source tree root 
 (a subdirectory is fine). For example, from the root of the ripple source tree:
@@ -107,6 +114,10 @@ followed by:
 cmake -DCMAKE_BUILD_TYPE=Debug ..
 ```
 
+If your operating system does not provide static libraries (Arch Linux, and 
+Manjaro Linux, for example), you must configure a non-static build by adding
+`-Dstatic=OFF` to the above cmake line.
+
 `CMAKE_BUILD_TYPE` can be changed as desired for `Debug` vs.
 `Release` builds (all four standard cmake build types are supported).
 
@@ -115,20 +126,6 @@ To select a different compiler (most likely gcc will be found by default), pass
 `-DCMAKE_CXX_COMPILER=</path/to/cxx-compiler>` when configuring. If you prefer, 
 you can instead set `CC` and `CXX` environment variables which cmake will honor.
 
-Once you have generated the build system, you can run the build via cmake:
-
-```
-cmake --build . -- -j <parallel jobs>
-```
-
-the `-j` parameter in this example tells the build tool to compile several
-files in parallel. This value should be chosen roughly based on the number of
-cores you have available and/or want to use for building.
-
-When the build completes succesfully, you will have a `rippled` executable in
-the current directory, which can be used to connect to the network (when
-properly configured) or to run unit tests.
-
 #### Options During Configuration:
 
 The CMake file defines a number of configure-time options which can be
@@ -146,10 +143,28 @@ testing and running.
 * `-Dsan=thread` to enable the thread sanitizer with clang
 * `-Dsan=address` to enable the address sanitizer with clang
 * `-Dstatic=ON` to enable static linking library dependencies
+* `-Dreporting=ON` to build code necessary for reporting mode (defaults to OFF)
 
 Several other infrequently used options are available - run `ccmake` or
 `cmake-gui` for a list of all options.
 
+### Build
+
+Once you have generated the build system, you can run the build via cmake:
+
+```
+cmake --build . -- -j <parallel jobs>
+```
+
+the `-j` parameter in this example tells the build tool to compile several
+files in parallel. This value should be chosen roughly based on the number of
+cores you have available and/or want to use for building.
+
+When the build completes successfully, you will have a `rippled` executable in
+the current directory, which can be used to connect to the network (when
+properly configured) or to run unit tests.
+
+
 #### Optional Installation
 
 The rippled cmake build supports an installation target that will install
@@ -224,5 +239,3 @@ change the `/opt/local` module path above to match your chosen installation pref
 `rippled` builds a set of unit tests into the server executable. To run these unit
 tests after building, pass the `--unittest` option to the compiled `rippled`
 executable. The executable will exit with summary info after running the unit tests.
-
-

Builds/macos/README.md

@@ -1,231 +1,3 @@
-# macos Build Instructions
-
-## Important
-
-We don't recommend macos for rippled production use at this time. Currently, the
-Ubuntu platform has received the highest level of quality assurance and
-testing. That said, macos is suitable for many development/test tasks.
-
-## Prerequisites
-
-You'll need macos 10.8 or later.
-
-To clone the source code repository, create branches for inspection or
-modification, build rippled using clang, and run the system tests you will need
-these software components:
-
-* [XCode](https://developer.apple.com/xcode/)
-* [Homebrew](http://brew.sh/)
-* [Boost](http://boost.org/)
-* other misc utilities and libraries installed via homebrew
-
-## Install Software
-
-### Install XCode
-
-If not already installed on your system, download and install XCode using the
-appstore or by using [this link](https://developer.apple.com/xcode/).
-
-For more info, see "Step 1: Download and Install the Command Line Tools"
-[here](http://www.moncefbelyamani.com/how-to-install-xcode-homebrew-git-rvm-ruby-on-mac)
-
-The command line tools can be installed through the terminal with the command:
-
-```
-xcode-select --install
-```
-
-### Install Homebrew
-
-> "[Homebrew](http://brew.sh/) installs the stuff you need that Apple didn’t."
-
-Open a terminal and type:
-
-```
-ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
-```
-
-For more info, see "Step 2: Install Homebrew"
-[here](http://www.moncefbelyamani.com/how-to-install-xcode-homebrew-git-rvm-ruby-on-mac#step-2)
-
-### Install Dependencies Using Homebrew
-
-`brew` will generally install the latest stable version of any package, which
-should satisfy the the minimum version requirements for rippled.
-
-```
-brew update
-brew install git cmake pkg-config protobuf openssl ninja
-```
-
-### Build Boost
-
-Boost 1.70 or later is required.
-
-We want to compile boost with clang/libc++
-
-Download [a release](https://dl.bintray.com/boostorg/release/1.70.0/source/boost_1_70_0.tar.bz2)
-
-Extract it to a folder, making note of where, open a terminal, then:
-
-```
-./bootstrap.sh
-./b2 cxxflags="-std=c++14"  visibility=global
-```
-
-Create an environment variable `BOOST_ROOT` in one of your `rc` files, pointing
-to the root of the extracted directory.
-
-### Dependencies for Building Source Documentation
-
-Source code documentation is not required for running/debugging rippled. That
-said, the documentation contains some helpful information about specific
-components of the application. For more information on how to install and run
-the necessary components, see [this document](../../docs/README.md)
-
-## Build
-
-### Clone the rippled repository
-
-From a shell:
-
-```
-git clone git@github.com:ripple/rippled.git
-cd rippled
-```
-
-For a stable release, choose the `master` branch or one of the tagged releases
-listed on [GitHub](https://github.com/ripple/rippled/releases GitHub). 
-
-```
-git checkout master
-```
-
-or to test the latest release candidate, choose the `release` branch.
-
-```
-git checkout release
-```
-
-If you are doing development work and want the latest set of untested
-features, you can consider using the `develop` branch instead.
-
-```
-git checkout develop
-```
-
-### Configure Library Paths
-
-If you didn't persistently set the `BOOST_ROOT` environment variable to the
-root of the extracted directory above, then you should set it temporarily.
-
-For example, assuming your username were `Abigail` and you extracted Boost
-1.70.0 in `/Users/Abigail/Downloads/boost_1_70_0`, you would do for any
-shell in which you want to build:
-
-```
-export BOOST_ROOT=/Users/Abigail/Downloads/boost_1_70_0
-```
-
-### Generate and Build
-
-For simple command line building we recommend using the *Unix Makefile* or
-*Ninja* generator with cmake. All builds should be done in a separate directory
-from the source tree root (a subdirectory is fine). For example, from the root
-of the ripple source tree:
-
-```
-mkdir my_build
-cd my_build
-```
-
-followed by:
-
-```
-cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Debug ..
-```
-
-or
-
-```
-cmake -G "Ninja" -DCMAKE_BUILD_TYPE=Debug ..
-```
-
-`CMAKE_BUILD_TYPE` can be changed as desired for `Debug` vs.
-`Release` builds (all four standard cmake build types are supported).
-
-Once you have generated the build system, you can run the build via cmake:
-
-```
-cmake --build . -- -j 4
-```
-
-the `-j` parameter in this example tells the build tool to compile several
-files in parallel. This value should be chosen roughly based on the number of
-cores you have available and/or want to use for building.
-
-When the build completes succesfully, you will have a `rippled` executable in
-the current directory, which can be used to connect to the network (when
-properly configured) or to run unit tests.
-
-If you prefer to have an XCode project to use for building, ask CMake to
-generate that instead:
-
-```
-cmake -GXcode ..
-```
-
-After generation succeeds, the xcode project file can be opened and used to
-build/debug. However, just as with other generators, cmake knows how to build
-using the xcode project as well:
-
-```
-cmake --build . -- -jobs 4
-```
-
-This will invoke the `xcodebuild` utility to compile the project. See `xcodebuild
---help` for details about build options.
-
-#### Optional installation
-
-If you'd like to install the artifacts of the build, we have preliminary
-support for standard CMake installation targets. We recommend explicitly
-setting the installation location when configuring, e.g.:
-
-```
-cmake -DCMAKE_INSTALL_PREFIX=/opt/local ..
-```
-
-(change the destination as desired), and then build the `install` target:
-
-```
-cmake --build . --target install -- -jobs 4
-```
-
-#### Options During Configuration:
-
-The CMake file defines a number of configure-time options which can be
-examined by running `cmake-gui` or `ccmake` to generated the build. In
-particular, the `unity` option allows you to select between the unity and
-non-unity builds. `unity` builds are faster to compile since they combine
-multiple sources into a single compiliation unit - this is the default if you
-don't specify. `nounity` builds can be helpful for detecting include omissions
-or for finding other build-related issues, but aren't generally needed for
-testing and running.
-
-* `-Dunity=ON` to enable/disable unity builds (defaults to ON) 
-* `-Dassert=ON` to enable asserts
-* `-Djemalloc=ON` to enable jemalloc support for heap checking
-* `-Dsan=thread` to enable the thread sanitizer with clang
-* `-Dsan=address` to enable the address sanitizer with clang
-
-Several other infrequently used options are available - run `ccmake` or
-`cmake-gui` for a list of all options.
-
-## Unit Tests (Recommended)
-
-`rippled` builds a set of unit tests into the server executable. To run these unit
-tests after building, pass the `--unittest` option to the compiled `rippled`
-executable. The executable will exit with summary info after running the unit tests.
-
+# macOS Build Instructions
 
+[Build and Run rippled on macOS](https://xrpl.org/build-run-rippled-macos.html)

Jenkinsfile

@@ -1,704 +0,0 @@
-#!/usr/bin/env groovy
-
-import groovy.json.JsonOutput
-import java.text.*
-
-all_status = [:]
-commit_id = ''
-git_fork = 'ripple'
-git_repo = 'rippled'
-//
-// this is not the actual token, but an ID/key into the jenkins
-// credential store which httpRequest can access.
-//
-github_cred = '6bd3f3b9-9a35-493e-8aef-975403c82d3e'
-//
-// root API url for our repo (default, overriden below)
-//
-github_api = 'https://api.github.com/repos/ripple/rippled'
-
-try {
-    stage ('Startup Checks') {
-        // here we check the commit author against collaborators
-        // we need a node to do this because readJSON requires
-        // a filesystem (although we just pass it text)
-        node {
-            checkout scm
-            commit_id = getCommitID()
-            //
-            // NOTE this getUserRemoteConfigs call requires a one-time
-            // In-process Script Approval (configure jenkins). We need this
-            // to detect the remote repo to interact with via the github API.
-            //
-            def remote_url = scm.getUserRemoteConfigs()[0].getUrl()
-            if (remote_url) {
-                echo "GIT URL scm: $remote_url"
-                git_fork = remote_url.tokenize('/')[2]
-                git_repo = remote_url.tokenize('/')[3].split('\\.')[0]
-                echo "GIT FORK: $git_fork"
-                echo "GIT REPO: $git_repo"
-                github_api = "https://api.github.com/repos/${git_fork}/${git_repo}"
-                echo "API URL REPO: $github_api"
-            }
-
-            if (env.CHANGE_AUTHOR) {
-                def collab_found = false;
-                //
-                // this means we have some sort of PR , so verify the author
-                //
-                echo "CHANGE AUTHOR ---> $CHANGE_AUTHOR"
-                echo "CHANGE TARGET ---> $CHANGE_TARGET"
-                echo "CHANGE ID ---> $CHANGE_ID"
-                //
-                // check the commit author against collaborators
-                // we need a node to do this because readJSON requires
-                // a filesystem (although we just pass it text)
-                //
-                def response = httpRequest(
-                    timeout: 10,
-                    authentication: github_cred,
-                    url: "${github_api}/collaborators")
-                def collab_data = readJSON(
-                    text: response.content)
-                for (collaborator in collab_data) {
-                    if (collaborator['login'] == "$CHANGE_AUTHOR") {
-                        echo "$CHANGE_AUTHOR is a collaborator!"
-                        collab_found = true;
-                        break;
-                    }
-                }
-
-                if (! collab_found) {
-                    echo "$CHANGE_AUTHOR is not a collaborator - waiting for manual approval."
-                    sendToSlack("A <${env.BUILD_URL}|jenkins job (PR)> is waiting for approval - please review.")
-
-                    try {
-                        httpRequest(
-                            timeout: 10,
-                            authentication: github_cred,
-                            url: getCommentURL(),
-                            contentType: 'APPLICATION_JSON',
-                            httpMode: 'POST',
-                            requestBody: JsonOutput.toJson([
-                                body: """
-**Thank you** for your submission. It will be reviewed soon and submitted for processing in CI.
-"""
-                            ])
-                        )
-                    }
-                    catch (e) {
-                        echo 'had a problem interacting with github...comments are probably not updated'
-                    }
-
-                    try {
-                        input (
-                            message: "User $CHANGE_AUTHOR has submitted PR #$CHANGE_ID. " +
-                                "**Please review** the changes for any CI/security concerns " +
-                                "and then decide whether to proceed with building.")
-                    }
-                    catch(e) {
-                        def user = e.getCauses()[0].getUser().toString()
-                        all_status['startup'] = [
-                            false,
-                            'Approval Check',
-                            "Build aborted by [${user}]",
-                            "[console](${env.BUILD_URL}/console)"]
-                        error "Aborted by: [${user}]"
-                    }
-                }
-            }
-        }
-
-        // UNCOMMENT the following if we want one message for every job...
-        //node('rippled-dev') {
-        //    sendToSlack("<${env.BUILD_URL}|Job ${env.BUILD_TAG}> has started.")
-        //}
-    }
-
-    stage ('Parallel Build') {
-        String[][] variants = [
-            ['gcc.Release'    ,'-Dassert=ON'     ,'MANUAL_TESTS=true'     ],
-            ['gcc.Debug'      ,'-Dcoverage=ON'   ,'TARGET=coverage_report', 'SKIP_TESTS=true'],
-            ['docs'           ,''                ,'TARGET=docs'           ],
-            ['msvc.Debug'                                                 ],
-            ['msvc.Debug'     ,''                ,'NINJA_BUILD=true'      ],
-            ['msvc.Debug'     ,'-Dunity=OFF'                              ],
-            ['msvc.Release'                                               ],
-            ['clang.Debug'                                                ],
-            ['clang.Debug'    ,'-Dunity=OFF'                              ],
-            ['gcc.Debug'                                                  ],
-            ['gcc.Debug'      ,'-Dunity=OFF'                              ],
-            ['clang.Release'  ,'-Dassert=ON'                              ],
-            ['gcc.Release'    ,'-Dassert=ON'                              ],
-            ['gcc.Debug'      ,'-Dstatic=OFF'                             ],
-            ['gcc.Debug'      ,'-Dstatic=OFF -DBUILD_SHARED_LIBS=ON'      ],
-            ['gcc.Debug'      ,''                ,'NINJA_BUILD=true'      ],
-            ['clang.Debug'      ,'-Dunity=OFF -Dsan=address'   ,'PARALLEL_TESTS=false', 'DEBUGGER=false'],
-            ['clang.Debug'      ,'-Dunity=OFF -Dsan=undefined' ,'PARALLEL_TESTS=false'],
-            // TODO - tsan runs currently fail/hang
-            //['clang.Debug'      ,'-Dunity=OFF -Dsan=thread'    ,'PARALLEL_TESTS=false'],
-        ]
-
-        // create a map of all builds
-        // that we want to run. The map
-        // is string keys and node{} object values
-        def builds = [:]
-        for (int index = 0; index < variants.size(); index++) {
-            def bldtype = variants[index][0]
-            def cmake_extra = variants[index].size() > 1 ? variants[index][1] : ''
-            def bldlabel = bldtype + cmake_extra
-            def extra_env = variants[index].size() > 2 ? variants[index][2..-1] : []
-            for (int j = 0; j < extra_env.size(); j++) {
-                bldlabel += "_" + extra_env[j]
-            }
-            bldlabel = bldlabel.replace('-', '_')
-            bldlabel = bldlabel.replace(' ', '')
-            bldlabel = bldlabel.replace('=', '_')
-
-            def compiler = getFirstPart(bldtype)
-            def config = getSecondPart(bldtype)
-            def target = 'install' // currently ignored for windows builds
-            if (compiler == 'docs') {
-                compiler = 'gcc'
-                config = 'Release'
-                target = 'docs'
-            }
-            def cc =
-                (compiler == 'clang') ? '/opt/llvm-5.0.1/bin/clang' : 'gcc'
-            def cxx =
-                (compiler == 'clang') ? '/opt/llvm-5.0.1/bin/clang++' : 'g++'
-            def ucc = isNoUnity(cmake_extra) ? 'true' : 'false'
-            def node_type =
-                (compiler == 'msvc') ? 'rippled-win' : 'rippled-dev'
-            // the default disposition for parallel test..disabled
-            // for coverage, enabled otherwise. Can still be overridden
-            // by explicitly setting with extra env settings above.
-            def pt = isCoverage(cmake_extra) ? 'false' : 'true'
-            def max_minutes = 25
-
-            def env_vars = [
-                "TARGET=${target}",
-                "BUILD_TYPE=${config}",
-                "COMPILER=${compiler}",
-                "PARALLEL_TESTS=${pt}",
-                'BUILD=cmake',
-                "MAX_TIME=${max_minutes}m",
-                "BUILD_DIR=${bldlabel}",
-                "CMAKE_EXTRA_ARGS=-Dwerr=ON ${cmake_extra}",
-                'VERBOSE_BUILD=true']
-
-            builds[bldlabel] = {
-                node(node_type) {
-                    checkout scm
-                    dir ('build') {
-                        deleteDir()
-                    }
-                    def cdir = upDir(pwd())
-                    echo "BASEDIR: ${cdir}"
-                    echo "COMPILER: ${compiler}"
-                    echo "BUILD_TYPE: ${config}"
-                    echo "USE_CC: ${ucc}"
-                    env_vars.addAll([
-                        "NIH_CACHE_ROOT=${cdir}/.nih_c"])
-                    if (compiler == 'msvc') {
-                        env_vars.addAll([
-                            'BOOST_ROOT=c:\\lib\\boost_1_70',
-                            'PROJECT_NAME=rippled',
-                            'MSBUILDDISABLENODEREUSE=1',  // this ENV setting is probably redundant since we also pass /nr:false to msbuild
-                            'OPENSSL_ROOT=c:\\OpenSSL-Win64'])
-                    }
-                    else {
-                        env_vars.addAll([
-                            'NINJA_BUILD=false',
-                            "CCACHE_BASEDIR=${cdir}",
-                            'PLANTUML_JAR=/opt/plantuml/plantuml.jar',
-                            'APP_ARGS=--unittest-ipv6',
-                            'CCACHE_NOHASHDIR=true',
-                            "CC=${cc}",
-                            "CXX=${cxx}",
-                            'LCOV_ROOT=""',
-                            'PATH+CMAKE_BIN=/opt/local/cmake',
-                            'GDB_ROOT=/opt/local/gdb',
-                            'BOOST_ROOT=/opt/local/boost_1_70_0',
-                            "USE_CCACHE=${ucc}"])
-                    }
-
-                    if (extra_env.size() > 0) {
-                        env_vars.addAll(extra_env)
-                    }
-
-                    // try to figure out codecov token to use. Look for
-                    // MY_CODECOV_TOKEN id first so users can set that
-                    // on job scope but then default to RIPPLED_CODECOV_TOKEN
-                    // which should be globally scoped
-                    def codecov_token = ''
-                    try {
-                        withCredentials( [string( credentialsId: 'MY_CODECOV_TOKEN', variable: 'CODECOV_TOKEN')]) {
-                            codecov_token = env.CODECOV_TOKEN
-                        }
-                    }
-                    catch (e) {
-                        // this might throw when MY_CODECOV_TOKEN doesn't exist
-                    }
-                    if (codecov_token == '') {
-                        withCredentials( [string( credentialsId: 'RIPPLED_CODECOV_TOKEN', variable: 'CODECOV_TOKEN')]) {
-                            codecov_token = env.CODECOV_TOKEN
-                        }
-                    }
-                    env_vars.addAll(["CODECOV_TOKEN=${codecov_token}"])
-
-                    withEnv(env_vars) {
-                        myStage(bldlabel)
-                        def thrown = '';
-                        try {
-                            timeout(
-                              time: max_minutes * 2,
-                              units: 'MINUTES')
-                            {
-                                if (compiler == 'msvc') {
-                                    powershell "Remove-Item -Path \"${bldlabel}.txt\" -Force -ErrorAction Ignore"
-                                    // we capture stdout to variable because I could
-                                    // not figure out how to make powershell redirect internally
-                                    output = powershell (
-                                            returnStdout: true,
-                                            script: windowsBuildCmd())
-                                    // if the powershell command fails (has nonzero exit)
-                                    // then the command above throws, we don't get our output,
-                                    // and we never create this output file.
-                                    //  SEE https://issues.jenkins-ci.org/browse/JENKINS-44930
-                                    // Alternatively, figure out how to reliably redirect
-                                    // all output above to a file (Start/Stop transcript does not work)
-                                    writeFile(
-                                        file: "${bldlabel}.txt",
-                                        text: output)
-                                }
-                                else {
-                                    sh "rm -fv ${bldlabel}.txt"
-                                    // execute the bld command in a redirecting shell
-                                    // to capture output
-                                    sh redhatBuildCmd(bldlabel)
-                                }
-                            }
-                        }
-                        catch(e) {
-                           thrown = "${e}"
-                           throw e
-                        }
-                        finally {
-                            if (bldtype == 'docs') {
-                                publishHTML(
-                                    allowMissing: true,
-                                    alwaysLinkToLastBuild: true,
-                                    keepAll: true,
-                                    reportName:  'Doxygen',
-                                    reportDir:   "build/${bldlabel}/html_doc",
-                                    reportFiles: 'index.html')
-                            }
-                            if (isCoverage(cmake_extra)) {
-                                publishHTML(
-                                    allowMissing: true,
-                                    alwaysLinkToLastBuild: false,
-                                    keepAll: true,
-                                    reportName:  'Coverage',
-                                    reportDir:   "build/${bldlabel}/coverage",
-                                    reportFiles: 'index.html')
-                            }
-                            def envs = ''
-                            for (int j = 0; j < extra_env.size(); j++) {
-                                envs += ", <br/>" + extra_env[j]
-                            }
-                            def cmake_txt = cmake_extra
-                            if (cmake_txt != '') {
-                                cmake_txt = " <br/>" + cmake_txt
-                            }
-                            def st = reportStatus(bldlabel, bldtype + cmake_txt + envs, env.BUILD_URL, thrown)
-                            lock('rippled_dev_status') {
-                                all_status[bldlabel] = st
-                            }
-                            if (thrown == '') {
-                                assert st[0] : "Unit Test Failures"
-                            }
-                        } //try-catch-finally
-                    } //withEnv
-                } //node
-            } //builds item
-        } //for variants
-
-        // this actually executes all the builds we just defined
-        // above, in parallel as slaves are available
-        parallel builds
-    }
-}
-finally {
-    // anything here should run always...
-    stage ('Final Status') {
-        node {
-            def results = makeResultText()
-            try {
-                def res = getCommentID() //get array return b/c jenkins does not allow multiple direct return/assign
-                def comment_id = res[0]
-                def url_comment = res[1]
-                def mode = 'PATCH'
-                if (comment_id == 0) {
-                    echo 'no existing status comment found'
-                    mode =  'POST'
-                }
-
-                def body = JsonOutput.toJson([
-                    body: results
-                ])
-
-                response = httpRequest(
-                    timeout: 10,
-                    authentication: github_cred,
-                    url: url_comment,
-                    contentType: 'APPLICATION_JSON',
-                    httpMode: mode,
-                    requestBody: body)
-            }
-            catch (e) {
-                echo 'had a problem interacting with github...status is probably not updated'
-            }
-        }
-    }
-}
-
-// ---------------
-// util functions
-// ---------------
-def myStage(name) {
-    echo """
-+++++++++++++++++++++++++++++++++++++++++
-  >> building ${name}
-+++++++++++++++++++++++++++++++++++++++++
-"""
-}
-
-def printGitInfo(id, log) {
-    echo """
-+++++++++++++++++++++++++++++++++++++++++
-  >> Building commit ID ${id}
-  >>
-${log}
-+++++++++++++++++++++++++++++++++++++++++
-"""
-}
-
-def makeResultText () {
-    def start_time = new Date()
-    def sdf = new SimpleDateFormat('yyyyMMdd - HH:mm:ss')
-    def datestamp = sdf.format(start_time)
-
-    def results = """
-## Jenkins Build Summary
-
-Built from [this commit](https://github.com/${git_fork}/${git_repo}/commit/${commit_id})
-
-Built at __${datestamp}__
-
-### Test Results
-
-Build Type | Log | Result | Status
----------- | --- | ------ | ------
-"""
-    for ( e in all_status) {
-        results += e.value[1] + ' | ' + e.value[3] + ' | ' + e.value[2] + ' | ' +
-            (e.value[0] ? 'PASS :white_check_mark: ' : 'FAIL :red_circle: ') + '\n'
-    }
-    results += '\n'
-    echo 'FINAL BUILD RESULTS'
-    echo results
-    results
-}
-
-def getCommentURL () {
-    def url_c = ''
-    if (env.CHANGE_ID && env.CHANGE_ID ==~ /\d+/) {
-        //
-        // CHANGE_ID indicates we are building a PR
-        // find PR comments
-        //
-        def resp = httpRequest(
-            timeout: 10,
-            authentication: github_cred,
-            url: "${github_api}/pulls/$CHANGE_ID")
-        def result = readJSON(text: resp.content)
-        //
-        // follow issue comments link
-        //
-        url_c = result['_links']['issue']['href'] + '/comments'
-    }
-    else {
-        //
-        // if not a PR, just search comments for our commit ID
-        //
-        url_c =
-            "${github_api}/commits/${commit_id}/comments"
-    }
-    url_c
-}
-
-def getCommentID () {
-    def url_c = getCommentURL()
-    def response = httpRequest(
-        timeout: 10,
-        authentication: github_cred,
-        url: url_c)
-    def data = readJSON(text: response.content)
-    def comment_id = 0
-    // see if we can find and existing comment here with
-    // a heading that matches ours...
-    for (comment in data) {
-        if (comment['body'] =~ /(?m)^##\s+Jenkins Build/) {
-            comment_id = comment['id']
-            echo "existing status comment ${comment_id} found"
-            url_c = comment['url']
-            break;
-        }
-    }
-    [comment_id, url_c]
-}
-
-def getCommitID () {
-    def cid = sh (
-        script: 'git rev-parse HEAD',
-        returnStdout: true)
-    cid = cid.trim()
-    echo "commit ID is ${cid}"
-    commit_log = sh (
-         script: "git show --name-status ${cid}",
-         returnStdout: true)
-    printGitInfo (cid, commit_log)
-    cid
-}
-
-@NonCPS
-def getResults(text, label) {
-    // example:
-    ///   194.5s, 154 suites, 948 cases, 360485 tests total, 0 failures
-    // or build log format:
-    //    [msvc.release] 71.3s, 162 suites, 995 cases, 318901 tests total, 1 failure
-    def matcher =
-        text == '' ?
-        manager.getLogMatcher(/\[${label}\].+?(\d+) case[s]?, (\d+) test[s]? total, (\d+) (failure(s?))/) :
-        text =~ /(\d+) case[s]?, (\d+) test[s]? total, (\d+) (failure(s?))/
-    matcher ? matcher[0][1] + ' cases, ' + matcher[0][3] + ' failed' : 'no test results'
-}
-
-@NonCPS
-def getFailures(text, label) {
-    // [see above for format]
-    def matcher =
-        text == '' ?
-        manager.getLogMatcher(/\[${label}\].+?(\d+) test[s]? total, (\d+) (failure(s?))/) :
-        text =~ /(\d+) test[s]? total, (\d+) (failure(s?))/
-    // if we didn't match, then return -1 since something is
-    // probably wrong, e.g. maybe the build failed...
-    matcher ? matcher[0][2] as Integer : -1i
-}
-
-@NonCPS
-def getTime(text, label) {
-    // look for text following a label 'real' for
-    // wallclock time. Some `time`s report fractional
-    // seconds and we can omit those in what we report
-    def matcher =
-        text == '' ?
-        manager.getLogMatcher(/(?m)^\[${label}\]\s+real\s+(.+)\.(\d+?)[s]?/) :
-        text =~ /(?m)^real\s+(.+)\.(\d+?)[s]?/
-    if (matcher) {
-        return matcher[0][1] + 's'
-    }
-
-    // alternatively, look for powershell elapsed time
-    // format, e.g. :
-    //    TotalSeconds      : 523.2140529
-    def matcher2 =
-        text == '' ?
-        manager.getLogMatcher(/(?m)^\[${label}\]\s+TotalSeconds\s+:\s+(\d+)\.(\d+?)?/) :
-        text =~ /(?m)^TotalSeconds\s+:\s+(\d+)\.(\d+?)?/
-    matcher2 ? matcher2[0][1] + 's' : 'n/a'
-}
-
-@NonCPS
-def getFirstPart(bld) {
-    def matcher = bld =~ /^(.+?)\.(.+)$/
-    matcher ? matcher[0][1] : bld
-}
-
-@NonCPS
-def isNoUnity(bld) {
-    def matcher = bld =~ /-Dunity=(off|OFF)/
-    matcher ? true : false
-}
-
-@NonCPS
-def isCoverage(bld) {
-    def matcher = bld =~ /-Dcoverage=(on|ON)/
-    matcher ? true : false
-}
-
-@NonCPS
-def getSecondPart(bld) {
-    def matcher = bld =~ /^(.+?)\.(.+)$/
-    matcher ? matcher[0][2] : bld
-}
-
-// because I can't seem to find path manipulation
-// functions in groovy....
-@NonCPS
-def upDir(path) {
-    def matcher = path =~ /^(.+)[\/\\](.+?)/
-    matcher ? matcher[0][1] : path
-}
-
-def sendToSlack(message) {
-    try {
-        withCredentials( [string( credentialsId: 'RIPPLED_SLACK_INCOMING_URL', variable: 'SLACK_URL')]) {
-            // I was unable to make httpRequest method work with the
-            // formdata required by slack API, so resorting
-            // to curl commands...
-            sh '''\
-CONTENT=$(tr -d '[\n]' <<JSON
-  payload={
-    "channel": "#cpp-notifications",
-    "username": "JenkinsCI",
-    "text": "''' + message + '''",
-    "icon_emoji": ":jenkins:"}
-JSON
-)
-curl ${SLACK_URL} --data-urlencode "${CONTENT}"
-'''
-        }
-    }
-    catch (e) {
-        echo "had a problem posting to slack: ${e}"
-    }
-}
-
-// the shell command used for building on redhat
-def redhatBuildCmd(bldlabel) {
-'''\
-#!/bin/bash
-set -ex
-log_file=''' + "${bldlabel}.txt" + '''
-exec 3>&1 1>>${log_file} 2>&1
-ccache -s
-source /opt/rh/devtoolset-7/enable
-/usr/bin/time -p ./bin/ci/ubuntu/build-and-test.sh 2>&1
-ccache -s
-'''
-}
-
-// the powershell command used for building
-def windowsBuildCmd() {
-'''
-# Enable streams 3-6
-$WarningPreference = 'Continue'
-$VerbosePreference = 'Continue'
-$DebugPreference = 'Continue'
-$InformationPreference = 'Continue'
-
-Invoke-BatchFile "${env:ProgramFiles(x86)}\\Microsoft Visual Studio\\2017\\Community\\VC\\Auxiliary\\Build\\vcvarsall.bat" x86_amd64
-Get-ChildItem env:* | Sort-Object name
-cl
-cmake --version
-New-Item -ItemType Directory -Force -Path "build/$env:BUILD_DIR" -ErrorAction Stop
-$sw = [Diagnostics.Stopwatch]::StartNew()
-try {
-    Push-Location "build/$env:BUILD_DIR"
-    if ($env:NINJA_BUILD -eq "true") {
-        Invoke-Expression "& cmake -G`"Ninja`" -DCMAKE_BUILD_TYPE=$env:BUILD_TYPE -DCMAKE_VERBOSE_MAKEFILE=ON $env:CMAKE_EXTRA_ARGS ../.."
-    }
-    else {
-        Invoke-Expression "& cmake -G`"Visual Studio 15 2017 Win64`" -DCMAKE_VERBOSE_MAKEFILE=ON $env:CMAKE_EXTRA_ARGS ../.."
-    }
-    if ($LastExitCode -ne 0) { throw "CMake failed" }
-
-    ## as of 01/2018, DO NOT USE cmake to run the actual build step. for some
-    ## reason, cmake spawning the build under jenkins causes MSBUILD/ninja to
-    ## get stuck at the end of the build. Perhaps cmake is spawning
-    ## incorrectly or failing to pass certain params
-
-    if ($env:NINJA_BUILD -eq "true") {
-        ninja -j $env:NUMBER_OF_PROCESSORS -v
-    }
-    else {
-        msbuild /fl /m /nr:false /p:Configuration="$env:BUILD_TYPE" /p:Platform=x64 /p:GenerateFullPaths=True /v:normal /nologo /clp:"ShowCommandLine;DisableConsoleColor" "$env:PROJECT_NAME.vcxproj"
-    }
-    if ($LastExitCode -ne 0) { throw "CMake build failed" }
-
-    $exe = "./$env:BUILD_TYPE/$env:PROJECT_NAME"
-    if ($env:NINJA_BUILD -eq "true") {
-        $exe = "./$env:PROJECT_NAME"
-    }
-    "Exe is at $exe"
-    $params = '--unittest', '--quiet', '--unittest-log'
-    if ($env:PARALLEL_TESTS -eq "true") {
-        $params = $params += "--unittest-jobs=$env:NUMBER_OF_PROCESSORS"
-    }
-    & $exe $params
-    if ($LastExitCode -ne 0) { throw "Unit tests failed" }
-}
-catch {
-    throw
-}
-finally {
-    $sw.Stop()
-    $sw.Elapsed
-    Pop-Location
-}
-'''
-}
-
-// post processing step after each build:
-//   * archives the log file
-//   * adds short description/status to build status
-//   * returns an array of result info to add to the all_build summary
-def reportStatus(label, type, bldurl, errmsg) {
-    def outstr = ''
-    def loglink = "[console](${bldurl}/console)"
-    def logfile = "${label}.txt"
-    if ( fileExists(logfile) ) {
-        archiveArtifacts( artifacts: logfile )
-        outstr = readFile(logfile)
-        loglink = "[logfile](${bldurl}/artifact/${logfile})"
-    }
-    def st = getResults(outstr, label)
-    def time = getTime(outstr, label)
-    def fail_count = getFailures(outstr, label)
-    outstr = null
-    def txtcolor =
-        (fail_count == 0 && errmsg == '') ? 'DarkGreen' : 'Crimson'
-    def shortbld = label
-    // this is just an attempt to shorten the
-    // summary text label to the point of absurdity..
-    shortbld = shortbld.replace('Debug', 'dbg')
-    shortbld = shortbld.replace('Release', 'rel')
-    shortbld = shortbld.replace('true', 'Y')
-    shortbld = shortbld.replace('false', 'N')
-    shortbld = shortbld.replace('Dcoverage', 'cov')
-    shortbld = shortbld.replace('Dassert', 'asrt')
-    shortbld = shortbld.replace('Dunity', 'unty')
-    shortbld = shortbld.replace('Dsan=address', 'asan')
-    shortbld = shortbld.replace('Dsan=thread', 'tsan')
-    shortbld = shortbld.replace('Dsan=undefined', 'ubsan')
-    shortbld = shortbld.replace('PARALLEL_TEST', 'PL')
-    shortbld = shortbld.replace('MANUAL_TESTS', 'MAN')
-    shortbld = shortbld.replace('NINJA_BUILD', 'ninja')
-    shortbld = shortbld.replace('DEBUGGER', 'gdb')
-    shortbld = shortbld.replace('ON', 'Y')
-    shortbld = shortbld.replace('OFF', 'N')
-    def stattext = "${st}, t: ${time}"
-    if (fail_count <= 0 && errmsg != '') {
-        stattext += " [BAD EXIT]"
-    }
-    manager.addShortText(
-        "${shortbld}: ${stattext}",
-        txtcolor,
-        'white',
-        '0px',
-        'white')
-    [fail_count == 0 && errmsg == '', type, stattext, loglink]
-}
-

LICENSE.md

@@ -0,0 +1,17 @@
+ISC License 
+
+Copyright (c) 2011, Arthur Britto, David Schwartz, Jed McCaleb, Vinnie Falco, Bob Way, Eric Lombrozo, Nikolaos D. Bougalis, Howard Hinnant.
+Copyright (c) 2012-2020, the XRP Ledger developers.
+
+Permission to use, copy, modify, and distribute this software for any
+purpose with or without fee is hereby granted, provided that the above
+copyright notice and this permission notice appear in all copies.
+
+THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+

README.md

@@ -1,40 +1,40 @@
 # The XRP Ledger
 
-The XRP Ledger is a decentralized cryptographic ledger powered by a network of peer-to-peer servers. The XRP Ledger uses a novel Byzantine Fault Tolerant consensus algorithm to settle and record transactions in a secure distributed database without a central operator.
+The [XRP Ledger](https://xrpl.org/) is a decentralized cryptographic ledger powered by a network of peer-to-peer nodes. The XRP Ledger uses a novel Byzantine Fault Tolerant consensus algorithm to settle and record transactions in a secure distributed database without a central operator.
 
 ## XRP
-XRP is a public, counterparty-less asset native to the XRP Ledger, and is designed to bridge the many different currencies in use worldwide. XRP is traded on the open-market and is available for anyone to access. The XRP Ledger was created in 2012 with a finite supply of 100 billion units of XRP. Its creators gifted 80 billion XRP to a company, now called [Ripple](https://ripple.com/), to develop the XRP Ledger and its ecosystem.  Ripple uses XRP the help build the Internet of Value, ushering in a world in which money moves as fast and efficiently as information does today.
+[XRP](https://xrpl.org/xrp.html) is a public, counterparty-free asset native to the XRP Ledger, and is designed to bridge the many different currencies in use worldwide. XRP is traded on the open-market and is available for anyone to access. The XRP Ledger was created in 2012 with a finite supply of 100 billion units of XRP. Its creators gifted 80 billion XRP to a company, now called [Ripple](https://ripple.com/), to develop the XRP Ledger and its ecosystem. Ripple uses XRP to help build the Internet of Value, ushering in a world in which money moves as fast and efficiently as information does today.
 
 ## rippled
-The server software that powers the XRP Ledger is called `rippled` and is available in this repository under the permissive [ISC open-source license](LICENSE). The `rippled` server is written primarily in C++ and runs on a variety of platforms.
+The server software that powers the XRP Ledger is called `rippled` and is available in this repository under the permissive [ISC open-source license](LICENSE.md). The `rippled` server software is written primarily in C++ and runs on a variety of platforms. The `rippled` server software can run in several modes depending on its [configuration](https://xrpl.org/rippled-server-modes.html).  
 
 ### Build from Source
 
 * [Linux](Builds/linux/README.md)
-* [Mac](Builds/macos/README.md)
-* [Windows](Builds/VisualStudio2017/README.md)
+* [Mac](Builds/macos/README.md) (Not recommended for production)
+* [Windows](Builds/VisualStudio2017/README.md) (Not recommended for production)
 
 ## Key Features of the XRP Ledger
 
 - **[Censorship-Resistant Transaction Processing][]:** No single party decides which transactions succeed or fail, and no one can "roll back" a transaction after it completes. As long as those who choose to participate in the network keep it healthy, they can settle transactions in seconds.
 - **[Fast, Efficient Consensus Algorithm][]:** The XRP Ledger's consensus algorithm settles transactions in 4 to 5 seconds, processing at a throughput of up to 1500 transactions per second. These properties put XRP at least an order of magnitude ahead of other top digital assets.
-- **[Finite XRP Supply][]:** When the XRP Ledger began, 100 billion XRP were created, and no more XRP will ever be created. (Each XRP is subdivisible down to 6 decimal places, for a grand total of 100 quintillion _drops_ of XRP.) The available supply of XRP decreases slowly over time as small amounts are destroyed to pay transaction costs.
+- **[Finite XRP Supply][]:** When the XRP Ledger began, 100 billion XRP were created, and no more XRP will ever be created. The available supply of XRP decreases slowly over time as small amounts are destroyed to pay transaction costs.
 - **[Responsible Software Governance][]:** A team of full-time, world-class developers at Ripple maintain and continually improve the XRP Ledger's underlying software with contributions from the open-source community. Ripple acts as a steward for the technology and an advocate for its interests, and builds constructive relationships with governments and financial institutions worldwide.
 - **[Secure, Adaptable Cryptography][]:** The XRP Ledger relies on industry standard digital signature systems like ECDSA (the same scheme used by Bitcoin) but also supports modern, efficient algorithms like Ed25519. The extensible nature of the XRP Ledger's software makes it possible to add and disable algorithms as the state of the art in cryptography advances.
 - **[Modern Features for Smart Contracts][]:** Features like Escrow, Checks, and Payment Channels support cutting-edge financial applications including the [Interledger Protocol](https://interledger.org/). This toolbox of advanced features comes with safety features like a process for amending the network and separate checks against invariant constraints.
 - **[On-Ledger Decentralized Exchange][]:** In addition to all the features that make XRP useful on its own, the XRP Ledger also has a fully-functional accounting system for tracking and trading obligations denominated in any way users want, and an exchange built into the protocol. The XRP Ledger can settle long, cross-currency payment paths and exchanges of multiple currencies in atomic transactions, bridging gaps of trust with XRP.
 
-[Censorship-Resistant Transaction Processing]: https://developers.ripple.com/xrp-ledger-overview.html#censorship-resistant-transaction-processing
-[Fast, Efficient Consensus Algorithm]: https://developers.ripple.com/xrp-ledger-overview.html#fast-efficient-consensus-algorithm
-[Finite XRP Supply]: https://developers.ripple.com/xrp-ledger-overview.html#finite-xrp-supply
-[Responsible Software Governance]: https://developers.ripple.com/xrp-ledger-overview.html#responsible-software-governance
-[Secure, Adaptable Cryptography]: https://developers.ripple.com/xrp-ledger-overview.html#secure-adaptable-cryptography
-[Modern Features for Smart Contracts]: https://developers.ripple.com/xrp-ledger-overview.html#modern-features-for-smart-contracts
-[On-Ledger Decentralized Exchange]: https://developers.ripple.com/xrp-ledger-overview.html#on-ledger-decentralized-exchange
+[Censorship-Resistant Transaction Processing]: https://xrpl.org/xrp-ledger-overview.html#censorship-resistant-transaction-processing
+[Fast, Efficient Consensus Algorithm]: https://xrpl.org/xrp-ledger-overview.html#fast-efficient-consensus-algorithm
+[Finite XRP Supply]: https://xrpl.org/xrp-ledger-overview.html#finite-xrp-supply
+[Responsible Software Governance]: https://xrpl.org/xrp-ledger-overview.html#responsible-software-governance
+[Secure, Adaptable Cryptography]: https://xrpl.org/xrp-ledger-overview.html#secure-adaptable-cryptography
+[Modern Features for Smart Contracts]: https://xrpl.org/xrp-ledger-overview.html#modern-features-for-smart-contracts
+[On-Ledger Decentralized Exchange]: https://xrpl.org/xrp-ledger-overview.html#on-ledger-decentralized-exchange
 
 
 ## Source Code
-[![travis-ci.org: Build Status](https://travis-ci.org/ripple/rippled.png?branch=develop)](https://travis-ci.org/ripple/rippled)
+[![travis-ci.com: Build Status](https://travis-ci.com/ripple/rippled.svg?branch=develop)](https://travis-ci.com/ripple/rippled)
 [![codecov.io: Code Coverage](https://codecov.io/gh/ripple/rippled/branch/develop/graph/badge.svg)](https://codecov.io/gh/ripple/rippled)
 
 ### Repository Contents
@@ -53,9 +53,7 @@ git-subtree. See those directories' README files for more details.
 
 ## See Also
 
-* [XRP Ledger Dev Portal](https://developers.ripple.com/)
-* [XRP News](https://ripple.com/category/xrp/)
-* [Setup and Installation](https://developers.ripple.com/install-rippled.html)
-
-To learn about how Ripple is transforming global payments, visit
-<https://ripple.com/contact/>.
+* [XRP Ledger Dev Portal](https://xrpl.org/)
+* [Setup and Installation](https://xrpl.org/install-rippled.html)
+* [Source Documentation (Doxygen)](https://ripple.github.io/rippled)
+* [Learn more about the XRP Ledger (YouTube)](https://www.youtube.com/playlist?list=PLJQ55Tj1hIVZtJ_JdTvSum2qMTsedWkNi)

RELEASENOTES.md

@@ -1,19 +1,352 @@
 # Release Notes
 
-![Ripple](docs/images/ripple.png)
+![XRP](docs/images/xrp-text-mark-black-small@2x.png)
 
-This document contains the release notes for `rippled`, the reference server implementation of the Ripple protocol. To learn more about how to build and run a `rippled` server, visit https://ripple.com/build/rippled-setup/
+This document contains the release notes for `rippled`, the reference server implementation of the XRP Ledger protocol. To learn more about how to build, run or update a `rippled` server, visit https://xrpl.org/install-rippled.html
 
-> **Do you work at a digital asset exchange or wallet provider?** 
-> 
-> Please [contact us](mailto:support@ripple.com). We can help guide your integration.
+ 
+Have new ideas? Need help with setting up your node? Come visit us [here](https://github.com/ripple/rippled/issues/new/choose)
 
-## Updating `rippled`
+# Change log
 
-If you are using Red Hat Enterprise Linux 7 or CentOS 7, you can [update using `yum`](https://ripple.com/build/rippled-setup/#updating-rippled). For other platforms, please [compile from source](https://wiki.ripple.com/Rippled_build_instructions).
+- API version 2 will now return `signer_lists` in the root of the `account_info` response, no longer nested under `account_data`.
 
 # Releases
 
+## Version 1.8.5
+This is the 1.8.5 release of `rippled`, the reference implementation of the XRP Ledger protocol. This release includes fixes and updates for stability and security, and improvements to build scripts. There are no user-facing API or protocol changes in this release.
+
+### Bug Fixes
+
+This release contains the following bug fixes and under-the-hood improvements:
+
+- **Correct TaggedPointer move constructor:** Fixes a bug in unused code for the TaggedPointer class. The old code would fail if a caller explicitly tried to remove a child that is not actually part of the node. (227a12d)
+
+- **Ensure protocol buffer prerequisites are present:** The build scripts and packages now properly handle Protobuf packages and various packages. Prior to this change, building on Ubuntu 21.10 Impish Indri would fail unless the `libprotoc-dev` package was installed. (e06465f)
+
+- **Improve handling of endpoints during peer discovery.** This hardens and improves handling of incoming messages on the peer protocol. (289bc0a)
+
+- **Run tests on updated linux distros:** Test builds now run on Rocky Linux 8, Fedora 34 and 35, Ubuntu 18, 20, and 22, and Debian 9, 10, and 11. (a9ee802)
+
+- **Avoid dereferencing empty optional in ReportingETL:** Fixes a bug in Reporting Mode that could dereference an empty optional value when throwing an error. (cdc215d)
+
+- **Correctly add GIT_COMMIT_HASH into version string:** When building the server from a non-tagged release, the build files now add the commit ID in a way that follows the semantic-versioning standard, and correctly handle the case where the commit hash ID cannot be retrieved. (d23d37f)
+
+- **Update RocksDB to version 6.27.3:** Updates the version of RocksDB included in the server from 6.7.3 (which was released on 2020-03-18) to 6.27.3 (released 2021-12-10).
+
+
+
+## Version 1.8.4
+This is the 1.8.4 release of `rippled`, the reference implementation of the XRP Ledger protocol.
+
+This release corrects a technical flaw introduced with 1.8.3 that may result in failures if the newly-introduced 'fast loading' is enabled. The release also adjusts default parameters used to configure the pathfinding engine to reduce resource usage.
+
+### Bug Fixes
+- **Adjust mutex scope in `walkMapParallel`**: This commit corrects a technical flaw introduced with commit [7c12f0135897361398917ad2c8cda888249d42ae] that would result in undefined behavior if the server operator configured their server to use the 'fast loading' mechanism introduced with 1.8.3.
+
+- **Adjust pathfinding configuration defaults**: This commit adjusts the default configuration of the pathfinding engine, to account for the size of the XRP Ledger mainnet. Unless explicitly overriden, the changes mean that pathfinding operations will return fewer, shallower paths than previous releases.
+
+
+## Version 1.8.3
+This is the 1.8.3 release of `rippled`, the reference implementation of the XRP Ledger protocol.
+
+This release implements changes that improve the syncing performance of peers on the network, adds countermeasures to several routines involving LZ4 to defend against CVE-2021-3520, corrects a minor technical flaw that would result in the server not using a cache for nodestore operations, and adjusts tunable values to optimize disk I/O.
+
+### Summary of Issues
+Recently, servers in the XRP Ledger network have been taking an increasingly long time to sync back to the network after restartiningg. This is one of several releases which will be made to improve on this issue. 
+
+
+### Bug Fixes
+
+- **Parallel ledger loader & I/O performance improvements**: This commit makes several changes that, together, should decrease the time needed for a server to sync to the network. To make full use of this change, `rippled` needs to be using storage with high IOPS and operators need to explicitly enable this behavior by adding the following to their config file, under the `[node_db]` stanza:
+
+    [node_db]
+    ...
+    fast_load=1
+
+Note that when 'fast loading' is enabled the server will not open RPC and WebSocket interfaces until after the initial load is completed. Because of this, it may appear unresponsive or down.
+
+- **Detect CVE-2021-3520 when decompressing using LZ4**: This commit adds code to detect LZ4 payloads that may result in out-of-bounds memory accesses.
+
+- **Provide sensible default values for nodestore cache:**: The nodestore includes a built-in cache to reduce the disk I/O load but, by default, this cache was not initialized unless it was explicitly configured by the server operator. This commit introduces sensible defaults based on the server's configured node size. 
+
+- **Adjust the number of concurrent ledger data jobs**: Processing a large amount of data at once can effectively bottleneck a server's I/O subsystem. This commits helps optimize I/O performance by controlling how many jobs can concurrently process ledger data.
+
+- **Two small SHAMapSync improvements**: This commit makes minor changes to optimize the way memory is used and control the amount of background I/O performed when attempting to fetch missing `SHAMap` nodes.
+
+## Version 1.8.2
+Ripple has released version 1.8.2 of rippled, the reference server implementation of the XRP Ledger protocol.  This release addresses the full transaction queues and elevated transaction fees issue observed on the XRP ledger, and also provides some optimizations and small fixes to improve the server's performance overall.
+
+### Summary of Issues
+Recently, servers in the XRP Ledger network have had full transaction queues and transactions paying low fees have mostly not been able to be confirmed through the queue. After investigation, it was discovered that a large influx of transactions to the network caused it to raise the transaction costs to be proposed in the next ledger block, and defer transactions paying lower costs to later ledgers. The first part worked as designed, but deferred transactions were not being confirmed as the ledger had capacity to process them.
+
+The root cause was that there were very many low-cost transactions that different servers in the network received in a different order due to incidental differences in timing or network topology, which caused validators to propose different sets of low-cost transactions from the queue. Since none of these transactions had support from a majority of validators, they were removed from the proposed transaction set. Normally, any transactions removed from a proposed transaction set are supposed to be retried in the next ledger, but servers attempted to put these deferred transactions into their transaction queues first, which had filled up. As a result, the deferred transactions were discarded, and the network was only able to confirm transactions that paid high costs.
+
+### Bug Fixes
+
+- **Address elevated transaction fees**: This change addresses the full queue problems in two ways. First, it puts deferred transactions directly into the open ledger, rather than transaction queue. This reverts a subset of the changes from [ximinez@62127d7](https://github.com/ximinez/rippled/commit/62127d725d801641bfaa61dee7d88c95e48820c5). A transaction that is in the open ledger but doesn't get validated should stay in the open ledger so that it can be proposed again right away. Second, it changes the order in which transactions are pulled from the transaction queue to increase the overlap in servers' initial transaction consensus proposals. Like the old rules, transactions paying higher fee levels are selected first. Unlike the old rules, transactions paying the same fee level are ordered by transaction ID / hash ascending. (Previously, transactions paying the same fee level were unsorted, resulting in each server having a different order.)
+
+- **Add ignore_default option to account_lines API**: This flag, if present, suppresses the output of incoming trust lines in the default state. This is primarily motivated by observing that users often have many unwanted incoming trust lines in a default state, which are not useful in the vast majority of cases. Being able to suppress those when doing `account_lines` saves bandwidth and resources. ([#3980](https://github.com/ripple/rippled/pull/3980))
+
+- **Make I/O and prefetch worker threads configurable**: This commit adds the ability to specify **io_workers** and **prefetch_workers** in the config file which can be used to specify the number of threads for processing raw inbound and outbound IO and configure the number of threads for performing node store prefetching. ([#3994](https://github.com/ripple/rippled/pull/3994))
+
+- **Enforce account RPC limits by objects traversed**: This changes the way the account_objects API method counts and limits the number of objects it returns. Instead of limiting results by the number of objects found, it counts by the number of objects traversed. Additionally, the default and maximum limits for non-admin connections have been decreased. This reduces the amount of work that one API call can do so that public API servers can share load more effectively. ([#4032](https://github.com/ripple/rippled/pull/4032))
+
+- **Fix a crash on shutdown**: The NuDB backend class could throw an error in its destructor, resulting in a crash while the server was shutting down gracefully. This crash was harmless but resulted in false alarms and noise when tracking down other possible crashes. ([#4017](https://github.com/ripple/rippled/pull/4017))
+
+- **Improve reporting of job queue in admin server_info**: The server_info command, when run with admin permissions, provides information about jobs in the server's job queue. This commit provides more descriptive names and more granular categories for many jobs that were previously all identified as "clientCommand". ([#4031](https://github.com/ripple/rippled/pull/4031))
+
+- **Improve full & compressed inner node deserialization**: Remove a redundant copy operation from low-level SHAMap deserialization. ([#4004](https://github.com/ripple/rippled/pull/4004))
+
+- **Reporting mode: only forward to P2P nodes that are synced**: Previously, reporting mode servers forwarded to any of their configured P2P nodes at random. This commit improves the selection so that it only chooses from P2P nodes that are fully synced with the network. ([#4028](https://github.com/ripple/rippled/pull/4028))
+
+- **Improve handling of HTTP X-Forwarded-For and Forwarded headers**: Fixes the way the server handles IPv6 addresses in these HTTP headers. ([#4009](https://github.com/ripple/rippled/pull/4009), [#4030](https://github.com/ripple/rippled/pull/4030))
+
+- **Other minor improvements to logging and Reporting Mode.**
+
+
+## Version 1.8.0
+Ripple has released version 1.8.0 of rippled, the reference server implementation of the XRP Ledger protocol. This release brings several features and improvements.
+
+### New and Improved Features
+
+- **Improve History Sharding**: Shards of ledger history are now assembled in a deterministic way so that any server can make a binary-identical shard for a given range of ledgers. This makes it possible to retrieve a shard from multiple sources in parallel, then verify its integrity by comparing checksums with peers' checksums for the same shard. Additionally, there's a new admin RPC command to import ledger history from the shard store, and the crawl_shards command has been expanded with more info. ([#2688](https://github.com/ripple/rippled/issues/2688), [#3726](https://github.com/ripple/rippled/pull/3726), [#3875](https://github.com/ripple/rippled/pull/3875))
+- **New CheckCashMakesTrustLine Amendment**: If enabled, this amendment will change the CheckCash transaction type so that cashing a check for an issued token automatically creates a trust line to hold the token, similar to how purchasing a token in the decentralized exchange creates a trust line to hold the token. This change provides a way for issuers to send tokens to a user before that user has set up a trust line, but without forcing anyone to hold tokens they don't want. ([#3823](https://github.com/ripple/rippled/pull/3823))
+- **Automatically determine the node size**: The server now selects an appropriate `[node_size]` configuration value by default if it is not explicitly specified. This parameter tunes various settings to the specs of the hardware that the server is running on, especially the amount of RAM and the number of CPU threads available in the system. Previously the server always chose the smallest value by default.
+- **Improve transaction relaying logic**: Previously, the server relayed every transaction to all its peers (except the one that it received the transaction from). To reduce redundant messages, the server now relays transactions to a subset of peers using a randomized algorithm. Peers can determine whether there are transactions they have not seen and can request them from a peer that has them. It is expected that this feature will further reduce the bandwidth needed to operate a server.
+- **Improve the Byzantine validator detector**: This expands the detection capabilities of the Byzantine validation detector. Previously, the server only monitored validators on its own UNL. Now, the server monitors for Byzantine behavior in all validations it sees.
+- **Experimental tx stream with history for sidechains**: Adds an experimental subscription stream for sidechain federators to track messages on the main chain in canonical order. This stream is expected to change or be replaced in future versions as work on sidechains matures.
+- **Support Debian 11 Bullseye**: This is the first release that is compatible with Debian Linux version 11.x, "Bullseye." The .deb packages now use absolute paths only, for compatibility with Bullseye's stricter package requirements. ([#3909](https://github.com/ripple/rippled/pull/3909))
+- **Improve Cache Performance**: The server uses a new storage structure for several in-memory caches for greatly improved overall performance. The process of purging old data from these caches, called "sweeping", was time-consuming and blocked other important activities necessary for maintaining ledger state and participating in consensus. The new structure divides the caches into smaller partitions that can be swept in parallel.
+- **Amendment default votes:** Introduces variable default votes per amendment. Previously the server always voted "yes" on any new amendment unless an admin explicitly configured a voting preference for that amendment. Now the server's default vote can be "yes" or "no" in the source code. This should allow a safer, more gradual roll-out of new amendments, as new releases can be configured to understand a new amendment but not vote for it by default. ([#3877](https://github.com/ripple/rippled/pull/3877))
+- **More fields in the `validations` stream:** The `validations` subscription stream in the API now reports additional fields that were added to validation messages by the HardenedValidations amendment. These fields make it easier to detect misconfigurations such as multiple servers sharing a validation key pair. ([#3865](https://github.com/ripple/rippled/pull/3865))
+- **Reporting mode supports `validations` and `manifests` streams:** In the API it is now possible to connect to these streams when connected to a servers running in reporting. Previously, attempting to subscribe to these streams on a reporting server failed with the error `reportingUnsupported`. ([#3905](https://github.com/ripple/rippled/pull/3905))
+
+### Bug Fixes
+
+- **Clarify the safety of NetClock::time_point arithmetic**: * NetClock::rep is uint32_t and can be error-prone when   used with subtraction. * Fixes [#3656](https://github.com/ripple/rippled/pull/3656)
+- **Fix out-of-bounds reserve, and some minor optimizations**
+- **Fix nested locks in ValidatorSite**
+- **Fix clang warnings about copies vs references**
+- **Fix reporting mode build issue**
+- **Fix potential deadlock in Validator sites**
+- **Use libsecp256k1 instead of OpenSSL for key derivation**: The deterministic key derivation code was still using calls to OpenSSL. This replaces the OpenSSL-based routines with new libsecp256k1-based implementations
+- **Improve NodeStore to ShardStore imports**: This runs the import process in a background thread while preventing online_delete from removing ledgers pending import
+- **Simplify SHAMapItem construction**: The existing class offered several constructors which were mostly unnecessary. This eliminates all existing constructors and introduces a single new one, taking a `Slice`. The internal buffer is switched from `std::vector` to `Buffer` to save a minimum of 8 bytes (plus the buffer slack that is inherent in `std::vector`) per SHAMapItem instance.
+- **Redesign stoppable objects**: Stoppable is no longer an abstract base class, but a pattern, modeled after the well-understood `std::thread`. The immediate benefits are less code, less synchronization, less runtime work, and (subjectively) more readable code. The end goal is to adhere to RAII in our object design, and this is one necessary step on that path.
+
+## Version 1.7.3
+
+This is the 1.7.3 release of `rippled`, the reference implementation of the XRP Ledger protocol. This release addresses an OOB memory read identified by Guido Vranken, as well as an unrelated issue identified by the Ripple C++ team that could result in incorrect use of SLEs. Additionally, this version also introduces the `NegativeUNL` amendment, which corresponds to the feature which was introduced with the 1.6.0 release.
+
+## Action Required
+
+If you operate an XRP Ledger server, then you should upgrade to version 1.7.3 at your earliest convenience to mitigate the issues addressed in this hotfix. If a sufficient majority of servers on the network upgrade, the `NegativeUNL` amendment may gain a majority, at which point a two week activation countdown will begin. If the `NegativeUNL` amendment activates, servers running versions of `rippled` prior to 1.7.3 will become [amendment blocked](https://xrpl.org/amendments.html#amendment-blocked).
+
+### Bug Fixes
+
+- **Improve SLE usage in check cashing**: Fixes a situation which could result in the incorrect use of SLEs.
+- **Address OOB in base58 decoder**: Corrects a technical flaw that could allow an out-of-bounds memory read in the Base58 decoder.
+- **Add `NegativeUNL` as a supported amendment**: Introduces an amendment for the Negative UNL feature introduced in `rippled` 1.6.0.
+
+## Version 1.7.2
+
+This the 1.7.2 release of rippled, the reference server implementation of the XRP Ledger protocol. This release protects against the security issue [CVE-2021-3499](https://www.openssl.org/news/secadv/20210325.txt) affecting OpenSSL, adds an amendment to fix an issue with small offers not being properly removed from order books in some cases, and includes various other minor fixes.
+Version 1.7.2 supersedes version 1.7.1 and adds fixes for more issues that were discovered during the release cycle.
+
+## Action Required
+
+This release introduces a new amendment to the XRP Ledger protocol: `fixRmSmallIncreasedQOffers`. This amendments is now open for voting according to the XRP Ledger's amendment process, which enables protocol changes following two weeks of >80% support from trusted validators.
+If you operate an XRP Ledger server, then you should upgrade to version 1.7.2 within two weeks, to ensure service continuity. The exact time that protocol changes take effect depends on the voting decisions of the decentralized network.
+If you operate an XRP Ledger validator, please learn more about this amendment so you can make informed decisions about how your validator votes. If you take no action, your validator begins voting in favor of any new amendments as soon as it has been upgraded. 
+
+### Bug Fixes
+
+- **fixRmSmallIncreasedQOffers Amendment:** This amendment fixes an issue where certain small offers can be left at the tip of an order book without being consumed or removed when appropriate and causes some payments and Offers to fail when they should have succeeded [(#3827)](https://github.com/ripple/rippled/pull/3827).
+- **Adjust OpenSSL defaults and mitigate CVE-2021-3499:** Prior to this fix, servers compiled against a vulnerable version of OpenSSL could have a crash triggered by a malicious network connection. This fix disables renegotiation support in OpenSSL so that the rippled server is not vulnerable to this bug regardless of the OpenSSL version used to compile the server. This also removes support for deprecated TLS versions 1.0 and 1.1 and ciphers that are not part of TLS 1.2 [(#79e69da)](https://github.com/ripple/rippled/pull/3843/commits/79e69da3647019840dca49622621c3d88bc3883f).
+- **Support HTTP health check in reporting mode:** Enables the Health Check special method when running the server in the new Reporting Mode introduced in 1.7.0 [(9c8cadd)](https://github.com/ripple/rippled/pull/3843/commits/9c8caddc5a197bdd642556f8beb14f06d53cdfd3).
+- **Maintain compatibility for forwarded RPC responses:** Fixes a case in API responses from servers in Reporting Mode, where requests that were forwarded to a P2P-mode server would have the result field nested inside another result field [(8579eb0)](https://github.com/ripple/rippled/pull/3843/commits/8579eb0c191005022dcb20641444ab471e277f67).
+- **Add load_factor in reporting mode:** Adds a load_factor value to the server info method response when running the server in Reporting Mode so that the response is compatible with the format returned by servers in P2P mode (the default) [(430802c)](https://github.com/ripple/rippled/pull/3843/commits/430802c1cf6d4179f2249a30bfab9eff8e1fa748).
+- **Properly encode metadata from tx RPC command:** Fixes a problem where transaction metadata in the tx API method response would be in JSON format even when binary was requested [(7311629)](https://github.com/ripple/rippled/pull/3843/commits/73116297aa94c4acbfc74c2593d1aa2323b4cc52).
+- **Updates to Windows builds:** When building on Windows, use vcpkg 2021 by default and add compatibility with MSVC 2019 [(36fe196)](https://github.com/ripple/rippled/pull/3843/commits/36fe1966c3cd37f668693b5d9910fab59c3f8b1f), [(30fd458)](https://github.com/ripple/rippled/pull/3843/commits/30fd45890b1d3d5f372a2091d1397b1e8e29d2ca).
+
+## Version 1.7.0 
+
+Ripple has released version 1.7.0 of `rippled`, the reference server implementation of the XRP Ledger protocol. 
+This release [significantly improves memory usage](https://blog.ripplex.io/how-ripples-c-team-cut-rippleds-memory-footprint-down-to-size/), introduces a protocol amendment to allow out-of-order transaction execution with Tickets, and brings several other features and improvements. 
+
+## Upgrading (SPECIAL ACTION REQUIRED)
+If you use the precompiled binaries of rippled that Ripple publishes for supported platforms, please note that Ripple has renewed the GPG key used to sign these packages. 
+If you are upgrading from a previous install, you must download and trust the renewed key. Automatic upgrades will not work until you have re-trusted the key.
+### Red Hat Enterprise Linux / CentOS
+
+Perform a [manual upgrade](https://xrpl.org/update-rippled-manually-on-centos-rhel.html). When prompted, confirm that the key's fingerprint matches the following example, then press `y` to accept the updated key:
+
+```
+$ sudo yum install rippled
+Loaded plugins: fastestmirror
+Loading mirror speeds from cached hostfile
+* base: mirror.web-ster.com
+* epel: mirrors.syringanetworks.net
+* extras: ftp.osuosl.org
+* updates: mirrors.vcea.wsu.edu
+ripple-nightly/signature                                                                                                                                                                                                                                 |  650 B  00:00:00    
+Retrieving key from https://repos.ripple.com/repos/rippled-rpm/nightly/repodata/repomd.xml.key
+Importing GPG key 0xCCAFD9A2:
+Userid     : "TechOps Team at Ripple <techops+rippled@ripple.com>"
+Fingerprint: c001 0ec2 05b3 5a33 10dc 90de 395f 97ff ccaf d9a2
+From       : https://repos.ripple.com/repos/rippled-rpm/nightly/repodata/repomd.xml.key
+Is this ok [y/N]: y
+```
+
+### Ubuntu / Debian
+
+Download and trust the updated public key, then perform a [manual upgrade](https://xrpl.org/update-rippled-manually-on-ubuntu.html) as follows:
+
+```
+wget -q -O - "https://repos.ripple.com/repos/api/gpg/key/public" | \
+    sudo apt-key add -
+sudo apt -y update
+sudo apt -y install rippled
+```
+
+### New and Improved Features
+
+- **Rework deferred node logic and async fetch behavior:** This change significantly improves ledger sync and fetch times while reducing memory consumption.  (https://blog.ripplex.io/how-ripples-c-team-cut-rippleds-memory-footprint-down-to-size/)
+- **New Ticket feature:** Tickets are a mechanism to prepare and send certain transactions outside of the normal sequence order. This version reworks and completes the implementation for Tickets after more than 6 years of development. This feature is now open for voting as the newly-introduced `TicketBatch` amendment, which replaces the previously-proposed `Tickets` amendment. The specification for this change can be found at: [xrp-community/standards-drafts#16](https://github.com/xrp-community/standards-drafts/issues/16)
+- **Add Reporting Mode:** The server can be compiled to operate in a new mode that serves API requests for validated ledger data without connecting directly to the peer-to-peer network. (The server needs a gRPC connection to another server that is on the peer-to-peer network.) Reporting Mode servers can share access to ledger data via Apache Cassandra and PostgreSQL to more efficiently serve API requests while peer-to-peer servers specialize in broadcasting and processing transactions.
+- **Optimize relaying of validation and proposal messages:** Servers typically receive multiple copies of any given message from directly connected peers; in particular, consensus proposal and validation messages are often relayed with extremely high redundancy. For servers with several peers, this can cause redundant work. This commit introduces experimental code that attempts to optimize the relaying of proposals and validations by allowing servers to instruct their peers to "squelch" delivery of selected proposals and validations. This change is considered experimental at this time and is disabled by default because the functioning of the consensus network depends on messages propagating with high reliability through the constantly-changing peer-to-peer network. Server operators who wish to test the optimized code can enable it in their server config file.
+- **Report server domain to other servers:** Server operators now have the option to configure a domain name to be associated with their servers. The value is communicated to other servers and is also reported via the `server_info` API. The value is meant for third-party applications and tools to group servers together. For example, a tool that visualizes the network's topology can show how many servers are operated by different stakeholders. An operator can claim any domain, so tools should use the [xrp-ledger.toml file](https://xrpl.org/xrp-ledger-toml.html) to confirm that the domain also claims ownership of the servers.
+- **Improve handling of peers that aren't synced:** When evaluating the fitness and usefulness of an outbound peer, the code would incorrectly calculate the amount of time that the peer spent in a non-useful state. This release fixes the calculation and makes the timeout values configurable by server operators. Two new options are introduced in the 'overlay' stanza of the config file. 
+- **Persist API-configured voting settings:** Previously, the amendments that a server would vote in support of or against could be configured both via the configuration file and via the ["feature" API method](https://xrpl.org/feature.html). Changes made in the configuration file were only loaded at server startup; changes made via the command line take effect immediately but were not persisted across restarts. Starting with this release, changes made via the API are saved to the wallet.db database file so that they persist even if the server is restarted.
+Amendment voting in the config file is deprecated. The first time the server starts with v1.7.0 or higher, it reads any amendment voting settings in the config file and saves the settings to the database; on later restarts the server prints a warning message and ignores the [amendments] and [veto_amendments] stanzas of the config file.
+Going forward, use the [feature method](https://xrpl.org/feature.html) to view and configure amendment votes. If you want to use the config file to configure amendment votes, add a line to the [rpc_startup] stanza such as the following:
+[rpc_startup]
+{ "command": "feature", "feature": "FlowSortStrands", "vetoed": true }
+- **Support UNLs with future effective dates:** Updates the format for the recommended validator list file format, allowing publishers to pre-publish the next recommended UNL while the current one is still valid. The server is still backwards compatible with the previous format, but the new format removes some uncertainty during the transition from one list to the next. Also, starting with this release, the server locks down and reports an error if it has no valid validator list. You can clear the error by loading a validator list from a file or by configuring a different UNL and restarting; the error also goes away on its own if the server is able to obtain a trusted validator list from the network (for example, after an network outage resolves itself).
+- **Improve manifest relaying:** Servers now propagate change messages for validators' ephemeral public keys ("manifests") on a best-effort basis, to make manifests more available throughout the peer-to-peer network. Previously, the server would only relay manifests from validators it trusts locally, which made it difficult to detect and track validators that are not broadly trusted.
+- **Implement ledger forward replay feature:** The server can now sync up to the network by "playing forward" transactions from a previously saved ledger until it catches up to the network. Compared with the default behavior of fetching the latest state and working backwards, forward replay can save time and bandwidth by reconstructing previous ledgers' state data rather than downloading the pre-calculated results from the network. As an added bonus, forward replay confirms that the rest of the network followed the same transaction processing rules as the local server when processing the intervening ledgers. This feature is considered experimental this time and can be enabled with an option in the config file.
+- **Make the transaction job queue limit adjustable:** The server uses a job queue to manage tasks, with limits on how many jobs of a particular type can be queued. The previously hard-coded limit associated with transactions is now configurable. Server operators can increase the number of transactions their server is able to queue, which may be useful if your server has a large memory capacity or you expect an influx of transactions. (https://github.com/ripple/rippled/issues/3556)
+- **Add public_key to the Validator List method response:** The [Validator List method](https://xrpl.org/validator-list.html) can be used to request a recommended validator list from a rippled instance. The response now includes the public key of the requested list. (https://github.com/ripple/rippled/issues/3392)
+- **Server operators can now configure maximum inbound and outbound peers separately:** The new `peers_in_max` and `peers_out_max` config options allow server operators to independently control the maximum number of inbound and outbound peers the server allows. [70c4ecc]
+- **Improvements to shard downloading:** Previously the download_shard command could only load shards over HTTPS. Compressed shards can now also be downloaded over plain HTTP. The server fully checks the data for integrity and consistency, so the encryption is not strictly necessary. When initiating multiple shard downloads, the server now returns an error if there is not enough space to store all the shards currently being downloaded.
+- **The manifest command is now public:** The manifest API method returns public information about a given validator. The required permissions have been changed so it is now part of the public API.
+
+### Bug Fixes
+
+- **Implement sticky DNS resolution for validator list retrieval:** When attempting to load a validator list from a configured site, attempt to reuse the last IP that was successfully used if that IP is still present in the DNS response. (https://github.com/ripple/rippled/issues/3494).
+- **Improve handling of RPC ledger_index argument:** You can now provide the `ledger_index` as a numeric string. This allows you to copy and use the numeric string `ledger_index` value returned by certain RPC commands. Previously you could only send native JSON numbers or shortcut strings such as "validated" in the `ledger_index` field. (https://github.com/ripple/rippled/issues/3533)
+- **Fix improper promotion of bool on return**  [6968da1]
+- **Fix ledger sequence on copynode** [ef53197] 
+-  **Fix parsing of node public keys in `manifest` CLI:** The previous code attempts to validate the provided node public key using a function that assumes that the encoded public key is for an account. This causes the parsing to fail. This commit fixes #3317 (https://github.com/ripple/rippled/issues/3317) by letting the caller specify the type of the public key being checked.
+- **Fix idle peer timer:** Fixes a bug where a function to remove idle peers was called every second instead of every 4 seconds. #3754 (https://github.com/ripple/rippled/issues/3754)
+- **Add database counters:** Fix bug where DatabaseRotateImp::getBackend and ::sync utilized the writable backend without a lock. ::getBackend was replaced with ::getCounters.
+- **Improve online_delete configuration and DB tuning** [6e9051e]
+- **Improve handling of burst writes in NuDB database** ( https://github.com/ripple/rippled/pull/3662 )
+- **Fix excessive logging after disabling history shards.** Previously if you configured the server with a shard store, then disabled it, the server output excessive warning messages about the shard limit being exceeded.
+- **Fixed some issues with negotiating link compression.** ( https://github.com/ripple/rippled/pull/3705 )
+- **Fixed a potential thread deadlock with history sharding.** ( https://github.com/ripple/rippled/pull/3683 )
+- **Various fixes to typos and comments, refactoring, and build system improvements**
+
+## Version 1.6.0
+
+This release introduces several new features including changes to the XRP Ledger's consensus mechanism to make it even more robust in 
+adverse conditions, as well as numerous bug fixes and optimizations.
+
+### New and Improved Features
+
+- Initial implementation of Negative UNL functionality: This change can improve the liveness of the network during periods of network instability, by allowing servers to track which validators are temporarily offline and to adjust quorum calculations to match. This change requires an amendment, but the amendment is not in the **1.6.0** release. Ripple expects to run extensive public testing for Negative UNL functionality on the Devnet in the coming weeks. If public testing satisfies all requirements across security, reliability, stability, and performance, then the amendment could be included in a version 2.0 release. [[#3380](https://github.com/ripple/rippled/pull/3380)]
+- Validation Hardening: This change allows servers to detect accidental misconfiguration of validators, as well as potentially Byzantine behavior by malicious validators. Servers can now log a message to notify operators if they detect a single validator issuing validations for multiple, incompatible ledger versions, or validations from multiple servers sharing a key. As part of this update, validators report the version of `rippled` they are using, as well as the hash of the last ledger they consider to be fully validated, in validation messages. [[#3291](https://github.com/ripple/rippled/pull/3291)] ![Amendment: Required](https://img.shields.io/badge/Amendment-Required-red)
+- Software Upgrade Monitoring & Notification: After the `HardenedValidations` amendment is enabled and the validators begin reporting the versions of `rippled` they are running, a server can check how many of the validators on its UNL run a newer version of the software than itself. If more than 60% of a server's validators are running a newer version, the server writes a message to notify the operator to consider upgrading their software. [[#3447](https://github.com/ripple/rippled/pull/3447)]
+- Link Compression: Beginning with **1.6.0**, server operators can enable support for compressing peer-to-peer messages. This can save bandwidth at a cost of higher CPU usage. This support is disabled by default and should prove useful for servers with a large number of peers. [[#3287](https://github.com/ripple/rippled/pull/3287)]
+- Unconditionalize Amendments that were enabled in 2017: This change removes legacy code which the network has not used since 2017. This change limits the ability to [replay](https://github.com/xrp-community/standards-drafts/issues/14) ledgers that rely on the pre-2017 behavior. [[#3292](https://github.com/ripple/rippled/pull/3292)]
+- New Health Check Method: Perform a simple HTTP request to get a summary of the health of the server: Healthy, Warning, or Critical. [[#3365](https://github.com/ripple/rippled/pull/3365)]
+- Start work on API version 2. Version 2 of the API will be part of a future release. The first breaking change will be to consolidate several closely related error messages that can occur when the server is not synced into a single "notSynced" error message. [[#3269](https://github.com/ripple/rippled/pull/3269)]
+- Improved shard concurrency: Improvements to the shard engine have helped reduce the lock scope on all public functions, increasing the concurrency of the code. [[#3251](https://github.com/ripple/rippled/pull/3251)]
+- Default Port: In the config file, the `[ips_fixed]` and `[ips]` stanzas now use the [IANA-assigned port](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=2459) for the XRP Ledger protocol (2459) when no port is specified. The `connect` API method also uses the same port by default. [[#2861](https://github.com/ripple/rippled/pull/2861)].
+- Improve proposal and validation relaying. The peer-to-peer protocol always relays trusted proposals and validations (as part of the [consensus process](https://xrpl.org/consensus.html)), but only relays _untrusted_ proposals and validations in certain circumstances. This update adds configuration options so server operators can fine-tune how their server handles untrusted proposals and validations, and changes the default behavior to prioritize untrusted validations higher than untrusted proposals.  [[#3391](https://github.com/ripple/rippled/pull/3391)]
+- Various Build and CI Improvements including updates to RocksDB 6.7.3 [[#3356](https://github.com/ripple/rippled/pull/3356)], NuDB 2.0.3 [[#3437](https://github.com/ripple/rippled/pull/3437)], adjusting CMake settings so that rippled can be built as a submodule [[#3449](https://github.com/ripple/rippled/pull/3449)], and adding Travis CI settings for Ubuntu Bionic Beaver [[#3319](https://github.com/ripple/rippled/pull/3319)].
+- Better documentation in the config file for online deletion and database tuning. [[#3429](https://github.com/ripple/rippled/pull/3429)]
+
+
+### Bug Fixes
+
+- Fix the 14 day timer to enable amendment to start at the correct quorum size [[#3396](https://github.com/ripple/rippled/pull/3396)]
+- Improve online delete backend lock which addresses a possibility in the online delete process where one or more backend shared pointer references may become invalid during rotation. [[#3342](https://github.com/ripple/rippled/pull/3342)]
+- Address an issue that can occur during the loading of validator tokens, where a deliberately malformed token could cause the server to crash during startup. [[#3326](https://github.com/ripple/rippled/pull/3326)]
+- Add delivered amount to GetAccountTransactionHistory. The delivered_amount field was not being populated when calling GetAccountTransactionHistory. In contrast, the delivered_amount field was being populated when calling GetTransaction. This change populates delivered_amount in the response to GetAccountTransactionHistory, and adds a unit test to make sure the results delivered by GetTransaction and GetAccountTransactionHistory match each other. [[#3370](https://github.com/ripple/rippled/pull/3370)]
+- Fix build issues for GCC 10 [[#3393](https://github.com/ripple/rippled/pull/3393)]
+- Fix historical ledger acquisition - this fixes an issue where historical ledgers were acquired only since the last online deletion interval instead of the configured value to allow deletion.[[#3369](https://github.com/ripple/rippled/pull/3369)]
+- Fix build issue with Docker [#3416](https://github.com/ripple/rippled/pull/3416)]
+- Add Shard family. The App Family utilizes a single shared Tree Node and Full Below cache for all history shards. This can create a problem when acquiring a shard that shares an account state node that was recently cached from another shard operation. The new Shard Family class solves this issue by managing separate Tree Node and Full Below caches for each shard. [#3448](https://github.com/ripple/rippled/pull/3448)]
+- Amendment table clean up which fixes a calculation issue with majority. [#3428](https://github.com/ripple/rippled/pull/3428)]
+- Add the `ledger_cleaner` command to rippled command line help [[#3305](https://github.com/ripple/rippled/pull/3305)]
+- Various typo and comments fixes.
+
+
+## Version 1.5.0
+
+The `rippled` 1.5.0 release introduces several improvements and new features, including support for gRPC API, API versioning, UNL propagation via the peer network, new RPC methods `validator_info` and `manifest`, augmented `submit` method, improved `tx` method response, improved command line parsing, improved handshake protocol, improved package building and various other minor bug fixes and improvements.
+
+This release also introduces two new amendments: `fixQualityUpperBound` and `RequireFullyCanonicalSig`.
+
+Several improvements to the sharding system are currently being evaluated for inclusion into the upcoming 1.6 release of `rippled`. These changes are incompatible with shards generated by previous versions of the code. 
+Additionally, an issue with the existing sharding engine can result in a server running versions 1.4 or 1.5 of the software to experience a deadlock and automatically restart when running with the sharding feature enabled. 
+At this time, the developers recommend running with sharding disabled, pending the improvements scheduled to be introduced with 1.6. For more information on how to disable sharding, please visit https://xrpl.org/configure-history-sharding.html
+
+ 
+**New and Updated Features**
+- The `RequireFullyCanonicalSig` amendment which changes the signature requirements for the XRP Ledger protocol so that non-fully-canonical signatures are no longer valid. This protects against transaction malleability on all transactions, instead of just transactions with the tfFullyCanonicalSig flag enabled. Without this amendment, a transaction is malleable if it uses a secp256k1 signature and does not have tfFullyCanonicalSig enabled. Most signing utilities enable tfFullyCanonicalSig by default, but there are exceptions. With this amendment, no single-signed transactions are malleable. (Multi-signed transactions may still be malleable if signers provide more signatures than are necessary.) All transactions must use the fully canonical form of the signature, regardless of the tfFullyCanonicalSig flag. Signing utilities that do not create fully canonical signatures are not supported. All of Ripple's signing utilities have been providing fully-canonical signatures exclusively since at least 2014. For more information. [`ec137044a`](https://github.com/ripple/rippled/commit/ec137044a014530263cd3309d81643a5a3c1fdab)
+- Native [gRPC API](https://grpc.io/) support. Currently, this API provides a subset of the full `rippled` [API](https://xrpl.org/rippled-api.html). You can enable the gRPC API on your server with a new configuration stanza. [`7d867b806`](https://github.com/ripple/rippled/commit/7d867b806d70fc41fb45e3e61b719397033b272c)
+- API Versioning which allows for future breaking change of RPC methods to co-exist with existing versions. [`2aa11fa41`](https://github.com/ripple/rippled/commit/2aa11fa41d4a7849ae6a5d7a11df6f367191e3ef)
+- Nodes now receive and broadcast UNLs over the peer network under various conditions. [`2c71802e3`](https://github.com/ripple/rippled/commit/2c71802e389a59118024ea0152123144c084b31c)
+- Augmented `submit` method to include additional details on the status of the command. [`79e9085dd`](https://github.com/ripple/rippled/commit/79e9085dd1eb72864afe841225b78ec96e72b5ca)
+- Improved `tx` method response with additional details on ledgers searched. [`47501b7f9`](https://github.com/ripple/rippled/commit/47501b7f99d4103d9ad405e399169fc251161548)
+- New `validator_info` method which returns information pertaining to the current validator's keys, manifest sequence, and domain. [`3578acaf0`](https://github.com/ripple/rippled/commit/3578acaf0b5f2d27ddc33f5b4cc81d21be1903ae)
+- New `manifest` method which looks up manifest information for the specified key (either master or ephemeral). [`3578acaf0`](https://github.com/ripple/rippled/commit/3578acaf0b5f2d27ddc33f5b4cc81d21be1903ae)
+- Introduce handshake protocol for compression negotiation (compression is not implemented at this point) and other minor improvements. [`f6916bfd4`](https://github.com/ripple/rippled/commit/f6916bfd429ce654e017ae9686cb023d9e05408b)
+- Remove various old conditionals introduced by amendments. [`(51ed7db00`](https://github.com/ripple/rippled/commit/51ed7db0027ba822739bd9de6f2613f97c1b227b), [`6e4945c56)`](https://github.com/ripple/rippled/commit/6e4945c56b1a1c063b32921d7750607587ec3063)
+- Add `getRippledInfo` info gathering script to `rippled` Linux packages. [`acf4b7889`](https://github.com/ripple/rippled/commit/acf4b78892074303cb1fa22b778da5e7e7eddeda)
+
+**Bug Fixes and Improvements**
+- The `fixQualityUpperBound` amendment which fixes a bug in unused code for estimating the ratio of input to output of individual steps in cross-currency payments. [`9d3626fec`](https://github.com/ripple/rippled/commit/9d3626fec5b610100f401dc0d25b9ec8e4a9a362)
+- `tx` method now properly fetches all historical tx if they are incorporated into a validated ledger under rules that applied at the time. [`11cf27e00`](https://github.com/ripple/rippled/commit/11cf27e00698dbfc099b23463927d1dac829ed19)
+- Fix to how `fail_hard` flag is handled with the `submit` method - transactions that are submitted with the `fail_hard` flag that result in any TER code besides tesSUCCESS is neither queued nor held. [`cd9732b47`](https://github.com/ripple/rippled/commit/cd9732b47a9d4e95bcb74e048d2c76fa118b80fb)
+- Remove unused `Beast` code. [`172ead822`](https://github.com/ripple/rippled/commit/172ead822159a3c1f9b73217da4316df48851ab6)
+- Lag ratchet code fix to use proper ephemeral public keys instead of the long-term master public keys.[`6529d3e6f`](https://github.com/ripple/rippled/commit/6529d3e6f7333fc5226e5aa9ae65f834cb93dfe5)
+
+
+## Version 1.4.0
+
+The `rippled` 1.4.0 release introduces several improvements and new features, including support for deleting accounts, improved peer slot management, improved CI integration and package building and support for [C++17](https://en.wikipedia.org/wiki/C%2B%2B17) and [Boost 1.71](https://www.boost.org/users/history/version_1_71_0.html). Finally, this release removes the code for the `SHAMapV2` amendment which failed to gain majority support and has been obsoleted by other improvements.
+ 
+**New and Updated Features**
+- The `DeletableAccounts` amendment which, if enabled, will make it possible for users to delete unused or unneeded accounts, recovering the account's reserve.
+- Support for improved management of peer slots and the ability to add and removed reserved connections without requiring a restart of the server.
+- Tracking and reporting of cumulative and instantaneous peer bandwidth usage. 
+- Preliminary support for post-processing historical shards after downloading to index their contents.
+- Reporting the master public key alongside the ephemeral public key in the `validation` stream [subscriptions](https://xrpl.org/subscribe.html).
+- Reporting consensus phase changes in the `server` stream [subscription](https://xrpl.org/subscribe.html).
+
+**Bug Fixes**
+- The `fixPayChanRecipientOwnerDir` amendment which corrects a minor technical flaw that would cause a payment channel to not appear in the recipient's owner directory, which made it unnecessarily difficult for users to enumerate all their payment channels.
+- The `fixCheckThreading` amendment which corrects a minor technical flaw that caused checks to not be properly threaded against the account of the check's recipient.
+- Respect the `ssl_verify` configuration option in the `SSLHTTPDownloader` and `HTTPClient` classes.
+- Properly update the `server_state` when a server detects a disagreement between itself and the network.
+- Allow Ed25519 keys to be used with the `channel_authorize` command.
+
+## Version 1.3.1
+
+The `rippled` 1.3.1 release improves the built-in deadlock detection code, improves logging during process startup, changes the package build pipeline and improves the build documentation.
+
+**New and Updated Features**
+
+This release has no new features.
+
+**Bug Fixes**
+- Add a LogicError when a deadlock is detected (355a7b04)
+- Improve logging during process startup (7c24f7b1)
+
 ## Version 1.3.0
 The `rippled` 1.3.0 release introduces several new features and overall improvements to the codebase, including the `fixMasterKeyAsRegularKey` amendment, code to adjust the timing of the consensus process and support for decentralized validator domain verification. The release also includes miscellaneous improvements including in the transaction censorship detection code, transaction validation code, manifest parsing code, configuration file parsing code, log file rotation code, and in the build, continuous integration, testing and package building pipelines.
 
@@ -24,7 +357,7 @@ The `rippled` 1.3.0 release introduces several new features and overall improvem
 
 **Bug Fixes**
 - Improve ledger trie ancestry tracking to reduce unnecessary error messages.
-- More efficient detection of dry paths in the payment engine. Although not a transaction-breaking change, this should reduces spurious error messages in the log files.
+- More efficient detection of dry paths in the payment engine. Although not a transaction-breaking change, this should reduce spurious error messages in the log files.
 
 ## Version 1.2.4
 

SECURITY.md

@@ -0,0 +1,149 @@
+### Operating an XRP Ledger server securely
+
+For more details on operating an XRP Ledger server securely, please visit https://xrpl.org/manage-the-rippled-server.html.
+
+
+# Security Policy
+
+## Supported Versions
+
+Software constantly evolves. In order to focus resources, we only generally only accept vulnerability reports that affect recent and current versions of the software. We always accept reports for issues present in the **master**, **release** or **develop** branches, and with proposed, [open pull requests](https://github.com/ripple/rippled/pulls).
+
+## Identifying and Reporting Vulnerabilities
+
+We take security seriously and we do our best to ensure that all our releases are bug free. But we aren't perfect and sometimes things will slip through.
+
+### Responsible Investigation
+
+We urge you to examine our code carefully and responsibly, and to disclose any issues that you identify in a responsible fashion.
+
+Responsible investigation includes, but isn't limited to, the following:
+
+- Not performing tests on the main network. If testing is necessary, use the [Testnet or Devnet](https://xrpl.org/xrp-testnet-faucet.html).
+- Not targeting physical security measures, or attempting to use social engineering, spam, distributed denial of service (DDOS) attacks, etc.
+- Investigating bugs in a way that makes a reasonable, good faith effort not to be disruptive or harmful to the XRP Ledger and the broader ecosystem.
+
+### Responsible Disclosure
+
+If you discover a vulnerability or potential threat, or if you _think_
+you have, please reach out by dropping an email using the contact
+information below.
+
+Your report should include the following:
+
+- Your contact information (typically, an email address);
+- The description of the vulnerability;
+- The attack scenario (if any);
+- The steps to reproduce the vulnerability;
+- Any other relevant details or artifacts, including code, scripts or patches.
+
+In your mail, please describe of the issue or the potential threat; if possible, please include a "repro" (code that can reproduce the issue) or describe the best way to reproduce and replicate the issue. Please make your report as extensive as possible.
+
+For more information on responsible disclosure, please read this [Wikipedia article](https://en.wikipedia.org/wiki/Responsible_disclosure).
+
+## Report Handling Process
+
+Please report the bug directly to us and limit further disclosure. If you want to prove that you knew the bug as of a given time, consider using a cryptographic precommitment: hash the content of your report and publish the hash on a medium of your choice (e.g. on Twitter or as a memo in a transaction) as "proof" that you had written the text at a given point in time.
+
+Once we receive a report, we:
+
+1. Assign two people to independently evaluate the report;
+2. Consider their recommendations;
+3. If action is necessary, formulate a plan to address the issue;
+4. Communicate privately with the reporter to explain our plan.
+5. Prepare, test and release a version which fixes the issue; and
+6. Announce the vulnerability publicly.
+
+We will triage and respond to your disclosure within 24 hours. Beyond that, we will work to analyze the issue in more detail, formulate, develop and test a fix.
+
+While we commit to responding with 24 hours of your initial report with our triage assessment, we cannot guarantee a response time for the remaining steps. We will communicate with you throughout this process, letting you know where we are and keeping you updated on the timeframe.
+
+## Bug Bounty Program
+
+[Ripple](https://ripple.com) is generously sponsoring a bug bounty program for vulnerabilities in [`rippled`](https://github.com/ripple/rippled) (and other related projects, like [`ripple-lib`](https://github.com/ripple/ripple-lib)).
+
+This program allows us to recognize and reward individuals or groups that identify and report bugs. In summary, order to qualify for a bounty, the bug must be:
+
+1. **In scope**. Only bugs in software under the scope of the program qualify. Currently, that means `rippled` and `ripple-lib`.
+2. **Relevant**. A security issue, posing a danger to user funds, privacy or the operation of the XRP Ledger.
+3. **Original and previously unknown**. Bugs that are already known and discussed in public do not qualify. Previously reported bugs, even if publicly unknown, are not eligible.
+4. **Specific**. We welcome general security advice or recommendations, but we cannot pay bounties for that.
+5. **Fixable**. There has to be something we can do to permanently fix the problem. Note that bugs in other people’s software may still qualify in some cases. For example, if you find a bug in a library that we use which can compromises the security of software that is in scope and we can get it fixed, you may qualify for a bounty.
+6. **Unused**. If you use the exploit to attack the XRP Ledger, you do not qualify for a bounty. If you report a vulnerability used in an ongoing or past attack and there is specific, concrete evidence that suggests you are the attacker we reserve the right not to pay a bounty.
+
+The amount paid varies dramatically. Vulnerabilities that are harmless on their own, but could form part of a critical exploit will usually receive a bounty. Full-blown exploits can receive much higher bounties. Please don’t hold back partial vulnerabilities while trying to construct a full-blown exploit. We will pay a bounty to anyone who reports a complete chain of vulnerabilities even if they have reported each component of the exploit separately and those vulnerabilities have been fixed in the meantime. However, to qualify for a the full bounty, you must to have been the first to report each of the partial exploits.
+
+### Contacting Us
+
+To report a qualifying bug, please send a detailed report to:
+
+|Email Address|bugs@ripple.com                                      |
+|:-----------:|:----------------------------------------------------|
+|Short Key ID | `0xC57929BE`                                        |
+|Long Key ID  | `0xCD49A0AFC57929BE`                                |
+|Fingerprint  | `24E6 3B02 37E0 FA9C 5E96 8974 CD49 A0AF C579 29BE` |
+
+The full PGP key for this address, which is also available on several key servers (e.g. on [keys.gnupg.net](https://keys.gnupg.net)), is: 
+```
+-----BEGIN PGP PUBLIC KEY BLOCK-----
+mQINBFUwGHYBEAC0wpGpBPkd8W1UdQjg9+cEFzeIEJRaoZoeuJD8mofwI5Ejnjdt
+kCpUYEDal0ygkKobu8SzOoATcDl18iCrScX39VpTm96vISFZMhmOryYCIp4QLJNN
+4HKc2ZdBj6W4igNi6vj5Qo6JMyGpLY2mz4CZskbt0TNuUxWrGood+UrCzpY8x7/N
+a93fcvNw+prgCr0rCH3hAPmAFfsOBbtGzNnmq7xf3jg5r4Z4sDiNIF1X1y53DAfV
+rWDx49IKsuCEJfPMp1MnBSvDvLaQ2hKXs+cOpx1BCZgHn3skouEUxxgqbtTzBLt1
+xXpmuijsaltWngPnGO7mOAzbpZSdBm82/Emrk9bPMuD0QaLQjWr7HkTSUs6ZsKt4
+7CLPdWqxyY/QVw9UaxeHEtWGQGMIQGgVJGh1fjtUr5O1sC9z9jXcQ0HuIHnRCTls
+GP7hklJmfH5V4SyAJQ06/hLuEhUJ7dn+BlqCsT0tLmYTgZYNzNcLHcqBFMEZHvHw
+9GENMx/tDXgajKql4bJnzuTK0iGU/YepanANLd1JHECJ4jzTtmKOus9SOGlB2/l1
+0t0ADDYAS3eqOdOcUvo9ElSLCI5vSVHhShSte/n2FMWU+kMUboTUisEG8CgQnrng
+g2CvvQvqDkeOtZeqMcC7HdiZS0q3LJUWtwA/ViwxrVlBDCxiTUXCotyBWwARAQAB
+tDBSaXBwbGUgTGFicyBCdWcgQm91bnR5IFByb2dyYW0gPGJ1Z3NAcmlwcGxlLmNv
+bT6JAjcEEwEKACEFAlUwGHYCGwMFCwkIBwMFFQoJCAsFFgIDAQACHgECF4AACgkQ
+zUmgr8V5Kb6R0g//SwY/mVJY59k87iL26/KayauSoOcz7xjcST26l4ZHVVX85gOY
+HYZl8k0+m8X3zxeYm9a3QAoAml8sfoaFRFQP8ynnefRrLUPaZ2MjbJ0SACMwZNef
+T6o7Mi8LBAaiNZdYVyIfX1oM6YXtqYkuJdav6ZCyvVYqc9OvMJPY2ZzJYuI/ZtvQ
+/lTndxCeg9ALNX/iezOLGdfMpf4HuIFVwcPPlwGi+HDlB9/bggDEHC8z434SXVFc
+aQatXAPcDkjMUweU7y0CZtYEj00HITd4pSX6MqGiHrxlDZTqinCOPs1Ieqp7qufs
+MzlM6irLGucxj1+wa16ieyYvEtGaPIsksUKkywx0O7cf8N2qKg+eIkUk6O0Uc6eO
+CszizmiXIXy4O6OiLlVHGKkXHMSW9Nwe9GE95O8G9WR8OZCEuDv+mHPAutO+IjdP
+PDAAUvy+3XnkceO+HGWRpVvJZfFP2YH4A33InFL5yqlJmSoR/yVingGLxk55bZDM
++HYGR3VeMb8Xj1rf/02qERsZyccMCFdAvKDbTwmvglyHdVLu5sPmktxbBYiemfyJ
+qxMxmYXCc9S0hWrWZW7edktBa9NpE58z1mx+hRIrDNbS2sDHrib9PULYCySyVYcF
+P+PWEe1CAS5jqkR2ker5td2/pHNnJIycynBEs7l6zbc9fu+nktFJz0q2B+GJAhwE
+EAEKAAYFAlUwGaQACgkQ+tiY1qQ2QkjMFw//f2hNY3BPNe+1qbhzumMDCnbTnGif
+kLuAGl9OKt81VHG1f6RnaGiLpR696+6Ja45KzH15cQ5JJl5Bgs1YkR/noTGX8IAD
+c70eNwiFu8JXTaaeeJrsmFkF9Tueufb364risYkvPP8tNUD3InBFEZT3WN7JKwix
+coD4/BwekUwOZVDd/uCFEyhlhZsROxdKNisNo3VtAq2s+3tIBAmTrriFUl0K+ZC5
+zgavcpnPN57zMtW9aK+VO3wXqAKYLYmtgxkVzSLUZt2M7JuwOaAdyuYWAneKZPCu
+1AXkmyo+d84sd5mZaKOr5xArAFiNMWPUcZL4rkS1Fq4dKtGAqzzR7a7hWtA5o27T
+6vynuxZ1n0PPh0er2O/zF4znIjm5RhTlfjp/VmhZdQfpulFEQ/dMxxGkQ9z5IYbX
+mTlSDbCSb+FMsanRBJ7Drp5EmBIudVGY6SHI5Re1RQiEh7GoDfUMUwZO+TVDII5R
+Ra7WyuimYleJgDo/+7HyfuIyGDaUCVj6pwVtYtYIdOI3tTw1R1Mr0V8yaNVnJghL
+CHcEJQL+YHSmiMM3ySil3O6tm1By6lFz8bVe/rgG/5uklQrnjMR37jYboi1orCC4
+yeIoQeV0ItlxeTyBwYIV/o1DBNxDevTZvJabC93WiGLw2XFjpZ0q/9+zI2rJUZJh
+qxmKP+D4e27lCI65Ag0EVTAYdgEQAMvttYNqeRNBRpSX8fk45WVIV8Fb21fWdwk6
+2SkZnJURbiC0LxQnOi7wrtii7DeFZtwM2kFHihS1VHekBnIKKZQSgGoKuFAQMGyu
+a426H4ZsSmA9Ufd7kRbvdtEcp7/RTAanhrSL4lkBhaKJrXlxBJ27o3nd7/rh7r3a
+OszbPY6DJ5bWClX3KooPTDl/RF2lHn+fweFk58UvuunHIyo4BWJUdilSXIjLun+P
+Qaik4ZAsZVwNhdNz05d+vtai4AwbYoO7adboMLRkYaXSQwGytkm+fM6r7OpXHYuS
+cR4zB/OK5hxCVEpWfiwN71N2NMvnEMaWd/9uhqxJzyvYgkVUXV9274TUe16pzXnW
+ZLfmitjwc91e7mJBBfKNenDdhaLEIlDRwKTLj7k58f9srpMnyZFacntu5pUMNblB
+cjXwWxz5ZaQikLnKYhIvrIEwtWPyjqOzNXNvYfZamve/LJ8HmWGCKao3QHoAIDvB
+9XBxrDyTJDpxbog6Qu4SY8AdgVlan6c/PsLDc7EUegeYiNTzsOK+eq3G5/E92eIu
+TsUXlciypFcRm1q8vLRr+HYYe2mJDo4GetB1zLkAFBcYJm/x9iJQbu0hn5NxJvZO
+R0Y5nOJQdyi+muJzKYwhkuzaOlswzqVXkq/7+QCjg7QsycdcwDjiQh3OrsgXHrwl
+M7gyafL9ABEBAAGJAh8EGAEKAAkFAlUwGHYCGwwACgkQzUmgr8V5Kb50BxAAhj9T
+TwmNrgRldTHszj+Qc+v8RWqV6j+R+zc0cn5XlUa6XFaXI1OFFg71H4dhCPEiYeN0
+IrnocyMNvCol+eKIlPKbPTmoixjQ4udPTR1DC1Bx1MyW5FqOrsgBl5t0e1VwEViM
+NspSStxu5Hsr6oWz2GD48lXZWJOgoL1RLs+uxjcyjySD/em2fOKASwchYmI+ezRv
+plfhAFIMKTSCN2pgVTEOaaz13M0U+MoprThqF1LWzkGkkC7n/1V1f5tn83BWiagG
+2N2Q4tHLfyouzMUKnX28kQ9sXfxwmYb2sA9FNIgxy+TdKU2ofLxivoWT8zS189z/
+Yj9fErmiMjns2FzEDX+bipAw55X4D/RsaFgC+2x2PDbxeQh6JalRA2Wjq32Ouubx
+u+I4QhEDJIcVwt9x6LPDuos1F+M5QW0AiUhKrZJ17UrxOtaquh/nPUL9T3l2qPUn
+1ChrZEEEhHO6vA8+jn0+cV9n5xEz30Str9iHnDQ5QyR5LyV4UBPgTdWyQzNVKA69
+KsSr9lbHEtQFRzGuBKwt6UlSFv9vPWWJkJit5XDKAlcKuGXj0J8OlltToocGElkF
++gEBZfoOWi/IBjRLrFW2cT3p36DTR5O1Ud/1DLnWRqgWNBLrbs2/KMKE6EnHttyD
+7Tz8SQkuxltX/yBXMV3Ddy0t6nWV2SZEfuxJAQI=
+=spg4
+-----END PGP PUBLIC KEY BLOCK-----
+```