Add coverage_report target (#1058)

This commit is contained in:
Bronek Kozicki
2023-12-21 15:08:32 +00:00
committed by GitHub
parent fe9649d872
commit a8bae96ad4
8 changed files with 602 additions and 132 deletions

View File

@@ -292,6 +292,55 @@ E.g.:
export CLIO_CLANG_TIDY_BIN=/opt/homebrew/opt/llvm@17/bin/clang-tidy
```
## Coverage report
Coverage report is intended for the developers using compilers GCC
or Clang (including Apple Clang). It is generated by the build target `coverage_report`,
which is only enabled when both `tests` and `coverage` options are set, e.g. with
`-o coverage=True -o tests=True` in `conan`
Prerequisites for the coverage report:
- [gcovr tool](https://gcovr.com/en/stable/getting-started.html) (can be installed e.g. with `pip install gcovr`)
- `gcov` for GCC (installed with the compiler by default) or
- `llvm-cov` for Clang (installed with the compiler by default, also on Apple)
- `Debug` build type
Coverage report is created when the following steps are completed, in order:
1. `clio_tests` binary built with the instrumentation data, enabled by the `coverage`
option mentioned above
2. completed run of unit tests, which populates coverage capture data
3. completed run of `gcovr` tool (which internally invokes either `gcov` or `llvm-cov`)
to assemble both instrumentation data and coverage capture data into a coverage report
Above steps are automated into a single target `coverage_report`. The instrumented
`clio_tests` binary can be also used for running regular unit tests. In case of a
spurious failure of unit tests, it is possile to re-run `coverage_report` target without
rebuilding the `clio_tests` binary (since it is simply a dependency of the coverage report target).
The default coverage report format is `html-details`, but the developers
can override it to any of the formats listed in `CMake/CodeCoverage.cmake`
by setting `CODE_COVERAGE_REPORT_FORMAT` variable in `cmake`. For example, CI
is setting this parameter to `xml` for the [codecov](codecov.io) integration.
In case if some unit tests predictably fail e.g. due to absence of a Cassandra database, it is possible
to set unit tests options in `CODE_COVERAGE_TESTS_ARGS` cmake variable, as demonstrated below:
```
cd .build
conan install .. --output-folder . --build missing --settings build_type=Debug -o tests=True -o coverage=True
cmake -DCODE_COVERAGE_REPORT_FORMAT=json-details -DCMAKE_BUILD_TYPE=Debug -DCODE_COVERAGE_TESTS_ARGS="--gtest_filter=-BackendCassandra*" -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake ..
cmake --build . --target coverage_report
```
After `coverage_report` target is completed, the generated coverage report will be
stored inside the build directory, as either of:
- file named `coverage_report.*`, with a suitable extension for the report format, or
- directory named `coverage_report`, with `index.html` and other files inside, for `html-details` or `html-nested` report formats.
## Developing against `rippled` in standalone mode
If you wish you develop against a `rippled` instance running in standalone