clio/docs/coverage-report.md

Coverage report

Coverage report is intended for developers using compilers GCC or Clang (including Apple Clang). It is generated by the build target coverage_report, which is only enabled when both tests and coverage options are set (e.g., with -o coverage=True -o tests=True in conan).

Prerequisites

To generate the coverage report you need:

• gcovr tool (can be installed e.g. with pip install gcovr)
• gcov for GCC (installed with the compiler by default)
• llvm-cov for Clang (installed with the compiler by default, also on Apple)
• Debug build type

Creating the coverage report

The coverage report is created when the following steps are completed, in order:

1. clio_tests binary built with the instrumentation data, enabled by the coverage option mentioned above.
2. Completed run of unit tests, which populates coverage capture data.
3. Completed run of gcovr tool, which internally invokes either gcov or llvm-cov to assemble both instrumentation data and coverage capture data into a coverage report.

The above steps are automated into a single target coverage_report. The instrumented clio_tests binary can also be used for running regular unit tests.

In case of a spurious failure of unit tests, it is possible to re-run the coverage_report target without rebuilding the clio_tests binary (since it is simply a dependency of the coverage report target).

The default coverage report format is html-details, but developers can override it to any of the formats listed in cmake/CodeCoverage.cmake by setting CODE_COVERAGE_REPORT_FORMAT variable in cmake. For example, CI is setting this parameter to xml for the codecov integration.

If some unit tests predictably fail (e.g., due to absence of a Cassandra database), it is possible to set unit tests options in the CODE_COVERAGE_TESTS_ARGS cmake variable, as demonstrated below:

cd .build
conan install .. --output-folder . --build missing --settings build_type=Debug -o tests=True -o coverage=True
cmake -DCODE_COVERAGE_REPORT_FORMAT=json-details -DCMAKE_BUILD_TYPE=Debug -DCODE_COVERAGE_TESTS_ARGS="--gtest_filter=-BackendCassandra*" -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake ..
cmake --build . --target coverage_report

After the coverage_report target is completed, the generated coverage report will be stored inside the build directory as either:

• A File named coverage_report.*, with a suitable extension for the report format.
• A Directory named coverage_report, with index.html and other files inside, for html-details or html-nested report formats.