mirror of
https://github.com/XRPLF/rippled.git
synced 2026-09-27 23:38:08 +00:00
ci: Detect TSan lock-order inversions and add a TSan CI job
tsan.supp turned lock-order checking off wholesale: deadlock:pthread_create, deadlock:pthread_rwlock_rdlock and deadlock:boost::asio name locking primitives rather than source files, so between them they covered every std::shared_mutex read lock and any inversion reached through a strand or a thread start, and detect_deadlocks defaults to true. Seven more named source files, among them ValidatorList.cpp, ValidatorSite.cpp and Manifest.cpp. Drop all ten, because the job cannot fail a build, so a suppression here costs a finding and buys nothing. The Manifest.cpp line alone was hiding a two-thread deadlock between ManifestCache::save() and the validator list, which a run without it reported 52 times on the shutdown path. Also repoint eight patterns at files that moved into libxrpl and the gtest tree, and empty sanitizer-ignorelist.txt of 24 entries that never matched anything, either through suppression syntax a clang ignorelist never consults or through a glob with no leading star. Both rules are now written down in docs/build/sanitizers.md. Add ubuntu-clang-debug-amd64-tsan to the Linux matrix, in a config of its own so TSan stays on clang and Debug, and give the matrix a third tier so it runs at night rather than on every labelled pull request. A Linux config may now declare "extended", which holds it out of both the minimal and the full matrix; generate.py emits those configs only for --extended, which the workflow passes on a schedule or a manual run. The name says which matrix a config belongs to, like "minimal", rather than naming a trigger, because the trigger set already grew from the schedule to manual runs and a config property outlives that. It avoids "maximal", which reads as a synonym for the full matrix it is meant to be larger than. It is a whole hour of runner time, which is too much to spend per pull request, and it reports nothing back anyway, because the workflow appends exitcode=0 to TSAN_OPTIONS. That belongs in the workflow rather than in runtime-tsan-options.txt, which the documented local command also reads and which must keep failing on a finding. Cap the test jobs at 4 under TSan, which is measured rather than chosen. One per core is 30 on the current runner and twice starved it until it lost contact with the server; 12 was killed by the OOM killer with code 137 before a single suite finished; 4 completes the suite. The machine has 32 cores and no swap, so nothing absorbs the peak, and the runner is one pod among several on a node, so the ceiling is not ours alone. Check that a build carries the instrumentation it asked for, by the __asan, __tsan and __ubsan symbols in the binary, since instrumented code references its runtime however that runtime is linked. The version string is checked too, but cannot stand alone, because cmake sets the SANITIZERS macro separately from the flags. Widen the voidstar step for this rather than add a second one, and add ASAN_ENABLED, TSAN_ENABLED and UBSAN_ENABLED so a step needing one sanitizer does not parse the list. Define XRPL_ASAN, XRPL_TSAN and XRPL_UBSAN so a test can skip when its sanitizer is inactive, and drop the -Dcoverage_test_parallelism example from BUILD.md, which neither cmake nor conanfile.py defines.
This commit is contained in:
37
.github/scripts/strategy-matrix/generate.py
vendored
37
.github/scripts/strategy-matrix/generate.py
vendored
@@ -60,6 +60,11 @@ def get_cmake_args(build_type: str, extra_args: str) -> str:
|
||||
# Every config must declare 'minimal'. Minimal configs form the reduced matrix
|
||||
# built for pull requests by default; the full matrix adds the rest.
|
||||
#
|
||||
# A Linux config may instead declare 'extended'. Neither the minimal nor the full
|
||||
# matrix includes such a config; only the extended matrix does, which the nightly
|
||||
# schedule and a manual run ask for. Use it for a config too expensive to run per
|
||||
# pull request. It is Linux only because nothing else needs it yet.
|
||||
#
|
||||
# Configs may also opt into 'benchmark' to smoke-run the benchmarks, or carry a
|
||||
# 'package' map to be packaged as well. Note that either applies to every entry
|
||||
# a config expands into, so only set them on configs that expand to a single
|
||||
@@ -94,6 +99,7 @@ class LinuxConfig:
|
||||
build_type: list[str]
|
||||
arch: list[str]
|
||||
minimal: bool
|
||||
extended: bool = False
|
||||
benchmark: bool = False # if true, smoke-run the benchmarks after testing
|
||||
sanitizers: list[str] = dataclasses.field(default_factory=list)
|
||||
suffix: str = ""
|
||||
@@ -215,12 +221,22 @@ _ARCHS: dict[str, Architecture] = {
|
||||
}
|
||||
|
||||
|
||||
def expand_linux_matrix(linux: LinuxFile, minimal: bool) -> list[MatrixEntry]:
|
||||
def expand_linux_matrix(
|
||||
linux: LinuxFile, minimal: bool, extended: bool = False
|
||||
) -> list[MatrixEntry]:
|
||||
"""Expand a LinuxFile into a flat list of matrix entries.
|
||||
|
||||
Each config entry is expanded over the cross-product of its
|
||||
compiler, build_type, sanitizers, and architecture lists. When 'minimal' is
|
||||
true, only configs flagged as minimal are included.
|
||||
|
||||
@param linux The parsed linux.json.
|
||||
@param minimal Emit only the configs flagged 'minimal'.
|
||||
@param extended Emit the configs flagged 'extended'. Both the minimal and
|
||||
the full matrix leave those out, so this is the only way to get them.
|
||||
@return One entry per combination the surviving configs expand into.
|
||||
@note Do not set both 'minimal' and 'extended'. The command line rejects the
|
||||
pair, but a direct caller gets the minimal matrix rather than an error.
|
||||
"""
|
||||
entries: list[MatrixEntry] = []
|
||||
|
||||
@@ -228,6 +244,8 @@ def expand_linux_matrix(linux: LinuxFile, minimal: bool) -> list[MatrixEntry]:
|
||||
for cfg in configs:
|
||||
if minimal and not cfg.minimal:
|
||||
continue
|
||||
if not extended and cfg.extended:
|
||||
continue
|
||||
# An empty sanitizers list means "one entry with no sanitizer".
|
||||
effective_sanitizers = cfg.sanitizers or [""]
|
||||
effective_archs = {arch: _ARCHS[arch] for arch in cfg.arch}
|
||||
@@ -367,7 +385,12 @@ if __name__ == "__main__":
|
||||
help="Emit the Linux packaging matrix instead of the build/test matrix.",
|
||||
action="store_true",
|
||||
)
|
||||
parser.add_argument(
|
||||
# Each flag picks a matrix size, and the sizes nest: minimal is a subset of
|
||||
# the full matrix, which is a subset of extended. So one flag narrows and the
|
||||
# other widens the same default, and asking for both has no answer. Argparse
|
||||
# rejects the pair rather than letting the filters intersect into a surprise.
|
||||
size = parser.add_mutually_exclusive_group()
|
||||
size.add_argument(
|
||||
"-m",
|
||||
"--minimal",
|
||||
help="Emit only the minimal matrix (the configs flagged 'minimal'), "
|
||||
@@ -375,6 +398,14 @@ if __name__ == "__main__":
|
||||
"emitted.",
|
||||
action="store_true",
|
||||
)
|
||||
size.add_argument(
|
||||
"-x",
|
||||
"--extended",
|
||||
help="Emit the extended matrix: the full one plus the configs flagged "
|
||||
"'extended', which no other matrix includes. Used for the nightly "
|
||||
"schedule and for a manual run.",
|
||||
action="store_true",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
matrix: list[MatrixEntry] | list[PackagingEntry] = []
|
||||
@@ -388,7 +419,7 @@ if __name__ == "__main__":
|
||||
else:
|
||||
if args.config in ("linux", None):
|
||||
matrix += expand_linux_matrix(
|
||||
LinuxFile.load(THIS_DIR / "linux.json"), args.minimal
|
||||
LinuxFile.load(THIS_DIR / "linux.json"), args.minimal, args.extended
|
||||
)
|
||||
if args.config in ("macos", None):
|
||||
matrix += expand_platform_matrix(
|
||||
|
||||
8
.github/scripts/strategy-matrix/linux.json
vendored
8
.github/scripts/strategy-matrix/linux.json
vendored
@@ -38,6 +38,14 @@
|
||||
"minimal": false,
|
||||
"sanitizers": ["address", "undefinedbehavior"]
|
||||
},
|
||||
{
|
||||
"compiler": ["clang"],
|
||||
"build_type": ["Debug"],
|
||||
"arch": ["amd64"],
|
||||
"minimal": false,
|
||||
"extended": true,
|
||||
"sanitizers": ["thread"]
|
||||
},
|
||||
|
||||
{
|
||||
"compiler": ["clang"],
|
||||
|
||||
Reference in New Issue
Block a user