tsan.supp turned lock-order checking off wholesale: deadlock:pthread_create,
deadlock:pthread_rwlock_rdlock and deadlock:boost::asio name locking
primitives rather than source files, so between them they covered every
std::shared_mutex read lock and any inversion reached through a strand or a
thread start, and detect_deadlocks defaults to true. Seven more named source
files, among them ValidatorList.cpp, ValidatorSite.cpp and Manifest.cpp. Drop
all ten, because the job cannot fail a build, so a suppression here costs a
finding and buys nothing. The Manifest.cpp line alone was hiding a two-thread
deadlock between ManifestCache::save() and the validator list, which a run
without it reported 52 times on the shutdown path. Also repoint eight
patterns at files that moved into libxrpl and the gtest tree, and empty
sanitizer-ignorelist.txt of 24 entries that never matched anything, either
through suppression syntax a clang ignorelist never consults or through a
glob with no leading star. Both rules are now written down in
docs/build/sanitizers.md.
Add ubuntu-clang-debug-amd64-tsan to the Linux matrix, in a config of its
own so TSan stays on clang and Debug, and give the matrix a third tier so
it runs at night rather than on every labelled pull request. A Linux config
may now declare "extended", which holds it out of both the minimal and the
full matrix; generate.py emits those configs only for --extended, which the
workflow passes on a schedule or a manual run. The name says which matrix a
config belongs to, like "minimal", rather than naming a trigger, because the
trigger set already grew from the schedule to manual runs and a config
property outlives that. It avoids "maximal", which reads as a synonym for
the full matrix it is meant to be larger than. It is a whole hour of
runner time, which is too much to spend per pull request, and it reports
nothing back anyway, because the workflow appends exitcode=0 to
TSAN_OPTIONS. That belongs in the workflow rather than in
runtime-tsan-options.txt, which the documented local command also reads and
which must keep failing on a finding.
Cap the test jobs at 4 under TSan, which is measured rather than chosen.
One per core is 30 on the current runner and twice starved it until it lost
contact with the server; 12 was killed by the OOM killer with code 137
before a single suite finished; 4 completes the suite. The machine has 32
cores and no swap, so nothing absorbs the peak, and the runner is one pod
among several on a node, so the ceiling is not ours alone.
Check that a build carries the instrumentation it asked for, by the __asan,
__tsan and __ubsan symbols in the binary, since instrumented code
references its runtime however that runtime is linked. The version string
is checked too, but cannot stand alone, because cmake sets the SANITIZERS
macro separately from the flags. Widen the voidstar step for this rather
than add a second one, and add ASAN_ENABLED, TSAN_ENABLED and UBSAN_ENABLED
so a step needing one sanitizer does not parse the list. Define XRPL_ASAN,
XRPL_TSAN and XRPL_UBSAN so a test can skip when its sanitizer is inactive,
and drop the -Dcoverage_test_parallelism example from BUILD.md, which
neither cmake nor conanfile.py defines.
This change moves the sanitizer runtime options out to dedicated files, such that they can be used in multiple places (CI, local runs) without any need to rewrite them.
This change adds support for sanitizer build options in CI builds workflow. Currently `asan+ubsan` is enabled, while `tsan+ubsan` is left disabled as more changes are required.
The .gitignore and .gitattributes files contain references to files and directories that the current build no longer produces, so this change removes obsolete entries in these files, and does some general reorganizing of the remaining entries.
This change updates BUILD.md for Conan 2, add fixes/workarounds for Apple Clang 17, Clang 20 and CMake 4. This also removes (from BUILD.md only) workarounds for compiler versions which we no longer support e.g. Clang 15 and adds compilation flag -Wno-deprecated-declarations to enable building with Clang 20 on Linux.
- Detects if the consensus process is "stalled". If it is, then we can declare a
consensus and end successfully even if we do not have 80% agreement on
our proposal.
- "Stalled" is defined as:
- We have a close time consensus
- Each disputed transaction is individually stalled:
- It has been in the final "stuck" 95% requirement for at least 2
(avMIN_ROUNDS) "inner rounds" of phaseEstablish,
- and either all of the other trusted proposers or this validator, if proposing,
have had the same vote(s) for at least 4 (avSTALLED_ROUNDS) "inner
rounds", and at least 80% of the validators (including this one, if
appropriate) agree about the vote (whether yes or no).
- If we have been in the establish phase for more than 10x the previous
consensus establish phase's time, then consensus is considered "expired",
and we will leave the round, which sends a partial validation (indicating
that the node is moving on without validating). Two restrictions avoid
prematurely exiting, or having an extended exit in extreme situations.
- The 10x time is clamped to be within a range of 15s
(ledgerMAX_CONSENSUS) to 120s (ledgerABANDON_CONSENSUS).
- If consensus has not had an opportunity to walk through all avalanche
states (defined as not going through 8 "inner rounds" of phaseEstablish),
then ConsensusState::Expired is treated as ConsensusState::No.
- When enough nodes leave the round, any remaining nodes will see they've
fallen behind, and move on, too, generally before hitting the timeout. Any
validations or partial validations sent during this time will help the
consensus process bring the nodes back together.
On macOS, if you have not installed something that depends on `xz`, then your
system may lack `lzma`, resulting in a build error similar to:
```
Downloading libarchive-3.6.0.tar.xz completed [6250.61k]
libarchive/3.6.0:
ERROR: libarchive/3.6.0: Error in source() method, line 120
get(self, **self.conan_data["sources"][self.version], strip_root=True)
ReadError: file could not be opened successfully:
- method gz: ReadError('not a gzip file')
- method bz2: ReadError('not a bzip2 file')
- method xz: CompressionError('lzma module is not available')
- method tar: ReadError('invalid header')
```
The solution is to ensure that `lzma` is installed by installing `xz`.
Add instructions for installing rippled using the package managers APT
and YUM. Some steps were adapted from xrpl.org.
---------
Co-authored-by: Michael Legleux <mlegleux@ripple.com>
Make it easy for projects to depend on libxrpl by adding an `ALIAS`
target named `xrpl::libxrpl` for projects to link.
The name was chosen because:
* The current library target is named `xrpl_core`. There is no other
"non-core" library target against which we need to distinguish the
"core" library. We only export one library target, and it should just
be named after the project to keep things simple and predictable.
* Underscores in target or library names are generally discouraged.
* Every target exported in CMake should be prefixed with the project
name.
By adding an `ALIAS` target, existing consumers who use the `xrpl_core`
target will not be affected.
* In the future, there can be a migration plan to make `xrpl_core` the
`ALIAS` target (and `libxrpl` the "real" target, which will affect the
filename of the compiled binary), and eventually remove it entirely.
Also:
* Fix the Conan recipe so that consumers using Conan import a target
named `xrpl::libxrpl`. This way, every consumer can use the same
instructions.
* Document the two easiest methods to depend on libxrpl. Both have been
tested.
* See #4443.
* Remove obsolete build instructions.
* By using Conan, builders can choose which dependencies specifically to
build and link as shared objects.
* Refactor the build instructions based on the plan in #4433.
