mirror of https://github.com/XRPLF/rippled.git synced 2025-11-19 18:45:52 +00:00

Files

Mayukha Vadari 911c0466c0 Merge develop into ripple/smart-escrow (#5357 )

* Set version to 2.4.0

* refactor: Remove unused and add missing includes (#5293)

The codebase is filled with includes that are unused, and which thus can be removed. At the same time, the files often do not include all headers that contain the definitions used in those files. This change uses clang-format and clang-tidy to clean up the includes, with minor manual intervention to ensure the code compiles on all platforms.

* refactor: Calculate numFeatures automatically (#5324)

Requiring manual updates of numFeatures is an annoying manual process that is easily forgotten, and leads to frequent merge conflicts. This change takes advantage of the `XRPL_FEATURE` and `XRPL_FIX` macros, and adds a new `XRPL_RETIRE` macro to automatically set `numFeatures`.

* refactor: Improve ordering of headers with clang-format (#5343)

Removes all manual header groupings from source and header files by leveraging clang-format options.

* Rename "deadlock" to "stall" in `LoadManager` (#5341)

What the LoadManager class does is stall detection, which is not the same as deadlock detection. In the condition of severe CPU starvation, LoadManager will currently intentionally crash rippled reporting `LogicError: Deadlock detected`. This error message is misleading as the condition being detected is not a deadlock. This change fixes and refactors the code in response.

* Adds hub.xrpl-commons.org as a new Bootstrap Cluster (#5263)

* fix: Error message for ledger_entry rpc (#5344)

Changes the error to `malformedAddress` for `permissioned_domain` in the `ledger_entry` rpc, when the account is not a string. This change makes it more clear to a user what is wrong with their request.

* fix: Handle invalid marker parameter in grpc call (#5317)

The `end_marker` is used to limit the range of ledger entries to fetch. If `end_marker` is less than `marker`, a crash can occur. This change adds an additional check.

* fix: trust line RPC no ripple flag (#5345)

The Trustline RPC `no_ripple` flag gets set depending on `lsfDefaultRipple` flag, which is not a flag of a trustline but of the account root. The `lsfDefaultRipple` flag does not provide any insight if this particular trust line has `lsfLowNoRipple` or `lsfHighNoRipple` flag set, so it should not be used here at all. This change simplifies the logic.

* refactor: Updates Conan dependencies: RocksDB (#5335)

Updates RocksDB to version 9.7.3, the latest version supported in Conan 1.x. A patch for 9.7.4 that fixes a memory leak is included.

* fix: Remove null pointer deref, just do abort (#5338)

This change removes the existing undefined behavior from `LogicError`, so we can be certain that there will be always a stacktrace.

De-referencing a null pointer is an old trick to generate `SIGSEGV`, which would typically also create a stacktrace. However it is also an undefined behaviour and compilers can do something else. A more robust way to create a stacktrace while crashing the program is to use `std::abort`, which we have also used in this location for a long time. If we combine the two, we might not get the expected behaviour - namely, the nullpointer deref followed by `std::abort`, as handled in certain compiler versions may not immediately cause a crash. We have observed stacktrace being wiped instead, and thread put in indeterminate state, then stacktrace created without any useful information.

* chore: Add PR number to payload (#5310)

This PR adds one more payload field to the libXRPL compatibility check workflow - the PR number itself.

* chore: Update link to ripple-binary-codec (#5355)

The link to ripple-binary-codec's definitions.json appears to be outdated. The updated link is also documented here: https://xrpl.org/docs/references/protocol/binary-format#definitions-file

* Prevent consensus from getting stuck in the establish phase (#5277)

- Detects if the consensus process is "stalled". If it is, then we can declare a
consensus and end successfully even if we do not have 80% agreement on
our proposal.
- "Stalled" is defined as:
- We have a close time consensus
- Each disputed transaction is individually stalled:
- It has been in the final "stuck" 95% requirement for at least 2
(avMIN_ROUNDS) "inner rounds" of phaseEstablish,
- and either all of the other trusted proposers or this validator, if proposing,
have had the same vote(s) for at least 4 (avSTALLED_ROUNDS) "inner
rounds", and at least 80% of the validators (including this one, if
appropriate) agree about the vote (whether yes or no).
- If we have been in the establish phase for more than 10x the previous
consensus establish phase's time, then consensus is considered "expired",
and we will leave the round, which sends a partial validation (indicating
that the node is moving on without validating). Two restrictions avoid
prematurely exiting, or having an extended exit in extreme situations.
- The 10x time is clamped to be within a range of 15s
(ledgerMAX_CONSENSUS) to 120s (ledgerABANDON_CONSENSUS).
- If consensus has not had an opportunity to walk through all avalanche
states (defined as not going through 8 "inner rounds" of phaseEstablish),
then ConsensusState::Expired is treated as ConsensusState::No.
- When enough nodes leave the round, any remaining nodes will see they've
fallen behind, and move on, too, generally before hitting the timeout. Any
validations or partial validations sent during this time will help the
consensus process bring the nodes back together.

---------

Co-authored-by: Michael Legleux <mlegleux@ripple.com>
Co-authored-by: Bart <bthomee@users.noreply.github.com>
Co-authored-by: Ed Hennis <ed@ripple.com>
Co-authored-by: Bronek Kozicki <brok@incorrekt.com>
Co-authored-by: Darius Tumas <Tokeiito@users.noreply.github.com>
Co-authored-by: Sergey Kuznetsov <skuznetsov@ripple.com>
Co-authored-by: cyan317 <120398799+cindyyan317@users.noreply.github.com>
Co-authored-by: Vlad <129996061+vvysokikh1@users.noreply.github.com>
Co-authored-by: Alex Kremer <akremer@ripple.com>

2025-03-20 16:47:14 -04:00

results

Merge develop into ripple/smart-escrow (#5357 )

2025-03-20 16:47:14 -04:00

levelization.sh

Fix levelization script to ignore commented includes (#5194 )

2025-01-16 15:23:40 -05:00

README.md

Refactor to fix levelization:

2022-03-01 14:32:14 -08:00

README.md

Levelization

Levelization is the term used to describe efforts to prevent rippled from having or creating cyclic dependencies.

rippled code is organized into directories under src/rippled (and src/test) representing modules. The modules are intended to be organized into "tiers" or "levels" such that a module from one level can only include code from lower levels. Additionally, a module in one level should never include code in an impl folder of any level other than it's own.

Unfortunately, over time, enforcement of levelization has been inconsistent, so the current state of the code doesn't necessarily reflect these rules. Whenever possible, developers should refactor any levelization violations they find (by moving files or individual classes). At the very least, don't make things worse.

The table below summarizes the desired division of modules, based on the state of the rippled code when it was created. The levels are numbered from the bottom up with the lower level, lower numbered, more independent modules listed first, and the higher level, higher numbered modules with more dependencies listed later.

tl;dr: The modules listed first are more independent than the modules listed later.

Level / Tier	Module(s)
01	ripple/beast ripple/unity
02	ripple/basics
03	ripple/json ripple/crypto
04	ripple/protocol
05	ripple/core ripple/conditions ripple/consensus ripple/resource ripple/server
06	ripple/peerfinder ripple/ledger ripple/nodestore ripple/net
07	ripple/shamap ripple/overlay
08	ripple/app
09	ripple/rpc
10	ripple/perflog
11	test/jtx test/beast test/csf
12	test/unit_test
13	test/crypto test/conditions test/json test/resource test/shamap test/peerfinder test/basics test/overlay
14	test
15	test/net test/protocol test/ledger test/consensus test/core test/server test/nodestore
16	test/rpc test/app

(Note that test levelization is much less important and much less strictly enforced than ripple levelization, other than the requirement that test code should never be included in ripple code.)

Validation

The levelization.sh script takes no parameters, reads no environment variables, and can be run from any directory, as long as it is in the expected location in the rippled repo. It can be run at any time from within a checked out repo, and will do an analysis of all the #includes in the rippled source. The only caveat is that it runs much slower under Windows than in Linux. It hasn't yet been tested under MacOS. It generates many files of results:

rawincludes.txt: The raw dump of the #includes
paths.txt: A second dump grouping the source module to the destination module, deduped, and with frequency counts.
includes/: A directory where each file represents a module and contains a list of modules and counts that the module includes.
includedby/: Similar to includes/, but the other way around. Each file represents a module and contains a list of modules and counts that include the module.
loops.txt: A list of direct loops detected between modules as they actually exist, as opposed to how they are desired as described above. In a perfect repo, this file will be empty. This file is committed to the repo, and is used by the levelization Github workflow to validate that nothing changed.
ordering.txt: A list showing relationships between modules where there are no loops as they actually exist, as opposed to how they are desired as described above. This file is committed to the repo, and is used by the levelization Github workflow to validate that nothing changed.
levelization.yml Github Actions workflow to test that levelization loops haven't changed. Unfortunately, if changes are detected, it can't tell if they are improvements or not, so if you have resolved any issues or done anything else to improve levelization, run levelization.sh, and commit the updated results.

The loops.txt and ordering.txt files relate the modules using comparison signs, which indicate the number of times each module is included in the other.

A > B means that A should probably be at a higher level than B, because B is included in A significantly more than A is included in B. These results can be included in both loops.txt and ordering.txt. Because ordering.txtonly includes relationships where B is not included in A at all, it will only include these types of results.
A ~= B means that A and B are included in each other a different number of times, but the values are so close that the script can't definitively say that one should be above the other. These results will only be included in loops.txt.
A == B means that A and B include each other the same number of times, so the script has no clue which should be higher. These results will only be included in loops.txt.

The committed files hide the detailed values intentionally, to prevent false alarms and merging issues, and because it's easy to get those details locally.

Run levelization.sh
Grep the modules in paths.txt.
- For example, if a cycle is found A ~= B, simply grep -w A Builds/levelization/results/paths.txt | grep -w B