Compare commits

..

238 Commits

Author SHA1 Message Date
Valentin Balaschenko
068cea41b6 Track latencies of certain code blocks, and log if they take too long 2025-06-15 22:01:36 +09:00
John Freeman
4cf97ad212 Use error codes throughout fast Base58 implementation 2025-06-15 21:59:37 +09:00
Mayukha Vadari
a0e19f9dcd Improve error handling in some RPC commands 2025-06-15 21:59:24 +09:00
Alex Kremer
28a0d88e5c Add xrpl.libpp as an exported lib in conan (#5022) 2025-06-15 21:30:47 +09:00
Gregory Tsipenyuk
7755a03c1f Fix Oracle's token pair deterministic order: (#5021)
Price Oracle data-series logic uses `unordered_map` to update the Oracle object.
This results in different servers disagreeing on the order of that hash table.
Consequently, the generated ledgers will have different hashes.
The fix uses `map` instead to guarantee the order of the token pairs
in the data-series.
2025-06-15 21:30:38 +09:00
Gregory Tsipenyuk
06606c124e Fix last Liquidity Provider withdrawal:
Due to the rounding, LPTokenBalance of the last
Liquidity Provider (LP), might not match this LP's
trustline balance. This fix sets LPTokenBalance on
last LP withdrawal to this LP's LPToken trustline
balance.
2025-06-15 21:29:28 +09:00
Gregory Tsipenyuk
abd0621fef Fix offer crossing via single path AMM with transfer fee:
Single path AMM offer has to factor in the transfer in rate
when calculating the upper bound quality and the quality function
because single path AMM's offer quality is not constant.
This fix factors in the transfer fee in
BookStep::adjustQualityWithFees().
2025-06-15 21:29:12 +09:00
Gregory Tsipenyuk
0609949d25 Fix adjustAmountsByLPTokens():
The fix is to return the actual adjusted lp tokens and amounts
by the function.
2025-06-15 21:29:03 +09:00
Gregory Tsipenyuk
cac001b826 Add the fixAMMOfferRounding amendment: (#4983)
* Fix AMM offer rounding and low quality LOB offer blocking AMM:

A single-path AMM offer with account offer on DEX, is always generated
starting with the takerPays first, which is rounded up, and then
the takerGets, which is rounded down. This rounding ensures that the pool's
product invariant is maintained. However, when one of the offer's side
is XRP, this rounding can result in the AMM offer having a lower
quality, potentially causing offer generation to fail if the quality
is lower than the account's offer quality.

To address this issue, the proposed fix adjusts the offer generation process
to start with the XRP side first and always rounds it down. This results
in a smaller offer size, improving the offer's quality. Regardless if the offer
has XRP or not, the rounding is done so that the offer size is minimized.
This change still ensures the product invariant, as the other generated
side is the exact result of the swap-in or swap-out equations.

If a liquidity can be provided by both AMM and LOB offer on offer crossing
then AMM offer is generated so that it matches LOB offer quality. If LOB
offer quality is less than limit quality then generated AMM offer quality
is also less than limit quality and the offer doesn't cross. To address
this issue, if LOB quality is better than limit quality then use LOB
quality to generate AMM offer. Otherwise, don't use the quality to generate
AMM offer. In this case, limitOut() function in StrandFlow limits
the out amount to match strand's quality to limit quality and consume
maximum AMM liquidity.
2025-06-15 21:28:21 +09:00
Gregory Tsipenyuk
0601fd4916 Price Oracle: validate input parameters and extend test coverage: (#5013)
* Price Oracle: validate input parameters and extend test coverage:

Validate trim, time_threshold, document_id are valid
Int, UInt, or string convertible to UInt. Validate base_asset
and quote_asset are valid currency. Update error codes.
Extend Oracle and GetAggregatePrice unit-tests.
Denote unreachable coverage code.

* Set one-line LCOV_EXCL_LINE

* Move ledger_entry tests to LedgerRPC_test.cpp

* Add constants for "None"

* Fix LedgerRPC test

---------

Co-authored-by: Scott Determan <scott.determan@yahoo.com>
2025-06-15 21:27:12 +09:00
Michael Legleux
b6285bfda2 Add external directory to Conan recipe's exports (#5006) 2025-06-15 21:27:01 +09:00
John Freeman
88ccbecc87 Add missing includes (#5011) 2025-06-15 21:26:51 +09:00
seelabs
6adce3f703 Remove flow assert: (#5009)
Rounding in the payment engine is causing an assert to sometimes fire
with "dust" amounts. This is causing issues when running debug builds of
rippled. This issue will be addressed, but the assert is no longer
serving its purpose.
2025-06-15 21:26:39 +09:00
seelabs
ad5a1d2385 fix amendment: AMM swap should honor invariants: (#5002)
The AMM has an invariant for swaps where:
new_balance_1*new_balance_2 >= old_balance_1*old_balance_2

Due to rounding, this invariant could sometimes be violated (although by
very small amounts).

This patch introduces an amendment `fixAMMRounding` that changes the
rounding to always favor the AMM. Doing this should maintain the
invariant.

Co-authored-by: Bronek Kozicki
Co-authored-by: thejohnfreeman
2025-06-15 21:26:10 +09:00
seelabs
00f2e28a7b Add global access to the current ledger rules:
It can be difficult to make transaction breaking changes to low level
code because the low level code does not have access to a ledger and the
current activated amendments in that ledger (the "rules"). This patch
adds global access to the current ledger rules as a `std::optional`. If
the optional is not seated, then there is no active transaction.
2025-06-15 21:20:55 +09:00
Snoppy
05618c9b98 chore: fix typos (#4958) 2025-06-15 21:20:45 +09:00
Ed Hennis
09cb6a884a test: Add RPC error checking support to unit tests (#4987) 2025-06-15 21:20:23 +09:00
John Freeman
1caf3f0fee Ignore more commits 2025-06-15 21:13:18 +09:00
John Freeman
796ca637d6 Address compiler warnings 2025-06-15 21:13:08 +09:00
John Freeman
946d34b25d Add markers around source lists 2025-06-15 21:12:57 +09:00
John Freeman
f3be6e8a73 Fix source lists 2025-06-15 21:12:30 +09:00
Pretty Printer
47ddb288e6 Rewrite includes
$ find src/ripple/ src/test/ -type f -exec sed -i 's:include\s*["<]ripple/\(.*\)\.h\(pp\)\?[">]:include <ripple/\1.h>:' {} +
2025-06-15 21:08:08 +09:00
Pretty Printer
c9898416bc Format formerly .hpp files 2025-06-15 21:07:53 +09:00
Pretty Printer
f7f6e019bf Rename .hpp to .h 2025-06-15 21:07:42 +09:00
John Freeman
13ea38bcbb Simplify protobuf generation 2025-06-15 21:07:29 +09:00
Pretty Printer
3bf1229b42 Consolidate external libraries 2025-06-06 16:07:10 +09:00
John Freeman
4f476237c0 Remove unused files 2025-06-06 15:46:57 +09:00
Ed Hennis
0c34036549 fix: Remove redundant STAmount conversion in test (#4996) 2025-06-06 15:46:54 +09:00
Scott Determan
ef7ac442e6 fix: resolve database deadlock: (#4989)
The `rotateWithLock` function holds a lock while it calls a callback
function that's passed in by the caller. This is a problematic design
that needs to be used very carefully. In this case, at least one caller
passed in a callback that eventually relocks the mutex on the same
thread, causing UB (a deadlock was observed). The caller was from
SHAMapStoreImpl, and it called `clearCaches`. This `clearCaches` can
potentially call `fetchNodeObject`, which tried to relock the mutex.

This patch resolves the issue by changing the mutex type to a
`recursive_mutex`. Ideally, the code should be rewritten so it doesn't
hold the mutex during the callback and the mutex should be changed back
to a regular mutex.

Co-authored-by: Ed Hennis <ed@ripple.com>
2025-06-06 15:46:51 +09:00
Michael Legleux
0d6f8afeac fix Conan component reference typo 2025-06-06 15:46:47 +09:00
Bronek Kozicki
fe71d30665 Remove unused lambdas from MultiApiJson_test 2025-06-06 15:46:36 +09:00
Chenna Keshava B S
0f86e7ad54 test: verify the rounding behavior of equal-asset AMM deposits (#4982)
* Specifically, test using tfLPToken flag
2025-06-06 15:46:27 +09:00
John Freeman
4d03f30571 test: Add tests to raise coverage of AMM (#4971)
---------

Co-authored-by: Howard Hinnant <howard.hinnant@gmail.com>
Co-authored-by: Mark Travis <mtravis@ripple.com>
Co-authored-by: Bronek Kozicki <brok@incorrekt.com>
Co-authored-by: Mayukha Vadari <mvadari@gmail.com>
Co-authored-by: Chenna Keshava <ckeshavabs@gmail.com>
2025-06-06 15:46:13 +09:00
John Freeman
415ef13480 test: Add tests to raise coverage of AMM (#4971)
---------

Co-authored-by: Howard Hinnant <howard.hinnant@gmail.com>
Co-authored-by: Mark Travis <mtravis@ripple.com>
Co-authored-by: Bronek Kozicki <brok@incorrekt.com>
Co-authored-by: Mayukha Vadari <mvadari@gmail.com>
Co-authored-by: Chenna Keshava <ckeshavabs@gmail.com>
2025-06-06 15:44:24 +09:00
Bronek Kozicki
632c2982a4 test: Unit test for AMM offer overflow (#4986) 2025-06-06 14:48:00 +09:00
Mayukha Vadari
c97148218f fix amendment to add PreviousTxnID/PreviousTxnLgrSequence (#4751)
This amendment, `fixPreviousTxnID`, adds `PreviousTxnID` and
`PreviousTxnLgrSequence` as fields to all ledger objects that did
not already have them included (`DirectoryNode`, `Amendments`,
`FeeSettings`, `NegativeUNL`, and `AMM`). This makes it much easier
to go through the history of these ledger objects.
2025-06-06 14:47:47 +09:00
Ed Hennis
6bbd3cd9e0 chore: Default validator-keys-tool to master branch: (#4943)
* master is the default branch for that project. There's no point in
  using develop.
2025-06-06 13:09:45 +09:00
Scott Determan
df6da572d0 fixXChainRewardRounding: round reward shares down: (#4933)
When calculating reward shares, the amount should always be rounded
down. If the `fixUniversalNumber` amendment is not active, this works
correctly. If it is not active, then the amount is incorrectly rounded
up. This patch introduces an amendment so it will be rounded down.
2025-06-06 13:08:48 +09:00
Mark Travis
e2f562ae91 Don't reach consensus as quickly if no other proposals seen: (#4763)
This fixes a case where a peer can desync under a certain timing
circumstance--if it reaches a certain point in consensus before it receives
proposals. 

This was noticed under high transaction volumes. Namely, when we arrive at the
point of deciding whether consensus is reached after minimum establish phase
duration but before having received any proposals. This could be caused by
finishing the previous round slightly faster and/or having some delay in
receiving proposals. Existing behavior arrives at consensus immediately after
the minimum establish duration with no proposals. This causes us to desync
because we then close a non-validated ledger. The change in this PR causes us to
wait for a configured threshold before making the decision to arrive at
consensus with no proposals. This allows validators to catch up and for brief
delays in receiving proposals to be absorbed. There should be no drawback since,
with no proposals coming in, we needn't be in a huge rush to jump ahead.
2025-06-06 13:07:46 +09:00
Bronek Kozicki
97fb26f491 Write improved forAllApiVersions used in NetworkOPs (#4833) 2025-06-06 13:07:19 +09:00
Bronek Kozicki
7c89952537 Enforce no duplicate slots from incoming connections: (#4944)
We do not currently enforce that incoming peer connection does not have
remote_endpoint which is already used (either by incoming or outgoing
connection), hence already stored in slots_. If we happen to receive a
connection from such a duplicate remote_endpoint, it will eventually result in a
crash (when disconnecting) or weird behavior (when updating slot state), as a
result of an apparently matching remote_endpoint in slots_ being used by a
different connection.
2025-06-06 13:01:11 +09:00
Mayukha Vadari
d66b67fccb fixEmptyDID: fix amendment to handle empty DID edge case: (#4950)
This amendment fixes an edge case where an empty DID object can be
created. It adds an additional check to ensure that DIDs are
non-empty when created, and returns a `tecEMPTY_DID` error if the DID
would be empty.
2025-06-06 13:00:47 +09:00
Ed Hennis
12fb3abb13 test: Env unit test RPC errors return a unique result: (#4877)
* telENV_RPC_FAILED is a new code, reserved exclusively
  for unit tests when RPC fails. This will
  make those types of errors distinct and easier to test
  for when expected and/or diagnose when not.
* Output RPC command result when result is not expected.
2025-06-06 12:53:00 +09:00
Bronek Kozicki
2a509960c5 Upgrade to xxhash 0.8.2 as a Conan requirement, enable SIMD hashing (#4893)
We are currently using old version 0.6.2 of `xxhash`, as a verbatim copy and paste of its header file `xxhash.h`. Switch to the more recent version 0.8.2. Since this version is in Conan Center (and properly protects its ABI by keeping the state object incomplete), add it as a Conan requirement. Switch to the SIMD instructions (in the new `XXH3` family) supported by the new version.
2025-06-06 12:41:55 +09:00
Michael Legleux
dbeca5a2ee Install more public headers (#4940)
Fixes some mistakes in #4885
2025-06-06 12:30:28 +09:00
Scott Determan
646fb1a860 fix: order book update variable swap: (#4890)
This is likely the result of a typo when the code was simplified.
2025-06-06 12:21:02 +09:00
John Freeman
7f2ddf431b Embed patched recipe for RocksDB 6.29.5 (#4947) 2025-06-06 12:20:45 +09:00
Gregory Tsipenyuk
9d5b013b34 build: add STCurrency.h to xrpl_core to fix clio build (#4939) 2025-06-06 12:10:58 +09:00
Mayukha Vadari
266e2a26fc feat: add user version of feature RPC (#4781)
* uses same formatting as admin RPC
* hides potentially sensitive data
2025-06-06 12:10:42 +09:00
Scott Determan
c8373de952 Fast base58 codec: (#4327)
This algorithm is about an order of magnitude faster than the existing
algorithm (about 10x faster for encoding and about 15x faster for
decoding - including the double hash for the checksum). The algorithms
use gcc's int128 (fast MS version will have to wait, in the meantime MS
falls back to the slow code).
2025-06-06 12:01:35 +09:00
Chenna Keshava B S
fd36796bbd Remove default ctors from SecretKey and PublicKey: (#4607)
* It is now an invariant that all constructed Public Keys are valid,
  non-empty and contain 33 bytes of data.
* Additionally, the memory footprint of the PublicKey class is reduced.
  The size_ data member is declared as static.
* Distinguish and identify the PublisherList retrieved from the local
  config file, versus the ones obtained from other validators.
* Fixes #2942
2025-06-06 12:01:19 +09:00
Gregory Tsipenyuk
4ac704384e fix compile error on gcc 13: (#4932)
The compilation fails due to an issue in the initializer list
of an optional argument, which holds a vector of pairs.
The code compiles correctly on earlier gcc versions, but fails on gcc 13.
2025-06-04 20:51:42 +09:00
Gregory Tsipenyuk
6fbf018e33 Price Oracle (XLS-47d): (#4789) (#4789)
Implement native support for Price Oracles.

 A Price Oracle is used to bring real-world data, such as market prices,
 onto the blockchain, enabling dApps to access and utilize information
 that resides outside the blockchain.

 Add Price Oracle functionality:
 - OracleSet: create or update the Oracle object
 - OracleDelete: delete the Oracle object

 To support this functionality add:
 - New RPC method, `get_aggregate_price`, to calculate aggregate price for a token pair of the specified oracles
 - `ltOracle` object

 The `ltOracle` object maintains:
 - Oracle Owner's account
 - Oracle's metadata
 - Up to ten token pairs with the scaled price
 - The last update time the token pairs were updated

 Add Oracle unit-tests
2025-06-04 20:51:00 +09:00
tequ
89890e40d9 Merge branch 'sync-2.0.1-conan' into sync-2.1.1-conan 2025-06-04 20:10:10 +09:00
tequ
0a1a289bd4 Merge commit '827e6fe617808139556e4eb1513f468941f20912' into sync-2.0.1-conan 2025-06-04 20:10:00 +09:00
Mayukha Vadari
827e6fe617 feat(rpc): add server_definitions method (#4703)
Add a new RPC / WS call for `server_definitions`, which returns an
SDK-compatible `definitions.json` (binary enum definitions) generated by
the server. This enables clients/libraries to dynamically work with new
fields and features, such as ones that may become available on side
chains. Clients query `server_definitions` on a node from the network
they want to work with, and immediately know how to speak that node's
binary "language", even if new features are added to it in the future
(as long as there are no new serialized types that the software doesn't
know how to serialize/deserialize).

Example:

```js
> {"command": "server_definitions"}
< {
    "result": {
        "FIELDS": [
            [
                "Generic",
                {
                    "isSerialized": false,
                    "isSigningField": false,
                    "isVLEncoded": false,
                    "nth": 0,
                    "type": "Unknown"
                }
            ],
            [
                "Invalid",
                {
                    "isSerialized": false,
                    "isSigningField": false,
                    "isVLEncoded": false,
                    "nth": -1,
                    "type": "Unknown"
                }
            ],
            [
                "ObjectEndMarker",
                {
                    "isSerialized": false,
                    "isSigningField": true,
                    "isVLEncoded": false,
                    "nth": 1,
                    "type": "STObject"
                }
            ],
        ...
```

Close #3657

---------

Co-authored-by: Richard Holland <richard.holland@starstone.co.nz>
2025-06-04 20:07:53 +09:00
tequ
99ee7814e1 Merge branch 'sync-2.1.0-conan' into sync-2.1.1-conan 2025-06-04 18:45:47 +09:00
tequ
bdb5582818 Merge branch 'sync-2.0.1-conan' into sync-2.1.0-conan 2025-06-04 18:45:39 +09:00
tequ
5436d1fffd Merge branch 'sync-2.0.0-conan' into sync-2.0.1-conan 2025-06-04 18:45:29 +09:00
tequ
85e8ec2acf Merge branch 'sync-1.12.0-conan' into sync-2.0.0-conan 2025-06-04 18:45:17 +09:00
tequ
58fa7fda82 Merge branch 'dev' into sync-1.12.0-conan 2025-06-04 13:54:52 +09:00
Gregory Tsipenyuk
60e80c0427 fix: improper handling of large synthetic AMM offers:
A large synthetic offer was not handled correctly in the payment engine.
This patch fixes that issue and introduces a new invariant check while
processing synthetic offers.
2025-05-06 15:14:35 +09:00
tequ
dc7b404538 Merge branch 'sync-2.0.1-conan' into sync-2.1.0-conan 2025-05-06 14:58:44 +09:00
tequ
e04d5911de Merge branch 'sync-2.0.0-conan' into sync-2.0.1-conan 2025-05-06 14:58:18 +09:00
tequ
8f4a9f44f6 Merge branch 'sync-1.12.0-conan' into sync-2.0.0-conan 2025-05-06 14:57:52 +09:00
tequ
a4bb9b9629 Merge branch 'tmp-conan' into sync-1.12.0-conan 2025-05-05 12:56:05 +09:00
RichardAH
9b47d06c64 Merge branch 'dev' into tmp-conan 2025-05-03 12:31:28 +10:00
Denis Angell
34b3442e00 Merge branch 'dev' into tmp-conan 2025-04-30 11:52:25 +02:00
Denis Angell
dd5d580b97 [fold] bad merge 2025-04-30 10:39:12 +02:00
Denis Angell
e4478fa883 revert build in docker changes 2025-04-30 10:33:43 +02:00
tequ
2876888a20 Merge branch 'tmp-conan' into sync-1.12.0-conan 2025-04-29 16:34:04 +09:00
RichardAH
857eefc829 Merge branch 'dev' into tmp-conan 2025-04-29 13:42:34 +10:00
Denis Angell
8ac2887961 Merge branch 'dev' into tmp-conan 2025-04-23 15:57:12 +02:00
Denis Angell
b87120ebc6 update conan boost 2025-04-23 15:56:13 +02:00
Ed Hennis
805e550f12 test: guarantee proper lifetime for temporary Rules object: (#4917)
* Commit 01c37fe introduced a change to the STTx unit test where a local
  "defaultRules" object was created with a temporary inline "presets"
  value provided to the ctor. Rules::Impl stores a const ref to the
  presets provided to the ctor.  This particular call provided an inline
  temp variable, which goes out of scope as soon as the object is
  created. On Windows, attempting to use the presets (e.g. via the
  enabled() function) causes an access violation, which crashes the test
  run.
* An audit of the code indicates that all other instances of Rules use
  the Application's config.features list, which will have a sufficient
  lifetime.
2025-04-23 22:21:09 +09:00
Gregory Tsipenyuk
9f5fb80647 fixInnerObjTemplate: set inner object template (#4906)
Add `STObject` constructor to explicitly set the inner object template.
This allows certain AMM transactions to apply in the same ledger:

There is no issue if the trading fee is greater than or equal to 0.01%.
If the trading fee is less than 0.01%, then:
- After AMM create, AMM transactions must wait for one ledger to close
  (3-5 seconds).
- After one ledger is validated, all AMM transactions succeed, as
  appropriate, except for AMMVote.
- The first AMMVote which votes for a 0 trading fee in a ledger will
  succeed. Subsequent AMMVote transactions which vote for a 0 trading
  fee will wait for the next ledger (3-5 seconds). This behavior repeats
  for each ledger.

This has no effect on the ultimate correctness of AMM. This amendment
will allow the transactions described above to succeed as expected, even
if the trading fee is 0 and the transactions are applied within one
ledger (block).
2025-04-23 22:20:50 +09:00
tequ
9d09ee12b5 Remove #ifndef DEBUG guards and exception handling wrappers 2025-04-23 22:12:16 +09:00
Chenna Keshava B S
2e586b3f12 feat: allow port_grpc to be specified in [server] stanza (#4728)
Prior to this commit, `port_grpc` could not be added to the [server]
stanza. Instead of validating gRPC IP/Port/Protocol information in
ServerHandler, validate grpc port info in GRPCServer constructor. This
should not break backwards compatibility.

gRPC-related config info must be in a section (stanza) called
[port_gprc].

* Close #4015 - That was an alternate solution. It was decided that with
  relaxed validation, it is not necessary to rename port_grpc.
* Fix #4557
2025-04-23 16:42:05 +09:00
Michael Legleux
51eee6254c build: add headers needed in Conan package for libxrpl (#4885)
These headers are required in the xrpl Conan package in order for
xbridge witness server (xbwd) to build. This change to libxrpl may help
any dependents of libxrpl. This addition does not change any C++ code.
2025-04-23 16:30:18 +09:00
Shawn Xie
4847bebea4 fixNFTokenReserve: ensure NFT tx fails when reserve is not met (#4767)
Without this amendment, an NFTokenAcceptOffer transaction can succeed
even when the NFToken recipient does not have sufficient reserves for
the new NFTokenPage. This allowed accounts to accept NFT sell offers
without having a sufficient reserve. (However, there was no issue in
brokered mode or when a buy offer is involved.)

Instead, the transaction should fail with `tecINSUFFICIENT_RESERVE` as
appropriate. The `fixNFTokenReserve` amendment adds checks in the
NFTokenAcceptOffer transactor to check if the OwnerCount changed. If it
did, then it checks the new reserve requirement.

Fix #4679
2025-04-23 16:29:41 +09:00
tequ
7168266ccf Merge branch 'sync-2.0.1-conan' into sync-2.1.0-conan 2025-04-23 16:21:58 +09:00
tequ
e38dcd3acd Merge branch 'sync-2.0.0-conan' into sync-2.0.1-conan 2025-04-23 11:33:15 +09:00
tequ
5ba9e871c5 bad merge: RPCCall_test, Transaction_test 2025-04-23 11:32:35 +09:00
Ed Hennis
a95d4ad6ca Fix cahce bug introduced in 2.0.1
Partially chery-picked from f419c18056
2025-04-23 10:41:05 +09:00
John Freeman
53dd3d1892 fix(libxrpl): change library names in Conan recipe (#4831)
Use consistent platform-agnostic library names on all platforms.

Fix an issue that prevents dependents like validator-keys-tool from
linking to libxrpl on Windows.

It is bad practice to change the binary base name depending on the
platform. CMake already manipulates the base name into a final name that
fits the conventions of the platform. Linkers accept base names on the
command line and then look for conventional names on disk.
2025-04-22 15:00:33 +09:00
Bronek Kozicki
70513b8fca test: add unit test for redundant payment (#4860)
If the payee and payer are the same account, then the transaction fails
in preflight with temREDUNDANT.
2025-04-22 14:59:52 +09:00
Bronek Kozicki
ad90852fa5 test: improve code coverage reporting (#4849)
* Speed up the generation of coverage reports by using multiple cores.

* Add codecov step to coverage workflow.
2025-04-22 14:38:45 +09:00
Chenna Keshava B S
469a3fb3b0 docs: update help message about unit test-suite pattern matching (#4846)
Update the "rippled --help" message for the "-u" parameter. This
documents the unit test name pattern matching rule implemented by #4634.

Fix #4800
2025-04-22 14:28:09 +09:00
Elliot Lee
0bc2a3ba0f docs: add Performance type to PR template (#4875) 2025-04-22 14:27:36 +09:00
Bronek Kozicki
fda383b6a4 test: add DeliverMax to more JSONRPC tests (#4826)
Minor change in unit tests to improve testing scope.
2025-04-22 14:27:11 +09:00
John Freeman
387f789a20 fix: change default send_queue_limit to 500 (#4867)
Clients subscribed to `transactions` over WebSocket are being
disconnected because the traffic exceeds the default `send_queue_limit`
of 100.

This commit changes the default configuration, not the default in code.

Fix #4866
2025-04-22 14:27:02 +09:00
Ed Hennis
fbe221c987 Improve lifetime management of ledger objects (SLEs) to prevent runaway memory usage: (#4822)
* Add logging for Application.cpp sweep()
* Improve lifetime management of ledger objects (`SLE`s)
* Only store SLE digest in CachedView; get SLEs from CachedSLEs
* Also force release of last ledger used for path finding if there are
  no path finding requests to process
* Count more ST objects (derive from `CountedObject`)
* Track CachedView stats in CountedObjects
* Rename the CachedView counters
* Fix the scope of the digest lookup lock

Before this patch, if you asked "is it caching?" It was always caching.
2025-04-22 14:25:27 +09:00
Ed Hennis
a124e54f6b WebSocket should only call async_close once (#4848)
Prevent WebSocket connections from trying to close twice.

The issue only occurs in debug builds (assertions are disabled in
release builds, including published packages), and when the WebSocket
connections are unprivileged. The assert (and WRN log) occurs when a
client drives up the resource balance enough to be forcibly disconnected
while there are still messages pending to be sent.

Thanks to @lathanbritz for discovering this issue in #4822.
2025-04-22 14:25:15 +09:00
Hussein Badakhchani
8f30d8dcfc fix typo: 'of' instead of 'on' (#4821)
Co-authored-by: Hussein Badakhchani <hoos@alsoug.com>
2025-04-19 01:10:17 +09:00
Bronek Kozicki
3e0324081b Workarounds for gcc-13 compatibility (#4817)
Workaround for compilation errors with gcc-13 and other compilers
relying on `libstdc++` version 13. This is temporary until actual fix is
available for us to use: https://github.com/boostorg/beast/pull/2682

Some boost.beast files (which we do use) rely on an old gcc-12 behaviour
where `#include <cstdint>` was not needed even though types from this
header were used. This was broken by a change in libstdc++ version 13:
https://gcc.gnu.org/gcc-13/porting_to.html#header-dep-changes

The necessary fix was implemented in boost.beast, however it is not yet
available. Until it is available, we can use this workaround to enable
compilation of `rippled` with gcc-13, clang-16, etc.
2025-04-19 01:10:02 +09:00
Bronek Kozicki
8ea213413a APIv2: show DeliverMax in submit, submit_multisigned (#4827)
Show `DeliverMax` instead of `Amount` in output from `submit`,
`submit_multisigned`, `sign`, and `sign_for`.

Fix #4829
2025-04-19 01:09:24 +09:00
Bronek Kozicki
7b3da1b0f7 APIv2: consistently return ledger_index as integer (#4820)
For api_version 2, always return ledger_index as integer in JSON output.

api_version 1 retains prior behavior.
2025-04-19 01:09:00 +09:00
Bronek Kozicki
07de989780 Fix 2.0 regression in tx method with binary output (#4812)
* Fix binary output from tx method

* Formatting fix

* Minor test improvement

* Minor test improvements
2025-04-19 00:57:27 +09:00
Bronek Kozicki
f589fb3f03 Promote API version 2 to supported (#4803)
* Promote API version 2 to supported

* Switch command line to API version 1

* Fix LedgerRequestRPC test

* Remove obsolete tx_account method

This method is not implemented, the only parts which are removed are related to command-line parsing

* Fix RPCCall test

* Reduce diff size, small test improvements

* Minor fixes

* Support for the mold linker

* [fold] handle case where both mold and gold are installed

* [fold] Use first non-default linker

* Fix TransactionEntry_test

* Fix AccountTx_test

---------

Co-authored-by: seelabs <scott.determan@yahoo.com>
2025-04-19 00:56:50 +09:00
Scott Determan
691a1a5610 Support for the mold linker (#4807) 2025-04-19 00:56:12 +09:00
Bronek Kozicki
36eb3d113a Unify JSON serialization format of transactions (#4775)
* Remove include <ranges>

* Formatting fix

* Output for subscriptions

* Output from sign, submit etc.

* Output from ledger

* Output from account_tx

* Output from transaction_entry

* Output from tx

* Store close_time_iso in API v2 output

* Add small APIv2 unit test for subscribe

* Add unit test for transaction_entry

* Add unit test for tx

* Remove inLedger from API version 2

* Set ledger_hash and ledger_index

* Move isValidated from RPCHelpers to LedgerMaster

* Store closeTime in LedgerFill

* Time formatting fix

* additional tests for Subscribe unit tests

* Improved comments

* Rename mInLedger to mLedgerIndex

* Minor fixes

* Set ledger_hash on closed ledger, even if not validated

* Update API-CHANGELOG.md

* Add ledger_hash, ledger_index to transaction_entry

* Fix validated and close_time_iso in account_tx

* Fix typos

* Improve getJson for Transaction and STTx

* Minor improvements

* Replace class enum JsonOptions with struct

We may consider turning this into a general-purpose template and using it elsewhere

* simplify the extraction of transactionID from Transaction object

* Remove obsolete comments

* Unconditionally set validated in account_tx output

* Minor improvements

* Minor fixes

---------

Co-authored-by: Chenna Keshava <ckeshavabs@gmail.com>
2025-04-19 00:54:42 +09:00
Scott Determan
c6c2983cf2 fix: check for valid public key in attestations (#4798) 2025-04-19 00:08:08 +09:00
pwang200
99225d5aee Fix unit test api_version to enable api_version 2 (#4785)
The command line API still uses `apiMaximumSupportedVersion`.
The unit test RPCs use `apiMinimumSupportedVersion` if unspecified.

Context:
- #4568
- #4552
2025-04-19 00:07:53 +09:00
Gregory Tsipenyuk
46b1b9e63c fixFillOrKill: fix offer crossing with tfFillOrKill (#4694)
Introduce the `fixFillOrKill` amendment.

Fix an edge case occurring when an offer with `tfFillOrKill` set (but
without `tfSell` set) fails to cross an offer with a better rate. If
`tfFillOrKill` is set, then the owner must receive the full TakerPays.
Without this amendment, an offer fails if the entire `TakerGets` is not
spent. With this amendment, when `tfSell` is not set, the entire
`TakerGets` does not have to be spent.

For details about OfferCreate, see: https://xrpl.org/offercreate.html

Fix #4684

---------

Co-authored-by: Scott Schurr <scott@ripple.com>
2025-04-19 00:02:29 +09:00
Bronek Kozicki
961054dc6b fix: remove include <ranges> (#4788)
Remove dependency on `<ranges>` header, since it is not implemented by
all compilers which we want to support.

This code change only affects unit tests.

Resolve https://github.com/XRPLF/rippled/issues/4787
2025-04-18 23:52:27 +09:00
Bronek Kozicki
53f8bf5b0b APIv2: remove tx_history and ledger_header (#4759)
Remove `tx_history` and `ledger_header` methods from API version 2.

Update `RPC::Handler` to allow for methods (or method implementations)
to be API version specific. This partially resolves #4727. We can now
store multiple handlers with the same name, as long as they belong to
different (non-overlapping) API versions. This necessarily impacts the
handler lookup algorithm and its complexity; however, there is no
performance loss on x86_64 architecture, and only minimal performance
loss on arm64 (around 10ns). This design change gives us extra
flexibility evolving the API in the future, including other parts of

In API version 2, `tx_history` and `ledger_header` are no longer
recognised; if they are called, `rippled` will return error
`unknownCmd`

Resolve #3638

Resolve #3539
2025-04-18 23:49:34 +09:00
Mark Travis
413a2076f4 docs: clarify definition of network health (#4729)
Update the documentation to describe network health with more nuance as
well as context about related factors.
2025-04-18 23:45:22 +09:00
tequ
f3970631aa fix temCode: bad merge 2025-04-18 23:44:22 +09:00
Bronek Kozicki
5e232bf785 APIv2(DeliverMax): add alias for Amount in Payment transactions (#4733)
Using the "Amount" field in Payment transactions can cause incorrect
interpretation. There continue to be problems from the use of this
field. "Amount" is rarely the correct field to use; instead,
"delivered_amount" (or "DeliveredAmount") should be used.

Rename the "Amount" field to "DeliverMax", a less misleading name. With
api_version: 2, remove the "Amount" field from Payment transactions.

- Input: "DeliverMax" in `tx_json` is an alias for "Amount"
  - sign
  - submit (in sign-and-submit mode)
  - submit_multisigned
  - sign_for
- Output: Add "DeliverMax" where transactions are provided by the API
  - ledger
  - tx
  - tx_history
  - account_tx
  - transaction_entry
  - subscribe (transactions stream)
- Output: Remove "Amount" from API version 2

Fix #3484

Fix #3902
2025-04-18 23:43:40 +09:00
Mayukha Vadari
ae39ac3cfb DID: Decentralized identifiers (DIDs) (XLS-40): (#4636)
Implement native support for W3C DIDs.

Add a new ledger object: `DID`.

Add two new transactions:
1. `DIDSet`: create or update the `DID` object.
2. `DIDDelete`: delete the `DID` object.

This meets the requirements specified in the DID v1.0 specification
currently recommended by the W3C Credentials Community Group.

The DID format for the XRP Ledger conforms to W3C DID standards.
The objects can be created and owned by any XRPL account holder.
The transactions can be integrated by any service, wallet, or application.
2025-04-18 23:15:02 +09:00
Scott Schurr
31b61b5b94 refactor(peerfinder): use LogicError in PeerFinder::Logic (#4562)
It might be possible for the server code to indirect through certain
`end()` iterators. While a debug build would catch this problem with
`assert()`s, a release build would crash. If there are problems in this
area in the future, it is best to get a definitive indication of the
nature of the error regardless of whether it's a debug or release build.
To accomplish this, these `assert`s are converted into `LogicError`s
that will produce a reasonable error message when they fire.
2025-04-18 22:34:37 +09:00
Ed Hennis
c1c2a9b3a6 fix(PathRequest): remove incorrect assert (#4743)
The assert is saying that the only reason `pathFinder` would be null is
if the request was aborted (connection dropped, etc.). That's what
`continueCallback()` checks. But that is very clearly not true if you
look at `getPathFinder`, which calls `findPaths`, which can return false
for many reasons.

Fix #4744
2025-04-18 22:33:10 +09:00
Ed Hennis
bcfb37bc5c docs(API-CHANGELOG): add XRPFees change (#4741)
* Add a new API Changelog section for release 1.10.
* Mark `jss::fee_ref` as deprecated.
* Fix a copy-paste error in one of the unit tests.
2025-04-18 22:31:27 +09:00
Florent
f4c475ab49 docs(rippled-example.cfg): add P2P link compression (#4753)
P2P link compression is a feature added in 1.6.0 by #3287.

https://xrpl.org/enable-link-compression.html

If the default changes in the future - for example, as currently
proposed by #4387 - the comment will be updated at that time.

Fix #4656
2025-04-18 22:28:34 +09:00
Denis Angell
7ed38c0ae3 fixDisallowIncomingV1: allow issuers to authorize trust lines (#4721)
Context: The `DisallowIncoming` amendment provides an option to block
incoming trust lines from reaching your account. The
asfDisallowIncomingTrustline AccountSet Flag, when enabled, prevents any
incoming trust line from being created. However, it was too restrictive:
it would block an issuer from authorizing a trust line, even if the
trust line already exists. Consider:

1. Issuer sets asfRequireAuth on their account.
2. User sets asfDisallowIncomingTrustline on their account.
3. User submits tx to SetTrust to Issuer.

At this point, without `fixDisallowIncomingV1` active, the issuer would
not be able to authorize the trust line.

The `fixDisallowIncomingV1` amendment, once activated, allows an issuer
to authorize a trust line even after the user sets the
asfDisallowIncomingTrustline flag, as long as the trust line already
exists.
2025-04-18 22:27:41 +09:00
Scott Determan
b06d2bfbcc refactor: reduce boilerplate in applySteps: (#4710)
When a new transactor is added, there are several places in applySteps
that need to be modified. This patch refactors the code so only one
function needs to be modified.
2025-04-18 22:23:21 +09:00
Rome Reginelli
0be2afdb9c refactor: reunify transaction common fields: (#4715)
Make transactions and pseudo-transactions share the same commonFields
again. This regularizes the code in a nice way.

While this technically allows pseudo-transactions to have a
TicketSequence field, pseudo-transactions are only ever constructed by
code paths that don't add such a field, so this is not a transaction
processing change. It may be possible to add a separate check to ensure
TicketSequence (and other fields that don't make sense on
pseudo-transactions) are never added to pseudo-transactions, but that
should not be necessary. (TicketSequence is not the only common field
that can not and does not appear in pseudo-transactions.) Note:
TicketSequence is already documented as a common field.

Related: #4637

Fix #4714
2025-04-18 22:09:59 +09:00
Chenna Keshava B S
3e0af5ba3b docs(BUILD.md): require GCC 11 or higher (#4700)
Update minimum compiler requirement for building the codebase. The
feature "using enum" is required. This feature was introduced in C++20.

Updating the C++ compiler to version 11 or later fixes this error:

```
Building CXX object CMakeFiles/xrpl_core.dir/src/ripple/protocol/impl/STAmount.cpp.o
/build/ripple/binary/src/ripple/protocol/impl/STAmount.cpp: In lambda function:
/build/ripple/binary/src/ripple/protocol/impl/STAmount.cpp:1577:15: error: expected nested-name-specifier before 'enum'
 1577 |         using enum Number::rounding_mode;
      |               ^~~~
```

Fix #4693
2025-04-18 21:59:19 +09:00
Scott Determan
a27a09a6c4 fix(XLS-38): disallow the same bridge on one chain: (#4720)
Modify the `XChainBridge` amendment.

Before this patch, two door accounts on the same chain could could own
the same bridge spec (of course, one would have to be the issuer and one
would have to be the locker). While this is silly, it does not violate
any bridge invariants. However, on further review, if we allow this then
the `claim` transactions would need to change. Since it's hard to see a
use case for two doors to own the same bridge, this patch disallows
it. (The transaction will return tecDUPLICATE).
2025-04-18 21:58:16 +09:00
Scott Schurr
78d9be7e09 fix: stabilize voting threshold for amendment majority mechanism (#4410)
Amendment "flapping" (an amendment repeatedly gaining and losing
majority) usually occurs when an amendment is on the verge of gaining
majority, and a validator not in favor of the amendment goes offline or
loses sync. This fix makes two changes:

1. The number of validators in the UNL determines the threshold required
   for an amendment to gain majority.
2. The AmendmentTable keeps a record of the most recent Amendment vote
   received from each trusted validator (and, with `trustChanged`, stays
   up-to-date when the set of trusted validators changes). If no
   validation arrives from a given validator, then the AmendmentTable
   assumes that the previously-received vote has not changed.

In other words, when missing an `STValidation` from a remote validator,
each server now uses the last vote seen. There is a 24 hour timeout for
recorded validator votes.

These changes do not require an amendment because they do not impact
transaction processing, but only the threshold at which each individual
validator decides to propose an EnableAmendment pseudo-transaction.

Fix #4350
2025-04-18 21:57:58 +09:00
Ed Hennis
f4bb4b6951 fix(build): uint is not defined on Windows platform (#4731)
Fix the Windows build by using `unsigned int` (instead of `uint`).

The error, introduced by #4618, looks something like:
  rpc\impl\RPCHelpers.h(299,5): error C2061: syntax error: identifier
  'uint' (compiling source file app\ledger\Ledger.cpp)
2025-04-18 21:57:17 +09:00
Nik Bougalis
67930a9ceb Eliminate the built-in SNTP support (fixes #4207): (#4628) 2025-04-18 21:52:43 +09:00
John Freeman
e7aebad065 fix: accept all valid currency codes in API (#4566)
A few methods, including `book_offers`, take currency codes as
parameters. The XRPL doesn't care if the letters in those codes are
lowercase or uppercase, as long as they come from an alphabet defined
internally. rippled doesn't care either, when they are submitted in a
hex representation. When they are submitted in an ASCII string
representation, rippled, but not XRPL, is more restrictive, preventing
clients from interacting with some currencies already in the XRPL.

This change gets rippled out of the way and lets clients submit currency
codes in ASCII using the full alphabet.

Fixes #4112
2025-04-18 21:41:03 +09:00
Bronek Kozicki
c2b14ee11e chore: add .build to .gitignore (#4722)
Currently, the `BUILD.md` instructions suggest using `.build` as the
build directory, so this change helps to reduce confusion.

An alternative would be to instruct developers to add `/.build/` to
`.git/info/exclude` or to user-level `.gitignore` (although the latter
is very intrusive). However, it is being added here because it is a good
practice to have a sensible default that's consistent with the build
instructions.
2025-04-18 21:40:34 +09:00
ForwardSlashBack
d3a4111d57 Fix typo in BUILD.md (#4718)
Co-authored-by: Chenna Keshava B S <21219765+ckeshava@users.noreply.github.com>
2025-04-18 21:39:05 +09:00
Peter Chen
77cb6d11fb APIv2(gateway_balances, channel_authorize): update errors (#4618)
gateway_balances
* When `account` does not exist in the ledger, return `actNotFound`
  * (Previously, a normal response was returned)
  * Fix #4290
* When required field(s) are missing, return `invalidParams`
  * (Previously, `invalidHotWallet` was incorrectly returned)
  * Fix #4548

channel_authorize
* When the specified `key_type` is invalid, return `badKeyType`
  * (Previously, `invalidParams` was returned)
  * Fix #4289

Since these are breaking changes, they apply only to API version 2.

Supersedes #4577
2025-04-18 21:38:05 +09:00
John Freeman
1c12f1511b build: use Boost 1.82 and link Boost.Json (#4632)
Add Boost::json to the list of linked Boost libraries.

This seems to be required for macOS.
2025-04-18 21:27:49 +09:00
Chenna Keshava B S
0fb7ef4051 docs(overlay): add URL of blog post and clarify wording (#4635) 2025-04-18 21:13:55 +09:00
Elliot Lee
ee92042c19 docs(RELEASENOTES): update 1.12.0 notes to match dev blog (#4691)
* Reorganize some changelog entries
* Add note about portable binaries
* Dev blog: https://xrpl.org/blog
2025-04-18 21:12:51 +09:00
John Freeman
b9f0c9209f Update secp256k1 to 0.3.2 (#4653)
Copy the new code to `src/secp256k1` without changes:
`src/secp256k1` is identical to bitcoin-core/secp256k1@acf5c55 (v0.3.2).

We could consider changing to a Git submodule, though that would require
changes to the build instructions because we are not using submodules
anywhere else.
2025-04-18 21:11:30 +09:00
Chenna Keshava B S
8497c6fd49 docs: fix comment for LedgerHistory::fixIndex return value (#4574)
`LedgerHistory::fixIndex` returns `false` if a repair was performed.

Fix #4572
2025-04-18 21:04:25 +09:00
Ed Hennis
6f25fcd394 fix: remove unused variable causing clang 14 build errors (#4672)
Removed the unused variable `none` from `Writer.cpp` which was causing
build errors on clang version 14.
2025-04-18 21:04:01 +09:00
Elliot Lee
b295c7691c docs(BUILD): make it easier to find environment.md (#4507)
Make the instructions a bit easier to follow. Users on different
platforms can look for their platform name to find relevant information.
2025-04-18 21:03:43 +09:00
Michael Legleux
af8a130c19 Revert CMake changes (#4707)
This was likely put back when #4292 was rebased.
2025-04-18 21:02:46 +09:00
Scott Determan
49d2c731f7 Change XChainBridge amendment to Supported::yes (#4709) 2025-04-18 21:01:09 +09:00
Scott Determan
d5b6c63a80 Fix Windows build by removing two unused declarations (#4708)
Remove the `verify` and `message` function declarations. The explicit
instantiation requests could not be completed because there were no
implementations for those two member functions. It is helpful that the
Microsoft (MSVC) compiler on Windows appears to be strict when it comes
to template instantiation.

This resolves the warning:

  XChainAttestations.h(450): warning C4661: 'bool
  ripple::XChainAttestationsBase<ripple::XChainClaimAttestation>::verify(void)
  const': no suitable definition provided for explicit template
  instantiation request
2025-04-18 20:59:02 +09:00
Ed Hennis
b3a670b541 Match unit tests on start of test name (#4634)
* For example, without this change, to run the TxQ tests, must specify
  `--unittest=TxQ1,TxQ2` on the command line. With this change, can use
  `--unittest=TxQ`, and both will be run.
* An exact match will prevent any further partial matching.
* This could have some side effects for different tests with a common
  name beginning. For example, NFToken, NFTokenBurn, NFTokenDir. This
  might be useful. If not, the shorter-named test(s) can be renamed. For
  example, NFToken to NFTokens.
* Split the NFToken, NFTokenBurn, and Offer test classes. Potentially speeds
  up parallel tests by a factor of 5.
2025-04-18 20:56:59 +09:00
Howard Hinnant
8159b0f0ff Revert ThreadName due to problems on Windows (#4702)
* Revert "Remove CurrentThreadName.h from RippledCore.cmake (#4697)"

This reverts commit 3b5fcd587313f5ebc762bc21c6a4ec3e6c275e83.

* Revert "Introduce replacement for getting and setting thread name: (#4312)"

This reverts commit 36cb5f90e233f975eb3f80d819b2fbadab0a9387.
2025-04-18 20:34:29 +09:00
Scott Determan
f2dc0b8cc1 XChainBridge: Introduce sidechain support (XLS-38): (#4292)
A bridge connects two blockchains: a locking chain and an issuing
chain (also called a mainchain and a sidechain). Both are independent
ledgers, with their own validators and potentially their own custom
transactions. Importantly, there is a way to move assets from the
locking chain to the issuing chain and a way to return those assets from
the issuing chain back to the locking chain: the bridge. This key
operation is called a cross-chain transfer. A cross-chain transfer is
not a single transaction. It happens on two chains, requires multiple
transactions, and involves an additional server type called a "witness".

A bridge does not exchange assets between two ledgers. Instead, it locks
assets on one ledger (the "locking chain") and represents those assets
with wrapped assets on another chain (the "issuing chain"). A good model
to keep in mind is a box with an infinite supply of wrapped assets.
Putting an asset from the locking chain into the box will release a
wrapped asset onto the issuing chain. Putting a wrapped asset from the
issuing chain back into the box will release one of the existing locking
chain assets back onto the locking chain. There is no other way to get
assets into or out of the box. Note that there is no way for the box to
"run out of" wrapped assets - it has an infinite supply.

Co-authored-by: Gregory Popovitch <greg7mdp@gmail.com>
2025-04-18 20:31:02 +09:00
Peter Chen
ed90432fbe APIv2(account_tx, noripple_check): return error on invalid input (#4620)
For the `account_tx` and `noripple_check` methods, perform input
validation for optional parameters such as "binary", "forward",
"strict", "transactions". Previously, when these parameters had invalid
values (e.g. not a bool), no error would be returned. Now, it returns an
`invalidParams` error.

* This updates the behavior to match Clio
  (https://github.com/XRPLF/clio).
* Since this is potentially a breaking change, it only applies to
  requests specifying api_version: 2.
* Fix #4543.
2025-04-18 18:02:18 +09:00
Howard Hinnant
057180860d Remove CurrentThreadName.h from RippledCore.cmake (#4697)
(File was already removed from the source)
2025-04-18 17:59:41 +09:00
Mayukha Vadari
89212fd03e refactor: simplify TxFormats common fields logic (#4637)
Minor refactor to `TxFormats.cpp`:
- Rename `commonFields` to `pseudoCommonFields` (since it is the common fields
  that all pseudo-transactions need)
- Add a new static variable, `commonFields`, which represents all the common
  fields that non-pseudo transactions need (essentially everything that
  `pseudoCommonFields` contains, plus `sfTicketSequence`)

This makes it harder to accidentally leave out `sfTicketSequence` in a new
transaction.
2025-04-18 17:58:38 +09:00
Peter Chen
ea0f8760f0 APIv2(ledger_entry): return invalidParams for bad parameters (#4630)
- Verify "check", used to retrieve a Check object, is a string.
- Verify "nft_page", used to retrieve an NFT Page, is a string.
- Verify "index", used to retrieve any type of ledger object by its
  unique ID, is a string.
- Verify "directory", used to retrieve a DirectoryNode, is a string or
  an object.

This change only impacts api_version 2 since it is a breaking change.

https://xrpl.org/ledger_entry.html

Fix #4550
2025-04-18 17:48:34 +09:00
Mark Pevec
2461ecbba9 docs(rippled-example.cfg): clarify ssl_cert vs ssl_chain (#4667)
Clarify usage of ssl_cert vs ssl_chain
2025-04-18 16:23:52 +09:00
Howard Hinnant
429a3599b9 Introduce replacement for getting and setting thread name: (#4312)
* In namespace ripple, introduces get_name function that takes a
  std:🧵:native_handle_type and returns a std::string.
* In namespace ripple, introduces get_name function that takes a
  std::thread or std::jthread and returns a std::string.
* In namespace ripple::this_thread, introduces get_name function
  that takes no parameters and returns the name of the current
  thread as a std::string.
* In namespace ripple::this_thread, introduces set_name function
  that takes a std::string_view and sets the name of the current
  thread.
* Intended to replace the beast utilities setCurrentThreadName
  and getCurrentThreadName.
2025-04-18 16:23:09 +09:00
Gregory Tsipenyuk
8fc04d5c1f amm_info: fetch by amm account id; add AMM object entry (#4682)
- Update amm_info to fetch AMM by amm account id.
  - This is an additional way to retrieve an AMM object.
  - Alternatively, AMM can still be fetched by the asset pair as well.
- Add owner directory entry for AMM object.

Context:

- Add back the AMM object directory entry, which was deleted by #4626.
  - This fixes `account_objects` for `amm` type.
2025-04-17 20:12:25 +09:00
Rome Reginelli
f645e42e6e AMMBid: use tecINTERNAL for 'impossible' errors (#4674)
Modify two error cases in AMMBid transactor to return `tecINTERNAL` to
more clearly indicate that these errors should not be possible unless
operating in unforeseen circumstances. It likely indicates a bug.

The log level has been updated to `fatal()` since it indicates a
(potentially network-wide) unexpected condition when either of these
errors occurs.

Details:

The two specific transaction error cases changed are:

- `tecAMM_BALANCE` - In this case, this error (total LP Tokens
  outstanding is lower than the amount to be burned for the bid) is a
  subset of the case where the user doesn't have enough LP Tokens to pay
  for the bid. When this case is reached, the bidder's LP Tokens balance
  has already been checked first. The user's LP Tokens should always be
  a subset of total LP Tokens issued, so this should be impossible.
- `tecINSUFFICIENT_PAYMENT` - In this case, the amount to be refunded as
  a result of the bid is greater than the price paid for the auction
  slot. This should never occur unless something is wrong with the math
  for calculating the refund amount.

Both error cases in question are "defense in depth" measures meant to
protect against making things worse if the code has already reached a
state that is supposed to be impossible, likely due to a bug elsewhere.

Such "shouldn't ever occur" checks should use an error code that
categorically indicates a larger problem. This is similar to how
`tecINVARIANT_FAILED` is a warning sign that something went wrong and
likely could've been worse, but since there isn't an Invariant Check
applying here, `tecINTERNAL` is the appropriate error code.

This is "debatably" a transaction processing change since it could
hypothetically change how transactions are processed if there's a bug we
don't know about.
2025-04-17 20:07:55 +09:00
Ikko Eltociear Ashimine
666f5eabd0 refactor: fix typo in FeeUnits.h (#4644)
covert -> convert
2025-04-17 18:38:04 +09:00
Arihant Kothari
a1ae22a0d6 test: add forAllApiVersions helper function (#4611)
Introduce a new variadic template helper function, `forAllApiVersions`,
that accepts callables to execute a set of functions over a range of
versions - from RPC::apiMinimumSupportedVersion to RPC::apiBetaVersion.
This avoids the duplication of code.

Context: #4552
2025-04-17 18:36:29 +09:00
Mayukha Vadari
da8973b86f add view updates for account SLEs (#4629)
Signed-off-by: Manoj Doshi <mdoshi@ripple.com>
2025-04-17 18:33:18 +09:00
John Freeman
29592347c0 Fix the package recipe for consumers of libxrpl (#4631)
- "Rename" the type `LedgerInfo` to `LedgerHeader` (but leave an alias
  for `LedgerInfo` to not yet disturb existing uses). Put it in its own
  public header, named after itself, so that it is more easily found.
- Move the type `Fees` and NFT serialization functions into public
  (installed) headers.
- Compile the XRPL and gRPC protocol buffers directly into `libxrpl` and
  install their headers. Fix the Conan recipe to correctly export these
  types.

Addresses change (2) in
https://github.com/XRPLF/XRPL-Standards/discussions/121.

For context: This work supports Clio's dependence on libxrpl. Clio is
just an example consumer. These changes should benefit all current and
future consumers.

---------

Co-authored-by: cyan317 <120398799+cindyyan317@users.noreply.github.com>
Signed-off-by: Manoj Doshi <mdoshi@ripple.com>
2025-04-17 18:32:56 +09:00
John Freeman
8684cb8bf9 Fix package definition for Conan (#4485)
Fix the libxrpl library target for consumers using Conan.

* Fix installation issues and update includes.
* Update requirements in the Conan package info.
  * libxrpl requires openssl::crypto.

(Conan is a software package manager for C++.)
2025-04-17 16:31:04 +09:00
tequ
4077db3389 Merge branch 'tmp-conan' into sync-1.12.0-conan 2025-04-17 16:27:00 +09:00
Denis Angell
c30b73db62 Merge branch 'dev' into tmp-conan 2025-04-17 08:41:34 +02:00
Denis Angell
8dc718a560 fix conan 2025-04-17 08:40:27 +02:00
Denis Angell
658eb73e00 build: add interface library libxrpl: (#4449)
Make it easy for projects to depend on libxrpl by adding an `ALIAS`
target named `xrpl::libxrpl` for projects to link.

The name was chosen because:

* The current library target is named `xrpl_core`. There is no other
  "non-core" library target against which we need to distinguish the
  "core" library. We only export one library target, and it should just
  be named after the project to keep things simple and predictable.
* Underscores in target or library names are generally discouraged.
* Every target exported in CMake should be prefixed with the project
  name.

By adding an `ALIAS` target, existing consumers who use the `xrpl_core`
target will not be affected.

* In the future, there can be a migration plan to make `xrpl_core` the
  `ALIAS` target (and `libxrpl` the "real" target, which will affect the
  filename of the compiled binary), and eventually remove it entirely.

Also:

* Fix the Conan recipe so that consumers using Conan import a target
  named `xrpl::libxrpl`. This way, every consumer can use the same
  instructions.
* Document the two easiest methods to depend on libxrpl. Both have been
  tested.
* See #4443.
2025-04-16 11:29:35 +02:00
RichardAH
df2e2ecd7c Merge branch 'dev' into tmp-conan 2025-04-16 10:29:29 +10:00
Alphonse Noni Mousse
542fbdcb61 refactor: improve checking of path lengths (#4519)
Improve the checking of the path lengths during Payments. Previously,
the code that did the check of the payment path lengths was sometimes
executed, but without any effect. This changes it to only check when it
matters, and to not make unnecessary copies of the path vectors.

Signed-off-by: Manoj Doshi <mdoshi@ripple.com>
2025-04-15 21:06:11 +09:00
Alphonse N. Mousse
e40a97100b refactor: use C++20 function std::popcount (#4389)
- Replace custom popcnt16 implementation with std::popcount from C++20
- Maintain compatibility with older compilers and MacOS by providing a
  conditional compilation fallback to __builtin_popcount and a lookup
  table method
- Move and inline related functions within SHAMapInnerNode for
  performance and readability

Signed-off-by: Manoj Doshi <mdoshi@ripple.com>
2025-04-15 21:05:44 +09:00
Gregory Tsipenyuk
913866afb2 fix(AMM): prevent orphaned objects, inconsistent ledger state: (#4626)
When an AMM account is deleted, the owner directory entries must be
deleted in order to ensure consistent ledger state.

* When deleting AMM account:
  * Clean up AMM owner dir, linking AMM account and AMM object
  * Delete trust lines to AMM
* Disallow `CheckCreate` to AMM accounts
  * AMM cannot cash a check
* Constrain entries in AuthAccounts array to be accounts
  * AuthAccounts is an array of objects for the AMMBid transaction
* SetTrust (TrustSet): Allow on AMM only for LP tokens
  * If the destination is an AMM account and the trust line doesn't
    exist, then:
    * If the asset is not the AMM LP token, then fail the tx with
      `tecNO_PERMISSION`
    * If the AMM is in empty state, then fail the tx with `tecAMM_EMPTY`
      * This disallows trustlines to AMM in empty state
* Add AMMID to AMM root account
  * Remove lsfAMM flag and use sfAMMID instead
* Remove owner dir entry for ltAMM
* Add `AMMDelete` transaction type to handle amortized deletion
  * Limit number of trust lines to delete on final withdraw + AMMDelete
  * Put AMM in empty state when LPTokens is 0 upon final withdraw
  * Add `tfTwoAssetIfEmpty` deposit option in AMM empty state
  * Fail all AMM transactions in AMM empty state except special deposit
  * Add `tecINCOMPLETE` to indicate that not all AMM trust lines are
    deleted (i.e. partial deletion)
    * This is handled in Transactor similar to deleted offers
  * Fail AMMDelete with `tecINTERNAL` if AMM root account is nullptr
  * Don't validate for invalid asset pair in AMMDelete
* AMMWithdraw deletes AMM trust lines and AMM account/object only if the
  number of trust lines is less than max
  * Current `maxDeletableAMMTrustLines` = 512
  * Check no directory left after AMM trust lines are deleted
  * Enable partial trustline deletion in AMMWithdraw
* Add `tecAMM_NOT_EMPTY` to fail any transaction that expects an AMM in
  empty state
* Clawback considerations
  * Disallow clawback out of AMM account
  * Disallow AMM create if issuer can claw back

This patch applies to the AMM implementation in #4294.

Acknowledgements:
Richard Holland and Nik Bougalis for responsibly disclosing this issue.

Bug Bounties and Responsible Disclosures:
We welcome reviews of the project code and urge researchers to
responsibly disclose any issues they may find.

To report a bug, please send a detailed report to:

    bugs@xrpl.org

Signed-off-by: Manoj Doshi <mdoshi@ripple.com>
2025-04-15 21:04:47 +09:00
RichardAH
13acc00c95 feat: support Concise Transaction Identifier (CTID) (XLS-37) (#4418)
* add CTIM to tx rpc

---------

Co-authored-by: Rome Reginelli <mduo13@gmail.com>
Co-authored-by: Elliot Lee <github.public@intelliot.com>
Co-authored-by: Denis Angell <dangell@transia.co>
2025-04-15 20:43:29 +09:00
Shawn Xie
85a636d494 Rename allowClawback flag to allowTrustLineClawback (#4617)
Reason for this change is here XRPLF/XRPL-Standards#119

We would want to be explicit that this flag is exclusively for trustline. For new token types(eg. CFT), they will not utilize this flag for clawback, instead, they will turn clawback on/off on the token-level, which is more versatile.
2025-04-15 20:28:06 +09:00
Gregory Tsipenyuk
eefc6eab53 Introduce AMM support (XLS-30d): (#4294)
Add AMM functionality:
- InstanceCreate
- Deposit
- Withdraw
- Governance
- Auctioning
- payment engine integration

To support this functionality, add:
- New RPC method, `amm_info`, to fetch pool and LPT balances
- AMM Root Account
- trust line for each IOU AMM token
- trust line to track Liquidity Provider Tokens (LPT)
- `ltAMM` object

The `ltAMM` object tracks:
- fee votes
- auction slot bids
- AMM tokens pair
- total outstanding tokens balance
- `AMMID` to AMM `RootAccountID` mapping

Add new classes to facilitate AMM integration into the payment engine.
`BookStep` uses these classes to infer if AMM liquidity can be consumed.

The AMM formula implementation uses the new Number class added in #4192.
IOUAmount and STAmount use Number arithmetic.

Add AMM unit tests for all features.

AMM requires the following amendments:
- featureAMM
- fixUniversalNumber
- featureFlowCross

Notes:
- Current trading fee threshold is 1%
- AMM currency is generated by: 0x03 + 152 bits of sha256{cur1, cur2}
- Current max AMM Offers is 30

---------

Co-authored-by: Howard Hinnant <howard.hinnant@gmail.com>
2025-04-15 19:56:16 +09:00
Elliot Lee
1d2ad17dc9 docs(CONTRIBUTING): push beta releases to release (#4589)
Sections that were rewrapped were wrapped to 72 characters, the same as
the recommendation for commit messages.
2025-04-15 17:17:32 +09:00
Arihant Kothari
97fb65c6d3 APIv2(ledger_entry): return "invalidParams" when fields missing (#4552)
Improve error handling for ledger_entry by returning an "invalidParams"
error when one or more request fields are specified incorrectly, or one
or more required fields are missing.

For example, if none of of the following fields is provided, then the
API should return an invalidParams error:
* index, account_root, directory, offer, ripple_state, check, escrow,
  payment_channel, deposit_preauth, ticket

Prior to this commit, the API returned an "unknownOption" error instead.
Since the error was actually due to invalid parameters, rather than
unknown options, this error was misleading.

Since this is an API breaking change, the "invalidParams" error is only
returned for requests using api_version: 2 and above. To maintain
backward compatibility, the "unknownOption" error is still returned for
api_version: 1.

Related: #4573

Fix #4303
2025-04-15 17:15:43 +09:00
Chenna Keshava B S
4bb7195db4 refactor: change the return type of mulDiv to std::optional (#4243)
- Previously, mulDiv had `std::pair<bool, uint64_t>` as the output type.
  - This is an error-prone interface as it is easy to ignore when
    overflow occurs.
- Using a return type of `std::optional` should decrease the likelihood
  of ignoring overflow.
  - It also allows for the use of optional::value_or() as a way to
    explicitly recover from overflow.
- Include limits.h header file preprocessing directive in order to
  satisfy gcc's numeric_limits incomplete_type requirement.

Fix #3495

---------

Co-authored-by: John Freeman <jfreeman08@gmail.com>
2025-04-15 17:10:55 +09:00
Shawn Xie
06b8d11317 fix: add allowClawback flag for account_info (#4590)
* Update the `account_info` API so that the `allowClawback` flag is
  included in the response.
  * The proposed `Clawback` amendement added an `allowClawback` flag in
    the `AccountRoot` object.
  * In the API response, under `account_flags`, there is now an
    `allowClawback` field with a boolean (`true` or `false`) value.
  * For reference, the XLS-39 Clawback implementation can be found in
    #4553

Fix #4588
2025-04-15 16:51:02 +09:00
Peter Chen
40818bb350 APIv2(account_info): handle invalid "signer_lists" value (#4585)
When requesting `account_info` with an invalid `signer_lists` value, the
API should return an "invalidParams" error.

`signer_lists` should have a value of type boolean. If it is not a
boolean, then it is invalid input. The response now indicates that.

* This is an API breaking change, so the change is only reflected for
  requests containing `"api_version": 2`
* Fix #4539
2025-04-15 16:50:45 +09:00
Chenna Keshava B S
a07d6f939b fix: Update Handler::Condition enum values #3417 (#4239)
- Use powers of two to clearly indicate the bitmask
- Replace bitmask with explicit if-conditions to better indicate predicates

Change enum values to be powers of two (fix #3417) #4239

Implement the simplified condition evaluation
removes the complex bitwise and(&) operator
Implement the second proposed solution in Nik Bougalis's comment - Software does not distinguish between different Conditions (Version: 1.5) #3417 (comment)
I have tested this code change by performing RPC calls with the commands server_info, server_state, peers and validation_info. These commands worked as expected.
2025-04-15 16:50:21 +09:00
Peter Chen
546bc31038 APIv2: add error messages for account_tx (#4571)
Certain inputs for the AccountTx method should return an error. In other
words, an invalid request from a user or client now results in an error
message.

Since this can change the response from the API, it is an API breaking
change. This commit maintains backward compatibility by keeping the
existing behavior for existing requests. When clients specify
"api_version": 2, they will be able to get the updated error messages.

Update unit tests to check the error based on the API version.

* Fix #4288
* Fix #4545
2025-04-15 16:49:59 +09:00
Ed Hennis
a8698bdf7f Fix build references to deleted ServerHandlerImp: (#4592)
* Commits 0b812cd (#4427) and 11e914f (#4516) conflict. The first added
  references to `ServerHandlerImp` in files outside of that class's
  organizational unit (which is technically incorrect). The second
  removed `ServerHandlerImp`, but was not up to date with develop. This
  results in the build failing.
* Fixes the build by changing references to `ServerHandlerImp` to
  the more correct `ServerHandler`.
2025-04-15 16:42:48 +09:00
Scott Schurr
b9b70e3e4d refactor: rename ServerHandlerImp to ServerHandler (#4516)
Rename `ServerHandlerImp` to `ServerHandler`. There was no other
ServerHandler definition despite the existence of a header suggesting
that there was.

This resolves a piece of historical confusion in the code, which was
identified during a code review.

The changes in the diff may look more extensive than they actually are.
The contents of `impl/ServerHandlerImp.h` were merged into
`ServerHandler.h`, making the latter file appear to have undergone
significant modifications. However, this is a non-breaking refactor that
only restructures code.
2025-04-15 16:34:46 +09:00
Chenna Keshava B S
4fa3890a02 fix: remove deprecated fields in ledger method (#4244)
Remove deprecated fields from the ledger command:
* accepted
* hash (use ledger_hash instead)
* seqNum (use ledger_index instead)
* totalCoins (use total_coins instead)

Update SHAMapStore unit tests to use `jss:ledger_hash` instead of the
deprecated `hash` field.

Fix #3214
2025-04-15 16:34:31 +09:00
Denis Angell
0e4470e662 refactor: replace hand-rolled lexicalCast (#4473)
Replace hand-rolled code with std::from_chars for better
maintainability.

The C++ std::from_chars function is intended to be as fast as possible,
so it is unlikely to be slower than the code it replaces. This change is
a net gain because it reduces the amount of hand-rolled code.
2025-04-15 16:33:55 +09:00
Shawn Xie
3448e17de8 XLS-39 Clawback: (#4553)
Introduces:
* AccountRoot flag: lsfAllowClawback
* New Clawback transaction
* More info on clawback spec: https://github.com/XRPLF/XRPL-Standards/tree/master/XLS-39d-clawback
2025-04-15 16:33:21 +09:00
Howard Hinnant
5605e9215a refactor: remove TypedField's move constructor (#4567)
Apply a minor cleanup in `TypedField`:
* Remove a non-working and unused move constructor.
* Constrain the remaining constructor to not be overly generic enough as
  to be used as a copy or move constructor.
2025-04-15 16:16:47 +09:00
drlongle
a024b52261 Add RPC/WS ports to server_info (#4427)
Enhance the /crawl endpoint by publishing WebSocket/RPC ports in the
server_info response. The function processing requests to the /crawl
endpoint actually calls server_info internally, so this change enables a
server to advertise its WebSocket/RPC port(s) to peers via the /crawl
endpoint. `grpc` and `peer` ports are included as well.

The new `ports` array contains objects, each containing a `port` for the
listening port (number string), and an array `protocol` listing the
supported protocol(s).

This allows crawlers to build a richer topology without needing to
port-scan nodes. For non-admin users (including peers), the info about
*admin* ports is excluded.

Also increase test coverage for RPC ServerInfo.

Fix #2837.
2025-04-15 16:16:02 +09:00
Scott Schurr
30f4654469 fixReducedOffersV1: prevent offers from blocking order books: (#4512)
Curtail the occurrence of order books that are blocked by reduced offers
with the implementation of the fixReducedOffersV1 amendment.

This commit identifies three ways in which offers can be reduced:

1. A new offer can be partially crossed by existing offers, so the new
   offer is reduced when placed in the ledger.

2. An in-ledger offer can be partially crossed by a new offer in a
   transaction. So the in-ledger offer is reduced by the new offer.

3. An in-ledger offer may be under-funded. In this case the in-ledger
   offer is scaled down to match the available funds.

Reduced offers can block order books if the effective quality of the
reduced offer is worse than the quality of the original offer (from the
perspective of the taker). It turns out that, for small values, the
quality of the reduced offer can be significantly affected by the
rounding mode used during scaling computations.

This commit adjusts some rounding modes so that the quality of a reduced
offer is always at least as good (from the taker's perspective) as the
original offer.

The amendment is titled fixReducedOffersV1 because additional ways of
producing reduced offers may come to light. Therefore, there may be a
future need for a V2 amendment.
2025-04-15 16:05:31 +09:00
Ed Hennis
c7255d8d8b Enable the Beta RPC API (v2) for all unit tests: (#4573)
* Enable api_version 2, which is currently in beta. It is expected to be
  marked stable by the next stable release.
* This does not change any defaults.
* The only existing tests changed were one that set the same flag, which
  was now redundant, and a couple that tested versioning explicitly.
2025-04-15 15:41:34 +09:00
tequ
5833b022d8 Merge branch 'tmp-conan' into sync-1.12.0-conan 2025-04-15 15:41:04 +09:00
Denis Angell
26cb84fa6c remove more kInfo 2025-04-11 11:22:00 +02:00
Denis Angell
938feb3c23 Update release-builder.sh 2025-04-11 10:50:52 +02:00
Denis Angell
7d6b2e76d2 remove info log from catalogue tests 2025-04-11 10:45:46 +02:00
Denis Angell
ab38e48fff Merge branch 'dev' into tmp-conan 2025-04-10 11:35:45 +02:00
Denis Angell
9b7103ea4d rename workflow 2025-04-10 11:33:08 +02:00
Denis Angell
0bf50c5dc7 eof 2025-03-30 16:48:43 +02:00
Denis Angell
8b0592ce06 Update xahau-sh-build-in-docker.yml 2025-03-29 04:22:35 +01:00
Denis Angell
c27e2a9c05 Update xahau-sh-build-in-docker.yml 2025-03-29 03:26:09 +01:00
Denis Angell
f8d22bbc24 add caching 2025-03-29 03:08:58 +01:00
Denis Angell
95ab35a185 Update release-builder.sh 2025-03-29 02:43:43 +01:00
Denis Angell
313ad7134b test workflow 2025-03-29 02:40:38 +01:00
Denis Angell
859dcbcabf remove EXIT_IF_CONTAINER_RUNNING 2025-03-29 02:26:01 +01:00
Denis Angell
a8e9448b1a revert docker dependency cache 2025-03-29 02:15:13 +01:00
Denis Angell
cab4d63a07 revert some cmake builds 2025-03-28 20:44:09 +01:00
Denis Angell
9a90a7ffdc optional release builder 2025-03-28 20:42:18 +01:00
Denis Angell
ddbbe91e5a attempt fix builder 2025-03-28 19:32:37 +01:00
Denis Angell
2d9ac91cd1 fix build dir 2025-03-28 14:35:41 +01:00
Denis Angell
ed243745f8 fix checkpatterns 2025-03-28 11:49:45 +01:00
Denis Angell
5eac2d3e44 change on fields 2025-03-28 11:44:22 +01:00
Denis Angell
245cf4a676 refactor checkpatterns remove comments 2025-03-28 10:44:18 +01:00
Denis Angell
864734f607 Update xahau-sh-build-in-docker.yml 2025-03-28 10:35:21 +01:00
Niq Dudfield
689740d818 Use GA runners with conan/ccache caches (#482) 2025-03-28 10:29:55 +01:00
Denis Angell
48919f028c [GA] Remove build caching 2025-03-25 11:40:18 +01:00
Denis Angell
9c26f1f213 [GA] revert 2025-03-25 10:45:28 +01:00
Denis Angell
69b7728b2d [GA] tmp disable build cache 2025-03-25 10:37:17 +01:00
Denis Angell
9178aa9d8a [GA] remove unused workflows 2025-03-25 10:17:33 +01:00
Denis Angell
17fdd09640 [GA] update conan profile 2025-03-25 10:13:05 +01:00
Denis Angell
40dc69d642 [GA] conan mac os build 2025-03-25 10:06:17 +01:00
Denis Angell
58e3840345 remove cxxflags for macos action 2025-03-24 12:02:01 +01:00
Denis Angell
b73dd584e7 Merge branch 'dev' into tmp-conan 2025-03-21 15:15:03 +01:00
Denis Angell
0726789d77 update conan to 1.86 2025-03-21 15:09:59 +01:00
Denis Angell
5ef3795cea [fix] release build issue 2025-03-06 08:47:01 +01:00
Denis Angell
da5f1d189b Merge branch 'dev' into tmp-conan 2025-03-05 08:22:27 +01:00
Denis Angell
6fdf788b13 Merge branch 'dev' into tmp-conan 2025-02-28 10:22:52 +01:00
Denis Angell
802ea6c568 [temp] remove negative rate test 2025-02-11 14:32:29 +01:00
Denis Angell
ecc779346e Merge branch 'dev' into tmp-conan 2025-02-11 12:11:27 +01:00
Denis Angell
680c6095d4 Update macos.yml 2025-02-11 12:06:20 +01:00
Denis Angell
6858861660 fix failing test 2025-02-07 11:01:20 +01:00
Denis Angell
9faef17407 fix bad commit 2025-02-06 11:13:29 +01:00
Denis Angell
ee68cc2cd2 clang-format 2025-02-05 11:31:06 +01:00
Denis Angell
7c360bad33 add optional include 2025-02-05 11:31:00 +01:00
Denis Angell
26a66bc2ef fix misc warnings 2025-02-04 12:24:34 +01:00
Denis Angell
4b93e1657f fix macos github action 2025-02-04 12:18:32 +01:00
Denis Angell
cd45285cab include optional 2025-02-04 12:18:25 +01:00
Denis Angell
b7acfb9803 fix misc warnings 2025-02-04 12:18:09 +01:00
Denis Angell
97a10d6556 [fold] fix test 2025-02-04 12:08:12 +01:00
Denis Angell
fe43029272 [fold] clang-format 2025-02-02 23:13:25 +01:00
Denis Angell
bf33b6f637 [fold] update conan 2025-02-02 22:25:33 +01:00
Denis Angell
c27518b846 [fold] use rippleci docker image 2025-02-02 22:23:26 +01:00
Denis Angell
7d8f5de93d [fold] remove conan patch 2025-02-02 22:23:15 +01:00
Denis Angell
858ea1bf25 [fold] add external snappy 2025-02-02 21:46:08 +01:00
Denis Angell
7162fe0497 [fold] upgrade artifact@v4 2025-02-02 21:09:52 +01:00
Denis Angell
56c0e0dd5f [fold] fix bad assert 2025-02-02 21:07:09 +01:00
Denis Angell
20ca066454 [fold] update workflow 2025-02-02 21:06:55 +01:00
Denis Angell
171610d1a9 [release] add release build 2025-02-01 16:54:35 +01:00
Denis Angell
2f6cf0ab4b [patch] remit transfer rate 2025-02-01 13:11:56 +01:00
Denis Angell
71884ad48a sync 2025-02-01 12:11:01 +01:00
Denis Angell
799a056313 conan 2025-02-01 12:00:35 +01:00
1840 changed files with 81559 additions and 138785 deletions

View File

@@ -45,11 +45,9 @@ DisableFormat: false
ExperimentalAutoDetectBinPacking: false
ForEachMacros: [ Q_FOREACH, BOOST_FOREACH ]
IncludeCategories:
- Regex: '^<(test)/'
- Regex: '^<(BeastConfig)'
Priority: 0
- Regex: '^<(xrpld)/'
Priority: 1
- Regex: '^<(xrpl)/'
- Regex: '^<(ripple)/'
Priority: 2
- Regex: '^<(boost)/'
Priority: 3
@@ -58,7 +56,6 @@ IncludeCategories:
IncludeIsMainRegex: '$'
IndentCaseLabels: true
IndentFunctionDeclarationAfterType: false
IndentRequiresClause: true
IndentWidth: 4
IndentWrappedFunctionNames: false
KeepEmptyLinesAtTheStartOfBlocks: false
@@ -74,7 +71,6 @@ PenaltyExcessCharacter: 1000000
PenaltyReturnTypeOnItsOwnLine: 200
PointerAlignment: Left
ReflowComments: true
RequiresClausePosition: OwnLine
SortIncludes: true
SpaceAfterCStyleCast: false
SpaceBeforeAssignmentOperators: true

View File

@@ -1,37 +0,0 @@
codecov:
require_ci_to_pass: true
comment:
behavior: default
layout: reach,diff,flags,tree,reach
show_carryforward_flags: false
coverage:
range: "60..80"
precision: 1
round: nearest
status:
project:
default:
target: 60%
threshold: 2%
patch:
default:
target: auto
threshold: 2%
changes: false
github_checks:
annotations: true
parsers:
cobertura:
partials_as_hits: true
handle_missing_conditions : true
slack_app: false
ignore:
- "src/test/"
- "include/xrpl/beast/test/"
- "include/xrpl/beast/unit_test/"

View File

@@ -1,25 +1,8 @@
# This feature requires Git >= 2.24
# To use it by default in git blame:
# git config blame.ignoreRevsFile .git-blame-ignore-revs
# Format first-party source according to .clang-format
50760c693510894ca368e90369b0cc2dabfd07f3
# Reintroduce Clang-Format & Levelization
da1d20d6d5d862716125d60899b80fab5302954a
# Consolidate external libraries
da1d20d6d5d862716125d60899b80fab5302954a
# Rename .hpp to .h
0345a2645d0f5ad900f4fbbcaff96040d3a887fc
# Format formerly .hpp files
5a227dc719016e10045e17c9396ad401118044f1
# Rewrite includes
e61880699997398f5a746e6c4034edc7632661f5
# Move CMake directory (#4997)
e47b1c1b3b97c3f6d11858ee02f463596e29e7f0
# Rearrange sources (#4997)
bfafa2bb39e562901736d656806bd700c3699a2f
# Rewrite includes (#4997)
e61880699997398f5a746e6c4034edc7632661f5
# Recompute loops (#4997)
d25b5dcd568bb96c18e347d55fac10fe901a1bfb
# Reformat code with clang-format-18
02749feea88ce61c1f7eeb2d61a57d8ecf07ab11
e2384885f5f630c8f0ffe4bf21a169b433a16858
241b9ddde9e11beb7480600fd5ed90e1ef109b21
760f16f56835663d9286bd29294d074de26a7ba6
0eebe6a5f4246fced516d52b83ec4e7f47373edd

View File

@@ -1,12 +0,0 @@
#!/bin/bash
# Pre-commit hook that runs the suspicious patterns check on staged files
# Get the repository's root directory
repo_root=$(git rev-parse --show-toplevel)
# Run the suspicious patterns script in pre-commit mode
"$repo_root/suspicious_patterns.sh" --pre-commit
# Exit with the same code as the script
exit $?

View File

@@ -1,4 +0,0 @@
#!/bin/bash
echo "Configuring git to use .githooks directory..."
git config core.hooksPath .githooks

View File

@@ -0,0 +1,31 @@
name: 'Configure ccache'
description: 'Sets up ccache with consistent configuration'
inputs:
max_size:
description: 'Maximum cache size'
required: false
default: '2G'
hash_dir:
description: 'Whether to include directory paths in hash'
required: false
default: 'true'
compiler_check:
description: 'How to check compiler for changes'
required: false
default: 'content'
runs:
using: 'composite'
steps:
- name: Configure ccache
shell: bash
run: |
mkdir -p ~/.ccache
export CONF_PATH="${CCACHE_CONFIGPATH:-${CCACHE_DIR:-$HOME/.ccache}/ccache.conf}"
mkdir -p $(dirname "$CONF_PATH")
echo "max_size = ${{ inputs.max_size }}" > "$CONF_PATH"
echo "hash_dir = ${{ inputs.hash_dir }}" >> "$CONF_PATH"
echo "compiler_check = ${{ inputs.compiler_check }}" >> "$CONF_PATH"
ccache -p # Print config for verification
ccache -z # Zero statistics before the build

View File

@@ -2,14 +2,6 @@ name: build
description: 'Builds the project with ccache integration'
inputs:
cmake-target:
description: 'CMake target to build'
required: false
default: all
cmake-args:
description: 'Additional CMake arguments'
required: false
default: null
generator:
description: 'CMake generator to use'
required: true
@@ -28,22 +20,14 @@ inputs:
description: 'C++ compiler to use'
required: false
default: ''
gcov:
description: 'Gcov to use'
required: false
default: ''
compiler-id:
description: 'Unique identifier: compiler-version-stdlib[-gccversion] (e.g. clang-14-libstdcxx-gcc11, gcc-13-libstdcxx)'
description: 'Unique identifier for compiler/version combination used for cache keys'
required: false
default: ''
cache_version:
description: 'Cache version for invalidation'
required: false
default: '1'
gha_cache_enabled:
description: 'Whether to use actions/cache (disable for self-hosted with volume mounts)'
required: false
default: 'true'
ccache_enabled:
description: 'Whether to use ccache'
required: false
@@ -52,30 +36,6 @@ inputs:
description: 'Main branch name for restore keys'
required: false
default: 'dev'
stdlib:
description: 'C++ standard library to use (default = compiler default, e.g. GCC always uses libstdc++)'
required: true
type: choice
options:
- default
- libstdcxx
- libcxx
clang_gcc_toolchain:
description: 'GCC version to use for Clang toolchain (e.g. 11, 13)'
required: false
default: ''
ccache_max_size:
description: 'Maximum ccache size'
required: false
default: '2G'
ccache_hash_dir:
description: 'Whether to include directory paths in hash'
required: false
default: 'true'
ccache_compiler_check:
description: 'How to check compiler for changes'
required: false
default: 'content'
runs:
using: 'composite'
@@ -88,32 +48,18 @@ runs:
SAFE_BRANCH=$(echo "${{ github.ref_name }}" | tr -c 'a-zA-Z0-9_.-' '-')
echo "name=${SAFE_BRANCH}" >> $GITHUB_OUTPUT
- name: Configure ccache
- name: Restore ccache directory
if: inputs.ccache_enabled == 'true'
shell: bash
run: |
# Create cache directories
mkdir -p ~/.ccache-cache
# Keep config separate from cache_dir so configs aren't swapped when CCACHE_DIR changes between steps
mkdir -p ~/.config/ccache
export CCACHE_CONFIGPATH="$HOME/.config/ccache/ccache.conf"
echo "CCACHE_CONFIGPATH=$CCACHE_CONFIGPATH" >> $GITHUB_ENV
# Configure ccache settings AFTER cache restore (prevents stale cached config)
ccache --set-config=max_size=${{ inputs.ccache_max_size }}
ccache --set-config=hash_dir=${{ inputs.ccache_hash_dir }}
ccache --set-config=compiler_check=${{ inputs.ccache_compiler_check }}
ccache --set-config=cache_dir="$HOME/.ccache-cache"
echo "CCACHE_DIR=$HOME/.ccache-cache" >> $GITHUB_ENV
echo "📦 using ~/.ccache-cache as ccache cache directory"
# Print config for verification
echo "=== ccache configuration ==="
ccache -p
# Zero statistics before the build
ccache -z
id: ccache-restore
uses: actions/cache/restore@v4
with:
path: ~/.ccache
key: ${{ runner.os }}-ccache-v${{ inputs.cache_version }}-${{ inputs.compiler-id }}-${{ inputs.configuration }}-${{ steps.safe-branch.outputs.name }}
restore-keys: |
${{ runner.os }}-ccache-v${{ inputs.cache_version }}-${{ inputs.compiler-id }}-${{ inputs.configuration }}-${{ inputs.main_branch }}
${{ runner.os }}-ccache-v${{ inputs.cache_version }}-${{ inputs.compiler-id }}-${{ inputs.configuration }}-
${{ runner.os }}-ccache-v${{ inputs.cache_version }}-${{ inputs.compiler-id }}-
${{ runner.os }}-ccache-v${{ inputs.cache_version }}-
- name: Configure project
shell: bash
@@ -129,102 +75,34 @@ runs:
if [ -n "${{ inputs.cxx }}" ]; then
export CXX="${{ inputs.cxx }}"
fi
if [ -n "${{ inputs.gcov }}" ]; then
ln -sf /usr/bin/${{ inputs.gcov }} /usr/local/bin/gcov
fi
# Create wrapper toolchain that overlays ccache on top of Conan's toolchain
# This enables ccache for the main app build without affecting Conan dependency builds
if [ "${{ inputs.ccache_enabled }}" = "true" ]; then
cat > wrapper_toolchain.cmake <<'EOF'
# Include Conan's generated toolchain first (sets compiler, flags, etc.)
# Note: CMAKE_CURRENT_LIST_DIR is the directory containing this wrapper (.build/)
include(${CMAKE_CURRENT_LIST_DIR}/build/generators/conan_toolchain.cmake)
# Overlay ccache configuration for main application build
# This does NOT affect Conan dependency builds (already completed)
set(CMAKE_C_COMPILER_LAUNCHER ccache CACHE STRING "C compiler launcher" FORCE)
set(CMAKE_CXX_COMPILER_LAUNCHER ccache CACHE STRING "C++ compiler launcher" FORCE)
EOF
TOOLCHAIN_FILE="wrapper_toolchain.cmake"
echo "✅ Created wrapper toolchain with ccache enabled"
else
TOOLCHAIN_FILE="build/generators/conan_toolchain.cmake"
echo " Using Conan toolchain directly (ccache disabled)"
fi
# Configure C++ standard library if specified
# libstdcxx used for clang-14/16 to work around missing lexicographical_compare_three_way in libc++
# libcxx can be used with clang-17+ which has full C++20 support
# Note: -stdlib flag is Clang-specific, GCC always uses libstdc++
CMAKE_CXX_FLAGS=""
if [[ "${{ inputs.cxx }}" == clang* ]]; then
# Only Clang needs the -stdlib flag
if [ "${{ inputs.stdlib }}" = "libstdcxx" ]; then
CMAKE_CXX_FLAGS="-stdlib=libstdc++"
elif [ "${{ inputs.stdlib }}" = "libcxx" ]; then
CMAKE_CXX_FLAGS="-stdlib=libc++"
fi
fi
# GCC always uses libstdc++ and doesn't need/support the -stdlib flag
# Configure GCC toolchain for Clang if specified
if [ -n "${{ inputs.clang_gcc_toolchain }}" ] && [[ "${{ inputs.cxx }}" == clang* ]]; then
# Extract Clang version from compiler executable name (e.g., clang++-14 -> 14)
clang_version=$(echo "${{ inputs.cxx }}" | grep -oE '[0-9]+$')
# Clang 16+ supports --gcc-install-dir (precise path specification)
# Clang <16 only has --gcc-toolchain (uses discovery heuristics)
if [ -n "$clang_version" ] && [ "$clang_version" -ge "16" ]; then
# Clang 16+ uses --gcc-install-dir (canonical, precise)
CMAKE_CXX_FLAGS="$CMAKE_CXX_FLAGS --gcc-install-dir=/usr/lib/gcc/x86_64-linux-gnu/${{ inputs.clang_gcc_toolchain }}"
else
# Clang 14-15 uses --gcc-toolchain (deprecated but necessary)
# Note: This still uses discovery, so we hide newer GCC versions in the workflow
CMAKE_CXX_FLAGS="$CMAKE_CXX_FLAGS --gcc-toolchain=/usr"
fi
# Configure ccache launcher args
CCACHE_ARGS=""
if [ "${{ inputs.ccache_enabled }}" = "true" ]; then
CCACHE_ARGS="-DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache"
fi
# Run CMake configure
# Note: conanfile.py hardcodes 'build/generators' as the output path.
# If we're in a 'build' folder, Conan detects this and uses just 'generators/'
# If we're in '.build' (non-standard), Conan adds the full 'build/generators/'
# So we get: .build/build/generators/ with our non-standard folder name
cmake .. \
-G "${{ inputs.generator }}" \
${CMAKE_CXX_FLAGS:+-DCMAKE_CXX_FLAGS="$CMAKE_CXX_FLAGS"} \
-DCMAKE_TOOLCHAIN_FILE:FILEPATH=${TOOLCHAIN_FILE} \
-DCMAKE_BUILD_TYPE=${{ inputs.configuration }} \
-Dtests=TRUE \
-Dxrpld=TRUE \
${{ inputs.cmake-args }}
- name: Show ccache config before build
if: inputs.ccache_enabled == 'true'
shell: bash
run: |
echo "=========================================="
echo "ccache configuration before build"
echo "=========================================="
ccache -p
echo ""
$CCACHE_ARGS \
-DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake \
-DCMAKE_BUILD_TYPE=${{ inputs.configuration }}
- name: Build project
shell: bash
run: |
cd ${{ inputs.build_dir }}
# Check for verbose build flag in commit message
VERBOSE_FLAG=""
if echo "${XAHAU_GA_COMMIT_MSG}" | grep -q '\[ci-ga-cmake-verbose\]'; then
echo "🔊 [ci-ga-cmake-verbose] detected - enabling verbose output"
VERBOSE_FLAG="-- -v"
fi
cmake --build . --config ${{ inputs.configuration }} --parallel $(nproc) --target ${{ inputs.cmake-target }} ${VERBOSE_FLAG}
cmake --build . --config ${{ inputs.configuration }} --parallel $(nproc)
- name: Show ccache statistics
if: inputs.ccache_enabled == 'true'
shell: bash
run: ccache -s
- name: Save ccache directory
if: inputs.ccache_enabled == 'true'
uses: actions/cache/save@v4
with:
path: ~/.ccache
key: ${{ steps.ccache-restore.outputs.cache-primary-key }}

View File

@@ -1,146 +0,0 @@
name: 'Cache Restore'
description: 'Restores cache with optional clearing based on commit message tags'
inputs:
path:
description: 'A list of files, directories, and wildcard patterns to cache'
required: true
key:
description: 'An explicit key for restoring the cache'
required: true
restore-keys:
description: 'An ordered list of prefix-matched keys to use for restoring stale cache if no cache hit occurred for key'
required: false
default: ''
cache-type:
description: 'Type of cache (for logging purposes, e.g., "ccache-main", "Conan")'
required: false
default: 'cache'
fail-on-cache-miss:
description: 'Fail the workflow if cache entry is not found'
required: false
default: 'false'
lookup-only:
description: 'Check if a cache entry exists for the given input(s) without downloading it'
required: false
default: 'false'
additional-clear-keys:
description: 'Additional cache keys to clear (newline separated)'
required: false
default: ''
outputs:
cache-hit:
description: 'A boolean value to indicate an exact match was found for the primary key'
value: ${{ steps.restore-cache.outputs.cache-hit }}
cache-primary-key:
description: 'The key that was used to restore the cache'
value: ${{ steps.restore-cache.outputs.cache-primary-key }}
cache-matched-key:
description: 'The key that was used to restore the cache (exact or prefix match)'
value: ${{ steps.restore-cache.outputs.cache-matched-key }}
runs:
using: 'composite'
steps:
- name: Clear cache if requested via commit message
shell: bash
env:
GH_TOKEN: ${{ github.token }}
run: |
echo "=========================================="
echo "${{ inputs.cache-type }} cache clear tag detection"
echo "=========================================="
echo "Searching for: [ci-ga-clear-cache] or [ci-ga-clear-cache:*]"
echo ""
CACHE_KEY="${{ inputs.key }}"
# Extract search terms if present (e.g., "ccache" from "[ci-ga-clear-cache:ccache]")
SEARCH_TERMS=$(echo "${XAHAU_GA_COMMIT_MSG}" | grep -o '\[ci-ga-clear-cache:[^]]*\]' | sed 's/\[ci-ga-clear-cache://;s/\]//' || echo "")
SHOULD_CLEAR=false
if [ -n "${SEARCH_TERMS}" ]; then
# Search terms provided - check if THIS cache key matches ALL terms (AND logic)
echo "🔍 [ci-ga-clear-cache:${SEARCH_TERMS}] detected"
echo "Checking if cache key matches search terms..."
echo " Cache key: ${CACHE_KEY}"
echo " Search terms: ${SEARCH_TERMS}"
echo ""
MATCHES=true
for term in ${SEARCH_TERMS}; do
if ! echo "${CACHE_KEY}" | grep -q "${term}"; then
MATCHES=false
echo " ✗ Key does not contain '${term}'"
break
else
echo " ✓ Key contains '${term}'"
fi
done
if [ "${MATCHES}" = "true" ]; then
echo ""
echo "✅ Cache key matches all search terms - will clear cache"
SHOULD_CLEAR=true
else
echo ""
echo "⏭️ Cache key doesn't match search terms - skipping cache clear"
fi
elif echo "${XAHAU_GA_COMMIT_MSG}" | grep -q '\[ci-ga-clear-cache\]'; then
# No search terms - always clear this job's cache
echo "🗑️ [ci-ga-clear-cache] detected in commit message"
echo "Clearing ${{ inputs.cache-type }} cache for key: ${CACHE_KEY}"
SHOULD_CLEAR=true
fi
if [ "${SHOULD_CLEAR}" = "true" ]; then
echo ""
echo "Deleting ${{ inputs.cache-type }} caches via GitHub API..."
# Delete primary cache key
echo "Checking for cache: ${CACHE_KEY}"
if gh cache list --key "${CACHE_KEY}" --json key --jq '.[].key' | grep -q "${CACHE_KEY}"; then
echo " Deleting: ${CACHE_KEY}"
gh cache delete "${CACHE_KEY}" || true
echo " ✓ Deleted"
else
echo " Not found"
fi
# Delete additional keys if provided
if [ -n "${{ inputs.additional-clear-keys }}" ]; then
echo ""
echo "Checking additional keys..."
while IFS= read -r key; do
[ -z "${key}" ] && continue
echo "Checking for cache: ${key}"
if gh cache list --key "${key}" --json key --jq '.[].key' | grep -q "${key}"; then
echo " Deleting: ${key}"
gh cache delete "${key}" || true
echo " ✓ Deleted"
else
echo " Not found"
fi
done <<< "${{ inputs.additional-clear-keys }}"
fi
echo ""
echo "✅ ${{ inputs.cache-type }} cache cleared successfully"
echo "Build will proceed from scratch"
else
echo ""
echo " No ${{ inputs.cache-type }} cache clear requested"
fi
echo "=========================================="
- name: Restore cache
id: restore-cache
uses: actions/cache/restore@v4
with:
path: ${{ inputs.path }}
key: ${{ inputs.key }}
restore-keys: ${{ inputs.restore-keys }}
fail-on-cache-miss: ${{ inputs.fail-on-cache-miss }}
lookup-only: ${{ inputs.lookup-only }}

View File

@@ -10,46 +10,21 @@ inputs:
required: false
default: '.build'
compiler-id:
description: 'Unique identifier: compiler-version-stdlib[-gccversion] (e.g. clang-14-libstdcxx-gcc11, gcc-13-libstdcxx)'
description: 'Unique identifier for compiler/version combination used for cache keys'
required: false
default: ''
cache_version:
description: 'Cache version for invalidation'
required: false
default: '1'
cache_enabled:
description: 'Whether to use caching'
required: false
default: 'true'
main_branch:
description: 'Main branch name for restore keys'
required: false
default: 'dev'
os:
description: 'Operating system (Linux, Macos)'
required: false
default: 'Linux'
arch:
description: 'Architecture (x86_64, armv8)'
required: false
default: 'x86_64'
compiler:
description: 'Compiler type (gcc, clang, apple-clang)'
required: true
compiler_version:
description: 'Compiler version (11, 13, 14, etc.)'
required: true
cc:
description: 'C compiler executable (gcc-13, clang-14, etc.), empty for macOS'
required: false
default: ''
cxx:
description: 'C++ compiler executable (g++-14, clang++-14, etc.), empty for macOS'
required: false
default: ''
stdlib:
description: 'C++ standard library for Conan configuration (note: also in compiler-id)'
required: true
type: choice
options:
- libstdcxx
- libcxx
outputs:
cache-hit:
@@ -59,96 +34,36 @@ outputs:
runs:
using: 'composite'
steps:
- name: Configure Conan cache paths
if: inputs.os == 'Linux'
- name: Generate safe branch name
if: inputs.cache_enabled == 'true'
id: safe-branch
shell: bash
run: |
mkdir -p /.conan-cache/conan2 /.conan-cache/conan2_download /.conan-cache/conan2_sources
echo 'core.cache:storage_path=/.conan-cache/conan2' > ~/.conan2/global.conf
echo 'core.download:download_cache=/.conan-cache/conan2_download' >> ~/.conan2/global.conf
echo 'core.sources:download_cache=/.conan-cache/conan2_sources' >> ~/.conan2/global.conf
SAFE_BRANCH=$(echo "${{ github.ref_name }}" | tr -c 'a-zA-Z0-9_.-' '-')
echo "name=${SAFE_BRANCH}" >> $GITHUB_OUTPUT
- name: Configure Conan cache paths
if: inputs.gha_cache_enabled == 'false'
shell: bash
# For self-hosted runners, register cache paths to be used as volumes
# This allows the cache to be shared between containers
run: |
mkdir -p /.conan-cache/conan2 /.conan-cache/conan2_download /.conan-cache/conan2_sources
echo 'core.cache:storage_path=/.conan-cache/conan2' > ~/.conan2/global.conf
echo 'core.download:download_cache=/.conan-cache/conan2_download' >> ~/.conan2/global.conf
echo 'core.sources:download_cache=/.conan-cache/conan2_sources' >> ~/.conan2/global.conf
- name: Configure Conan
shell: bash
run: |
# Create the default profile directory if it doesn't exist
mkdir -p ~/.conan2/profiles
# Determine the correct libcxx based on stdlib parameter
if [ "${{ inputs.stdlib }}" = "libcxx" ]; then
LIBCXX="libc++"
else
LIBCXX="libstdc++11"
fi
# Create profile with our specific settings
# This overwrites any cached profile to ensure fresh configuration
cat > ~/.conan2/profiles/default <<EOF
[settings]
arch=${{ inputs.arch }}
build_type=${{ inputs.configuration }}
compiler=${{ inputs.compiler }}
compiler.cppstd=20
compiler.libcxx=${LIBCXX}
compiler.version=${{ inputs.compiler_version }}
os=${{ inputs.os }}
EOF
# Add buildenv and conf sections for Linux (not needed for macOS)
if [ "${{ inputs.os }}" = "Linux" ] && [ -n "${{ inputs.cc }}" ]; then
cat >> ~/.conan2/profiles/default <<EOF
[buildenv]
CC=/usr/bin/${{ inputs.cc }}
CXX=/usr/bin/${{ inputs.cxx }}
[conf]
tools.build:compiler_executables={"c": "/usr/bin/${{ inputs.cc }}", "cpp": "/usr/bin/${{ inputs.cxx }}"}
EOF
fi
# Add macOS-specific conf if needed
if [ "${{ inputs.os }}" = "Macos" ]; then
cat >> ~/.conan2/profiles/default <<EOF
[conf]
# Workaround for gRPC with newer Apple Clang
tools.build:cxxflags=["-Wno-missing-template-arg-list-after-template-kw"]
EOF
fi
# Display profile for verification
conan profile show
- name: Restore Conan cache
if: inputs.cache_enabled == 'true'
id: cache-restore-conan
uses: actions/cache/restore@v4
with:
path: |
~/.conan
~/.conan2
key: ${{ runner.os }}-conan-v${{ inputs.cache_version }}-${{ inputs.compiler-id }}-${{ hashFiles('**/conanfile.txt', '**/conanfile.py') }}-${{ inputs.configuration }}
restore-keys: |
${{ runner.os }}-conan-v${{ inputs.cache_version }}-${{ inputs.compiler-id }}-${{ hashFiles('**/conanfile.txt', '**/conanfile.py') }}-
${{ runner.os }}-conan-v${{ inputs.cache_version }}-${{ inputs.compiler-id }}-
${{ runner.os }}-conan-v${{ inputs.cache_version }}-
- name: Export custom recipes
shell: bash
run: |
# Export snappy if not already exported
conan list snappy/1.1.10@xahaud/stable 2>/dev/null | (grep -q "not found" && exit 1 || exit 0) || \
conan export external/snappy --version 1.1.10 --user xahaud --channel stable
# Export soci if not already exported
conan list soci/4.0.3@xahaud/stable 2>/dev/null | (grep -q "not found" && exit 1 || exit 0) || \
conan export external/soci --version 4.0.3 --user xahaud --channel stable
# Export wasmedge if not already exported
conan list wasmedge/0.11.2@xahaud/stable 2>/dev/null | (grep -q "not found" && exit 1 || exit 0) || \
conan export external/wasmedge --version 0.11.2 --user xahaud --channel stable
conan export external/snappy snappy/1.1.9@
conan export external/soci soci/4.0.3@
- name: Install dependencies
shell: bash
env:
CONAN_REQUEST_TIMEOUT: 180 # Increase timeout to 3 minutes for slow mirrors
run: |
# Create build directory
mkdir -p ${{ inputs.build_dir }}
@@ -160,3 +75,12 @@ runs:
--build missing \
--settings build_type=${{ inputs.configuration }} \
..
- name: Save Conan cache
if: inputs.cache_enabled == 'true' && steps.cache-restore-conan.outputs.cache-hit != 'true'
uses: actions/cache/save@v4
with:
path: |
~/.conan
~/.conan2
key: ${{ steps.cache-restore-conan.outputs.cache-primary-key }}

View File

@@ -1,74 +0,0 @@
name: 'Get Commit Message'
description: 'Gets commit message for both push and pull_request events and sets XAHAU_GA_COMMIT_MSG env var'
inputs:
event-name:
description: 'The event name (push or pull_request)'
required: true
head-commit-message:
description: 'The head commit message (for push events)'
required: false
default: ''
pr-head-sha:
description: 'The PR head SHA (for pull_request events)'
required: false
default: ''
runs:
using: 'composite'
steps:
- name: Get commit message and set environment variable
shell: python
env:
GH_TOKEN: ${{ github.token }}
run: |
import json
import os
import secrets
import urllib.request
event_name = "${{ inputs.event-name }}"
pr_head_sha = "${{ inputs.pr-head-sha }}"
repository = "${{ github.repository }}"
print("==========================================")
print("Setting XAHAU_GA_COMMIT_MSG environment variable")
print("==========================================")
print(f"Event: {event_name}")
if event_name == 'push':
# For push events, use the input directly
message = """${{ inputs.head-commit-message }}"""
print("Source: workflow input (github.event.head_commit.message)")
elif event_name == 'pull_request' and pr_head_sha:
# For PR events, fetch via GitHub API
print(f"Source: GitHub API (fetching commit {pr_head_sha})")
try:
url = f"https://api.github.com/repos/{repository}/commits/{pr_head_sha}"
req = urllib.request.Request(url, headers={
"Accept": "application/vnd.github.v3+json",
"Authorization": f"Bearer {os.environ.get('GH_TOKEN', '')}"
})
with urllib.request.urlopen(req) as response:
data = json.load(response)
message = data["commit"]["message"]
except Exception as e:
print(f"Failed to fetch commit message: {e}")
message = ""
else:
message = ""
print(f"Warning: Unknown event type: {event_name}")
print(f"Commit message (first 100 chars): {message[:100]}")
# Write to GITHUB_ENV using heredoc with random delimiter (prevents injection attacks)
# See: https://securitylab.github.com/resources/github-actions-untrusted-input/
delimiter = f"EOF_{secrets.token_hex(16)}"
with open(os.environ['GITHUB_ENV'], 'a') as f:
f.write(f'XAHAU_GA_COMMIT_MSG<<{delimiter}\n')
f.write(message)
f.write(f'\n{delimiter}\n')
print(f"✓ XAHAU_GA_COMMIT_MSG set (available to all subsequent steps)")
print("==========================================")

View File

@@ -32,9 +32,19 @@ jobs:
clean: true
fetch-depth: 2 # Only get the last 2 commits, to avoid fetching all history
checkpatterns:
runs-on: [self-hosted, vanity]
needs: checkout
defaults:
run:
working-directory: ${{ needs.checkout.outputs.checkout_path }}
steps:
- name: Check for suspicious patterns
run: /bin/bash suspicious_patterns.sh
build:
runs-on: [self-hosted, xahaud-build]
needs: [checkout]
runs-on: [self-hosted, vanity]
needs: [checkpatterns, checkout]
defaults:
run:
working-directory: ${{ needs.checkout.outputs.checkout_path }}
@@ -74,7 +84,7 @@ jobs:
fi
tests:
runs-on: [self-hosted, xahaud-build]
runs-on: [self-hosted, vanity]
needs: [build, checkout]
defaults:
run:
@@ -84,7 +94,7 @@ jobs:
run: /bin/bash docker-unit-tests.sh
cleanup:
runs-on: [self-hosted, xahaud-build]
runs-on: [self-hosted, vanity]
needs: [tests, checkout]
if: always()
steps:

View File

@@ -1,107 +0,0 @@
name: Check Genesis Hooks
on:
push:
pull_request:
jobs:
check-genesis-hooks:
runs-on: ubuntu-24.04
env:
CLANG_VERSION: 18
name: Verify xahau.h is in sync with genesis hooks
steps:
- name: Checkout repository
uses: actions/checkout@v6
# Install binaryen from GitHub Releases (pinned to version 100)
- name: Install binaryen (version 100)
run: |
curl -LO https://github.com/WebAssembly/binaryen/releases/download/version_100/binaryen-version_100-x86_64-linux.tar.gz
tar -xzf binaryen-version_100-x86_64-linux.tar.gz
sudo cp binaryen-version_100/bin/* /usr/local/bin/
wasm-opt --version
- name: Install clang-format
run: |
codename=$( lsb_release --codename --short )
sudo tee /etc/apt/sources.list.d/llvm.list >/dev/null <<EOF
deb http://apt.llvm.org/${codename}/ llvm-toolchain-${codename}-${CLANG_VERSION} main
deb-src http://apt.llvm.org/${codename}/ llvm-toolchain-${codename}-${CLANG_VERSION} main
EOF
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add
sudo apt-get update
sudo apt-get install clang-format-${CLANG_VERSION}
clang-format --version
# Install wasienv (WebAssembly SDK)
- name: Install wasienv
run: |
# Download install.sh
curl -o /tmp/wasienv-install.sh https://raw.githubusercontent.com/wasienv/wasienv/master/install.sh
# Replace /bin to /local/bin
sed -i 's|/bin|/local/bin|g' /tmp/wasienv-install.sh
# Execute the installed script
bash /tmp/wasienv-install.sh
# Add wasienv to PATH for subsequent steps
- name: Setup wasienv
run: |
echo "$HOME/.wasienv/bin" >> $GITHUB_PATH
wasmcc -v || true
# Build and install hook-cleaner tool
- name: Build and install hook-cleaner
run: |
git clone https://github.com/richardah/hook-cleaner-c.git /tmp/hook-cleaner
cd /tmp/hook-cleaner
make
cp hook-cleaner /usr/local/bin/
chmod +x /usr/local/bin/hook-cleaner
# Build and install guard_checker tool
- name: Build and install guard_checker
run: |
cd include/xrpl/hook
make
cp guard_checker /usr/local/bin/
chmod +x /usr/local/bin/guard_checker
# Verify all required tools are available
- name: Verify required tools
run: |
echo "Checking tool availability..."
command -v wasmcc || (echo "Error: wasmcc not found" && exit 1)
command -v wasm-opt || (echo "Error: wasm-opt not found" && exit 1)
command -v hook-cleaner || (echo "Error: hook-cleaner not found" && exit 1)
command -v guard_checker || (echo "Error: guard_checker not found" && exit 1)
command -v xxd || (echo "Error: xxd not found" && exit 1)
command -v clang-format || (echo "Error: clang-format not found" && exit 1)
echo "All tools verified successfully"
# Execute build script to regenerate xahau.h
- name: Run build_xahau_h.sh
run: |
cd hook/genesis
./build_xahau_h.sh
# Check if xahau.h has changed (fail if out of sync)
- name: Verify xahau.h is in sync
run: |
if ! git diff --exit-code include/xrpl/hook/xahau.h; then
echo ""
echo "❌ ERROR: xahau.h is out of sync with genesis hooks"
echo ""
echo "The generated xahau.h differs from the committed version."
echo "Please run the following command and commit the changes:"
echo ""
echo " cd hook/genesis && ./build_xahau_h.sh"
echo ""
echo "Diff:"
git diff include/xrpl/hook/xahau.h
exit 1
fi
echo "✅ xahau.h is in sync with genesis hooks"

20
.github/workflows/checkpatterns.yml vendored Normal file
View File

@@ -0,0 +1,20 @@
name: checkpatterns
on: [push, pull_request]
jobs:
checkpatterns:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Check for suspicious patterns
run: |
if [ -f "suspicious_patterns.sh" ]; then
bash suspicious_patterns.sh
else
echo "Warning: suspicious_patterns.sh not found, skipping check"
# Still exit with success for compatibility with dependent jobs
exit 0
fi

View File

@@ -4,23 +4,36 @@ on: [push, pull_request]
jobs:
check:
runs-on: ubuntu-24.04
runs-on: ubuntu-22.04
env:
CLANG_VERSION: 18
CLANG_VERSION: 10
steps:
- uses: actions/checkout@v4
- name: Install clang-format
- uses: actions/checkout@v3
# - name: Install clang-format
# run: |
# codename=$( lsb_release --codename --short )
# sudo tee /etc/apt/sources.list.d/llvm.list >/dev/null <<EOF
# deb http://apt.llvm.org/${codename}/ llvm-toolchain-${codename}-${CLANG_VERSION} main
# deb-src http://apt.llvm.org/${codename}/ llvm-toolchain-${codename}-${CLANG_VERSION} main
# EOF
# wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add
# sudo apt-get update -y
# sudo apt-get install -y clang-format-${CLANG_VERSION}
# Temporary fix until this commit is merged
# https://github.com/XRPLF/rippled/commit/552377c76f55b403a1c876df873a23d780fcc81c
- name: Download and install clang-format
run: |
codename=$( lsb_release --codename --short )
sudo tee /etc/apt/sources.list.d/llvm.list >/dev/null <<EOF
deb http://apt.llvm.org/${codename}/ llvm-toolchain-${codename}-${CLANG_VERSION} main
deb-src http://apt.llvm.org/${codename}/ llvm-toolchain-${codename}-${CLANG_VERSION} main
EOF
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add
sudo apt-get update
sudo apt-get install clang-format-${CLANG_VERSION}
- name: Format first-party sources
run: find include src -type f \( -name '*.cpp' -o -name '*.hpp' -o -name '*.h' -o -name '*.ipp' \) -exec clang-format-${CLANG_VERSION} -i {} +
sudo apt-get update -y
sudo apt-get install -y libtinfo5
curl -LO https://github.com/llvm/llvm-project/releases/download/llvmorg-10.0.1/clang+llvm-10.0.1-x86_64-linux-gnu-ubuntu-16.04.tar.xz
tar -xf clang+llvm-10.0.1-x86_64-linux-gnu-ubuntu-16.04.tar.xz
sudo mv clang+llvm-10.0.1-x86_64-linux-gnu-ubuntu-16.04 /opt/clang-10
sudo ln -s /opt/clang-10/bin/clang-format /usr/local/bin/clang-format-10
- name: Format src/ripple
run: find src/ripple -type f \( -name '*.cpp' -o -name '*.h' -o -name '*.ipp' \) -print0 | xargs -0 clang-format-${CLANG_VERSION} -i
- name: Format src/test
run: find src/test -type f \( -name '*.cpp' -o -name '*.h' -o -name '*.ipp' \) -print0 | xargs -0 clang-format-${CLANG_VERSION} -i
- name: Check for differences
id: assert
run: |
@@ -50,7 +63,7 @@ jobs:
To fix it, you can do one of two things:
1. Download and apply the patch generated as an artifact of this
job to your repo, commit, and push.
2. Run 'git-clang-format --extensions cpp,h,hpp,ipp develop'
2. Run 'git-clang-format --extensions c,cpp,h,cxx,ipp dev'
in your repo, commit, and push.
run: |
echo "${PREAMBLE}"

View File

@@ -1,127 +0,0 @@
name: Formal Verification (Lean)
on:
push:
branches: ["feature-export-rng-lean"]
pull_request:
branches: ["**"]
types: [opened, synchronize, reopened]
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
lean-consensus:
name: Lean/C++ drift checks
runs-on: [self-hosted, macOS]
env:
BUILD_DIR: .build-formal
CMAKE_BUILD_DIR: .build-formal-cmake
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Add Homebrew to PATH
run: |
echo "/opt/homebrew/bin" >> "$GITHUB_PATH"
echo "/opt/homebrew/sbin" >> "$GITHUB_PATH"
- name: Install core tools
run: |
brew install coreutils
echo "Num proc: $(nproc)"
- name: Setup toolchain (mise)
uses: jdx/mise-action@v3.6.1
with:
cache: false
install: true
mise_toml: |
[tools]
cmake = "3.25.3"
python = "3.12"
pipx = "latest"
conan = "2"
ninja = "latest"
- name: Install tools via mise
run: |
mise install
mise reshim
echo "$HOME/.local/share/mise/shims" >> "$GITHUB_PATH"
- name: Install Lean toolchain
run: |
toolchain="$(cat formal_verification/lean-toolchain)"
curl -sSfL https://raw.githubusercontent.com/leanprover/elan/master/elan-init.sh \
| sh -s -- -y --default-toolchain "$toolchain"
echo "$HOME/.elan/bin" >> "$GITHUB_PATH"
"$HOME/.elan/bin/lake" --version
"$HOME/.elan/bin/lean" --version
- name: Build Lean proofs
run: |
cd formal_verification
"$HOME/.elan/bin/lake" build XahauConsensus:static
- name: Detect compiler version
id: detect-compiler
run: |
compiler_version=$(clang --version | grep -oE 'version [0-9]+' | grep -oE '[0-9]+')
echo "compiler_version=${compiler_version}" >> "$GITHUB_OUTPUT"
echo "Detected Apple Clang version: ${compiler_version}"
- name: Configure Conan profile
run: |
mkdir -p ~/.conan2/profiles
cat > ~/.conan2/profiles/default <<EOF
[settings]
arch=armv8
build_type=Debug
compiler=apple-clang
compiler.cppstd=20
compiler.libcxx=libc++
compiler.version=${{ steps.detect-compiler.outputs.compiler_version }}
os=Macos
[conf]
tools.build:cxxflags=["-Wno-missing-template-arg-list-after-template-kw"]
EOF
conan profile show
- name: Export custom Conan recipes
run: |
conan export external/snappy --version 1.1.10 --user xahaud --channel stable
conan export external/soci --version 4.0.3 --user xahaud --channel stable
conan export external/wasmedge --version 0.11.2 --user xahaud --channel stable
- name: Install Conan dependencies
env:
CONAN_REQUEST_TIMEOUT: 180
run: |
conan install . \
--output-folder "$BUILD_DIR" \
--build missing \
--settings build_type=Debug \
-o '&:tests=True' \
-o '&:xrpld=True' \
-o '&:formal_verification=True'
- name: Configure formal build
run: |
cmake -S . -B "$CMAKE_BUILD_DIR" -G Ninja \
-DCMAKE_TOOLCHAIN_FILE="$PWD/$BUILD_DIR/build/generators/conan_toolchain.cmake" \
-DCMAKE_BUILD_TYPE=Debug \
-Dtests=ON \
-Dxrpld=ON \
-Dformal_verification=ON
- name: Build formal-enabled rippled
run: |
cmake --build "$CMAKE_BUILD_DIR" --target rippled --parallel "$(nproc)"
- name: Run Lean/C++ drift checks
run: |
"$CMAKE_BUILD_DIR/rippled" --unittest=LeanConsensus --unittest-log

View File

@@ -1,24 +0,0 @@
name: Guard Checker Build
on:
push:
pull_request:
jobs:
guard-checker-build:
strategy:
fail-fast: false
matrix:
include:
- run-on: ubuntu-latest
- run-on: macos-latest
runs-on: ${{ matrix.run-on }}
name: Guard Checker Build - ${{ matrix.run-on }}
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Build Guard Checker
run: |
cd include/xrpl/hook
make guard_checker

View File

@@ -1,104 +0,0 @@
name: instrumentation
on:
pull_request:
push:
# If the branches list is ever changed, be sure to change it on all
# build/test jobs (nix, macos, windows, instrumentation)
branches:
# Always build the package branches
- develop
- release
- master
# Branches that opt-in to running
- 'ci/**'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
# NOTE we are not using dependencies built inside nix because nix is lagging
# with compiler versions. Instrumentation requires clang version 16 or later
instrumentation-build:
if: false # disable for now
env:
CLANG_RELEASE: 16
strategy:
fail-fast: false
runs-on: [self-hosted, heavy]
container: debian:bookworm
steps:
- name: install prerequisites
env:
DEBIAN_FRONTEND: noninteractive
run: |
apt-get update
apt-get install --yes --no-install-recommends \
clang-${CLANG_RELEASE} clang++-${CLANG_RELEASE} \
python3-pip python-is-python3 make cmake git wget
apt-get clean
update-alternatives --install \
/usr/bin/clang clang /usr/bin/clang-${CLANG_RELEASE} 100 \
--slave /usr/bin/clang++ clang++ /usr/bin/clang++-${CLANG_RELEASE}
update-alternatives --auto clang
pip install --no-cache --break-system-packages "conan<2"
- name: checkout
uses: actions/checkout@v4
- name: prepare environment
run: |
mkdir ${GITHUB_WORKSPACE}/.build
echo "SOURCE_DIR=$GITHUB_WORKSPACE" >> $GITHUB_ENV
echo "BUILD_DIR=$GITHUB_WORKSPACE/.build" >> $GITHUB_ENV
echo "CC=/usr/bin/clang" >> $GITHUB_ENV
echo "CXX=/usr/bin/clang++" >> $GITHUB_ENV
- name: configure Conan
run: |
conan profile new --detect default
conan profile update settings.compiler=clang default
conan profile update settings.compiler.version=${CLANG_RELEASE} default
conan profile update settings.compiler.libcxx=libstdc++11 default
conan profile update settings.compiler.cppstd=20 default
conan profile update options.rocksdb=False default
conan profile update \
'conf.tools.build:compiler_executables={"c": "/usr/bin/clang", "cpp": "/usr/bin/clang++"}' default
conan profile update 'env.CXXFLAGS="-DBOOST_ASIO_DISABLE_CONCEPTS"' default
conan profile update 'conf.tools.build:cxxflags+=["-DBOOST_ASIO_DISABLE_CONCEPTS"]' default
conan export external/snappy snappy/1.1.10@
conan export external/soci soci/4.0.3@
- name: build dependencies
run: |
cd ${BUILD_DIR}
conan install ${SOURCE_DIR} \
--output-folder ${BUILD_DIR} \
--install-folder ${BUILD_DIR} \
--build missing \
--settings build_type=Debug
- name: build with instrumentation
run: |
cd ${BUILD_DIR}
cmake -S ${SOURCE_DIR} -B ${BUILD_DIR} \
-Dvoidstar=ON \
-Dtests=ON \
-Dxrpld=ON \
-DCMAKE_BUILD_TYPE=Debug \
-DSECP256K1_BUILD_BENCHMARK=OFF \
-DSECP256K1_BUILD_TESTS=OFF \
-DSECP256K1_BUILD_EXHAUSTIVE_TESTS=OFF \
-DCMAKE_TOOLCHAIN_FILE=${BUILD_DIR}/build/generators/conan_toolchain.cmake
cmake --build . --parallel $(nproc)
- name: verify instrumentation enabled
run: |
cd ${BUILD_DIR}
./rippled --version | grep libvoidstar
- name: run unit tests
run: |
cd ${BUILD_DIR}
./rippled -u --unittest-jobs $(( $(nproc)/4 ))

View File

@@ -10,7 +10,7 @@ jobs:
steps:
- uses: actions/checkout@v3
- name: Check levelization
run: python Builds/levelization/levelization.py
run: Builds/levelization/levelization.sh
- name: Check for differences
id: assert
run: |
@@ -40,7 +40,7 @@ jobs:
To fix it, you can do one of two things:
1. Download and apply the patch generated as an artifact of this
job to your repo, commit, and push.
2. Run 'python Builds/levelization/levelization.py' in your repo,
2. Run './Builds/levelization/levelization.sh' in your repo,
commit, and push.
See Builds/levelization/README.md for more info.

View File

@@ -1,54 +0,0 @@
name: Verify Generated Hook Headers
on:
push:
pull_request:
jobs:
verify-generated-headers:
strategy:
fail-fast: false
matrix:
include:
- target: hook/error.h
generator: ./hook/generate_error.sh
- target: hook/extern.h
generator: ./hook/generate_extern.sh
- target: hook/sfcodes.h
generator: bash ./hook/generate_sfcodes.sh
- target: hook/tts.h
generator: ./hook/generate_tts.sh
- target: hook/ls_flags.h
generator: ./hook/generate_lsflags.sh
- target: hook/tx_flags.h
generator: ./hook/generate_txflags.sh
runs-on: ubuntu-24.04
env:
CLANG_VERSION: 18
name: ${{ matrix.target }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Install clang-format
run: |
codename=$( lsb_release --codename --short )
sudo tee /etc/apt/sources.list.d/llvm.list >/dev/null <<EOF
deb http://apt.llvm.org/${codename}/ llvm-toolchain-${codename}-${CLANG_VERSION} main
deb-src http://apt.llvm.org/${codename}/ llvm-toolchain-${codename}-${CLANG_VERSION} main
EOF
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add
sudo apt-get update
sudo apt-get install clang-format-${CLANG_VERSION}
sudo ln -sf /usr/bin/clang-format-${CLANG_VERSION} /usr/local/bin/clang-format
- name: Verify ${{ matrix.target }}
run: |
set -euo pipefail
chmod +x hook/generate_*.sh || true
tmp=$(mktemp)
trap 'rm -f "$tmp"' EXIT
${{ matrix.generator }} > "$tmp"
diff -u ${{ matrix.target }} "$tmp"

View File

@@ -4,10 +4,7 @@ on:
push:
branches: ["dev", "candidate", "release"]
pull_request:
branches: ["**"]
types: [opened, synchronize, reopened, labeled, unlabeled]
schedule:
- cron: '0 0 * * *'
branches: ["dev", "candidate", "release"]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -15,57 +12,68 @@ concurrency:
jobs:
test:
if: >
github.event_name != 'pull_request' ||
contains(fromJson('["dev","candidate","release"]'), github.base_ref) ||
contains(join(github.event.pull_request.labels.*.name, ','), 'ci-full-build')
strategy:
matrix:
generator:
- Ninja
configuration:
- Debug
runs-on: [self-hosted, macOS]
runs-on: macos-15
env:
build_dir: .build
# Bump this number to invalidate all caches globally.
CACHE_VERSION: 3
CACHE_VERSION: 1
MAIN_BRANCH_NAME: dev
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Add Homebrew to PATH
- name: Install Conan
run: |
echo "/opt/homebrew/bin" >> "$GITHUB_PATH"
echo "/opt/homebrew/sbin" >> "$GITHUB_PATH"
brew install conan@1
# Add Conan 1 to the PATH for this job
echo "$(brew --prefix conan@1)/bin" >> $GITHUB_PATH
- name: Install Coreutils
run: |
brew install coreutils
echo "Num proc: $(nproc)"
# To isolate environments for each Runner, instead of installing globally with brew,
# use mise to isolate environments for each Runner directory.
- name: Setup toolchain (mise)
uses: jdx/mise-action@v3.6.1
with:
cache: false
install: true
mise_toml: |
[tools]
cmake = "3.25.3"
python = "3.12"
pipx = "latest"
conan = "2"
ninja = "latest"
ccache = "latest"
- name: Install Ninja
if: matrix.generator == 'Ninja'
run: brew install ninja
- name: Install tools via mise
- name: Install Python
run: |
mise install
mise reshim
echo "$HOME/.local/share/mise/shims" >> "$GITHUB_PATH"
if which python3 > /dev/null 2>&1; then
echo "Python 3 executable exists"
python3 --version
else
brew install python@3.12
fi
# Create 'python' symlink if it doesn't exist (for tools expecting 'python')
if ! which python > /dev/null 2>&1; then
sudo ln -sf $(which python3) /usr/local/bin/python
fi
- name: Install CMake
run: |
if which cmake > /dev/null 2>&1; then
echo "cmake executable exists"
cmake --version
else
brew install cmake
fi
- name: Install ccache
run: brew install ccache
- name: Configure ccache
uses: ./.github/actions/xahau-configure-ccache
with:
max_size: 2G
hash_dir: true
compiler_check: content
- name: Check environment
run: |
@@ -79,20 +87,10 @@ jobs:
echo "---- Full Environment ----"
env
- name: Get commit message
id: get-commit-message
uses: ./.github/actions/xahau-ga-get-commit-message
with:
event-name: ${{ github.event_name }}
head-commit-message: ${{ github.event.head_commit.message }}
pr-head-sha: ${{ github.event.pull_request.head.sha }}
- name: Detect compiler version
id: detect-compiler
- name: Configure Conan
run: |
COMPILER_VERSION=$(clang --version | grep -oE 'version [0-9]+' | grep -oE '[0-9]+')
echo "compiler_version=${COMPILER_VERSION}" >> $GITHUB_OUTPUT
echo "Detected Apple Clang version: ${COMPILER_VERSION}"
conan profile new default --detect || true # Ignore error if profile exists
conan profile update settings.compiler.cppstd=20 default
- name: Install dependencies
uses: ./.github/actions/xahau-ga-dependencies
@@ -102,11 +100,6 @@ jobs:
compiler-id: clang
cache_version: ${{ env.CACHE_VERSION }}
main_branch: ${{ env.MAIN_BRANCH_NAME }}
os: Macos
arch: armv8
compiler: apple-clang
compiler_version: ${{ steps.detect-compiler.outputs.compiler_version }}
stdlib: libcxx
- name: Build
uses: ./.github/actions/xahau-ga-build
@@ -117,9 +110,7 @@ jobs:
compiler-id: clang
cache_version: ${{ env.CACHE_VERSION }}
main_branch: ${{ env.MAIN_BRANCH_NAME }}
stdlib: libcxx
ccache_max_size: '100G'
- name: Test
run: |
${{ env.build_dir }}/rippled --unittest --unittest-jobs $(nproc)
${{ env.build_dir }}/rippled --unittest --unittest-jobs $(nproc)

View File

@@ -4,252 +4,31 @@ on:
push:
branches: ["dev", "candidate", "release"]
pull_request:
branches: ["**"]
types: [opened, synchronize, reopened, labeled, unlabeled]
schedule:
- cron: '0 0 * * *'
workflow_dispatch:
inputs:
full_matrix:
description: "Force full matrix (6 configs)"
required: false
default: "false"
branches: ["dev", "candidate", "release"]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
matrix-setup:
if: >
github.event_name != 'pull_request' ||
contains(fromJson('["dev","candidate","release"]'), github.base_ref) ||
contains(join(github.event.pull_request.labels.*.name, ','), 'ci-full-build')
runs-on: [self-hosted, generic, 20.04]
container: python:3-slim
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- name: escape double quotes
id: escape
shell: bash
env:
PR_TITLE: ${{ github.event.pull_request.title }}
run: |
ESCAPED_PR_TITLE="${PR_TITLE//\"/\\\"}"
echo "title=${ESCAPED_PR_TITLE}" >> "$GITHUB_OUTPUT"
- name: Generate build matrix
id: set-matrix
shell: python
env:
GH_TOKEN: ${{ github.token }}
run: |
import json
import os
import urllib.request
# Full matrix with all 6 compiler configurations
# Each configuration includes all parameters needed by the build job
full_matrix = [
{
"compiler_id": "gcc-11-libstdcxx",
"compiler": "gcc",
"cc": "gcc-11",
"cxx": "g++-11",
"compiler_version": 11,
"stdlib": "default",
"configuration": "Debug",
"job_type": "build"
},
{
"compiler_id": "gcc-13-libstdcxx",
"compiler": "gcc",
"cc": "gcc-13",
"cxx": "g++-13",
"compiler_version": 13,
"stdlib": "default",
"configuration": "Debug",
"job_type": "build"
},
{
"compiler_id": "gcc-13-libstdcxx",
"compiler": "gcc",
"cc": "gcc-13",
"cxx": "g++-13",
"gcov": "gcov-13",
"compiler_version": 13,
"stdlib": "default",
"configuration": "Debug",
"job_type": "coverage"
},
{
"compiler_id": "clang-14-libstdcxx-gcc11",
"compiler": "clang",
"cc": "clang-14",
"cxx": "clang++-14",
"compiler_version": 14,
"stdlib": "libstdcxx",
"clang_gcc_toolchain": 11,
"configuration": "Debug",
"job_type": "build"
},
{
"compiler_id": "clang-16-libstdcxx-gcc13",
"compiler": "clang",
"cc": "clang-16",
"cxx": "clang++-16",
"compiler_version": 16,
"stdlib": "libstdcxx",
"clang_gcc_toolchain": 13,
"configuration": "Debug",
"job_type": "build"
},
{
"compiler_id": "clang-17-libcxx",
"compiler": "clang",
"cc": "clang-17",
"cxx": "clang++-17",
"compiler_version": 17,
"stdlib": "libcxx",
"configuration": "Debug",
"job_type": "build"
},
{
# Clang 18 - testing if it's faster than Clang 17 with libc++
# Requires patching Conan v1 settings.yml to add version 18
"compiler_id": "clang-18-libcxx",
"compiler": "clang",
"cc": "clang-18",
"cxx": "clang++-18",
"compiler_version": 18,
"stdlib": "libcxx",
"configuration": "Debug",
"job_type": "build"
}
]
# Minimal matrix for PRs and feature branches
minimal_matrix = [
full_matrix[1], # gcc-13 (middle-ground gcc)
full_matrix[2], # gcc-13 coverage
full_matrix[3] # clang-14 (mature, stable clang)
]
# Determine which matrix to use based on the target branch
ref = "${{ github.ref }}"
base_ref = "${{ github.base_ref }}" # For PRs, this is the target branch
event_name = "${{ github.event_name }}"
pr_title = """${{ steps.escape.outputs.title }}"""
pr_labels = """${{ join(github.event.pull_request.labels.*.name, ',') }}"""
pr_head_sha = "${{ github.event.pull_request.head.sha }}"
# Get commit message - for PRs, fetch via API since head_commit.message is empty
if event_name == "pull_request" and pr_head_sha:
try:
url = f"https://api.github.com/repos/${{ github.repository }}/commits/{pr_head_sha}"
req = urllib.request.Request(url, headers={
"Accept": "application/vnd.github.v3+json",
"Authorization": f"Bearer {os.environ.get('GH_TOKEN', '')}"
})
with urllib.request.urlopen(req) as response:
data = json.load(response)
commit_message = data["commit"]["message"]
except Exception as e:
print(f"Failed to fetch commit message: {e}")
commit_message = ""
else:
commit_message = """${{ github.event.head_commit.message }}"""
# Debug logging
print(f"Event: {event_name}")
print(f"Ref: {ref}")
print(f"Base ref: {base_ref}")
print(f"PR head SHA: {pr_head_sha}")
print(f"PR title: {pr_title}")
print(f"PR labels: {pr_labels}")
print(f"Commit message: {commit_message}")
# Manual trigger input to force full matrix.
manual_full = "${{ github.event.inputs.full_matrix || 'false' }}" == "true"
# Label/manual overrides, while preserving existing title/commit behavior.
force_full = (
manual_full
or "[ci-nix-full-matrix]" in commit_message
or "[ci-nix-full-matrix]" in pr_title
or ("ci-full-build" in pr_labels and "ci-nix-full-matrix" in pr_labels)
)
force_min = (
"ci-full-build" in pr_labels
)
print(f"Force full matrix: {force_full}")
print(f"Force min matrix: {force_min}")
# Check if this is targeting a main branch
# For PRs: check base_ref (target branch)
# For pushes: check ref (current branch)
main_branches = ["refs/heads/dev", "refs/heads/release", "refs/heads/candidate"]
if force_full:
# Override: always use full matrix if forced by manual input or label.
use_full = True
elif force_min:
# Override: always use minimal matrix if ci-full-build label is present.
use_full = False
elif event_name == "pull_request":
# For PRs, base_ref is just the branch name (e.g., "dev", not "refs/heads/dev")
# Check if the PR targets release or candidate (more critical branches)
use_full = base_ref in ["release", "candidate"]
else:
# For pushes, ref is the full reference (e.g., "refs/heads/dev")
use_full = ref in main_branches
# Select the appropriate matrix
if use_full:
if force_full:
print(f"Using FULL matrix (7 configs) - forced by [ci-nix-full-matrix] tag")
else:
print(f"Using FULL matrix (7 configs) - targeting main branch")
matrix = full_matrix
else:
print(f"Using MINIMAL matrix (3 configs) - feature branch/PR")
matrix = minimal_matrix
# Add runs_on based on job_type
for entry in matrix:
if entry.get("job_type") == "coverage":
entry["runs_on"] = '["self-hosted", "generic", 24.04]'
else:
entry["runs_on"] = '["self-hosted", "generic", 20.04]'
# Output the matrix as JSON
output = json.dumps({"include": matrix})
with open(os.environ['GITHUB_OUTPUT'], 'a') as f:
f.write(f"matrix={output}\n")
build:
needs: matrix-setup
runs-on: ${{ fromJSON(matrix.runs_on) }}
permissions:
id-token: write
contents: read
container:
image: ubuntu:24.04
volumes:
- /home/runner/.conan-cache:/.conan-cache
- /home/runner/.ccache-cache:/github/home/.ccache-cache
defaults:
run:
shell: bash
build-job:
runs-on: ubuntu-latest
outputs:
artifact_name: ${{ steps.set-artifact-name.outputs.artifact_name }}
strategy:
fail-fast: false
matrix: ${{ fromJSON(needs.matrix-setup.outputs.matrix) }}
matrix:
compiler: [gcc]
configuration: [Debug]
include:
- compiler: gcc
cc: gcc-11
cxx: g++-11
compiler_id: gcc-11
env:
build_dir: .build
# Bump this number to invalidate all caches globally.
CACHE_VERSION: 3
CACHE_VERSION: 1
MAIN_BRANCH_NAME: dev
steps:
- name: Checkout
@@ -257,86 +36,36 @@ jobs:
- name: Install build dependencies
run: |
apt-get update
apt-get install -y software-properties-common
add-apt-repository ppa:ubuntu-toolchain-r/test -y
apt-get update
apt-get install -y git python3 python-is-python3 pipx
pipx ensurepath
apt-get install -y cmake ninja-build ${{ matrix.cc }} ${{ matrix.cxx }} ccache
apt-get install -y perl # for openssl build
apt-get install -y libsqlite3-dev # for xahaud build
sudo apt-get update
sudo apt-get install -y ninja-build ${{ matrix.cc }} ${{ matrix.cxx }} ccache
# Install specific Conan version needed
pip install --upgrade "conan<2"
# Install the specific GCC version needed for Clang
if [ -n "${{ matrix.clang_gcc_toolchain }}" ]; then
echo "=== Installing GCC ${{ matrix.clang_gcc_toolchain }} for Clang ==="
apt-get install -y gcc-${{ matrix.clang_gcc_toolchain }} g++-${{ matrix.clang_gcc_toolchain }} libstdc++-${{ matrix.clang_gcc_toolchain }}-dev
- name: Configure ccache
uses: ./.github/actions/xahau-configure-ccache
with:
max_size: 2G
hash_dir: true
compiler_check: content
echo "=== GCC versions available after installation ==="
ls -la /usr/lib/gcc/x86_64-linux-gnu/ | grep -E "^d"
fi
# For Clang < 16 with --gcc-toolchain, hide newer GCC versions
# This is needed because --gcc-toolchain still picks the highest version
#
# THE GREAT GCC HIDING TRICK (for Clang < 16):
# Clang versions before 16 don't have --gcc-install-dir, only --gcc-toolchain
# which is deprecated and still uses discovery heuristics that ALWAYS pick
# the highest version number. So we play a sneaky game...
#
# We rename newer GCC versions to very low integers (1, 2, 3...) which makes
# Clang think they're ancient GCC versions. Since 11 > 3 > 2 > 1, Clang will
# pick GCC 11 over our renamed versions. It's dumb but it works!
#
# Example: GCC 12→1, GCC 13→2, GCC 14→3, so Clang picks 11 (highest number)
if [ -n "${{ matrix.clang_gcc_toolchain }}" ] && [ "${{ matrix.compiler_version }}" -lt "16" ]; then
echo "=== Hiding GCC versions newer than ${{ matrix.clang_gcc_toolchain }} for Clang < 16 ==="
target_version=${{ matrix.clang_gcc_toolchain }}
counter=1 # Start with 1 - these will be seen as "GCC version 1, 2, 3" etc
for dir in /usr/lib/gcc/x86_64-linux-gnu/*/; do
if [ -d "$dir" ]; then
version=$(basename "$dir")
# Check if version is numeric and greater than target
if [[ "$version" =~ ^[0-9]+$ ]] && [ "$version" -gt "$target_version" ]; then
echo "Hiding GCC $version -> renaming to $counter (will be seen as GCC version $counter)"
# Safety check: ensure target doesn't already exist
if [ ! -e "/usr/lib/gcc/x86_64-linux-gnu/$counter" ]; then
mv "$dir" "/usr/lib/gcc/x86_64-linux-gnu/$counter"
else
echo "ERROR: Cannot rename GCC $version - /usr/lib/gcc/x86_64-linux-gnu/$counter already exists"
exit 1
fi
counter=$((counter + 1))
fi
fi
done
fi
# Verify what Clang will use
if [ -n "${{ matrix.clang_gcc_toolchain }}" ]; then
echo "=== Verifying GCC toolchain selection ==="
echo "Available GCC versions:"
ls -la /usr/lib/gcc/x86_64-linux-gnu/ | grep -E "^d.*[0-9]+$" || true
echo ""
echo "Clang's detected GCC installation:"
${{ matrix.cxx }} -v -E -x c++ /dev/null -o /dev/null 2>&1 | grep "Found candidate GCC installation" || true
fi
# Install libc++ dev packages if using libc++ (not needed for libstdc++)
if [ "${{ matrix.stdlib }}" = "libcxx" ]; then
apt-get install -y libc++-${{ matrix.compiler_version }}-dev libc++abi-${{ matrix.compiler_version }}-dev
fi
# Install Conan 2
pipx install "conan>=2.0,<3"
echo "$HOME/.local/bin" >> $GITHUB_PATH
# Install gcovr for coverage jobs
if [ "${{ matrix.job_type }}" = "coverage" ]; then
pipx install "gcovr>=7,<9"
apt-get install -y curl lcov
- name: Configure Conan
run: |
conan profile new default --detect || true # Ignore error if profile exists
conan profile update settings.compiler.cppstd=20 default
conan profile update settings.compiler=${{ matrix.compiler }} default
conan profile update settings.compiler.libcxx=libstdc++11 default
conan profile update env.CC=/usr/bin/${{ matrix.cc }} default
conan profile update env.CXX=/usr/bin/${{ matrix.cxx }} default
conan profile update conf.tools.build:compiler_executables='{"c": "/usr/bin/${{ matrix.cc }}", "cpp": "/usr/bin/${{ matrix.cxx }}"}' default
# Set correct compiler version based on matrix.compiler
if [ "${{ matrix.compiler }}" = "gcc" ]; then
conan profile update settings.compiler.version=11 default
elif [ "${{ matrix.compiler }}" = "clang" ]; then
conan profile update settings.compiler.version=14 default
fi
# Display profile for verification
conan profile show default
- name: Check environment
run: |
@@ -347,24 +76,9 @@ jobs:
which ${{ matrix.cc }} && ${{ matrix.cc }} --version || echo "${{ matrix.cc }} not found"
which ${{ matrix.cxx }} && ${{ matrix.cxx }} --version || echo "${{ matrix.cxx }} not found"
which ccache && ccache --version || echo "ccache not found"
# Check gcovr for coverage jobs
if [ "${{ matrix.job_type }}" = "coverage" ]; then
which gcov && gcov --version || echo "gcov not found"
which gcovr && gcovr --version || echo "gcovr not found"
fi
echo "---- Full Environment ----"
env
- name: Get commit message
id: get-commit-message
uses: ./.github/actions/xahau-ga-get-commit-message
with:
event-name: ${{ github.event_name }}
head-commit-message: ${{ github.event.head_commit.message }}
pr-head-sha: ${{ github.event.pull_request.head.sha }}
- name: Install dependencies
uses: ./.github/actions/xahau-ga-dependencies
with:
@@ -373,15 +87,8 @@ jobs:
compiler-id: ${{ matrix.compiler_id }}
cache_version: ${{ env.CACHE_VERSION }}
main_branch: ${{ env.MAIN_BRANCH_NAME }}
compiler: ${{ matrix.compiler }}
compiler_version: ${{ matrix.compiler_version }}
cc: ${{ matrix.cc }}
cxx: ${{ matrix.cxx }}
stdlib: ${{ matrix.stdlib }}
gha_cache_enabled: 'false' # Disable caching for self hosted runner
- name: Build
if: matrix.job_type == 'build'
uses: ./.github/actions/xahau-ga-build
with:
generator: Ninja
@@ -392,31 +99,8 @@ jobs:
compiler-id: ${{ matrix.compiler_id }}
cache_version: ${{ env.CACHE_VERSION }}
main_branch: ${{ env.MAIN_BRANCH_NAME }}
stdlib: ${{ matrix.stdlib }}
clang_gcc_toolchain: ${{ matrix.clang_gcc_toolchain || '' }}
ccache_max_size: '100G'
- name: Build (Coverage)
if: matrix.job_type == 'coverage'
uses: ./.github/actions/xahau-ga-build
with:
generator: Ninja
configuration: ${{ matrix.configuration }}
build_dir: ${{ env.build_dir }}
cc: ${{ matrix.cc }}
cxx: ${{ matrix.cxx }}
gcov: ${{ matrix.gcov }}
compiler-id: ${{ matrix.compiler_id }}
cache_version: ${{ env.CACHE_VERSION }}
main_branch: ${{ env.MAIN_BRANCH_NAME }}
stdlib: ${{ matrix.stdlib }}
# Coverage builds are slower due to instrumentation; use fewer parallel jobs to avoid flakiness
cmake-args: '-Dcoverage=ON -Dcoverage_format=xml -Dcoverage_test_parallelism=$(($(nproc)/2)) -DCODE_COVERAGE_VERBOSE=ON -DCMAKE_CXX_FLAGS="-O0" -DCMAKE_C_FLAGS="-O0"'
cmake-target: 'coverage'
ccache_max_size: '100G'
- name: Set artifact name
if: matrix.job_type == 'build'
id: set-artifact-name
run: |
ARTIFACT_NAME="build-output-nix-${{ github.run_id }}-${{ matrix.compiler }}-${{ matrix.configuration }}"
@@ -429,7 +113,6 @@ jobs:
ls -la ${{ env.build_dir }} || echo "Build directory not found or empty"
- name: Run tests
if: matrix.job_type == 'build'
run: |
# Ensure the binary exists before trying to run
if [ -f "${{ env.build_dir }}/rippled" ]; then
@@ -437,43 +120,4 @@ jobs:
else
echo "Error: rippled executable not found in ${{ env.build_dir }}"
exit 1
fi
# Coverage-specific steps
- name: Move coverage report
if: matrix.job_type == 'coverage'
shell: bash
run: |
mv "${{ env.build_dir }}/coverage.xml" ./
- name: Archive coverage report
if: matrix.job_type == 'coverage'
uses: actions/upload-artifact@v4
with:
name: coverage.xml
path: coverage.xml
retention-days: 30
- name: Upload coverage report
if: matrix.job_type == 'coverage'
uses: codecov/codecov-action@v5
with:
files: coverage.xml
fail_ci_if_error: true
disable_search: true
verbose: true
plugins: noop
use_oidc: true
- name: Export server definitions
if: matrix.job_type == 'build' && matrix.compiler_id == 'gcc-13-libstdcxx'
run: |
${{ env.build_dir }}/rippled --definitions | python3 -m json.tool > server_definitions.json
- name: Upload server definitions
if: matrix.job_type == 'build' && matrix.compiler_id == 'gcc-13-libstdcxx'
uses: actions/upload-artifact@v7
with:
name: server-definitions
path: server_definitions.json
archive: false
fi

16
.gitignore vendored
View File

@@ -24,11 +24,6 @@ bin/project-cache.jam
build/docker
# Ignore release builder files
.env
release-build
cmake-*.tar.gz
# Ignore object files.
*.o
build
@@ -53,9 +48,6 @@ Builds/levelization/results/paths.txt
Builds/levelization/results/includes/
Builds/levelization/results/includedby/
# Python
__pycache__
# Ignore tmp directory.
tmp
@@ -79,7 +71,6 @@ docs/html_doc
# Xcode
.DS_Store
*/build/*
!/docs/build/
*.pbxuser
!default.pbxuser
*.mode1v3
@@ -127,12 +118,5 @@ bld.rippled/
generated
.vscode
# AI docs (local working documents)
.ai-docs/
# Local formal-methods workspace; kept as a separate repository and optionally
# symlinked here for navigation.
formal/lean/xahau_consensus
# Suggested in-tree build directory
/.build/

View File

@@ -1,6 +1,6 @@
# .pre-commit-config.yaml
repos:
- repo: https://github.com/pre-commit/mirrors-clang-format
rev: v18.1.3
rev: v10.0.1
hooks:
- id: clang-format

4
.testnet/.gitignore vendored
View File

@@ -1,4 +0,0 @@
output/
__pycache__/
scenarios/odd-cases/
scenarios/suite-experiments.yml

View File

@@ -1,29 +0,0 @@
"""Scenario: ConsensusEntropy amendment crashes non-supporting node.
Votes ConsensusEntropy accept on all nodes except n4, then waits for n4
to crash as the amendment activates without its support.
x-testnet run --scenario-script consensus_entropy_crash.py
"""
from helpers import CONSENSUS_ENTROPY_FEATURE
async def scenario(ctx, log):
await ctx.wait_for_ledger_close()
ctx.feature(CONSENSUS_ENTROPY_FEATURE, vetoed=False, exclude_nodes=[4])
log("Waiting for ConsensusEntropy to be voted for...")
await ctx.wait_for_feature(
CONSENSUS_ENTROPY_FEATURE,
check=lambda s: not s.get("vetoed"),
exclude_nodes=[4],
timeout=60,
)
log("Waiting for n4 to crash...")
op = await ctx.wait_for_nodes_down(nodes=[4], timeout=600)
ctx.assert_log("unsupported amendments activated", since=op.started, nodes=[4])
ctx.assert_exit_status(0, nodes=[4])
log("PASS: n4 shut down due to unsupported amendment")

View File

@@ -1,52 +0,0 @@
""":descr: entropy stays valid under transaction load"""
from __future__ import annotations
from helpers import require_entropy, get_entropy_tx, assert_valid_entropy
variants = [
{"label": "light", "min_txns": 5, "max_txns": 10},
{"label": "heavy", "min_txns": 50, "max_txns": 60},
{"label": "super_heavy", "min_txns": 90, "max_txns": 120},
]
async def scenario(ctx, log, *, min_txns=5, max_txns=10, **_):
await require_entropy(ctx, log)
gen = ctx.txn_generator(min_txns=min_txns, max_txns=max_txns)
await gen.start()
await gen.wait_until_ready()
log(f"Transaction generator ready ({min_txns}-{max_txns} txns/ledger)")
# Wait for pipeline warmup + a few txn-bearing ledgers.
await ctx.wait_for_ledgers(3, node_id=0, timeout=60)
start_seq = ctx.validated_ledger_index(0)
await ctx.wait_for_ledgers(10, node_id=0, timeout=120)
end_seq = ctx.validated_ledger_index(0)
log(f"Inspecting ledgers {start_seq + 1}{end_seq}")
digests = set()
total_user_txns = 0
for seq in range(start_seq + 1, end_seq + 1):
ce, user_txns = get_entropy_tx(ctx, seq)
digest, count = assert_valid_entropy(ce, seq, seen_digests=digests)
total_user_txns += len(user_txns)
log(
f" Ledger {seq}: EntropyCount={count} "
f"user_txns={len(user_txns)} Digest={digest[:16]}..."
)
await gen.stop()
log(
f"Verified {end_seq - start_seq} ledgers: {total_user_txns} user txns, "
f"all entropy valid and unique"
)
if total_user_txns == 0:
raise AssertionError("No user transactions were included in any ledger")
log("PASS")

View File

@@ -1,28 +0,0 @@
""":descr: healthy non-standalone testnet without UNLReport mints Tier 1 fallback"""
from __future__ import annotations
from helpers import require_entropy, get_entropy_tx, assert_consensus_fallback
async def scenario(ctx, log):
await require_entropy(ctx, log)
# Non-standalone nodes require a ledger-anchored UNLReport before assigning
# validator_quorum / participant_aligned labels. Without it, the RNG pipeline
# may still collect commits/reveals, but injection must remain Tier 1.
await ctx.wait_for_ledgers(3, node_id=0, timeout=60)
log("Pipeline warmed up without UNLReport")
start_seq = ctx.validated_ledger_index(0)
await ctx.wait_for_ledgers(5, node_id=0, timeout=90)
end_seq = ctx.validated_ledger_index(0)
log(f"Inspecting ledgers {start_seq + 1} -> {end_seq}")
for seq in range(start_seq + 1, end_seq + 1):
ce, _ = get_entropy_tx(ctx, seq)
digest, count = assert_consensus_fallback(ce, seq)
log(f" Ledger {seq}: EntropyCount={count} Digest={digest[:16]}...")
log(f"Verified {end_seq - start_seq} ledgers: all consensus_fallback")
log("PASS")

View File

@@ -1,160 +0,0 @@
""":descr: 5/6 validator_quorum, 4/6 participant_aligned (tier 2), recovery
Requires node_count: 6 (see suite.yml) — the smallest NON-degenerate Tier 2
size. At n=6: tier2 floor = 4, validator quorum = 5, validation quorum = 5. So
6/6, 5/6 present -> validator_quorum (EntropyTier=3)
4/6 present -> participant_aligned (EntropyTier=2, count 4) <-- the band
3/6 present -> consensus_fallback (EntropyTier=1)
n=5 has NO tier-2 band (tier2 == quorum == 4), which is why the existing
degradation smoke at 5 nodes only ever sees tier 3 / fallback.
KEY: the 4/6 window is BELOW the 80% validation quorum (5). The 4 survivors
keep CLOSING ledgers that carry tier-2 entropy, but those ledgers do NOT
validate until the network recovers — exactly the transition window Tier 2
serves. So validated_ledger_index() stalls; we instead inspect a surviving
node's CLOSED ledger (its LCL) directly, and cross-check the injection from the
cohort's logs.
"""
from __future__ import annotations
from helpers import (
require_entropy,
get_entropy_tx,
assert_participant_aligned,
assert_validator_quorum,
)
def _closed_entropy(result):
"""(seq, ConsensusEntropy tx) from a ctx.ledger('closed', transactions=True)
result, or (None, None) if the fetch returned no usable ledger.
Enforces the per-ledger invariant that an entropy-enabled closed ledger
carries EXACTLY ONE ConsensusEntropy pseudo-tx (mirroring get_entropy_tx):
a duplicate or missing injection raises here with a clear error instead of
being silently skipped and resurfacing later as a generic 'no tier-2 ledger'.
"""
if not result or not isinstance(result.get("ledger"), dict):
return None, None
led = result["ledger"]
try:
seq = int(led.get("ledger_index"))
except (TypeError, ValueError):
return None, None
ce = [
t
for t in led.get("transactions", [])
if isinstance(t, dict) and t.get("TransactionType") == "ConsensusEntropy"
]
if len(ce) != 1:
raise AssertionError(
f"Closed ledger {seq}: expected 1 ConsensusEntropy txn, got {len(ce)}"
)
return seq, ce[0]
async def scenario(ctx, log):
await require_entropy(ctx, log)
# Baseline: healthy 6/6 produces validator_quorum entropy.
await ctx.wait_for_ledgers(1, node_id=0, timeout=30)
# --- 5/6: settles back to validator_quorum (5 present >= quorum 5) ---
val_before_drop = ctx.validated_ledger_index(0)
ctx.stop_node(5)
await ctx.wait_for_nodes_down(nodes=[5], timeout=30)
# Settle a few ledgers past the membership change. The ledger right at a
# validator drop can carry a transient consensus_fallback (tier 1, count 0,
# deterministic and by design) before the commit/reveal pipeline re-primes,
# so we do NOT assume any single post-drop ledger is already tier 3.
await ctx.wait_for_ledgers(4, node_id=0, timeout=90)
# 5/6 is at/above the 80% quorum (5), so steady state is validator_quorum.
# Scan the post-drop validated ledgers (all carry the 5-node cohort, so a
# tier-3 here has count == 5) and require at least one clean validator_quorum
# — EntropyTier=3, count >= quorum, non-zero digest — tolerating the
# transition fallback instead of depending on where the tip happened to land.
val_5of6 = ctx.validated_ledger_index(0)
t3_seq = None
for seq in range(val_5of6, val_before_drop, -1):
ce, _ = get_entropy_tx(ctx, seq)
tier = ce.get("EntropyTier")
log(f" 5/6 ledger {seq}: tier={tier} count={ce.get('EntropyCount')}")
if tier == 3:
assert_validator_quorum(ce, seq, min_count=5)
t3_seq = seq
break
if t3_seq is None:
raise AssertionError(
f"5/6: no validator_quorum (tier 3) entropy in post-drop validated "
f"ledgers {val_before_drop + 1}..{val_5of6}"
)
log(f"5/6: validator_quorum at validated seq {t3_seq}")
#@@start test-participant-aligned-window
# --- 4/6: participant_aligned (Tier 2) degraded window ---
ctx.stop_node(4)
await ctx.wait_for_nodes_down(nodes=[4], timeout=30)
# ~12s window: confirm tier-2 INJECTION from the cohort's logs, and that the
# round is NOT the impossible/fallback path (which is what distinguishes the
# tier-2 band from the tier-1 fallback regime).
op = await ctx.sleep(12, name="tier2_window")
selected_t2 = ctx.search_logs(
r"RNG: entropy selected seq=\d+ tier=2 count=4",
within=op.window,
nodes=[0, 1, 2, 3],
)
log(f"4/6: 'entropy selected tier=2 count=4' logs: {selected_t2.count}")
if selected_t2.count == 0:
raise AssertionError(
"4/6 window injected no participant_aligned (tier 2) entropy: no "
"'RNG: entropy selected ... tier=2 count=4' on the surviving cohort"
)
ctx.assert_not_log(
r"reason=impossible-entropy-gate", within=op.window, nodes=[0, 1, 2, 3]
)
# Verify the on-ledger EntropyTier=2 DIRECTLY: validation is stalled (4 < 5),
# so sample the surviving cohort's CLOSED ledger (its LCL — built but not yet
# validated). At least one must be participant_aligned with EntropyCount=4.
tier2_on_ledger = 0
last_seq = None
for _ in range(5):
seq, ce = _closed_entropy(
ctx.ledger("closed", transactions=True, node_id=0)
)
if ce is not None and seq is not None and seq != last_seq:
last_seq = seq
tier = ce.get("EntropyTier")
count = ce.get("EntropyCount", -1)
log(f" closed ledger {seq}: tier={tier} count={count}")
if tier == 2:
assert_participant_aligned(ce, seq, expected_count=4)
tier2_on_ledger += 1
await ctx.sleep(3)
if tier2_on_ledger == 0:
raise AssertionError(
"no closed participant_aligned (tier 2) ledger observed during the "
"4/6 window (tier 2 was injected per logs, but not seen on a closed "
"ledger)"
)
log(f"4/6: {tier2_on_ledger} participant_aligned closed ledger(s) verified")
#@@end test-participant-aligned-window
# --- Recovery: liveness — validation resumes once quorum is restored ---
ctx.start_node(4)
ctx.start_node(5)
await ctx.wait_for_ledgers(1, node_id=0, timeout=120)
val_recovered = ctx.validated_ledger_index(0)
if not val_recovered or val_recovered <= val_5of6:
raise AssertionError(
f"Validated ledger did not advance after recovery "
f"({val_5of6} -> {val_recovered})"
)
log(f"Recovered: validated seq {val_5of6} -> {val_recovered}")
log("PASS")

View File

@@ -1,148 +0,0 @@
""":descr: 4/5 liveness, 3/5 fallback-entropy (consensus_fallback), recovery"""
from __future__ import annotations
from helpers import ZERO_DIGEST, require_entropy, get_entropy_tx, entropy_fields
async def scenario(ctx, log):
await require_entropy(ctx, log)
# Baseline: wait 1 ledger to confirm network is healthy.
await ctx.wait_for_ledgers(1, node_id=0, timeout=30)
# --- 4/5 liveness ---
ctx.stop_node(4)
await ctx.wait_for_nodes_down(nodes=[4], timeout=30)
await ctx.wait_for_ledgers(1, node_id=0, timeout=30)
log("4/5: liveness OK")
# Snapshot validated seq before dropping to 3/5.
val_before = ctx.validated_ledger_index(0)
# --- 3/5 degraded window ---
ctx.stop_node(3)
await ctx.wait_for_nodes_down(nodes=[3], timeout=30)
# 10s ≈ 3 rounds at 3s cadence.
await ctx.sleep(10)
val_after = ctx.validated_ledger_index(0)
log(f"3/5: validated ledger {val_before}{val_after}")
# Accepted/built ledgers may still later appear as validated once the full
# network rejoins. For ConsensusEntropy the key invariant is that every
# ledger created during this sub-quorum window carries FALLBACK entropy
# (consensus_fallback: non-zero consensus-bound digest, count 0) — never
# validator-tier entropy.
degraded_fallback = 0
degraded_end = val_after or val_before
if val_before and degraded_end and degraded_end > val_before:
for seq in range(val_before + 1, degraded_end + 1):
ce, _ = get_entropy_tx(ctx, seq)
digest, entropy_count, is_fallback = entropy_fields(ce)
tier = ce.get("EntropyTier")
# consensus_fallback (EntropyTier=1): explicit tier, count 0,
# deterministic NON-zero digest.
if tier != 1:
raise AssertionError(
f"Ledger {seq}: expected EntropyTier==1 "
f"(consensus_fallback) during 3/5 window, got {tier} "
f"(EntropyCount={entropy_count})"
)
if entropy_count != 0:
raise AssertionError(
f"Ledger {seq}: fallback EntropyCount must be 0, got "
f"{entropy_count}"
)
if not digest or digest == ZERO_DIGEST:
raise AssertionError(
f"Ledger {seq}: fallback digest must be non-zero "
f"(consensus_fallback), got {digest[:16]}..."
)
assert is_fallback # tier==1 implies fallback
degraded_fallback += 1
log(
f" Degraded ledger {seq}: EntropyCount={entropy_count} "
f"FALLBACK"
)
log(f"3/5 entropy summary: {degraded_fallback} fallback")
# Log checks tied to current transition mechanics:
# - commit-set SHAMap publication is the observable output of entering the
# commit sidecar phase
# - ConvergingCommit transition is the gateway out of seq=0-only behavior
# - reason=impossible-entropy-gate is the explicit degraded-window fallback path
ctx.log_level("LedgerConsensus", "trace")
ctx.log_level("ConsensusExtensions", "trace")
op = await ctx.sleep(6, name="stall_window")
ctx.assert_not_log(
r"RNG: transitioned to ConvergingCommit", within=op.window, nodes=[0, 1, 2]
)
ctx.assert_not_log(
r"RNG: built commitSet SHAMap", within=op.window, nodes=[0, 1, 2]
)
gate_blocked = ctx.search_logs(
r"STALLDIAG: establish gate blocked reason=(pause|no-tx-consensus)",
within=op.window,
nodes=[0, 1, 2],
)
log(f"3/5: establish gate-blocked logs in 6s: {gate_blocked.count}")
impossible = ctx.search_logs(
r"RNG: skipping commit wait reason=impossible-entropy-gate",
within=op.window,
nodes=[0, 1, 2],
)
log(f"3/5: RNG impossible-entropy-gate skips in 6s: {impossible.count}")
# --- Recovery: restart nodes, verify ledger advancement ---
ctx.start_node(3)
ctx.start_node(4)
await ctx.wait_for_ledgers(1, node_id=0, timeout=120)
val_recovered = ctx.validated_ledger_index(0)
pre_recovery = max(v for v in [val_before, val_after] if v is not None)
log(f"Recovered: validated seq {pre_recovery}{val_recovered}")
if not val_recovered or val_recovered <= pre_recovery:
raise AssertionError(
f"Validated ledger did not advance after recovery "
f"({pre_recovery}{val_recovered})"
)
# Inspect post-recovery ledgers separately from the degraded window above.
# Once the network is back at quorum, validator-tier entropy is expected
# again (transitional fallback ledgers are fine) and must be quorum-met.
fallback_count = 0
validator_count = 0
for seq in range(pre_recovery + 1, val_recovered + 1):
ce, _ = get_entropy_tx(ctx, seq)
digest, entropy_count, is_fallback = entropy_fields(ce)
if is_fallback:
fallback_count += 1
else:
validator_count += 1
if entropy_count < 4:
raise AssertionError(
f"Ledger {seq}: validator entropy with sub-quorum "
f"EntropyCount={entropy_count} (need >= 4)"
)
log(
f" Ledger {seq}: EntropyCount={entropy_count} "
f"{'FALLBACK' if is_fallback else 'VALIDATOR'}"
)
log(
f"Entropy summary: {fallback_count} fallback, "
f"{validator_count} validator"
)
log("PASS")

View File

@@ -1,44 +0,0 @@
""":descr: drop 2 nodes (3/5 stall), restart both, verify recovery"""
from __future__ import annotations
from helpers import require_entropy
async def scenario(ctx, log):
await require_entropy(ctx, log)
await ctx.wait_for_ledgers(1, node_id=0, timeout=60)
log("Baseline OK")
# Drop 2 nodes → validation stall.
ctx.stop_node(3)
ctx.stop_node(4)
await ctx.wait_for_nodes_down(nodes=[3, 4], timeout=30)
info = ctx.rpc.server_info(node_id=0)
val_before = info.get("info", {}).get("validated_ledger", {}).get("seq", 0)
log(f"Stalled at validated seq {val_before}")
# Let it sit for a few rounds in degraded state.
await ctx.sleep(6)
# Bring both nodes back.
ctx.start_node(3)
ctx.start_node(4)
log("Restarted n3 and n4, waiting for recovery...")
# Recovery: wait for ANY validated ledger advance on n0.
await ctx.wait_for_ledger_close(node_id=0, timeout=60)
info = ctx.rpc.server_info(node_id=0)
val_after = info.get("info", {}).get("validated_ledger", {}).get("seq", 0)
log(f"Recovered: validated seq {val_before}{val_after}")
if val_after <= val_before:
raise AssertionError(
f"Validated ledger did not advance after recovery "
f"({val_before}{val_after})"
)
log("PASS")

View File

@@ -1,27 +0,0 @@
""":descr: all 5 nodes healthy, every ledger has valid unique quorum-met entropy"""
from __future__ import annotations
from helpers import require_entropy, get_entropy_tx, assert_valid_entropy
async def scenario(ctx, log):
await require_entropy(ctx, log)
# Wait for RNG pipeline to warm up past bootstrap skip.
await ctx.wait_for_ledgers(3, node_id=0, timeout=60)
log("Pipeline warmed up")
start_seq = ctx.validated_ledger_index(0)
await ctx.wait_for_ledgers(10, node_id=0, timeout=120)
end_seq = ctx.validated_ledger_index(0)
log(f"Inspecting ledgers {start_seq + 1}{end_seq}")
digests = set()
for seq in range(start_seq + 1, end_seq + 1):
ce, _ = get_entropy_tx(ctx, seq)
digest, count = assert_valid_entropy(ce, seq, seen_digests=digests)
log(f" Ledger {seq}: EntropyCount={count} Digest={digest[:16]}...")
log(f"Verified {end_seq - start_seq} ledgers: all quorum entropy, all unique")
log("PASS")

View File

@@ -1,104 +0,0 @@
defaults:
network:
node_count: 5
launcher: tmux
find_ports: true
slave_delay: 0.2
features:
- ConsensusEntropy
- Export
track_features:
- ConsensusEntropy
- Export
unl_report: true
log_levels:
TxQ: info
Protocol: debug
Peer: debug
LedgerConsensus: debug
ConsensusExtensions: debug
NetworkOPs: info
env:
XAHAU_RESOURCE_PER_PORT: "1"
rc:
- rng_poll_ms=333
tests:
# --- CE + Export (80% quorum, SHAMap convergence) ---
- name: steady_state_export_ce
script: .testnet/scenarios/export/steady_state_export.py
- name: retriable_export_ce
script: .testnet/scenarios/export/retriable_export.py
- name: export_degradation_ce
script: .testnet/scenarios/export/export_degradation.py
network:
rc:
- rng_poll_ms=333
- n3:no_export_sig=true
- n4:no_export_sig=true
- name: export_without_unl_report
script: .testnet/scenarios/export/export_without_unl_report.py
network:
features:
- Export
track_features:
- Export
unl_report: false
- name: export_no_veto_missing_observation
script: .testnet/scenarios/export/export_no_veto_missing_observation.py
network:
rc:
- rng_poll_ms=333
- n4:no_export_sig_hash=true
# CE + Export: 1 node suppressed, 4/5 = 80% quorum, should succeed
- name: export_ce_one_node_down
script: .testnet/scenarios/export/export_quorum.py
params:
expect_success: true
network:
rc:
- rng_poll_ms=333
- n4:no_export_sig=true
# --- Export only, no CE (80% active-view quorum) ---
- name: export_only_all_up
script: .testnet/scenarios/export/export_quorum.py
params:
expect_success: true
network:
features:
- Export
track_features:
- Export
- name: export_only_one_node_down
script: .testnet/scenarios/export/export_quorum.py
params:
expect_success: true
network:
features:
- Export
track_features:
- Export
rc:
- rng_poll_ms=333
- n4:no_export_sig=true
- name: export_only_two_nodes_down
script: .testnet/scenarios/export/export_quorum.py
params:
expect_success: false
network:
features:
- Export
track_features:
- Export
rc:
- rng_poll_ms=333
- n3:no_export_sig=true
- n4:no_export_sig=true

View File

@@ -1,123 +0,0 @@
""":descr: Submit ttEXPORT with 2 nodes suppressing export sigs, verify it
retries via terRETRY_EXPORT until LLS expiry (insufficient signatures).
Nodes 3 and 4 have runtime_config no_export_sig=true, so only 3/5 nodes
provide export signatures. With 80% quorum = ceil(5*0.8) = 4 required,
the export cannot reach quorum and should expire via tecEXPORT_EXPIRED.
Flow:
1. Fund alice and bob
2. alice submits ttEXPORT with tight LLS
3. Export retries (only 3/5 sigs available, need 4)
4. Verify export expires with tecEXPORT_EXPIRED
5. Verify subsequent payment still works (sequence not permanently blocked)
"""
from __future__ import annotations
from export_helpers import require_export, assert_shadow_ticket
async def scenario(ctx, log):
await require_export(ctx, log)
# --- Setup ---
await ctx.fund_accounts({"alice": 10000, "bob": 1000})
log("Accounts funded")
alice = ctx.account("alice")
bob = ctx.account("bob")
current_seq = ctx.validated_ledger_index(0)
log(f"Current ledger: {current_seq}")
log("Nodes 3,4 have runtime_config no_export_sig=true (3/5 sigs, need 4)")
#@@start test-export-below-quorum-expiry
# --- Submit ttEXPORT (should retry then expire -- only 3/5 sigs) ---
export_start = ctx.mark("export-degradation-submit-start")
result = await ctx.submit_and_wait(
{
"TransactionType": "Export",
"LastLedgerSequence": current_seq + 8,
"Fee": "1000000",
"ExportedTxn": {
"TransactionType": "Payment",
"Account": alice.address,
"Destination": bob.address,
"Amount": "1000000",
"Fee": "10",
"Sequence": 0,
"TicketSequence": 1,
"FirstLedgerSequence": current_seq + 1,
"LastLedgerSequence": current_seq + 6,
"Flags": 2147483648,
"SigningPubKey": "",
},
},
alice.wallet,
timeout=60,
)
export_end = ctx.mark("export-degradation-submit-end")
final_seq = ctx.validated_ledger_index(0)
engine_result = result.get("engine_result", "")
log(f"Export completed at ledger {final_seq}, result: {engine_result}")
# With only 3/5 sigs and 80% quorum (4 required), export MUST fail
if engine_result == "tesSUCCESS":
raise AssertionError(
"Export should NOT have succeeded with only 3/5 sigs "
"(need 4 for 80% quorum) -- check runtime_config no_export_sig"
)
# Should be tecEXPORT_EXPIRED (LLS reached without quorum). Be exact here:
# any other non-success means the retry/expiry boundary regressed.
if engine_result != "tecEXPORT_EXPIRED":
raise AssertionError(
f"Expected tecEXPORT_EXPIRED below quorum, got {engine_result}"
)
log(f"Export failed as expected ({engine_result})")
retry_logs = ctx.assert_log(
r"Export: insufficient signatures .*result=terRETRY_EXPORT",
since=export_start,
until=export_end,
)
log(f"Export insufficient-signature retries: {retry_logs.count}")
expired_logs = ctx.assert_log(
r"Export: last ledger expired .*result=tecEXPORT_EXPIRED",
since=export_start,
until=export_end,
)
log(f"Export LLS expiry logs: {expired_logs.count}")
# No shadow ticket should exist (export never reached quorum)
assert_shadow_ticket(ctx, alice.address, log, expect_exists=False)
#@@end test-export-below-quorum-expiry
# --- Verify subsequent payment works regardless ---
log("Submitting payment from alice to bob...")
pay_result = await ctx.submit_and_wait(
{
"TransactionType": "Payment",
"Destination": bob.address,
"Amount": "1000000",
"Fee": "12",
},
alice.wallet,
timeout=30,
)
pay_engine = pay_result.get("engine_result", "")
log(f"Payment result: {pay_engine}")
if pay_engine != "tesSUCCESS":
raise AssertionError(
f"Payment failed after expired export: {pay_engine} "
f"-- sequence may be blocked"
)
log("Payment succeeded -- account not permanently blocked")
log("PASS")

View File

@@ -1,181 +0,0 @@
"""Shared helpers for Export scenario tests."""
from __future__ import annotations
from xahaud_scripts.testnet.config import _unl_report_index, feature_name_to_hash
async def require_export(
ctx, log, *, require_unl_report=True, require_runtime_config=True
):
"""Wait for first ledger and assert Export is enabled.
Network-mode Export success requires a parent-ledger UNLReport-backed
active validator view. Most export scenarios seed that report in genesis;
assert it here so a success-path test cannot accidentally pass setup
without the condition Export::doApply requires. The no-UNLReport retry
scenario opts out deliberately.
The tracked export suite also uses XAHAUD_RUNTIME_TEST_CONFIG for polling
and fault-injection knobs. Default binaries reject the runtime_config RPC,
so check it up front rather than silently running without those knobs.
"""
await ctx.wait_for_ledger_close(timeout=120)
if require_runtime_config:
result = ctx.rpc.runtime_config(0)
if not result or result.get("error"):
raise AssertionError(
"Export suite requires a binary built with "
"xahaud_runtime_test_config=ON; runtime_config RPC returned "
f"{result}"
)
log("RuntimeConfig RPC active")
feature = ctx.feature_check(feature_name_to_hash("Export"), node_id=0)
if not feature or not feature.get("enabled", False):
raise AssertionError(f"Export not enabled: {feature}")
log("Export enabled")
if require_unl_report:
result = ctx.rpc.ledger_entry(0, _unl_report_index())
node = (result or {}).get("node", {})
active = node.get("ActiveValidators", [])
if node.get("LedgerEntryType") != "UNLReport" or not active:
raise AssertionError(
"Export success scenario requires a ledger UNLReport with "
f"ActiveValidators, got: {result}"
)
log(f"UNLReport active validators: {len(active)}")
def find_export_txns(ctx, seq):
"""Find Export transactions in a ledger.
Returns list of Export transaction dicts.
"""
result = ctx.ledger(seq, transactions=True)
if not result:
return []
txns = result.get("ledger", {}).get("transactions", [])
return [tx for tx in txns if tx.get("TransactionType") == "Export"]
def dst_param(address):
"""Encode an address as a HookParameter entry for the DST param."""
from xrpl.core.addresscodec import decode_classic_address
dst_hex = decode_classic_address(address).hex().upper()
return {
"HookParameter": {
"HookParameterName": "445354", # "DST"
"HookParameterValue": dst_hex,
}
}
def assert_hook_accepted(meta, log, *, expected_emits=1):
"""Assert hook executed with ACCEPT and the expected emit count.
Checks sfHookExecutions in transaction metadata.
Returns the hook execution entry for further inspection.
"""
hook_execs = meta.get("HookExecutions", [])
if not hook_execs:
raise AssertionError("No HookExecutions in metadata")
exec_entry = hook_execs[0].get("HookExecution", {})
hook_result = exec_entry.get("HookResult", -1)
emit_count = exec_entry.get("HookEmitCount", -1)
return_code = exec_entry.get("HookReturnCode", "")
log(f" HookResult={hook_result} EmitCount={emit_count} ReturnCode={return_code}")
# HookResult 3 = ExitType::ACCEPT
if hook_result != 3:
raise AssertionError(
f"Hook did not ACCEPT: HookResult={hook_result} "
f"ReturnCode={return_code}"
)
if emit_count != expected_emits:
raise AssertionError(
f"Expected {expected_emits} emits, got {emit_count}"
)
# ReturnCode 0 = success; non-zero = ASSERT line number in hook
if return_code and str(return_code) != "0":
raise AssertionError(
f"Hook returned error code {return_code} "
f"(likely ASSERT failure at that line)"
)
return exec_entry
def assert_export_result(meta, log, *, require_signers=True):
"""Assert ExportResult is present and well-formed in metadata.
Returns the ExportResult dict.
"""
export_result = meta.get("ExportResult", {})
if not export_result:
raise AssertionError("ExportResult not found in metadata")
# Must have LedgerSequence and TransactionHash
if "LedgerSequence" not in export_result:
raise AssertionError("ExportResult missing LedgerSequence")
if "TransactionHash" not in export_result:
raise AssertionError("ExportResult missing TransactionHash")
# Must have the inner ExportedTxn object
inner = export_result.get("ExportedTxn", {})
if not inner:
raise AssertionError("ExportResult missing ExportedTxn (multisigned blob)")
log(f" ExportResult: seq={export_result['LedgerSequence']} "
f"hash={export_result['TransactionHash'][:16]}...")
# Inner tx should have Account, Destination, TransactionType
if "Account" not in inner:
raise AssertionError("ExportedTxn missing Account")
if "TransactionType" not in inner:
raise AssertionError("ExportedTxn missing TransactionType")
# Should have empty SigningPubKey (multisigned)
if inner.get("SigningPubKey", "NOT_EMPTY") != "":
raise AssertionError(
f"ExportedTxn SigningPubKey should be empty, "
f"got '{inner.get('SigningPubKey')}'"
)
if require_signers:
signers = inner.get("Signers", [])
if not signers:
raise AssertionError("ExportedTxn has no Signers (multisig not applied)")
log(f" Signers: {len(signers)} validator(s)")
return export_result
def assert_shadow_ticket(ctx, account_address, log, *, expect_exists=True):
"""Assert shadow ticket exists (or doesn't) for the account."""
obj_result = ctx.rpc.request(
0, "account_objects", {"account": account_address}
)
all_objects = (obj_result or {}).get("account_objects", [])
shadow_tickets = [
obj for obj in all_objects
if obj.get("LedgerEntryType") == "ShadowTicket"
]
log(f" Shadow tickets: {len(shadow_tickets)}")
if expect_exists and not shadow_tickets:
raise AssertionError("Expected shadow ticket but none found")
if not expect_exists and shadow_tickets:
raise AssertionError(
f"Expected no shadow tickets but found {len(shadow_tickets)}"
)
return shadow_tickets

View File

@@ -1,87 +0,0 @@
""":descr: Export succeeds when quorum sidecar material exists but one active
validator withholds exportSigSetHash observation.
Node 4 has runtime_config no_export_sig_hash=true. It still attaches export
signatures, but it does not publish its exportSigSetHash in proposals. The
remaining 4/5 active validators can still align on the same export sidecar
hash, so the round must not retry/expire just because fullObservation is false.
"""
from __future__ import annotations
from export_helpers import (
require_export,
assert_export_result,
assert_shadow_ticket,
)
async def scenario(ctx, log):
await require_export(ctx, log)
await ctx.fund_accounts({"alice": 10000, "bob": 1000})
log("Accounts funded")
alice = ctx.account("alice")
bob = ctx.account("bob")
current_seq = ctx.validated_ledger_index(0)
log(f"Current ledger: {current_seq}")
log("Node 4 withholds exportSigSetHash but still attaches export signatures")
export_start = ctx.mark("export-no-veto-submit-start")
result = await ctx.submit_and_wait(
{
"TransactionType": "Export",
"LastLedgerSequence": current_seq + 10,
"Fee": "1000000",
"ExportedTxn": {
"TransactionType": "Payment",
"Account": alice.address,
"Destination": bob.address,
"Amount": "1000000",
"Fee": "10",
"Sequence": 0,
"TicketSequence": 1,
"FirstLedgerSequence": current_seq + 1,
"LastLedgerSequence": current_seq + 8,
"Flags": 2147483648,
"SigningPubKey": "",
},
},
alice.wallet,
timeout=60,
)
export_end = ctx.mark("export-no-veto-submit-end")
final_seq = ctx.validated_ledger_index(0)
engine_result = result.get("engine_result", "")
meta = result.get("meta", {})
log(f"Export completed at ledger {final_seq}, result: {engine_result}")
if engine_result != "tesSUCCESS":
raise AssertionError(f"Expected tesSUCCESS, got {engine_result}")
export_result = assert_export_result(meta, log, require_signers=True)
signers = export_result.get("ExportedTxn", {}).get("Signers", [])
if len(signers) < 4:
raise AssertionError(f"Expected at least 4 signers, got {len(signers)}")
log(f"Export signer count: {len(signers)}")
no_veto_logs = ctx.assert_log(
r"Export: missing exportSigSetHash observation ignored",
since=export_start,
until=export_end,
)
log(f"Export no-veto missing-observation logs: {no_veto_logs.count}")
withhold_logs = ctx.assert_log(
r"Export: withholding exportSigSetHash",
since=export_start,
until=export_end,
)
log(f"Export sidecar hash withholding logs: {withhold_logs.count}")
assert_shadow_ticket(ctx, alice.address, log, expect_exists=True)
log("PASS")

View File

@@ -1,117 +0,0 @@
""":descr: Test Export quorum behavior. When enough active validators sign,
the export should succeed whether or not CE is enabled. When fewer than the
active-view quorum sign, the export should expire.
Parameterized via `expect_success` kwarg from suite.yml.
Flow:
1. Fund alice and bob
2. alice submits ttEXPORT
3. Verify result matches expectation (tesSUCCESS or tecEXPORT_EXPIRED)
4. Verify ExportResult + shadow ticket on success, absence on failure
5. Verify subsequent payment works regardless
"""
from __future__ import annotations
from export_helpers import (
require_export,
assert_export_result,
assert_shadow_ticket,
)
async def scenario(ctx, log, expect_success=True):
await require_export(ctx, log)
# --- Setup ---
await ctx.fund_accounts({"alice": 10000, "bob": 1000})
log("Accounts funded")
alice = ctx.account("alice")
bob = ctx.account("bob")
current_seq = ctx.validated_ledger_index(0)
log(f"Current ledger: {current_seq}")
outcome = "success" if expect_success else "failure (below quorum)"
log(f"Expecting export {outcome}")
# --- Submit ttEXPORT ---
result = await ctx.submit_and_wait(
{
"TransactionType": "Export",
"LastLedgerSequence": current_seq + 10,
"Fee": "1000000",
"ExportedTxn": {
"TransactionType": "Payment",
"Account": alice.address,
"Destination": bob.address,
"Amount": "1000000",
"Fee": "10",
"Sequence": 0,
"TicketSequence": 1,
"FirstLedgerSequence": current_seq + 1,
"LastLedgerSequence": current_seq + 8,
"Flags": 2147483648,
"SigningPubKey": "",
},
},
alice.wallet,
timeout=60,
)
final_seq = ctx.validated_ledger_index(0)
engine_result = result.get("engine_result", "")
meta = result.get("meta", {})
log(f"Export at ledger {final_seq}, result: {engine_result}")
if expect_success:
if engine_result != "tesSUCCESS":
raise AssertionError(
f"Expected tesSUCCESS, got {engine_result}"
)
# Assert ExportResult is well-formed with signers
assert_export_result(meta, log, require_signers=True)
# Assert shadow ticket was created
assert_shadow_ticket(ctx, alice.address, log, expect_exists=True)
log("Export succeeded as expected (active-view quorum reached)")
else:
if engine_result == "tesSUCCESS":
raise AssertionError(
"Export should NOT have succeeded below active-view quorum"
)
if engine_result != "tecEXPORT_EXPIRED":
raise AssertionError(
"Expected tecEXPORT_EXPIRED below active-view quorum, "
f"got {engine_result}"
)
log(f"Export failed as expected ({engine_result})")
# No shadow ticket should exist
assert_shadow_ticket(ctx, alice.address, log, expect_exists=False)
# --- Verify subsequent payment works ---
log("Submitting payment from alice to bob...")
pay_result = await ctx.submit_and_wait(
{
"TransactionType": "Payment",
"Destination": bob.address,
"Amount": "1000000",
"Fee": "12",
},
alice.wallet,
timeout=30,
)
pay_engine = pay_result.get("engine_result", "")
log(f"Payment result: {pay_engine}")
if pay_engine != "tesSUCCESS":
raise AssertionError(f"Payment failed: {pay_engine}")
log("Payment succeeded -- account not blocked")
log("PASS")

View File

@@ -1,92 +0,0 @@
""":descr: Export retries/expires without a ledger-anchored UNLReport view.
All validators may sign, but network-mode Export must not assemble quorum
material from a node-local trusted-config view. Without UNLReport, the export
should retry until LastLedgerSequence and expire without creating a shadow
ticket.
"""
from __future__ import annotations
from export_helpers import require_export, assert_shadow_ticket
async def scenario(ctx, log):
await require_export(ctx, log, require_unl_report=False)
await ctx.fund_accounts({"alice": 10000, "bob": 1000})
log("Accounts funded")
alice = ctx.account("alice")
bob = ctx.account("bob")
current_seq = ctx.validated_ledger_index(0)
log(f"Current ledger: {current_seq}")
log("UNLReport intentionally absent; export must not use local config view")
export_start = ctx.mark("export-without-unlreport-submit-start")
result = await ctx.submit_and_wait(
{
"TransactionType": "Export",
"LastLedgerSequence": current_seq + 8,
"Fee": "1000000",
"ExportedTxn": {
"TransactionType": "Payment",
"Account": alice.address,
"Destination": bob.address,
"Amount": "1000000",
"Fee": "10",
"Sequence": 0,
"TicketSequence": 1,
"FirstLedgerSequence": current_seq + 1,
"LastLedgerSequence": current_seq + 6,
"Flags": 2147483648,
"SigningPubKey": "",
},
},
alice.wallet,
timeout=60,
)
export_end = ctx.mark("export-without-unlreport-submit-end")
final_seq = ctx.validated_ledger_index(0)
engine_result = result.get("engine_result", "")
log(f"Export completed at ledger {final_seq}, result: {engine_result}")
if engine_result == "tesSUCCESS":
raise AssertionError(
"Export should not succeed without a ledger-anchored UNLReport view"
)
# Be exact: without a UNLReport view the export should retry until LLS and
# expire, not fail by some unrelated terminal code.
if engine_result != "tecEXPORT_EXPIRED":
raise AssertionError(
"Expected tecEXPORT_EXPIRED without UNLReport view, "
f"got {engine_result}"
)
warning_logs = ctx.assert_log(
r"Export: retrying without ledger-anchored validator view",
since=export_start,
until=export_end,
)
log(f"Export no-UNLReport retry warnings: {warning_logs.count}")
retry_logs = ctx.assert_log(
r"Export: insufficient signatures .*result=terRETRY_EXPORT",
since=export_start,
until=export_end,
)
log(f"Export retry logs: {retry_logs.count}")
expired_logs = ctx.assert_log(
r"Export: last ledger expired .*result=tecEXPORT_EXPIRED",
since=export_start,
until=export_end,
)
log(f"Export expiry logs: {expired_logs.count}")
assert_shadow_ticket(ctx, alice.address, log, expect_exists=False)
log("PASS")

View File

@@ -1,94 +0,0 @@
""":descr: Submit ttEXPORT directly (no hook), verify it succeeds with
ExportResult in metadata. Then submit a payment from the same account
to verify sequence handling doesn't block subsequent transactions.
Flow:
1. Fund alice and bob
2. alice submits ttEXPORT with inner payment -> tesSUCCESS (provisional)
3. Validators attach sigs via proposals -> quorum -> ExportResult in metadata
4. alice submits a Payment to bob -> should succeed (sequence not blocked)
"""
from __future__ import annotations
from export_helpers import require_export, assert_export_result, assert_shadow_ticket
async def scenario(ctx, log):
await require_export(ctx, log)
# --- Setup ---
await ctx.fund_accounts({"alice": 10000, "bob": 1000})
log("Accounts funded")
alice = ctx.account("alice")
bob = ctx.account("bob")
current_seq = ctx.validated_ledger_index(0)
log(f"Current ledger: {current_seq}")
# --- 1. Submit ttEXPORT ---
result = await ctx.submit_and_wait(
{
"TransactionType": "Export",
"LastLedgerSequence": current_seq + 15,
"Fee": "1000000",
"ExportedTxn": {
"TransactionType": "Payment",
"Account": alice.address,
"Destination": bob.address,
"Amount": "1000000",
"Fee": "10",
"Sequence": 0,
"TicketSequence": 1,
"FirstLedgerSequence": current_seq + 1,
"LastLedgerSequence": current_seq + 10,
"Flags": 2147483648,
"SigningPubKey": "",
},
},
alice.wallet,
timeout=60,
)
export_seq = ctx.validated_ledger_index(0)
engine_result = result.get("engine_result", "")
log(f"Export completed at ledger {export_seq}, result: {engine_result}")
if engine_result != "tesSUCCESS":
raise AssertionError(
f"Expected tesSUCCESS for export, got {engine_result}"
)
# Assert ExportResult is well-formed with signers
meta = result.get("meta", {})
assert_export_result(meta, log, require_signers=True)
# Assert shadow ticket was created
assert_shadow_ticket(ctx, alice.address, log, expect_exists=True)
# --- 2. Submit Payment from same account ---
log("Submitting payment from alice to bob...")
pay_result = await ctx.submit_and_wait(
{
"TransactionType": "Payment",
"Destination": bob.address,
"Amount": "1000000",
"Fee": "12",
},
alice.wallet,
timeout=30,
)
pay_engine = pay_result.get("engine_result", "")
log(f"Payment result: {pay_engine}")
if pay_engine != "tesSUCCESS":
raise AssertionError(f"Payment failed: {pay_engine}")
log(
f"Both transactions succeeded: "
f"Export at ledger {export_seq}, Payment at ledger {ctx.validated_ledger_index(0)}"
)
log("Sequence handling OK - export didn't block subsequent txns")
log("PASS")

View File

@@ -1,211 +0,0 @@
""":descr: install xport hook, trigger export, verify emitted ttEXPORT lifecycle
1. Fund alice (hook holder), bob (trigger), carol (export destination)
2. Install xport hook on alice
3. bob pays alice with DST=carol → hook calls xport() → emits ttEXPORT
4. Emitted ttEXPORT enters open ledger, validators attach sigs via proposals
5. Verify Export transaction appears in a subsequent ledger
"""
from __future__ import annotations
from export_helpers import (
require_export,
find_export_txns,
dst_param,
assert_hook_accepted,
assert_export_result,
assert_shadow_ticket,
)
# C source for the xport hook — verbatim from src/test/app/Export_test_hooks.h
# On Payment to the hook account, exports a 1 XAH payment to the DST param.
XPORT_HOOK_C = r"""
#include <stdint.h>
extern int32_t _g(uint32_t id, uint32_t maxiter);
extern int64_t accept(uint32_t read_ptr, uint32_t read_len, int64_t error_code);
extern int64_t rollback(uint32_t read_ptr, uint32_t read_len, int64_t error_code);
extern int64_t xport(uint32_t write_ptr, uint32_t write_len, uint32_t read_ptr, uint32_t read_len);
extern int64_t xport_reserve(uint32_t count);
extern int64_t hook_account(uint32_t write_ptr, uint32_t write_len);
extern int64_t otxn_param(uint32_t write_ptr, uint32_t write_len, uint32_t name_ptr, uint32_t name_len);
extern int64_t otxn_type(void);
extern int64_t ledger_seq(void);
#define SBUF(x) (uint32_t)(x), sizeof(x)
#define ASSERT(x) if (!(x)) rollback((uint32_t)#x, sizeof(#x), __LINE__)
#define ttPAYMENT 0
#define tfCANONICAL 0x80000000UL
#define amAMOUNT 1
#define amFEE 8
#define atACCOUNT 1
#define atDESTINATION 3
#define ENCODE_TT(buf_out, tt) \
buf_out[0] = 0x12U; buf_out[1] = (tt >> 8) & 0xFFU; buf_out[2] = tt & 0xFFU; buf_out += 3;
#define ENCODE_FLAGS(buf_out, flags) \
buf_out[0] = 0x22U; buf_out[1] = (flags >> 24) & 0xFFU; buf_out[2] = (flags >> 16) & 0xFFU; \
buf_out[3] = (flags >> 8) & 0xFFU; buf_out[4] = flags & 0xFFU; buf_out += 5;
#define ENCODE_SEQUENCE(buf_out, seq) \
buf_out[0] = 0x24U; buf_out[1] = (seq >> 24) & 0xFFU; buf_out[2] = (seq >> 16) & 0xFFU; \
buf_out[3] = (seq >> 8) & 0xFFU; buf_out[4] = seq & 0xFFU; buf_out += 5;
#define ENCODE_FLS(buf_out, fls) \
buf_out[0] = 0x20U; buf_out[1] = 0x1AU; buf_out[2] = (fls >> 24) & 0xFFU; \
buf_out[3] = (fls >> 16) & 0xFFU; buf_out[4] = (fls >> 8) & 0xFFU; \
buf_out[5] = fls & 0xFFU; buf_out += 6;
#define ENCODE_LLS(buf_out, lls) \
buf_out[0] = 0x20U; buf_out[1] = 0x1BU; buf_out[2] = (lls >> 24) & 0xFFU; \
buf_out[3] = (lls >> 16) & 0xFFU; buf_out[4] = (lls >> 8) & 0xFFU; \
buf_out[5] = lls & 0xFFU; buf_out += 6;
#define ENCODE_DROPS(buf_out, drops, amt_type) \
buf_out[0] = 0x60U + amt_type; buf_out[1] = 0x40U + ((drops >> 56) & 0x3FU); \
buf_out[2] = (drops >> 48) & 0xFFU; buf_out[3] = (drops >> 40) & 0xFFU; \
buf_out[4] = (drops >> 32) & 0xFFU; buf_out[5] = (drops >> 24) & 0xFFU; \
buf_out[6] = (drops >> 16) & 0xFFU; buf_out[7] = (drops >> 8) & 0xFFU; \
buf_out[8] = drops & 0xFFU; buf_out += 9;
#define ENCODE_SIGNING_PUBKEY_EMPTY(buf_out) \
buf_out[0] = 0x73U; buf_out[1] = 0x00U; buf_out += 2;
#define ENCODE_ACCOUNT(buf_out, acc, acc_type) \
buf_out[0] = 0x80U + acc_type; buf_out[1] = 0x14U; \
for (int i = 0; i < 20; ++i) buf_out[2+i] = acc[i]; buf_out += 22;
#define PREPARE_PAYMENT_SIMPLE_SIZE 270U
int64_t hook(uint32_t reserved) {
_g(1, 1);
if (otxn_type() != ttPAYMENT)
return accept(0, 0, 0);
ASSERT(xport_reserve(1) == 1);
uint8_t dst[20];
int64_t dst_len = otxn_param(SBUF(dst), "DST", 3);
ASSERT(dst_len == 20);
uint8_t acc[20];
ASSERT(hook_account(SBUF(acc)) == 20);
uint32_t cls = (uint32_t)ledger_seq();
uint8_t tx[PREPARE_PAYMENT_SIMPLE_SIZE];
uint8_t* buf = tx;
ENCODE_TT(buf, ttPAYMENT);
ENCODE_FLAGS(buf, tfCANONICAL);
ENCODE_SEQUENCE(buf, 0);
ENCODE_FLS(buf, cls + 1);
ENCODE_LLS(buf, cls + 5);
// sfTicketSequence = UINT32 field 41 = 0x20 0x29
buf[0] = 0x20U; buf[1] = 0x29U;
buf[2] = 0; buf[3] = 0; buf[4] = 0; buf[5] = 1;
buf += 6;
uint64_t drops = 1000000;
ENCODE_DROPS(buf, drops, amAMOUNT);
ENCODE_DROPS(buf, 10, amFEE);
ENCODE_SIGNING_PUBKEY_EMPTY(buf);
ENCODE_ACCOUNT(buf, acc, atACCOUNT);
ENCODE_ACCOUNT(buf, dst, atDESTINATION);
uint8_t hash[32];
int64_t xport_result = xport(SBUF(hash), (uint32_t)tx, buf - tx);
ASSERT(xport_result == 32);
return accept(0, 0, 0);
}
"""
async def scenario(ctx, log):
# Wait for network to start and amendments to activate
await require_export(ctx, log)
# --- Setup ---
await ctx.fund_accounts({"alice": 10000, "bob": 10000, "carol": 1000})
log("Accounts funded")
alice = ctx.account("alice")
carol = ctx.account("carol")
# Compile and install xport hook on alice
wasm = ctx.compile_hook(XPORT_HOOK_C, label="xport")
await ctx.submit_and_wait(
{
"TransactionType": "SetHook",
"Hooks": [
{
"Hook": {
"CreateCode": wasm.hex().upper(),
"HookOn": "0" * 64,
"HookNamespace": "0" * 64,
"HookApiVersion": 0,
"Flags": 1, # hsfOVERRIDE
}
}
],
"Fee": "100000000",
},
alice.wallet,
)
log(
f"Hook installed on alice ({alice.address[:12]}...) "
f"ledger {ctx.validated_ledger_index(0)}"
)
# --- Trigger ---
# bob pays alice → hook calls xport() → emits ttEXPORT
trigger_result = await ctx.submit_and_wait(
{
"TransactionType": "Payment",
"Destination": alice.address,
"Amount": "100000000",
"Fee": "1000000",
"HookParameters": [dst_param(carol.address)],
},
ctx.account("bob").wallet,
)
trigger_seq = ctx.validated_ledger_index(0)
log(f"Export triggered at ledger {trigger_seq}")
# Assert hook fired with ACCEPT and emitted 1 tx
trigger_meta = trigger_result.get("meta", {})
assert_hook_accepted(trigger_meta, log, expected_emits=1)
# --- Verify: check each ledger close for the Export transaction ---
max_ledgers = 10
for i in range(max_ledgers):
await ctx.wait_for_ledgers(1, node_id=0, timeout=30)
seq = ctx.validated_ledger_index(0)
exports = find_export_txns(ctx, seq)
if exports:
export_tx = exports[0]
meta = export_tx.get("meta", export_tx.get("metaData", {}))
result = meta.get("TransactionResult", "")
log(f"Ledger {seq}: Export txn found, result={result}")
if result != "tesSUCCESS":
raise AssertionError(f"Export did not succeed: {result}")
# Assert ExportResult is well-formed with signers and inner tx
assert_export_result(meta, log, require_signers=True)
# Assert shadow ticket was created
assert_shadow_ticket(ctx, alice.address, log, expect_exists=True)
log("PASS")
return
log(f"Ledger {seq}: no Export txn yet")
raise AssertionError(
f"No Export transaction found after {max_ledgers} ledger closes"
)

View File

@@ -1,180 +0,0 @@
"""Shared helpers for ConsensusEntropy scenario tests."""
from __future__ import annotations
from xahaud_scripts.testnet.config import feature_name_to_hash
ZERO_DIGEST = "0" * 64
CONSENSUS_ENTROPY_FEATURE = feature_name_to_hash("ConsensusEntropy")
def feature_hash(name: str) -> str:
"""Return the amendment hash accepted by feature RPC."""
return feature_name_to_hash(name)
def feature_status(ctx, name: str, node_id=0):
"""Query a feature by amendment hash; feature RPC names are ambiguous."""
return ctx.feature_check(feature_hash(name), node_id=node_id)
def consensus_entropy_feature(ctx, node_id=0):
"""Query ConsensusEntropy by amendment hash."""
return feature_status(ctx, "ConsensusEntropy", node_id=node_id)
async def require_entropy(ctx, log):
"""Wait for first ledger and assert ConsensusEntropy is enabled."""
await ctx.wait_for_ledger_close(timeout=120)
feature = consensus_entropy_feature(ctx, node_id=0)
if not feature or not feature.get("enabled", False):
raise AssertionError(f"ConsensusEntropy not enabled: {feature}")
log("ConsensusEntropy enabled")
def get_entropy_tx(ctx, seq):
"""Fetch ledger and return (ce_tx, user_txns) or raise."""
result = ctx.ledger(seq, transactions=True)
if not result:
raise AssertionError(f"Ledger {seq}: fetch failed")
ledger = result.get("ledger")
if not isinstance(ledger, dict):
raise AssertionError(f"Ledger {seq}: fetch returned no ledger: {result}")
txns = ledger.get("transactions", [])
ce = [tx for tx in txns if tx.get("TransactionType") == "ConsensusEntropy"]
user = [tx for tx in txns if tx.get("TransactionType") != "ConsensusEntropy"]
if len(ce) != 1:
raise AssertionError(
f"Ledger {seq}: expected 1 ConsensusEntropy txn, got {len(ce)}"
)
return ce[0], user
def entropy_fields(ce_tx):
"""Return (digest, entropy_count, is_fallback) from a ConsensusEntropy tx.
consensus_fallback rounds carry a deterministic non-zero consensus-bound
digest with EntropyCount=0 and EntropyTier=1 (consensus_fallback).
Validator entropy has EntropyTier=3 (validator_quorum).
WARNING: is_fallback is ``tier != 3``, so it lumps participant_aligned
(Tier 2) in with fallback. It is only safe where no Tier 2 band exists
(e.g. 5-node networks, where tier2 == quorum). For band-aware scenarios use
the explicit assert_consensus_fallback / assert_participant_aligned /
assert_validator_quorum helpers, which check EntropyTier directly.
"""
digest = ce_tx.get("Digest", "")
entropy_count = ce_tx.get("EntropyCount", -1)
tier = ce_tx.get("EntropyTier", None)
if tier is not None:
is_fallback = tier != 3
else:
is_fallback = entropy_count == 0
return digest, entropy_count, is_fallback
def assert_participant_aligned(ce_tx, seq, expected_count=None):
"""Assert participant_aligned (Tier 2) entropy on a ConsensusEntropy tx.
Tier 2 is the sub-quorum band: the agreed reveal cohort is >= the
participant floor but < the 80% validator quorum, so it carries
EntropyTier=2 with a deterministic non-zero digest. NOTE entropy_fields()'s
is_fallback lumps tier 2 in with fallback (is_fallback = tier != 3), so the
tier must be checked EXPLICITLY here.
"""
digest = ce_tx.get("Digest", "")
count = ce_tx.get("EntropyCount", -1)
tier = ce_tx.get("EntropyTier", None)
if tier != 2:
raise AssertionError(
f"Ledger {seq}: expected EntropyTier==2 (participant_aligned), "
f"got {tier} (EntropyCount={count})"
)
if not digest or digest == ZERO_DIGEST:
raise AssertionError(
f"Ledger {seq}: participant_aligned digest must be non-zero, got "
f"{digest[:16]}..."
)
if expected_count is not None and count != expected_count:
raise AssertionError(
f"Ledger {seq}: participant_aligned EntropyCount must be "
f"{expected_count} (the surviving cohort), got {count}"
)
return digest, count
def assert_validator_quorum(ce_tx, seq, min_count=None):
"""Assert validator_quorum (Tier 3) entropy on a ConsensusEntropy tx:
EntropyTier=3, a deterministic non-zero digest, and (optionally)
EntropyCount >= min_count (the active quorum). The count can EXCEED the
quorum (e.g. a still-full 6/6 ledger caught at a 6->5 transition), so check
>=, not ==.
"""
digest = ce_tx.get("Digest", "")
count = ce_tx.get("EntropyCount", -1)
tier = ce_tx.get("EntropyTier", None)
if tier != 3:
raise AssertionError(
f"Ledger {seq}: expected EntropyTier==3 (validator_quorum), got "
f"{tier} (EntropyCount={count})"
)
if not digest or digest == ZERO_DIGEST:
raise AssertionError(
f"Ledger {seq}: validator_quorum digest must be non-zero, got "
f"{digest[:16]}..."
)
if min_count is not None and count < min_count:
raise AssertionError(
f"Ledger {seq}: validator_quorum EntropyCount={count} < quorum "
f"{min_count}"
)
return digest, count
def assert_consensus_fallback(ce_tx, seq):
"""Assert consensus_fallback (Tier 1) entropy on a ConsensusEntropy tx:
EntropyTier=1, EntropyCount=0, and a deterministic NON-zero digest.
"""
digest = ce_tx.get("Digest", "")
count = ce_tx.get("EntropyCount", -1)
tier = ce_tx.get("EntropyTier", None)
if tier != 1:
raise AssertionError(
f"Ledger {seq}: expected EntropyTier==1 (consensus_fallback), got "
f"{tier} (EntropyCount={count})"
)
if count != 0:
raise AssertionError(
f"Ledger {seq}: consensus_fallback EntropyCount must be 0, got "
f"{count}"
)
if not digest or digest == ZERO_DIGEST:
raise AssertionError(
f"Ledger {seq}: consensus_fallback digest must be non-zero, got "
f"{digest[:16]}..."
)
return digest, count
def assert_valid_entropy(ce_tx, seq, seen_digests=None):
"""Assert quorum-met validator entropy. Optionally check uniqueness."""
digest, entropy_count, is_fallback = entropy_fields(ce_tx)
if is_fallback or not digest or digest == ZERO_DIGEST:
raise AssertionError(f"Ledger {seq}: fallback/empty Digest")
if entropy_count < 4:
raise AssertionError(
f"Ledger {seq}: EntropyCount={entropy_count} < 4 (sub-quorum)"
)
if seen_digests is not None:
if digest in seen_digests:
raise AssertionError(f"Ledger {seq}: duplicate Digest {digest[:16]}...")
seen_digests.add(digest)
return digest, entropy_count

View File

@@ -1,92 +0,0 @@
defaults:
network:
node_count: 5
launcher: tmux
find_ports: true
slave_delay: 0.2
features:
- ConsensusEntropy
- Export
track_features:
- ConsensusEntropy
- Export
unl_report: true
log_levels:
TxQ: info
Protocol: debug
Peer: debug
LedgerConsensus: debug
ConsensusExtensions: debug
NetworkOPs: info
env:
XAHAU_RESOURCE_PER_PORT: "1"
rc:
- rng_poll_ms=250
tests:
- name: latency_baseline_ce
script: .testnet/scenarios/perf/ce_export_latency_probe.py
params:
warmup_ledgers: 3
ledgers: 8
submit_export: false
- name: latency_baseline_export
script: .testnet/scenarios/perf/ce_export_latency_probe.py
params:
warmup_ledgers: 3
ledgers: 8
submit_export: true
- name: latency_proposal_delay_export
script: .testnet/scenarios/perf/ce_export_latency_probe.py
params:
warmup_ledgers: 3
ledgers: 8
submit_export: true
network:
rc:
- rng_poll_ms=250
- delay=100,jitter=25,msg=proposal
- name: latency_directed_pair_delay_export
script: .testnet/scenarios/perf/ce_export_latency_probe.py
params:
warmup_ledgers: 3
ledgers: 8
submit_export: true
network:
rc:
- rng_poll_ms=250
- n0->n2:delay=750,jitter=100,msg=proposal
- n2->n0:delay=750,jitter=100,msg=proposal
- name: latency_slow_minority_export
script: .testnet/scenarios/perf/ce_export_latency_probe.py
params:
warmup_ledgers: 3
ledgers: 8
submit_export: true
export_timeout: 120
network:
rc:
- rng_poll_ms=250
- n3->n0:delay=500,jitter=100,msg=proposal
- n3->n1:delay=500,jitter=100,msg=proposal
- n3->n2:delay=500,jitter=100,msg=proposal
- n4->n0:delay=500,jitter=100,msg=proposal
- n4->n1:delay=500,jitter=100,msg=proposal
- n4->n2:delay=500,jitter=100,msg=proposal
- n0->n3:delay=500,jitter=100,msg=proposal
- n1->n3:delay=500,jitter=100,msg=proposal
- n2->n3:delay=500,jitter=100,msg=proposal
- n0->n4:delay=500,jitter=100,msg=proposal
- n1->n4:delay=500,jitter=100,msg=proposal
- n2->n4:delay=500,jitter=100,msg=proposal
- name: latency_export_no_veto_with_delay
script: .testnet/scenarios/export/export_no_veto_missing_observation.py
network:
rc:
- rng_poll_ms=250
- delay=300,jitter=100,msg=proposal
- n4:no_export_sig_hash=true

View File

@@ -1,196 +0,0 @@
""":descr: measure CE/export behavior while RuntimeConfig injects latency/drop.
The suite supplies runtime fault injection through network.rc. This scenario
does not mutate RuntimeConfig itself; it observes what the launched network does
under that condition and logs enough counters to compare variants.
"""
from __future__ import annotations
from collections import Counter
import json
from export.export_helpers import assert_export_result, require_export
from helpers import consensus_entropy_feature, get_entropy_tx
async def _require_runtime_config(ctx, log):
result = ctx.rpc.runtime_config(0)
if not result or result.get("error"):
raise AssertionError(
"Latency probe requires a binary built with "
"xahaud_runtime_test_config=ON; runtime_config RPC returned "
f"{result}"
)
log("RuntimeConfig RPC active")
async def _require_consensus_entropy(ctx, log):
feature = consensus_entropy_feature(ctx, node_id=0)
if not feature or not feature.get("enabled", False):
raise AssertionError(f"ConsensusEntropy not enabled: {feature}")
log("ConsensusEntropy enabled")
def _log_runtime_config(ctx, log):
for node_id in range(ctx.node_count):
cfg = ctx.rpc.runtime_config(node_id)
if cfg is None:
raise AssertionError(f"runtime_config RPC failed on node {node_id}")
log(
f"runtime_config n{node_id}: "
f"{json.dumps(cfg, sort_keys=True, separators=(',', ':'))}"
)
async def _submit_direct_export(ctx, log, *, timeout):
await ctx.fund_accounts({"alice": 10000, "bob": 1000})
alice = ctx.account("alice")
bob = ctx.account("bob")
current_seq = ctx.validated_ledger_index(0)
if current_seq is None:
raise AssertionError("validated ledger is not available before Export")
log(f"Submitting direct Export at validated ledger {current_seq}")
started = ctx.mark("latency-export-submit-start")
result = await ctx.submit_and_wait(
{
"TransactionType": "Export",
"LastLedgerSequence": current_seq + 12,
"Fee": "1000000",
"ExportedTxn": {
"TransactionType": "Payment",
"Account": alice.address,
"Destination": bob.address,
"Amount": "1000000",
"Fee": "10",
"Sequence": 0,
"TicketSequence": 1,
"FirstLedgerSequence": current_seq + 1,
"LastLedgerSequence": current_seq + 10,
"Flags": 2147483648,
"SigningPubKey": "",
},
},
alice.wallet,
timeout=timeout,
)
ended = ctx.mark("latency-export-submit-end")
elapsed = (ended.monotonic_ns - started.monotonic_ns) / 1_000_000_000
engine_result = result.get("engine_result", "")
log(f"Export result={engine_result} elapsed={elapsed:.3f}s")
if engine_result != "tesSUCCESS":
raise AssertionError(f"Expected Export tesSUCCESS, got {engine_result}")
export_result = assert_export_result(result.get("meta", {}), log)
signers = export_result.get("ExportedTxn", {}).get("Signers", [])
log(f"Export signer count={len(signers)}")
return started, ended
def _summarize_logs(ctx, log, *, label, started, ended):
patterns = {
"rng_selected": r"RNG: entropy selected",
"rng_fallback": r"tier=1",
"rng_participant_aligned": r"tier=2",
"rng_validator_quorum": r"tier=3",
"export_retry": r"terRETRY_EXPORT",
"export_quorum_timeout": r"Export: exportSigSet quorum alignment timeout",
"export_missing_observation_ignored": (
r"Export: missing exportSigSetHash observation ignored"
),
}
for name, pattern in patterns.items():
result = ctx.search_logs(pattern, since=started, until=ended, limit=500)
log(f"log_count {label}.{name}={result.count}")
async def scenario(
ctx,
log,
*,
warmup_ledgers=3,
ledgers=8,
submit_export=False,
export_timeout=90,
):
await ctx.wait_for_ledger_close(timeout=120)
await _require_runtime_config(ctx, log)
_log_runtime_config(ctx, log)
await _require_consensus_entropy(ctx, log)
if submit_export:
# require_export also asserts the UNLReport precondition for successful
# network-mode Export. Keep that explicit in perf runs so a missing
# report does not masquerade as a latency failure.
await require_export(ctx, log, require_runtime_config=False)
await ctx.wait_for_ledgers(warmup_ledgers, node_id=0, timeout=120)
warm_seq = ctx.validated_ledger_index(0)
log(f"Warmup complete at validated ledger {warm_seq}")
export_window = None
if submit_export:
export_window = await _submit_direct_export(
ctx, log, timeout=export_timeout
)
started = ctx.mark("latency-probe-start")
start_seq = ctx.validated_ledger_index(0)
await ctx.wait_for_ledgers(ledgers, node_id=0, timeout=max(120, ledgers * 30))
ended = ctx.mark("latency-probe-end")
end_seq = ctx.validated_ledger_index(0)
if start_seq is None or end_seq is None:
raise AssertionError("validated ledger index unavailable during probe")
elapsed = (ended.monotonic_ns - started.monotonic_ns) / 1_000_000_000
closed = max(0, end_seq - start_seq)
cadence = elapsed / closed if closed else 0.0
log(
f"Observed validated ledgers {start_seq + 1}..{end_seq} "
f"closed={closed} elapsed={elapsed:.3f}s cadence={cadence:.3f}s/ledger"
)
tiers: Counter[int] = Counter()
counts: Counter[int] = Counter()
missing_entropy = 0
for seq in range(start_seq + 1, end_seq + 1):
try:
ce, user_txns = get_entropy_tx(ctx, seq)
except AssertionError as exc:
missing_entropy += 1
log(f" Ledger {seq}: no ConsensusEntropy tx ({exc})")
continue
tier = ce.get("EntropyTier", -1)
count = ce.get("EntropyCount", -1)
tiers[tier] += 1
counts[count] += 1
log(
f" Ledger {seq}: tier={tier} count={count} "
f"user_txns={len(user_txns)} digest={ce.get('Digest', '')[:16]}..."
)
log(
"SUMMARY "
f"closed={closed} elapsed_s={elapsed:.3f} cadence_s={cadence:.3f} "
f"tiers={dict(sorted(tiers.items()))} "
f"counts={dict(sorted(counts.items()))} "
f"missing_entropy={missing_entropy}"
)
_summarize_logs(ctx, log, label="probe", started=started, ended=ended)
if export_window is not None:
_summarize_logs(
ctx,
log,
label="export",
started=export_window[0],
ended=export_window[1],
)
log("PASS")

View File

@@ -1,62 +0,0 @@
defaults:
network:
node_count: 5
launcher: tmux
find_ports: true
slave_delay: 0.2
features:
- ConsensusEntropy
track_features:
- ConsensusEntropy
unl_report: true
log_levels:
TxQ: info
Protocol: debug
Peer: debug
LedgerConsensus: debug
ConsensusExtensions: debug
NetworkOPs: info
env:
XAHAU_RESOURCE_PER_PORT: "1"
rc:
- rng_poll_ms=333
tests:
- name: steady_state_entropy
script: .testnet/scenarios/entropy/steady_state_entropy.py
- name: fallback_without_unl_report
script: .testnet/scenarios/entropy/fallback_without_unl_report.py
network:
unl_report: false
- name: steady_state_entropy_fast_start
script: .testnet/scenarios/entropy/steady_state_entropy.py
network:
env:
XAHAUD_RUNTIME_TEST_CONFIG: '{"set":{"global":{"rng_poll_ms":333,"bootstrap_fast_start":true}}}'
- name: entropy_with_transactions
script: .testnet/scenarios/entropy/entropy_with_transactions.py
- name: quorum_recovery_smoke
script: .testnet/scenarios/entropy/quorum_recovery_smoke.py
- name: quorum_degradation_smoke
script: .testnet/scenarios/entropy/quorum_degradation_smoke.py
network:
log_levels:
LedgerConsensus: trace
ConsensusExtensions: trace
# Tier 2 (participant_aligned) needs 6 nodes: n=5 has no band (tier2 ==
# quorum). At 6, the 4/6 window is the participant_aligned band.
- name: participant_aligned_smoke
script: .testnet/scenarios/entropy/participant_aligned_smoke.py
network:
node_count: 6
log_levels:
LedgerConsensus: trace
ConsensusExtensions: trace
# Export scenarios: see export-suite.yml

View File

@@ -8,6 +8,6 @@
"editor.semanticHighlighting.enabled": true,
"editor.tabSize": 4,
"editor.defaultFormatter": "xaver.clang-format",
"editor.formatOnSave": true
"editor.formatOnSave": false
}
}

View File

@@ -8,130 +8,47 @@ The API version controls the API behavior you see. This includes what properties
For a log of breaking changes, see the **API Version [number]** headings. In general, breaking changes are associated with a particular API Version number. For non-breaking changes, scroll to the **XRP Ledger version [x.y.z]** headings. Non-breaking changes are associated with a particular XRP Ledger (`rippled`) release.
## API Version 2
API version 2 is available in `rippled` version 2.0.0 and later. To use this API, clients specify `"api_version" : 2` in each request.
#### Removed methods
In API version 2, the following deprecated methods are no longer available: (https://github.com/XRPLF/rippled/pull/4759)
- `tx_history` - Instead, use other methods such as `account_tx` or `ledger` with the `transactions` field set to `true`.
- `ledger_header` - Instead, use the `ledger` method.
#### Modifications to JSON transaction element in V2
In API version 2, JSON elements for transaction output have been changed and made consistent for all methods which output transactions. (https://github.com/XRPLF/rippled/pull/4775)
This helps to unify the JSON serialization format of transactions. (https://github.com/XRPLF/clio/issues/722, https://github.com/XRPLF/rippled/issues/4727)
- JSON transaction element is named `tx_json`
- Binary transaction element is named `tx_blob`
- JSON transaction metadata element is named `meta`
- Binary transaction metadata element is named `meta_blob`
Additionally, these elements are now consistently available next to `tx_json` (i.e. sibling elements), where possible:
- `hash` - Transaction ID. This data was stored inside transaction output in API version 1, but in API version 2 is a sibling element.
- `ledger_index` - Ledger index (only set on validated ledgers)
- `ledger_hash` - Ledger hash (only set on closed or validated ledgers)
- `close_time_iso` - Ledger close time expressed in ISO 8601 time format (only set on validated ledgers)
- `validated` - Bool element set to `true` if the transaction is in a validated ledger, otherwise `false`
This change affects the following methods:
- `tx` - Transaction data moved into element `tx_json` (was inline inside `result`) or, if binary output was requested, moved from `tx` to `tx_blob`. Renamed binary transaction metadata element (if it was requested) from `meta` to `meta_blob`. Changed location of `hash` and added new elements
- `account_tx` - Renamed transaction element from `tx` to `tx_json`. Renamed binary transaction metadata element (if it was requested) from `meta` to `meta_blob`. Changed location of `hash` and added new elements
- `transaction_entry` - Renamed transaction metadata element from `metadata` to `meta`. Changed location of `hash` and added new elements
- `subscribe` - Renamed transaction element from `transaction` to `tx_json`. Changed location of `hash` and added new elements
- `sign`, `sign_for`, `submit` and `submit_multisigned` - Changed location of `hash` element.
#### Modification to `Payment` transaction JSON schema
When reading Payments, the `Amount` field should generally **not** be used. Instead, use [delivered_amount](https://xrpl.org/partial-payments.html#the-delivered_amount-field) to see the amount that the Payment delivered. To clarify its meaning, the `Amount` field is being renamed to `DeliverMax`. (https://github.com/XRPLF/rippled/pull/4733)
- In `Payment` transaction type, JSON RPC field `Amount` is renamed to `DeliverMax`. To enable smooth client transition, `Amount` is still handled, as described below: (https://github.com/XRPLF/rippled/pull/4733)
- On JSON RPC input (e.g. `submit_multisigned` etc. methods), `Amount` is recognized as an alias to `DeliverMax` for both API version 1 and version 2 clients.
- On JSON RPC input, submitting both `Amount` and `DeliverMax` fields is allowed _only_ if they are identical; otherwise such input is rejected with `rpcINVALID_PARAMS` error.
- On JSON RPC output (e.g. `subscribe`, `account_tx` etc. methods), `DeliverMax` is present in both API version 1 and version 2.
- On JSON RPC output, `Amount` is only present in API version 1 and _not_ in version 2.
#### Modifications to account_info response
- `signer_lists` is returned in the root of the response. (In API version 1, it was nested under `account_data`.) (https://github.com/XRPLF/rippled/pull/3770)
- When using an invalid `signer_lists` value, the API now returns an "invalidParams" error. (https://github.com/XRPLF/rippled/pull/4585)
- (`signer_lists` must be a boolean. In API version 1, strings were accepted and may return a normal response - i.e. as if `signer_lists` were `true`.)
#### Modifications to [account_tx](https://xrpl.org/account_tx.html#account_tx) response
- Using `ledger_index_min`, `ledger_index_max`, and `ledger_index` returns `invalidParams` because if you use `ledger_index_min` or `ledger_index_max`, then it does not make sense to also specify `ledger_index`. In API version 1, no error was returned. (https://github.com/XRPLF/rippled/pull/4571)
- The same applies for `ledger_index_min`, `ledger_index_max`, and `ledger_hash`. (https://github.com/XRPLF/rippled/issues/4545#issuecomment-1565065579)
- Using a `ledger_index_min` or `ledger_index_max` beyond the range of ledgers that the server has:
- returns `lgrIdxMalformed` in API version 2. Previously, in API version 1, no error was returned. (https://github.com/XRPLF/rippled/issues/4288)
- Attempting to use a non-boolean value (such as a string) for the `binary` or `forward` parameters returns `invalidParams` (`rpcINVALID_PARAMS`). Previously, in API version 1, no error was returned. (https://github.com/XRPLF/rippled/pull/4620)
#### Modifications to [noripple_check](https://xrpl.org/noripple_check.html#noripple_check) response
- Attempting to use a non-boolean value (such as a string) for the `transactions` parameter returns `invalidParams` (`rpcINVALID_PARAMS`). Previously, in API version 1, no error was returned. (https://github.com/XRPLF/rippled/pull/4620)
## API Version 1
This version is supported by all `rippled` versions. For WebSocket and HTTP JSON-RPC requests, it is currently the default API version used when no `api_version` is specified.
This version is supported by all `rippled` versions. At time of writing, it is the default API version, used when no `api_version` is specified. When a new API version is introduced, the command line interface will default to the latest API version. The command line is intended for ad-hoc usage by humans, not programs or automated scripts. The command line is not meant for use in production code.
The [commandline](https://xrpl.org/docs/references/http-websocket-apis/api-conventions/request-formatting/#commandline-format) always uses the latest API version. The command line is intended for ad-hoc usage by humans, not programs or automated scripts. The command line is not meant for use in production code.
### Idiosyncrasies
### Inconsistency: server_info - network_id
#### V1 account_info response
The `network_id` field was added in the `server_info` response in version 1.5.0 (2019), but it is not returned in [reporting mode](https://xrpl.org/rippled-server-modes.html#reporting-mode). However, use of reporting mode is now discouraged, in favor of using [Clio](https://github.com/XRPLF/clio) instead.
In [the response to the `account_info` command](https://xrpl.org/account_info.html#response-format), there is `account_data` - which is supposed to be an `AccountRoot` object - and `signer_lists` is returned in this object. However, the docs say that `signer_lists` should be at the root level of the reponse.
## XRP Ledger server version 2.4.0
It makes sense for `signer_lists` to be at the root level because signer lists are not part of the AccountRoot object. (First reported in [xrpl-dev-portal#938](https://github.com/XRPLF/xrpl-dev-portal/issues/938).)
As of 2025-01-28, version 2.4.0 is in development. You can use a pre-release version by building from source or [using the `nightly` package](https://xrpl.org/docs/infrastructure/installation/install-rippled-on-ubuntu).
In `api_version: 2`, the `signer_lists` field [will be moved](#modifications-to-account_info-response-in-v2) to the root level of the account_info response. (https://github.com/XRPLF/rippled/pull/3770)
### Additions and bugfixes in 2.4.0
#### server_info - network_id
- `ledger_entry`: `state` is added an alias for `ripple_state`.
- `ledger_entry`: Enables case-insensitive filtering by canonical name in addition to case-sensitive filtering by RPC name.
- `validators`: Added new field `validator_list_threshold` in response.
- `simulate`: A new RPC that executes a [dry run of a transaction submission](https://github.com/XRPLF/XRPL-Standards/tree/master/XLS-0069d-simulate#2-rpc-simulate)
- Signing methods autofill fees better and properly handle transactions that don't have a base fee, and will also autofill the `NetworkID` field.
## XRP Ledger server version 2.3.0
[Version 2.3.0](https://github.com/XRPLF/rippled/releases/tag/2.3.0) was released on Nov 25, 2024.
### Breaking changes in 2.3.0
- `book_changes`: If the requested ledger version is not available on this node, a `ledgerNotFound` error is returned and the node does not attempt to acquire the ledger from the p2p network (as with other non-admin RPCs).
Admins can still attempt to retrieve old ledgers with the `ledger_request` RPC.
### Additions and bugfixes in 2.3.0
- `book_changes`: Returns a `validated` field in its response, which was missing in prior versions.
## XRP Ledger server version 2.2.0
[Version 2.2.0](https://github.com/XRPLF/rippled/releases/tag/2.2.0) was released on Jun 5, 2024. The following additions are non-breaking (because they are purely additive):
- The `feature` method now has a non-admin mode for users. (It was previously only available to admin connections.) The method returns an updated list of amendments, including their names and other information. ([#4781](https://github.com/XRPLF/rippled/pull/4781))
The `network_id` field was added in the `server_info` response in version 1.5.0 (2019), but it is not returned in [reporting mode](https://xrpl.org/rippled-server-modes.html#reporting-mode).
## XRP Ledger server version 2.0.0
[Version 2.0.0](https://github.com/XRPLF/rippled/releases/tag/2.0.0) was released on Jan 9, 2024. The following additions are non-breaking (because they are purely additive):
### Additions in 2.2
Additions are intended to be non-breaking (because they are purely additive).
- `feature`: A non-admin mode that uses the same formatting as admin RPC, but hides potentially-sensitive data.
### Additions in 2.0
Additions are intended to be non-breaking (because they are purely additive).
- `server_definitions`: A new RPC that generates a `definitions.json`-like output that can be used in XRPL libraries.
- In `Payment` transactions, `DeliverMax` has been added. This is a replacement for the `Amount` field, which should not be used. Typically, the `delivered_amount` (in transaction metadata) should be used. To ease the transition, `DeliverMax` is present regardless of API version, since adding a field is non-breaking.
- API version 2 has been moved from beta to supported, meaning that it is generally available (regardless of the `beta_rpc_api` setting).
## XRP Ledger server version 2.2.0
The following is a non-breaking addition to the API.
- The `feature` method now has a non-admin mode for users. (It was previously only available to admin connections.) The method returns an updated list of amendments, including their names and other information. ([#4781](https://github.com/XRPLF/rippled/pull/4781))
## XRP Ledger server version 1.12.0
[Version 1.12.0](https://github.com/XRPLF/rippled/releases/tag/1.12.0) was released on Sep 6, 2023. The following additions are non-breaking (because they are purely additive).
[Version 1.12.0](https://github.com/XRPLF/rippled/releases/tag/1.12.0) was released on Sep 6, 2023.
### Additions in 1.12
Additions are intended to be non-breaking (because they are purely additive).
- `server_info`: Added `ports`, an array which advertises the RPC and WebSocket ports. This information is also included in the `/crawl` endpoint (which calls `server_info` internally). `grpc` and `peer` ports are also included. (https://github.com/XRPLF/rippled/pull/4427)
- `ports` contains objects, each containing a `port` for the listening port (a number string), and a `protocol` array listing the supported protocols on that port.
@@ -208,6 +125,71 @@ was released on Mar 14, 2023.
removed from the [ledger subscription stream](https://xrpl.org/subscribe.html#ledger-stream), because it will no longer
have any meaning.
## API Version 2
API version 2 is introduced in `rippled` version 2.0. Users can request it explicitly by specifying `"api_version" : 2`.
#### Removed methods
In API version 2, the following deprecated methods are no longer available: (https://github.com/XRPLF/rippled/pull/4759)
- `tx_history` - Instead, use other methods such as `account_tx` or `ledger` with the `transactions` field set to `true`.
- `ledger_header` - Instead, use the `ledger` method.
#### Modifications to JSON transaction element in V2
In API version 2, JSON elements for transaction output have been changed and made consistent for all methods which output transactions. (https://github.com/XRPLF/rippled/pull/4775)
This helps to unify the JSON serialization format of transactions. (https://github.com/XRPLF/clio/issues/722, https://github.com/XRPLF/rippled/issues/4727)
- JSON transaction element is named `tx_json`
- Binary transaction element is named `tx_blob`
- JSON transaction metadata element is named `meta`
- Binary transaction metadata element is named `meta_blob`
Additionally, these elements are now consistently available next to `tx_json` (i.e. sibling elements), where possible:
- `hash` - Transaction ID. This data was stored inside transaction output in API version 1, but in API version 2 is a sibling element.
- `ledger_index` - Ledger index (only set on validated ledgers)
- `ledger_hash` - Ledger hash (only set on closed or validated ledgers)
- `close_time_iso` - Ledger close time expressed in ISO 8601 time format (only set on validated ledgers)
- `validated` - Bool element set to `true` if the transaction is in a validated ledger, otherwise `false`
This change affects the following methods:
- `tx` - Transaction data moved into element `tx_json` (was inline inside `result`) or, if binary output was requested, moved from `tx` to `tx_blob`. Renamed binary transaction metadata element (if it was requested) from `meta` to `meta_blob`. Changed location of `hash` and added new elements
- `account_tx` - Renamed transaction element from `tx` to `tx_json`. Renamed binary transaction metadata element (if it was requested) from `meta` to `meta_blob`. Changed location of `hash` and added new elements
- `transaction_entry` - Renamed transaction metadata element from `metadata` to `meta`. Changed location of `hash` and added new elements
- `subscribe` - Renamed transaction element from `transaction` to `tx_json`. Changed location of `hash` and added new elements
- `sign`, `sign_for`, `submit` and `submit_multisigned` - Changed location of `hash` element.
#### Modification to `Payment` transaction JSON schema
When reading Payments, the `Amount` field should generally **not** be used. Instead, use [delivered_amount](https://xrpl.org/partial-payments.html#the-delivered_amount-field) to see the amount that the Payment delivered. To clarify its meaning, the `Amount` field is being renamed to `DeliverMax`. (https://github.com/XRPLF/rippled/pull/4733)
- In `Payment` transaction type, JSON RPC field `Amount` is renamed to `DeliverMax`. To enable smooth client transition, `Amount` is still handled, as described below: (https://github.com/XRPLF/rippled/pull/4733)
- On JSON RPC input (e.g. `submit_multisigned` etc. methods), `Amount` is recognized as an alias to `DeliverMax` for both API version 1 and version 2 clients.
- On JSON RPC input, submitting both `Amount` and `DeliverMax` fields is allowed _only_ if they are identical; otherwise such input is rejected with `rpcINVALID_PARAMS` error.
- On JSON RPC output (e.g. `subscribe`, `account_tx` etc. methods), `DeliverMax` is present in both API version 1 and version 2.
- On JSON RPC output, `Amount` is only present in API version 1 and _not_ in version 2.
#### Modifications to account_info response
- `signer_lists` is returned in the root of the response. In API version 1, it was nested under `account_data`. (https://github.com/XRPLF/rippled/pull/3770)
- When using an invalid `signer_lists` value, the API now returns an "invalidParams" error. (https://github.com/XRPLF/rippled/pull/4585)
- (`signer_lists` must be a boolean. In API version 1, strings were accepted and may return a normal response - i.e. as if `signer_lists` were `true`.)
#### Modifications to [account_tx](https://xrpl.org/account_tx.html#account_tx) response
- Using `ledger_index_min`, `ledger_index_max`, and `ledger_index` returns `invalidParams` because if you use `ledger_index_min` or `ledger_index_max`, then it does not make sense to also specify `ledger_index`. In API version 1, no error was returned. (https://github.com/XRPLF/rippled/pull/4571)
- The same applies for `ledger_index_min`, `ledger_index_max`, and `ledger_hash`. (https://github.com/XRPLF/rippled/issues/4545#issuecomment-1565065579)
- Using a `ledger_index_min` or `ledger_index_max` beyond the range of ledgers that the server has:
- returns `lgrIdxMalformed` in API version 2. Previously, in API version 1, no error was returned. (https://github.com/XRPLF/rippled/issues/4288)
- Attempting to use a non-boolean value (such as a string) for the `binary` or `forward` parameters returns `invalidParams` (`rpcINVALID_PARAMS`). Previously, in API version 1, no error was returned. (https://github.com/XRPLF/rippled/pull/4620)
#### Modifications to [noripple_check](https://xrpl.org/noripple_check.html#noripple_check) response
- Attempting to use a non-boolean value (such as a string) for the `transactions` parameter returns `invalidParams` (`rpcINVALID_PARAMS`). Previously, in API version 1, no error was returned. (https://github.com/XRPLF/rippled/pull/4620)
# Unit tests for API changes
The following information is useful to developers contributing to this project:

199
BUILD.md
View File

@@ -10,7 +10,7 @@
## Branches
For a stable release, choose the `master` branch or one of the [tagged
releases](https://github.com/Xahau/xahaud/releases).
releases](https://github.com/ripple/rippled/releases).
```
git checkout master
@@ -36,10 +36,10 @@ See [System Requirements](https://xrpl.org/system-requirements.html).
Building rippled generally requires git, Python, Conan, CMake, and a C++ compiler. Some guidance on setting up such a [C++ development environment can be found here](./docs/build/environment.md).
- [Python 3.7](https://www.python.org/downloads/)
- [Conan 2.x](https://conan.io/downloads)
- [Conan 1.55](https://conan.io/downloads.html)
- [CMake 3.16](https://cmake.org/download/)
`xahaud` is written in the C++20 dialect and includes the `<concepts>` header.
`rippled` is written in the C++20 dialect and includes the `<concepts>` header.
The [minimum compiler versions][2] required are:
| Compiler | Version |
@@ -58,19 +58,20 @@ Here are [sample instructions for setting up a C++ development environment on Li
### Mac
Many xahaud engineers use macOS for development.
Many rippled engineers use macOS for development.
Here are [sample instructions for setting up a C++ development environment on macOS](./docs/build/environment.md#macos).
### Windows
We don't recommend Windows for `xahaud` production at this time. As of
November 2025, Ubuntu has the highest level of quality assurance, testing,
and support.
Windows is not recommended for production use at this time.
Windows developers should use Visual Studio 2019. `xahaud` isn't
compatible with [Boost](https://www.boost.org/) 1.78 or 1.79, and Conan
can't build earlier Boost versions.
- Additionally, 32-bit Windows development is not supported.
- Visual Studio 2022 is not yet supported.
- rippled generally requires [Boost][] 1.77, which Conan cannot build with VS 2022.
- Until rippled is updated for compatibility with later versions of Boost, Windows developers may need to use Visual Studio 2019.
[Boost]: https://www.boost.org/
## Steps
@@ -85,30 +86,13 @@ If you are unfamiliar with Conan, then please read [this crash course](./docs/bu
You'll need at least one Conan profile:
```
conan profile detect --force
conan profile new default --detect
```
Update the compiler settings:
For Conan 2, you can edit the profile directly at `~/.conan2/profiles/default`,
or use the Conan CLI. Ensure C++20 is set:
```
conan profile show
```
Look for `compiler.cppstd=20` in the output. If it's not set, edit the profile:
```
# Edit ~/.conan2/profiles/default and ensure these settings exist:
[settings]
compiler.cppstd=20
```
Configure Conan (1.x only) to use recipe revisions:
```
conan config set general.revisions_enabled=1
conan profile update settings.compiler.cppstd=20 default
```
**Linux** developers will commonly have a default Conan [profile][] that compiles
@@ -117,54 +101,35 @@ If you are linking with libstdc++ (see profile setting `compiler.libcxx`),
then you will need to choose the `libstdc++11` ABI:
```
# In ~/.conan2/profiles/default, ensure:
[settings]
compiler.libcxx=libstdc++11
conan profile update settings.compiler.libcxx=libstdc++11 default
```
Ensure inter-operability between `boost::string_view` and `std::string_view` types:
```
conan profile update 'conf.tools.build:cxxflags+=["-DBOOST_BEAST_USE_STD_STRING_VIEW"]' default
conan profile update 'env.CXXFLAGS="-DBOOST_BEAST_USE_STD_STRING_VIEW"' default
```
If you have other flags in the `conf.tools.build` or `env.CXXFLAGS` sections, make sure to retain the existing flags and append the new ones. You can check them with:
```
conan profile show default
```
**Windows** developers may need to use the x64 native build tools.
An easy way to do that is to run the shortcut "x64 Native Tools Command
Prompt" for the version of Visual Studio that you have installed.
Windows developers must also build `xahaud` and its dependencies for the x64
architecture.
Windows developers must also build `rippled` and its dependencies for the x64
architecture:
```
# In ~/.conan2/profiles/default, ensure:
[settings]
arch=x86_64
conan profile update settings.arch=x86_64 default
```
### Multiple compilers
```
# In ~/.conan2/profiles/default, add under [conf] section:
[conf]
tools.build:compiler_executables={"c": "<path>", "cpp": "<path>"}
```
When `/usr/bin/g++` exists on a platform, it is the default cpp compiler. This
default works for some users.
For setting environment variables for dependencies:
However, if this compiler cannot build rippled or its dependencies, then you can
install another compiler and set Conan and CMake to use it.
Update the `conf.tools.build:compiler_executables` setting in order to set the correct variables (`CMAKE_<LANG>_COMPILER`) in the
generated CMake toolchain file.
For example, on Ubuntu 20, you may have gcc at `/usr/bin/gcc` and g++ at `/usr/bin/g++`; if that is the case, you can select those compilers with:
```
conan profile update 'conf.tools.build:compiler_executables={"c": "/usr/bin/gcc", "cpp": "/usr/bin/g++"}' default
```
```
# In ~/.conan2/profiles/default, add under [buildenv] section:
[buildenv]
CC=<path>
CXX=<path>
```
Replace `/usr/bin/gcc` and `/usr/bin/g++` with paths to the desired compilers.
It should choose the compiler for dependencies as well,
but not all of them have a Conan recipe that respects this setting (yet).
@@ -179,41 +144,21 @@ It does not explicitly link the C++ standard library,
which allows you to statically link it with GCC, if you want.
```
conan export external/snappy --version 1.1.10 --user xahaud --channel stable
conan export external/snappy snappy/1.1.9@
```
Export our [Conan recipe for RocksDB](./external/rocksdb).
It does not override paths to dependencies when building with Visual Studio.
```
# Conan 1.x
conan export external/rocksdb rocksdb/6.29.5@
# Conan 2.x
conan export --version 6.29.5 external/rocksdb
```
Export our [Conan recipe for SOCI](./external/soci).
It patches their CMake to correctly import its dependencies.
```
conan export external/soci --version 4.0.3 --user xahaud --channel stable
```
6. Export our [Conan recipe for WasmEdge](./external/wasmedge).
```
conan export external/wasmedge --version 0.11.2 --user xahaud --channel stable
```
Export our [Conan recipe for NuDB](./external/nudb).
It fixes some source files to add missing `#include`s.
```
# Conan 1.x
conan export external/nudb nudb/2.0.8@
# Conan 2.x
conan export --version 2.0.8 external/nudb
conan export external/soci soci/4.0.3@
```
### Build and Test
@@ -234,15 +179,13 @@ It fixes some source files to add missing `#include`s.
the `install-folder` or `-if` option to every `conan install` command
in the next step.
2. Use conan to generate CMake files for every configuration you want to build:
2. Generate CMake files for every configuration you want to build.
```
conan install .. --output-folder . --build missing --settings build_type=Release
conan install .. --output-folder . --build missing --settings build_type=Debug
```
To build Debug, in the next step, be sure to set `-DCMAKE_BUILD_TYPE=Debug`
For a single-configuration generator, e.g. `Unix Makefiles` or `Ninja`,
you only need to run this command once.
For a multi-configuration generator, e.g. `Visual Studio`, you may want to
@@ -253,13 +196,13 @@ It fixes some source files to add missing `#include`s.
generated by the first. You can pass the build type on the command line with
`--settings build_type=$BUILD_TYPE` or in the profile itself,
under the section `[settings]` with the key `build_type`.
If you are using a Microsoft Visual C++ compiler,
then you will need to ensure consistency between the `build_type` setting
and the `compiler.runtime` setting.
When `build_type` is `Release`, `compiler.runtime` should be `MT`.
When `build_type` is `Debug`, `compiler.runtime` should be `MTd`.
```
@@ -271,32 +214,28 @@ It fixes some source files to add missing `#include`s.
`$OUTPUT_FOLDER/build/generators/conan_toolchain.cmake`.
Single-config generators:
```
cmake -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=Release ..
```
Pass the CMake variable [`CMAKE_BUILD_TYPE`][build_type]
and make sure it matches the one of the `build_type` settings
you chose in the previous step.
For example, to build Debug, in the next command, replace "Release" with "Debug"
```
cmake -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=Release -Dxrpld=ON -Dtests=ON ..
```
and make sure it matches the `build_type` setting you chose in the previous
step.
Multi-config generators:
```
cmake -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake -Dxrpld=ON -Dtests=ON ..
cmake -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake ..
```
**Note:** You can pass build options for `xahaud` in this step.
**Note:** You can pass build options for `rippled` in this step.
4. Build `xahaud`.
4. Build `rippled`.
For a single-configuration generator, it will build whatever configuration
you passed for `CMAKE_BUILD_TYPE`. For a multi-configuration generator,
you must pass the option `--config` to select the build configuration.
The output file is currently named 'rippled'.
Single-config generators:
@@ -305,13 +244,13 @@ It fixes some source files to add missing `#include`s.
```
Multi-config generators:
```
cmake --build . --config Release
cmake --build . --config Debug
```
5. Test xahaud.
5. Test rippled.
Single-config generators:
@@ -326,7 +265,7 @@ It fixes some source files to add missing `#include`s.
./Debug/rippled --unittest
```
The location of `xahaud` in your build directory depends on your CMake
The location of `rippled` in your build directory depends on your CMake
generator. Pass `--help` to see the rest of the command line options.
@@ -362,7 +301,7 @@ to select only specific tests for the purpose of the coverage report, by setting
the `coverage_test` variable in `cmake`
The default coverage report format is `html-details`, but the user
can override it to any of the formats listed in `cmake/CodeCoverage.cmake`
can override it to any of the formats listed in `Builds/CMake/CodeCoverage.cmake`
by setting the `coverage_format` variable in `cmake`. It is also possible
to generate more than one format at a time by setting the `coverage_extra_args`
variable in `cmake`. The specific command line used to run the `gcovr` tool will be
@@ -378,7 +317,7 @@ Example use with some cmake variables set:
```
cd .build
conan install .. --output-folder . --build missing --settings build_type=Debug
cmake -DCMAKE_BUILD_TYPE=Debug -Dcoverage=ON -Dxrpld=ON -Dtests=ON -Dcoverage_test_parallelism=2 -Dcoverage_format=html-details -Dcoverage_extra_args="--json coverage.json" -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake ..
cmake -DCMAKE_BUILD_TYPE=Debug -Dcoverage=ON -Dcoverage_test_parallelism=2 -Dcoverage_format=html-details -Dcoverage_extra_args="--json coverage.json" -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake ..
cmake --build . --target coverage
```
@@ -394,11 +333,11 @@ stored inside the build directory, as either of:
| Option | Default Value | Description |
| --- | ---| ---|
| `assert` | OFF | Enable assertions.
| `reporting` | OFF | Build the reporting mode feature. |
| `coverage` | OFF | Prepare the coverage report. |
| `san` | N/A | Enable a sanitizer with Clang. Choices are `thread` and `address`. |
| `tests` | OFF | Build tests. |
| `tests` | ON | Build tests. |
| `unity` | ON | Configure a unity build. |
| `xrpld` | OFF | Build the xrpld (`rippled`) application, and not just the libxrpl library. |
| `san` | N/A | Enable a sanitizer with Clang. Choices are `thread` and `address`. |
[Unity builds][5] may be faster for the first build
(at the cost of much more memory) since they concatenate sources into fewer
@@ -414,38 +353,23 @@ and can be helpful for detecting `#include` omissions.
If you have trouble building dependencies after changing Conan settings,
try removing the Conan cache.
For Conan 2:
```
rm -rf ~/.conan2/p
```
Or clear the entire Conan 2 cache:
```
conan cache clean "*"
rm -rf ~/.conan/data
```
### macOS compilation with Apple Clang 17+
### no std::result_of
If you're on macOS with Apple Clang 17 or newer, you need to add a compiler flag to work around a compilation error in gRPC dependencies.
Edit `~/.conan2/profiles/default` and add under the `[conf]` section:
If your compiler version is recent enough to have removed `std::result_of` as
part of C++20, e.g. Apple Clang 15.0, then you might need to add a preprocessor
definition to your build.
```
[conf]
tools.build:cxxflags=["-Wno-missing-template-arg-list-after-template-kw"]
```
### call to 'async_teardown' is ambiguous
If you are compiling with an early version of Clang 16, then you might hit
a [regression][6] when compiling C++20 that manifests as an [error in a Boost
header][7]. You can workaround it by adding this preprocessor definition:
```
conan profile update 'env.CXXFLAGS="-DBOOST_ASIO_DISABLE_CONCEPTS"' default
conan profile update 'conf.tools.build:cxxflags+=["-DBOOST_ASIO_DISABLE_CONCEPTS"]' default
conan profile update 'options.boost:extra_b2_flags="define=BOOST_ASIO_HAS_STD_INVOKE_RESULT"' default
conan profile update 'env.CFLAGS="-DBOOST_ASIO_HAS_STD_INVOKE_RESULT"' default
conan profile update 'env.CXXFLAGS="-DBOOST_ASIO_HAS_STD_INVOKE_RESULT"' default
conan profile update 'conf.tools.build:cflags+=["-DBOOST_ASIO_HAS_STD_INVOKE_RESULT"]' default
conan profile update 'conf.tools.build:cxxflags+=["-DBOOST_ASIO_HAS_STD_INVOKE_RESULT"]' default
```
@@ -613,4 +537,3 @@ but it is more convenient to put them in a [profile][profile].
[search]: https://cmake.org/cmake/help/latest/command/find_package.html#search-procedure
[prefix_path]: https://cmake.org/cmake/help/latest/variable/CMAKE_PREFIX_PATH.html
[profile]: https://docs.conan.io/en/latest/reference/profiles.html

View File

@@ -95,9 +95,6 @@
# - replace both functions setup_target_for_coverage_gcovr_* with a single setup_target_for_coverage_gcovr
# - add support for all gcovr output formats
#
# 2024-04-03, Bronek Kozicki
# - add support for output formats: jacoco, clover, lcov
#
# USAGE:
#
# 1. Copy this file into your cmake modules path.
@@ -259,10 +256,10 @@ endif()
# BASE_DIRECTORY "../" # Base directory for report
# # (defaults to PROJECT_SOURCE_DIR)
# FORMAT "cobertura" # Output format, one of:
# # xml cobertura sonarqube jacoco clover
# # json-summary json-details coveralls csv
# # txt html-single html-nested html-details
# # lcov (xml is an alias to cobertura;
# # xml cobertura sonarqube json-summary
# # json-details coveralls csv txt
# # html-single html-nested html-details
# # (xml is an alias to cobertura;
# # if no format is set, defaults to xml)
# EXCLUDE "src/dir1/*" "src/dir2/*" # Patterns to exclude (can be relative
# # to BASE_DIRECTORY, with CMake 3.4+)
@@ -311,8 +308,6 @@ function(setup_target_for_coverage_gcovr)
set(GCOVR_OUTPUT_FILE ${Coverage_NAME}.txt)
elseif(Coverage_FORMAT STREQUAL "csv")
set(GCOVR_OUTPUT_FILE ${Coverage_NAME}.csv)
elseif(Coverage_FORMAT STREQUAL "lcov")
set(GCOVR_OUTPUT_FILE ${Coverage_NAME}.lcov)
else()
set(GCOVR_OUTPUT_FILE ${Coverage_NAME}.xml)
endif()
@@ -325,14 +320,6 @@ function(setup_target_for_coverage_gcovr)
set(Coverage_FORMAT cobertura) # overwrite xml
elseif(Coverage_FORMAT STREQUAL "sonarqube")
list(APPEND GCOVR_ADDITIONAL_ARGS --sonarqube "${GCOVR_OUTPUT_FILE}" )
elseif(Coverage_FORMAT STREQUAL "jacoco")
list(APPEND GCOVR_ADDITIONAL_ARGS --jacoco "${GCOVR_OUTPUT_FILE}" )
list(APPEND GCOVR_ADDITIONAL_ARGS --jacoco-pretty )
elseif(Coverage_FORMAT STREQUAL "clover")
list(APPEND GCOVR_ADDITIONAL_ARGS --clover "${GCOVR_OUTPUT_FILE}" )
list(APPEND GCOVR_ADDITIONAL_ARGS --clover-pretty )
elseif(Coverage_FORMAT STREQUAL "lcov")
list(APPEND GCOVR_ADDITIONAL_ARGS --lcov "${GCOVR_OUTPUT_FILE}" )
elseif(Coverage_FORMAT STREQUAL "json-summary")
list(APPEND GCOVR_ADDITIONAL_ARGS --json-summary "${GCOVR_OUTPUT_FILE}" )
list(APPEND GCOVR_ADDITIONAL_ARGS --json-summary-pretty)
@@ -393,7 +380,6 @@ function(setup_target_for_coverage_gcovr)
${GCOVR_PATH}
--gcov-executable ${GCOV_TOOL}
--gcov-ignore-parse-errors=negative_hits.warn_once_per_file
--gcov-ignore-parse-errors=suspicious_hits.warn_once_per_file
-r ${BASEDIR}
${GCOVR_ADDITIONAL_ARGS}
${GCOVR_EXCLUDE_ARGS}

View File

@@ -17,9 +17,7 @@ target_compile_features (common INTERFACE cxx_std_20)
target_compile_definitions (common
INTERFACE
$<$<CONFIG:Debug>:DEBUG _DEBUG>
$<$<AND:$<BOOL:${profile}>,$<NOT:$<BOOL:${assert}>>>:NDEBUG>
# TODO: Remove once we have migrated functions from OpenSSL 1.x to 3.x.
OPENSSL_SUPPRESS_DEPRECATED)
$<$<AND:$<BOOL:${profile}>,$<NOT:$<BOOL:${assert}>>>:NDEBUG>)
# ^^^^ NOTE: CMAKE release builds already have NDEBUG
# defined, so no need to add it explicitly except for
# this special case of (profile ON) and (assert OFF)
@@ -122,7 +120,6 @@ else ()
target_link_libraries (common
INTERFACE
-rdynamic
$<$<BOOL:${is_linux}>:-Wl,-z,relro,-z,now,--build-id>
# link to static libc/c++ iff:
# * static option set and
# * NOT APPLE (AppleClang does not support static libc/c++) and
@@ -133,17 +130,6 @@ else ()
>)
endif ()
# Antithesis instrumentation will only be built and deployed using machines running Linux.
if (voidstar)
if (NOT CMAKE_BUILD_TYPE STREQUAL "Debug")
message(FATAL_ERROR "Antithesis instrumentation requires Debug build type, aborting...")
elseif (NOT is_linux)
message(FATAL_ERROR "Antithesis instrumentation requires Linux, aborting...")
elseif (NOT (is_clang AND CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 16.0))
message(FATAL_ERROR "Antithesis instrumentation requires Clang version 16 or later, aborting...")
endif ()
endif ()
if (use_mold)
# use mold linker if available
execute_process (
@@ -151,16 +137,6 @@ if (use_mold)
ERROR_QUIET OUTPUT_VARIABLE LD_VERSION)
if ("${LD_VERSION}" MATCHES "mold")
target_link_libraries (common INTERFACE -fuse-ld=mold)
else ()
# Checking for mold linker (< GCC 12.1.0)
execute_process (
COMMAND ${CMAKE_CXX_COMPILER} -B/usr/libexec/mold -Wl,--version
OUTPUT_VARIABLE LD_VERSION_OUT
ERROR_VARIABLE LD_VERSION_ERR)
set(LD_VERSION "${LD_VERSION_OUT}${LD_VERSION_ERR}")
if ("${LD_VERSION}" MATCHES "mold")
target_link_libraries (common INTERFACE -B/usr/libexec/mold)
endif ()
endif ()
unset (LD_VERSION)
elseif (use_gold AND is_gcc)

File diff suppressed because it is too large Load Diff

View File

@@ -33,6 +33,6 @@ setup_target_for_coverage_gcovr(
FORMAT ${coverage_format}
EXECUTABLE rippled
EXECUTABLE_ARGS --unittest$<$<BOOL:${coverage_test}>:=${coverage_test}> --unittest-jobs ${coverage_test_parallelism} --quiet --unittest-log
EXCLUDE "src/test" "include/xrpl/beast/test" "include/xrpl/beast/unit_test" "${CMAKE_BINARY_DIR}/pb-xrpl.libpb"
EXCLUDE "src/test" "${CMAKE_BINARY_DIR}/proto_gen" "${CMAKE_BINARY_DIR}/proto_gen_grpc"
DEPENDENCIES rippled
)

View File

@@ -26,7 +26,8 @@ if (tests)
src/ripple/*.cpp
src/ripple/*.md
src/test/*.h
src/test/*.md)
src/test/*.md
Builds/*/README.md)
list (APPEND doxygen_input
README.md
RELEASENOTES.md

View File

@@ -2,44 +2,20 @@
install stuff
#]===================================================================]
include(create_symbolic_link)
install (
TARGETS
common
opts
ripple_syslibs
ripple_boost
xrpl.imports.main
xrpl_core
xrpl.libpb
xrpl.libxrpl.basics
xrpl.libxrpl.beast
xrpl.libxrpl.hook
xrpl.libxrpl.crypto
xrpl.libxrpl.json
xrpl.libxrpl.protocol
xrpl.libxrpl.resource
xrpl.libxrpl.server
xrpl.libxrpl
antithesis-sdk-cpp
EXPORT RippleExports
LIBRARY DESTINATION lib
ARCHIVE DESTINATION lib
RUNTIME DESTINATION bin
INCLUDES DESTINATION include)
install(
DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/include/xrpl"
DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}"
)
install(CODE "
set(CMAKE_MODULE_PATH \"${CMAKE_MODULE_PATH}\")
include(create_symbolic_link)
create_symbolic_link(xrpl \
\${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_INCLUDEDIR}/ripple)
")
install (EXPORT RippleExports
FILE RippleTargets.cmake
NAMESPACE Ripple::
@@ -50,9 +26,14 @@ write_basic_package_version_file (
VERSION ${rippled_version}
COMPATIBILITY SameMajorVersion)
if (is_root_project AND TARGET rippled)
if (is_root_project)
install (TARGETS rippled RUNTIME DESTINATION bin)
set_target_properties(rippled PROPERTIES INSTALL_RPATH_USE_LINK_PATH ON)
install (
FILES
${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake/RippleConfig.cmake
${CMAKE_CURRENT_BINARY_DIR}/RippleConfigVersion.cmake
DESTINATION lib/cmake/ripple)
# sample configs should not overwrite existing files
# install if-not-exists workaround as suggested by
# https://cmake.org/Bug/view.php?id=12646
@@ -67,16 +48,4 @@ if (is_root_project AND TARGET rippled)
copy_if_not_exists(\"${CMAKE_CURRENT_SOURCE_DIR}/cfg/rippled-example.cfg\" etc rippled.cfg)
copy_if_not_exists(\"${CMAKE_CURRENT_SOURCE_DIR}/cfg/validators-example.txt\" etc validators.txt)
")
install(CODE "
set(CMAKE_MODULE_PATH \"${CMAKE_MODULE_PATH}\")
include(create_symbolic_link)
create_symbolic_link(rippled${suffix} \
\${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_BINDIR}/xrpld${suffix})
")
endif ()
install (
FILES
${CMAKE_CURRENT_SOURCE_DIR}/cmake/RippleConfig.cmake
${CMAKE_CURRENT_BINARY_DIR}/RippleConfigVersion.cmake
DESTINATION lib/cmake/ripple)

View File

@@ -7,9 +7,6 @@ add_library (Ripple::opts ALIAS opts)
target_compile_definitions (opts
INTERFACE
BOOST_ASIO_DISABLE_HANDLER_TYPE_REQUIREMENTS
BOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT
BOOST_CONTAINER_FWD_BAD_DEQUE
HAS_UNCAUGHT_EXCEPTIONS=1
$<$<BOOL:${boost_show_deprecated}>:
BOOST_ASIO_NO_DEPRECATED
BOOST_FILESYSTEM_NO_DEPRECATED
@@ -21,15 +18,10 @@ target_compile_definitions (opts
>
$<$<BOOL:${beast_no_unit_test_inline}>:BEAST_NO_UNIT_TEST_INLINE=1>
$<$<BOOL:${beast_disable_autolink}>:BEAST_DONT_AUTOLINK_TO_WIN32_LIBRARIES=1>
$<$<BOOL:${single_io_service_thread}>:RIPPLE_SINGLE_IO_SERVICE_THREAD=1>
# Enhanced logging is enabled for Debug builds, or explicitly via
# -DBEAST_ENHANCED_LOGGING=ON for other build types.
$<$<OR:$<CONFIG:Debug>,$<BOOL:${BEAST_ENHANCED_LOGGING}>>:BEAST_ENHANCED_LOGGING=1>
$<$<BOOL:${voidstar}>:ENABLE_VOIDSTAR>)
$<$<BOOL:${single_io_service_thread}>:RIPPLE_SINGLE_IO_SERVICE_THREAD=1>)
target_compile_options (opts
INTERFACE
$<$<AND:$<BOOL:${is_gcc}>,$<COMPILE_LANGUAGE:CXX>>:-Wsuggest-override>
$<$<BOOL:${is_gcc}>:-Wno-maybe-uninitialized>
$<$<BOOL:${perf}>:-fno-omit-frame-pointer>
$<$<AND:$<BOOL:${is_gcc}>,$<BOOL:${coverage}>>:-g --coverage -fprofile-abs-path>
$<$<AND:$<BOOL:${is_clang}>,$<BOOL:${coverage}>>:-g --coverage>

View File

@@ -0,0 +1,39 @@
#[===================================================================[
multiconfig misc
#]===================================================================]
if (is_multiconfig)
# This code finds all source files in the src subdirectory for inclusion
# in the IDE file tree as non-compiled sources. Since this file list will
# have some overlap with files we have already added to our targets to
# be compiled, we explicitly remove any of these target source files from
# this list.
file (GLOB_RECURSE all_sources RELATIVE ${CMAKE_CURRENT_SOURCE_DIR}
CONFIGURE_DEPENDS
src/*.* Builds/*.md docs/*.md src/*.md Builds/*.cmake)
file(GLOB md_files RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} CONFIGURE_DEPENDS
*.md)
LIST(APPEND all_sources ${md_files})
foreach (_target secp256k1::secp256k1 ed25519::ed25519 xrpl_core rippled)
get_target_property (_type ${_target} TYPE)
if(_type STREQUAL "INTERFACE_LIBRARY")
continue()
endif()
get_target_property (_src ${_target} SOURCES)
list (REMOVE_ITEM all_sources ${_src})
endforeach ()
target_sources (rippled PRIVATE ${all_sources})
set_property (
SOURCE ${all_sources}
APPEND
PROPERTY HEADER_FILE_ONLY true)
if (MSVC)
set_property(
DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
PROPERTY VS_STARTUP_PROJECT rippled)
endif ()
group_sources(src)
group_sources(docs)
group_sources(Builds)
endif ()

View File

@@ -0,0 +1,33 @@
#[===================================================================[
NIH prefix path..this is where we will download
and build any ExternalProjects, and they will hopefully
survive across build directory deletion (manual cleans)
#]===================================================================]
string (REGEX REPLACE "[ \\/%]+" "_" gen_for_path ${CMAKE_GENERATOR})
string (TOLOWER ${gen_for_path} gen_for_path)
# HACK: trying to shorten paths for windows CI (which hits 260 MAXPATH easily)
# @see: https://issues.jenkins-ci.org/browse/JENKINS-38706?focusedCommentId=339847
string (REPLACE "visual_studio" "vs" gen_for_path ${gen_for_path})
if (NOT DEFINED NIH_CACHE_ROOT)
if (DEFINED ENV{NIH_CACHE_ROOT})
set (NIH_CACHE_ROOT $ENV{NIH_CACHE_ROOT})
else ()
set (NIH_CACHE_ROOT "${CMAKE_CURRENT_SOURCE_DIR}/.nih_c")
endif ()
endif ()
set (nih_cache_path
"${NIH_CACHE_ROOT}/${gen_for_path}/${CMAKE_CXX_COMPILER_ID}_${CMAKE_CXX_COMPILER_VERSION}")
if (NOT is_multiconfig)
set (nih_cache_path "${nih_cache_path}/${CMAKE_BUILD_TYPE}")
endif ()
file(TO_CMAKE_PATH "${nih_cache_path}" nih_cache_path)
message (STATUS "NIH-EP cache path: ${nih_cache_path}")
## two convenience variables:
set (ep_lib_prefix ${CMAKE_STATIC_LIBRARY_PREFIX})
set (ep_lib_suffix ${CMAKE_STATIC_LIBRARY_SUFFIX})
# this is a setting for FetchContent and needs to be
# a cache variable
# https://cmake.org/cmake/help/latest/module/FetchContent.html#populating-the-content
set (FETCHCONTENT_BASE_DIR ${nih_cache_path} CACHE STRING "" FORCE)

View File

@@ -8,34 +8,15 @@ ProcessorCount(PROCESSOR_COUNT)
option(assert "Enables asserts, even in release builds" OFF)
option(xrpld "Build xrpld" ON)
option(reporting "Build rippled with reporting mode enabled" OFF)
option(tests "Build tests" ON)
option(xahaud_runtime_test_config
"Enable XAHAUD_RUNTIME_TEST_CONFIG env and runtime_config RPC fault-injection controls"
OFF)
# Conan 2 local opt-in:
# [conf]
# tools.cmake.cmaketoolchain:extra_variables={"xahaud_runtime_test_config":"ON"}
option(formal_verification
"Enable Lean-backed formal-verification cross-check tests"
OFF)
# Default off: this pulls the Lean runtime and the vendored formal model into
# the test binary. Conan/local opt-in mirrors the runtime-test-config pattern:
# [conf]
# tools.cmake.cmaketoolchain:extra_variables={"formal_verification":"ON"}
option(unity "Creates a build using UNITY support in cmake. This is the default" ON)
if(unity)
if(NOT is_ci)
set(CMAKE_UNITY_BUILD_BATCH_SIZE 15 CACHE STRING "")
endif()
set(CMAKE_UNITY_BUILD ON CACHE BOOL "Do a unity build")
endif()
if(is_clang AND is_linux)
option(voidstar "Enable Antithesis instrumentation." OFF)
endif()
if(is_gcc OR is_clang)
option(coverage "Generates coverage info." OFF)

View File

@@ -0,0 +1,15 @@
#[===================================================================[
read version from source
#]===================================================================]
file (STRINGS src/ripple/protocol/impl/BuildInfo.cpp BUILD_INFO)
foreach (line_ ${BUILD_INFO})
if (line_ MATCHES "versionString[ ]*=[ ]*\"(.+)\"")
set (rippled_version ${CMAKE_MATCH_1})
endif ()
endforeach ()
if (rippled_version)
message (STATUS "rippled version: ${rippled_version}")
else ()
message (FATAL_ERROR "unable to determine rippled version")
endif ()

View File

@@ -1,4 +1,4 @@
find_package(Boost 1.86 REQUIRED
find_package(Boost 1.83 REQUIRED
COMPONENTS
chrono
container
@@ -7,7 +7,6 @@ find_package(Boost 1.86 REQUIRED
date_time
filesystem
json
json
program_options
regex
system

View File

@@ -0,0 +1,102 @@
#[===================================================================[
NIH dep: boost
#]===================================================================]
if((NOT DEFINED BOOST_ROOT) AND(DEFINED ENV{BOOST_ROOT}))
set(BOOST_ROOT $ENV{BOOST_ROOT})
endif()
if((NOT DEFINED BOOST_LIBRARYDIR) AND(DEFINED ENV{BOOST_LIBRARYDIR}))
set(BOOST_LIBRARYDIR $ENV{BOOST_LIBRARYDIR})
endif()
file(TO_CMAKE_PATH "${BOOST_ROOT}" BOOST_ROOT)
if(WIN32 OR CYGWIN)
# Workaround for MSVC having two boost versions - x86 and x64 on same PC in stage folders
if((NOT DEFINED BOOST_LIBRARYDIR) AND (DEFINED BOOST_ROOT))
if(IS_DIRECTORY ${BOOST_ROOT}/stage64/lib)
set(BOOST_LIBRARYDIR ${BOOST_ROOT}/stage64/lib)
elseif(IS_DIRECTORY ${BOOST_ROOT}/stage/lib)
set(BOOST_LIBRARYDIR ${BOOST_ROOT}/stage/lib)
elseif(IS_DIRECTORY ${BOOST_ROOT}/lib)
set(BOOST_LIBRARYDIR ${BOOST_ROOT}/lib)
else()
message(WARNING "Did not find expected boost library dir. "
"Defaulting to ${BOOST_ROOT}")
set(BOOST_LIBRARYDIR ${BOOST_ROOT})
endif()
endif()
endif()
message(STATUS "BOOST_ROOT: ${BOOST_ROOT}")
message(STATUS "BOOST_LIBRARYDIR: ${BOOST_LIBRARYDIR}")
# uncomment the following as needed to debug FindBoost issues:
#set(Boost_DEBUG ON)
#[=========================================================[
boost dynamic libraries don't trivially support @rpath
linking right now (cmake's default), so just force
static linking for macos, or if requested on linux by flag
#]=========================================================]
if(static)
set(Boost_USE_STATIC_LIBS ON)
endif()
set(Boost_USE_MULTITHREADED ON)
if(static AND NOT APPLE)
set(Boost_USE_STATIC_RUNTIME ON)
else()
set(Boost_USE_STATIC_RUNTIME OFF)
endif()
# TBD:
# Boost_USE_DEBUG_RUNTIME: When ON, uses Boost libraries linked against the
find_package(Boost 1.86 REQUIRED
COMPONENTS
chrono
container
context
coroutine
date_time
filesystem
json
program_options
regex
system
iostreams
thread)
add_library(ripple_boost INTERFACE)
add_library(Ripple::boost ALIAS ripple_boost)
if(is_xcode)
target_include_directories(ripple_boost BEFORE INTERFACE ${Boost_INCLUDE_DIRS})
target_compile_options(ripple_boost INTERFACE --system-header-prefix="boost/")
else()
target_include_directories(ripple_boost SYSTEM BEFORE INTERFACE ${Boost_INCLUDE_DIRS})
endif()
target_link_libraries(ripple_boost
INTERFACE
Boost::boost
Boost::chrono
Boost::container
Boost::coroutine
Boost::date_time
Boost::filesystem
Boost::json
Boost::iostreams
Boost::program_options
Boost::regex
Boost::system
Boost::thread)
if(Boost_COMPILER)
target_link_libraries(ripple_boost INTERFACE Boost::disable_autolinking)
endif()
if(san AND is_clang)
# TODO: gcc does not support -fsanitize-blacklist...can we do something else
# for gcc ?
if(NOT Boost_INCLUDE_DIRS AND TARGET Boost::headers)
get_target_property(Boost_INCLUDE_DIRS Boost::headers INTERFACE_INCLUDE_DIRECTORIES)
endif()
message(STATUS "Adding [${Boost_INCLUDE_DIRS}] to sanitizer blacklist")
file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/san_bl.txt "src:${Boost_INCLUDE_DIRS}/*")
target_compile_options(opts
INTERFACE
# ignore boost headers for sanitizing
-fsanitize-blacklist=${CMAKE_CURRENT_BINARY_DIR}/san_bl.txt)
endif()

View File

@@ -0,0 +1,28 @@
#[===================================================================[
NIH dep: ed25519-donna
#]===================================================================]
add_library (ed25519-donna STATIC
external/ed25519-donna/ed25519.c)
target_include_directories (ed25519-donna
PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/external>
$<INSTALL_INTERFACE:include>
PRIVATE
${CMAKE_CURRENT_SOURCE_DIR}/external/ed25519-donna)
#[=========================================================[
NOTE for macos:
https://github.com/floodyberry/ed25519-donna/issues/29
our source for ed25519-donna-portable.h has been
patched to workaround this.
#]=========================================================]
target_link_libraries (ed25519-donna PUBLIC OpenSSL::SSL)
add_library (NIH::ed25519-donna ALIAS ed25519-donna)
target_link_libraries (ripple_libs INTERFACE NIH::ed25519-donna)
#[===========================[
headers installation
#]===========================]
install (
FILES
external/ed25519-donna/ed25519.h
DESTINATION include/ed25519-donna)

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,47 @@
# - Try to find jemalloc
# Once done this will define
# JEMALLOC_FOUND - System has jemalloc
# JEMALLOC_INCLUDE_DIRS - The jemalloc include directories
# JEMALLOC_LIBRARIES - The libraries needed to use jemalloc
if(NOT USE_BUNDLED_JEMALLOC)
find_package(PkgConfig)
if (PKG_CONFIG_FOUND)
pkg_check_modules(PC_JEMALLOC QUIET jemalloc)
endif()
else()
set(PC_JEMALLOC_INCLUDEDIR)
set(PC_JEMALLOC_INCLUDE_DIRS)
set(PC_JEMALLOC_LIBDIR)
set(PC_JEMALLOC_LIBRARY_DIRS)
set(LIMIT_SEARCH NO_DEFAULT_PATH)
endif()
set(JEMALLOC_DEFINITIONS ${PC_JEMALLOC_CFLAGS_OTHER})
find_path(JEMALLOC_INCLUDE_DIR jemalloc/jemalloc.h
PATHS ${PC_JEMALLOC_INCLUDEDIR} ${PC_JEMALLOC_INCLUDE_DIRS}
${LIMIT_SEARCH})
# If we're asked to use static linkage, add libjemalloc.a as a preferred library name.
if(JEMALLOC_USE_STATIC)
list(APPEND JEMALLOC_NAMES
"${CMAKE_STATIC_LIBRARY_PREFIX}jemalloc${CMAKE_STATIC_LIBRARY_SUFFIX}")
endif()
list(APPEND JEMALLOC_NAMES jemalloc)
find_library(JEMALLOC_LIBRARY NAMES ${JEMALLOC_NAMES}
HINTS ${PC_JEMALLOC_LIBDIR} ${PC_JEMALLOC_LIBRARY_DIRS}
${LIMIT_SEARCH})
set(JEMALLOC_LIBRARIES ${JEMALLOC_LIBRARY})
set(JEMALLOC_INCLUDE_DIRS ${JEMALLOC_INCLUDE_DIR})
include(FindPackageHandleStandardArgs)
# handle the QUIETLY and REQUIRED arguments and set JEMALLOC_FOUND to TRUE
# if all listed variables are TRUE
find_package_handle_standard_args(JeMalloc DEFAULT_MSG
JEMALLOC_LIBRARY JEMALLOC_INCLUDE_DIR)
mark_as_advanced(JEMALLOC_INCLUDE_DIR JEMALLOC_LIBRARY)

View File

@@ -0,0 +1,22 @@
find_package (PkgConfig REQUIRED)
pkg_search_module (libarchive_PC QUIET libarchive>=3.4.3)
if(static)
set(LIBARCHIVE_LIB libarchive.a)
else()
set(LIBARCHIVE_LIB archive)
endif()
find_library (archive
NAMES ${LIBARCHIVE_LIB}
HINTS
${libarchive_PC_LIBDIR}
${libarchive_PC_LIBRARY_DIRS}
NO_DEFAULT_PATH)
find_path (LIBARCHIVE_INCLUDE_DIR
NAMES archive.h
HINTS
${libarchive_PC_INCLUDEDIR}
${libarchive_PC_INCLUDEDIRS}
NO_DEFAULT_PATH)

View File

@@ -0,0 +1,24 @@
find_package (PkgConfig)
if (PKG_CONFIG_FOUND)
pkg_search_module (lz4_PC QUIET liblz4>=1.9)
endif ()
if(static)
set(LZ4_LIB liblz4.a)
else()
set(LZ4_LIB lz4.so)
endif()
find_library (lz4
NAMES ${LZ4_LIB}
HINTS
${lz4_PC_LIBDIR}
${lz4_PC_LIBRARY_DIRS}
NO_DEFAULT_PATH)
find_path (LZ4_INCLUDE_DIR
NAMES lz4.h
HINTS
${lz4_PC_INCLUDEDIR}
${lz4_PC_INCLUDEDIRS}
NO_DEFAULT_PATH)

View File

@@ -0,0 +1,24 @@
find_package (PkgConfig)
if (PKG_CONFIG_FOUND)
pkg_search_module (secp256k1_PC QUIET libsecp256k1)
endif ()
if(static)
set(SECP256K1_LIB libsecp256k1.a)
else()
set(SECP256K1_LIB secp256k1)
endif()
find_library(secp256k1
NAMES ${SECP256K1_LIB}
HINTS
${secp256k1_PC_LIBDIR}
${secp256k1_PC_LIBRARY_PATHS}
NO_DEFAULT_PATH)
find_path (SECP256K1_INCLUDE_DIR
NAMES secp256k1.h
HINTS
${secp256k1_PC_INCLUDEDIR}
${secp256k1_PC_INCLUDEDIRS}
NO_DEFAULT_PATH)

View File

@@ -0,0 +1,24 @@
find_package (PkgConfig)
if (PKG_CONFIG_FOUND)
pkg_search_module (snappy_PC QUIET snappy>=1.1.7)
endif ()
if(static)
set(SNAPPY_LIB libsnappy.a)
else()
set(SNAPPY_LIB libsnappy.so)
endif()
find_library (snappy
NAMES ${SNAPPY_LIB}
HINTS
${snappy_PC_LIBDIR}
${snappy_PC_LIBRARY_DIRS}
NO_DEFAULT_PATH)
find_path (SNAPPY_INCLUDE_DIR
NAMES snappy.h
HINTS
${snappy_PC_INCLUDEDIR}
${snappy_PC_INCLUDEDIRS}
NO_DEFAULT_PATH)

View File

@@ -0,0 +1,19 @@
find_package (PkgConfig)
if (PKG_CONFIG_FOUND)
# TBD - currently no soci pkgconfig
#pkg_search_module (soci_PC QUIET libsoci_core>=3.2)
endif ()
if(static)
set(SOCI_LIB libsoci.a)
else()
set(SOCI_LIB libsoci_core.so)
endif()
find_library (soci
NAMES ${SOCI_LIB})
find_path (SOCI_INCLUDE_DIR
NAMES soci/soci.h)
message("SOCI FOUND AT: ${SOCI_LIB}")

View File

@@ -0,0 +1,24 @@
find_package (PkgConfig)
if (PKG_CONFIG_FOUND)
pkg_search_module (sqlite_PC QUIET sqlite3>=3.26.0)
endif ()
if(static)
set(SQLITE_LIB libsqlite3.a)
else()
set(SQLITE_LIB sqlite3.so)
endif()
find_library (sqlite3
NAMES ${SQLITE_LIB}
HINTS
${sqlite_PC_LIBDIR}
${sqlite_PC_LIBRARY_DIRS}
NO_DEFAULT_PATH)
find_path (SQLITE_INCLUDE_DIR
NAMES sqlite3.h
HINTS
${sqlite_PC_INCLUDEDIR}
${sqlite_PC_INCLUDEDIRS}
NO_DEFAULT_PATH)

View File

@@ -0,0 +1,163 @@
#[===================================================================[
NIH dep: libarchive
#]===================================================================]
option (local_libarchive "use local build of libarchive." OFF)
add_library (archive_lib UNKNOWN IMPORTED GLOBAL)
if (NOT local_libarchive)
if (NOT WIN32)
find_package(libarchive_pc REQUIRED)
endif ()
if (archive)
message (STATUS "Found libarchive using pkg-config. Using ${archive}.")
set_target_properties (archive_lib PROPERTIES
IMPORTED_LOCATION_DEBUG
${archive}
IMPORTED_LOCATION_RELEASE
${archive}
INTERFACE_INCLUDE_DIRECTORIES
${LIBARCHIVE_INCLUDE_DIR})
# pkg-config can return extra info for static lib linking
# this is probably needed/useful generally, but apply
# to APPLE for now (mostly for homebrew)
if (APPLE AND static AND libarchive_PC_STATIC_LIBRARIES)
message(STATUS "NOTE: libarchive static libs: ${libarchive_PC_STATIC_LIBRARIES}")
# also, APPLE seems to need iconv...maybe linux does too (TBD)
target_link_libraries (archive_lib
INTERFACE iconv ${libarchive_PC_STATIC_LIBRARIES})
endif ()
else ()
## now try searching using the minimal find module that cmake provides
find_package(LibArchive 3.4.3 QUIET)
if (LibArchive_FOUND)
if (static)
# find module doesn't find static libs currently, so we re-search
get_filename_component(_loc ${LibArchive_LIBRARY} DIRECTORY)
find_library(_la_static
NAMES libarchive.a archive_static.lib archive.lib
PATHS ${_loc})
if (_la_static)
set (_la_lib ${_la_static})
else ()
message (WARNING "unable to find libarchive static lib - switching to local build")
set (local_libarchive ON CACHE BOOL "" FORCE)
endif ()
else ()
set (_la_lib ${LibArchive_LIBRARY})
endif ()
if (NOT local_libarchive)
message (STATUS "Found libarchive using module/config. Using ${_la_lib}.")
set_target_properties (archive_lib PROPERTIES
IMPORTED_LOCATION_DEBUG
${_la_lib}
IMPORTED_LOCATION_RELEASE
${_la_lib}
INTERFACE_INCLUDE_DIRECTORIES
${LibArchive_INCLUDE_DIRS})
endif ()
else ()
set (local_libarchive ON CACHE BOOL "" FORCE)
endif ()
endif ()
endif()
if (local_libarchive)
set (lib_post "")
if (MSVC)
set (lib_post "_static")
endif ()
ExternalProject_Add (libarchive
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/libarchive/libarchive.git
GIT_TAG v3.4.3
CMAKE_ARGS
# passing the compiler seems to be needed for windows CI, sadly
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
$<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
-DCMAKE_DEBUG_POSTFIX=_d
$<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
-DENABLE_LZ4=ON
-ULZ4_*
-DLZ4_INCLUDE_DIR=$<JOIN:$<TARGET_PROPERTY:lz4_lib,INTERFACE_INCLUDE_DIRECTORIES>,::>
# because we are building a static lib, this lz4 library doesn't
# actually matter since you can't generally link static libs to other static
# libs. The include files are needed, but the library itself is not (until
# we link our application, at which point we use the lz4 we built above).
# nonetheless, we need to provide a library to libarchive else it will
# NOT include lz4 support when configuring
-DLZ4_LIBRARY=$<IF:$<CONFIG:Debug>,$<TARGET_PROPERTY:lz4_lib,IMPORTED_LOCATION_DEBUG>,$<TARGET_PROPERTY:lz4_lib,IMPORTED_LOCATION_RELEASE>>
-DENABLE_WERROR=OFF
-DENABLE_TAR=OFF
-DENABLE_TAR_SHARED=OFF
-DENABLE_INSTALL=ON
-DENABLE_NETTLE=OFF
-DENABLE_OPENSSL=OFF
-DENABLE_LZO=OFF
-DENABLE_LZMA=OFF
-DENABLE_ZLIB=OFF
-DENABLE_BZip2=OFF
-DENABLE_LIBXML2=OFF
-DENABLE_EXPAT=OFF
-DENABLE_PCREPOSIX=OFF
-DENABLE_LibGCC=OFF
-DENABLE_CNG=OFF
-DENABLE_CPIO=OFF
-DENABLE_CPIO_SHARED=OFF
-DENABLE_CAT=OFF
-DENABLE_CAT_SHARED=OFF
-DENABLE_XATTR=OFF
-DENABLE_ACL=OFF
-DENABLE_ICONV=OFF
-DENABLE_TEST=OFF
-DENABLE_COVERAGE=OFF
$<$<BOOL:${MSVC}>:
"-DCMAKE_C_FLAGS=-GR -Gd -fp:precise -FS -MP"
"-DCMAKE_C_FLAGS_DEBUG=-MTd"
"-DCMAKE_C_FLAGS_RELEASE=-MT"
>
LIST_SEPARATOR ::
LOG_BUILD ON
LOG_CONFIGURE ON
BUILD_COMMAND
${CMAKE_COMMAND}
--build .
--config $<CONFIG>
--target archive_static
--parallel ${ep_procs}
$<$<BOOL:${is_multiconfig}>:
COMMAND
${CMAKE_COMMAND} -E copy
<BINARY_DIR>/libarchive/$<CONFIG>/${ep_lib_prefix}archive${lib_post}$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
<BINARY_DIR>/libarchive
>
TEST_COMMAND ""
INSTALL_COMMAND ""
DEPENDS lz4_lib
BUILD_BYPRODUCTS
<BINARY_DIR>/libarchive/${ep_lib_prefix}archive${lib_post}${ep_lib_suffix}
<BINARY_DIR>/libarchive/${ep_lib_prefix}archive${lib_post}_d${ep_lib_suffix}
)
ExternalProject_Get_Property (libarchive BINARY_DIR)
ExternalProject_Get_Property (libarchive SOURCE_DIR)
if (CMAKE_VERBOSE_MAKEFILE)
print_ep_logs (libarchive)
endif ()
file (MAKE_DIRECTORY ${SOURCE_DIR}/libarchive)
set_target_properties (archive_lib PROPERTIES
IMPORTED_LOCATION_DEBUG
${BINARY_DIR}/libarchive/${ep_lib_prefix}archive${lib_post}_d${ep_lib_suffix}
IMPORTED_LOCATION_RELEASE
${BINARY_DIR}/libarchive/${ep_lib_prefix}archive${lib_post}${ep_lib_suffix}
INTERFACE_INCLUDE_DIRECTORIES
${SOURCE_DIR}/libarchive
INTERFACE_COMPILE_DEFINITIONS
LIBARCHIVE_STATIC)
endif()
add_dependencies (archive_lib libarchive)
target_link_libraries (archive_lib INTERFACE lz4_lib)
target_link_libraries (ripple_libs INTERFACE archive_lib)
exclude_if_included (libarchive)
exclude_if_included (archive_lib)

View File

@@ -0,0 +1,79 @@
#[===================================================================[
NIH dep: lz4
#]===================================================================]
add_library (lz4_lib STATIC IMPORTED GLOBAL)
if (NOT WIN32)
find_package(lz4)
endif()
if(lz4)
set_target_properties (lz4_lib PROPERTIES
IMPORTED_LOCATION_DEBUG
${lz4}
IMPORTED_LOCATION_RELEASE
${lz4}
INTERFACE_INCLUDE_DIRECTORIES
${LZ4_INCLUDE_DIR})
else()
ExternalProject_Add (lz4
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/lz4/lz4.git
GIT_TAG v1.9.2
SOURCE_SUBDIR contrib/cmake_unofficial
CMAKE_ARGS
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
$<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
-DCMAKE_DEBUG_POSTFIX=_d
$<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
-DBUILD_STATIC_LIBS=ON
-DBUILD_SHARED_LIBS=OFF
$<$<BOOL:${MSVC}>:
"-DCMAKE_C_FLAGS=-GR -Gd -fp:precise -FS -MP"
"-DCMAKE_C_FLAGS_DEBUG=-MTd"
"-DCMAKE_C_FLAGS_RELEASE=-MT"
>
LOG_BUILD ON
LOG_CONFIGURE ON
BUILD_COMMAND
${CMAKE_COMMAND}
--build .
--config $<CONFIG>
--target lz4_static
--parallel ${ep_procs}
$<$<BOOL:${is_multiconfig}>:
COMMAND
${CMAKE_COMMAND} -E copy
<BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}lz4$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
<BINARY_DIR>
>
TEST_COMMAND ""
INSTALL_COMMAND ""
BUILD_BYPRODUCTS
<BINARY_DIR>/${ep_lib_prefix}lz4${ep_lib_suffix}
<BINARY_DIR>/${ep_lib_prefix}lz4_d${ep_lib_suffix}
)
ExternalProject_Get_Property (lz4 BINARY_DIR)
ExternalProject_Get_Property (lz4 SOURCE_DIR)
file (MAKE_DIRECTORY ${SOURCE_DIR}/lz4)
set_target_properties (lz4_lib PROPERTIES
IMPORTED_LOCATION_DEBUG
${BINARY_DIR}/${ep_lib_prefix}lz4_d${ep_lib_suffix}
IMPORTED_LOCATION_RELEASE
${BINARY_DIR}/${ep_lib_prefix}lz4${ep_lib_suffix}
INTERFACE_INCLUDE_DIRECTORIES
${SOURCE_DIR}/lib)
if (CMAKE_VERBOSE_MAKEFILE)
print_ep_logs (lz4)
endif ()
add_dependencies (lz4_lib lz4)
target_link_libraries (ripple_libs INTERFACE lz4_lib)
exclude_if_included (lz4)
endif()
exclude_if_included (lz4_lib)

View File

@@ -0,0 +1,31 @@
#[===================================================================[
NIH dep: nudb
NuDB is header-only, thus is an INTERFACE lib in CMake.
TODO: move the library definition into NuDB repo and add
proper targets and export/install
#]===================================================================]
if (is_root_project) # NuDB not needed in the case of xrpl_core inclusion build
add_library (nudb INTERFACE)
FetchContent_Declare(
nudb_src
GIT_REPOSITORY https://github.com/CPPAlliance/NuDB.git
GIT_TAG 2.0.5
)
FetchContent_GetProperties(nudb_src)
if(NOT nudb_src_POPULATED)
message (STATUS "Pausing to download NuDB...")
FetchContent_Populate(nudb_src)
endif()
file(TO_CMAKE_PATH "${nudb_src_SOURCE_DIR}" nudb_src_SOURCE_DIR)
# specify as system includes so as to avoid warnings
target_include_directories (nudb SYSTEM INTERFACE ${nudb_src_SOURCE_DIR}/include)
target_link_libraries (nudb
INTERFACE
Boost::thread
Boost::system)
add_library (NIH::nudb ALIAS nudb)
target_link_libraries (ripple_libs INTERFACE NIH::nudb)
endif ()

View File

@@ -0,0 +1,48 @@
#[===================================================================[
NIH dep: openssl
#]===================================================================]
#[===============================================[
OPENSSL_ROOT_DIR is the only variable that
FindOpenSSL honors for locating, so convert any
OPENSSL_ROOT vars to this
#]===============================================]
if (NOT DEFINED OPENSSL_ROOT_DIR)
if (DEFINED ENV{OPENSSL_ROOT})
set (OPENSSL_ROOT_DIR $ENV{OPENSSL_ROOT})
elseif (HOMEBREW)
execute_process (COMMAND ${HOMEBREW} --prefix openssl
OUTPUT_VARIABLE OPENSSL_ROOT_DIR
OUTPUT_STRIP_TRAILING_WHITESPACE)
endif ()
file (TO_CMAKE_PATH "${OPENSSL_ROOT_DIR}" OPENSSL_ROOT_DIR)
endif ()
if (static)
set (OPENSSL_USE_STATIC_LIBS ON)
endif ()
set (OPENSSL_MSVC_STATIC_RT ON)
find_package (OpenSSL 1.1.1 REQUIRED)
target_link_libraries (ripple_libs
INTERFACE
OpenSSL::SSL
OpenSSL::Crypto)
# disable SSLv2...this can also be done when building/configuring OpenSSL
set_target_properties(OpenSSL::SSL PROPERTIES
INTERFACE_COMPILE_DEFINITIONS OPENSSL_NO_SSL2)
#[=========================================================[
https://gitlab.kitware.com/cmake/cmake/issues/16885
depending on how openssl is built, it might depend
on zlib. In fact, the openssl find package should
figure this out for us, but it does not currently...
so let's add zlib ourselves to the lib list
TODO: investigate linking to static zlib for static
build option
#]=========================================================]
find_package (ZLIB)
set (has_zlib FALSE)
if (TARGET ZLIB::ZLIB)
set_target_properties(OpenSSL::Crypto PROPERTIES
INTERFACE_LINK_LIBRARIES ZLIB::ZLIB)
set (has_zlib TRUE)
endif ()

View File

@@ -0,0 +1,70 @@
if(reporting)
find_package(PostgreSQL)
if(NOT PostgreSQL_FOUND)
message("find_package did not find postgres")
find_library(postgres NAMES pq libpq libpq-dev pq-dev postgresql-devel)
find_path(libpq-fe NAMES libpq-fe.h PATH_SUFFIXES postgresql pgsql include)
if(NOT libpq-fe_FOUND OR NOT postgres_FOUND)
message("No system installed Postgres found. Will build")
add_library(postgres SHARED IMPORTED GLOBAL)
add_library(pgport SHARED IMPORTED GLOBAL)
add_library(pgcommon SHARED IMPORTED GLOBAL)
ExternalProject_Add(postgres_src
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/postgres/postgres.git
GIT_TAG REL_14_5
CONFIGURE_COMMAND ./configure --without-readline > /dev/null
BUILD_COMMAND ${CMAKE_COMMAND} -E env --unset=MAKELEVEL make
UPDATE_COMMAND ""
BUILD_IN_SOURCE 1
INSTALL_COMMAND ""
BUILD_BYPRODUCTS
<BINARY_DIR>/src/interfaces/libpq/${ep_lib_prefix}pq.a
<BINARY_DIR>/src/common/${ep_lib_prefix}pgcommon.a
<BINARY_DIR>/src/port/${ep_lib_prefix}pgport.a
LOG_BUILD TRUE
)
ExternalProject_Get_Property (postgres_src SOURCE_DIR)
ExternalProject_Get_Property (postgres_src BINARY_DIR)
set (postgres_src_SOURCE_DIR "${SOURCE_DIR}")
file (MAKE_DIRECTORY ${postgres_src_SOURCE_DIR})
list(APPEND INCLUDE_DIRS
${SOURCE_DIR}/src/include
${SOURCE_DIR}/src/interfaces/libpq
)
set_target_properties(postgres PROPERTIES
IMPORTED_LOCATION
${BINARY_DIR}/src/interfaces/libpq/${ep_lib_prefix}pq.a
INTERFACE_INCLUDE_DIRECTORIES
"${INCLUDE_DIRS}"
)
set_target_properties(pgcommon PROPERTIES
IMPORTED_LOCATION
${BINARY_DIR}/src/common/${ep_lib_prefix}pgcommon.a
INTERFACE_INCLUDE_DIRECTORIES
"${INCLUDE_DIRS}"
)
set_target_properties(pgport PROPERTIES
IMPORTED_LOCATION
${BINARY_DIR}/src/port/${ep_lib_prefix}pgport.a
INTERFACE_INCLUDE_DIRECTORIES
"${INCLUDE_DIRS}"
)
add_dependencies(postgres postgres_src)
add_dependencies(pgcommon postgres_src)
add_dependencies(pgport postgres_src)
file(TO_CMAKE_PATH "${postgres_src_SOURCE_DIR}" postgres_src_SOURCE_DIR)
target_link_libraries(ripple_libs INTERFACE postgres pgcommon pgport)
else()
message("Found system installed Postgres via find_libary")
target_include_directories(ripple_libs INTERFACE ${libpq-fe})
target_link_libraries(ripple_libs INTERFACE ${postgres})
endif()
else()
message("Found system installed Postgres via find_package")
target_include_directories(ripple_libs INTERFACE ${PostgreSQL_INCLUDE_DIRS})
target_link_libraries(ripple_libs INTERFACE ${PostgreSQL_LIBRARIES})
endif()
endif()

View File

@@ -0,0 +1,156 @@
#[===================================================================[
import protobuf (lib and compiler) and create a lib
from our proto message definitions. If the system protobuf
is not found, fallback on EP to download and build a version
from official source.
#]===================================================================]
if (static)
set (Protobuf_USE_STATIC_LIBS ON)
endif ()
find_package (Protobuf 3.8)
if (is_multiconfig)
set(protobuf_protoc_lib ${Protobuf_PROTOC_LIBRARIES})
else ()
string(TOUPPER ${CMAKE_BUILD_TYPE} upper_cmake_build_type)
set(protobuf_protoc_lib ${Protobuf_PROTOC_LIBRARY_${upper_cmake_build_type}})
endif ()
if (local_protobuf OR NOT (Protobuf_FOUND AND Protobuf_PROTOC_EXECUTABLE AND protobuf_protoc_lib))
include (GNUInstallDirs)
message (STATUS "using local protobuf build.")
set(protobuf_reqs Protobuf_PROTOC_EXECUTABLE protobuf_protoc_lib)
foreach(lib ${protobuf_reqs})
if(NOT ${lib})
message(STATUS "Couldn't find ${lib}")
endif()
endforeach()
if (WIN32)
# protobuf prepends lib even on windows
set (pbuf_lib_pre "lib")
else ()
set (pbuf_lib_pre ${ep_lib_prefix})
endif ()
# for the external project build of protobuf, we currently ignore the
# static option and always build static libs here. This is consistent
# with our other EP builds. Dynamic libs in an EP would add complexity
# because we'd need to get them into the runtime path, and probably
# install them.
ExternalProject_Add (protobuf_src
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/protocolbuffers/protobuf.git
GIT_TAG v3.8.0
SOURCE_SUBDIR cmake
CMAKE_ARGS
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
-DCMAKE_INSTALL_PREFIX=<BINARY_DIR>/_installed_
-Dprotobuf_BUILD_TESTS=OFF
-Dprotobuf_BUILD_EXAMPLES=OFF
-Dprotobuf_BUILD_PROTOC_BINARIES=ON
-Dprotobuf_MSVC_STATIC_RUNTIME=ON
-DBUILD_SHARED_LIBS=OFF
-Dprotobuf_BUILD_SHARED_LIBS=OFF
-DCMAKE_DEBUG_POSTFIX=_d
-Dprotobuf_DEBUG_POSTFIX=_d
-Dprotobuf_WITH_ZLIB=$<IF:$<BOOL:${has_zlib}>,ON,OFF>
$<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
$<$<BOOL:${unity}>:-DCMAKE_UNITY_BUILD=ON}>
$<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
$<$<BOOL:${MSVC}>:
"-DCMAKE_CXX_FLAGS=-GR -Gd -fp:precise -FS -EHa -MP"
>
LOG_BUILD ON
LOG_CONFIGURE ON
BUILD_COMMAND
${CMAKE_COMMAND}
--build .
--config $<CONFIG>
--parallel ${ep_procs}
TEST_COMMAND ""
INSTALL_COMMAND
${CMAKE_COMMAND} -E env --unset=DESTDIR ${CMAKE_COMMAND} --build . --config $<CONFIG> --target install
BUILD_BYPRODUCTS
<BINARY_DIR>/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protobuf${ep_lib_suffix}
<BINARY_DIR>/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protobuf_d${ep_lib_suffix}
<BINARY_DIR>/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protoc${ep_lib_suffix}
<BINARY_DIR>/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protoc_d${ep_lib_suffix}
<BINARY_DIR>/_installed_/bin/protoc${CMAKE_EXECUTABLE_SUFFIX}
)
ExternalProject_Get_Property (protobuf_src BINARY_DIR)
ExternalProject_Get_Property (protobuf_src SOURCE_DIR)
if (CMAKE_VERBOSE_MAKEFILE)
print_ep_logs (protobuf_src)
endif ()
exclude_if_included (protobuf_src)
if (NOT TARGET protobuf::libprotobuf)
add_library (protobuf::libprotobuf STATIC IMPORTED GLOBAL)
endif ()
file (MAKE_DIRECTORY ${BINARY_DIR}/_installed_/include)
set_target_properties (protobuf::libprotobuf PROPERTIES
IMPORTED_LOCATION_DEBUG
${BINARY_DIR}/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protobuf_d${ep_lib_suffix}
IMPORTED_LOCATION_RELEASE
${BINARY_DIR}/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protobuf${ep_lib_suffix}
INTERFACE_INCLUDE_DIRECTORIES
${BINARY_DIR}/_installed_/include)
add_dependencies (protobuf::libprotobuf protobuf_src)
exclude_if_included (protobuf::libprotobuf)
if (NOT TARGET protobuf::libprotoc)
add_library (protobuf::libprotoc STATIC IMPORTED GLOBAL)
endif ()
set_target_properties (protobuf::libprotoc PROPERTIES
IMPORTED_LOCATION_DEBUG
${BINARY_DIR}/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protoc_d${ep_lib_suffix}
IMPORTED_LOCATION_RELEASE
${BINARY_DIR}/_installed_/${CMAKE_INSTALL_LIBDIR}/${pbuf_lib_pre}protoc${ep_lib_suffix}
INTERFACE_INCLUDE_DIRECTORIES
${BINARY_DIR}/_installed_/include)
add_dependencies (protobuf::libprotoc protobuf_src)
exclude_if_included (protobuf::libprotoc)
if (NOT TARGET protobuf::protoc)
add_executable (protobuf::protoc IMPORTED)
exclude_if_included (protobuf::protoc)
endif ()
set_target_properties (protobuf::protoc PROPERTIES
IMPORTED_LOCATION "${BINARY_DIR}/_installed_/bin/protoc${CMAKE_EXECUTABLE_SUFFIX}")
add_dependencies (protobuf::protoc protobuf_src)
else ()
if (NOT TARGET protobuf::protoc)
if (EXISTS "${Protobuf_PROTOC_EXECUTABLE}")
add_executable (protobuf::protoc IMPORTED)
set_target_properties (protobuf::protoc PROPERTIES
IMPORTED_LOCATION "${Protobuf_PROTOC_EXECUTABLE}")
else ()
message (FATAL_ERROR "Protobuf import failed")
endif ()
endif ()
endif ()
set(output_dir ${CMAKE_BINARY_DIR}/proto_gen)
file(MAKE_DIRECTORY ${output_dir})
set(ccbd ${CMAKE_CURRENT_BINARY_DIR})
set(CMAKE_CURRENT_BINARY_DIR ${output_dir})
protobuf_generate_cpp(PROTO_SRCS PROTO_HDRS src/ripple/proto/ripple.proto)
set(CMAKE_CURRENT_BINARY_DIR ${ccbd})
target_include_directories(xrpl_core SYSTEM PUBLIC
# The generated implementation imports the header relative to the output
# directory.
$<BUILD_INTERFACE:${output_dir}>
$<BUILD_INTERFACE:${output_dir}/src>
)
target_sources(xrpl_core PRIVATE ${output_dir}/src/ripple/proto/ripple.pb.cc)
install(
FILES ${output_dir}/src/ripple/proto/ripple.pb.h
DESTINATION include/ripple/proto)
target_link_libraries(xrpl_core PUBLIC protobuf::libprotobuf)
target_compile_options(xrpl_core
PUBLIC
$<$<BOOL:${is_xcode}>:
--system-header-prefix="google/protobuf"
-Wno-deprecated-dynamic-exception-spec
>
)

View File

@@ -0,0 +1,177 @@
#[===================================================================[
NIH dep: rocksdb
#]===================================================================]
add_library (rocksdb_lib UNKNOWN IMPORTED GLOBAL)
set_target_properties (rocksdb_lib
PROPERTIES INTERFACE_COMPILE_DEFINITIONS RIPPLE_ROCKSDB_AVAILABLE=1)
option (local_rocksdb "use local build of rocksdb." OFF)
if (NOT local_rocksdb)
find_package (RocksDB 6.27 QUIET CONFIG)
if (TARGET RocksDB::rocksdb)
message (STATUS "Found RocksDB using config.")
get_target_property (_rockslib_l RocksDB::rocksdb IMPORTED_LOCATION_DEBUG)
if (_rockslib_l)
set_target_properties (rocksdb_lib PROPERTIES IMPORTED_LOCATION_DEBUG ${_rockslib_l})
endif ()
get_target_property (_rockslib_l RocksDB::rocksdb IMPORTED_LOCATION_RELEASE)
if (_rockslib_l)
set_target_properties (rocksdb_lib PROPERTIES IMPORTED_LOCATION_RELEASE ${_rockslib_l})
endif ()
get_target_property (_rockslib_l RocksDB::rocksdb IMPORTED_LOCATION)
if (_rockslib_l)
set_target_properties (rocksdb_lib PROPERTIES IMPORTED_LOCATION ${_rockslib_l})
endif ()
get_target_property (_rockslib_i RocksDB::rocksdb INTERFACE_INCLUDE_DIRECTORIES)
if (_rockslib_i)
set_target_properties (rocksdb_lib PROPERTIES INTERFACE_INCLUDE_DIRECTORIES ${_rockslib_i})
endif ()
target_link_libraries (ripple_libs INTERFACE RocksDB::rocksdb)
else ()
# using a find module with rocksdb is difficult because
# you have no idea how it was configured (transitive dependencies).
# the code below will generally find rocksdb using the module, but
# will then result in linker errors for static linkage since the
# transitive dependencies are unknown. force local build here for now, but leave the code as
# a placeholder for future investigation.
if (static)
set (local_rocksdb ON CACHE BOOL "" FORCE)
# TBD if there is some way to extract transitive deps..then:
#set (RocksDB_USE_STATIC ON)
else ()
find_package (RocksDB 6.27 MODULE)
if (ROCKSDB_FOUND)
if (RocksDB_LIBRARY_DEBUG)
set_target_properties (rocksdb_lib PROPERTIES IMPORTED_LOCATION_DEBUG ${RocksDB_LIBRARY_DEBUG})
endif ()
set_target_properties (rocksdb_lib PROPERTIES IMPORTED_LOCATION_RELEASE ${RocksDB_LIBRARIES})
set_target_properties (rocksdb_lib PROPERTIES IMPORTED_LOCATION ${RocksDB_LIBRARIES})
set_target_properties (rocksdb_lib PROPERTIES INTERFACE_INCLUDE_DIRECTORIES ${RocksDB_INCLUDE_DIRS})
else ()
set (local_rocksdb ON CACHE BOOL "" FORCE)
endif ()
endif ()
endif ()
endif ()
if (local_rocksdb)
message (STATUS "Using local build of RocksDB.")
ExternalProject_Add (rocksdb
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/facebook/rocksdb.git
GIT_TAG v6.27.3
PATCH_COMMAND
# only used by windows build
${CMAKE_COMMAND} -E copy_if_different
${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake/rocks_thirdparty.inc
<SOURCE_DIR>/thirdparty.inc
COMMAND
# fixup their build version file to keep the values
# from changing always
${CMAKE_COMMAND} -E copy_if_different
${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake/rocksdb_build_version.cc.in
<SOURCE_DIR>/util/build_version.cc.in
CMAKE_ARGS
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
$<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
$<$<BOOL:${unity}>:-DCMAKE_UNITY_BUILD=ON}>
-DCMAKE_DEBUG_POSTFIX=_d
$<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
-DBUILD_SHARED_LIBS=OFF
-DCMAKE_POSITION_INDEPENDENT_CODE=ON
-DWITH_JEMALLOC=$<IF:$<BOOL:${jemalloc}>,ON,OFF>
-DWITH_SNAPPY=ON
-DWITH_LZ4=ON
-DWITH_ZLIB=OFF
-DUSE_RTTI=ON
-DWITH_ZSTD=OFF
-DWITH_GFLAGS=OFF
-DWITH_BZ2=OFF
-ULZ4_*
-Ulz4_*
-Dlz4_INCLUDE_DIRS=$<JOIN:$<TARGET_PROPERTY:lz4_lib,INTERFACE_INCLUDE_DIRECTORIES>,::>
-Dlz4_LIBRARIES=$<IF:$<CONFIG:Debug>,$<TARGET_PROPERTY:lz4_lib,IMPORTED_LOCATION_DEBUG>,$<TARGET_PROPERTY:lz4_lib,IMPORTED_LOCATION_RELEASE>>
-Dlz4_FOUND=ON
-USNAPPY_*
-Usnappy_*
-USnappy_*
-Dsnappy_INCLUDE_DIRS=$<JOIN:$<TARGET_PROPERTY:snappy_lib,INTERFACE_INCLUDE_DIRECTORIES>,::>
-Dsnappy_LIBRARIES=$<IF:$<CONFIG:Debug>,$<TARGET_PROPERTY:snappy_lib,IMPORTED_LOCATION_DEBUG>,$<TARGET_PROPERTY:snappy_lib,IMPORTED_LOCATION_RELEASE>>
-Dsnappy_FOUND=ON
-DSnappy_INCLUDE_DIRS=$<JOIN:$<TARGET_PROPERTY:snappy_lib,INTERFACE_INCLUDE_DIRECTORIES>,::>
-DSnappy_LIBRARIES=$<IF:$<CONFIG:Debug>,$<TARGET_PROPERTY:snappy_lib,IMPORTED_LOCATION_DEBUG>,$<TARGET_PROPERTY:snappy_lib,IMPORTED_LOCATION_RELEASE>>
-DSnappy_FOUND=ON
-DWITH_MD_LIBRARY=OFF
-DWITH_RUNTIME_DEBUG=$<IF:$<CONFIG:Debug>,ON,OFF>
-DFAIL_ON_WARNINGS=OFF
-DWITH_ASAN=OFF
-DWITH_TSAN=OFF
-DWITH_UBSAN=OFF
-DWITH_NUMA=OFF
-DWITH_TBB=OFF
-DWITH_WINDOWS_UTF8_FILENAMES=OFF
-DWITH_XPRESS=OFF
-DPORTABLE=ON
-DFORCE_SSE42=OFF
-DDISABLE_STALL_NOTIF=OFF
-DOPTDBG=ON
-DROCKSDB_LITE=OFF
-DWITH_FALLOCATE=ON
-DWITH_LIBRADOS=OFF
-DWITH_JNI=OFF
-DROCKSDB_INSTALL_ON_WINDOWS=OFF
-DWITH_TESTS=OFF
-DWITH_TOOLS=OFF
$<$<BOOL:${MSVC}>:
"-DCMAKE_CXX_FLAGS=-GR -Gd -fp:precise -FS -MP /DNDEBUG"
>
$<$<NOT:$<BOOL:${MSVC}>>:
"-DCMAKE_CXX_FLAGS=-DNDEBUG"
>
LOG_BUILD ON
LOG_CONFIGURE ON
BUILD_COMMAND
${CMAKE_COMMAND}
--build .
--config $<CONFIG>
--parallel ${ep_procs}
$<$<BOOL:${is_multiconfig}>:
COMMAND
${CMAKE_COMMAND} -E copy
<BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}rocksdb$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
<BINARY_DIR>
>
LIST_SEPARATOR ::
TEST_COMMAND ""
INSTALL_COMMAND ""
DEPENDS snappy_lib lz4_lib
BUILD_BYPRODUCTS
<BINARY_DIR>/${ep_lib_prefix}rocksdb${ep_lib_suffix}
<BINARY_DIR>/${ep_lib_prefix}rocksdb_d${ep_lib_suffix}
)
ExternalProject_Get_Property (rocksdb BINARY_DIR)
ExternalProject_Get_Property (rocksdb SOURCE_DIR)
if (CMAKE_VERBOSE_MAKEFILE)
print_ep_logs (rocksdb)
endif ()
file (MAKE_DIRECTORY ${SOURCE_DIR}/include)
set_target_properties (rocksdb_lib PROPERTIES
IMPORTED_LOCATION_DEBUG
${BINARY_DIR}/${ep_lib_prefix}rocksdb_d${ep_lib_suffix}
IMPORTED_LOCATION_RELEASE
${BINARY_DIR}/${ep_lib_prefix}rocksdb${ep_lib_suffix}
INTERFACE_INCLUDE_DIRECTORIES
${SOURCE_DIR}/include)
add_dependencies (rocksdb_lib rocksdb)
exclude_if_included (rocksdb)
endif ()
target_link_libraries (rocksdb_lib
INTERFACE
snappy_lib
lz4_lib
$<$<BOOL:${MSVC}>:rpcrt4>)
exclude_if_included (rocksdb_lib)
target_link_libraries (ripple_libs INTERFACE rocksdb_lib)

View File

@@ -0,0 +1,58 @@
#[===================================================================[
NIH dep: secp256k1
#]===================================================================]
add_library (secp256k1_lib STATIC IMPORTED GLOBAL)
if (NOT WIN32)
find_package(secp256k1)
endif()
if(secp256k1)
set_target_properties (secp256k1_lib PROPERTIES
IMPORTED_LOCATION_DEBUG
${secp256k1}
IMPORTED_LOCATION_RELEASE
${secp256k1}
INTERFACE_INCLUDE_DIRECTORIES
${SECP256K1_INCLUDE_DIR})
add_library (secp256k1 ALIAS secp256k1_lib)
add_library (NIH::secp256k1 ALIAS secp256k1_lib)
else()
set(INSTALL_SECP256K1 true)
add_library (secp256k1 STATIC
external/secp256k1/src/secp256k1.c)
target_compile_definitions (secp256k1
PRIVATE
USE_NUM_NONE
USE_FIELD_10X26
USE_FIELD_INV_BUILTIN
USE_SCALAR_8X32
USE_SCALAR_INV_BUILTIN)
target_include_directories (secp256k1
PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/external>
$<INSTALL_INTERFACE:include>
PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/external/secp256k1)
target_compile_options (secp256k1
PRIVATE
$<$<BOOL:${MSVC}>:-wd4319>
$<$<NOT:$<BOOL:${MSVC}>>:
-Wno-deprecated-declarations
-Wno-unused-function
>
$<$<BOOL:${is_gcc}>:-Wno-nonnull-compare>)
target_link_libraries (ripple_libs INTERFACE NIH::secp256k1)
#[===========================[
headers installation
#]===========================]
install (
FILES
external/secp256k1/include/secp256k1.h
DESTINATION include/secp256k1/include)
add_library (NIH::secp256k1 ALIAS secp256k1)
endif()

View File

@@ -0,0 +1,77 @@
#[===================================================================[
NIH dep: snappy
#]===================================================================]
add_library (snappy_lib STATIC IMPORTED GLOBAL)
if (NOT WIN32)
find_package(snappy)
endif()
if(snappy)
set_target_properties (snappy_lib PROPERTIES
IMPORTED_LOCATION_DEBUG
${snappy}
IMPORTED_LOCATION_RELEASE
${snappy}
INTERFACE_INCLUDE_DIRECTORIES
${SNAPPY_INCLUDE_DIR})
else()
ExternalProject_Add (snappy
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/google/snappy.git
GIT_TAG 1.1.7
CMAKE_ARGS
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
$<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
-DCMAKE_DEBUG_POSTFIX=_d
$<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
-DBUILD_SHARED_LIBS=OFF
-DCMAKE_POSITION_INDEPENDENT_CODE=ON
-DSNAPPY_BUILD_TESTS=OFF
$<$<BOOL:${MSVC}>:
"-DCMAKE_CXX_FLAGS=-GR -Gd -fp:precise -FS -EHa -MP"
"-DCMAKE_CXX_FLAGS_DEBUG=-MTd"
"-DCMAKE_CXX_FLAGS_RELEASE=-MT"
>
LOG_BUILD ON
LOG_CONFIGURE ON
BUILD_COMMAND
${CMAKE_COMMAND}
--build .
--config $<CONFIG>
--parallel ${ep_procs}
$<$<BOOL:${is_multiconfig}>:
COMMAND
${CMAKE_COMMAND} -E copy
<BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}snappy$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
<BINARY_DIR>
>
TEST_COMMAND ""
INSTALL_COMMAND
${CMAKE_COMMAND} -E copy_if_different <BINARY_DIR>/config.h <BINARY_DIR>/snappy-stubs-public.h <SOURCE_DIR>
BUILD_BYPRODUCTS
<BINARY_DIR>/${ep_lib_prefix}snappy${ep_lib_suffix}
<BINARY_DIR>/${ep_lib_prefix}snappy_d${ep_lib_suffix}
)
ExternalProject_Get_Property (snappy BINARY_DIR)
ExternalProject_Get_Property (snappy SOURCE_DIR)
if (CMAKE_VERBOSE_MAKEFILE)
print_ep_logs (snappy)
endif ()
file (MAKE_DIRECTORY ${SOURCE_DIR}/snappy)
set_target_properties (snappy_lib PROPERTIES
IMPORTED_LOCATION_DEBUG
${BINARY_DIR}/${ep_lib_prefix}snappy_d${ep_lib_suffix}
IMPORTED_LOCATION_RELEASE
${BINARY_DIR}/${ep_lib_prefix}snappy${ep_lib_suffix}
INTERFACE_INCLUDE_DIRECTORIES
${SOURCE_DIR})
endif()
add_dependencies (snappy_lib snappy)
target_link_libraries (ripple_libs INTERFACE snappy_lib)
exclude_if_included (snappy)
exclude_if_included (snappy_lib)

View File

@@ -0,0 +1,165 @@
#[===================================================================[
NIH dep: soci
#]===================================================================]
foreach (_comp core empty sqlite3)
add_library ("soci_${_comp}" STATIC IMPORTED GLOBAL)
endforeach ()
if (NOT WIN32)
find_package(soci)
endif()
if (soci)
foreach (_comp core empty sqlite3)
set_target_properties ("soci_${_comp}" PROPERTIES
IMPORTED_LOCATION_DEBUG
${soci}
IMPORTED_LOCATION_RELEASE
${soci}
INTERFACE_INCLUDE_DIRECTORIES
${SOCI_INCLUDE_DIR})
endforeach ()
else()
set (soci_lib_pre ${ep_lib_prefix})
set (soci_lib_post "")
if (WIN32)
# for some reason soci on windows still prepends lib (non-standard)
set (soci_lib_pre lib)
# this version in the name might change if/when we change versions of soci
set (soci_lib_post "_4_0")
endif ()
get_target_property (_boost_incs Boost::date_time INTERFACE_INCLUDE_DIRECTORIES)
get_target_property (_boost_dt Boost::date_time IMPORTED_LOCATION)
if (NOT _boost_dt)
get_target_property (_boost_dt Boost::date_time IMPORTED_LOCATION_RELEASE)
endif ()
if (NOT _boost_dt)
get_target_property (_boost_dt Boost::date_time IMPORTED_LOCATION_DEBUG)
endif ()
ExternalProject_Add (soci
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/SOCI/soci.git
GIT_TAG 04e1870294918d20761736743bb6136314c42dd5
# We had an issue with soci integer range checking for boost::optional
# and needed to remove the exception that SOCI throws in this case.
# This is *probably* a bug in SOCI, but has never been investigated more
# nor reported to the maintainers.
# This cmake script comments out the lines in question.
# This patch process is likely fragile and should be reviewed carefully
# whenever we update the GIT_TAG above.
PATCH_COMMAND
${CMAKE_COMMAND} -D RIPPLED_SOURCE=${CMAKE_CURRENT_SOURCE_DIR}
-P ${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake/soci_patch.cmake
CMAKE_ARGS
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
$<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
$<$<BOOL:${CMAKE_TOOLCHAIN_FILE}>:-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE}>
$<$<BOOL:${VCPKG_TARGET_TRIPLET}>:-DVCPKG_TARGET_TRIPLET=${VCPKG_TARGET_TRIPLET}>
$<$<BOOL:${unity}>:-DCMAKE_UNITY_BUILD=ON}>
-DCMAKE_PREFIX_PATH=${CMAKE_BINARY_DIR}/sqlite3
-DCMAKE_MODULE_PATH=${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake
-DCMAKE_INCLUDE_PATH=$<JOIN:$<TARGET_PROPERTY:sqlite,INTERFACE_INCLUDE_DIRECTORIES>,::>
-DCMAKE_LIBRARY_PATH=${sqlite_BINARY_DIR}
-DCMAKE_DEBUG_POSTFIX=_d
$<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
-DSOCI_CXX_C11=ON
-DSOCI_STATIC=ON
-DSOCI_LIBDIR=lib
-DSOCI_SHARED=OFF
-DSOCI_TESTS=OFF
# hacks to workaround the fact that soci doesn't currently use
# boost imported targets in its cmake. If they switch to
# proper imported targets, this next line can be removed
# (as well as the get_property above that sets _boost_incs)
-DBoost_INCLUDE_DIRS=$<JOIN:${_boost_incs},::>
-DBoost_INCLUDE_DIR=$<JOIN:${_boost_incs},::>
-DBOOST_ROOT=${BOOST_ROOT}
-DWITH_BOOST=ON
-DBoost_FOUND=ON
-DBoost_NO_BOOST_CMAKE=ON
-DBoost_DATE_TIME_FOUND=ON
-DSOCI_HAVE_BOOST=ON
-DSOCI_HAVE_BOOST_DATE_TIME=ON
-DBoost_DATE_TIME_LIBRARY=${_boost_dt}
-DSOCI_DB2=OFF
-DSOCI_FIREBIRD=OFF
-DSOCI_MYSQL=OFF
-DSOCI_ODBC=OFF
-DSOCI_ORACLE=OFF
-DSOCI_POSTGRESQL=OFF
-DSOCI_SQLITE3=ON
-DSQLITE3_INCLUDE_DIR=$<JOIN:$<TARGET_PROPERTY:sqlite,INTERFACE_INCLUDE_DIRECTORIES>,::>
-DSQLITE3_LIBRARY=$<IF:$<CONFIG:Debug>,$<TARGET_PROPERTY:sqlite,IMPORTED_LOCATION_DEBUG>,$<TARGET_PROPERTY:sqlite,IMPORTED_LOCATION_RELEASE>>
$<$<BOOL:${APPLE}>:-DCMAKE_FIND_FRAMEWORK=LAST>
$<$<BOOL:${MSVC}>:
"-DCMAKE_CXX_FLAGS=-GR -Gd -fp:precise -FS -EHa -MP"
"-DCMAKE_CXX_FLAGS_DEBUG=-MTd"
"-DCMAKE_CXX_FLAGS_RELEASE=-MT"
>
$<$<NOT:$<BOOL:${MSVC}>>:
"-DCMAKE_CXX_FLAGS=-Wno-deprecated-declarations"
>
# SEE: https://github.com/SOCI/soci/issues/640
$<$<AND:$<BOOL:${is_gcc}>,$<VERSION_GREATER_EQUAL:${CMAKE_CXX_COMPILER_VERSION},8>>:
"-DCMAKE_CXX_FLAGS=-Wno-deprecated-declarations -Wno-error=format-overflow -Wno-format-overflow -Wno-error=format-truncation"
>
LIST_SEPARATOR ::
LOG_BUILD ON
LOG_CONFIGURE ON
BUILD_COMMAND
${CMAKE_COMMAND}
--build .
--config $<CONFIG>
--parallel ${ep_procs}
$<$<BOOL:${is_multiconfig}>:
COMMAND
${CMAKE_COMMAND} -E copy
<BINARY_DIR>/lib/$<CONFIG>/${soci_lib_pre}soci_core${soci_lib_post}$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
<BINARY_DIR>/lib/$<CONFIG>/${soci_lib_pre}soci_empty${soci_lib_post}$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
<BINARY_DIR>/lib/$<CONFIG>/${soci_lib_pre}soci_sqlite3${soci_lib_post}$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
<BINARY_DIR>/lib
>
TEST_COMMAND ""
INSTALL_COMMAND ""
DEPENDS sqlite
BUILD_BYPRODUCTS
<BINARY_DIR>/lib/${soci_lib_pre}soci_core${soci_lib_post}${ep_lib_suffix}
<BINARY_DIR>/lib/${soci_lib_pre}soci_core${soci_lib_post}_d${ep_lib_suffix}
<BINARY_DIR>/lib/${soci_lib_pre}soci_empty${soci_lib_post}${ep_lib_suffix}
<BINARY_DIR>/lib/${soci_lib_pre}soci_empty${soci_lib_post}_d${ep_lib_suffix}
<BINARY_DIR>/lib/${soci_lib_pre}soci_sqlite3${soci_lib_post}${ep_lib_suffix}
<BINARY_DIR>/lib/${soci_lib_pre}soci_sqlite3${soci_lib_post}_d${ep_lib_suffix}
)
ExternalProject_Get_Property (soci BINARY_DIR)
ExternalProject_Get_Property (soci SOURCE_DIR)
if (CMAKE_VERBOSE_MAKEFILE)
print_ep_logs (soci)
endif ()
file (MAKE_DIRECTORY ${SOURCE_DIR}/include)
file (MAKE_DIRECTORY ${BINARY_DIR}/include)
foreach (_comp core empty sqlite3)
set_target_properties ("soci_${_comp}" PROPERTIES
IMPORTED_LOCATION_DEBUG
${BINARY_DIR}/lib/${soci_lib_pre}soci_${_comp}${soci_lib_post}_d${ep_lib_suffix}
IMPORTED_LOCATION_RELEASE
${BINARY_DIR}/lib/${soci_lib_pre}soci_${_comp}${soci_lib_post}${ep_lib_suffix}
INTERFACE_INCLUDE_DIRECTORIES
"${SOURCE_DIR}/include;${BINARY_DIR}/include")
add_dependencies ("soci_${_comp}" soci) # something has to depend on the ExternalProject to trigger it
target_link_libraries (ripple_libs INTERFACE "soci_${_comp}")
if (NOT _comp STREQUAL "core")
target_link_libraries ("soci_${_comp}" INTERFACE soci_core)
endif ()
endforeach ()
endif()
foreach (_comp core empty sqlite3)
exclude_if_included ("soci_${_comp}")
endforeach ()
exclude_if_included (soci)

View File

@@ -0,0 +1,93 @@
#[===================================================================[
NIH dep: sqlite
#]===================================================================]
add_library (sqlite STATIC IMPORTED GLOBAL)
if (NOT WIN32)
find_package(sqlite)
endif()
if(sqlite3)
set_target_properties (sqlite PROPERTIES
IMPORTED_LOCATION_DEBUG
${sqlite3}
IMPORTED_LOCATION_RELEASE
${sqlite3}
INTERFACE_INCLUDE_DIRECTORIES
${SQLITE_INCLUDE_DIR})
else()
ExternalProject_Add (sqlite3
PREFIX ${nih_cache_path}
# sqlite doesn't use git, but it provides versioned tarballs
URL https://www.sqlite.org/2018/sqlite-amalgamation-3260000.zip
http://www.sqlite.org/2018/sqlite-amalgamation-3260000.zip
https://www2.sqlite.org/2018/sqlite-amalgamation-3260000.zip
http://www2.sqlite.org/2018/sqlite-amalgamation-3260000.zip
# ^^^ version is apparent in the URL: 3260000 => 3.26.0
URL_HASH SHA256=de5dcab133aa339a4cf9e97c40aa6062570086d6085d8f9ad7bc6ddf8a52096e
# Don't need to worry about MITM attacks too much because the download
# is checked against a strong hash
TLS_VERIFY false
# we wrote a very simple CMake file to build sqlite
# so that's what we copy here so that we can build with
# CMake. sqlite doesn't generally provided a build system
# for the single amalgamation source file.
PATCH_COMMAND
${CMAKE_COMMAND} -E copy_if_different
${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake/CMake_sqlite3.txt
<SOURCE_DIR>/CMakeLists.txt
CMAKE_ARGS
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
$<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
-DCMAKE_DEBUG_POSTFIX=_d
$<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
$<$<BOOL:${MSVC}>:
"-DCMAKE_C_FLAGS=-GR -Gd -fp:precise -FS -MP"
"-DCMAKE_C_FLAGS_DEBUG=-MTd"
"-DCMAKE_C_FLAGS_RELEASE=-MT"
>
LOG_BUILD ON
LOG_CONFIGURE ON
BUILD_COMMAND
${CMAKE_COMMAND}
--build .
--config $<CONFIG>
--parallel ${ep_procs}
$<$<BOOL:${is_multiconfig}>:
COMMAND
${CMAKE_COMMAND} -E copy
<BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}sqlite3$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
<BINARY_DIR>
>
TEST_COMMAND ""
INSTALL_COMMAND ""
BUILD_BYPRODUCTS
<BINARY_DIR>/${ep_lib_prefix}sqlite3${ep_lib_suffix}
<BINARY_DIR>/${ep_lib_prefix}sqlite3_d${ep_lib_suffix}
)
ExternalProject_Get_Property (sqlite3 BINARY_DIR)
ExternalProject_Get_Property (sqlite3 SOURCE_DIR)
if (CMAKE_VERBOSE_MAKEFILE)
print_ep_logs (sqlite3)
endif ()
set_target_properties (sqlite PROPERTIES
IMPORTED_LOCATION_DEBUG
${BINARY_DIR}/${ep_lib_prefix}sqlite3_d${ep_lib_suffix}
IMPORTED_LOCATION_RELEASE
${BINARY_DIR}/${ep_lib_prefix}sqlite3${ep_lib_suffix}
INTERFACE_INCLUDE_DIRECTORIES
${SOURCE_DIR})
add_dependencies (sqlite sqlite3)
exclude_if_included (sqlite3)
endif()
target_link_libraries (sqlite INTERFACE $<$<NOT:$<BOOL:${MSVC}>>:dl>)
target_link_libraries (ripple_libs INTERFACE sqlite)
exclude_if_included (sqlite)
set(sqlite_BINARY_DIR ${BINARY_DIR})

View File

@@ -0,0 +1,84 @@
#[===================================================================[
NIH dep: wasmedge: web assembly runtime for hooks.
#]===================================================================]
find_package(Curses)
if(CURSES_FOUND)
include_directories(${CURSES_INCLUDE_DIR})
target_link_libraries(ripple_libs INTERFACE ${CURSES_LIBRARY})
else()
message(WARNING "CURSES library not found... (only important for mac builds)")
endif()
find_package(LLVM REQUIRED CONFIG)
message(STATUS "Found LLVM ${LLVM_PACKAGE_VERSION}")
message(STATUS "Using LLVMConfig.cmake in: ${LLVM_DIR}")
ExternalProject_Add (wasmedge_src
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/WasmEdge/WasmEdge.git
GIT_TAG 0.11.2
CMAKE_ARGS
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
$<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
-DCMAKE_DEBUG_POSTFIX=_d
-DWASMEDGE_BUILD_SHARED_LIB=OFF
-DWASMEDGE_BUILD_STATIC_LIB=ON
-DWASMEDGE_BUILD_AOT_RUNTIME=ON
-DWASMEDGE_FORCE_DISABLE_LTO=ON
-DWASMEDGE_LINK_LLVM_STATIC=ON
-DWASMEDGE_LINK_TOOLS_STATIC=ON
-DWASMEDGE_BUILD_PLUGINS=OFF
-DCMAKE_POSITION_INDEPENDENT_CODE=ON
-DLLVM_DIR=${LLVM_DIR}
-DLLVM_LIBRARY_DIR=${LLVM_LIBRARY_DIR}
-DLLVM_ENABLE_TERMINFO=OFF
$<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
$<$<BOOL:${MSVC}>:
"-DCMAKE_C_FLAGS=-GR -Gd -fp:precise -FS -MP -march=native"
"-DCMAKE_C_FLAGS_DEBUG=-MTd"
"-DCMAKE_C_FLAGS_RELEASE=-MT"
>
LOG_CONFIGURE ON
LOG_BUILD ON
LOG_CONFIGURE ON
COMMAND
pwd
BUILD_COMMAND
${CMAKE_COMMAND}
--build .
--config $<CONFIG>
$<$<VERSION_GREATER_EQUAL:${CMAKE_VERSION},3.12>:--parallel ${ep_procs}>
TEST_COMMAND ""
INSTALL_COMMAND ""
BUILD_BYPRODUCTS
<BINARY_DIR>/lib/api/libwasmedge.a
)
add_library (wasmedge STATIC IMPORTED GLOBAL)
ExternalProject_Get_Property (wasmedge_src BINARY_DIR)
ExternalProject_Get_Property (wasmedge_src SOURCE_DIR)
set (wasmedge_src_BINARY_DIR "${BINARY_DIR}")
add_dependencies (wasmedge wasmedge_src)
execute_process(
COMMAND
mkdir -p "${wasmedge_src_BINARY_DIR}/include/api"
)
set_target_properties (wasmedge PROPERTIES
IMPORTED_LOCATION_DEBUG
"${wasmedge_src_BINARY_DIR}/lib/api/libwasmedge.a"
IMPORTED_LOCATION_RELEASE
"${wasmedge_src_BINARY_DIR}/lib/api/libwasmedge.a"
INTERFACE_INCLUDE_DIRECTORIES
"${wasmedge_src_BINARY_DIR}/include/api/"
)
target_link_libraries (ripple_libs INTERFACE wasmedge)
#RH NOTE: some compilers / versions of some libraries need these, most don't
find_library(XAR_LIBRARY NAMES xar)
if(XAR_LIBRARY)
target_link_libraries(ripple_libs INTERFACE ${XAR_LIBRARY})
else()
message(WARNING "xar library not found... (only important for mac builds)")
endif()
add_library (wasmedge::wasmedge ALIAS wasmedge)

View File

@@ -0,0 +1,167 @@
if(reporting)
find_library(cassandra NAMES cassandra)
if(NOT cassandra)
message("System installed Cassandra cpp driver not found. Will build")
find_library(zlib NAMES zlib1g-dev zlib-devel zlib z)
if(NOT zlib)
message("zlib not found. will build")
add_library(zlib STATIC IMPORTED GLOBAL)
ExternalProject_Add(zlib_src
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/madler/zlib.git
GIT_TAG v1.2.12
INSTALL_COMMAND ""
BUILD_BYPRODUCTS <BINARY_DIR>/${ep_lib_prefix}z.a
LOG_BUILD TRUE
LOG_CONFIGURE TRUE
)
ExternalProject_Get_Property (zlib_src SOURCE_DIR)
ExternalProject_Get_Property (zlib_src BINARY_DIR)
set (zlib_src_SOURCE_DIR "${SOURCE_DIR}")
file (MAKE_DIRECTORY ${zlib_src_SOURCE_DIR}/include)
set_target_properties (zlib PROPERTIES
IMPORTED_LOCATION
${BINARY_DIR}/${ep_lib_prefix}z.a
INTERFACE_INCLUDE_DIRECTORIES
${SOURCE_DIR}/include)
add_dependencies(zlib zlib_src)
file(TO_CMAKE_PATH "${zlib_src_SOURCE_DIR}" zlib_src_SOURCE_DIR)
endif()
find_library(krb5 NAMES krb5-dev libkrb5-dev)
if(NOT krb5)
message("krb5 not found. will build")
add_library(krb5 STATIC IMPORTED GLOBAL)
ExternalProject_Add(krb5_src
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/krb5/krb5.git
GIT_TAG krb5-1.20-final
UPDATE_COMMAND ""
CONFIGURE_COMMAND autoreconf src && CFLAGS=-fcommon ./src/configure --enable-static --disable-shared > /dev/null
BUILD_IN_SOURCE 1
BUILD_COMMAND make
INSTALL_COMMAND ""
BUILD_BYPRODUCTS <SOURCE_DIR>/lib/${ep_lib_prefix}krb5.a
LOG_BUILD TRUE
)
ExternalProject_Get_Property (krb5_src SOURCE_DIR)
ExternalProject_Get_Property (krb5_src BINARY_DIR)
set (krb5_src_SOURCE_DIR "${SOURCE_DIR}")
file (MAKE_DIRECTORY ${krb5_src_SOURCE_DIR}/include)
set_target_properties (krb5 PROPERTIES
IMPORTED_LOCATION
${BINARY_DIR}/lib/${ep_lib_prefix}krb5.a
INTERFACE_INCLUDE_DIRECTORIES
${SOURCE_DIR}/include)
add_dependencies(krb5 krb5_src)
file(TO_CMAKE_PATH "${krb5_src_SOURCE_DIR}" krb5_src_SOURCE_DIR)
endif()
find_library(libuv1 NAMES uv1 libuv1 liubuv1-dev libuv1:amd64)
if(NOT libuv1)
message("libuv1 not found, will build")
add_library(libuv1 STATIC IMPORTED GLOBAL)
ExternalProject_Add(libuv_src
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/libuv/libuv.git
GIT_TAG v1.44.2
INSTALL_COMMAND ""
BUILD_BYPRODUCTS <BINARY_DIR>/${ep_lib_prefix}uv_a.a
LOG_BUILD TRUE
LOG_CONFIGURE TRUE
)
ExternalProject_Get_Property (libuv_src SOURCE_DIR)
ExternalProject_Get_Property (libuv_src BINARY_DIR)
set (libuv_src_SOURCE_DIR "${SOURCE_DIR}")
file (MAKE_DIRECTORY ${libuv_src_SOURCE_DIR}/include)
set_target_properties (libuv1 PROPERTIES
IMPORTED_LOCATION
${BINARY_DIR}/${ep_lib_prefix}uv_a.a
INTERFACE_INCLUDE_DIRECTORIES
${SOURCE_DIR}/include)
add_dependencies(libuv1 libuv_src)
file(TO_CMAKE_PATH "${libuv_src_SOURCE_DIR}" libuv_src_SOURCE_DIR)
endif()
add_library (cassandra STATIC IMPORTED GLOBAL)
ExternalProject_Add(cassandra_src
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/datastax/cpp-driver.git
GIT_TAG 2.16.2
CMAKE_ARGS
-DLIBUV_ROOT_DIR=${BINARY_DIR}
-DLIBUV_LIBARY=${BINARY_DIR}/libuv_a.a
-DLIBUV_INCLUDE_DIR=${SOURCE_DIR}/include
-DCASS_BUILD_STATIC=ON
-DCASS_BUILD_SHARED=OFF
-DOPENSSL_ROOT_DIR=/opt/local/openssl
INSTALL_COMMAND ""
BUILD_BYPRODUCTS <BINARY_DIR>/${ep_lib_prefix}cassandra_static.a
LOG_BUILD TRUE
LOG_CONFIGURE TRUE
)
ExternalProject_Get_Property (cassandra_src SOURCE_DIR)
ExternalProject_Get_Property (cassandra_src BINARY_DIR)
set (cassandra_src_SOURCE_DIR "${SOURCE_DIR}")
file (MAKE_DIRECTORY ${cassandra_src_SOURCE_DIR}/include)
set_target_properties (cassandra PROPERTIES
IMPORTED_LOCATION
${BINARY_DIR}/${ep_lib_prefix}cassandra_static.a
INTERFACE_INCLUDE_DIRECTORIES
${SOURCE_DIR}/include)
add_dependencies(cassandra cassandra_src)
if(NOT libuv1)
ExternalProject_Add_StepDependencies(cassandra_src build libuv1)
target_link_libraries(cassandra INTERFACE libuv1)
else()
target_link_libraries(cassandra INTERFACE ${libuv1})
endif()
if(NOT krb5)
ExternalProject_Add_StepDependencies(cassandra_src build krb5)
target_link_libraries(cassandra INTERFACE krb5)
else()
target_link_libraries(cassandra INTERFACE ${krb5})
endif()
if(NOT zlib)
ExternalProject_Add_StepDependencies(cassandra_src build zlib)
target_link_libraries(cassandra INTERFACE zlib)
else()
target_link_libraries(cassandra INTERFACE ${zlib})
endif()
file(TO_CMAKE_PATH "${cassandra_src_SOURCE_DIR}" cassandra_src_SOURCE_DIR)
target_link_libraries(ripple_libs INTERFACE cassandra)
else()
message("Found system installed cassandra cpp driver")
find_path(cassandra_includes NAMES cassandra.h REQUIRED)
target_link_libraries (ripple_libs INTERFACE ${cassandra})
target_include_directories(ripple_libs INTERFACE ${cassandra_includes})
endif()
exclude_if_included (cassandra)
endif()

View File

@@ -0,0 +1,18 @@
#[===================================================================[
NIH dep: date
the main library is header-only, thus is an INTERFACE lib in CMake.
NOTE: this has been accepted into c++20 so can likely be replaced
when we update to that standard
#]===================================================================]
find_package (date QUIET)
if (NOT TARGET date::date)
FetchContent_Declare(
hh_date_src
GIT_REPOSITORY https://github.com/HowardHinnant/date.git
GIT_TAG fc4cf092f9674f2670fb9177edcdee870399b829
)
FetchContent_MakeAvailable(hh_date_src)
endif ()

View File

@@ -0,0 +1,392 @@
# currently linking to unsecure versions...if we switch, we'll
# need to add ssl as a link dependency to the grpc targets
option (use_secure_grpc "use TLS version of grpc libs." OFF)
if (use_secure_grpc)
set (grpc_suffix "")
else ()
set (grpc_suffix "_unsecure")
endif ()
find_package (gRPC 1.23 CONFIG QUIET)
if (TARGET gRPC::gpr AND NOT local_grpc)
get_target_property (_grpc_l gRPC::gpr IMPORTED_LOCATION_DEBUG)
if (NOT _grpc_l)
get_target_property (_grpc_l gRPC::gpr IMPORTED_LOCATION_RELEASE)
endif ()
if (NOT _grpc_l)
get_target_property (_grpc_l gRPC::gpr IMPORTED_LOCATION)
endif ()
message (STATUS "Found cmake config for gRPC. Using ${_grpc_l}.")
else ()
find_package (PkgConfig QUIET)
if (PKG_CONFIG_FOUND)
pkg_check_modules (grpc QUIET "grpc${grpc_suffix}>=1.25" "grpc++${grpc_suffix}" gpr)
endif ()
if (grpc_FOUND)
message (STATUS "Found gRPC using pkg-config. Using ${grpc_gpr_PREFIX}.")
endif ()
add_executable (gRPC::grpc_cpp_plugin IMPORTED)
exclude_if_included (gRPC::grpc_cpp_plugin)
if (grpc_FOUND AND NOT local_grpc)
# use installed grpc (via pkg-config)
macro (add_imported_grpc libname_)
if (static)
set (_search "${CMAKE_STATIC_LIBRARY_PREFIX}${libname_}${CMAKE_STATIC_LIBRARY_SUFFIX}")
else ()
set (_search "${CMAKE_SHARED_LIBRARY_PREFIX}${libname_}${CMAKE_SHARED_LIBRARY_SUFFIX}")
endif()
find_library(_found_${libname_}
NAMES ${_search}
HINTS ${grpc_LIBRARY_DIRS})
if (_found_${libname_})
message (STATUS "importing ${libname_} as ${_found_${libname_}}")
else ()
message (FATAL_ERROR "using pkg-config for grpc, can't find ${_search}")
endif ()
add_library ("gRPC::${libname_}" STATIC IMPORTED GLOBAL)
set_target_properties ("gRPC::${libname_}" PROPERTIES IMPORTED_LOCATION ${_found_${libname_}})
if (grpc_INCLUDE_DIRS)
set_target_properties ("gRPC::${libname_}" PROPERTIES INTERFACE_INCLUDE_DIRECTORIES ${grpc_INCLUDE_DIRS})
endif ()
target_link_libraries (ripple_libs INTERFACE "gRPC::${libname_}")
exclude_if_included ("gRPC::${libname_}")
endmacro ()
set_target_properties (gRPC::grpc_cpp_plugin PROPERTIES
IMPORTED_LOCATION "${grpc_gpr_PREFIX}/bin/grpc_cpp_plugin${CMAKE_EXECUTABLE_SUFFIX}")
pkg_check_modules (cares QUIET libcares)
if (cares_FOUND)
if (static)
set (_search "${CMAKE_STATIC_LIBRARY_PREFIX}cares${CMAKE_STATIC_LIBRARY_SUFFIX}")
set (_prefix cares_STATIC)
set (_static STATIC)
else ()
set (_search "${CMAKE_SHARED_LIBRARY_PREFIX}cares${CMAKE_SHARED_LIBRARY_SUFFIX}")
set (_prefix cares)
set (_static)
endif()
find_library(_location NAMES ${_search} HINTS ${cares_LIBRARY_DIRS})
if (NOT _location)
message (FATAL_ERROR "using pkg-config for grpc, can't find c-ares")
endif ()
if(${_location} MATCHES "\\.a$")
add_library(c-ares::cares STATIC IMPORTED GLOBAL)
else()
add_library(c-ares::cares SHARED IMPORTED GLOBAL)
endif()
set_target_properties (c-ares::cares PROPERTIES
IMPORTED_LOCATION ${_location}
INTERFACE_INCLUDE_DIRECTORIES "${${_prefix}_INCLUDE_DIRS}"
INTERFACE_LINK_OPTIONS "${${_prefix}_LDFLAGS}"
)
exclude_if_included (c-ares::cares)
else ()
message (FATAL_ERROR "using pkg-config for grpc, can't find c-ares")
endif ()
else ()
#[===========================[
c-ares (grpc requires)
#]===========================]
ExternalProject_Add (c-ares_src
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/c-ares/c-ares.git
GIT_TAG cares-1_15_0
CMAKE_ARGS
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
$<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
-DCMAKE_DEBUG_POSTFIX=_d
$<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
-DCMAKE_INSTALL_PREFIX=<BINARY_DIR>/_installed_
-DCARES_SHARED=OFF
-DCARES_STATIC=ON
-DCARES_STATIC_PIC=ON
-DCARES_INSTALL=ON
-DCARES_MSVC_STATIC_RUNTIME=ON
$<$<BOOL:${MSVC}>:
"-DCMAKE_C_FLAGS=-GR -Gd -fp:precise -FS -MP"
>
LOG_BUILD ON
LOG_CONFIGURE ON
BUILD_COMMAND
${CMAKE_COMMAND}
--build .
--config $<CONFIG>
--parallel ${ep_procs}
TEST_COMMAND ""
INSTALL_COMMAND
${CMAKE_COMMAND} -E env --unset=DESTDIR ${CMAKE_COMMAND} --build . --config $<CONFIG> --target install
BUILD_BYPRODUCTS
<BINARY_DIR>/_installed_/lib/${ep_lib_prefix}cares${ep_lib_suffix}
<BINARY_DIR>/_installed_/lib/${ep_lib_prefix}cares_d${ep_lib_suffix}
)
exclude_if_included (c-ares_src)
ExternalProject_Get_Property (c-ares_src BINARY_DIR)
set (cares_binary_dir "${BINARY_DIR}")
add_library (c-ares::cares STATIC IMPORTED GLOBAL)
file (MAKE_DIRECTORY ${BINARY_DIR}/_installed_/include)
set_target_properties (c-ares::cares PROPERTIES
IMPORTED_LOCATION_DEBUG
${BINARY_DIR}/_installed_/lib/${ep_lib_prefix}cares_d${ep_lib_suffix}
IMPORTED_LOCATION_RELEASE
${BINARY_DIR}/_installed_/lib/${ep_lib_prefix}cares${ep_lib_suffix}
INTERFACE_INCLUDE_DIRECTORIES
${BINARY_DIR}/_installed_/include)
add_dependencies (c-ares::cares c-ares_src)
exclude_if_included (c-ares::cares)
if (NOT has_zlib)
#[===========================[
zlib (grpc requires)
#]===========================]
if (MSVC)
set (zlib_debug_postfix "d") # zlib cmake sets this internally for MSVC, so we really don't have a choice
set (zlib_base "zlibstatic")
else ()
set (zlib_debug_postfix "_d")
set (zlib_base "z")
endif ()
ExternalProject_Add (zlib_src
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/madler/zlib.git
GIT_TAG v1.2.11
CMAKE_ARGS
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
$<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
-DCMAKE_DEBUG_POSTFIX=${zlib_debug_postfix}
$<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
-DCMAKE_INSTALL_PREFIX=<BINARY_DIR>/_installed_
-DBUILD_SHARED_LIBS=OFF
$<$<BOOL:${MSVC}>:
"-DCMAKE_C_FLAGS=-GR -Gd -fp:precise -FS -MP"
"-DCMAKE_C_FLAGS_DEBUG=-MTd"
"-DCMAKE_C_FLAGS_RELEASE=-MT"
>
LOG_BUILD ON
LOG_CONFIGURE ON
BUILD_COMMAND
${CMAKE_COMMAND}
--build .
--config $<CONFIG>
--parallel ${ep_procs}
TEST_COMMAND ""
INSTALL_COMMAND
${CMAKE_COMMAND} -E env --unset=DESTDIR ${CMAKE_COMMAND} --build . --config $<CONFIG> --target install
BUILD_BYPRODUCTS
<BINARY_DIR>/_installed_/lib/${ep_lib_prefix}${zlib_base}${ep_lib_suffix}
<BINARY_DIR>/_installed_/lib/${ep_lib_prefix}${zlib_base}${zlib_debug_postfix}${ep_lib_suffix}
)
exclude_if_included (zlib_src)
ExternalProject_Get_Property (zlib_src BINARY_DIR)
set (zlib_binary_dir "${BINARY_DIR}")
add_library (ZLIB::ZLIB STATIC IMPORTED GLOBAL)
file (MAKE_DIRECTORY ${BINARY_DIR}/_installed_/include)
set_target_properties (ZLIB::ZLIB PROPERTIES
IMPORTED_LOCATION_DEBUG
${BINARY_DIR}/_installed_/lib/${ep_lib_prefix}${zlib_base}${zlib_debug_postfix}${ep_lib_suffix}
IMPORTED_LOCATION_RELEASE
${BINARY_DIR}/_installed_/lib/${ep_lib_prefix}${zlib_base}${ep_lib_suffix}
INTERFACE_INCLUDE_DIRECTORIES
${BINARY_DIR}/_installed_/include)
add_dependencies (ZLIB::ZLIB zlib_src)
exclude_if_included (ZLIB::ZLIB)
endif ()
#[===========================[
grpc
#]===========================]
ExternalProject_Add (grpc_src
PREFIX ${nih_cache_path}
GIT_REPOSITORY https://github.com/grpc/grpc.git
GIT_TAG v1.25.0
CMAKE_ARGS
-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
-DCMAKE_CXX_STANDARD=17
$<$<BOOL:${CMAKE_VERBOSE_MAKEFILE}>:-DCMAKE_VERBOSE_MAKEFILE=ON>
$<$<BOOL:${CMAKE_TOOLCHAIN_FILE}>:-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE}>
$<$<BOOL:${VCPKG_TARGET_TRIPLET}>:-DVCPKG_TARGET_TRIPLET=${VCPKG_TARGET_TRIPLET}>
$<$<BOOL:${unity}>:-DCMAKE_UNITY_BUILD=ON}>
-DCMAKE_DEBUG_POSTFIX=_d
$<$<NOT:$<BOOL:${is_multiconfig}>>:-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}>
-DgRPC_BUILD_TESTS=OFF
-DgRPC_BENCHMARK_PROVIDER=""
-DgRPC_BUILD_CSHARP_EXT=OFF
-DgRPC_MSVC_STATIC_RUNTIME=ON
-DgRPC_INSTALL=OFF
-DgRPC_CARES_PROVIDER=package
-Dc-ares_DIR=${cares_binary_dir}/_installed_/lib/cmake/c-ares
-DgRPC_SSL_PROVIDER=package
-DOPENSSL_ROOT_DIR=${OPENSSL_ROOT_DIR}
-DgRPC_PROTOBUF_PROVIDER=package
-DProtobuf_USE_STATIC_LIBS=$<IF:$<AND:$<BOOL:${Protobuf_FOUND}>,$<NOT:$<BOOL:${static}>>>,OFF,ON>
-DProtobuf_INCLUDE_DIR=$<JOIN:$<TARGET_PROPERTY:protobuf::libprotobuf,INTERFACE_INCLUDE_DIRECTORIES>,:_:>
-DProtobuf_LIBRARY=$<IF:$<CONFIG:Debug>,$<TARGET_PROPERTY:protobuf::libprotobuf,IMPORTED_LOCATION_DEBUG>,$<TARGET_PROPERTY:protobuf::libprotobuf,IMPORTED_LOCATION_RELEASE>>
-DProtobuf_PROTOC_LIBRARY=$<IF:$<CONFIG:Debug>,$<TARGET_PROPERTY:protobuf::libprotoc,IMPORTED_LOCATION_DEBUG>,$<TARGET_PROPERTY:protobuf::libprotoc,IMPORTED_LOCATION_RELEASE>>
-DProtobuf_PROTOC_EXECUTABLE=$<TARGET_PROPERTY:protobuf::protoc,IMPORTED_LOCATION>
-DgRPC_ZLIB_PROVIDER=package
$<$<NOT:$<BOOL:${has_zlib}>>:-DZLIB_ROOT=${zlib_binary_dir}/_installed_>
$<$<BOOL:${MSVC}>:
"-DCMAKE_CXX_FLAGS=-GR -Gd -fp:precise -FS -EHa -MP"
"-DCMAKE_C_FLAGS=-GR -Gd -fp:precise -FS -MP"
>
LOG_BUILD ON
LOG_CONFIGURE ON
BUILD_COMMAND
${CMAKE_COMMAND}
--build .
--config $<CONFIG>
--parallel ${ep_procs}
$<$<BOOL:${is_multiconfig}>:
COMMAND
${CMAKE_COMMAND} -E copy
<BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}grpc${grpc_suffix}$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
<BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}grpc++${grpc_suffix}$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
<BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}address_sorting$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
<BINARY_DIR>/$<CONFIG>/${ep_lib_prefix}gpr$<$<CONFIG:Debug>:_d>${ep_lib_suffix}
<BINARY_DIR>/$<CONFIG>/grpc_cpp_plugin${CMAKE_EXECUTABLE_SUFFIX}
<BINARY_DIR>
>
LIST_SEPARATOR :_:
TEST_COMMAND ""
INSTALL_COMMAND ""
DEPENDS c-ares_src
BUILD_BYPRODUCTS
<BINARY_DIR>/${ep_lib_prefix}grpc${grpc_suffix}${ep_lib_suffix}
<BINARY_DIR>/${ep_lib_prefix}grpc${grpc_suffix}_d${ep_lib_suffix}
<BINARY_DIR>/${ep_lib_prefix}grpc++${grpc_suffix}${ep_lib_suffix}
<BINARY_DIR>/${ep_lib_prefix}grpc++${grpc_suffix}_d${ep_lib_suffix}
<BINARY_DIR>/${ep_lib_prefix}address_sorting${ep_lib_suffix}
<BINARY_DIR>/${ep_lib_prefix}address_sorting_d${ep_lib_suffix}
<BINARY_DIR>/${ep_lib_prefix}gpr${ep_lib_suffix}
<BINARY_DIR>/${ep_lib_prefix}gpr_d${ep_lib_suffix}
<BINARY_DIR>/grpc_cpp_plugin${CMAKE_EXECUTABLE_SUFFIX}
)
if (TARGET protobuf_src)
ExternalProject_Add_StepDependencies(grpc_src build protobuf_src)
endif ()
exclude_if_included (grpc_src)
ExternalProject_Get_Property (grpc_src BINARY_DIR)
ExternalProject_Get_Property (grpc_src SOURCE_DIR)
set (grpc_binary_dir "${BINARY_DIR}")
set (grpc_source_dir "${SOURCE_DIR}")
if (CMAKE_VERBOSE_MAKEFILE)
print_ep_logs (grpc_src)
endif ()
file (MAKE_DIRECTORY ${SOURCE_DIR}/include)
macro (add_imported_grpc libname_)
add_library ("gRPC::${libname_}" STATIC IMPORTED GLOBAL)
set_target_properties ("gRPC::${libname_}" PROPERTIES
IMPORTED_LOCATION_DEBUG
${grpc_binary_dir}/${ep_lib_prefix}${libname_}_d${ep_lib_suffix}
IMPORTED_LOCATION_RELEASE
${grpc_binary_dir}/${ep_lib_prefix}${libname_}${ep_lib_suffix}
INTERFACE_INCLUDE_DIRECTORIES
${grpc_source_dir}/include)
add_dependencies ("gRPC::${libname_}" grpc_src)
target_link_libraries (ripple_libs INTERFACE "gRPC::${libname_}")
exclude_if_included ("gRPC::${libname_}")
endmacro ()
set_target_properties (gRPC::grpc_cpp_plugin PROPERTIES
IMPORTED_LOCATION "${grpc_binary_dir}/grpc_cpp_plugin${CMAKE_EXECUTABLE_SUFFIX}")
add_dependencies (gRPC::grpc_cpp_plugin grpc_src)
endif ()
add_imported_grpc (gpr)
add_imported_grpc ("grpc${grpc_suffix}")
add_imported_grpc ("grpc++${grpc_suffix}")
add_imported_grpc (address_sorting)
target_link_libraries ("gRPC::grpc${grpc_suffix}" INTERFACE c-ares::cares gRPC::gpr gRPC::address_sorting ZLIB::ZLIB)
target_link_libraries ("gRPC::grpc++${grpc_suffix}" INTERFACE "gRPC::grpc${grpc_suffix}" gRPC::gpr)
endif ()
#[=================================[
generate protobuf sources for
grpc defs and bundle into a
static lib
#]=================================]
set(output_dir "${CMAKE_BINARY_DIR}/proto_gen_grpc")
set(GRPC_GEN_DIR "${output_dir}/ripple/proto")
file(MAKE_DIRECTORY ${GRPC_GEN_DIR})
set(GRPC_PROTO_SRCS)
set(GRPC_PROTO_HDRS)
set(GRPC_PROTO_ROOT "${CMAKE_CURRENT_SOURCE_DIR}/src/ripple/proto/org")
file(GLOB_RECURSE GRPC_DEFINITION_FILES "${GRPC_PROTO_ROOT}/*.proto")
foreach(file ${GRPC_DEFINITION_FILES})
# /home/user/rippled/src/ripple/proto/org/.../v1/get_ledger.proto
get_filename_component(_abs_file ${file} ABSOLUTE)
# /home/user/rippled/src/ripple/proto/org/.../v1
get_filename_component(_abs_dir ${_abs_file} DIRECTORY)
# get_ledger
get_filename_component(_basename ${file} NAME_WE)
# /home/user/rippled/src/ripple/proto
get_filename_component(_proto_inc ${GRPC_PROTO_ROOT} DIRECTORY) # updir one level
# org/.../v1/get_ledger.proto
file(RELATIVE_PATH _rel_root_file ${_proto_inc} ${_abs_file})
# org/.../v1
get_filename_component(_rel_root_dir ${_rel_root_file} DIRECTORY)
# src/ripple/proto/org/.../v1
file(RELATIVE_PATH _rel_dir ${CMAKE_CURRENT_SOURCE_DIR} ${_abs_dir})
set(src_1 "${GRPC_GEN_DIR}/${_rel_root_dir}/${_basename}.grpc.pb.cc")
set(src_2 "${GRPC_GEN_DIR}/${_rel_root_dir}/${_basename}.pb.cc")
set(hdr_1 "${GRPC_GEN_DIR}/${_rel_root_dir}/${_basename}.grpc.pb.h")
set(hdr_2 "${GRPC_GEN_DIR}/${_rel_root_dir}/${_basename}.pb.h")
add_custom_command(
OUTPUT ${src_1} ${src_2} ${hdr_1} ${hdr_2}
COMMAND protobuf::protoc
ARGS --grpc_out=${GRPC_GEN_DIR}
--cpp_out=${GRPC_GEN_DIR}
--plugin=protoc-gen-grpc=$<TARGET_FILE:gRPC::grpc_cpp_plugin>
-I ${_proto_inc} -I ${_rel_dir}
${_abs_file}
DEPENDS ${_abs_file} protobuf::protoc gRPC::grpc_cpp_plugin
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
COMMENT "Running gRPC C++ protocol buffer compiler on ${file}"
VERBATIM)
set_source_files_properties(${src_1} ${src_2} ${hdr_1} ${hdr_2} PROPERTIES
GENERATED TRUE
SKIP_UNITY_BUILD_INCLUSION ON
)
list(APPEND GRPC_PROTO_SRCS ${src_1} ${src_2})
list(APPEND GRPC_PROTO_HDRS ${hdr_1} ${hdr_2})
endforeach()
target_include_directories(xrpl_core SYSTEM PUBLIC
$<BUILD_INTERFACE:${output_dir}>
$<BUILD_INTERFACE:${output_dir}/ripple/proto>
# The generated sources include headers relative to this path. Fix it later.
$<INSTALL_INTERFACE:include/ripple/proto>
)
target_sources(xrpl_core PRIVATE ${GRPC_PROTO_SRCS})
install(
DIRECTORY ${output_dir}/ripple
DESTINATION include/
FILES_MATCHING PATTERN "*.h"
)
target_link_libraries(xrpl_core PUBLIC
"gRPC::grpc++"
# libgrpc is missing references.
absl::random_random
)
target_compile_options(xrpl_core
PRIVATE
$<$<BOOL:${MSVC}>:-wd4065>
$<$<NOT:$<BOOL:${MSVC}>>:-Wno-deprecated-declarations>
PUBLIC
$<$<BOOL:${MSVC}>:-wd4996>
$<$<BOOL:${is_xcode}>:
--system-header-prefix="google/protobuf"
-Wno-deprecated-dynamic-exception-spec
>)
# target_link_libraries (ripple_libs INTERFACE Ripple::grpc_pbufs)
# exclude_if_included (grpc_pbufs)

View File

@@ -50,7 +50,7 @@ that `test` code should *never* be included in `ripple` code.)
## Validation
The [levelization.py](levelization.py) script takes no parameters,
The [levelization.sh](levelization.sh) script takes no parameters,
reads no environment variables, and can be run from any directory,
as long as it is in the expected location in the rippled repo.
It can be run at any time from within a checked out repo, and will
@@ -84,7 +84,7 @@ It generates many files of [results](results):
Github Actions workflow to test that levelization loops haven't
changed. Unfortunately, if changes are detected, it can't tell if
they are improvements or not, so if you have resolved any issues or
done anything else to improve levelization, run `levelization.py`,
done anything else to improve levelization, run `levelization.sh`,
and commit the updated results.
The `loops.txt` and `ordering.txt` files relate the modules
@@ -108,7 +108,7 @@ The committed files hide the detailed values intentionally, to
prevent false alarms and merging issues, and because it's easy to
get those details locally.
1. Run `levelization.py`
1. Run `levelization.sh`
2. Grep the modules in `paths.txt`.
* For example, if a cycle is found `A ~= B`, simply `grep -w
A Builds/levelization/results/paths.txt | grep -w B`

View File

@@ -1,283 +0,0 @@
#!/usr/bin/env python3
"""
Usage: levelization.py
This script takes no parameters, and can be called from any directory in the file system.
"""
import os
import re
import sys
from collections import defaultdict
from pathlib import Path
# Compile regex patterns once at module level
INCLUDE_PATTERN = re.compile(r"^\s*#include.*/.*\.h")
INCLUDE_PATH_PATTERN = re.compile(r'[<"]([^>"]+)[>"]')
def dictionary_sort_key(s):
"""
Create a sort key that mimics 'sort -d' (dictionary order).
Dictionary order only considers blanks and alphanumeric characters.
"""
return "".join(c for c in s if c.isalnum() or c.isspace())
def get_level(file_path):
"""
Extract the level from a file path (second and third directory components).
Equivalent to bash: cut -d/ -f 2,3
Examples:
src/ripple/app/main.cpp -> ripple.app
src/test/app/Import_test.cpp -> test.app
"""
parts = file_path.split("/")
if len(parts) >= 3:
level = f"{parts[1]}/{parts[2]}"
elif len(parts) >= 2:
level = f"{parts[1]}/toplevel"
else:
level = file_path
# If the "level" indicates a file, cut off the filename
if "." in level.split("/")[-1]:
# Use the "toplevel" label as a workaround for `sort`
# inconsistencies between different utility versions
level = level.rsplit("/", 1)[0] + "/toplevel"
return level.replace("/", ".")
def extract_include_level(include_line):
"""
Extract the include path from an #include directive.
Gets the first two directory components from the include path.
Equivalent to bash: cut -d/ -f 1,2
Examples:
#include <ripple/basics/base_uint.h> -> ripple.basics
#include "ripple/app/main/Application.h" -> ripple.app
"""
match = INCLUDE_PATH_PATTERN.search(include_line)
if not match:
return None
include_path = match.group(1)
parts = include_path.split("/")
if len(parts) >= 2:
include_level = f"{parts[0]}/{parts[1]}"
else:
include_level = include_path
# If the "includelevel" indicates a file, cut off the filename
if "." in include_level.split("/")[-1]:
include_level = include_level.rsplit("/", 1)[0] + "/toplevel"
return include_level.replace("/", ".")
def find_repository_directories(start_path, depth_limit=10):
"""
Find the repository root by looking for src or include folders.
Walks up the directory tree from the start path.
"""
current = start_path.resolve()
for _ in range(depth_limit):
src_path = current / "src"
include_path = current / "include"
has_src = src_path.exists()
has_include = include_path.exists()
if has_src or has_include:
dirs = []
if has_src:
dirs.append(src_path)
if has_include:
dirs.append(include_path)
return current, dirs
parent = current.parent
if parent == current:
break
current = parent
raise RuntimeError(
"Could not find repository root. "
"Expected to find a directory containing 'src' and/or 'include' folders."
)
def main():
script_dir = Path(__file__).parent.resolve()
os.chdir(script_dir)
# Clean up and create results directory.
results_dir = script_dir / "results"
if results_dir.exists():
import shutil
shutil.rmtree(results_dir)
results_dir.mkdir()
# Find the repository root.
try:
repo_root, scan_dirs = find_repository_directories(script_dir)
print(f"Found repository root: {repo_root}")
for scan_dir in scan_dirs:
print(f" Scanning: {scan_dir.relative_to(repo_root)}")
except RuntimeError as e:
print(f"Error: {e}", file=sys.stderr)
sys.exit(1)
# Find all #include directives.
print("\nScanning for raw includes...")
raw_includes = []
rawincludes_file = results_dir / "rawincludes.txt"
with open(rawincludes_file, "w", buffering=8192) as raw_f:
for dir_path in scan_dirs:
for file_path in dir_path.rglob("*"):
if not file_path.is_file():
continue
try:
rel_path_str = str(file_path.relative_to(repo_root))
with open(
file_path, "r", encoding="utf-8", errors="ignore", buffering=8192
) as f:
for line in f:
if "#include" not in line or "boost" in line:
continue
if INCLUDE_PATTERN.match(line):
line_stripped = line.strip()
entry = f"{rel_path_str}:{line_stripped}\n"
print(entry, end="")
raw_f.write(entry)
raw_includes.append((rel_path_str, line_stripped))
except Exception as e:
print(f"Error reading {file_path}: {e}", file=sys.stderr)
# Build levelization paths and count directly.
print("Build levelization paths")
path_counts = defaultdict(int)
for file_path, include_line in raw_includes:
include_level = extract_include_level(include_line)
if not include_level:
continue
level = get_level(file_path)
if level != include_level:
path_counts[(level, include_level)] += 1
# Sort and deduplicate paths.
print("Sort and deduplicate paths")
sorted_items = sorted(
path_counts.items(),
key=lambda x: (dictionary_sort_key(x[0][0]), dictionary_sort_key(x[0][1])),
)
paths_file = results_dir / "paths.txt"
with open(paths_file, "w") as f:
for (level, include_level), count in sorted_items:
line = f"{count:7} {level} {include_level}\n"
print(line.rstrip())
f.write(line)
# Split into flat-file database.
print("Split into flat-file database")
includes_dir = results_dir / "includes"
includedby_dir = results_dir / "includedby"
includes_dir.mkdir()
includedby_dir.mkdir()
includes_data = defaultdict(list)
includedby_data = defaultdict(list)
for (level, include_level), count in sorted_items:
includes_data[level].append((include_level, count))
includedby_data[include_level].append((level, count))
for level in sorted(includes_data.keys(), key=dictionary_sort_key):
with open(includes_dir / level, "w") as f:
for include_level, count in includes_data[level]:
line = f"{include_level} {count}\n"
print(line.rstrip())
f.write(line)
for include_level in sorted(includedby_data.keys(), key=dictionary_sort_key):
with open(includedby_dir / include_level, "w") as f:
for level, count in includedby_data[include_level]:
line = f"{level} {count}\n"
print(line.rstrip())
f.write(line)
# Search for loops.
print("Search for loops")
loops_file = results_dir / "loops.txt"
ordering_file = results_dir / "ordering.txt"
# Pre-load all include files into memory for fast lookup.
includes_cache = {}
includes_lookup = {}
for include_file in sorted(includes_dir.iterdir(), key=lambda p: p.name):
if not include_file.is_file():
continue
includes_cache[include_file.name] = []
includes_lookup[include_file.name] = {}
with open(include_file, "r") as f:
for line in f:
parts = line.strip().split()
if len(parts) >= 2:
name, count = parts[0], int(parts[1])
includes_cache[include_file.name].append((name, count))
includes_lookup[include_file.name][name] = count
loops_found = set()
with open(loops_file, "w", buffering=8192) as loops_f, open(
ordering_file, "w", buffering=8192
) as ordering_f:
for source in sorted(includes_cache.keys()):
for include, include_freq in includes_cache[source]:
if include not in includes_lookup:
continue
source_freq = includes_lookup[include].get(source)
if source_freq is not None:
loop_key = tuple(sorted([source, include]))
if loop_key in loops_found:
continue
loops_found.add(loop_key)
loops_f.write(f"Loop: {source} {include}\n")
diff = include_freq - source_freq
if diff > 3:
loops_f.write(f" {source} > {include}\n\n")
elif diff < -3:
loops_f.write(f" {include} > {source}\n\n")
elif source_freq == include_freq:
loops_f.write(f" {include} == {source}\n\n")
else:
loops_f.write(f" {include} ~= {source}\n\n")
else:
ordering_f.write(f"{source} > {include}\n")
# Print results.
print("\nOrdering:")
with open(ordering_file, "r") as f:
print(f.read(), end="")
print("\nLoops:")
with open(loops_file, "r") as f:
print(f.read(), end="")
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,127 @@
#!/bin/bash
# Usage: levelization.sh
# This script takes no parameters, reads no environment variables,
# and can be run from any directory, as long as it is in the expected
# location in the repo.
pushd $( dirname $0 )
if [ -v PS1 ]
then
# if the shell is interactive, clean up any flotsam before analyzing
git clean -ix
fi
rm -rfv results
mkdir results
includes="$( pwd )/results/rawincludes.txt"
pushd ../..
echo Raw includes:
grep -r '#include.*/.*\.h' src/ripple/ src/test/ | \
grep -v boost | tee ${includes}
popd
pushd results
oldifs=${IFS}
IFS=:
mkdir includes
mkdir includedby
echo Build levelization paths
exec 3< ${includes} # open rawincludes.txt for input
while read -r -u 3 file include
do
level=$( echo ${file} | cut -d/ -f 2,3 )
# If the "level" indicates a file, cut off the filename
if [[ "${level##*.}" != "${level}" ]]
then
# Use the "toplevel" label as a workaround for `sort`
# inconsistencies between different utility versions
level="$( dirname ${level} )/toplevel"
fi
level=$( echo ${level} | tr '/' '.' )
includelevel=$( echo ${include} | sed 's/.*["<]//; s/[">].*//' | \
cut -d/ -f 1,2 )
if [[ "${includelevel##*.}" != "${includelevel}" ]]
then
# Use the "toplevel" label as a workaround for `sort`
# inconsistencies between different utility versions
includelevel="$( dirname ${includelevel} )/toplevel"
fi
includelevel=$( echo ${includelevel} | tr '/' '.' )
if [[ "$level" != "$includelevel" ]]
then
echo $level $includelevel | tee -a paths.txt
fi
done
echo Sort and dedup paths
sort -ds paths.txt | uniq -c | tee sortedpaths.txt
mv sortedpaths.txt paths.txt
exec 3>&- #close fd 3
IFS=${oldifs}
unset oldifs
echo Split into flat-file database
exec 4<paths.txt # open paths.txt for input
while read -r -u 4 count level include
do
echo ${include} ${count} | tee -a includes/${level}
echo ${level} ${count} | tee -a includedby/${include}
done
exec 4>&- #close fd 4
loops="$( pwd )/loops.txt"
ordering="$( pwd )/ordering.txt"
pushd includes
echo Search for loops
# Redirect stdout to a file
exec 4>&1
exec 1>"${loops}"
for source in *
do
if [[ -f "$source" ]]
then
exec 5<"${source}" # open for input
while read -r -u 5 include includefreq
do
if [[ -f $include ]]
then
if grep -q -w $source $include
then
if grep -q -w "Loop: $include $source" "${loops}"
then
continue
fi
sourcefreq=$( grep -w $source $include | cut -d\ -f2 )
echo "Loop: $source $include"
# If the counts are close, indicate that the two modules are
# on the same level, though they shouldn't be
if [[ $(( $includefreq - $sourcefreq )) -gt 3 ]]
then
echo -e " $source > $include\n"
elif [[ $(( $sourcefreq - $includefreq )) -gt 3 ]]
then
echo -e " $include > $source\n"
elif [[ $sourcefreq -eq $includefreq ]]
then
echo -e " $include == $source\n"
else
echo -e " $include ~= $source\n"
fi
else
echo "$source > $include" >> "${ordering}"
fi
fi
done
exec 5>&- #close fd 5
fi
done
exec 1>&4 #close fd 1
exec 4>&- #close fd 4
cat "${ordering}"
cat "${loops}"
popd
popd
popd

View File

@@ -1,3 +1,54 @@
Loop: ripple.app ripple.core
ripple.app > ripple.core
Loop: ripple.app ripple.ledger
ripple.app > ripple.ledger
Loop: ripple.app ripple.net
ripple.app > ripple.net
Loop: ripple.app ripple.nodestore
ripple.app > ripple.nodestore
Loop: ripple.app ripple.overlay
ripple.overlay ~= ripple.app
Loop: ripple.app ripple.peerfinder
ripple.app > ripple.peerfinder
Loop: ripple.app ripple.protocol
ripple.app > ripple.protocol
Loop: ripple.app ripple.rpc
ripple.rpc > ripple.app
Loop: ripple.app ripple.shamap
ripple.app > ripple.shamap
Loop: ripple.basics ripple.core
ripple.core > ripple.basics
Loop: ripple.basics ripple.json
ripple.json ~= ripple.basics
Loop: ripple.basics ripple.protocol
ripple.protocol > ripple.basics
Loop: ripple.core ripple.net
ripple.net > ripple.core
Loop: ripple.ledger ripple.protocol
ripple.ledger > ripple.protocol
Loop: ripple.net ripple.rpc
ripple.rpc > ripple.net
Loop: ripple.nodestore ripple.overlay
ripple.overlay ~= ripple.nodestore
Loop: ripple.overlay ripple.rpc
ripple.rpc ~= ripple.overlay
Loop: test.app test.jtx
test.app > test.jtx
@@ -10,45 +61,3 @@ Loop: test.jtx test.toplevel
Loop: test.jtx test.unit_test
test.unit_test == test.jtx
Loop: xrpl.protocol xrpld.app
xrpld.app > xrpl.protocol
Loop: xrpld.app xrpld.core
xrpld.app > xrpld.core
Loop: xrpld.app xrpld.ledger
xrpld.app > xrpld.ledger
Loop: xrpld.app xrpld.net
xrpld.app > xrpld.net
Loop: xrpld.app xrpld.nodestore
xrpld.app > xrpld.nodestore
Loop: xrpld.app xrpld.overlay
xrpld.overlay == xrpld.app
Loop: xrpld.app xrpld.peerfinder
xrpld.app > xrpld.peerfinder
Loop: xrpld.app xrpld.rpc
xrpld.rpc > xrpld.app
Loop: xrpld.app xrpld.shamap
xrpld.app > xrpld.shamap
Loop: xrpld.core xrpld.net
xrpld.net > xrpld.core
Loop: xrpld.core xrpld.perflog
xrpld.perflog == xrpld.core
Loop: xrpld.net xrpld.rpc
xrpld.rpc > xrpld.net
Loop: xrpld.overlay xrpld.rpc
xrpld.rpc ~= xrpld.overlay
Loop: xrpld.perflog xrpld.rpc
xrpld.rpc ~= xrpld.perflog

View File

@@ -1,214 +1,227 @@
libxrpl.basics > xrpl.basics
libxrpl.crypto > xrpl.basics
libxrpl.json > xrpl.basics
libxrpl.json > xrpl.json
libxrpl.protocol > xrpl.basics
libxrpl.protocol > xrpl.hook
libxrpl.protocol > xrpl.json
libxrpl.protocol > xrpl.protocol
libxrpl.resource > xrpl.basics
libxrpl.resource > xrpl.resource
libxrpl.server > xrpl.basics
libxrpl.server > xrpl.json
libxrpl.server > xrpl.protocol
libxrpl.server > xrpl.server
test.app > test.shamap
ripple.app > ripple.basics
ripple.app > ripple.beast
ripple.app > ripple.conditions
ripple.app > ripple.consensus
ripple.app > ripple.crypto
ripple.app > ripple.json
ripple.app > ripple.resource
ripple.app > test.unit_test
ripple.basics > ripple.beast
ripple.conditions > ripple.basics
ripple.conditions > ripple.protocol
ripple.consensus > ripple.basics
ripple.consensus > ripple.beast
ripple.consensus > ripple.json
ripple.consensus > ripple.protocol
ripple.core > ripple.beast
ripple.core > ripple.json
ripple.core > ripple.protocol
ripple.crypto > ripple.basics
ripple.json > ripple.beast
ripple.ledger > ripple.basics
ripple.ledger > ripple.beast
ripple.ledger > ripple.core
ripple.ledger > ripple.json
ripple.net > ripple.basics
ripple.net > ripple.beast
ripple.net > ripple.json
ripple.net > ripple.protocol
ripple.net > ripple.resource
ripple.nodestore > ripple.basics
ripple.nodestore > ripple.beast
ripple.nodestore > ripple.core
ripple.nodestore > ripple.json
ripple.nodestore > ripple.protocol
ripple.nodestore > ripple.unity
ripple.overlay > ripple.basics
ripple.overlay > ripple.beast
ripple.overlay > ripple.core
ripple.overlay > ripple.json
ripple.overlay > ripple.peerfinder
ripple.overlay > ripple.protocol
ripple.overlay > ripple.resource
ripple.overlay > ripple.server
ripple.peerfinder > ripple.basics
ripple.peerfinder > ripple.beast
ripple.peerfinder > ripple.core
ripple.peerfinder > ripple.protocol
ripple.perflog > ripple.basics
ripple.perflog > ripple.beast
ripple.perflog > ripple.core
ripple.perflog > ripple.json
ripple.perflog > ripple.nodestore
ripple.perflog > ripple.protocol
ripple.perflog > ripple.rpc
ripple.protocol > ripple.beast
ripple.protocol > ripple.crypto
ripple.protocol > ripple.json
ripple.resource > ripple.basics
ripple.resource > ripple.beast
ripple.resource > ripple.json
ripple.resource > ripple.protocol
ripple.rpc > ripple.basics
ripple.rpc > ripple.beast
ripple.rpc > ripple.core
ripple.rpc > ripple.crypto
ripple.rpc > ripple.json
ripple.rpc > ripple.ledger
ripple.rpc > ripple.nodestore
ripple.rpc > ripple.protocol
ripple.rpc > ripple.resource
ripple.rpc > ripple.server
ripple.rpc > ripple.shamap
ripple.server > ripple.basics
ripple.server > ripple.beast
ripple.server > ripple.crypto
ripple.server > ripple.json
ripple.server > ripple.protocol
ripple.shamap > ripple.basics
ripple.shamap > ripple.beast
ripple.shamap > ripple.crypto
ripple.shamap > ripple.nodestore
ripple.shamap > ripple.protocol
test.app > ripple.app
test.app > ripple.basics
test.app > ripple.beast
test.app > ripple.core
test.app > ripple.json
test.app > ripple.ledger
test.app > ripple.overlay
test.app > ripple.protocol
test.app > ripple.resource
test.app > ripple.rpc
test.app > test.toplevel
test.app > test.unit_test
test.app > xrpl.basics
test.app > xrpld.app
test.app > xrpld.core
test.app > xrpld.ledger
test.app > xrpld.nodestore
test.app > xrpld.overlay
test.app > xrpld.rpc
test.app > xrpld.shamap
test.app > xrpl.hook
test.app > xrpl.json
test.app > xrpl.protocol
test.app > xrpl.resource
test.basics > ripple.basics
test.basics > ripple.beast
test.basics > ripple.json
test.basics > ripple.protocol
test.basics > ripple.rpc
test.basics > test.jtx
test.basics > test.unit_test
test.basics > xrpl.basics
test.basics > xrpld.perflog
test.basics > xrpld.rpc
test.basics > xrpl.json
test.basics > xrpl.protocol
test.beast > xrpl.basics
test.conditions > xrpl.basics
test.conditions > xrpld.conditions
test.beast > ripple.basics
test.beast > ripple.beast
test.conditions > ripple.basics
test.conditions > ripple.beast
test.conditions > ripple.conditions
test.consensus > ripple.app
test.consensus > ripple.basics
test.consensus > ripple.beast
test.consensus > ripple.consensus
test.consensus > ripple.core
test.consensus > ripple.ledger
test.consensus > ripple.protocol
test.consensus > test.csf
test.consensus > test.toplevel
test.consensus > test.unit_test
test.consensus > xrpl.basics
test.consensus > xrpld.app
test.consensus > xrpld.consensus
test.consensus > xrpld.core
test.consensus > xrpld.ledger
test.consensus > xrpl.json
test.consensus > xrpl.protocol
test.core > ripple.basics
test.core > ripple.beast
test.core > ripple.core
test.core > ripple.crypto
test.core > ripple.json
test.core > ripple.server
test.core > test.jtx
test.core > test.toplevel
test.core > test.unit_test
test.core > xrpl.basics
test.core > xrpld.core
test.core > xrpld.perflog
test.core > xrpl.json
test.core > xrpl.server
test.csf > xrpl.basics
test.csf > xrpld.consensus
test.csf > xrpl.json
test.csf > xrpl.protocol
test.formal_verification > xrpld.app
test.formal_verification > xrpld.consensus
test.csf > ripple.basics
test.csf > ripple.beast
test.csf > ripple.consensus
test.csf > ripple.json
test.csf > ripple.protocol
test.json > ripple.beast
test.json > ripple.json
test.json > test.jtx
test.json > xrpl.json
test.jtx > xrpl.basics
test.jtx > xrpld.app
test.jtx > xrpld.consensus
test.jtx > xrpld.core
test.jtx > xrpld.ledger
test.jtx > xrpld.net
test.jtx > xrpld.rpc
test.jtx > xrpl.hook
test.jtx > xrpl.json
test.jtx > xrpl.protocol
test.jtx > xrpl.resource
test.jtx > xrpl.server
test.jtx > ripple.app
test.jtx > ripple.basics
test.jtx > ripple.beast
test.jtx > ripple.consensus
test.jtx > ripple.core
test.jtx > ripple.json
test.jtx > ripple.ledger
test.jtx > ripple.net
test.jtx > ripple.protocol
test.jtx > ripple.resource
test.jtx > ripple.rpc
test.jtx > ripple.server
test.ledger > ripple.app
test.ledger > ripple.basics
test.ledger > ripple.beast
test.ledger > ripple.core
test.ledger > ripple.ledger
test.ledger > ripple.protocol
test.ledger > test.jtx
test.ledger > test.toplevel
test.ledger > xrpl.basics
test.ledger > xrpld.app
test.ledger > xrpld.core
test.ledger > xrpld.ledger
test.ledger > xrpl.protocol
test.net > ripple.net
test.net > test.jtx
test.net > test.toplevel
test.net > test.unit_test
test.nodestore > ripple.app
test.nodestore > ripple.basics
test.nodestore > ripple.beast
test.nodestore > ripple.core
test.nodestore > ripple.nodestore
test.nodestore > ripple.protocol
test.nodestore > ripple.unity
test.nodestore > test.jtx
test.nodestore > test.toplevel
test.nodestore > test.unit_test
test.nodestore > xrpl.basics
test.nodestore > xrpld.core
test.nodestore > xrpld.nodestore
test.nodestore > xrpld.unity
test.nodestore > xrpl.protocol
test.overlay > ripple.app
test.overlay > ripple.basics
test.overlay > ripple.beast
test.overlay > ripple.overlay
test.overlay > ripple.peerfinder
test.overlay > ripple.protocol
test.overlay > ripple.shamap
test.overlay > test.jtx
test.overlay > test.toplevel
test.overlay > test.unit_test
test.overlay > xrpl.basics
test.overlay > xrpld.app
test.overlay > xrpld.overlay
test.overlay > xrpld.peerfinder
test.overlay > xrpld.shamap
test.overlay > xrpl.protocol
test.peerfinder > ripple.basics
test.peerfinder > ripple.beast
test.peerfinder > ripple.core
test.peerfinder > ripple.peerfinder
test.peerfinder > ripple.protocol
test.peerfinder > test.beast
test.peerfinder > test.unit_test
test.peerfinder > xrpl.basics
test.peerfinder > xrpld.core
test.peerfinder > xrpld.peerfinder
test.peerfinder > xrpl.protocol
test.protocol > ripple.basics
test.protocol > ripple.beast
test.protocol > ripple.crypto
test.protocol > ripple.json
test.protocol > ripple.protocol
test.protocol > test.toplevel
test.protocol > xrpl.basics
test.protocol > xrpl.json
test.protocol > xrpl.protocol
test.rdb > test.jtx
test.rdb > test.toplevel
test.rdb > xrpld.app
test.rdb > xrpld.core
test.resource > ripple.basics
test.resource > ripple.beast
test.resource > ripple.resource
test.resource > test.unit_test
test.resource > xrpl.basics
test.resource > xrpl.resource
test.rpc > ripple.app
test.rpc > ripple.basics
test.rpc > ripple.beast
test.rpc > ripple.core
test.rpc > ripple.json
test.rpc > ripple.net
test.rpc > ripple.nodestore
test.rpc > ripple.overlay
test.rpc > ripple.protocol
test.rpc > ripple.resource
test.rpc > ripple.rpc
test.rpc > test.jtx
test.rpc > test.nodestore
test.rpc > test.toplevel
test.rpc > xrpl.basics
test.rpc > xrpld.app
test.rpc > xrpld.core
test.rpc > xrpld.net
test.rpc > xrpld.overlay
test.rpc > xrpld.rpc
test.rpc > xrpld.shamap
test.rpc > xrpl.hook
test.rpc > xrpl.json
test.rpc > xrpl.protocol
test.rpc > xrpl.resource
test.server > ripple.app
test.server > ripple.basics
test.server > ripple.beast
test.server > ripple.core
test.server > ripple.json
test.server > ripple.rpc
test.server > ripple.server
test.server > test.jtx
test.server > test.toplevel
test.server > test.unit_test
test.server > xrpl.basics
test.server > xrpld.app
test.server > xrpld.core
test.server > xrpld.rpc
test.server > xrpl.json
test.server > xrpl.server
test.shamap > ripple.basics
test.shamap > ripple.beast
test.shamap > ripple.nodestore
test.shamap > ripple.protocol
test.shamap > ripple.shamap
test.shamap > test.unit_test
test.shamap > xrpl.basics
test.shamap > xrpld.nodestore
test.shamap > xrpld.shamap
test.shamap > xrpl.protocol
test.toplevel > ripple.json
test.toplevel > test.csf
test.toplevel > xrpl.json
test.unit_test > xrpl.basics
xrpl.hook > xrpl.basics
xrpl.hook > xrpl.protocol
xrpl.json > xrpl.basics
xrpl.protocol > xrpl.basics
xrpl.protocol > xrpl.json
xrpl.resource > xrpl.basics
xrpl.resource > xrpl.json
xrpl.resource > xrpl.protocol
xrpl.server > xrpl.basics
xrpl.server > xrpl.json
xrpl.server > xrpl.protocol
xrpld.app > test.unit_test
xrpld.app > xrpl.basics
xrpld.app > xrpld.conditions
xrpld.app > xrpld.consensus
xrpld.app > xrpld.perflog
xrpld.app > xrpl.hook
xrpld.app > xrpl.json
xrpld.app > xrpl.resource
xrpld.conditions > xrpl.basics
xrpld.conditions > xrpl.protocol
xrpld.consensus > xrpl.basics
xrpld.consensus > xrpl.json
xrpld.consensus > xrpl.protocol
xrpld.core > xrpl.basics
xrpld.core > xrpl.json
xrpld.core > xrpl.protocol
xrpld.ledger > xrpl.basics
xrpld.ledger > xrpld.core
xrpld.ledger > xrpl.json
xrpld.ledger > xrpl.protocol
xrpld.net > xrpl.basics
xrpld.net > xrpl.json
xrpld.net > xrpl.protocol
xrpld.net > xrpl.resource
xrpld.nodestore > xrpl.basics
xrpld.nodestore > xrpld.core
xrpld.nodestore > xrpld.unity
xrpld.nodestore > xrpl.json
xrpld.nodestore > xrpl.protocol
xrpld.overlay > xrpl.basics
xrpld.overlay > xrpld.core
xrpld.overlay > xrpld.peerfinder
xrpld.overlay > xrpld.perflog
xrpld.overlay > xrpl.json
xrpld.overlay > xrpl.protocol
xrpld.overlay > xrpl.resource
xrpld.overlay > xrpl.server
xrpld.peerfinder > xrpl.basics
xrpld.peerfinder > xrpld.core
xrpld.peerfinder > xrpl.protocol
xrpld.perflog > xrpl.basics
xrpld.perflog > xrpl.json
xrpld.perflog > xrpl.protocol
xrpld.rpc > xrpl.basics
xrpld.rpc > xrpld.core
xrpld.rpc > xrpld.ledger
xrpld.rpc > xrpld.nodestore
xrpld.rpc > xrpld.shamap
xrpld.rpc > xrpl.json
xrpld.rpc > xrpl.protocol
xrpld.rpc > xrpl.resource
xrpld.rpc > xrpl.server
xrpld.shamap > xrpl.basics
xrpld.shamap > xrpld.nodestore
xrpld.shamap > xrpl.protocol
test.unit_test > ripple.basics
test.unit_test > ripple.beast

View File

@@ -12,45 +12,27 @@ endif()
# Fix "unrecognized escape" issues when passing CMAKE_MODULE_PATH on Windows.
file(TO_CMAKE_PATH "${CMAKE_MODULE_PATH}" CMAKE_MODULE_PATH)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake")
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake")
if(POLICY CMP0144)
cmake_policy(SET CMP0144 NEW)
endif()
project (xrpl)
project (rippled)
set(Boost_NO_BOOST_CMAKE ON)
# make GIT_COMMIT_HASH define available to all sources
find_package(Git)
if(Git_FOUND)
execute_process(COMMAND ${GIT_EXECUTABLE} --git-dir=${CMAKE_CURRENT_SOURCE_DIR}/.git rev-parse HEAD
execute_process(COMMAND ${GIT_EXECUTABLE} --git-dir=${CMAKE_CURRENT_SOURCE_DIR}/.git describe --always --abbrev=40
OUTPUT_STRIP_TRAILING_WHITESPACE OUTPUT_VARIABLE gch)
if(gch)
set(GIT_COMMIT_HASH "${gch}")
message(STATUS gch: ${GIT_COMMIT_HASH})
add_definitions(-DGIT_COMMIT_HASH="${GIT_COMMIT_HASH}")
endif()
execute_process(COMMAND ${GIT_EXECUTABLE} --git-dir=${CMAKE_CURRENT_SOURCE_DIR}/.git rev-parse --abbrev-ref HEAD
OUTPUT_STRIP_TRAILING_WHITESPACE OUTPUT_VARIABLE gb)
if(gb)
set(GIT_BRANCH "${gb}")
message(STATUS gb: ${GIT_BRANCH})
add_definitions(-DGIT_BRANCH="${GIT_BRANCH}")
endif()
endif() #git
# make SOURCE_ROOT_PATH define available for logging
set(SOURCE_ROOT_PATH "${CMAKE_CURRENT_SOURCE_DIR}/src/")
add_definitions(-DSOURCE_ROOT_PATH="${SOURCE_ROOT_PATH}")
# BEAST_ENHANCED_LOGGING - adds file:line numbers and formatting to logs
# Automatically enabled for Debug builds via generator expression
# Can be explicitly controlled with -DBEAST_ENHANCED_LOGGING=ON/OFF
option(BEAST_ENHANCED_LOGGING "Include file and line numbers in log messages (auto: Debug=ON, Release=OFF)" OFF)
message(STATUS "BEAST_ENHANCED_LOGGING option: ${BEAST_ENHANCED_LOGGING}")
if(thread_safety_analysis)
add_compile_options(-Wthread-safety -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -DRIPPLE_ENABLE_THREAD_SAFETY_ANNOTATIONS)
add_compile_options("-stdlib=libc++")
@@ -68,6 +50,11 @@ if(CMAKE_TOOLCHAIN_FILE)
endif()
endif()
if (NOT USE_CONAN)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake")
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Builds/CMake/deps")
endif()
include (CheckCXXCompilerFlag)
include (FetchContent)
include (ExternalProject)
@@ -79,7 +66,9 @@ endif ()
include(RippledSanity)
include(RippledVersion)
include(RippledSettings)
if (NOT USE_CONAN)
include(RippledNIH)
endif()
# this check has to remain in the top-level cmake
# because of the early return statement
if (packages_only)
@@ -91,66 +80,102 @@ endif ()
include(RippledCompiler)
include(RippledInterface)
include(deps/Boost)
find_package(OpenSSL 1.1.1 REQUIRED)
set_target_properties(OpenSSL::SSL PROPERTIES
INTERFACE_COMPILE_DEFINITIONS OPENSSL_NO_SSL2
)
set(SECP256K1_INSTALL TRUE)
add_subdirectory(external/secp256k1)
add_library(secp256k1::secp256k1 ALIAS secp256k1)
add_subdirectory(external/ed25519-donna)
add_subdirectory(external/antithesis-sdk)
find_package(gRPC REQUIRED)
find_package(lz4 REQUIRED)
# Target names with :: are not allowed in a generator expression.
# We need to pull the include directories and imported location properties
# from separate targets.
find_package(LibArchive REQUIRED)
find_package(SOCI REQUIRED)
find_package(SQLite3 REQUIRED)
option(rocksdb "Enable RocksDB" ON)
if(rocksdb)
find_package(RocksDB REQUIRED)
set_target_properties(RocksDB::rocksdb PROPERTIES
INTERFACE_COMPILE_DEFINITIONS RIPPLE_ROCKSDB_AVAILABLE=1
)
target_link_libraries(ripple_libs INTERFACE RocksDB::rocksdb)
endif()
find_package(nudb REQUIRED)
find_package(date REQUIRED)
find_package(xxHash REQUIRED)
find_package(magic_enum REQUIRED)
include(deps/WasmEdge)
if(TARGET nudb::core)
set(nudb nudb::core)
elseif(TARGET NuDB::nudb)
set(nudb NuDB::nudb)
###
if (NOT USE_CONAN)
set(SECP256K1_INSTALL TRUE)
add_subdirectory(external/secp256k1)
add_library(secp256k1::secp256k1 ALIAS secp256k1)
add_subdirectory(external/ed25519-donna)
include(deps/Boost)
include(deps/OpenSSL)
# include(deps/Secp256k1)
# include(deps/Ed25519-donna)
include(deps/Lz4)
include(deps/Libarchive)
include(deps/Sqlite)
include(deps/Soci)
include(deps/Snappy)
include(deps/Rocksdb)
include(deps/Nudb)
include(deps/date)
# include(deps/Protobuf)
# include(deps/gRPC)
include(deps/cassandra)
include(deps/Postgres)
include(deps/WasmEdge)
else()
message(FATAL_ERROR "unknown nudb target")
endif()
target_link_libraries(ripple_libs INTERFACE ${nudb})
include(conan/Boost)
find_package(OpenSSL 1.1.1 REQUIRED)
set_target_properties(OpenSSL::SSL PROPERTIES
INTERFACE_COMPILE_DEFINITIONS OPENSSL_NO_SSL2
)
set(SECP256K1_INSTALL TRUE)
add_subdirectory(external/secp256k1)
add_library(secp256k1::secp256k1 ALIAS secp256k1)
add_subdirectory(external/ed25519-donna)
find_package(gRPC REQUIRED)
find_package(lz4 REQUIRED)
# Target names with :: are not allowed in a generator expression.
# We need to pull the include directories and imported location properties
# from separate targets.
find_package(LibArchive REQUIRED)
find_package(SOCI REQUIRED)
find_package(SQLite3 REQUIRED)
find_package(Snappy REQUIRED)
find_package(wasmedge REQUIRED)
option(rocksdb "Enable RocksDB" ON)
if(rocksdb)
find_package(RocksDB REQUIRED)
set_target_properties(RocksDB::rocksdb PROPERTIES
INTERFACE_COMPILE_DEFINITIONS RIPPLE_ROCKSDB_AVAILABLE=1
)
target_link_libraries(ripple_libs INTERFACE RocksDB::rocksdb)
endif()
find_package(nudb REQUIRED)
find_package(date REQUIRED)
find_package(xxHash REQUIRED)
if(TARGET nudb::core)
set(nudb nudb::core)
elseif(TARGET NuDB::nudb)
set(nudb NuDB::nudb)
else()
message(FATAL_ERROR "unknown nudb target")
endif()
target_link_libraries(ripple_libs INTERFACE ${nudb})
target_link_libraries(ripple_libs INTERFACE
ed25519::ed25519
LibArchive::LibArchive
lz4::lz4
OpenSSL::Crypto
OpenSSL::SSL
secp256k1::secp256k1
soci::soci
SQLite::SQLite3
)
if(reporting)
find_package(cassandra-cpp-driver REQUIRED)
find_package(PostgreSQL REQUIRED)
target_link_libraries(ripple_libs INTERFACE
cassandra-cpp-driver::cassandra-cpp-driver
PostgreSQL::PostgreSQL
)
endif()
target_link_libraries(ripple_libs INTERFACE
ed25519::ed25519
lz4::lz4
OpenSSL::Crypto
OpenSSL::SSL
# Ripple::grpc_pbufs
# Ripple::pbufs
secp256k1::secp256k1
soci::soci
SQLite::SQLite3
)
endif()
if(coverage)
include(RippledCov)
endif()
set(PROJECT_EXPORT_SET RippleExports)
###
include(RippledCore)
if (NOT USE_CONAN)
include(deps/Protobuf)
include(deps/gRPC)
endif()
include(RippledInstall)
include(RippledMultiConfig)
include(RippledDocs)
include(RippledValidatorKeys)

View File

@@ -71,248 +71,27 @@ The source must be formatted according to the style guide below.
Header includes must be [levelized](./Builds/levelization).
Changes should be usually squashed down into a single commit.
Some larger or more complicated change sets make more sense,
and are easier to review if organized into multiple logical commits.
Either way, all commits should fit the following criteria:
* Changes should be presented in a single commit or a logical
sequence of commits.
Specifically, chronological commits that simply
reflect the history of how the author implemented
the change, "warts and all", are not useful to
reviewers.
* Every commit should have a [good message](#good-commit-messages).
to explain a specific aspects of the change.
* Every commit should be signed.
* Every commit should be well-formed (builds successfully,
unit tests passing), as this helps to resolve merge
conflicts, and makes it easier to use `git bisect`
to find bugs.
### Good commit messages
Refer to
["How to Write a Git Commit Message"](https://cbea.ms/git-commit/)
for general rules on writing a good commit message.
tl;dr
> 1. Separate subject from body with a blank line.
> 2. Limit the subject line to 50 characters.
> * [...]shoot for 50 characters, but consider 72 the hard limit.
> 3. Capitalize the subject line.
> 4. Do not end the subject line with a period.
> 5. Use the imperative mood in the subject line.
> * A properly formed Git commit subject line should always be able
> to complete the following sentence: "If applied, this commit will
> _your subject line here_".
> 6. Wrap the body at 72 characters.
> 7. Use the body to explain what and why vs. how.
In addition to those guidelines, please add one of the following
prefixes to the subject line if appropriate.
* `fix:` - The primary purpose is to fix an existing bug.
* `perf:` - The primary purpose is performance improvements.
* `refactor:` - The changes refactor code without affecting
functionality.
* `test:` - The changes _only_ affect unit tests.
* `docs:` - The changes _only_ affect documentation. This can
include code comments in addition to `.md` files like this one.
* `build:` - The changes _only_ affect the build process,
including CMake and/or Conan settings.
* `chore:` - Other tasks that don't affect the binary, but don't fit
any of the other cases. e.g. formatting, git settings, updating
Github Actions jobs.
Whenever possible, when updating commits after the PR is open, please
add the PR number to the end of the subject line. e.g. `test: Add
unit tests for Feature X (#1234)`.
## Pull requests
In general, pull requests use `develop` as the base branch.
(Hotfixes are an exception.)
If your changes are not quite ready, but you want to make it easily available
for preliminary examination or review, you can create a "Draft" pull request.
While a pull request is marked as a "Draft", you can rebase or reorganize the
commits in the pull request as desired.
Github pull requests are created as "Ready" by default, or you can mark
a "Draft" pull request as "Ready".
Once a pull request is marked as "Ready",
any changes must be added as new commits. Do not
force-push to a branch in a pull request under review.
(This includes rebasing your branch onto the updated base branch.
Use a merge operation, instead or hit the "Update branch" button
at the bottom of the Github PR page.)
Changes to pull requests must be added as new commits.
Once code reviewers have started looking at your code, please avoid
force-pushing a branch in a pull request.
This preserves the ability for reviewers to filter changes since their last
review.
A pull request must obtain **approvals from at least two reviewers**
before it can be considered for merge by a Maintainer.
A pull request must obtain **approvals from at least two reviewers** before it
can be considered for merge by a Maintainer.
Maintainers retain discretion to require more approvals if they feel the
credibility of the existing approvals is insufficient.
Pull requests must be merged by [squash-and-merge][2]
to preserve a linear history for the `develop` branch.
### When and how to merge pull requests
#### "Passed"
A pull request should only have the "Passed" label added when it
meets a few criteria:
1. It must have two approving reviews [as described
above](#pull-requests). (Exception: PRs that are deemed "trivial"
only need one approval.)
2. All CI checks must be complete and passed. (One-off failures may
be acceptable if they are related to a known issue.)
3. The PR must have a [good commit message](#good-commit-messages).
* If the PR started with a good commit message, and it doesn't
need to be updated, the author can indicate that in a comment.
* Any contributor, preferably the author, can leave a comment
suggesting a commit message.
* If the author squashes and rebases the code in preparation for
merge, they should also ensure the commit message(s) are updated
as well.
4. The PR branch must be up to date with the base branch (usually
`develop`). This is usually accomplised by merging the base branch
into the feature branch, but if the other criteria are met, the
changes can be squashed and rebased on top of the base branch.
5. Finally, and most importantly, the author of the PR must
positively indicate that the PR is ready to merge. That can be
accomplished by adding the "Passed" label if their role allows,
or by leaving a comment to the effect that the PR is ready to
merge.
Once the "Passed" label is added, a maintainer may merge the PR at
any time, so don't use it lightly.
#### Instructions for maintainers
The maintainer should double-check that the PR has met all the
necessary criteria, and can request additional information from the
owner, or additional reviews, and can always feel free to remove the
"Passed" label if appropriate. The maintainer has final say on
whether a PR gets merged, and are encouraged to communicate and
issues or concerns to other maintainers.
##### Most pull requests: "Squash and merge"
Most pull requests don't need special handling, and can simply be
merged using the "Squash and merge" button on the Github UI. Update
the suggested commit message if necessary.
##### Slightly more complicated pull requests
Some pull requests need to be pushed to `develop` as more than one
commit. There are multiple ways to accomplish this. If the author
describes a process, and it is reasonable, follow it. Otherwise, do
a fast forward only merge (`--ff-only`) on the command line and push.
Either way, check that:
* The commits are based on the current tip of `develop`.
* The commits are clean: No merge commits (except when reverse
merging), no "[FOLD]" or "fixup!" messages.
* All commits are signed. If the commits are not signed by the author, use
`git commit --amend -S` to sign them yourself.
* At least one (but preferably all) of the commits has the PR number
in the commit message.
**Never use the "Create a merge commit" or "Rebase and merge"
functions!**
##### Releases, release candidates, and betas
All releases, including release candidates and betas, are handled
differently from typical PRs. Most importantly, never use
the Github UI to merge a release.
1. There are two possible conditions that the `develop` branch will
be in when preparing a release.
1. Ready or almost ready to go: There may be one or two PRs that
need to be merged, but otherwise, the only change needed is to
update the version number in `BuildInfo.cpp`. In this case,
merge those PRs as appropriate, updating the second one, and
waiting for CI to finish in between. Then update
`BuildInfo.cpp`.
2. Several pending PRs: In this case, do not use the Github UI,
because the delays waiting for CI in between each merge will be
unnecessarily onerous. Instead, create a working branch (e.g.
`develop-next`) based off of `develop`. Squash the changes
from each PR onto the branch, one commit each (unless
more are needed), being sure to sign each commit and update
the commit message to include the PR number. You may be able
to use a fast-forward merge for the first PR. The workflow may
look something like:
```
git fetch upstream
git checkout upstream/develop
git checkout -b develop-next
# Use -S on the ff-only merge if prbranch1 isn't signed.
# Or do another branch first.
git merge --ff-only user1/prbranch1
git merge --squash user2/prbranch2
git commit -S
git merge --squash user3/prbranch3
git commit -S
[...]
git push --set-upstream origin develop-next
</pre>
```
2. Create the Pull Request with `release` as the base branch. If any
of the included PRs are still open,
[use closing keywords](https://docs.github.com/articles/closing-issues-using-keywords)
in the description to ensure they are closed when the code is
released. e.g. "Closes #1234"
3. Instead of the default template, reuse and update the message from
the previous release. Include the following verbiage somewhere in
the description:
```
The base branch is release. All releases (including betas) go in
release. This PR will be merged with --ff-only (not squashed or
rebased, and not using the GitHub UI) to both release and develop.
```
4. Sign-offs for the three platforms usually occur offline, but at
least one approval will be needed on the PR.
5. Once everything is ready to go, open a terminal, and do the
fast-forward merges manually. Do not push any branches until you
verify that all of them update correctly.
```
git fetch upstream
git checkout -b upstream--develop -t upstream/develop || git checkout upstream--develop
git reset --hard upstream/develop
# develop-next must be signed already!
git merge --ff-only origin/develop-next
git checkout -b upstream--release -t upstream/release || git checkout upstream--release
git reset --hard upstream/release
git merge --ff-only origin/develop-next
# Only do these 3 steps if pushing a release. No betas or RCs
git checkout -b upstream--master -t upstream/master || git checkout upstream--master
git reset --hard upstream/master
git merge --ff-only origin/develop-next
# Check that all of the branches are updated
git log -1 --oneline
# The output should look like:
# 02ec8b7962 (HEAD -> upstream--master, origin/develop-next, upstream--release, upstream--develop, develop-next) Set version to 2.2.0-rc1
# Note that all of the upstream--develop/release/master are on this commit.
# (Master will be missing for betas, etc.)
# Just to be safe, do a dry run first:
git push --dry-run upstream-push HEAD:develop
git push --dry-run upstream-push HEAD:release
# git push --dry-run upstream-push HEAD:master
# Now push
git push upstream-push HEAD:develop
git push upstream-push HEAD:release
# git push upstream-push HEAD:master
# Don't forget to tag the release, too.
git tag <version number>
git push upstream-push <version number>
```
6. Finally
[create a new release on Github](https://github.com/XRPLF/rippled/releases).
# Style guide
@@ -338,84 +117,12 @@ this:
You can format individual files in place by running `clang-format -i <file>...`
from any directory within this project.
There is a Continuous Integration job that runs clang-format on pull requests. If the code doesn't comply, a patch file that corrects auto-fixable formatting issues is generated.
To download the patch file:
1. Next to `clang-format / check (pull_request) Failing after #s` -> click **Details** to open the details page.
2. Left menu -> click **Summary**
3. Scroll down to near the bottom-right under `Artifacts` -> click **clang-format.patch**
4. Download the zip file and extract it to your local git repository. Run `git apply [patch-file-name]`.
5. Commit and push.
You can install a pre-commit hook to automatically run `clang-format` before every commit:
```
pip3 install pre-commit
pre-commit install
```
## Contracts and instrumentation
We are using [Antithesis](https://antithesis.com/) for continuous fuzzing,
and keep a copy of [Antithesis C++ SDK](https://github.com/antithesishq/antithesis-sdk-cpp/)
in `external/antithesis-sdk`. One of the aims of fuzzing is to identify bugs
by finding external conditions which cause contracts violations inside `rippled`.
The contracts are expressed as `XRPL_ASSERT` or `UNREACHABLE` (defined in
`include/xrpl/beast/utility/instrumentation.h`), which are effectively (outside
of Antithesis) wrappers for `assert(...)` with added name. The purpose of name
is to provide contracts with stable identity which does not rely on line numbers.
When `rippled` is built with the Antithesis instrumentation enabled
(using `voidstar` CMake option) and ran on the Antithesis platform, the
contracts become
[test properties](https://antithesis.com/docs/using_antithesis/properties.html);
otherwise they are just like a regular `assert`.
To learn more about Antithesis, see
[How Antithesis Works](https://antithesis.com/docs/introduction/how_antithesis_works.html)
and [C++ SDK](https://antithesis.com/docs/using_antithesis/sdk/cpp/overview.html#)
We continue to use the old style `assert` or `assert(false)` in certain
locations, where the reporting of contract violations on the Antithesis
platform is either not possible or not useful.
For this reason:
* The locations where `assert` or `assert(false)` contracts should continue to be used:
* `constexpr` functions
* unit tests i.e. files under `src/test`
* unit tests-related modules (files under `beast/test` and `beast/unit_test`)
* Outside of the listed locations, do not use `assert`; use `XRPL_ASSERT` instead,
giving it unique name, with the short description of the contract.
* Outside of the listed locations, do not use `assert(false)`; use
`UNREACHABLE` instead, giving it unique name, with the description of the
condition being violated
* The contract name should start with a full name (including scope) of the
function, optionally a named lambda, followed by a colon ` : ` and a brief
(typically at most five words) description. `UNREACHABLE` contracts
can use slightly longer descriptions. If there are multiple overloads of the
function, use common sense to balance both brevity and unambiguity of the
function name. NOTE: the purpose of name is to provide stable means of
unique identification of every contract; for this reason try to avoid elements
which can change in some obvious refactors or when reinforcing the condition.
* Contract description typically (except for `UNREACHABLE`) should describe the
_expected_ condition, as in "I assert that _expected_ is true".
* Contract description for `UNREACHABLE` should describe the _unexpected_
situation which caused the line to have been reached.
* Example good name for an
`UNREACHABLE` macro `"Json::operator==(Value, Value) : invalid type"`; example
good name for an `XRPL_ASSERT` macro `"Json::Value::asCString : valid type"`.
* Example **bad** name
`"RFC1751::insert(char* s, int x, int start, int length) : length is greater than or equal zero"`
(missing namespace, unnecessary full function signature, description too verbose).
Good name: `"ripple::RFC1751::insert : minimum length"`.
* In **few** well-justified cases a non-standard name can be used, in which case a
comment should be placed to explain the rationale (example in `contract.cpp`)
* Do **not** rename a contract without a good reason (e.g. the name no longer
reflects the location or the condition being checked)
* Do not use `std::unreachable`
* Do not put contracts where they can be violated by an external condition
(e.g. timing, data payload before mandatory validation etc.) as this creates
bogus bug reports (and causes crashes of Debug builds)
## Unit Tests
To execute all unit tests:
@@ -488,10 +195,9 @@ existing maintainer without a vote.
## Current Maintainers
* [Richard Holland](https://github.com/RichardAH) (XRPL Labs + INFTF)
* [Denis Angell](https://github.com/dangell7) (XRPL Labs + INFTF)
* [Wietse Wind](https://github.com/WietseWind) (XRPL Labs + INFTF)
* [tequ](https://github.com/tequdev) (Independent + INFTF)
* [Richard Holland](https://github.com/RichardAH) (XRPL Labs + XRP Ledger Foundation)
* [Denis Angell](https://github.com/dangell7) (XRPL Labs + XRP Ledger Foundation)
* [Wietse Wind](https://github.com/WietseWind) (XRPL Labs + XRP Ledger Foundation)
[1]: https://docs.github.com/en/get-started/quickstart/contributing-to-projects

Some files were not shown because too many files have changed in this diff Show More