Compare commits

...

256 Commits

Author SHA1 Message Date
Mayukha Vadari
53f57cad83 fix typos 2025-10-09 16:39:33 -04:00
Mayukha Vadari
df8768d24a Merge branch 'develop' into copilot/fix-f350b804-905b-4a06-ab84-d0f12e5b0dd1 2025-10-09 16:05:54 -04:00
Mayukha Vadari
9e4bc87146 clean up 2025-10-09 16:05:45 -04:00
Bart
46ba8a28fe refactor: Update Conan dependencies: OpenSSL (#5873)
This change bumps OpenSSL from 1.1.1w to 3.6.0.
2025-10-09 13:27:26 -04:00
Bronek Kozicki
5ecde3cf39 Add vault invariants (#5518)
This change adds invariants for SingleAssetVault #5224 (XLS-065), which had been intentionally skipped earlier to keep the SAV PR size manageable.
2025-10-08 15:04:02 +00:00
tequ
620fb26823 test: Add more tests for Simulate RPC metadata (#5827) 2025-10-08 14:36:09 +00:00
Bronek Kozicki
6b6b213cf5 chore: Fix release build error (#5864)
This change fixes a release build error with GCC 15.2.

The `fields` variable is only used in `XRPL_ASSERT`, which evaluates to nothing in a Release build, leaving the variable unused. This change silences the build warning.

Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
2025-10-08 13:45:44 +00:00
Bart
f61086b43c refactor: Update CI strategy matrix to use new RHEL 9 and RHEL 10 images (#5856)
This change uses the new RHEL 9 and 10 images to build and test the binary, and adds support for having different Docker image SHAs per distro-compiler combination.

Instead of supporting RHEL each minor version, we are simplifying our pipelines by only supporting RHEL major versions. Our CI Docker images have already been updated accordingly, and we recently added support for RHEL 10 as well. Up until now, the CI Docker images had all been rebuilt at the same time, but that is not necessarily true as the most recent push to the CI repo has shown where the RHEL images now have a different SHA than the Debian and Ubuntu ones.

Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
2025-10-08 13:15:24 +00:00
Mayukha Vadari
176fd2b6e4 chore: exclude all UNREACHABLE blocks from codecov (#5846) 2025-10-08 09:25:51 +01:00
Bart
2df730438d Set version to 3.0.0-b1 (#5859) 2025-10-07 20:28:19 +00:00
Mayukha Vadari
3f5c350c5a refactor source code 2025-10-07 14:09:40 -04:00
Mayukha Vadari
5b22d7a574 fix tests 2025-10-06 16:50:46 -04:00
Mayukha Vadari
9cc2e2cc16 clean up testCreateOfferExpiration 2025-10-06 16:50:46 -04:00
Mayukha Vadari
7e0d5ad933 roll back unnecessary changes 2025-10-06 16:50:46 -04:00
Mayukha Vadari
1425345fed clean up testAcceptOfferInvalid 2025-10-06 16:50:46 -04:00
Mayukha Vadari
5524610c82 exclude tecINTERNALs from codecov 2025-10-06 16:50:46 -04:00
Mayukha Vadari
11de1418e6 [Claude] clean up 2025-10-06 16:50:46 -04:00
Mayukha Vadari
36f27a63fc [manual] fix all the tests 2025-10-06 16:50:46 -04:00
Mayukha Vadari
b5a94445aa [Claude] make progress 2025-10-06 16:50:46 -04:00
Mayukha Vadari
ab71cfb206 [WIP] fix some stuff, hit the limit before I could continue 2025-10-06 16:50:46 -04:00
Mayukha Vadari
85120ed502 [Claude] Temporarily disable new tests to isolate amendment impact 2025-10-06 16:50:45 -04:00
Mayukha Vadari
9f7e9e4197 [Claude] Make existing NFToken tests amendment-aware for fixExpiredNFTokenOfferRemoval 2025-10-06 16:50:45 -04:00
Mayukha Vadari
55b3bc2332 [Claude] Fix NFToken expired offer test timing logic 2025-10-06 16:50:45 -04:00
Mayukha Vadari
cbac095cf2 [Claude] attempt fix 2025-10-06 16:50:45 -04:00
copilot-swe-agent[bot]
b589551a5d Fix test logic: use robust timing pattern for expired offer tests
Co-authored-by: mvadari <8029314+mvadari@users.noreply.github.com>
2025-10-06 16:50:45 -04:00
copilot-swe-agent[bot]
0fd1dc12a5 Fix test logic: use fresh expiration times for buy offer and brokered scenarios
Co-authored-by: mvadari <8029314+mvadari@users.noreply.github.com>
2025-10-06 16:50:45 -04:00
copilot-swe-agent[bot]
858dc2b63b Fix test logic: create offers with future expiration times, then advance ledger time
Co-authored-by: mvadari <8029314+mvadari@users.noreply.github.com>
2025-10-06 16:50:45 -04:00
copilot-swe-agent[bot]
41bfa3819c Fix test logic error: buy offer should exist after creation regardless of amendment
Co-authored-by: mvadari <8029314+mvadari@users.noreply.github.com>
2025-10-06 16:50:45 -04:00
copilot-swe-agent[bot]
be5d105f07 Simplify amendment check using tweakedFeatures[] operator instead of count()
Co-authored-by: mvadari <8029314+mvadari@users.noreply.github.com>
2025-10-06 16:50:45 -04:00
copilot-swe-agent[bot]
0493180658 Fix build error: use bo->key() instead of getIndex() and resolve clang-format issues
Co-authored-by: mvadari <8029314+mvadari@users.noreply.github.com>
2025-10-06 16:50:45 -04:00
copilot-swe-agent[bot]
76a1831c6d Improve test coverage for brokered expired offers
Co-authored-by: mvadari <8029314+mvadari@users.noreply.github.com>
2025-10-06 16:50:45 -04:00
copilot-swe-agent[bot]
98e9fce6ca Add fixExpiredNFTokenOfferRemoval amendment and implementation
Co-authored-by: mvadari <8029314+mvadari@users.noreply.github.com>
2025-10-06 16:50:43 -04:00
copilot-swe-agent[bot]
35eaf4fe8a Initial plan 2025-10-06 16:50:16 -04:00
Bronek Kozicki
5d79bfc531 Remove bogus coverage warning (#5838) 2025-10-02 11:54:09 +01:00
Ed Hennis
51ef35ab55 fix: Transaction sig checking functions do not get a full context (#5829)
Fixes a (currently harmless) bug introduced by PR #5594
2025-10-01 20:58:43 +00:00
Valentin Balaschenko
330a3215bc fix: FD/handle guarding + exponential backoff (#5823) 2025-10-01 12:57:33 +01:00
Ed Hennis
85c2ceacde Merge tag '2.6.1' into ximinez/merge261
2.6.1

* tag '2.6.1':
  Set version to 2.6.1
  Set version to 2.6.1-rc2
  Mark PermissionDelegation as unsupported
2025-09-30 19:10:51 -04:00
Ed Hennis
70d5c624e8 Set version to 2.6.1 2025-09-30 16:09:11 -04:00
yinyiqian1
8e4fda160d Rename flags for DynamicMPT (#5820) 2025-09-30 18:49:53 +00:00
Bart
072b1c442c chore: Set free-form CI inputs as env vars (#5822)
This change moves CI values that could be user-provided into environment variables.
2025-09-30 19:46:10 +02:00
Ayaz Salikhov
294e03ecf5 ci: Upload artifacts during build and test in a separate job (#5817) 2025-09-30 16:15:24 +00:00
Ed Hennis
550f90a75e refactor: Add support for extra transaction signatures (#5594)
* Restructures Transactor signature checking code to be able to handle a `sigObject`, which may be the full transaction, or may be an object field containing a separate signature. Either way, the `sigObject` can be a single- or multi-sign signature.
2025-09-29 22:11:53 +00:00
Ed Hennis
d67dcfe3c4 refactor: Restructure Transactor::preflight to reduce boilerplate (#5592)
* Restructures `Transactor::preflight` to create several functions that will remove the need for error-prone boilerplate code in derived classes' implementations of `preflight`.
2025-09-29 17:31:42 -04:00
Mayukha Vadari
0fd2f715bb switch fixIncludeKeyletFields to Supported::yes (#5819) 2025-09-27 09:04:04 +02:00
Mayukha Vadari
807462b191 Add STInt32 as a new SType (#5788)
This change adds `STInt32` as a new `SType` under the `STInteger` umbrella, with `SType` value `12`. This is the first and only `STInteger` type that supports negative values.
2025-09-26 20:13:15 +00:00
Ayaz Salikhov
19c4226d3d ci: Call all reusable workflows reusable (#5818) 2025-09-26 18:33:42 +00:00
Mayukha Vadari
d02c306f1e test: add more comprehensive tests for FeeVote (#5746)
This change adds more comprehensive tests for the `FeeVote` module, which previously only checked the basics, and not the more comprehensive flows in that class.
2025-09-26 17:40:19 +00:00
Jingchen
cfd26f444c fix: Address http header case sensitivity (#5767)
This change makes the regex in `HttpClient.cpp` that matches the content-length http header case insensitive to improve compatibility, as http headers are case insensitive.
2025-09-26 11:40:43 +00:00
tequ
2c3024716b change fixPriceOracleOrder to Supported::yes (#5749) 2025-09-26 12:07:48 +01:00
Bart
a12f5de68d chore: Pin all CI Docker tags (#5813)
To avoid surprises and ensure reproducibility, this change pins all CI Docker image tags to the latest version in the XRPLF/CI repo.
2025-09-25 16:08:07 +00:00
Bronek Kozicki
51c5f2bfc9 Improve ValidatorList invalid UNL manifest logging (#5804)
This change raises logging severity from `INFO` to `WARN` when handling UNL manifest signed with an unexpected / invalid key. It also changes the internal error code for an invalid format of UNL manifest to `invalid` (from `untrusted`).

This is a follow up to problems experienced by an UNL node due to old manifest key configured in `validators.txt`, which would be easier to diagnose with improved logging.

It also replaces a log line with `UNREACHABLE` for an impossible situation when we match UNL manifest key against a configured key which has an invalid type (we cannot configure such a key because of checks when loading configured keys).
2025-09-25 16:14:29 +02:00
Valentin Balaschenko
73ff54143d docs: Add warning about using std::counting_semaphore (#5595)
This adds a comment to avoid using `std::counting_semaphore` until the minimum compiler versions of GCC and Clang have been updated to no longer contain the bug that is present in older compilers.
2025-09-23 13:26:26 +02:00
Bart
08b136528e Revert "Update Conan dependencies: OpenSSL" (#5807)
This change reverts #5617, because it will require extensive testing that will take up more time than we have before the next scheduled release.

Reverting this change does not mean we are abandoning it. We aim to pick it back up once there's a sufficient time window to allow for testing on multiple distros running a mixture of OpenSSL 1.x and 3.x.
2025-09-22 18:27:02 +00:00
Mayukha Vadari
6b8a589447 test: Add STInteger and STParsedJSON tests (#5726)
This change is to improve code coverage (and to simplify #5720 and #5725); there is otherwise no change in functionality. The change adds basic tests for `STInteger` and `STParsedJSON`, so it becomes easier to test smaller changes to the types, as well as removes `STParsedJSONArray`, since it is not used anywhere (including in Clio).
2025-09-22 20:00:31 +02:00
Ed Hennis
ffeabc9642 refactor: Simplify STParsedJSON with some helper functions (#5591)
- Add code coverage for STParsedJSON edge cases

Co-authored-by: Denis Angell <dangell@transia.co>
2025-09-18 19:04:40 +00:00
Ed Hennis
3cbdf818a7 Miscellaneous refactors and updates (#5590)
- Added a new Invariant: `ValidPseudoAccounts` which checks that all pseudo-accounts behave consistently through creation and updates, and that no "real" accounts look like pseudo-accounts (which means they don't have a 0 sequence). 
- `to_short_string(base_uint)`. Like `to_string`, but only returns the first 8 characters. (Similar to how a git commit ID can be abbreviated.) Used as a wrapped sink to prefix most transaction-related messages. More can be added later.
- `XRPL_ASSERT_PARTS`. Convenience wrapper for `XRPL_ASSERT`, which takes the `function` and `description` as separate parameters.
- `SField::sMD_PseudoAccount`. Metadata option for `SField` definitions to indicate that the field, if set in an `AccountRoot` indicates that account is a pseudo-account. Removes the need for hard-coded field lists all over the place. Added the flag to `AMMID` and `VaultID`.
- Added functionality to `SField` ctor to detect both code and name collisions using asserts. And require all SFields to have a name
- Convenience type aliases `STLedgerEntry::const_pointer` and `STLedgerEntry::const_ref`. (`SLE` is an alias to `STLedgerEntry`.)
- Generalized `feeunit.h` (`TaggedFee`) into `unit.h` (`ValueUnit`) and added new "BIPS"-related tags for future use. Also refactored the type restrictions to use Concepts.
- Restructured `transactions.macro` to do two big things
	1. Include the `#include` directives for transactor header files directly in the macro file. Removes the need to update `applySteps.cpp` and the resulting conflicts.
	2. Added a `privileges` parameter to the `TRANSACTION` macro, which specifies some of the operations a transaction is allowed to do. These `privileges` are enforced by invariant checks. Again, removed the need to update scattered lists of transaction types in various checks.
- Unit tests:
	1.  Moved more helper functions into `TestHelpers.h` and `.cpp`. 
	2. Cleaned up the namespaces to prevent / mitigate random collisions and ambiguous symbols, particularly in unity builds.
	3. Generalized `Env::balance` to add support for `MPTIssue` and `Asset`.
	4. Added a set of helper classes to simplify `Env` transaction parameter classes: `JTxField`, `JTxFieldWrapper`, and a bunch of classes derived or aliased from it. For an example of how awesome it is, check the changes `src/test/jtx/escrow.h` for how much simpler the definitions are for `finish_time`, `cancel_time`, `condition`, and `fulfillment`. 
	5. Generalized several of the amount-related helper classes to understand `Asset`s.
     6. `env.balance` for an MPT issuer will return a negative number (or 0) for consistency with IOUs.
2025-09-18 17:55:49 +00:00
Bronek Kozicki
c46888f8f7 Set version to 2.6.1-rc2 2025-09-18 18:09:04 +01:00
Bronek Kozicki
2ae65d2fdb Mark PermissionDelegation as unsupported 2025-09-18 18:04:12 +01:00
Ed Hennis
bd834c87e0 Merge tag '2.6.1-rc1' into ximinez/merge-261rc1
2.6.1-rc1

* tag '2.6.1-rc1':
  Set version to 2.6.1-rc1
  Downgrade to boost 1.83
2025-09-18 11:46:22 -04:00
Jingchen
dc8b37a524 refactor: Modularise ledger (#5493)
This change moves the ledger code to libxrpl.
2025-09-18 11:12:24 -04:00
Bronek Kozicki
617a895af5 chore: Add unit tests dir to code coverage excludes (#5803)
This change excludes unit test code from code coverage reporting.
2025-09-18 06:30:34 -04:00
Bart
1af1048c58 chore: Build and test all configs for daily scheduled run (#5801)
This change re-enables building and testing all configurations, but only for the daily scheduled run. Previously all configurations were run for each merge into the develop branch, but that overwhelmed both the GitHub runners and the Conan remote, and thus they were limited to just a subset of configurations. Now that the number of jobs is limited via `max-parallel: 10`, we should be able to safely enable building all configurations again. However, building them all once a day instead of for each PR merge should be sufficient.
2025-09-17 19:17:48 -04:00
Ed Hennis
f07ba87e51 Merge tag '2.5.1' into upstream--develop
- Ensures the commits don't get orphaned, even though the relevant code
  changes are already included.

* tag '2.5.1':
  Set version to 2.5.1
  Fix: Don't flag consensus as stalled prematurely (#5658)
2025-09-17 19:05:14 -04:00
Bart
e66558a883 chore: Limits CI build and test parallelism to reduce resource contention (#5799)
GitHub runners have a limit on how many concurrent jobs they can actually process (even though they will try to run them all at the same time), and similarly the Conan remote cannot handle hundreds of concurrent requests. Previously, the Conan dependency uploading was already limited to max 10 jobs running in parallel, and this change makes the same change to the build+test workflow.
2025-09-17 22:55:00 +00:00
Mayukha Vadari
510314d344 fix(amendment): Add missing fields for keylets to ledger objects (#5646)
This change adds a fix amendment (`fixIncludeKeyletFields`) that adds:
* `sfSequence` to `Escrow` and `PayChannel`
* `sfOwner` to `SignerList`
* `sfOracleDocumentID` to `Oracle`

This ensures that all ledger entries hold all the information needed to determine their keylet.
2025-09-17 21:34:47 +00:00
yinyiqian1
37b951859c Rename mutable flags (#5797)
This is a minor change on top of #5705
2025-09-17 21:43:04 +01:00
Jingchen
9494fc9668 chore: Use self hosted windows runners (#5780)
This changes switches from the GitHub-managed Windows runners to self-hosted runners to significantly reduce build time.
2025-09-17 09:29:15 -04:00
Bronek Kozicki
8d01f35eb9 Set version to 2.6.1-rc1 2025-09-16 15:35:54 -04:00
Bronek Kozicki
1020a32d76 Downgrade to boost 1.83 2025-09-16 15:35:47 -04:00
Vito Tumas
17a2606591 Bugfix: Adds graceful peer disconnection (#5669)
The XRPL establishes connections in three stages: first a TCP connection, then a TLS/SSL handshake to secure the connection, and finally an upgrade to the bespoke XRP Ledger peer-to-peer protocol. During connection termination, xrpld directly closes the TCP connection, bypassing the TLS/SSL shutdown handshake. This makes peer disconnection diagnostics more difficult - abrupt TCP termination appears as if the peer crashed rather than disconnected gracefully.

This change refactors the connection lifecycle with the following changes:
- Enhanced outgoing connection logic with granular timeouts for each connection stage (TCP, TLS, XRPL handshake) to improve diagnostic capabilities
- Updated both PeerImp and ConnectAttempt to use proper asynchronous TLS shutdown procedures for graceful connection termination
2025-09-16 10:51:55 +01:00
yinyiqian1
ccb9f1e42d Support DynamicMPT XLS-94d (#5705)
* extends the functionality of the MPTokenIssuanceSet transaction, allowing the issuer to update fields or flags that were explicitly marked as mutable during creation.
2025-09-15 19:42:36 +00:00
Bart
3e4e9a2ddc Only notify clio for PRs targeting the release and master branches (#5794)
Clio should only be notified when releases are about to be made, instead of for all PR, so this change only notifies Clio when a PR targets the release or master branch.
2025-09-15 13:28:47 -04:00
Bart
4caebfbd0e refactor: Wrap GitHub CI conditionals in curly braces (#5796)
This change wraps all GitHub conditionals in `${{ .. }}`, both for consistency and to reduce unexpected failures, because it was previously noticed that not all conditionals work without those curly braces.
2025-09-15 16:26:08 +00:00
Denis Angell
37c377a1b6 Fix: EscrowTokenV1 (#5571)
* resolves an accounting inconsistency in MPT escrows where transfer fees were not properly handled when unlocking escrowed tokens.
2025-09-15 14:48:47 +00:00
Jingchen
bd182c0a3e fix: Skip processing transaction batch if the batch is empty (#5670)
Avoids an assertion failure in NetworkOPsImp::apply in the unlikely event that all incoming transactions are invalid.
2025-09-15 13:51:19 +00:00
Ayaz Salikhov
406c26cc72 ci: Fix conan secrets in upload-conan-deps (#5785)
- Accounts for some variables that were changed and missed when the reusable workflow was removed.
2025-09-12 17:09:42 +00:00
Jingchen
9bd1ce436a Fix code coverage error (#5765)
* Fix the issue where COVERAGE_CXX_COMPILER_FLAGS is never used
2025-09-12 15:13:27 +00:00
Ayaz Salikhov
f69ad4eff6 docs: Add remote to conan lock create command (#5770)
* docs: Add remote to `conan lock create` command
* Document error resolution for conan package issues
* Update BUILD.md
* Add more info about lockfiles
2025-09-11 15:42:27 +00:00
Mayukha Vadari
6fe0599cc2 refactor: clean up CTID.h (#5681) 2025-09-11 14:49:26 +00:00
tequ
e6f8bc720f Add additional metadata to simulate response (#5754) 2025-09-11 15:17:06 +01:00
Ayaz Salikhov
fbd60fc000 ci: Use pre-commit reusable workflow (#5772) 2025-09-11 13:58:11 +01:00
yinyiqian1
61d628d654 fix: Add restrictions to Permission Delegation: fixDelegateV1_1 (#5650)
- Amendment: fixDelegateV1_1
- In DelegateSet, disallow invalid PermissionValues like 0, and transaction values when the transaction's amendment is not enabled. Acts as if the transaction doesn't exist, which is the same thing older versions without the amendment will do.
- Payment burn/mint should disallow DEX currency exchange.
- Support MPT for Payment burn/mint.
2025-09-10 17:47:33 +00:00
Ayaz Salikhov
3d92375d12 ci: Add missing dependencies to workflows (#5783) 2025-09-10 08:20:45 +00:00
Ayaz Salikhov
cdbe70b2a7 ci: Use default conan install format (#5784) 2025-09-10 07:35:58 +00:00
Bronek Kozicki
f6426ca183 Switch CI pipeline bookworm:gcc-13 from arm64 to amd64 (#5779) 2025-09-09 21:23:07 +00:00
Ayaz Salikhov
e5f7a8442d ci: Change upload-conan-deps workflow is run (#5782)
- Don't run upload-conan-deps in PRs, unless the PR changes the workflow file.
- Change cron schedule for uploading Conan dependencies to run after work hours for most dev.
2025-09-09 16:21:12 -04:00
Ayaz Salikhov
e67e0395df ci: Limit number of parallel jobs in "upload-conan-deps" (#5781)
- This should prevent Artifactory from being overloaded by too many requests at a time.
- Uses "max-parallel" to limit the build job to 10 simultaneous instances.
- Only run the minimal matrix on PRs.
2025-09-09 19:47:06 +00:00
Ed Hennis
148f669a25 chore: "passed" fails if any previous jobs fail or are cancelled (#5776)
For the purposes of being able to merge a PR, Github Actions jobs count as passed if they ran and passed, or were skipped.

With this change, if any of the jobs that "passed" depends on fail or are cancelled, then "passed" will fail. If they all succeed or are skipped, then "passed" is skipped, which does not prevent a merge.

This saves spinning up a runner in the usual case where things work, and will simplify our branch protection rules, so that only "passed" will need to be checked.
2025-09-09 18:07:04 +00:00
yinyiqian1
f1eaa6a264 enable fixAMMClawbackRounding (#5750) 2025-09-09 15:57:28 +00:00
Ayaz Salikhov
da4c8c9550 ci: Only run build-test/notify-clio if should-run indicates to (#5777)
- Fixes an issue introduced by #5762 which removed the transitive `should-run` check from these two jobs.
2025-09-09 11:25:41 -04:00
Wo Jake
bcde2790a4 Update old links & descriptions in README.md (#4701) 2025-09-08 18:03:20 +00:00
Ayaz Salikhov
9ebeb413e4 feat: Implement separate upload workflow (#5762)
* feat: Implement separate upload workflow
* Use cleanup-workspace
* Name some workflows reusable
* Add dependencies
2025-09-08 15:15:59 +00:00
Bronek Kozicki
6d40b882a4 Switch on-trigger to minimal build (#5773) 2025-09-08 13:54:50 +00:00
tzchenxixi
9fe0a154f1 chore: remove redundant word in comment (#5752) 2025-09-08 13:13:32 +00:00
Ayaz Salikhov
cb52c9af00 fix: Remove extra @ in notify-clio.yml (#5771) 2025-09-05 14:08:17 +01:00
Mayukha Vadari
6bf8338038 chore: Add conan.lock to workflow file checks (#5769)
* Add conan.lock to workflow file checks
* Add conan.lock to on-trigger.yml
2025-09-04 22:32:23 +00:00
Ayaz Salikhov
b0f4174e47 chore: Use tooling provided by pre-commit (#5753) 2025-09-04 20:30:54 +00:00
Ayaz Salikhov
3865dde0b8 fix: Add missing info to notify-clio workflow (#5761)
* Add missing info to notify-clio workflow, as conan_ref
2025-09-04 19:26:57 +00:00
Ayaz Salikhov
811c980821 ci: Use cleanup-workspace action (#5763)
* ci: Use cleanup-workspace action
* Use latest version
2025-09-04 16:27:30 +01:00
Bronek Kozicki
cf5f65b68e Add Scale to SingleAssetVault (#5652)
* Add and Scale to VaultCreate
* Add round-trip calculation to VaultDeposit VaultWithdraw and VaultClawback
* Implement Number::truncate() for VaultClawback
* Add rounding to DepositWithdraw
* Disallow zero shares withdraw or deposit with tecPRECISION_LOSS
* Return tecPATH_DRY on overflow when converting shares/assets
* Remove empty shares MPToken in clawback or withdraw (except for vault owner)
* Implicitly create shares MPToken for vault owner in VaultCreate
* Review feedback: defensive checks in shares/assets calculations

---------

Co-authored-by: Ed Hennis <ed@ripple.com>
2025-09-04 08:54:24 +00:00
Jingchen
c38f2a3f2e Fix coverage parameter (#5760) 2025-09-03 16:08:02 +00:00
Ed Hennis
16c2ff97cc Set version to 2.5.1 2025-09-03 10:20:12 -04:00
Ed Hennis
32043463a8 Fix: Don't flag consensus as stalled prematurely (#5658)
Fix stalled consensus detection to prevent false positives in situations where there are no disputed transactions.

Stalled consensus detection was added to 2.5.0 in response to a network consensus halt that caused a round to run for over an hour. However, it has a flaw that makes it very easy to have false positives. Those false positives are usually mitigated by other checks that prevent them from having an effect, but there have been several instances of validators "running ahead" because there are circumstances where the other checks are "successful", allowing the stall state to be checked.
2025-09-03 10:12:30 -04:00
Ayaz Salikhov
724e9b1313 chore: Use conan lockfile (#5751)
* chore: Use conan lockfile
* Add windows-specific dependencies as well
* Add more info about lockfiles
* Update lockfile to latest version
* Update BUILD.md with conan install note
2025-09-03 10:24:07 +00:00
Bronek Kozicki
2e6f00aef2 Add required disable_ccache option (#5756) 2025-09-03 09:25:52 +01:00
Mayukha Vadari
e0b9812fc5 Refactor ledger_entry RPC source code and tests (#5237)
This is a major refactor of LedgerEntry.cpp. It adds a number of helper functions to make the code easier to maintain.

It also splits up the ledger and ledger_entry tests into different files, and cleans up the ledger_entry tests to make them easier to write and maintain.

This refactor also caught a few bugs in some of the other RPC processing, so those are fixed along the way.
2025-08-29 15:52:09 -04:00
Vito Tumas
e4fdf33158 adds additional logging to differentiate why connections were refused (#5690)
This is a follow-up to PR #5664 that further improves the specificity of logging for refused peer connections. The previous changes did not account for several key scenarios, leading to potentially misleading log messages.

It addresses the following 

- Inbound Disabled: Connections are now explicitly logged as rejected when the server is not configured to accept inbound peers. Previously, this was logged as the server being "full," which was technically correct but lacked diagnostic clarity.
- Duplicate Connections: The logging now distinguishes between two types of duplicate connection refusals:
    - When a peer with the same node public key is already connected (duplicate connection).
    -  When a connection is rejected because the limit for connections from a single IP address has been reached.

These changes provide more accurate and actionable diagnostic information when analyzing peer connection behavior.
2025-08-29 00:00:38 +00:00
Ed Hennis
6e814d7ebd chore: Run CI jobs in more situations, and add "passed" job (#5739)
Test jobs will run if
* Either the PR is non-draft or has the "DraftRunCI" label set *AND*
* One of the following:
	* Certain files were changed *OR*
	* The PR is non-draft and has the "Ready to merge" flag *OR*
	* The workflow is being run from the merge queue.

Additionally, a meta "passed" job was added that is dependent on all the other test jobs, so the required jobs list under branch protection rules only needs to specify "passed" to ensure that *either* all the test jobs pass *or* all the test jobs are skipped because they don't need to be run.

This allows PRs that don't affect the build or binary to be merged without overriding.
2025-08-28 20:33:11 +00:00
Ayaz Salikhov
1e37d00d6c ci: Use XRPLF/prepare-runner action (#5740)
* ci: Use XRPLF/prepare-runner action
* Remove some old boost workaround
2025-08-28 19:32:49 +00:00
Michael Legleux
87ea3ba65d Merge remote-tracking branch 'upstream/release' into merge2.6.0 2025-08-28 13:51:17 -04:00
Bronek Kozicki
dedf3d3983 Remove extraneous // LCOV_EXCL_START, and fix CMake warning (#5744)
* Remove extraneous // LCOV_EXCL_START
* Fix "At least one COMMAND must be given" CMake warning
2025-08-28 10:15:17 -04:00
Michael Legleux
2df7dcfdeb Set version to 2.6.0 2025-08-27 10:25:53 -07:00
Alex Kremer
1506e65558 refactor: Update to Boost 1.88 (#5570)
This updates Boost to 1.88, which is needed because Clio wants to move to 1.88 as that fixes several ASAN false positives around coroutine usage. In order for Clio to move to newer boost, libXRPL needs to move too. Hence the changes in this PR. A lot has changed between 1.83 and 1.88 so there are lots of changes in the diff, especially in regards to Boost.Asio and coroutines in particular.
2025-08-27 09:34:50 +00:00
Bart
808c86663c fix: Add codecov token to trigger workflow (#5736)
This change adds the Codecov token to the on-trigger workflow.
2025-08-26 19:07:23 -04:00
Bart
92431a4238 chore: Add support for merge_group event (#5734)
This change adds support for the merge_group CI event, which will allow us to enable merge queues.
2025-08-26 17:12:37 -04:00
Bart
285120684c refactor: Replace 'on: pull_request: paths' by 'changed-files' action (#5728)
This PR moves the list of files from the `paths:` section in the `on: pull_request` into a separate job.
2025-08-26 16:00:00 -04:00
Bart
77fef8732b fix: Simplify PR pipeline trigger rules (#5727)
This change removes `labeled` and `unlabeled` as pipeline trigger actions, and instead adds `reopened` and `ready_for_review`. The logic whether to run the pipeline jobs is then simplified, although to get a draft PR with the `DraftCIRun` label to run it can be necessary to close and reopen a PR.
2025-08-25 13:32:07 -04:00
Ed Hennis
7775c725f3 Merge remote-tracking branch 'upstream/release' into ximinez/merge-release 2025-08-22 19:56:21 -04:00
Bart
c61096239c chore: Remove codecov token check to support tokenless uploads on forks (#5722) 2025-08-22 23:31:01 +00:00
Ed Hennis
c5fe970646 Set version to 2.6.0-rc3 2025-08-22 17:32:31 -04:00
Ed Hennis
c57cd8b23e Revert "perf: Move mutex to the partition level (#5486)"
This reverts commit 94decc753b.
2025-08-22 17:30:08 -04:00
Bart
c14ce956ad chore: Update clang-format and prettier with pre-commit (#5709)
The change updates how clang-format is called in CI and locally, and adds prettier to the pre-commit hook. Proto files are now also formatted, while external files are excluded.
2025-08-22 17:37:11 +00:00
Mayukha Vadari
095dc4d9cc fix(test): handle null metadata for unvalidated tx in Env::meta (#5715)
This change handles errors better when calling `env.meta`. It prints some debug help and throws an error if `env.meta` is going to return a `nullptr`.
2025-08-22 16:15:03 +00:00
Bronek Kozicki
2e255812ae chore: Workaround for CI build errors on arm64 (#5717)
CI builds with `clang-20` on `linux/arm64` are failing due to boost 1.86. This is hopefully fixed in version 1.88.
2025-08-22 10:58:36 -04:00
Bart
896b8c3b54 chore: Fix file formatting (#5718) 2025-08-22 10:02:56 -04:00
Bart
58dd07bbdf fix: Skip notify-clio when running in a fork, reorder config fields (#5712)
This change will skip running the notify-clio job when a PR is created from a fork, and reorders the strategy matrix configuration fields so GitHub will more clearly show which configuration is running.
2025-08-21 16:32:04 -04:00
Bart
b13370ac0d chore: Reverts formatting changes to external files, adds formatting changes to proto files (#5711)
This change reverts the formatting applied to external files and adds formatting of proto files.

As clang-format will complain if a proto file is modified or moved, since the .clang-format file does not explicitly contain a section for proto files, the change has been included in this PR as well.
2025-08-21 15:22:25 -04:00
Bart
f847e3287c Update Conan dependencies: OpenSSL (#5617)
This change updates OpenSSL from 1.1.1w to 3.5.2. The code works as-is, but many functions have been marked as deprecated and thus will need to be rewritten. For now we explicitly add the `-DOPENSSL_SUPPRESS_DEPRECATED` to give us time to do so, while providing us with the benefits of the updated version.
2025-08-21 07:41:00 -04:00
Bart
56c1e078f2 fix: Correctly check for build_only when deciding whether to run tests (#5708)
This change modifies the `build_only` check used to determine whether to run tests. For easier debugging in the future it also prints out the contents of the strategy matrix.
2025-08-20 19:25:40 -04:00
Bart
afc05659ed fix: Adjust the CI workflows (#5700) 2025-08-19 12:46:38 -04:00
Bart
b04d239926 fix: Modify jobs to use '>>' instead of 'tee' for GITHUB_OUTPUT (#5699) 2025-08-18 10:49:55 -04:00
Bart
dc1caa41b2 refactor: Revamp CI workflows (#5661)
This change refactors the CI workflows to leverage the new CI Docker images for Debian, Red Hat, and Ubuntu.
2025-08-18 10:21:43 -04:00
Jingchen
ceb0ce5634 refactor: Decouple net from xrpld and move rpc-related classes to the rpc folder (#5477)
As a step of modularisation, this change moves code from `xrpld` to `libxrpl`.
2025-08-15 23:27:13 +00:00
Michael Legleux
fb89213d4d Set version to 2.6.0-rc2 2025-08-15 14:50:35 -07:00
Bart
d8628d481d docs: Updates list of maintainers and reviewers (#5687) 2025-08-14 16:17:37 -04:00
Elliot.
a14551b151 fix: Change log to debug level for AMM offer retrieval and IOU payment check (#5686)
Reduce log noise by changing two log statements from error/warn level to debug level. These logs occur during normal operation when AMM offers are not available or when IOU authorization checks fail, which are expected scenarios that don't require an elevated log level.
2025-08-14 12:28:01 -04:00
Bart
de33a6a241 fix: Add -Wno-deprecated-declarations for Clang only (#5680)
This change adds `-Wno-deprecated-declarations` for Clang only (not for GCC) builds in `cmake/RippledCompiler.cmake`.
2025-08-14 06:07:09 -04:00
Elliot.
28eec6ce1b Update .git-blame-ignore-revs for #5657 (#5675)
Now that #5657 has been squashed and merged, we can add its commit hash to .git-blame-ignore-revs.
2025-08-13 18:00:22 +00:00
Bronek Kozicki
c9a723128a Fix BUILD.md instruction (#5676) 2025-08-13 07:23:36 -04:00
Michael Legleux
da82e52613 Set version to 2.6.0-rc1 2025-08-12 13:40:34 -07:00
Vito Tumas
c9d73b6135 fix: Improve logging of the reason to refuse a peer connection (#5664)
Currently, all peer connection rejections are logged with the reason "slots full". This is inaccurate, as the PeerFinder can also reject connections if they are a duplicate. This change updates the logging logic to correctly report the specific reason (full or duplicate) for a rejected peer connection, providing more accurate diagnostic information.
2025-08-11 18:52:47 +00:00
Oleksandr Hrabar
b7ed99426b fix: Make test suite names match the directory name (#5597)
This change fixes the suite names all around the test files, to make them match to the folder name in which this test files are located. Also, the RCL test files are relocated to the consensus folder, because they are testing consensus functionality.
2025-08-11 14:12:36 -04:00
Mayukha Vadari
97f0747e10 chore: Run prettier on all files (#5657) 2025-08-11 16:15:42 +00:00
Bronek Kozicki
abf12db788 chore: Set CONAN_REMOTE_URL also for forks (#5662)
This change replaces the configuration variable with the hardcoded `https://conan.ripplex.io`, making it possible for PRs from forks to use our Conan remote containing workarounds.
2025-08-11 13:02:03 +00:00
Bart
bdfc376951 chore: Cleanup bin/ directory (#5660)
This change removes ancient and unused files from the `bin/` directory.
2025-08-11 11:24:24 +00:00
Jingchen
b40a3684ae perf: Optimize hash performance by avoiding allocating hash state object (#5469)
We're currently calling `XXH3_createState` and `XXH3_freeState` when hashing an object. However, it may be slow because they call `malloc` and `free`, which may affect the performance. This change avoids the use of the streaming API as much as possible by using an internal buffer.
2025-08-11 06:21:26 -04:00
Ed Hennis
86ef16dbeb Fix: Don't flag consensus as stalled prematurely (#5627)
Fix stalled consensus detection to prevent false positives in situations where there are no disputed transactions.

Stalled consensus detection was added to 2.5.0 in response to a network consensus halt that caused a round to run for over an hour. However, it has a flaw that makes it very easy to have false positives. Those false positives are usually mitigated by other checks that prevent them from having an effect, but there have been several instances of validators "running ahead" because there are circumstances where the other checks are "successful", allowing the stall state to be checked.
2025-08-08 17:13:32 -04:00
Bart
39b5031ab5 Switch Conan 1 commands to Conan 2 and fix credentials (#5655)
This change updates some incorrect Conan commands for Conan 2. As some flags do not exist in Conan 2, such as --settings build_type=[configuration], the commands have been adjusted accordingly. This change further uses the org-level variables and secrets rather than the repo-level ones.
2025-08-08 12:47:36 +00:00
Valentin Balaschenko
94decc753b perf: Move mutex to the partition level (#5486)
This change introduces two key optimizations:
* Mutex scope reduction: Limits the lock to individual partitions within `TaggedCache`, reducing contention.
* Decoupling: Removes the tight coupling between `LedgerHistory` and `TaggedCache`, improving modularity and testability.

Lock contention analysis based on eBPF showed significant improvements as a result of this change.
2025-08-07 17:04:07 -04:00
Bart
991891625a Upload Conan dependencies upon merge into develop (#5654)
This change uploads built Conan dependencies to the Conan remote upon merge into the develop branch.

At the moment, whenever Conan dependencies change, we need to remember to manually push them to our Conan remote, so they are cached for future reuse. If we forget to do so, these changed dependencies need to be rebuilt over and over again, which can take a long time.
2025-08-07 06:52:58 -04:00
Bart
69314e6832 refactor: Remove external libraries as they are hosted in our Conan Center Index fork (#5643)
This change:
* Removes the patched Conan recipes from the `external/` directory.
* Adds instructions for contributors how to obtain our patched recipes.
* Updates the Conan remote name and remote URL (the underlying package repository isn't changed).
* If the remote already exists, updates the URL instead of removing and re-adding.
  * This is not done for the libXRPL job as it still uses Conan 1. This job will be switched to Conan 2 soon.
* Removes duplicate Conan remote CI pipeline steps.
* Overwrites the existing global.conf on MacOS and Windows machines, as those do not run CI pipelines in isolation but all share the same Conan installation; appending the same config over and over bloats the file.
2025-08-06 15:46:13 +00:00
Bronek Kozicki
dbeb841b5a docs: Update BUILD.md for Conan 2 (#5478)
This change updates BUILD.md for Conan 2, add fixes/workarounds for Apple Clang 17, Clang 20 and CMake 4. This also removes (from BUILD.md only) workarounds for compiler versions which we no longer support e.g. Clang 15 and adds compilation flag -Wno-deprecated-declarations to enable building with Clang 20 on Linux.
2025-08-06 10:18:41 +00:00
tequ
4eae037fee fix: Ensures canonical order for PriceDataSeries upon PriceOracle creation (#5485)
This change fixes an issue where the order of `PriceDataSeries` was out of sync between when `PriceOracle` was created and when it was updated. Although they are registered in the canonical order when updated, they are created using the order specified in the transaction; this change ensures that they are also registered in the canonical order when created.
2025-08-05 13:08:59 -04:00
Jingchen
b5a63b39d3 refactor: Decouple ledger from xrpld/app (#5492)
This change decouples `ledger` from `xrpld/app`, and therefore fully clears the path to the modularisation of the ledger component. Before this change, `View.cpp` relied on `MPTokenAuthorize::authorize; this change moves `MPTokenAuthorize::authorize` to `View.cpp` to invert the dependency, making ledger a standalone module.
2025-08-05 15:28:56 +00:00
Denis Angell
6419f9a253 docs: Set up developer environment with specific XCode version (#5645) 2025-08-04 10:54:54 -04:00
Ayaz Salikhov
31c99caa65 Revert "ci: Build all conan dependencies from source for now (#5623)" (#5639)
This reverts commit 9b45b6888b.
2025-07-31 14:01:43 -04:00
Bronek Kozicki
d835e97490 Fix crash in Slot::deletePeer (#5635)
Fix crash due to recurrent call to `Slot::deletePeer` (via `OverlayImpl::unsquelch`) when a peer is disconnected at just the wrong moment.
2025-07-31 13:08:34 -04:00
Shawn Xie
baf4b8381f fix DeliveredAmount and delivered_amount in transaction metadata for direct MPT transfer (#5569)
The Payment transaction metadata is missing the `DeliveredAmount` field that displays the actual amount delivered to the destination excluding transfer fees. This amendment fixes this problem.
2025-07-29 17:02:33 +00:00
Ayaz Salikhov
9b45b6888b ci: Build all conan dependencies from source for now (#5623) 2025-07-29 15:29:38 +00:00
Bronek Kozicki
7179ce9c58 Build options cleanup (#5581)
As we no longer support old compiler versions, we are bringing back some warnings by removing no longer relevant `-Wno-...` options.
2025-07-25 15:48:22 -04:00
Bart
921aef9934 Updates Conan dependencies: Boost 1.86 (#5264) 2025-07-25 11:54:02 -04:00
Bronek Kozicki
e7a7bb83c1 VaultWithdraw destination account bugfix (#5572)
#5224 added (among other things) a `VaultWithdraw` transaction that allows setting the recipient of the withdrawn funds in the `Destination` transaction field. This technically turns this transaction into a payment, and in some respect the implementation does follow payment rules, e.g. enforcement of `lsfRequireDestTag` or `lsfDepositAuth`, or that MPT transfer has destination `MPToken`. However for IOUs, it missed verification that the destination account has a trust line to the asset issuer. Since the default behavior of `accountSendIOU` is to create this trust line (if missing), this is what `VaultWithdraw` currently does. This is incorrect, since the `Destination` might not be interested in holding the asset in question; this basically enables spammy transfers. This change, therefore, removes automatic creation of a trust line to the `Destination` account in `VaultWithdraw`.
2025-07-25 13:53:25 +00:00
Bart
5c2a3a2779 refactor: Update rocksdb (#5568)
This change updates RocksDB to its latest version. RocksDB is backward-compatible, so even though this is a major version bump, databases created with previous versions will continue to function.

The external RocksDB folder is removed, as the latest version available via Conan Center no longer needs custom patches.
2025-07-24 14:53:14 -04:00
Bronek Kozicki
b2960b9e7f Switch instrumentation workflow to use dependencies (#5607)
Before `XRPLF/ci` images, we did not have a `dependencies:` job for clang-16, so `instrumentation:` had to build its own dependencies. Now we have clang-16 Conan dependencies built in a separate job that can be used.
2025-07-24 09:20:50 -04:00
Bronek Kozicki
5713f9782a chore: Rename conan profile to default (#5599)
This change renames the `libxrpl` profile to `default` to make it more usable.
2025-07-24 10:35:47 +00:00
Chenna Keshava B S
60e340d356 Include network_id in validations and subscription stream responses (#5579)
This change includes `network_id` data in the validations and ledger subscription stream responses, as well as unit tests to validate the response fields. Fixes #4783
2025-07-23 17:53:18 +00:00
Bronek Kozicki
80d82c5b2b Add support for DomainID in MPTokenIssuance transactions (#5509)
This change adds support for `DomainID` to existing transactions `MPTokenIssuanceCreate` and `MPTokenIssuanceSet`.

In #5224 `DomainID` was added as an access control mechanism for `SingleAssetVault`. The actual implementation of this feature lies in `MPToken` and `MPTokenIssuance`, hence it makes sense to enable the use of `DomainID` also in `MPTokenIssuanceCreate` and `MPTokenIssuanceSet`, following same rules as in Vault:

* `MPTokenIssuanceCreate` and `MPTokenIssuanceSet` can only set `DomainID` if flag `MPTRequireAuth` is set.
* `MPTokenIssuanceCreate` requires that `DomainID` be a non-zero, uint256 number.
* `MPTokenIssuanceSet` allows `DomainID` to be zero (or empty) in which case it will remove `DomainID` from the `MPTokenIssuance` object.

The change is amendment-gated by `SingleAssetVault`. This is a non-breaking change because `SingleAssetVault` amendment is `Supported::no`, i.e. at this moment considered a work in progress, which cannot be enabled on the network.
2025-07-23 13:21:30 -04:00
Vlad
433eeabfa5 chore: Remove unused code after flow cross retirement (#5575)
After the `FlowCross` amendment was retired (#5562), there was still some unused code left. This change removes the remaining remnants.
2025-07-23 13:57:51 +00:00
Jingchen
faa781b71f Remove obsolete owner pays fee feature and XRPL_ABANDON stanza (#5550)
If a feature was never voted on then it is safe to remove.
2025-07-23 13:27:41 +00:00
Valentin Balaschenko
c233df720a refactor: Makes HashRouter flags more type-safe (#5371)
This change addresses the issue #5336: Refactor HashRouter flags to be more type-safe.

* Switched numeric flags to enum type.
* Updated unit tests
2025-07-23 12:03:12 +00:00
Bronek Kozicki
7ff4f79d30 Fix clang-format CI job (#5598)
For jobs running in containers, $GITHUB_WORKSPACE and ${{ github.workspace }} might not be the same directory. The actions/checkout step is supposed to checkout into `$GITHUB_WORKSPACE` and then add it to safe.directory (see instructions at https://github.com/actions/checkout), but that's apparently not happening for some container images. We can't be sure what is actually happening, so we preemptively add both directories to `safe.directory`. See also the GitHub issue opened in 2022 that still has not been resolved https://github.com/actions/runner/issues/2058.
2025-07-23 10:44:18 +00:00
Luc des Trois Maisons
60909655d3 Restructure beast::rngfill (#5563)
The current implementation of rngfill is prone to false warnings from GCC about array bounds violations. Looking at the code, the implementation naively manipulates both the bytes count and the buffer pointer directly to ensure the trailing memcpy doesn't overrun the buffer. As expressed, there is a data dependency on both fields between loop iterations.

Now, ideally, an optimizing compiler would realize that these dependencies were unnecessary and end up restructuring its intermediate representation into a functionally equivalent form with them absent. However, the point at which this occurs may be disjoint from when warning analyses are performed, potentially rendering them more difficult to
determine precisely.

In addition, it may also consume a portion of the budget the optimizer has allocated to attempting to improve a translation unit's performance. Given this is a function template which requires context-sensitive instantiation, this code would be more prone than most to being inlined, with a decrease in optimization budget corresponding to the effort the optimizer has already expended, having already optimized one or more calling functions. Thus, the scope for impacting the the ultimate quality of the code generated is elevated.

For this change, we rearrange things so that the location and contents of each memcpy can be computed independently, relying on a simple loop iteration counter as the only changing input between iterations.
2025-07-22 11:42:43 -04:00
Bronek Kozicki
03e46cd026 Remove include(default) from libxrpl profile (#5587)
Remove `include(default)` from `conan/profiles/libxrpl`. This means that we will now rely on compiler workarounds stored elsewhere e.g. in global.conf.
2025-07-21 14:03:53 +00:00
Vito Tumas
e95683a0fb refactor: Change boost::shared_mutex to std::shared_mutex (#5576)
This change reverts the usage of boost::shared_mutex back to std::shared_mutex. The change was originally introduced as a workaround for a bug in glibc 2.28 and older versions, which could cause threads using std::shared_mutex to stall. This issue primarily affected Ubuntu 18.04 and earlier distributions, which we no longer support.
2025-07-21 13:14:22 +00:00
Jingchen
13353ae36d Fix macos runner (#5585)
This change fixes the MacOS pipeline issue by limiting GitHub to choose the existing runners, ensuring the new experimental runners are excluded until they are ready.
2025-07-21 12:22:32 +00:00
Chenna Keshava B S
1a40f18bdd Remove the type filter from "ledger" RPC command (#4934)
This issue was reported on the Javascript client library: XRPLF/xrpl.js#2611

The type filter (Note: as of the latest version of rippled, type parameter is deprecated) does not work as expected. This PR removes the type filter from the ledger command.
2025-07-18 17:58:46 +00:00
Bart
90e6380383 refactor: Update date, libarchive, nudb, openssl, sqlite3, xxhash packages (#5567)
This PR updates several dependencies to their latest versions. Not all dependencies have been updated, as some need to be patched and some require additional code changes due to backward incompatibilities introduced by the version bump.
2025-07-18 16:55:15 +00:00
Vlad
8bfaa7fe0a test: Run unit tests regardless of 'Supported' amendment status (#5537) 2025-07-16 11:47:54 +00:00
Vlad
c9135a63cd Retire Flow Cross amendment (#5562)
The FlowCross amendment is now permanently enabled, so all code branches that have this amendment disabled are removed.
2025-07-16 06:53:13 -04:00
Michael Legleux
452263eaa5 chore: Update CI to use Conan 2 (#5556)
This is a minimally invasive update to use Conan 2 provided by our new build images.
2025-07-15 22:17:22 +00:00
yinyiqian1
8aa94ea09a fixAMMClawbackRounding: adjust last holder's LPToken balance (#5513)
Due to rounding, the LPTokenBalance of the last LP might not match the LP's trustline balance. This was fixed for `AMMWithdraw` in `fixAMMv1_1` by adjusting the LPTokenBalance to be the same as the trustline balance. Since `AMMClawback` is also performing a withdrawal, we need to adjust LPTokenBalance as well in `AMMClawback.`

This change includes:
1. Refactored `verifyAndAdjustLPTokenBalance` function in `AMMUtils`, which both`AMMWithdraw` and `AMMClawback` call to adjust LPTokenBalance.
2. Added the unit test `testLastHolderLPTokenBalance` to test the scenario.
3. Modify the existing unit tests for `fixAMMClawbackRounding`.
2025-07-11 20:03:28 +00:00
Bronek Kozicki
258ba71363 chore: Add gcc-12 workaround (#5554)
This change silences a dummy warning, which is breaking builds with GCC 12 (but not newer versions of GCC) in release mode only.
2025-07-11 18:57:09 +00:00
Shawn Xie
b8626ea3c6 Add MPT related txns into issuer's account history (#5530)
Currently there is no easy way to track MPT related transactions for the issuer. This change allows MPT transactions to show up on issuer's AccountTx RPC (to align with how IOUs work).
2025-07-11 17:50:03 +00:00
Vlad
6534757d85 chore: Remove unused headers (#5526) 2025-07-10 18:15:42 +00:00
Denis Angell
8e94ea3154 fix: add allowTrustLineLocking flag for account_info (#5525)
* Update the `account_info` API so that the `allowTrustLineLocking` flag is included in the response.
* The proposed `TokenEscrow` amendment added an `allowTrustLineLocking` flag in the `AccountRoot` object.
* In the API response, under `account_flags`, there is now an `allowTrustLineLocking` field with a boolean (`true` or `false`) value.
* For reference, the XLS-85 Token-Enabled Escrows implementation can be found in https://github.com/XRPLF/rippled/pull/5185
2025-07-10 16:29:51 +00:00
Bronek Kozicki
b113190563 Downgrade required CMake version for Antithesis SDK (#5548)
The current version was copied from `antithesis-sdk-cpp` but there is no logical reason to require this specific version of CMake. This change downgrades the version to make the project build with older CMake versions.
2025-07-10 11:46:02 -04:00
Ayaz Salikhov
358b7f50a7 fix: Link with boost libraries explicitly (#5546)
Having `boost::boost` in `self.requires` makes clio link with all boost libraries. There are additionally several Boost stacktrace backends that are both linked with, which violate ODR.
This change fixes the problem.
2025-07-10 06:14:27 -04:00
Bronek Kozicki
f47e2f4e82 chore: Fix compilation error with clang-20 and cleanup (#5543)
Removes clutter for old compilers, defaults to non-unity builds in cmake to match conanfile.py, and workaround for clang-20 compilation errors.
2025-07-09 17:47:34 +00:00
Bronek Kozicki
a7eea9546f test: Remove circular jtx.h dependencies (#5544)
Circular includes in header files can yield unpredictable results.
2025-07-09 08:43:11 -04:00
Jingchen
9874d47d7f Decouple CredentialHelpers from xrpld/app/tx (#5487)
This PR refactors `CredentialHelpers` and removes some unnecessary dependencies as a step of modularization.

The ledger component is almost independent except that it references `MPTokenAuthorize` and `CredentialHelpers.h`, and the latter further references `Transactor.h`. This PR partially clears the path to modularizing the ledger component and decouples `CredentialHelpers` from xrpld.
2025-07-03 14:27:37 +00:00
Mayukha Vadari
c2f3e2e263 fix: crash when trace-logging in tests (#5529)
This PR fixes a crash in tests when the test `Env is run at trace/debug log level.

This issue only affects tests, and only if logging at trace/debug level, so really only relevant during rippled development, and does not affect production servers.
2025-07-02 19:10:25 +00:00
Vlad
e18f27f5f7 test: switch some unit tests to doctest (#5383)
This change moves some tests from the current unit tests that are compiled into the rippled binary to using the doctest framework.
2025-06-26 19:35:31 +00:00
Jingchen
df6daf0d8f Add XRPL_ABANDON and use it to abandon OwnerPaysFee (#5510) 2025-06-26 12:09:05 -04:00
Jingchen
e9d46f0bfc Remove OwnerPaysFee as it's never fully supported (#5435)
The OwnerPaysFee amendment was never fully supported, and this change removes the feature to the extent possible.
2025-06-24 18:56:58 +00:00
Bart
42fd74b77b Removes release notes from codebase (#5508) 2025-06-24 13:10:00 -04:00
tequ
c55ea56c5e Add nftoken_id, nftoken_ids, offer_id to meta for transaction stream (#5230) 2025-06-24 09:02:22 -04:00
Michael Legleux
1e01cd34f7 Set version to 2.5.0 2025-06-23 10:13:01 -07:00
Alex Kremer
e2fa5c1b7c chore: Change libXRPL check conan remote to dev (#5482)
This change aligns the Conan remote used by the libXRPL Clio compatibility check workflow with the recent changes applied to Clio.
2025-06-20 17:02:16 +00:00
Ed Hennis
fc0984d286 Require a message on "Application::signalStop" (#5255)
This change adds a message parameter to Application::signalStop for extra context.
2025-06-20 16:24:34 +00:00
Valentin Balaschenko
8b3dcd41f7 refactor: Change getNodeFat Missing Node State Tree error into warning (#5455) 2025-06-20 15:44:42 +00:00
Denis Angell
8f2f5310e2 Fix: Improve error handling in Batch RPC response (#5503) 2025-06-18 17:46:45 -04:00
Michael Legleux
edb4f0342c Set version to 2.5.0-rc2 2025-06-11 17:10:45 -07:00
yinyiqian1
ea17abb92a fix: Ensure delegate tests do not silently fail with batch (#5476)
The tests that ensure `tfInnerBatchTxn` won't block delegated transactions silently fail in `Delegate_test.cpp`. This change removes these cases from that file and adds them to `Batch_test.cpp` instead where they do not silently fail, because there the batch delegate results are explicitly checked. Moving them to that file further avoids refactoring many helper functions.
2025-06-11 13:21:24 +08:00
Mayukha Vadari
35a40a8e62 fix: Improve multi-sign usage of simulate (#5479)
This change allows users to submit simulate requests from a multi-sign account without needing to specify the accounts that are doing the multi-signing, and fixes an error with simulate that allowed double-"signed" (both single-sign and multi-sign public keys are provided) transactions.
2025-06-10 14:47:27 +08:00
Ed Hennis
d494bf45b2 refactor: Collapse some split log messages into one (#5347)
Multi-line log messages are hard to work with. Writing these handful of related messages as one message should make the log a tiny bit easier to manage.
2025-06-06 16:01:02 +00:00
Vlad
8bf4a5cbff chore: Remove external project build cores division (#5475)
The CMake statements that make it seem as if the number of cores used to build external project dependencies is halved don't actually do anything. This change removes these statements.
2025-06-05 13:37:30 +00:00
Denis Angell
58c2c82a30 fix: Amendment-guard TokenEscrow preclaim and expand tests (#5473)
This change amendment-guards the preclaim for `TokenEscrow`, as well as expands tests to increase code coverage.
2025-06-05 12:54:45 +00:00
Michael Legleux
11edaa441d Set version to 2.5.0-rc1 (#5472) 2025-06-04 17:55:23 +00:00
yinyiqian1
a5e953b191 fix: Add tecNO_DELEGATE_PERMISSION and fix flags (#5465)
* Adds `tecNO_DELEGATE_PERMISSION` for unauthorized transactions sent by a delegated account.
* Returns `tecNO_TARGET` instead of `terNO_ACCOUNT` for the `DelegateSet` transaction if the delegated account does not exist.
* Fixes `tfFullyCanonicalSig` and `tfInnerBatchTxn` blocking transactions issue by adding `tfUniversal` in the permission related masks in `txFlags.h`
2025-06-03 22:20:29 +00:00
Mark Travis
506ae12a8c Increase network i/o capacity (#5464)
The change increases the default network I/O worker thread pool size from 2 to 6. This will improve stability, as worker thread saturation correlates to desyncs, particularly on high-traffic peers, such as hubs.
2025-06-03 21:33:09 +00:00
Ayaz Salikhov
0310c5cbe0 fix: Specify transitive_headers when building with Conan 2 (#5462)
To be able to consume `rippled` in Conan 2, the recipe should specify transitive_headers for external libraries that are present in the exported header files. This change remains compatibility with Conan 1, where this flag was not present.
2025-06-03 17:33:32 +00:00
Denis Angell
053e1af7ff Add support for XLS-85 Token Escrow (#5185)
- Specification: https://github.com/XRPLF/XRPL-Standards/pull/272
- Amendment: `TokenEscrow`
- Enables escrowing of IOU and MPT tokens in addition to native XRP.
- Allows accounts to lock issued tokens (IOU/MPT) in escrow objects, with support for freeze, authorization, and transfer rates.
- Adds new ledger fields (`sfLockedAmount`, `sfIssuerNode`, etc.) to track locked balances for IOU and MPT escrows.
- Updates EscrowCreate, EscrowFinish, and EscrowCancel transaction logic to support IOU and MPT assets, including proper handling of trustlines and MPT authorization, transfer rates, and locked balances.
- Enforces invariant checks for escrowed IOU/MPT amounts.
- Extends GatewayBalances RPC to report locked (escrowed) balances.
2025-06-03 12:51:55 -04:00
Vlad
7e24adbdd0 fix: Address NFT interactions with trustlines (#5297)
The changes are focused on fixing NFT transactions bypassing the trustline authorization requirement and potential invariant violation when interacting with deep frozen trustlines.
2025-06-02 16:13:20 +00:00
Gregory Tsipenyuk
621df422a7 fix: Add AMMv1_3 amendment (#5203)
* Add AMM bid/create/deposit/swap/withdraw/vote invariants:
  - Deposit, Withdrawal invariants: `sqrt(asset1Balance * asset2Balance) >= LPTokens`.
  - Bid: `sqrt(asset1Balance * asset2Balance) > LPTokens` and the pool balances don't change.
  - Create: `sqrt(asset1Balance * assetBalance2) == LPTokens`.
  - Swap: `asset1BalanceAfter * asset2BalanceAfter >= asset1BalanceBefore * asset2BalanceBefore`
     and `LPTokens` don't change.
  - Vote: `LPTokens` and pool balances don't change.
  - All AMM and swap transactions: amounts and tokens are greater than zero, except on withdrawal if all tokens
    are withdrawn.
* Add AMM deposit and withdraw rounding to ensure AMM invariant:
  - On deposit, tokens out are rounded downward and deposit amount is rounded upward.
  - On withdrawal, tokens in are rounded upward and withdrawal amount is rounded downward.
* Add Order Book Offer invariant to verify consumed amounts. Consumed amounts are less than the offer.
* Fix Bid validation. `AuthAccount` can't have duplicate accounts or the submitter account.
2025-06-02 09:52:10 -04:00
Shawn Xie
0a34b5c691 Add support for XLS-81 Permissioned DEX (#5404)
Modified transactions:
- OfferCreate
- Payment

Modified RPCs:
- book_changes
- subscribe
- book_offers
- ripple_path_find
- path_find

Spec: https://github.com/XRPLF/XRPL-Standards/pull/281
2025-05-30 13:24:48 -04:00
Matt Mankins
e0bc3dd51f docs: update example keyserver host in SECURITY.md (#5460) 2025-05-30 08:46:08 -04:00
Bronek Kozicki
dacecd24ba Fix unit build error (#5459)
This change fixes the issue that there is a `using namespace` statement inside a namespace scope.
2025-05-29 20:53:31 +00:00
Mayukha Vadari
05105743e9 chore[tests]: improve env.meta usage (#5457)
This commit changes the ledger close in env.meta to be conditional on if it hasn't already been closed (i.e. the current ledger doesn't have any transactions in it). This change will make it a bit easier to use, as it will still work if you close the ledger outside of this usage. Previously, if you accidentally closed the ledger outside of the meta function, it would segfault and it was incredibly difficult to debug.
2025-05-29 16:28:09 +00:00
Bronek Kozicki
9e1fe9a85e Fix: Improve handling of expired credentials in VaultDeposit (#5452)
This change returns `tecEXPIRED` from VaultDeposit to allow the Transactor to remove the expired credentials.
2025-05-28 10:28:18 -04:00
Vito Tumas
d71ce51901 feat: improve squelching configuration (#5438)
This commit introduces the following changes:
* Renames `vp_enable config` option to `vp_base_squelch_enable` to enable squelching for validators.
* Removes `vp_squelch` config option which was used to configure whether to send squelch messages to peers or not. With this flag removed, if squelching is enabled, squelch messages will be sent. This was an option used for debugging.
* Introduces a temporary `vp_base_squelch_max_trusted_peers` config option to change the max number of peers who are selected as validator message sources. This is a temporary option, which will be removed once a good value is found.
* Adds a traffic counter to count the number of times peers ignored squelch messages and kept sending messages for squelched validators.
* Moves the decision whether squelching is enabled and ready into Slot.h.
2025-05-28 06:30:03 -04:00
Michael Legleux
be668ee26d chore: Update CPP ref source (#5453) 2025-05-27 20:46:25 +00:00
Bart
cae5294b4e chore: Rename docs job (#5398) 2025-05-27 20:03:23 +00:00
Elliot.
cd777f79ef docs: add -j $(nproc) to BUILD.md (#5288)
This improves build times.
2025-05-27 19:11:13 +00:00
Valentin Balaschenko
8b9e21e3f5 docs: Update build instructions for Ubuntu 22.04+ (#5292) 2025-05-27 18:32:25 +00:00
Denis Angell
2a61aee562 Add Batch feature (XLS-56) (#5060)
- Specification: [XRPLF/XRPL-Standards 56](https://github.com/XRPLF/XRPL-Standards/blob/master/XLS-0056d-batch/README.md)
- Amendment: `Batch`
- Implements execution of multiple transactions within a single batch transaction with four execution modes: `tfAllOrNothing`, `tfOnlyOne`, `tfUntilFailure`, and `tfIndependent`.
- Enables atomic multi-party transactions where multiple accounts can participate in a single batch, with up to 8 inner transactions and 8 batch signers per batch transaction.
- Inner transactions use `tfInnerBatchTxn` flag with zero fees, no signature, and empty signing public key.
- Inner transactions are applied after the outer batch succeeds via the `applyBatchTransactions` function in apply.cpp.
- Network layer prevents relay of transactions with `tfInnerBatchTxn` flag - each peer applies inner transactions locally from the batch.
- Batch transactions are excluded from AccountDelegate permissions but inner transactions retain full delegation support.
- Metadata includes `ParentBatchID` linking inner transactions to their containing batch for traceability and auditing.
- Extended STTx with batch-specific signature verification methods and added protocol structures (`sfRawTransactions`, `sfBatchSigners`).
2025-05-23 19:53:53 +00:00
Bronek Kozicki
40ce8a8833 fix: Fix pseudo-account ID calculation (#5447)
Before #5224, the pseudoaccount ID was calculated using prefix expressed in `std::uint16_t`. The refactoring to move the pseudoaccount ID calculation to View.cpp had accidentally changed the prefix type to `int` (derived from `auto i = 0`) which in turn changed the length of the input to `sha512Half` from 2 bytes to 4, altering the result.

This resulted in a different ID of the pseudoaccount calculated from the function after the refactoring, breaking the ledger. This impacts AMMCreate, even when the `SingleAssetVault` amendment is not active. This change restores the prefix type to `std::uint16_t`.
2025-05-23 14:05:36 +00:00
Bronek Kozicki
7713ff8c5c Add codecov badge, raise .codecov.yml thresholds (#5428) 2025-05-22 14:43:41 +00:00
Olek
70371a4344 Fix initializer list initialization for GCC-15 (#5443) 2025-05-21 13:28:18 -04:00
Bronek Kozicki
e514de76ed Add single asset vault (XLS-65d) (#5224)
- Specification: XRPLF/XRPL-Standards#239
- Amendment: `SingleAssetVault`
- Implements a vault feature used to store a fungible asset (XRP, IOU, or MPT, but not NFT) and to receive shares in the vault (an MPT) in exchange.
- A vault can be private or public.
- A private vault can use permissioned domains, subject to the `PermissionedDomains` amendment.
- Shares can be exchanged back into asset with `VaultWithdraw`.
- Permissions on the asset in the vault are transitively applied on shares in the vault.
- Issuer of the asset in the vault can clawback with `VaultClawback`.
- Extended `MPTokenIssuance` with `DomainID`, used by the permissioned domain on the vault shares.

Co-authored-by: John Freeman <jfreeman08@gmail.com>
2025-05-20 14:06:41 -04:00
Bart
dd62cfcc22 fix: Update path in CODEOWNERS (#5440) 2025-05-20 15:24:07 +00:00
Michael Legleux
09690f1b38 Set version to 2.5.0-b1 2025-05-18 20:39:18 +01:00
Valentin Balaschenko
380ba9f1c1 Fix: Resolve slow test on macOS pipeline (#5392)
Using std::barrier performs extremely poorly (~1 hour vs ~1 minute to run the test suite) in certain macOS environments.
To unblock our macOS CI pipeline, std::barrier has been replaced with a custom mutex-based barrier (Barrier) that significantly improves performance without compromising correctness.
2025-05-16 10:31:51 +00:00
brettmollin
c3e9380fb4 fix: Update validators-example.txt fix xrplf example URL (#5384) 2025-05-16 09:49:14 +00:00
Jingchen
e3ebc253fa fix: Ensure that coverage file generation is atomic. (#5426)
Running unit tests in parallel and multiple threads can write into one file can corrupt output files, and then gcovr won't be able to parse the corrupted file. This change adds -fprofile-update=atomic as instructed by https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68080.
2025-05-12 14:54:01 +00:00
Bart
c6c7c84355 Configure CODEOWNERS for changes to RPC code (#5266)
To ensure changes to any RPC-related code are compatible with other services, such as Clio, the RPC team will be required to review them.
2025-05-12 12:42:03 +00:00
yinyiqian1
28f50cb7cf fix: enable LedgerStateFix for delegation (#5427) 2025-05-10 10:36:11 -04:00
Vito Tumas
3e152fec74 refactor: use east const convention (#5409)
This change refactors the codebase to use the "east const convention", and adds a clang-format rule to follow this convention.
2025-05-08 11:00:42 +00:00
yinyiqian1
2db2791805 Add PermissionDelegation feature (#5354)
This change implements the account permission delegation described in XLS-75d, see https://github.com/XRPLF/XRPL-Standards/pull/257.

* Introduces transaction-level and granular permissions that can be delegated to other accounts.
* Adds `DelegateSet` transaction to grant specified permissions to another account.
* Adds `ltDelegate` ledger object to maintain the permission list for delegating/delegated account pair.
* Adds an optional `Delegate` field in common fields, allowing a delegated account to send transactions on behalf of the delegating account within the granted permission scope. The `Account` field remains the delegating account; the `Delegate` field specifies the delegated account. The transaction is signed by the delegated account.
2025-05-08 06:14:02 -04:00
Vito Tumas
9ec2d7f8ff Enable passive squelching (#5358)
This change updates the squelching logic to accept squelch messages for untrusted validators. As a result, servers will also squelch untrusted validator messages reducing duplicate traffic they generate.

In particular:
* Updates squelch message handling logic to squelch messages for all validators, not only trusted ones.
* Updates the logic to send squelch messages to peers that don't squelch themselves
* Increases the threshold for the number of messages that a peer has to deliver to consider it as a candidate for validator messages.
2025-05-02 11:01:45 -04:00
Ed Hennis
4a084ce34c Improve transaction relay logic (#4985)
Combines four related changes:
1. "Decrease `shouldRelay` limit to 30s." Pretty self-explanatory. Currently, the limit is 5 minutes, by which point the `HashRouter` entry could have expired, making this transaction look brand new (and thus causing it to be relayed back to peers which have sent it to us recently).
2.  "Give a transaction more chances to be retried." Will put a transaction into `LedgerMaster`'s held transactions if the transaction gets a `ter`, `tel`, or `tef` result. Old behavior was just `ter`.
     * Additionally, to prevent a transaction from being repeatedly held indefinitely, it must meet some extra conditions. (Documented in a comment in the code.)
3. "Pop all transactions with sequential sequences, or tickets." When a transaction is processed successfully, currently, one held transaction for the same account (if any) will be popped out of the held transactions list, and queued up for the next transaction batch. This change pops all transactions for the account, but only if they have sequential sequences (for non-ticket transactions) or use a ticket. This issue was identified from interactions with @mtrippled's #4504, which was merged, but unfortunately reverted later by #4852. When the batches were spaced out, it could potentially take a very long time for a large number of held transactions for an account to get processed through. However, whether batched or not, this change will help get held transactions cleared out, particularly if a missing earlier transaction is what held them up.
4. "Process held transactions through existing NetworkOPs batching." In the current processing, at the end of each consensus round, all held transactions are directly applied to the open ledger, then the held list is reset. This bypasses all of the logic in `NetworkOPs::apply` which, among other things, broadcasts successful transactions to peers. This means that the transaction may not get broadcast to peers for a really long time (5 minutes in the current implementation, or 30 seconds with this first commit). If the node is a bottleneck (either due to network configuration, or because the transaction was submitted locally), the transaction may not be seen by any other nodes or validators before it expires or causes other problems.
2025-05-01 13:58:18 -04:00
Vito Tumas
3502df2174 fix: Replaces random endpoint resolution with sequential (#5365)
This change addresses an issue where `rippled` attempts to connect to an IPv6 address, even when the local network lacks IPv6 support, resulting in a "Network is unreachable" error.

The fix replaces the custom endpoint selection logic with `boost::async_connect`, which sequentially attempts to connect to available endpoints until one succeeds or all fail.
2025-04-28 15:38:55 -04:00
Vlad
fa1e25abef chore: Small clarification to lsfDefaultRipple comment (#5410) 2025-04-25 15:21:27 +00:00
Denis Angell
217ba8dd4d fix: CTID to use correct ledger_index (#5408) 2025-04-24 10:24:10 -04:00
Ed Hennis
405f4613d8 chore: Run CI on PRs that are Ready or have the "DraftRunCI" label (#5400)
- Avoids costly overhead for idle PRs where the CI results don't add any
  value.
2025-04-11 22:20:59 +00:00
Mayukha Vadari
cba512068b refactor: Clean up test logging to make it easier to search (#5396)
This PR replaces the word `failed` with `failure` in any test names and renames some test files to fix MSVC warnings, so that it is easier to search through the test output to find tests that failed.
2025-04-11 09:07:42 +00:00
Valentin Balaschenko
1c99ea23d1 Temporary disable automatic triggering macOS pipeline (#5397)
We temporarily disable running unit tests on macOS on the CI pipeline while we are investigating the delays.
2025-04-10 21:58:29 +02:00
Denis Angell
c4308b216f fix: Adds CTID to RPC tx and updates error (#4738)
This change fixes a number of issues involved with CTID:
* CTID is not present on all RPC tx transactions.
* rpcWRONG_NETWORK is missing in the ErrorCodes.cpp
2025-04-10 12:38:52 +00:00
Wietse Wind
aafd2d8525 Fix: admin RPC webhook queue limit removal and timeout reduction (#5163)
When using subscribe at admin RPC port to send webhooks for the transaction stream to a backend, on large(r) ledgers the endpoint receives fewer HTTP POSTs with TX information than the amount of transactions in a ledger. This change removes the hardcoded queue length to avoid dropping TX notifications for the admin-only command. In addition, the per-request TTL for outgoing RPC HTTP calls has been reduced from 10 minutes to 30 seconds.
2025-04-10 06:37:24 +00:00
Denis Angell
a574ec6023 fix: fixPayChanV1 (#4717)
This change introduces a new fix amendment (`fixPayChanV1`) that prevents the creation of new `PaymentChannelCreate` transaction with a `CancelAfter` time less than the current ledger time. It piggy backs off of fix1571.

Once the amendment is activated, creating a new `PaymentChannel` will require that if you specify the `CancelAfter` time/value, that value must be greater than or equal to the current ledger time.

Currently users can create a payment channel where the `CancelAfter` time is before the current ledger time. This results in the payment channel being immediately closed on the next PaymentChannel transaction.
2025-04-09 22:08:44 +00:00
Mayukha Vadari
e429455f4d refactor(trivial): reorganize ledger entry tests and helper functions (#5376)
This PR splits out `ledger_entry` tests into its own file (`LedgerEntry_test.cpp`) and alphabetizes the helper functions in `LedgerEntry.cpp`. These commits were split out of #5237 to make that PR a little more manageable, since these basic trivial changes are most of the diff. There is no code change, just moving code around.
2025-04-09 17:02:03 +00:00
Vito Tumas
7692eeb9a0 Instrument proposal, validation and transaction messages (#5348)
Adds metric counters for the following P2P message types:

* Untrusted proposal and validation messages
* Duplicate proposal, validation and transaction messages
2025-04-09 15:33:17 +02:00
Bronek Kozicki
a099f5a804 Remove UNREACHABLE from NetworkOPsImp::processTrustedProposal (#5387)
It’s possible for this to happen legitimately if a set of peers, including a validator, are connected in a cycle, and the latency and message processing time between those peers is significantly less than the latency between the validator and the last peer. It’s unlikely in the real world, but obviously easy to simulate with Antithesis.
2025-04-08 14:43:34 +00:00
Michael Legleux
ca0bc767fe fix: Use the build image from ghcr.io (#5390)
The ci pipelines are constantly hitting Docker Hub's public rate limiting since increasing the number of jobs we're running. This change switches over to images hosted in GitHub's registry.
2025-04-05 02:24:31 +00:00
Mayukha Vadari
4ba9288935 fix: disable channel_authorize when signing_support is disabled (#5385) 2025-04-05 01:08:34 +00:00
Valentin Balaschenko
e923ec6d36 Fix to correct memory ordering for compare_exchange_weak and wait in the intrusive reference counting logic (#5381)
This change addresses a memory ordering assertion failure observed on one of the Windows test machines during the IntrusiveShared_test suite.
2025-04-04 18:21:17 +00:00
Vlad
851d99d99e fix: uint128 ambiguousness breaking macos unity build (#5386) 2025-04-04 08:28:33 -04:00
942 changed files with 63780 additions and 29170 deletions

View File

@@ -1,5 +1,21 @@
---
Language: Cpp
BreakBeforeBraces: Custom
BraceWrapping:
AfterClass: true
AfterControlStatement: true
AfterEnum: false
AfterFunction: true
AfterNamespace: false
AfterObjCDeclaration: true
AfterStruct: true
AfterUnion: true
BeforeCatch: true
BeforeElse: true
IndentBraces: false
KeepEmptyLinesAtTheStartOfBlocks: false
MaxEmptyLinesToKeep: 1
---
Language: Cpp
AccessModifierOffset: -4
AlignAfterOpenBracket: AlwaysBreak
AlignConsecutiveAssignments: false
@@ -18,56 +34,41 @@ AlwaysBreakBeforeMultilineStrings: true
AlwaysBreakTemplateDeclarations: true
BinPackArguments: false
BinPackParameters: false
BraceWrapping:
AfterClass: true
AfterControlStatement: true
AfterEnum: false
AfterFunction: true
AfterNamespace: false
AfterObjCDeclaration: true
AfterStruct: true
AfterUnion: true
BeforeCatch: true
BeforeElse: true
IndentBraces: false
BreakBeforeBinaryOperators: false
BreakBeforeBraces: Custom
BreakBeforeTernaryOperators: true
BreakConstructorInitializersBeforeComma: true
ColumnLimit: 80
CommentPragmas: '^ IWYU pragma:'
ColumnLimit: 80
CommentPragmas: "^ IWYU pragma:"
ConstructorInitializerAllOnOneLineOrOnePerLine: true
ConstructorInitializerIndentWidth: 4
ContinuationIndentWidth: 4
Cpp11BracedListStyle: true
DerivePointerAlignment: false
DisableFormat: false
DisableFormat: false
ExperimentalAutoDetectBinPacking: false
ForEachMacros: [ Q_FOREACH, BOOST_FOREACH ]
IncludeBlocks: Regroup
ForEachMacros: [Q_FOREACH, BOOST_FOREACH]
IncludeBlocks: Regroup
IncludeCategories:
- Regex: '^<(test)/'
Priority: 0
- Regex: '^<(xrpld)/'
Priority: 1
- Regex: '^<(xrpl)/'
Priority: 2
- Regex: '^<(boost)/'
Priority: 3
- Regex: '^.*/'
Priority: 4
- Regex: '^.*\.h'
Priority: 5
- Regex: '.*'
Priority: 6
IncludeIsMainRegex: '$'
- Regex: "^<(test)/"
Priority: 0
- Regex: "^<(xrpld)/"
Priority: 1
- Regex: "^<(xrpl)/"
Priority: 2
- Regex: "^<(boost)/"
Priority: 3
- Regex: "^.*/"
Priority: 4
- Regex: '^.*\.h'
Priority: 5
- Regex: ".*"
Priority: 6
IncludeIsMainRegex: "$"
IndentCaseLabels: true
IndentFunctionDeclarationAfterType: false
IndentRequiresClause: true
IndentWidth: 4
IndentWidth: 4
IndentWrappedFunctionNames: false
KeepEmptyLinesAtTheStartOfBlocks: false
MaxEmptyLinesToKeep: 1
NamespaceIndentation: None
ObjCSpaceAfterProperty: false
ObjCSpaceBeforeProtocolList: false
@@ -78,19 +79,25 @@ PenaltyBreakString: 1000
PenaltyExcessCharacter: 1000000
PenaltyReturnTypeOnItsOwnLine: 200
PointerAlignment: Left
ReflowComments: true
ReflowComments: true
RequiresClausePosition: OwnLine
SortIncludes: true
SortIncludes: true
SpaceAfterCStyleCast: false
SpaceBeforeAssignmentOperators: true
SpaceBeforeParens: ControlStatements
SpaceInEmptyParentheses: false
SpacesBeforeTrailingComments: 2
SpacesInAngles: false
SpacesInAngles: false
SpacesInContainerLiterals: true
SpacesInCStyleCastParentheses: false
SpacesInParentheses: false
SpacesInSquareBrackets: false
Standard: Cpp11
TabWidth: 8
UseTab: Never
Standard: Cpp11
TabWidth: 8
UseTab: Never
QualifierAlignment: Right
---
Language: Proto
BasedOnStyle: Google
ColumnLimit: 0
IndentWidth: 2

View File

@@ -7,13 +7,13 @@ comment:
show_carryforward_flags: false
coverage:
range: "60..80"
range: "70..85"
precision: 1
round: nearest
status:
project:
default:
target: 60%
target: 75%
threshold: 2%
patch:
default:
@@ -27,11 +27,12 @@ github_checks:
parsers:
cobertura:
partials_as_hits: true
handle_missing_conditions : true
handle_missing_conditions: true
slack_app: false
ignore:
- "src/test/"
- "src/tests/"
- "include/xrpl/beast/test/"
- "include/xrpl/beast/unit_test/"

View File

@@ -11,3 +11,6 @@ b9d007813378ad0ff45660dc07285b823c7e9855
fe9a5365b8a52d4acc42eb27369247e6f238a4f9
9a93577314e6a8d4b4a8368cc9d2b15a5d8303e8
552377c76f55b403a1c876df873a23d780fcc81c
97f0747e103f13e26e45b731731059b32f7679ac
b13370ac0d207217354f1fc1c29aef87769fb8a1
896b8c3b54a22b0497cb0d1ce95e1095f9a227ce

8
.github/CODEOWNERS vendored Normal file
View File

@@ -0,0 +1,8 @@
# Allow anyone to review any change by default.
*
# Require the rpc-reviewers team to review changes to the rpc code.
include/xrpl/protocol/ @xrplf/rpc-reviewers
src/libxrpl/protocol/ @xrplf/rpc-reviewers
src/xrpld/rpc/ @xrplf/rpc-reviewers
src/xrpld/app/misc/ @xrplf/rpc-reviewers

View File

@@ -2,30 +2,35 @@
name: Bug Report
about: Create a report to help us improve rippled
title: "[Title with short description] (Version: [rippled version])"
labels: ''
assignees: ''
labels: ""
assignees: ""
---
<!-- Please search existing issues to avoid creating duplicates.-->
## Issue Description
<!--Provide a summary for your issue/bug.-->
## Steps to Reproduce
<!--List in detail the exact steps to reproduce the unexpected behavior of the software.-->
## Expected Result
<!--Explain in detail what behavior you expected to happen.-->
## Actual Result
<!--Explain in detail what behavior actually happened.-->
## Environment
<!--Please describe your environment setup (such as Ubuntu 18.04 with Boost 1.70).-->
<!-- If you are using a formal release, please use the version returned by './rippled --version' as the version number-->
<!-- If you are working off of develop, please add the git hash via 'git rev-parse HEAD'-->
## Supporting Files
<!--If you have supporting files such as a log, feel free to post a link here using Github Gist.-->
<!--Consider adding configuration files with private information removed via Github Gist. -->

View File

@@ -3,19 +3,23 @@ name: Feature Request
about: Suggest a new feature for the rippled project
title: "[Title with short description] (Version: [rippled version])"
labels: Feature Request
assignees: ''
assignees: ""
---
<!-- Please search existing issues to avoid creating duplicates.-->
## Summary
<!-- Provide a summary to the feature request-->
## Motivation
<!-- Why do we need this feature?-->
## Solution
<!-- What is the solution?-->
## Paths Not Taken
<!-- What other alternatives have been considered?-->

37
.github/actions/build-deps/action.yml vendored Normal file
View File

@@ -0,0 +1,37 @@
name: Build Conan dependencies
description: "Install Conan dependencies, optionally forcing a rebuild of all dependencies."
# Note that actions do not support 'type' and all inputs are strings, see
# https://docs.github.com/en/actions/reference/workflows-and-actions/metadata-syntax#inputs.
inputs:
build_dir:
description: "The directory where to build."
required: true
build_type:
description: 'The build type to use ("Debug", "Release").'
required: true
force_build:
description: 'Force building of all dependencies ("true", "false").'
required: false
default: "false"
runs:
using: composite
steps:
- name: Install Conan dependencies
shell: bash
env:
BUILD_DIR: ${{ inputs.build_dir }}
BUILD_OPTION: ${{ inputs.force_build == 'true' && '*' || 'missing' }}
BUILD_TYPE: ${{ inputs.build_type }}
run: |
echo 'Installing dependencies.'
mkdir -p '${{ env.BUILD_DIR }}'
cd '${{ env.BUILD_DIR }}'
conan install \
--output-folder . \
--build=${{ env.BUILD_OPTION }} \
--options:host='&:tests=True' \
--options:host='&:xrpld=True' \
--settings:all build_type='${{ env.BUILD_TYPE }}' \
..

View File

@@ -1,34 +0,0 @@
name: build
inputs:
generator:
default: null
configuration:
required: true
cmake-args:
default: null
cmake-target:
default: all
# An implicit input is the environment variable `build_dir`.
runs:
using: composite
steps:
- name: configure
shell: bash
run: |
cd ${build_dir}
cmake \
${{ inputs.generator && format('-G "{0}"', inputs.generator) || '' }} \
-DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake \
-DCMAKE_BUILD_TYPE=${{ inputs.configuration }} \
-Dtests=TRUE \
-Dxrpld=TRUE \
${{ inputs.cmake-args }} \
..
- name: build
shell: bash
run: |
cmake \
--build ${build_dir} \
--config ${{ inputs.configuration }} \
--parallel ${NUM_PROCESSORS:-$(nproc)} \
--target ${{ inputs.cmake-target }}

View File

@@ -1,57 +0,0 @@
name: dependencies
inputs:
configuration:
required: true
# An implicit input is the environment variable `build_dir`.
runs:
using: composite
steps:
- name: unlock Conan
shell: bash
run: conan remove --locks
- name: export custom recipes
shell: bash
run: |
conan config set general.revisions_enabled=1
conan export external/snappy snappy/1.1.10@
conan export external/rocksdb rocksdb/9.7.3@
conan export external/soci soci/4.0.3@
conan export external/nudb nudb/2.0.8@
- name: add Ripple Conan remote
shell: bash
run: |
conan remote list
conan remote remove ripple || true
# Do not quote the URL. An empty string will be accepted (with
# a non-fatal warning), but a missing argument will not.
conan remote add ripple ${{ env.CONAN_URL }} --insert 0
- name: try to authenticate to Ripple Conan remote
id: remote
shell: bash
run: |
# `conan user` implicitly uses the environment variables
# CONAN_LOGIN_USERNAME_<REMOTE> and CONAN_PASSWORD_<REMOTE>.
# https://docs.conan.io/1/reference/commands/misc/user.html#using-environment-variables
# https://docs.conan.io/1/reference/env_vars.html#conan-login-username-conan-login-username-remote-name
# https://docs.conan.io/1/reference/env_vars.html#conan-password-conan-password-remote-name
echo outcome=$(conan user --remote ripple --password >&2 \
&& echo success || echo failure) | tee ${GITHUB_OUTPUT}
- name: list missing binaries
id: binaries
shell: bash
# Print the list of dependencies that would need to be built locally.
# A non-empty list means we have "failed" to cache binaries remotely.
run: |
echo missing=$(conan info . --build missing --settings build_type=${{ inputs.configuration }} --json 2>/dev/null | grep '^\[') | tee ${GITHUB_OUTPUT}
- name: install dependencies
shell: bash
run: |
mkdir ${build_dir}
cd ${build_dir}
conan install \
--output-folder . \
--build missing \
--options tests=True \
--options xrpld=True \
--settings build_type=${{ inputs.configuration }} \
..

43
.github/actions/print-env/action.yml vendored Normal file
View File

@@ -0,0 +1,43 @@
name: Print build environment
description: "Print environment and some tooling versions"
runs:
using: composite
steps:
- name: Check configuration (Windows)
if: ${{ runner.os == 'Windows' }}
shell: bash
run: |
echo 'Checking environment variables.'
set
echo 'Checking CMake version.'
cmake --version
echo 'Checking Conan version.'
conan --version
- name: Check configuration (Linux and macOS)
if: ${{ runner.os == 'Linux' || runner.os == 'macOS' }}
shell: bash
run: |
echo 'Checking path.'
echo ${PATH} | tr ':' '\n'
echo 'Checking environment variables.'
env | sort
echo 'Checking CMake version.'
cmake --version
echo 'Checking compiler version.'
${{ runner.os == 'Linux' && '${CC}' || 'clang' }} --version
echo 'Checking Conan version.'
conan --version
echo 'Checking Ninja version.'
ninja --version
echo 'Checking nproc version.'
nproc --version

46
.github/actions/setup-conan/action.yml vendored Normal file
View File

@@ -0,0 +1,46 @@
name: Setup Conan
description: "Set up Conan configuration, profile, and remote."
inputs:
conan_remote_name:
description: "The name of the Conan remote to use."
required: false
default: xrplf
conan_remote_url:
description: "The URL of the Conan endpoint to use."
required: false
default: https://conan.ripplex.io
runs:
using: composite
steps:
- name: Set up Conan configuration
shell: bash
run: |
echo 'Installing configuration.'
cat conan/global.conf ${{ runner.os == 'Linux' && '>>' || '>' }} $(conan config home)/global.conf
echo 'Conan configuration:'
conan config show '*'
- name: Set up Conan profile
shell: bash
run: |
echo 'Installing profile.'
conan config install conan/profiles/default -tf $(conan config home)/profiles/
echo 'Conan profile:'
conan profile show
- name: Set up Conan remote
shell: bash
env:
CONAN_REMOTE_NAME: ${{ inputs.conan_remote_name }}
CONAN_REMOTE_URL: ${{ inputs.conan_remote_url }}
run: |
echo "Adding Conan remote '${{ env.CONAN_REMOTE_NAME }}' at '${{ env.CONAN_REMOTE_URL }}'."
conan remote add --index 0 --force '${{ env.CONAN_REMOTE_NAME }}' '${{ env.CONAN_REMOTE_URL }}'
echo 'Listing Conan remotes.'
conan remote list

View File

@@ -25,32 +25,32 @@ more dependencies listed later.
**tl;dr:** The modules listed first are more independent than the modules
listed later.
| Level / Tier | Module(s) |
|--------------|-----------------------------------------------|
| 01 | ripple/beast ripple/unity
| 02 | ripple/basics
| 03 | ripple/json ripple/crypto
| 04 | ripple/protocol
| 05 | ripple/core ripple/conditions ripple/consensus ripple/resource ripple/server
| 06 | ripple/peerfinder ripple/ledger ripple/nodestore ripple/net
| 07 | ripple/shamap ripple/overlay
| 08 | ripple/app
| 09 | ripple/rpc
| 10 | ripple/perflog
| 11 | test/jtx test/beast test/csf
| 12 | test/unit_test
| 13 | test/crypto test/conditions test/json test/resource test/shamap test/peerfinder test/basics test/overlay
| 14 | test
| 15 | test/net test/protocol test/ledger test/consensus test/core test/server test/nodestore
| 16 | test/rpc test/app
| Level / Tier | Module(s) |
| ------------ | -------------------------------------------------------------------------------------------------------- |
| 01 | ripple/beast ripple/unity |
| 02 | ripple/basics |
| 03 | ripple/json ripple/crypto |
| 04 | ripple/protocol |
| 05 | ripple/core ripple/conditions ripple/consensus ripple/resource ripple/server |
| 06 | ripple/peerfinder ripple/ledger ripple/nodestore ripple/net |
| 07 | ripple/shamap ripple/overlay |
| 08 | ripple/app |
| 09 | ripple/rpc |
| 10 | ripple/perflog |
| 11 | test/jtx test/beast test/csf |
| 12 | test/unit_test |
| 13 | test/crypto test/conditions test/json test/resource test/shamap test/peerfinder test/basics test/overlay |
| 14 | test |
| 15 | test/net test/protocol test/ledger test/consensus test/core test/server test/nodestore |
| 16 | test/rpc test/app |
(Note that `test` levelization is *much* less important and *much* less
(Note that `test` levelization is _much_ less important and _much_ less
strictly enforced than `ripple` levelization, other than the requirement
that `test` code should *never* be included in `ripple` code.)
that `test` code should _never_ be included in `ripple` code.)
## Validation
The [levelization.sh](levelization.sh) script takes no parameters,
The [levelization](generate.sh) script takes no parameters,
reads no environment variables, and can be run from any directory,
as long as it is in the expected location in the rippled repo.
It can be run at any time from within a checked out repo, and will
@@ -59,48 +59,48 @@ the rippled source. The only caveat is that it runs much slower
under Windows than in Linux. It hasn't yet been tested under MacOS.
It generates many files of [results](results):
* `rawincludes.txt`: The raw dump of the `#includes`
* `paths.txt`: A second dump grouping the source module
- `rawincludes.txt`: The raw dump of the `#includes`
- `paths.txt`: A second dump grouping the source module
to the destination module, deduped, and with frequency counts.
* `includes/`: A directory where each file represents a module and
- `includes/`: A directory where each file represents a module and
contains a list of modules and counts that the module _includes_.
* `includedby/`: Similar to `includes/`, but the other way around. Each
- `includedby/`: Similar to `includes/`, but the other way around. Each
file represents a module and contains a list of modules and counts
that _include_ the module.
* [`loops.txt`](results/loops.txt): A list of direct loops detected
- [`loops.txt`](results/loops.txt): A list of direct loops detected
between modules as they actually exist, as opposed to how they are
desired as described above. In a perfect repo, this file will be
empty.
This file is committed to the repo, and is used by the [levelization
Github workflow](../../.github/workflows/levelization.yml) to validate
Github workflow](../../workflows/reusable-check-levelization.yml) to validate
that nothing changed.
* [`ordering.txt`](results/ordering.txt): A list showing relationships
- [`ordering.txt`](results/ordering.txt): A list showing relationships
between modules where there are no loops as they actually exist, as
opposed to how they are desired as described above.
This file is committed to the repo, and is used by the [levelization
Github workflow](../../.github/workflows/levelization.yml) to validate
Github workflow](../../workflows/reusable-check-levelization.yml) to validate
that nothing changed.
* [`levelization.yml`](../../.github/workflows/levelization.yml)
- [`levelization.yml`](../../workflows/reusable-check-levelization.yml)
Github Actions workflow to test that levelization loops haven't
changed. Unfortunately, if changes are detected, it can't tell if
changed. Unfortunately, if changes are detected, it can't tell if
they are improvements or not, so if you have resolved any issues or
done anything else to improve levelization, run `levelization.sh`,
and commit the updated results.
The `loops.txt` and `ordering.txt` files relate the modules
The `loops.txt` and `ordering.txt` files relate the modules
using comparison signs, which indicate the number of times each
module is included in the other.
* `A > B` means that A should probably be at a higher level than B,
- `A > B` means that A should probably be at a higher level than B,
because B is included in A significantly more than A is included in B.
These results can be included in both `loops.txt` and `ordering.txt`.
Because `ordering.txt`only includes relationships where B is not
included in A at all, it will only include these types of results.
* `A ~= B` means that A and B are included in each other a different
- `A ~= B` means that A and B are included in each other a different
number of times, but the values are so close that the script can't
definitively say that one should be above the other. These results
will only be included in `loops.txt`.
* `A == B` means that A and B include each other the same number of
- `A == B` means that A and B include each other the same number of
times, so the script has no clue which should be higher. These results
will only be included in `loops.txt`.
@@ -110,5 +110,5 @@ get those details locally.
1. Run `levelization.sh`
2. Grep the modules in `paths.txt`.
* For example, if a cycle is found `A ~= B`, simply `grep -w
A Builds/levelization/results/paths.txt | grep -w B`
- For example, if a cycle is found `A ~= B`, simply `grep -w
A .github/scripts/levelization/results/paths.txt | grep -w B`

View File

@@ -1,6 +1,6 @@
#!/bin/bash
# Usage: levelization.sh
# Usage: generate.sh
# This script takes no parameters, reads no environment variables,
# and can be run from any directory, as long as it is in the expected
# location in the repo.
@@ -19,7 +19,7 @@ export LANG=C
rm -rfv results
mkdir results
includes="$( pwd )/results/rawincludes.txt"
pushd ../..
pushd ../../..
echo Raw includes:
grep -r '^[ ]*#include.*/.*\.h' include src | \
grep -v boost | tee ${includes}

View File

@@ -7,12 +7,6 @@ Loop: test.jtx test.unit_test
Loop: xrpld.app xrpld.core
xrpld.app > xrpld.core
Loop: xrpld.app xrpld.ledger
xrpld.app > xrpld.ledger
Loop: xrpld.app xrpld.net
xrpld.app > xrpld.net
Loop: xrpld.app xrpld.overlay
xrpld.overlay > xrpld.app
@@ -25,15 +19,9 @@ Loop: xrpld.app xrpld.rpc
Loop: xrpld.app xrpld.shamap
xrpld.app > xrpld.shamap
Loop: xrpld.core xrpld.net
xrpld.net > xrpld.core
Loop: xrpld.core xrpld.perflog
xrpld.perflog == xrpld.core
Loop: xrpld.net xrpld.rpc
xrpld.rpc ~= xrpld.net
Loop: xrpld.overlay xrpld.rpc
xrpld.rpc ~= xrpld.overlay

View File

@@ -2,6 +2,12 @@ libxrpl.basics > xrpl.basics
libxrpl.crypto > xrpl.basics
libxrpl.json > xrpl.basics
libxrpl.json > xrpl.json
libxrpl.ledger > xrpl.basics
libxrpl.ledger > xrpl.json
libxrpl.ledger > xrpl.ledger
libxrpl.ledger > xrpl.protocol
libxrpl.net > xrpl.basics
libxrpl.net > xrpl.net
libxrpl.protocol > xrpl.basics
libxrpl.protocol > xrpl.json
libxrpl.protocol > xrpl.protocol
@@ -19,11 +25,11 @@ test.app > test.unit_test
test.app > xrpl.basics
test.app > xrpld.app
test.app > xrpld.core
test.app > xrpld.ledger
test.app > xrpld.nodestore
test.app > xrpld.overlay
test.app > xrpld.rpc
test.app > xrpl.json
test.app > xrpl.ledger
test.app > xrpl.protocol
test.app > xrpl.resource
test.basics > test.jtx
@@ -42,8 +48,8 @@ test.consensus > test.unit_test
test.consensus > xrpl.basics
test.consensus > xrpld.app
test.consensus > xrpld.consensus
test.consensus > xrpld.ledger
test.consensus > xrpl.json
test.consensus > xrpl.ledger
test.core > test.jtx
test.core > test.toplevel
test.core > test.unit_test
@@ -61,10 +67,10 @@ test.json > xrpl.json
test.jtx > xrpl.basics
test.jtx > xrpld.app
test.jtx > xrpld.core
test.jtx > xrpld.ledger
test.jtx > xrpld.net
test.jtx > xrpld.rpc
test.jtx > xrpl.json
test.jtx > xrpl.ledger
test.jtx > xrpl.net
test.jtx > xrpl.protocol
test.jtx > xrpl.resource
test.jtx > xrpl.server
@@ -73,7 +79,7 @@ test.ledger > test.toplevel
test.ledger > xrpl.basics
test.ledger > xrpld.app
test.ledger > xrpld.core
test.ledger > xrpld.ledger
test.ledger > xrpl.ledger
test.ledger > xrpl.protocol
test.nodestore > test.jtx
test.nodestore > test.toplevel
@@ -109,7 +115,6 @@ test.rpc > test.toplevel
test.rpc > xrpl.basics
test.rpc > xrpld.app
test.rpc > xrpld.core
test.rpc > xrpld.net
test.rpc > xrpld.overlay
test.rpc > xrpld.rpc
test.rpc > xrpl.json
@@ -132,7 +137,12 @@ test.shamap > xrpl.protocol
test.toplevel > test.csf
test.toplevel > xrpl.json
test.unit_test > xrpl.basics
tests.libxrpl > xrpl.basics
tests.libxrpl > xrpl.net
xrpl.json > xrpl.basics
xrpl.ledger > xrpl.basics
xrpl.ledger > xrpl.protocol
xrpl.net > xrpl.basics
xrpl.protocol > xrpl.basics
xrpl.protocol > xrpl.json
xrpl.resource > xrpl.basics
@@ -148,6 +158,8 @@ xrpld.app > xrpld.consensus
xrpld.app > xrpld.nodestore
xrpld.app > xrpld.perflog
xrpld.app > xrpl.json
xrpld.app > xrpl.ledger
xrpld.app > xrpl.net
xrpld.app > xrpl.protocol
xrpld.app > xrpl.resource
xrpld.conditions > xrpl.basics
@@ -157,14 +169,8 @@ xrpld.consensus > xrpl.json
xrpld.consensus > xrpl.protocol
xrpld.core > xrpl.basics
xrpld.core > xrpl.json
xrpld.core > xrpl.net
xrpld.core > xrpl.protocol
xrpld.ledger > xrpl.basics
xrpld.ledger > xrpl.json
xrpld.ledger > xrpl.protocol
xrpld.net > xrpl.basics
xrpld.net > xrpl.json
xrpld.net > xrpl.protocol
xrpld.net > xrpl.resource
xrpld.nodestore > xrpl.basics
xrpld.nodestore > xrpld.core
xrpld.nodestore > xrpld.unity
@@ -185,9 +191,10 @@ xrpld.perflog > xrpl.basics
xrpld.perflog > xrpl.json
xrpld.rpc > xrpl.basics
xrpld.rpc > xrpld.core
xrpld.rpc > xrpld.ledger
xrpld.rpc > xrpld.nodestore
xrpld.rpc > xrpl.json
xrpld.rpc > xrpl.ledger
xrpld.rpc > xrpl.net
xrpld.rpc > xrpl.protocol
xrpld.rpc > xrpl.resource
xrpld.rpc > xrpl.server

197
.github/scripts/strategy-matrix/generate.py vendored Executable file
View File

@@ -0,0 +1,197 @@
#!/usr/bin/env python3
import argparse
import itertools
import json
from pathlib import Path
from dataclasses import dataclass
THIS_DIR = Path(__file__).parent.resolve()
@dataclass
class Config:
architecture: list[dict]
os: list[dict]
build_type: list[str]
cmake_args: list[str]
'''
Generate a strategy matrix for GitHub Actions CI.
On each PR commit we will build a selection of Debian, RHEL, Ubuntu, MacOS, and
Windows configurations, while upon merge into the develop, release, or master
branches, we will build all configurations, and test most of them.
We will further set additional CMake arguments as follows:
- All builds will have the `tests`, `werr`, and `xrpld` options.
- All builds will have the `wextra` option except for GCC 12 and Clang 16.
- All release builds will have the `assert` option.
- Certain Debian Bookworm configurations will change the reference fee, enable
codecov, and enable voidstar in PRs.
'''
def generate_strategy_matrix(all: bool, config: Config) -> list:
configurations = []
for architecture, os, build_type, cmake_args in itertools.product(config.architecture, config.os, config.build_type, config.cmake_args):
# The default CMake target is 'all' for Linux and MacOS and 'install'
# for Windows, but it can get overridden for certain configurations.
cmake_target = 'install' if os["distro_name"] == 'windows' else 'all'
# We build and test all configurations by default, except for Windows in
# Debug, because it is too slow, as well as when code coverage is
# enabled as that mode already runs the tests.
build_only = False
if os['distro_name'] == 'windows' and build_type == 'Debug':
build_only = True
# Only generate a subset of configurations in PRs.
if not all:
# Debian:
# - Bookworm using GCC 13: Release and Unity on linux/amd64, set
# the reference fee to 500.
# - Bookworm using GCC 15: Debug and no Unity on linux/amd64, enable
# code coverage (which will be done below).
# - Bookworm using Clang 16: Debug and no Unity on linux/arm64,
# enable voidstar.
# - Bookworm using Clang 17: Release and no Unity on linux/amd64,
# set the reference fee to 1000.
# - Bookworm using Clang 20: Debug and Unity on linux/amd64.
if os['distro_name'] == 'debian':
skip = True
if os['distro_version'] == 'bookworm':
if f'{os['compiler_name']}-{os['compiler_version']}' == 'gcc-13' and build_type == 'Release' and '-Dunity=ON' in cmake_args and architecture['platform'] == 'linux/amd64':
cmake_args = f'-DUNIT_TEST_REFERENCE_FEE=500 {cmake_args}'
skip = False
if f'{os['compiler_name']}-{os['compiler_version']}' == 'gcc-15' and build_type == 'Debug' and '-Dunity=OFF' in cmake_args and architecture['platform'] == 'linux/amd64':
skip = False
if f'{os['compiler_name']}-{os['compiler_version']}' == 'clang-16' and build_type == 'Debug' and '-Dunity=OFF' in cmake_args and architecture['platform'] == 'linux/arm64':
cmake_args = f'-Dvoidstar=ON {cmake_args}'
skip = False
if f'{os['compiler_name']}-{os['compiler_version']}' == 'clang-17' and build_type == 'Release' and '-Dunity=ON' in cmake_args and architecture['platform'] == 'linux/amd64':
cmake_args = f'-DUNIT_TEST_REFERENCE_FEE=1000 {cmake_args}'
skip = False
if f'{os['compiler_name']}-{os['compiler_version']}' == 'clang-20' and build_type == 'Debug' and '-Dunity=ON' in cmake_args and architecture['platform'] == 'linux/amd64':
skip = False
if skip:
continue
# RHEL:
# - 9 using GCC 12: Debug and Unity on linux/amd64.
# - 10 using Clang: Release and no Unity on linux/amd64.
if os['distro_name'] == 'rhel':
skip = True
if os['distro_version'] == '9':
if f'{os['compiler_name']}-{os['compiler_version']}' == 'gcc-12' and build_type == 'Debug' and '-Dunity=ON' in cmake_args and architecture['platform'] == 'linux/amd64':
skip = False
elif os['distro_version'] == '10':
if f'{os['compiler_name']}-{os['compiler_version']}' == 'clang-any' and build_type == 'Release' and '-Dunity=OFF' in cmake_args and architecture['platform'] == 'linux/amd64':
skip = False
if skip:
continue
# Ubuntu:
# - Jammy using GCC 12: Debug and no Unity on linux/arm64.
# - Noble using GCC 14: Release and Unity on linux/amd64.
# - Noble using Clang 18: Debug and no Unity on linux/amd64.
# - Noble using Clang 19: Release and Unity on linux/arm64.
if os['distro_name'] == 'ubuntu':
skip = True
if os['distro_version'] == 'jammy':
if f'{os['compiler_name']}-{os['compiler_version']}' == 'gcc-12' and build_type == 'Debug' and '-Dunity=OFF' in cmake_args and architecture['platform'] == 'linux/arm64':
skip = False
elif os['distro_version'] == 'noble':
if f'{os['compiler_name']}-{os['compiler_version']}' == 'gcc-14' and build_type == 'Release' and '-Dunity=ON' in cmake_args and architecture['platform'] == 'linux/amd64':
skip = False
if f'{os['compiler_name']}-{os['compiler_version']}' == 'clang-18' and build_type == 'Debug' and '-Dunity=OFF' in cmake_args and architecture['platform'] == 'linux/amd64':
skip = False
if f'{os['compiler_name']}-{os['compiler_version']}' == 'clang-19' and build_type == 'Release' and '-Dunity=ON' in cmake_args and architecture['platform'] == 'linux/arm64':
skip = False
if skip:
continue
# MacOS:
# - Debug and no Unity on macos/arm64.
if os['distro_name'] == 'macos' and not (build_type == 'Debug' and '-Dunity=OFF' in cmake_args and architecture['platform'] == 'macos/arm64'):
continue
# Windows:
# - Release and Unity on windows/amd64.
if os['distro_name'] == 'windows' and not (build_type == 'Release' and '-Dunity=ON' in cmake_args and architecture['platform'] == 'windows/amd64'):
continue
# Additional CMake arguments.
cmake_args = f'{cmake_args} -Dtests=ON -Dwerr=ON -Dxrpld=ON'
if not f'{os['compiler_name']}-{os['compiler_version']}' in ['gcc-12', 'clang-16']:
cmake_args = f'{cmake_args} -Dwextra=ON'
if build_type == 'Release':
cmake_args = f'{cmake_args} -Dassert=ON'
# We skip all RHEL on arm64 due to a build failure that needs further
# investigation.
if os['distro_name'] == 'rhel' and architecture['platform'] == 'linux/arm64':
continue
# We skip all clang-20 on arm64 due to boost 1.86 build error
if f'{os['compiler_name']}-{os['compiler_version']}' == 'clang-20' and architecture['platform'] == 'linux/arm64':
continue
# Enable code coverage for Debian Bookworm using GCC 15 in Debug and no
# Unity on linux/amd64
if f'{os['compiler_name']}-{os['compiler_version']}' == 'gcc-15' and build_type == 'Debug' and '-Dunity=OFF' in cmake_args and architecture['platform'] == 'linux/amd64':
cmake_args = f'-Dcoverage=ON -Dcoverage_format=xml -DCODE_COVERAGE_VERBOSE=ON -DCMAKE_C_FLAGS=-O0 -DCMAKE_CXX_FLAGS=-O0 {cmake_args}'
cmake_target = 'coverage'
build_only = True
# Generate a unique name for the configuration, e.g. macos-arm64-debug
# or debian-bookworm-gcc-12-amd64-release-unity.
config_name = os['distro_name']
if (n := os['distro_version']) != '':
config_name += f'-{n}'
if (n := os['compiler_name']) != '':
config_name += f'-{n}'
if (n := os['compiler_version']) != '':
config_name += f'-{n}'
config_name += f'-{architecture['platform'][architecture['platform'].find('/')+1:]}'
config_name += f'-{build_type.lower()}'
if '-Dunity=ON' in cmake_args:
config_name += '-unity'
# Add the configuration to the list, with the most unique fields first,
# so that they are easier to identify in the GitHub Actions UI, as long
# names get truncated.
configurations.append({
'config_name': config_name,
'cmake_args': cmake_args,
'cmake_target': cmake_target,
'build_only': build_only,
'build_type': build_type,
'os': os,
'architecture': architecture,
})
return configurations
def read_config(file: Path) -> Config:
config = json.loads(file.read_text())
if config['architecture'] is None or config['os'] is None or config['build_type'] is None or config['cmake_args'] is None:
raise Exception('Invalid configuration file.')
return Config(**config)
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('-a', '--all', help='Set to generate all configurations (generally used when merging a PR) or leave unset to generate a subset of configurations (generally used when committing to a PR).', action="store_true")
parser.add_argument('-c', '--config', help='Path to the JSON file containing the strategy matrix configurations.', required=False, type=Path)
args = parser.parse_args()
matrix = []
if args.config is None or args.config == '':
matrix += generate_strategy_matrix(args.all, read_config(THIS_DIR / "linux.json"))
matrix += generate_strategy_matrix(args.all, read_config(THIS_DIR / "macos.json"))
matrix += generate_strategy_matrix(args.all, read_config(THIS_DIR / "windows.json"))
else:
matrix += generate_strategy_matrix(args.all, read_config(args.config))
# Generate the strategy matrix.
print(f'matrix={json.dumps({"include": matrix})}')

View File

@@ -0,0 +1,170 @@
{
"architecture": [
{
"platform": "linux/amd64",
"runner": ["self-hosted", "Linux", "X64", "heavy"]
},
{
"platform": "linux/arm64",
"runner": ["self-hosted", "Linux", "ARM64", "heavy-arm64"]
}
],
"os": [
{
"distro_name": "debian",
"distro_version": "bookworm",
"compiler_name": "gcc",
"compiler_version": "12",
"image_sha": "6f723eb"
},
{
"distro_name": "debian",
"distro_version": "bookworm",
"compiler_name": "gcc",
"compiler_version": "13",
"image_sha": "6f723eb"
},
{
"distro_name": "debian",
"distro_version": "bookworm",
"compiler_name": "gcc",
"compiler_version": "14",
"image_sha": "6f723eb"
},
{
"distro_name": "debian",
"distro_version": "bookworm",
"compiler_name": "gcc",
"compiler_version": "15",
"image_sha": "6f723eb"
},
{
"distro_name": "debian",
"distro_version": "bookworm",
"compiler_name": "clang",
"compiler_version": "16",
"image_sha": "6f723eb"
},
{
"distro_name": "debian",
"distro_version": "bookworm",
"compiler_name": "clang",
"compiler_version": "17",
"image_sha": "6f723eb"
},
{
"distro_name": "debian",
"distro_version": "bookworm",
"compiler_name": "clang",
"compiler_version": "18",
"image_sha": "6f723eb"
},
{
"distro_name": "debian",
"distro_version": "bookworm",
"compiler_name": "clang",
"compiler_version": "19",
"image_sha": "6f723eb"
},
{
"distro_name": "debian",
"distro_version": "bookworm",
"compiler_name": "clang",
"compiler_version": "20",
"image_sha": "6f723eb"
},
{
"distro_name": "rhel",
"distro_version": "9",
"compiler_name": "gcc",
"compiler_version": "12",
"image_sha": "d133ce3"
},
{
"distro_name": "rhel",
"distro_version": "9",
"compiler_name": "gcc",
"compiler_version": "13",
"image_sha": "d133ce3"
},
{
"distro_name": "rhel",
"distro_version": "9",
"compiler_name": "gcc",
"compiler_version": "14",
"image_sha": "d133ce3"
},
{
"distro_name": "rhel",
"distro_version": "9",
"compiler_name": "clang",
"compiler_version": "any",
"image_sha": "d133ce3"
},
{
"distro_name": "rhel",
"distro_version": "10",
"compiler_name": "gcc",
"compiler_version": "14",
"image_sha": "d133ce3"
},
{
"distro_name": "rhel",
"distro_version": "10",
"compiler_name": "clang",
"compiler_version": "any",
"image_sha": "d133ce3"
},
{
"distro_name": "ubuntu",
"distro_version": "jammy",
"compiler_name": "gcc",
"compiler_version": "12",
"image_sha": "6f723eb"
},
{
"distro_name": "ubuntu",
"distro_version": "noble",
"compiler_name": "gcc",
"compiler_version": "13",
"image_sha": "6f723eb"
},
{
"distro_name": "ubuntu",
"distro_version": "noble",
"compiler_name": "gcc",
"compiler_version": "14",
"image_sha": "6f723eb"
},
{
"distro_name": "ubuntu",
"distro_version": "noble",
"compiler_name": "clang",
"compiler_version": "16",
"image_sha": "6f723eb"
},
{
"distro_name": "ubuntu",
"distro_version": "noble",
"compiler_name": "clang",
"compiler_version": "17",
"image_sha": "6f723eb"
},
{
"distro_name": "ubuntu",
"distro_version": "noble",
"compiler_name": "clang",
"compiler_version": "18",
"image_sha": "6f723eb"
},
{
"distro_name": "ubuntu",
"distro_version": "noble",
"compiler_name": "clang",
"compiler_version": "19",
"image_sha": "6f723eb"
}
],
"build_type": ["Debug", "Release"],
"cmake_args": ["-Dunity=OFF", "-Dunity=ON"]
}

View File

@@ -0,0 +1,22 @@
{
"architecture": [
{
"platform": "macos/arm64",
"runner": ["self-hosted", "macOS", "ARM64", "mac-runner-m1"]
}
],
"os": [
{
"distro_name": "macos",
"distro_version": "",
"compiler_name": "",
"compiler_version": "",
"image_sha": ""
}
],
"build_type": ["Debug", "Release"],
"cmake_args": [
"-Dunity=OFF -DCMAKE_POLICY_VERSION_MINIMUM=3.5",
"-Dunity=ON -DCMAKE_POLICY_VERSION_MINIMUM=3.5"
]
}

View File

@@ -0,0 +1,19 @@
{
"architecture": [
{
"platform": "windows/amd64",
"runner": ["self-hosted", "Windows", "devbox"]
}
],
"os": [
{
"distro_name": "windows",
"distro_version": "",
"compiler_name": "",
"compiler_version": "",
"image_sha": ""
}
],
"build_type": ["Debug", "Release"],
"cmake_args": ["-Dunity=OFF", "-Dunity=ON"]
}

View File

@@ -1,59 +0,0 @@
name: clang-format
on: [push, pull_request]
jobs:
check:
runs-on: ubuntu-24.04
env:
CLANG_VERSION: 18
steps:
- uses: actions/checkout@v4
- name: Install clang-format
run: |
codename=$( lsb_release --codename --short )
sudo tee /etc/apt/sources.list.d/llvm.list >/dev/null <<EOF
deb http://apt.llvm.org/${codename}/ llvm-toolchain-${codename}-${CLANG_VERSION} main
deb-src http://apt.llvm.org/${codename}/ llvm-toolchain-${codename}-${CLANG_VERSION} main
EOF
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add
sudo apt-get update
sudo apt-get install clang-format-${CLANG_VERSION}
- name: Format first-party sources
run: find include src tests -type f \( -name '*.cpp' -o -name '*.hpp' -o -name '*.h' -o -name '*.ipp' \) -exec clang-format-${CLANG_VERSION} -i {} +
- name: Check for differences
id: assert
run: |
set -o pipefail
git diff --exit-code | tee "clang-format.patch"
- name: Upload patch
if: failure() && steps.assert.outcome == 'failure'
uses: actions/upload-artifact@v4
continue-on-error: true
with:
name: clang-format.patch
if-no-files-found: ignore
path: clang-format.patch
- name: What happened?
if: failure() && steps.assert.outcome == 'failure'
env:
PREAMBLE: |
If you are reading this, you are looking at a failed Github Actions
job. That means you pushed one or more files that did not conform
to the formatting specified in .clang-format. That may be because
you neglected to run 'git clang-format' or 'clang-format' before
committing, or that your version of clang-format has an
incompatibility with the one on this
machine, which is:
SUGGESTION: |
To fix it, you can do one of two things:
1. Download and apply the patch generated as an artifact of this
job to your repo, commit, and push.
2. Run 'git-clang-format --extensions cpp,h,hpp,ipp develop'
in your repo, commit, and push.
run: |
echo "${PREAMBLE}"
clang-format-${CLANG_VERSION} --version
echo "${SUGGESTION}"
exit 1

View File

@@ -1,37 +0,0 @@
name: Build and publish Doxygen documentation
# To test this workflow, push your changes to your fork's `develop` branch.
on:
push:
branches:
- develop
- doxygen
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
job:
runs-on: ubuntu-latest
permissions:
contents: write
container: rippleci/rippled-build-ubuntu:aaf5e3e
steps:
- name: checkout
uses: actions/checkout@v4
- name: check environment
run: |
echo ${PATH} | tr ':' '\n'
cmake --version
doxygen --version
env | sort
- name: build
run: |
mkdir build
cd build
cmake -Donly_docs=TRUE ..
cmake --build . --target docs --parallel $(nproc)
- name: publish
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: build/docs/html

View File

@@ -1,49 +0,0 @@
name: levelization
on: [push, pull_request]
jobs:
check:
runs-on: ubuntu-latest
env:
CLANG_VERSION: 10
steps:
- uses: actions/checkout@v4
- name: Check levelization
run: Builds/levelization/levelization.sh
- name: Check for differences
id: assert
run: |
set -o pipefail
git diff --exit-code | tee "levelization.patch"
- name: Upload patch
if: failure() && steps.assert.outcome == 'failure'
uses: actions/upload-artifact@v4
continue-on-error: true
with:
name: levelization.patch
if-no-files-found: ignore
path: levelization.patch
- name: What happened?
if: failure() && steps.assert.outcome == 'failure'
env:
MESSAGE: |
If you are reading this, you are looking at a failed Github
Actions job. That means you changed the dependency relationships
between the modules in rippled. That may be an improvement or a
regression. This check doesn't judge.
A rule of thumb, though, is that if your changes caused
something to be removed from loops.txt, that's probably an
improvement. If something was added, it's probably a regression.
To fix it, you can do one of two things:
1. Download and apply the patch generated as an artifact of this
job to your repo, commit, and push.
2. Run './Builds/levelization/levelization.sh' in your repo,
commit, and push.
See Builds/levelization/README.md for more info.
run: |
echo "${MESSAGE}"
exit 1

View File

@@ -1,89 +0,0 @@
name: Check libXRPL compatibility with Clio
env:
CONAN_URL: http://18.143.149.228:8081/artifactory/api/conan/conan-non-prod
CONAN_LOGIN_USERNAME_RIPPLE: ${{ secrets.CONAN_USERNAME }}
CONAN_PASSWORD_RIPPLE: ${{ secrets.CONAN_TOKEN }}
on:
pull_request:
paths:
- 'src/libxrpl/protocol/BuildInfo.cpp'
- '.github/workflows/libxrpl.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
publish:
name: Publish libXRPL
outputs:
outcome: ${{ steps.upload.outputs.outcome }}
version: ${{ steps.version.outputs.version }}
channel: ${{ steps.channel.outputs.channel }}
runs-on: [self-hosted, heavy]
container: rippleci/rippled-build-ubuntu:aaf5e3e
steps:
- name: Wait for essential checks to succeed
uses: lewagon/wait-on-check-action@v1.3.4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
running-workflow-name: wait-for-check-regexp
check-regexp: '(dependencies|test).*linux.*' # Ignore windows and mac tests but make sure linux passes
repo-token: ${{ secrets.GITHUB_TOKEN }}
wait-interval: 10
- name: Checkout
uses: actions/checkout@v4
- name: Generate channel
id: channel
shell: bash
run: |
echo channel="clio/pr_${{ github.event.pull_request.number }}" | tee ${GITHUB_OUTPUT}
- name: Export new package
shell: bash
run: |
conan export . ${{ steps.channel.outputs.channel }}
- name: Add Ripple Conan remote
shell: bash
run: |
conan remote list
conan remote remove ripple || true
# Do not quote the URL. An empty string will be accepted (with a non-fatal warning), but a missing argument will not.
conan remote add ripple ${{ env.CONAN_URL }} --insert 0
- name: Parse new version
id: version
shell: bash
run: |
echo version="$(cat src/libxrpl/protocol/BuildInfo.cpp | grep "versionString =" \
| awk -F '"' '{print $2}')" | tee ${GITHUB_OUTPUT}
- name: Try to authenticate to Ripple Conan remote
id: remote
shell: bash
run: |
# `conan user` implicitly uses the environment variables CONAN_LOGIN_USERNAME_<REMOTE> and CONAN_PASSWORD_<REMOTE>.
# https://docs.conan.io/1/reference/commands/misc/user.html#using-environment-variables
# https://docs.conan.io/1/reference/env_vars.html#conan-login-username-conan-login-username-remote-name
# https://docs.conan.io/1/reference/env_vars.html#conan-password-conan-password-remote-name
echo outcome=$(conan user --remote ripple --password >&2 \
&& echo success || echo failure) | tee ${GITHUB_OUTPUT}
- name: Upload new package
id: upload
if: (steps.remote.outputs.outcome == 'success')
shell: bash
run: |
echo "conan upload version ${{ steps.version.outputs.version }} on channel ${{ steps.channel.outputs.channel }}"
echo outcome=$(conan upload xrpl/${{ steps.version.outputs.version }}@${{ steps.channel.outputs.channel }} --remote ripple --confirm >&2 \
&& echo success || echo failure) | tee ${GITHUB_OUTPUT}
notify_clio:
name: Notify Clio
runs-on: ubuntu-latest
needs: publish
env:
GH_TOKEN: ${{ secrets.CLIO_NOTIFY_TOKEN }}
steps:
- name: Notify Clio about new version
if: (needs.publish.outputs.outcome == 'success')
shell: bash
run: |
gh api --method POST -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" \
/repos/xrplf/clio/dispatches -f "event_type=check_libxrpl" \
-F "client_payload[version]=${{ needs.publish.outputs.version }}@${{ needs.publish.outputs.channel }}" \
-F "client_payload[pr]=${{ github.event.pull_request.number }}"

View File

@@ -1,94 +0,0 @@
name: macos
on:
pull_request:
push:
# If the branches list is ever changed, be sure to change it on all
# build/test jobs (nix, macos, windows, instrumentation)
branches:
# Always build the package branches
- develop
- release
- master
# Branches that opt-in to running
- 'ci/**'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
test:
strategy:
matrix:
platform:
- macos
generator:
- Ninja
configuration:
- Release
runs-on: [self-hosted, macOS]
env:
# The `build` action requires these variables.
build_dir: .build
NUM_PROCESSORS: 12
steps:
- name: checkout
uses: actions/checkout@v4
- name: install Conan
run: |
brew install conan@1
echo '/opt/homebrew/opt/conan@1/bin' >> $GITHUB_PATH
- name: install Ninja
if: matrix.generator == 'Ninja'
run: brew install ninja
- name: install python
run: |
if which python > /dev/null 2>&1; then
echo "Python executable exists"
else
brew install python@3.13
ln -s /opt/homebrew/bin/python3 /opt/homebrew/bin/python
fi
- name: install cmake
run: |
if which cmake > /dev/null 2>&1; then
echo "cmake executable exists"
else
brew install cmake
fi
- name: install nproc
run: |
brew install coreutils
- name: check environment
run: |
env | sort
echo ${PATH} | tr ':' '\n'
python --version
conan --version
cmake --version
nproc --version
echo -n "nproc returns: "
nproc
- name: configure Conan
run : |
conan profile new default --detect || true
conan profile update settings.compiler.cppstd=20 default
- name: build dependencies
uses: ./.github/actions/dependencies
env:
CONAN_URL: http://18.143.149.228:8081/artifactory/api/conan/conan-non-prod
CONAN_LOGIN_USERNAME_RIPPLE: ${{ secrets.CONAN_USERNAME }}
CONAN_PASSWORD_RIPPLE: ${{ secrets.CONAN_TOKEN }}
with:
configuration: ${{ matrix.configuration }}
- name: build
uses: ./.github/actions/build
with:
generator: ${{ matrix.generator }}
configuration: ${{ matrix.configuration }}
cmake-args: "-Dassert=TRUE -Dwerr=TRUE ${{ matrix.cmake-args }}"
- name: test
run: |
n=$(nproc)
echo "Using $n test jobs"
${build_dir}/rippled --unittest --unittest-jobs $n

View File

@@ -1,60 +0,0 @@
name: missing-commits
on:
push:
branches:
# Only check that the branches are up to date when updating the
# relevant branches.
- develop
- release
jobs:
up_to_date:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Check for missing commits
id: commits
env:
SUGGESTION: |
If you are reading this, then the commits indicated above are
missing from "develop" and/or "release". Do a reverse-merge
as soon as possible. See CONTRIBUTING.md for instructions.
run: |
set -o pipefail
# Branches ordered by how "canonical" they are. Every commit in
# one branch should be in all the branches behind it
order=( master release develop )
branches=()
for branch in "${order[@]}"
do
# Check that the branches exist so that this job will work on
# forked repos, which don't necessarily have master and
# release branches.
if git ls-remote --exit-code --heads origin \
refs/heads/${branch} > /dev/null
then
branches+=( origin/${branch} )
fi
done
prior=()
for branch in "${branches[@]}"
do
if [[ ${#prior[@]} -ne 0 ]]
then
echo "Checking ${prior[@]} for commits missing from ${branch}"
git log --oneline --no-merges "${prior[@]}" \
^$branch | tee -a "missing-commits.txt"
echo
fi
prior+=( "${branch}" )
done
if [[ $( cat missing-commits.txt | wc -l ) -ne 0 ]]
then
echo "${SUGGESTION}"
exit 1
fi

View File

@@ -1,440 +0,0 @@
name: nix
on:
pull_request:
push:
# If the branches list is ever changed, be sure to change it on all
# build/test jobs (nix, macos, windows)
branches:
# Always build the package branches
- develop
- release
- master
# Branches that opt-in to running
- "ci/**"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
# This workflow has multiple job matrixes.
# They can be considered phases because most of the matrices ("test",
# "coverage", "conan", ) depend on the first ("dependencies").
#
# The first phase has a job in the matrix for each combination of
# variables that affects dependency ABI:
# platform, compiler, and configuration.
# It creates a GitHub artifact holding the Conan profile,
# and builds and caches binaries for all the dependencies.
# If an Artifactory remote is configured, they are cached there.
# If not, they are added to the GitHub artifact.
# GitHub's "cache" action has a size limit (10 GB) that is too small
# to hold the binaries if they are built locally.
# We must use the "{upload,download}-artifact" actions instead.
#
# The remaining phases have a job in the matrix for each test
# configuration. They install dependency binaries from the cache,
# whichever was used, and build and test rippled.
#
# "instrumentation" is independent, but is included here because it also
# builds on linux in the same "on:" conditions.
jobs:
dependencies:
strategy:
fail-fast: false
matrix:
platform:
- linux
compiler:
- gcc
- clang
configuration:
- Debug
- Release
include:
- compiler: gcc
profile:
version: 11
cc: /usr/bin/gcc
cxx: /usr/bin/g++
- compiler: clang
profile:
version: 14
cc: /usr/bin/clang-14
cxx: /usr/bin/clang++-14
runs-on: [self-hosted, heavy]
container: rippleci/rippled-build-ubuntu:aaf5e3e
env:
build_dir: .build
steps:
- name: upgrade conan
run: |
pip install --upgrade "conan<2"
- name: checkout
uses: actions/checkout@v4
- name: check environment
run: |
echo ${PATH} | tr ':' '\n'
lsb_release -a || true
${{ matrix.profile.cc }} --version
conan --version
cmake --version
env | sort
- name: configure Conan
run: |
conan profile new default --detect
conan profile update settings.compiler.cppstd=20 default
conan profile update settings.compiler=${{ matrix.compiler }} default
conan profile update settings.compiler.version=${{ matrix.profile.version }} default
conan profile update settings.compiler.libcxx=libstdc++11 default
conan profile update env.CC=${{ matrix.profile.cc }} default
conan profile update env.CXX=${{ matrix.profile.cxx }} default
conan profile update conf.tools.build:compiler_executables='{"c": "${{ matrix.profile.cc }}", "cpp": "${{ matrix.profile.cxx }}"}' default
- name: archive profile
# Create this archive before dependencies are added to the local cache.
run: tar -czf conan.tar -C ~/.conan .
- name: build dependencies
uses: ./.github/actions/dependencies
env:
CONAN_URL: http://18.143.149.228:8081/artifactory/api/conan/conan-non-prod
CONAN_LOGIN_USERNAME_RIPPLE: ${{ secrets.CONAN_USERNAME }}
CONAN_PASSWORD_RIPPLE: ${{ secrets.CONAN_TOKEN }}
with:
configuration: ${{ matrix.configuration }}
- name: upload archive
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.platform }}-${{ matrix.compiler }}-${{ matrix.configuration }}
path: conan.tar
if-no-files-found: error
test:
strategy:
fail-fast: false
matrix:
platform:
- linux
compiler:
- gcc
- clang
configuration:
- Debug
- Release
cmake-args:
-
- "-Dunity=ON"
needs: dependencies
runs-on: [self-hosted, heavy]
container: rippleci/rippled-build-ubuntu:aaf5e3e
env:
build_dir: .build
steps:
- name: upgrade conan
run: |
pip install --upgrade "conan<2"
- name: download cache
uses: actions/download-artifact@v4
with:
name: ${{ matrix.platform }}-${{ matrix.compiler }}-${{ matrix.configuration }}
- name: extract cache
run: |
mkdir -p ~/.conan
tar -xzf conan.tar -C ~/.conan
- name: check environment
run: |
env | sort
echo ${PATH} | tr ':' '\n'
conan --version
cmake --version
- name: checkout
uses: actions/checkout@v4
- name: dependencies
uses: ./.github/actions/dependencies
env:
CONAN_URL: http://18.143.149.228:8081/artifactory/api/conan/conan-non-prod
with:
configuration: ${{ matrix.configuration }}
- name: build
uses: ./.github/actions/build
with:
generator: Ninja
configuration: ${{ matrix.configuration }}
cmake-args: "-Dassert=TRUE -Dwerr=TRUE ${{ matrix.cmake-args }}"
- name: test
run: |
${build_dir}/rippled --unittest --unittest-jobs $(nproc)
reference-fee-test:
strategy:
fail-fast: false
matrix:
platform:
- linux
compiler:
- gcc
configuration:
- Debug
cmake-args:
- "-DUNIT_TEST_REFERENCE_FEE=200"
- "-DUNIT_TEST_REFERENCE_FEE=1000"
needs: dependencies
runs-on: [self-hosted, heavy]
container: rippleci/rippled-build-ubuntu:aaf5e3e
env:
build_dir: .build
steps:
- name: upgrade conan
run: |
pip install --upgrade "conan<2"
- name: download cache
uses: actions/download-artifact@v4
with:
name: ${{ matrix.platform }}-${{ matrix.compiler }}-${{ matrix.configuration }}
- name: extract cache
run: |
mkdir -p ~/.conan
tar -xzf conan.tar -C ~/.conan
- name: check environment
run: |
env | sort
echo ${PATH} | tr ':' '\n'
conan --version
cmake --version
- name: checkout
uses: actions/checkout@v4
- name: dependencies
uses: ./.github/actions/dependencies
env:
CONAN_URL: http://18.143.149.228:8081/artifactory/api/conan/conan-non-prod
with:
configuration: ${{ matrix.configuration }}
- name: build
uses: ./.github/actions/build
with:
generator: Ninja
configuration: ${{ matrix.configuration }}
cmake-args: "-Dassert=TRUE -Dwerr=TRUE ${{ matrix.cmake-args }}"
- name: test
run: |
${build_dir}/rippled --unittest --unittest-jobs $(nproc)
coverage:
strategy:
fail-fast: false
matrix:
platform:
- linux
compiler:
- gcc
configuration:
- Debug
needs: dependencies
runs-on: [self-hosted, heavy]
container: rippleci/rippled-build-ubuntu:aaf5e3e
env:
build_dir: .build
steps:
- name: upgrade conan
run: |
pip install --upgrade "conan<2"
- name: download cache
uses: actions/download-artifact@v4
with:
name: ${{ matrix.platform }}-${{ matrix.compiler }}-${{ matrix.configuration }}
- name: extract cache
run: |
mkdir -p ~/.conan
tar -xzf conan.tar -C ~/.conan
- name: install gcovr
run: pip install "gcovr>=7,<8"
- name: check environment
run: |
echo ${PATH} | tr ':' '\n'
conan --version
cmake --version
gcovr --version
env | sort
ls ~/.conan
- name: checkout
uses: actions/checkout@v4
- name: dependencies
uses: ./.github/actions/dependencies
env:
CONAN_URL: http://18.143.149.228:8081/artifactory/api/conan/conan-non-prod
with:
configuration: ${{ matrix.configuration }}
- name: build
uses: ./.github/actions/build
with:
generator: Ninja
configuration: ${{ matrix.configuration }}
cmake-args: >-
-Dassert=TRUE
-Dwerr=TRUE
-Dcoverage=ON
-Dcoverage_format=xml
-DCODE_COVERAGE_VERBOSE=ON
-DCMAKE_CXX_FLAGS="-O0"
-DCMAKE_C_FLAGS="-O0"
cmake-target: coverage
- name: move coverage report
shell: bash
run: |
mv "${build_dir}/coverage.xml" ./
- name: archive coverage report
uses: actions/upload-artifact@v4
with:
name: coverage.xml
path: coverage.xml
retention-days: 30
- name: upload coverage report
uses: wandalen/wretry.action@v1.4.10
with:
action: codecov/codecov-action@v4.5.0
with: |
files: coverage.xml
fail_ci_if_error: true
disable_search: true
verbose: true
plugin: noop
token: ${{ secrets.CODECOV_TOKEN }}
attempt_limit: 5
attempt_delay: 210000 # in milliseconds
conan:
needs: dependencies
runs-on: [self-hosted, heavy]
container: rippleci/rippled-build-ubuntu:aaf5e3e
env:
build_dir: .build
configuration: Release
steps:
- name: upgrade conan
run: |
pip install --upgrade "conan<2"
- name: download cache
uses: actions/download-artifact@v4
with:
name: linux-gcc-${{ env.configuration }}
- name: extract cache
run: |
mkdir -p ~/.conan
tar -xzf conan.tar -C ~/.conan
- name: check environment
run: |
env | sort
echo ${PATH} | tr ':' '\n'
conan --version
cmake --version
- name: checkout
uses: actions/checkout@v4
- name: dependencies
uses: ./.github/actions/dependencies
env:
CONAN_URL: http://18.143.149.228:8081/artifactory/api/conan/conan-non-prod
with:
configuration: ${{ env.configuration }}
- name: export
run: |
version=$(conan inspect --raw version .)
reference="xrpl/${version}@local/test"
conan remove -f ${reference} || true
conan export . local/test
echo "reference=${reference}" >> "${GITHUB_ENV}"
- name: build
run: |
cd tests/conan
mkdir ${build_dir}
cd ${build_dir}
conan install .. --output-folder . \
--require-override ${reference} --build missing
cmake .. \
-DCMAKE_TOOLCHAIN_FILE:FILEPATH=./build/${configuration}/generators/conan_toolchain.cmake \
-DCMAKE_BUILD_TYPE=${configuration}
cmake --build .
./example | grep '^[[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+'
# NOTE we are not using dependencies built above because it lags with
# compiler versions. Instrumentation requires clang version 16 or
# later
instrumentation-build:
env:
CLANG_RELEASE: 16
strategy:
fail-fast: false
runs-on: [self-hosted, heavy]
container: debian:bookworm
steps:
- name: install prerequisites
env:
DEBIAN_FRONTEND: noninteractive
run: |
apt-get update
apt-get install --yes --no-install-recommends \
clang-${CLANG_RELEASE} clang++-${CLANG_RELEASE} \
python3-pip python-is-python3 make cmake git wget
apt-get clean
update-alternatives --install \
/usr/bin/clang clang /usr/bin/clang-${CLANG_RELEASE} 100 \
--slave /usr/bin/clang++ clang++ /usr/bin/clang++-${CLANG_RELEASE}
update-alternatives --auto clang
pip install --no-cache --break-system-packages "conan<2"
- name: checkout
uses: actions/checkout@v4
- name: prepare environment
run: |
mkdir ${GITHUB_WORKSPACE}/.build
echo "SOURCE_DIR=$GITHUB_WORKSPACE" >> $GITHUB_ENV
echo "BUILD_DIR=$GITHUB_WORKSPACE/.build" >> $GITHUB_ENV
echo "CC=/usr/bin/clang" >> $GITHUB_ENV
echo "CXX=/usr/bin/clang++" >> $GITHUB_ENV
- name: configure Conan
run: |
conan profile new --detect default
conan profile update settings.compiler=clang default
conan profile update settings.compiler.version=${CLANG_RELEASE} default
conan profile update settings.compiler.libcxx=libstdc++11 default
conan profile update settings.compiler.cppstd=20 default
conan profile update options.rocksdb=False default
conan profile update \
'conf.tools.build:compiler_executables={"c": "/usr/bin/clang", "cpp": "/usr/bin/clang++"}' default
conan profile update 'env.CXXFLAGS="-DBOOST_ASIO_DISABLE_CONCEPTS"' default
conan profile update 'conf.tools.build:cxxflags+=["-DBOOST_ASIO_DISABLE_CONCEPTS"]' default
conan export external/snappy snappy/1.1.10@
conan export external/soci soci/4.0.3@
- name: build dependencies
run: |
cd ${BUILD_DIR}
conan install ${SOURCE_DIR} \
--output-folder ${BUILD_DIR} \
--install-folder ${BUILD_DIR} \
--build missing \
--settings build_type=Debug
- name: build with instrumentation
run: |
cd ${BUILD_DIR}
cmake -S ${SOURCE_DIR} -B ${BUILD_DIR} \
-Dvoidstar=ON \
-Dtests=ON \
-Dxrpld=ON \
-DCMAKE_BUILD_TYPE=Debug \
-DSECP256K1_BUILD_BENCHMARK=OFF \
-DSECP256K1_BUILD_TESTS=OFF \
-DSECP256K1_BUILD_EXHAUSTIVE_TESTS=OFF \
-DCMAKE_TOOLCHAIN_FILE=${BUILD_DIR}/build/generators/conan_toolchain.cmake
cmake --build . --parallel $(nproc)
- name: verify instrumentation enabled
run: |
cd ${BUILD_DIR}
./rippled --version | grep libvoidstar
- name: run unit tests
run: |
cd ${BUILD_DIR}
./rippled -u --unittest-jobs $(( $(nproc)/4 ))

132
.github/workflows/on-pr.yml vendored Normal file
View File

@@ -0,0 +1,132 @@
# This workflow runs all workflows to check, build and test the project on
# various Linux flavors, as well as on MacOS and Windows, on every push to a
# user branch. However, it will not run if the pull request is a draft unless it
# has the 'DraftRunCI' label.
name: PR
on:
merge_group:
types:
- checks_requested
pull_request:
types:
- opened
- reopened
- synchronize
- ready_for_review
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
defaults:
run:
shell: bash
jobs:
# This job determines whether the rest of the workflow should run. It runs
# when the PR is not a draft (which should also cover merge-group) or
# has the 'DraftRunCI' label.
should-run:
if: ${{ !github.event.pull_request.draft || contains(github.event.pull_request.labels.*.name, 'DraftRunCI') }}
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
- name: Determine changed files
# This step checks whether any files have changed that should
# cause the next jobs to run. We do it this way rather than
# using `paths` in the `on:` section, because all required
# checks must pass, even for changes that do not modify anything
# that affects those checks. We would therefore like to make the
# checks required only if the job runs, but GitHub does not
# support that directly. By always executing the workflow on new
# commits and by using the changed-files action below, we ensure
# that Github considers any skipped jobs to have passed, and in
# turn the required checks as well.
id: changes
uses: tj-actions/changed-files@ed68ef82c095e0d48ec87eccea555d944a631a4c # v46.0.5
with:
files: |
# These paths are unique to `on-pr.yml`.
.github/scripts/levelization/**
.github/workflows/reusable-check-levelization.yml
.github/workflows/reusable-notify-clio.yml
.github/workflows/on-pr.yml
# Keep the paths below in sync with those in `on-trigger.yml`.
.github/actions/build-deps/**
.github/actions/build-test/**
.github/actions/setup-conan/**
.github/scripts/strategy-matrix/**
.github/workflows/reusable-build.yml
.github/workflows/reusable-build-test-config.yml
.github/workflows/reusable-build-test.yml
.github/workflows/reusable-strategy-matrix.yml
.github/workflows/reusable-test.yml
.codecov.yml
cmake/**
conan/**
external/**
include/**
src/**
tests/**
CMakeLists.txt
conanfile.py
conan.lock
- name: Check whether to run
# This step determines whether the rest of the workflow should
# run. The rest of the workflow will run if this job runs AND at
# least one of:
# * Any of the files checked in the `changes` step were modified
# * The PR is NOT a draft and is labeled "Ready to merge"
# * The workflow is running from the merge queue
id: go
env:
FILES: ${{ steps.changes.outputs.any_changed }}
DRAFT: ${{ github.event.pull_request.draft }}
READY: ${{ contains(github.event.pull_request.labels.*.name, 'Ready to merge') }}
MERGE: ${{ github.event_name == 'merge_group' }}
run: |
echo "go=${{ (env.DRAFT != 'true' && env.READY == 'true') || env.FILES == 'true' || env.MERGE == 'true' }}" >> "${GITHUB_OUTPUT}"
cat "${GITHUB_OUTPUT}"
outputs:
go: ${{ steps.go.outputs.go == 'true' }}
check-levelization:
needs: should-run
if: ${{ needs.should-run.outputs.go == 'true' }}
uses: ./.github/workflows/reusable-check-levelization.yml
build-test:
needs: should-run
if: ${{ needs.should-run.outputs.go == 'true' }}
uses: ./.github/workflows/reusable-build-test.yml
strategy:
matrix:
os: [linux, macos, windows]
with:
os: ${{ matrix.os }}
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
notify-clio:
needs:
- should-run
- build-test
if: ${{ needs.should-run.outputs.go == 'true' && contains(fromJSON('["release", "master"]'), github.ref_name) }}
uses: ./.github/workflows/reusable-notify-clio.yml
secrets:
clio_notify_token: ${{ secrets.CLIO_NOTIFY_TOKEN }}
conan_remote_username: ${{ secrets.CONAN_REMOTE_USERNAME }}
conan_remote_password: ${{ secrets.CONAN_REMOTE_PASSWORD }}
passed:
if: failure() || cancelled()
needs:
- build-test
- check-levelization
runs-on: ubuntu-latest
steps:
- name: Fail
run: false

74
.github/workflows/on-trigger.yml vendored Normal file
View File

@@ -0,0 +1,74 @@
# This workflow runs all workflows to build the dependencies required for the
# project on various Linux flavors, as well as on MacOS and Windows, on a
# scheduled basis, on merge into the 'develop', 'release', or 'master' branches,
# or manually. The missing commits check is only run when the code is merged
# into the 'develop' or 'release' branches, and the documentation is built when
# the code is merged into the 'develop' branch.
name: Trigger
on:
push:
branches:
- develop
- release
- master
paths:
# These paths are unique to `on-trigger.yml`.
- ".github/workflows/reusable-check-missing-commits.yml"
- ".github/workflows/on-trigger.yml"
- ".github/workflows/publish-docs.yml"
# Keep the paths below in sync with those in `on-pr.yml`.
- ".github/actions/build-deps/**"
- ".github/actions/build-test/**"
- ".github/actions/setup-conan/**"
- ".github/scripts/strategy-matrix/**"
- ".github/workflows/reusable-build.yml"
- ".github/workflows/reusable-build-test-config.yml"
- ".github/workflows/reusable-build-test.yml"
- ".github/workflows/reusable-strategy-matrix.yml"
- ".github/workflows/reusable-test.yml"
- ".codecov.yml"
- "cmake/**"
- "conan/**"
- "external/**"
- "include/**"
- "src/**"
- "tests/**"
- "CMakeLists.txt"
- "conanfile.py"
- "conan.lock"
# Run at 06:32 UTC on every day of the week from Monday through Friday. This
# will force all dependencies to be rebuilt, which is useful to verify that
# all dependencies can be built successfully. Only the dependencies that
# are actually missing from the remote will be uploaded.
schedule:
- cron: "32 6 * * 1-5"
# Run when manually triggered via the GitHub UI or API.
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
defaults:
run:
shell: bash
jobs:
check-missing-commits:
if: ${{ github.event_name == 'push' && github.ref_type == 'branch' && contains(fromJSON('["develop", "release"]'), github.ref_name) }}
uses: ./.github/workflows/reusable-check-missing-commits.yml
build-test:
uses: ./.github/workflows/reusable-build-test.yml
strategy:
matrix:
os: [linux, macos, windows]
with:
os: ${{ matrix.os }}
strategy_matrix: ${{ github.event_name == 'schedule' && 'all' || 'minimal' }}
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

15
.github/workflows/pre-commit.yml vendored Normal file
View File

@@ -0,0 +1,15 @@
name: Run pre-commit hooks
on:
pull_request:
push:
branches: [develop, release, master]
workflow_dispatch:
jobs:
# Call the workflow in the XRPLF/actions repo that runs the pre-commit hooks.
run-hooks:
uses: XRPLF/actions/.github/workflows/pre-commit.yml@af1b0f0d764cda2e5435f5ac97b240d4bd4d95d3
with:
runs_on: ubuntu-latest
container: '{ "image": "ghcr.io/xrplf/ci/tools-rippled-pre-commit:sha-d1496b8" }'

60
.github/workflows/publish-docs.yml vendored Normal file
View File

@@ -0,0 +1,60 @@
# This workflow builds the documentation for the repository, and publishes it to
# GitHub Pages when changes are merged into the default branch.
name: Build and publish documentation
on:
push:
paths:
- ".github/workflows/publish-docs.yml"
- "*.md"
- "**/*.md"
- "docs/**"
- "include/**"
- "src/libxrpl/**"
- "src/xrpld/**"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
defaults:
run:
shell: bash
env:
BUILD_DIR: .build
jobs:
publish:
runs-on: ubuntu-latest
container: ghcr.io/xrplf/ci/tools-rippled-documentation:sha-d1496b8
permissions:
contents: write
steps:
- name: Checkout repository
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
- name: Check configuration
run: |
echo 'Checking path.'
echo ${PATH} | tr ':' '\n'
echo 'Checking environment variables.'
env | sort
echo 'Checking CMake version.'
cmake --version
echo 'Checking Doxygen version.'
doxygen --version
- name: Build documentation
run: |
mkdir -p ${{ env.BUILD_DIR }}
cd ${{ env.BUILD_DIR }}
cmake -Donly_docs=ON ..
cmake --build . --target docs --parallel $(nproc)
- name: Publish documentation
if: ${{ github.ref_type == 'branch' && github.ref_name == github.event.repository.default_branch }}
uses: peaceiris/actions-gh-pages@4f9cc6602d3f66b9c108549d475ec49e8ef4d45e # v4.0.0
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ${{ env.BUILD_DIR }}/docs/html

View File

@@ -0,0 +1,69 @@
name: Build and test configuration
on:
workflow_call:
inputs:
build_dir:
description: "The directory where to build."
required: true
type: string
build_only:
description: 'Whether to only build or to build and test the code ("true", "false").'
required: true
type: boolean
build_type:
description: 'The build type to use ("Debug", "Release").'
type: string
required: true
cmake_args:
description: "Additional arguments to pass to CMake."
required: false
type: string
default: ""
cmake_target:
description: "The CMake target to build."
type: string
required: true
runs_on:
description: Runner to run the job on as a JSON string
required: true
type: string
image:
description: "The image to run in (leave empty to run natively)"
required: true
type: string
config_name:
description: "The configuration string (used for naming artifacts and such)."
required: true
type: string
secrets:
CODECOV_TOKEN:
description: "The Codecov token to use for uploading coverage reports."
required: true
jobs:
build:
uses: ./.github/workflows/reusable-build.yml
with:
build_dir: ${{ inputs.build_dir }}
build_type: ${{ inputs.build_type }}
cmake_args: ${{ inputs.cmake_args }}
cmake_target: ${{ inputs.cmake_target }}
runs_on: ${{ inputs.runs_on }}
image: ${{ inputs.image }}
config_name: ${{ inputs.config_name }}
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
test:
needs: build
uses: ./.github/workflows/reusable-test.yml
with:
run_tests: ${{ !inputs.build_only }}
verify_voidstar: ${{ contains(inputs.cmake_args, '-Dvoidstar=ON') }}
runs_on: ${{ inputs.runs_on }}
image: ${{ inputs.image }}
config_name: ${{ inputs.config_name }}

View File

@@ -0,0 +1,58 @@
# This workflow builds and tests the binary for various configurations.
name: Build and test
# This workflow can only be triggered by other workflows. Note that the
# workflow_call event does not support the 'choice' input type, see
# https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#onworkflow_callinputsinput_idtype,
# so we use 'string' instead.
on:
workflow_call:
inputs:
build_dir:
description: "The directory where to build."
required: false
type: string
default: ".build"
os:
description: 'The operating system to use for the build ("linux", "macos", "windows").'
required: true
type: string
strategy_matrix:
# TODO: Support additional strategies, e.g. "ubuntu" for generating all Ubuntu configurations.
description: 'The strategy matrix to use for generating the configurations ("minimal", "all").'
required: false
type: string
default: "minimal"
secrets:
CODECOV_TOKEN:
description: "The Codecov token to use for uploading coverage reports."
required: true
jobs:
# Generate the strategy matrix to be used by the following job.
generate-matrix:
uses: ./.github/workflows/reusable-strategy-matrix.yml
with:
os: ${{ inputs.os }}
strategy_matrix: ${{ inputs.strategy_matrix }}
# Build and test the binary for each configuration.
build-test-config:
needs:
- generate-matrix
uses: ./.github/workflows/reusable-build-test-config.yml
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.generate-matrix.outputs.matrix) }}
max-parallel: 10
with:
build_dir: ${{ inputs.build_dir }}
build_only: ${{ matrix.build_only }}
build_type: ${{ matrix.build_type }}
cmake_args: ${{ matrix.cmake_args }}
cmake_target: ${{ matrix.cmake_target }}
runs_on: ${{ toJSON(matrix.architecture.runner) }}
image: ${{ contains(matrix.architecture.platform, 'linux') && format('ghcr.io/xrplf/ci/{0}-{1}:{2}-{3}-sha-{4}', matrix.os.distro_name, matrix.os.distro_version, matrix.os.compiler_name, matrix.os.compiler_version, matrix.os.image_sha) || '' }}
config_name: ${{ matrix.config_name }}
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

121
.github/workflows/reusable-build.yml vendored Normal file
View File

@@ -0,0 +1,121 @@
name: Build rippled
on:
workflow_call:
inputs:
build_dir:
description: "The directory where to build."
required: true
type: string
build_type:
description: 'The build type to use ("Debug", "Release").'
required: true
type: string
cmake_args:
description: "Additional arguments to pass to CMake."
required: true
type: string
cmake_target:
description: "The CMake target to build."
required: true
type: string
runs_on:
description: Runner to run the job on as a JSON string
required: true
type: string
image:
description: "The image to run in (leave empty to run natively)"
required: true
type: string
config_name:
description: "The name of the configuration."
required: true
type: string
secrets:
CODECOV_TOKEN:
description: "The Codecov token to use for uploading coverage reports."
required: true
defaults:
run:
shell: bash
jobs:
build:
name: Build ${{ inputs.config_name }}
runs-on: ${{ fromJSON(inputs.runs_on) }}
container: ${{ inputs.image != '' && inputs.image || null }}
steps:
- name: Cleanup workspace
if: ${{ runner.os == 'macOS' }}
uses: XRPLF/actions/.github/actions/cleanup-workspace@3f044c7478548e3c32ff68980eeb36ece02b364e
- name: Checkout repository
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
- name: Prepare runner
uses: XRPLF/actions/.github/actions/prepare-runner@638e0dc11ea230f91bd26622fb542116bb5254d5
with:
disable_ccache: false
- name: Print build environment
uses: ./.github/actions/print-env
- name: Setup Conan
uses: ./.github/actions/setup-conan
- name: Build dependencies
uses: ./.github/actions/build-deps
with:
build_dir: ${{ inputs.build_dir }}
build_type: ${{ inputs.build_type }}
- name: Configure CMake
shell: bash
working-directory: ${{ inputs.build_dir }}
env:
BUILD_TYPE: ${{ inputs.build_type }}
CMAKE_ARGS: ${{ inputs.cmake_args }}
run: |
cmake \
-G '${{ runner.os == 'Windows' && 'Visual Studio 17 2022' || 'Ninja' }}' \
-DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake \
-DCMAKE_BUILD_TYPE=${{ env.BUILD_TYPE }} \
${{ env.CMAKE_ARGS }} \
..
- name: Build the binary
shell: bash
working-directory: ${{ inputs.build_dir }}
env:
BUILD_TYPE: ${{ inputs.build_type }}
CMAKE_TARGET: ${{ inputs.cmake_target }}
run: |
cmake \
--build . \
--config ${{ env.BUILD_TYPE }} \
--parallel $(nproc) \
--target ${{ env.CMAKE_TARGET }}
- name: Upload rippled artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: rippled-${{ inputs.config_name }}
path: ${{ inputs.build_dir }}/${{ runner.os == 'Windows' && inputs.build_type || '' }}/rippled${{ runner.os == 'Windows' && '.exe' || '' }}
retention-days: 3
if-no-files-found: error
- name: Upload coverage report
if: ${{ inputs.cmake_target == 'coverage' }}
uses: codecov/codecov-action@18283e04ce6e62d37312384ff67231eb8fd56d24 # v5.4.3
with:
disable_search: true
disable_telem: true
fail_ci_if_error: true
files: ${{ inputs.build_dir }}/coverage.xml
plugins: noop
token: ${{ secrets.CODECOV_TOKEN }}
verbose: true

View File

@@ -0,0 +1,46 @@
# This workflow checks if the dependencies between the modules are correctly
# indexed.
name: Check levelization
# This workflow can only be triggered by other workflows.
on: workflow_call
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-levelization
cancel-in-progress: true
defaults:
run:
shell: bash
jobs:
levelization:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
- name: Check levelization
run: .github/scripts/levelization/generate.sh
- name: Check for differences
env:
MESSAGE: |
The dependency relationships between the modules in rippled have
changed, which may be an improvement or a regression.
A rule of thumb is that if your changes caused something to be
removed from loops.txt, it's probably an improvement, while if
something was added, it's probably a regression.
Run '.github/scripts/levelization/generate.sh' in your repo, commit
and push the changes. See .github/scripts/levelization/README.md for
more info.
run: |
DIFF=$(git status --porcelain)
if [ -n "${DIFF}" ]; then
# Print the differences to give the contributor a hint about what to
# expect when running levelization on their own machine.
git diff
echo "${MESSAGE}"
exit 1
fi

View File

@@ -0,0 +1,62 @@
# This workflow checks that all commits in the "master" branch are also in the
# "release" and "develop" branches, and that all commits in the "release" branch
# are also in the "develop" branch.
name: Check for missing commits
# This workflow can only be triggered by other workflows.
on: workflow_call
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-missing-commits
cancel-in-progress: true
defaults:
run:
shell: bash
jobs:
check:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
fetch-depth: 0
- name: Check for missing commits
env:
MESSAGE: |
If you are reading this, then the commits indicated above are missing
from the "develop" and/or "release" branch. Do a reverse-merge as soon
as possible. See CONTRIBUTING.md for instructions.
run: |
set -o pipefail
# Branches are ordered by how "canonical" they are. Every commit in one
# branch should be in all the branches behind it.
order=(master release develop)
branches=()
for branch in "${order[@]}"; do
# Check that the branches exist so that this job will work on forked
# repos, which don't necessarily have master and release branches.
echo "Checking if ${branch} exists."
if git ls-remote --exit-code --heads origin \
refs/heads/${branch} > /dev/null; then
branches+=(origin/${branch})
fi
done
prior=()
for branch in "${branches[@]}"; do
if [[ ${#prior[@]} -ne 0 ]]; then
echo "Checking ${prior[@]} for commits missing from ${branch}."
git log --oneline --no-merges "${prior[@]}" \
^$branch | tee -a "missing-commits.txt"
echo
fi
prior+=("${branch}")
done
if [[ $(cat missing-commits.txt | wc -l) -ne 0 ]]; then
echo "${MESSAGE}"
exit 1
fi

View File

@@ -0,0 +1,89 @@
# This workflow exports the built libxrpl package to the Conan remote on a
# a channel named after the pull request, and notifies the Clio repository about
# the new version so it can check for compatibility.
name: Notify Clio
# This workflow can only be triggered by other workflows.
on:
workflow_call:
inputs:
conan_remote_name:
description: "The name of the Conan remote to use."
required: false
type: string
default: xrplf
conan_remote_url:
description: "The URL of the Conan endpoint to use."
required: false
type: string
default: https://conan.ripplex.io
secrets:
clio_notify_token:
description: "The GitHub token to notify Clio about new versions."
required: true
conan_remote_username:
description: "The username for logging into the Conan remote."
required: true
conan_remote_password:
description: "The password for logging into the Conan remote."
required: true
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-clio
cancel-in-progress: true
defaults:
run:
shell: bash
jobs:
upload:
if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
runs-on: ubuntu-latest
container: ghcr.io/xrplf/ci/ubuntu-noble:gcc-13-sha-5dd7158
steps:
- name: Checkout repository
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
- name: Generate outputs
id: generate
env:
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
echo 'Generating user and channel.'
echo "user=clio" >> "${GITHUB_OUTPUT}"
echo "channel=pr_${{ env.PR_NUMBER }}" >> "${GITHUB_OUTPUT}"
echo 'Extracting version.'
echo "version=$(cat src/libxrpl/protocol/BuildInfo.cpp | grep "versionString =" | awk -F '"' '{print $2}')" >> "${GITHUB_OUTPUT}"
- name: Calculate conan reference
id: conan_ref
run: |
echo "conan_ref=${{ steps.generate.outputs.version }}@${{ steps.generate.outputs.user }}/${{ steps.generate.outputs.channel }}" >> "${GITHUB_OUTPUT}"
- name: Set up Conan
uses: ./.github/actions/setup-conan
with:
conan_remote_name: ${{ inputs.conan_remote_name }}
conan_remote_url: ${{ inputs.conan_remote_url }}
- name: Log into Conan remote
run: conan remote login ${{ inputs.conan_remote_name }} "${{ secrets.conan_remote_username }}" --password "${{ secrets.conan_remote_password }}"
- name: Upload package
env:
CONAN_REMOTE_NAME: ${{ inputs.conan_remote_name }}
run: |
conan export --user=${{ steps.generate.outputs.user }} --channel=${{ steps.generate.outputs.channel }} .
conan upload --confirm --check --remote=${{ env.CONAN_REMOTE_NAME }} xrpl/${{ steps.conan_ref.outputs.conan_ref }}
outputs:
conan_ref: ${{ steps.conan_ref.outputs.conan_ref }}
notify:
needs: upload
runs-on: ubuntu-latest
steps:
- name: Notify Clio
env:
GH_TOKEN: ${{ secrets.clio_notify_token }}
PR_URL: ${{ github.event.pull_request.html_url }}
run: |
gh api --method POST -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" \
/repos/xrplf/clio/dispatches -f "event_type=check_libxrpl" \
-F "client_payload[conan_ref]=${{ needs.upload.outputs.conan_ref }}" \
-F "client_payload[pr_url]=${{ env.PR_URL }}"

View File

@@ -0,0 +1,41 @@
name: Generate strategy matrix
on:
workflow_call:
inputs:
os:
description: 'The operating system to use for the build ("linux", "macos", "windows").'
required: false
type: string
strategy_matrix:
# TODO: Support additional strategies, e.g. "ubuntu" for generating all Ubuntu configurations.
description: 'The strategy matrix to use for generating the configurations ("minimal", "all").'
required: false
type: string
default: "minimal"
outputs:
matrix:
description: "The generated strategy matrix."
value: ${{ jobs.generate-matrix.outputs.matrix }}
jobs:
generate-matrix:
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.generate.outputs.matrix }}
steps:
- name: Checkout repository
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: 3.13
- name: Generate strategy matrix
working-directory: .github/scripts/strategy-matrix
id: generate
env:
GENERATE_CONFIG: ${{ inputs.os != '' && format('--config={0}.json', inputs.os) || '' }}
GENERATE_OPTION: ${{ inputs.strategy_matrix == 'all' && '--all' || '' }}
run: ./generate.py ${{ env.GENERATE_OPTION }} ${{ env.GENERATE_CONFIG }} >> "${GITHUB_OUTPUT}"

69
.github/workflows/reusable-test.yml vendored Normal file
View File

@@ -0,0 +1,69 @@
name: Test rippled
on:
workflow_call:
inputs:
verify_voidstar:
description: "Whether to verify the presence of voidstar instrumentation."
required: true
type: boolean
run_tests:
description: "Whether to run unit tests"
required: true
type: boolean
runs_on:
description: Runner to run the job on as a JSON string
required: true
type: string
image:
description: "The image to run in (leave empty to run natively)"
required: true
type: string
config_name:
description: "The name of the configuration."
required: true
type: string
jobs:
test:
name: Test ${{ inputs.config_name }}
runs-on: ${{ fromJSON(inputs.runs_on) }}
container: ${{ inputs.image != '' && inputs.image || null }}
steps:
- name: Download rippled artifact
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: rippled-${{ inputs.config_name }}
- name: Make binary executable (Linux and macOS)
shell: bash
if: ${{ runner.os == 'Linux' || runner.os == 'macOS' }}
run: |
chmod +x ./rippled
- name: Check linking (Linux)
if: ${{ runner.os == 'Linux' }}
shell: bash
run: |
ldd ./rippled
if [ "$(ldd ./rippled | grep -E '(libstdc\+\+|libgcc)' | wc -l)" -eq 0 ]; then
echo 'The binary is statically linked.'
else
echo 'The binary is dynamically linked.'
exit 1
fi
- name: Verifying presence of instrumentation
if: ${{ inputs.verify_voidstar }}
shell: bash
run: |
./rippled --version | grep libvoidstar
- name: Test the binary
if: ${{ inputs.run_tests }}
shell: bash
run: |
./rippled --unittest --unittest-jobs $(nproc)
ctest -j $(nproc) --output-on-failure

91
.github/workflows/upload-conan-deps.yml vendored Normal file
View File

@@ -0,0 +1,91 @@
name: Upload Conan Dependencies
on:
schedule:
- cron: "0 3 * * 2-6"
workflow_dispatch:
inputs:
force_source_build:
description: "Force source build of all dependencies"
required: false
default: false
type: boolean
force_upload:
description: "Force upload of all dependencies"
required: false
default: false
type: boolean
pull_request:
branches: [develop]
paths:
# This allows testing changes to the upload workflow in a PR
- .github/workflows/upload-conan-deps.yml
push:
branches: [develop]
paths:
- .github/workflows/upload-conan-deps.yml
- .github/workflows/reusable-strategy-matrix.yml
- .github/actions/build-deps/action.yml
- .github/actions/setup-conan/action.yml
- ".github/scripts/strategy-matrix/**"
- conanfile.py
- conan.lock
env:
CONAN_REMOTE_NAME: xrplf
CONAN_REMOTE_URL: https://conan.ripplex.io
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
# Generate the strategy matrix to be used by the following job.
generate-matrix:
uses: ./.github/workflows/reusable-strategy-matrix.yml
with:
strategy_matrix: ${{ github.event_name == 'pull_request' && 'minimal' || 'all' }}
# Build and upload the dependencies for each configuration.
run-upload-conan-deps:
needs:
- generate-matrix
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.generate-matrix.outputs.matrix) }}
max-parallel: 10
runs-on: ${{ matrix.architecture.runner }}
container: ${{ contains(matrix.architecture.platform, 'linux') && format('ghcr.io/xrplf/ci/{0}-{1}:{2}-{3}-sha-{4}', matrix.os.distro_name, matrix.os.distro_version, matrix.os.compiler_name, matrix.os.compiler_version, matrix.os.image_sha) || null }}
steps:
- name: Cleanup workspace
if: ${{ runner.os == 'macOS' }}
uses: XRPLF/actions/.github/actions/cleanup-workspace@3f044c7478548e3c32ff68980eeb36ece02b364e
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
- name: Prepare runner
uses: XRPLF/actions/.github/actions/prepare-runner@638e0dc11ea230f91bd26622fb542116bb5254d5
with:
disable_ccache: false
- name: Setup Conan
uses: ./.github/actions/setup-conan
with:
conan_remote_name: ${{ env.CONAN_REMOTE_NAME }}
conan_remote_url: ${{ env.CONAN_REMOTE_URL }}
- name: Build dependencies
uses: ./.github/actions/build-deps
with:
build_dir: .build
build_type: ${{ matrix.build_type }}
force_build: ${{ github.event_name == 'schedule' || github.event.inputs.force_source_build == 'true' }}
- name: Log into Conan remote
if: ${{ github.repository_owner == 'XRPLF' && github.event_name != 'pull_request' }}
run: conan remote login ${{ env.CONAN_REMOTE_NAME }} "${{ secrets.CONAN_REMOTE_USERNAME }}" --password "${{ secrets.CONAN_REMOTE_PASSWORD }}"
- name: Upload Conan packages
if: ${{ github.repository_owner == 'XRPLF' && github.event_name != 'pull_request' && github.event_name != 'schedule' }}
env:
FORCE_OPTION: ${{ github.event.inputs.force_upload == 'true' && '--force' || '' }}
run: conan upload "*" --remote='${{ env.CONAN_REMOTE_NAME }}' --confirm ${{ env.FORCE_OPTION }}

View File

@@ -1,97 +0,0 @@
name: windows
on:
pull_request:
push:
# If the branches list is ever changed, be sure to change it on all
# build/test jobs (nix, macos, windows, instrumentation)
branches:
# Always build the package branches
- develop
- release
- master
# Branches that opt-in to running
- 'ci/**'
# https://docs.github.com/en/actions/using-jobs/using-concurrency
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
test:
strategy:
fail-fast: false
matrix:
version:
- generator: Visual Studio 17 2022
runs-on: windows-2022
configuration:
- type: Release
tests: true
- type: Debug
# Skip running unit tests on debug builds, because they
# take an unreasonable amount of time
tests: false
runtime: d
runs-on: ${{ matrix.version.runs-on }}
env:
build_dir: .build
steps:
- name: checkout
uses: actions/checkout@v4
- name: choose Python
uses: actions/setup-python@v5
with:
python-version: 3.9
- name: learn Python cache directory
id: pip-cache
shell: bash
run: |
python -m pip install --upgrade pip
echo "dir=$(pip cache dir)" | tee ${GITHUB_OUTPUT}
- name: restore Python cache directory
uses: actions/cache@v4
with:
path: ${{ steps.pip-cache.outputs.dir }}
key: ${{ runner.os }}-${{ hashFiles('.github/workflows/windows.yml') }}
- name: install Conan
run: pip install wheel 'conan<2'
- name: check environment
run: |
dir env:
$env:PATH -split ';'
python --version
conan --version
cmake --version
- name: configure Conan
shell: bash
run: |
conan profile new default --detect
conan profile update settings.compiler.cppstd=20 default
conan profile update \
settings.compiler.runtime=MT${{ matrix.configuration.runtime }} \
default
- name: build dependencies
uses: ./.github/actions/dependencies
env:
CONAN_URL: http://18.143.149.228:8081/artifactory/api/conan/conan-non-prod
CONAN_LOGIN_USERNAME_RIPPLE: ${{ secrets.CONAN_USERNAME }}
CONAN_PASSWORD_RIPPLE: ${{ secrets.CONAN_TOKEN }}
with:
configuration: ${{ matrix.configuration.type }}
- name: build
uses: ./.github/actions/build
with:
generator: '${{ matrix.version.generator }}'
configuration: ${{ matrix.configuration.type }}
# Hard code for now. Move to the matrix if varied options are needed
cmake-args: '-Dassert=TRUE -Dwerr=TRUE -Dreporting=OFF -Dunity=ON'
cmake-target: install
- name: test
shell: bash
if: ${{ matrix.configuration.tests }}
run: |
${build_dir}/${{ matrix.configuration.type }}/rippled --unittest \
--unittest-jobs $(nproc)

9
.gitignore vendored
View File

@@ -37,10 +37,9 @@ Release/*.*
*.gcov
# Levelization checking
Builds/levelization/results/rawincludes.txt
Builds/levelization/results/paths.txt
Builds/levelization/results/includes/
Builds/levelization/results/includedby/
.github/scripts/levelization/results/*
!.github/scripts/levelization/results/loops.txt
!.github/scripts/levelization/results/ordering.txt
# Ignore tmp directory.
tmp
@@ -111,4 +110,4 @@ bld.rippled/
.vscode
# Suggested in-tree build directory
/.build/
/.build*/

View File

@@ -1,6 +1,39 @@
# .pre-commit-config.yaml
# To run pre-commit hooks, first install pre-commit:
# - `pip install pre-commit==${PRE_COMMIT_VERSION}`
#
# Then, run the following command to install the git hook scripts:
# - `pre-commit install`
# You can run all configured hooks against all files with:
# - `pre-commit run --all-files`
# To manually run a specific hook, use:
# - `pre-commit run <hook_id> --all-files`
# To run the hooks against only the staged files, use:
# - `pre-commit run`
repos:
- repo: https://github.com/pre-commit/mirrors-clang-format
rev: v18.1.3
hooks:
- id: clang-format
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: 3e8a8703264a2f4a69428a0aa4dcb512790b2c8c # frozen: v6.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: mixed-line-ending
- id: check-merge-conflict
args: [--assume-in-merge]
- repo: https://github.com/pre-commit/mirrors-clang-format
rev: 7d85583be209cb547946c82fbe51f4bc5dd1d017 # frozen: v18.1.8
hooks:
- id: clang-format
args: [--style=file]
"types_or": [c++, c, proto]
- repo: https://github.com/rbubley/mirrors-prettier
rev: 5ba47274f9b181bce26a5150a725577f3c336011 # frozen: v3.6.2
hooks:
- id: prettier
exclude: |
(?x)^(
external/.*|
.github/scripts/levelization/results/.*\.txt|
conan\.lock
)$

1
.prettierignore Normal file
View File

@@ -0,0 +1 @@
external

View File

@@ -83,9 +83,17 @@ The [commandline](https://xrpl.org/docs/references/http-websocket-apis/api-conve
The `network_id` field was added in the `server_info` response in version 1.5.0 (2019), but it is not returned in [reporting mode](https://xrpl.org/rippled-server-modes.html#reporting-mode). However, use of reporting mode is now discouraged, in favor of using [Clio](https://github.com/XRPLF/clio) instead.
## XRP Ledger server version 2.5.0
As of 2025-04-04, version 2.5.0 is in development. You can use a pre-release version by building from source or [using the `nightly` package](https://xrpl.org/docs/infrastructure/installation/install-rippled-on-ubuntu).
### Additions and bugfixes in 2.5.0
- `channel_authorize`: If `signing_support` is not enabled in the config, the RPC is disabled.
## XRP Ledger server version 2.4.0
As of 2025-01-28, version 2.4.0 is in development. You can use a pre-release version by building from source or [using the `nightly` package](https://xrpl.org/docs/infrastructure/installation/install-rippled-on-ubuntu).
[Version 2.4.0](https://github.com/XRPLF/rippled/releases/tag/2.4.0) was released on March 4, 2025.
### Additions and bugfixes in 2.4.0

644
BUILD.md
View File

@@ -3,29 +3,29 @@
| These instructions assume you have a C++ development environment ready with Git, Python, Conan, CMake, and a C++ compiler. For help setting one up on Linux, macOS, or Windows, [see this guide](./docs/build/environment.md). |
> These instructions also assume a basic familiarity with Conan and CMake.
> If you are unfamiliar with Conan,
> you can read our [crash course](./docs/build/conan.md)
> or the official [Getting Started][3] walkthrough.
> If you are unfamiliar with Conan, you can read our
> [crash course](./docs/build/conan.md) or the official [Getting Started][3]
> walkthrough.
## Branches
For a stable release, choose the `master` branch or one of the [tagged
releases](https://github.com/ripple/rippled/releases).
```
```bash
git checkout master
```
For the latest release candidate, choose the `release` branch.
```
```bash
git checkout release
```
For the latest set of untested features, or to contribute, choose the `develop`
branch.
```
```bash
git checkout develop
```
@@ -33,176 +33,327 @@ git checkout develop
See [System Requirements](https://xrpl.org/system-requirements.html).
Building rippled generally requires git, Python, Conan, CMake, and a C++ compiler. Some guidance on setting up such a [C++ development environment can be found here](./docs/build/environment.md).
Building rippled generally requires git, Python, Conan, CMake, and a C++
compiler. Some guidance on setting up such a [C++ development environment can be
found here](./docs/build/environment.md).
- [Python 3.7](https://www.python.org/downloads/)
- [Conan 1.60](https://conan.io/downloads.html)[^1]
- [CMake 3.16](https://cmake.org/download/)
- [Python 3.11](https://www.python.org/downloads/), or higher
- [Conan 2.17](https://conan.io/downloads.html)[^1], or higher
- [CMake 3.22](https://cmake.org/download/)[^2], or higher
[^1]: It is possible to build with Conan 2.x,
but the instructions are significantly different,
which is why we are not recommending it yet.
Notably, the `conan profile update` command is removed in 2.x.
Profiles must be edited by hand.
[^1]:
It is possible to build with Conan 1.60+, but the instructions are
significantly different, which is why we are not recommending it.
[^2]:
CMake 4 is not yet supported by all dependencies required by this project.
If you are affected by this issue, follow [conan workaround for cmake
4](#workaround-for-cmake-4)
`rippled` is written in the C++20 dialect and includes the `<concepts>` header.
The [minimum compiler versions][2] required are:
| Compiler | Version |
|-------------|---------|
| GCC | 11 |
| Clang | 13 |
| Apple Clang | 13.1.6 |
| MSVC | 19.23 |
| Compiler | Version |
| ----------- | --------- |
| GCC | 12 |
| Clang | 16 |
| Apple Clang | 16 |
| MSVC | 19.44[^3] |
### Linux
The Ubuntu operating system has received the highest level of
quality assurance, testing, and support.
The Ubuntu Linux distribution has received the highest level of quality
assurance, testing, and support. We also support Red Hat and use Debian
internally.
Here are [sample instructions for setting up a C++ development environment on Linux](./docs/build/environment.md#linux).
Here are [sample instructions for setting up a C++ development environment on
Linux](./docs/build/environment.md#linux).
### Mac
Many rippled engineers use macOS for development.
Here are [sample instructions for setting up a C++ development environment on macOS](./docs/build/environment.md#macos).
Here are [sample instructions for setting up a C++ development environment on
macOS](./docs/build/environment.md#macos).
### Windows
Windows is not recommended for production use at this time.
Windows is used by some engineers for development only.
- Additionally, 32-bit Windows development is not supported.
[Boost]: https://www.boost.org/
[^3]: Windows is not recommended for production use.
## Steps
### Set Up Conan
After you have a [C++ development environment](./docs/build/environment.md) ready with Git, Python, Conan, CMake, and a C++ compiler, you may need to set up your Conan profile.
After you have a [C++ development environment](./docs/build/environment.md) ready with Git, Python,
Conan, CMake, and a C++ compiler, you may need to set up your Conan profile.
These instructions assume a basic familiarity with Conan and CMake.
These instructions assume a basic familiarity with Conan and CMake. If you are
unfamiliar with Conan, then please read [this crash course](./docs/build/conan.md) or the official
[Getting Started][3] walkthrough.
If you are unfamiliar with Conan, then please read [this crash course](./docs/build/conan.md) or the official [Getting Started][3] walkthrough.
#### Default profile
You'll need at least one Conan profile:
We recommend that you import the provided `conan/profiles/default` profile:
```
conan profile new default --detect
```
Update the compiler settings:
```
conan profile update settings.compiler.cppstd=20 default
```
Configure Conan (1.x only) to use recipe revisions:
```
conan config set general.revisions_enabled=1
```
**Linux** developers will commonly have a default Conan [profile][] that compiles
with GCC and links with libstdc++.
If you are linking with libstdc++ (see profile setting `compiler.libcxx`),
then you will need to choose the `libstdc++11` ABI:
```
conan profile update settings.compiler.libcxx=libstdc++11 default
```
Ensure inter-operability between `boost::string_view` and `std::string_view` types:
```
conan profile update 'conf.tools.build:cxxflags+=["-DBOOST_BEAST_USE_STD_STRING_VIEW"]' default
conan profile update 'env.CXXFLAGS="-DBOOST_BEAST_USE_STD_STRING_VIEW"' default
```bash
conan config install conan/profiles/ -tf $(conan config home)/profiles/
```
If you have other flags in the `conf.tools.build` or `env.CXXFLAGS` sections, make sure to retain the existing flags and append the new ones. You can check them with:
```
conan profile show default
You can check your Conan profile by running:
```bash
conan profile show
```
#### Custom profile
**Windows** developers may need to use the x64 native build tools.
An easy way to do that is to run the shortcut "x64 Native Tools Command
Prompt" for the version of Visual Studio that you have installed.
If the default profile does not work for you and you do not yet have a Conan
profile, you can create one by running:
Windows developers must also build `rippled` and its dependencies for the x64
architecture:
```
conan profile update settings.arch=x86_64 default
```
### Multiple compilers
When `/usr/bin/g++` exists on a platform, it is the default cpp compiler. This
default works for some users.
However, if this compiler cannot build rippled or its dependencies, then you can
install another compiler and set Conan and CMake to use it.
Update the `conf.tools.build:compiler_executables` setting in order to set the correct variables (`CMAKE_<LANG>_COMPILER`) in the
generated CMake toolchain file.
For example, on Ubuntu 20, you may have gcc at `/usr/bin/gcc` and g++ at `/usr/bin/g++`; if that is the case, you can select those compilers with:
```
conan profile update 'conf.tools.build:compiler_executables={"c": "/usr/bin/gcc", "cpp": "/usr/bin/g++"}' default
```bash
conan profile detect
```
Replace `/usr/bin/gcc` and `/usr/bin/g++` with paths to the desired compilers.
You may need to make changes to the profile to suit your environment. You can
refer to the provided `conan/profiles/default` profile for inspiration, and you
may also need to apply the required [tweaks](#conan-profile-tweaks) to this
default profile.
It should choose the compiler for dependencies as well,
but not all of them have a Conan recipe that respects this setting (yet).
For the rest, you can set these environment variables.
Replace `<path>` with paths to the desired compilers:
### Patched recipes
- `conan profile update env.CC=<path> default`
- `conan profile update env.CXX=<path> default`
The recipes in Conan Center occasionally need to be patched for compatibility
with the latest version of `rippled`. We maintain a fork of the Conan Center
[here](https://github.com/XRPLF/conan-center-index/) containing the patches.
Export our [Conan recipe for Snappy](./external/snappy).
It does not explicitly link the C++ standard library,
which allows you to statically link it with GCC, if you want.
To ensure our patched recipes are used, you must add our Conan remote at a
higher index than the default Conan Center remote, so it is consulted first. You
can do this by running:
```
# Conan 1.x
conan export external/snappy snappy/1.1.10@
# Conan 2.x
conan export --version 1.1.10 external/snappy
```
```bash
conan remote add --index 0 xrplf https://conan.ripplex.io
```
Export our [Conan recipe for RocksDB](./external/rocksdb).
It does not override paths to dependencies when building with Visual Studio.
Alternatively, you can pull the patched recipes into the repository and use them
locally:
```
# Conan 1.x
conan export external/rocksdb rocksdb/9.7.3@
# Conan 2.x
conan export --version 9.7.3 external/rocksdb
```
```bash
cd external
git init
git remote add origin git@github.com:XRPLF/conan-center-index.git
git sparse-checkout init
git sparse-checkout set recipes/snappy
git sparse-checkout add recipes/soci
git fetch origin master
git checkout master
conan export --version 1.1.10 recipes/snappy/all
conan export --version 4.0.3 recipes/soci/all
rm -rf .git
```
Export our [Conan recipe for SOCI](./external/soci).
It patches their CMake to correctly import its dependencies.
In the case we switch to a newer version of a dependency that still requires a
patch, it will be necessary for you to pull in the changes and re-export the
updated dependencies with the newer version. However, if we switch to a newer
version that no longer requires a patch, no action is required on your part, as
the new recipe will be automatically pulled from the official Conan Center.
```
# Conan 1.x
conan export external/soci soci/4.0.3@
# Conan 2.x
conan export --version 4.0.3 external/soci
```
> [!NOTE]
> You might need to add `--lockfile=""` to your `conan install` command
> to avoid automatic use of the existing `conan.lock` file when you run `conan export` manually on your machine
Export our [Conan recipe for NuDB](./external/nudb).
It fixes some source files to add missing `#include`s.
### Conan profile tweaks
#### Missing compiler version
```
# Conan 1.x
conan export external/nudb nudb/2.0.8@
# Conan 2.x
conan export --version 2.0.8 external/nudb
```
If you see an error similar to the following after running `conan profile show`:
```bash
ERROR: Invalid setting '17' is not a valid 'settings.compiler.version' value.
Possible values are ['5.0', '5.1', '6.0', '6.1', '7.0', '7.3', '8.0', '8.1',
'9.0', '9.1', '10.0', '11.0', '12.0', '13', '13.0', '13.1', '14', '14.0', '15',
'15.0', '16', '16.0']
Read "http://docs.conan.io/2/knowledge/faq.html#error-invalid-setting"
```
you need to amend the list of compiler versions in
`$(conan config home)/settings.yml`, by appending the required version number(s)
to the `version` array specific for your compiler. For example:
```yaml
apple-clang:
version:
[
"5.0",
"5.1",
"6.0",
"6.1",
"7.0",
"7.3",
"8.0",
"8.1",
"9.0",
"9.1",
"10.0",
"11.0",
"12.0",
"13",
"13.0",
"13.1",
"14",
"14.0",
"15",
"15.0",
"16",
"16.0",
"17",
"17.0",
]
```
#### Multiple compilers
If you have multiple compilers installed, make sure to select the one to use in
your default Conan configuration **before** running `conan profile detect`, by
setting the `CC` and `CXX` environment variables.
For example, if you are running MacOS and have [homebrew
LLVM@18](https://formulae.brew.sh/formula/llvm@18), and want to use it as a
compiler in the new Conan profile:
```bash
export CC=$(brew --prefix llvm@18)/bin/clang
export CXX=$(brew --prefix llvm@18)/bin/clang++
conan profile detect
```
You should also explicitly set the path to the compiler in the profile file,
which helps to avoid errors when `CC` and/or `CXX` are set and disagree with the
selected Conan profile. For example:
```text
[conf]
tools.build:compiler_executables={'c':'/usr/bin/gcc','cpp':'/usr/bin/g++'}
```
#### Multiple profiles
You can manage multiple Conan profiles in the directory
`$(conan config home)/profiles`, for example renaming `default` to a different
name and then creating a new `default` profile for a different compiler.
#### Select language
The default profile created by Conan will typically select different C++ dialect
than C++20 used by this project. You should set `20` in the profile line
starting with `compiler.cppstd=`. For example:
```bash
sed -i.bak -e 's|^compiler\.cppstd=.*$|compiler.cppstd=20|' $(conan config home)/profiles/default
```
#### Select standard library in Linux
**Linux** developers will commonly have a default Conan [profile][] that
compiles with GCC and links with libstdc++. If you are linking with libstdc++
(see profile setting `compiler.libcxx`), then you will need to choose the
`libstdc++11` ABI:
```bash
sed -i.bak -e 's|^compiler\.libcxx=.*$|compiler.libcxx=libstdc++11|' $(conan config home)/profiles/default
```
#### Select architecture and runtime in Windows
**Windows** developers may need to use the x64 native build tools. An easy way
to do that is to run the shortcut "x64 Native Tools Command Prompt" for the
version of Visual Studio that you have installed.
Windows developers must also build `rippled` and its dependencies for the x64
architecture:
```bash
sed -i.bak -e 's|^arch=.*$|arch=x86_64|' $(conan config home)/profiles/default
```
**Windows** developers also must select static runtime:
```bash
sed -i.bak -e 's|^compiler\.runtime=.*$|compiler.runtime=static|' $(conan config home)/profiles/default
```
#### Workaround for CMake 4
If your system CMake is version 4 rather than 3, you may have to configure Conan
profile to use CMake version 3 for dependencies, by adding the following two
lines to your profile:
```text
[tool_requires]
!cmake/*: cmake/[>=3 <4]
```
This will force Conan to download and use a locally cached CMake 3 version, and
is needed because some of the dependencies used by this project do not support
CMake 4.
#### Clang workaround for grpc
If your compiler is clang, version 19 or later, or apple-clang, version 17 or
later, you may encounter a compilation error while building the `grpc`
dependency:
```text
In file included from .../lib/promise/try_seq.h:26:
.../lib/promise/detail/basic_seq.h:499:38: error: a template argument list is expected after a name prefixed by the template keyword [-Wmissing-template-arg-list-after-template-kw]
499 | Traits::template CallSeqFactory(f_, *cur_, std::move(arg)));
| ^
```
The workaround for this error is to add two lines to profile:
```text
[conf]
tools.build:cxxflags=['-Wno-missing-template-arg-list-after-template-kw']
```
#### Workaround for gcc 12
If your compiler is gcc, version 12, and you have enabled `werr` option, you may
encounter a compilation error such as:
```text
/usr/include/c++/12/bits/char_traits.h:435:56: error: 'void* __builtin_memcpy(void*, const void*, long unsigned int)' accessing 9223372036854775810 or more bytes at offsets [2, 9223372036854775807] and 1 may overlap up to 9223372036854775813 bytes at offset -3 [-Werror=restrict]
435 | return static_cast<char_type*>(__builtin_memcpy(__s1, __s2, __n));
| ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
cc1plus: all warnings being treated as errors
```
The workaround for this error is to add two lines to your profile:
```text
[conf]
tools.build:cxxflags=['-Wno-restrict']
```
#### Workaround for clang 16
If your compiler is clang, version 16, you may encounter compilation error such
as:
```text
In file included from .../boost/beast/websocket/stream.hpp:2857:
.../boost/beast/websocket/impl/read.hpp:695:17: error: call to 'async_teardown' is ambiguous
async_teardown(impl.role, impl.stream(),
^~~~~~~~~~~~~~
```
The workaround for this error is to add two lines to your profile:
```text
[conf]
tools.build:cxxflags=['-DBOOST_ASIO_DISABLE_CONCEPTS']
```
### Build and Test
@@ -224,66 +375,65 @@ It fixes some source files to add missing `#include`s.
2. Use conan to generate CMake files for every configuration you want to build:
```
conan install .. --output-folder . --build missing --settings build_type=Release
conan install .. --output-folder . --build missing --settings build_type=Debug
```
```
conan install .. --output-folder . --build missing --settings build_type=Release
conan install .. --output-folder . --build missing --settings build_type=Debug
```
To build Debug, in the next step, be sure to set `-DCMAKE_BUILD_TYPE=Debug`
To build Debug, in the next step, be sure to set `-DCMAKE_BUILD_TYPE=Debug`
For a single-configuration generator, e.g. `Unix Makefiles` or `Ninja`,
you only need to run this command once.
For a multi-configuration generator, e.g. `Visual Studio`, you may want to
run it more than once.
For a single-configuration generator, e.g. `Unix Makefiles` or `Ninja`,
you only need to run this command once.
For a multi-configuration generator, e.g. `Visual Studio`, you may want to
run it more than once.
Each of these commands should also have a different `build_type` setting.
A second command with the same `build_type` setting will overwrite the files
generated by the first. You can pass the build type on the command line with
`--settings build_type=$BUILD_TYPE` or in the profile itself,
under the section `[settings]` with the key `build_type`.
Each of these commands should also have a different `build_type` setting.
A second command with the same `build_type` setting will overwrite the files
generated by the first. You can pass the build type on the command line with
`--settings build_type=$BUILD_TYPE` or in the profile itself,
under the section `[settings]` with the key `build_type`.
If you are using a Microsoft Visual C++ compiler,
then you will need to ensure consistency between the `build_type` setting
and the `compiler.runtime` setting.
If you are using a Microsoft Visual C++ compiler,
then you will need to ensure consistency between the `build_type` setting
and the `compiler.runtime` setting.
When `build_type` is `Release`, `compiler.runtime` should be `MT`.
When `build_type` is `Release`, `compiler.runtime` should be `MT`.
When `build_type` is `Debug`, `compiler.runtime` should be `MTd`.
When `build_type` is `Debug`, `compiler.runtime` should be `MTd`.
```
conan install .. --output-folder . --build missing --settings build_type=Release --settings compiler.runtime=MT
conan install .. --output-folder . --build missing --settings build_type=Debug --settings compiler.runtime=MTd
```
```
conan install .. --output-folder . --build missing --settings build_type=Release --settings compiler.runtime=MT
conan install .. --output-folder . --build missing --settings build_type=Debug --settings compiler.runtime=MTd
```
3. Configure CMake and pass the toolchain file generated by Conan, located at
`$OUTPUT_FOLDER/build/generators/conan_toolchain.cmake`.
Single-config generators:
Single-config generators:
Pass the CMake variable [`CMAKE_BUILD_TYPE`][build_type]
and make sure it matches the one of the `build_type` settings
you chose in the previous step.
Pass the CMake variable [`CMAKE_BUILD_TYPE`][build_type]
and make sure it matches the one of the `build_type` settings
you chose in the previous step.
For example, to build Debug, in the next command, replace "Release" with "Debug"
For example, to build Debug, in the next command, replace "Release" with "Debug"
```
cmake -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=Release -Dxrpld=ON -Dtests=ON ..
```
```
cmake -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=Release -Dxrpld=ON -Dtests=ON ..
```
Multi-config generators:
Multi-config generators:
```
cmake -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake -Dxrpld=ON -Dtests=ON ..
```
```
cmake -DCMAKE_TOOLCHAIN_FILE:FILEPATH=build/generators/conan_toolchain.cmake -Dxrpld=ON -Dtests=ON ..
```
**Note:** You can pass build options for `rippled` in this step.
**Note:** You can pass build options for `rippled` in this step.
5. Build `rippled`.
4. Build `rippled`.
For a single-configuration generator, it will build whatever configuration
you passed for `CMAKE_BUILD_TYPE`. For a multi-configuration generator,
you must pass the option `--config` to select the build configuration.
you passed for `CMAKE_BUILD_TYPE`. For a multi-configuration generator, you
must pass the option `--config` to select the build configuration.
Single-config generators:
@@ -298,24 +448,54 @@ It fixes some source files to add missing `#include`s.
cmake --build . --config Debug
```
6. Test rippled.
5. Test rippled.
Single-config generators:
```
./rippled --unittest
./rippled --unittest --unittest-jobs N
```
Multi-config generators:
```
./Release/rippled --unittest
./Debug/rippled --unittest
./Release/rippled --unittest --unittest-jobs N
./Debug/rippled --unittest --unittest-jobs N
```
The location of `rippled` in your build directory depends on your CMake
generator. Pass `--help` to see the rest of the command line options.
Replace the `--unittest-jobs` parameter N with the desired unit tests
concurrency. Recommended setting is half of the number of available CPU
cores.
The location of `rippled` binary in your build directory depends on your
CMake generator. Pass `--help` to see the rest of the command line options.
#### Conan lockfile
To achieve reproducible dependencies, we use [Conan lockfile](https://docs.conan.io/2/tutorial/versioning/lockfiles.html).
The `conan.lock` file in the repository contains a "snapshot" of the current dependencies.
It is implicitly used when running `conan` commands, you don't need to specify it.
You have to update this file every time you add a new dependency or change a revision or version of an existing dependency.
> [!NOTE]
> Conan uses local cache by default when creating a lockfile.
>
> To ensure, that lockfile creation works the same way on all developer machines, you should clear the local cache before creating a new lockfile.
To create a new lockfile, run the following commands in the repository root:
```bash
conan remove '*' --confirm
rm conan.lock
# This ensure that xrplf remote is the first to be consulted
conan remote add --force --index 0 xrplf https://conan.ripplex.io
conan lock create . -o '&:jemalloc=True' -o '&:rocksdb=True'
```
> [!NOTE]
> If some dependencies are exclusive for some OS, you may need to run the last command for them adding `--profile:all <PROFILE>`.
## Coverage report
@@ -356,7 +536,7 @@ variable in `cmake`. The specific command line used to run the `gcovr` tool will
displayed if the `CODE_COVERAGE_VERBOSE` variable is set.
By default, the code coverage tool runs parallel unit tests with `--unittest-jobs`
set to the number of available CPU cores. This may cause spurious test
set to the number of available CPU cores. This may cause spurious test
errors on Apple. Developers can override the number of unit test jobs with
the `coverage_test_parallelism` variable in `cmake`.
@@ -372,45 +552,62 @@ cmake --build . --target coverage
After the `coverage` target is completed, the generated coverage report will be
stored inside the build directory, as either of:
- file named `coverage.`_extension_ , with a suitable extension for the report format, or
- file named `coverage.`_extension_, with a suitable extension for the report format, or
- directory named `coverage`, with the `index.html` and other files inside, for the `html-details` or `html-nested` report formats.
## Options
| Option | Default Value | Description |
| --- | ---| ---|
| `assert` | OFF | Enable assertions.
| `coverage` | OFF | Prepare the coverage report. |
| `san` | N/A | Enable a sanitizer with Clang. Choices are `thread` and `address`. |
| `tests` | OFF | Build tests. |
| `unity` | ON | Configure a unity build. |
| `xrpld` | OFF | Build the xrpld (`rippled`) application, and not just the libxrpl library. |
| Option | Default Value | Description |
| ---------- | ------------- | -------------------------------------------------------------------------- |
| `assert` | OFF | Enable assertions. |
| `coverage` | OFF | Prepare the coverage report. |
| `san` | N/A | Enable a sanitizer with Clang. Choices are `thread` and `address`. |
| `tests` | OFF | Build tests. |
| `unity` | OFF | Configure a unity build. |
| `xrpld` | OFF | Build the xrpld (`rippled`) application, and not just the libxrpl library. |
| `werr` | OFF | Treat compilation warnings as errors |
| `wextra` | OFF | Enable additional compilation warnings |
[Unity builds][5] may be faster for the first build
(at the cost of much more memory) since they concatenate sources into fewer
translation units. Non-unity builds may be faster for incremental builds,
and can be helpful for detecting `#include` omissions.
## Troubleshooting
### Conan
After any updates or changes to dependencies, you may need to do the following:
1. Remove your build directory.
2. Remove the Conan cache:
2. Remove individual libraries from the Conan cache, e.g.
```bash
conan remove 'grpc/*'
```
rm -rf ~/.conan/data
**or**
Remove all libraries from Conan cache:
```bash
conan remove '*'
```
4. Re-run [conan install](#build-and-test).
3. Re-run [conan export](#patched-recipes) if needed.
4. [Regenerate lockfile](#conan-lockfile).
5. Re-run [conan install](#build-and-test).
### 'protobuf/port_def.inc' file not found
#### ERROR: Package not resolved
If `cmake --build .` results in an error due to a missing a protobuf file, then you might have generated CMake files for a different `build_type` than the `CMAKE_BUILD_TYPE` you passed to conan.
If you're seeing an error like `ERROR: Package 'snappy/1.1.10' not resolved: Unable to find 'snappy/1.1.10#968fef506ff261592ec30c574d4a7809%1756234314.246' in remotes.`,
please add `xrplf` remote or re-run `conan export` for [patched recipes](#patched-recipes).
### `protobuf/port_def.inc` file not found
If `cmake --build .` results in an error due to a missing a protobuf file, then
you might have generated CMake files for a different `build_type` than the
`CMAKE_BUILD_TYPE` you passed to Conan.
```
/rippled/.build/pb-xrpl.libpb/xrpl/proto/ripple.pb.h:10:10: fatal error: 'google/protobuf/port_def.inc' file not found
@@ -424,70 +621,21 @@ For example, if you want to build Debug:
1. For conan install, pass `--settings build_type=Debug`
2. For cmake, pass `-DCMAKE_BUILD_TYPE=Debug`
### no std::result_of
If your compiler version is recent enough to have removed `std::result_of` as
part of C++20, e.g. Apple Clang 15.0, then you might need to add a preprocessor
definition to your build.
```
conan profile update 'options.boost:extra_b2_flags="define=BOOST_ASIO_HAS_STD_INVOKE_RESULT"' default
conan profile update 'env.CFLAGS="-DBOOST_ASIO_HAS_STD_INVOKE_RESULT"' default
conan profile update 'env.CXXFLAGS="-DBOOST_ASIO_HAS_STD_INVOKE_RESULT"' default
conan profile update 'conf.tools.build:cflags+=["-DBOOST_ASIO_HAS_STD_INVOKE_RESULT"]' default
conan profile update 'conf.tools.build:cxxflags+=["-DBOOST_ASIO_HAS_STD_INVOKE_RESULT"]' default
```
### call to 'async_teardown' is ambiguous
If you are compiling with an early version of Clang 16, then you might hit
a [regression][6] when compiling C++20 that manifests as an [error in a Boost
header][7]. You can workaround it by adding this preprocessor definition:
```
conan profile update 'env.CXXFLAGS="-DBOOST_ASIO_DISABLE_CONCEPTS"' default
conan profile update 'conf.tools.build:cxxflags+=["-DBOOST_ASIO_DISABLE_CONCEPTS"]' default
```
### recompile with -fPIC
If you get a linker error suggesting that you recompile Boost with
position-independent code, such as:
```
/usr/bin/ld.gold: error: /home/username/.conan/data/boost/1.77.0/_/_/package/.../lib/libboost_container.a(alloc_lib.o):
requires unsupported dynamic reloc 11; recompile with -fPIC
```
Conan most likely downloaded a bad binary distribution of the dependency.
This seems to be a [bug][1] in Conan just for Boost 1.77.0 compiled with GCC
for Linux. The solution is to build the dependency locally by passing
`--build boost` when calling `conan install`.
```
conan install --build boost ...
```
## Add a Dependency
If you want to experiment with a new package, follow these steps:
1. Search for the package on [Conan Center](https://conan.io/center/).
2. Modify [`conanfile.py`](./conanfile.py):
- Add a version of the package to the `requires` property.
- Change any default options for the package by adding them to the
`default_options` property (with syntax `'$package:$option': $value`).
- Add a version of the package to the `requires` property.
- Change any default options for the package by adding them to the
`default_options` property (with syntax `'$package:$option': $value`).
3. Modify [`CMakeLists.txt`](./CMakeLists.txt):
- Add a call to `find_package($package REQUIRED)`.
- Link a library from the package to the target `ripple_libs`
(search for the existing call to `target_link_libraries(ripple_libs INTERFACE ...)`).
- Add a call to `find_package($package REQUIRED)`.
- Link a library from the package to the target `ripple_libs`
(search for the existing call to `target_link_libraries(ripple_libs INTERFACE ...)`).
4. Start coding! Don't forget to include whatever headers you need from the package.
[1]: https://github.com/conan-io/conan-center-index/issues/13168
[2]: https://en.cppreference.com/w/cpp/compiler_support/20
[3]: https://docs.conan.io/en/latest/getting_started.html

View File

@@ -90,6 +90,11 @@ set_target_properties(OpenSSL::SSL PROPERTIES
INTERFACE_COMPILE_DEFINITIONS OPENSSL_NO_SSL2
)
set(SECP256K1_INSTALL TRUE)
set(SECP256K1_BUILD_BENCHMARK FALSE)
set(SECP256K1_BUILD_TESTS FALSE)
set(SECP256K1_BUILD_EXHAUSTIVE_TESTS FALSE)
set(SECP256K1_BUILD_CTIME_TESTS FALSE)
set(SECP256K1_BUILD_EXAMPLES FALSE)
add_subdirectory(external/secp256k1)
add_library(secp256k1::secp256k1 ALIAS secp256k1)
add_subdirectory(external/ed25519-donna)
@@ -144,3 +149,8 @@ set(PROJECT_EXPORT_SET RippleExports)
include(RippledCore)
include(RippledInstall)
include(RippledValidatorKeys)
if(tests)
include(CTest)
add_subdirectory(src/tests/libxrpl)
endif()

View File

@@ -8,13 +8,12 @@ We assume you are familiar with the general practice of [making
contributions on GitHub][contrib]. This file includes only special
instructions specific to this project.
## Before you start
The following branches exist in the main project repository:
- `develop`: The latest set of unreleased features, and the most common
starting point for contributions.
starting point for contributions.
- `release`: The latest beta release or release candidate.
- `master`: The latest stable release.
- `gh-pages`: The documentation for this project, built by Doxygen.
@@ -27,18 +26,18 @@ In general, external contributions should be developed in your personal
[fork][forking]. Contributions from developers with write permissions
should be done in [the main repository][rippled] in a branch with
a permitted prefix. Permitted prefixes are:
* XLS-[a-zA-Z0-9]+/.+
* e.g. XLS-0033d/mpt-clarify-STEitherAmount
* [GitHub username]/.+
* e.g. JoelKatz/fix-rpc-webhook-queue
* [Organization name]/.+
* e.g. ripple/antithesis
Regardless of where the branch is created, please open a *draft* pull
- XLS-[a-zA-Z0-9]+/.+
- e.g. XLS-0033d/mpt-clarify-STEitherAmount
- [GitHub username]/.+
- e.g. JoelKatz/fix-rpc-webhook-queue
- [Organization name]/.+
- e.g. ripple/antithesis
Regardless of where the branch is created, please open a _draft_ pull
request as soon as possible after pushing the branch to Github, to
increase visibility, and ease feedback during the development process.
## Major contributions
If your contribution is a major feature or breaking change, then you
@@ -55,8 +54,8 @@ responsibility of the XLS author to update the draft to match the final
implementation when its corresponding pull request is merged, unless the
author delegates that responsibility to others.
## Before making a pull request
(Or marking a draft pull request as ready.)
Changes that alter transaction processing must be guarded by an
@@ -73,30 +72,32 @@ automatic test run by `rippled --unittest`.
Otherwise, it must be a manual test.
If you create new source files, they must be organized as follows:
* If the files are in any of the `libxrpl` modules, the headers (`.h`) must go
- If the files are in any of the `libxrpl` modules, the headers (`.h`) must go
under `include/xrpl`, and source (`.cpp`) files must go under
`src/libxrpl`.
* All other non-test files must go under `src/xrpld`.
* All test source files must go under `src/test`.
- All other non-test files must go under `src/xrpld`.
- All test source files must go under `src/test`.
The source must be formatted according to the style guide below.
Header includes must be [levelized](./Builds/levelization).
Header includes must be [levelized](.github/scripts/levelization).
Changes should be usually squashed down into a single commit.
Some larger or more complicated change sets make more sense,
and are easier to review if organized into multiple logical commits.
Either way, all commits should fit the following criteria:
* Changes should be presented in a single commit or a logical
- Changes should be presented in a single commit or a logical
sequence of commits.
Specifically, chronological commits that simply
reflect the history of how the author implemented
the change, "warts and all", are not useful to
reviewers.
* Every commit should have a [good message](#good-commit-messages).
- Every commit should have a [good message](#good-commit-messages).
to explain a specific aspects of the change.
* Every commit should be signed.
* Every commit should be well-formed (builds successfully,
- Every commit should be signed.
- Every commit should be well-formed (builds successfully,
unit tests passing), as this helps to resolve merge
conflicts, and makes it easier to use `git bisect`
to find bugs.
@@ -108,13 +109,14 @@ Refer to
for general rules on writing a good commit message.
tl;dr
> 1. Separate subject from body with a blank line.
> 2. Limit the subject line to 50 characters.
> * [...]shoot for 50 characters, but consider 72 the hard limit.
> - [...]shoot for 50 characters, but consider 72 the hard limit.
> 3. Capitalize the subject line.
> 4. Do not end the subject line with a period.
> 5. Use the imperative mood in the subject line.
> * A properly formed Git commit subject line should always be able
> - A properly formed Git commit subject line should always be able
> to complete the following sentence: "If applied, this commit will
> _your subject line here_".
> 6. Wrap the body at 72 characters.
@@ -122,16 +124,17 @@ tl;dr
In addition to those guidelines, please add one of the following
prefixes to the subject line if appropriate.
* `fix:` - The primary purpose is to fix an existing bug.
* `perf:` - The primary purpose is performance improvements.
* `refactor:` - The changes refactor code without affecting
- `fix:` - The primary purpose is to fix an existing bug.
- `perf:` - The primary purpose is performance improvements.
- `refactor:` - The changes refactor code without affecting
functionality.
* `test:` - The changes _only_ affect unit tests.
* `docs:` - The changes _only_ affect documentation. This can
- `test:` - The changes _only_ affect unit tests.
- `docs:` - The changes _only_ affect documentation. This can
include code comments in addition to `.md` files like this one.
* `build:` - The changes _only_ affect the build process,
- `build:` - The changes _only_ affect the build process,
including CMake and/or Conan settings.
* `chore:` - Other tasks that don't affect the binary, but don't fit
- `chore:` - Other tasks that don't affect the binary, but don't fit
any of the other cases. e.g. formatting, git settings, updating
Github Actions jobs.
@@ -143,9 +146,10 @@ unit tests for Feature X (#1234)`.
In general, pull requests use `develop` as the base branch.
The exceptions are
* Fixes and improvements to a release candidate use `release` as the
- Fixes and improvements to a release candidate use `release` as the
base.
* Hotfixes use `master` as the base.
- Hotfixes use `master` as the base.
If your changes are not quite ready, but you want to make it easily available
for preliminary examination or review, you can create a "Draft" pull request.
@@ -182,11 +186,11 @@ meets a few criteria:
2. All CI checks must be complete and passed. (One-off failures may
be acceptable if they are related to a known issue.)
3. The PR must have a [good commit message](#good-commit-messages).
* If the PR started with a good commit message, and it doesn't
- If the PR started with a good commit message, and it doesn't
need to be updated, the author can indicate that in a comment.
* Any contributor, preferably the author, can leave a comment
- Any contributor, preferably the author, can leave a comment
suggesting a commit message.
* If the author squashes and rebases the code in preparation for
- If the author squashes and rebases the code in preparation for
merge, they should also ensure the commit message(s) are updated
as well.
4. The PR branch must be up to date with the base branch (usually
@@ -208,7 +212,6 @@ This is a non-exhaustive list of recommended style guidelines. These are
not always strictly enforced and serve as a way to keep the codebase
coherent rather than a set of _thou shalt not_ commandments.
## Formatting
All code must conform to `clang-format` version 18,
@@ -237,6 +240,7 @@ To download the patch file:
5. Commit and push.
You can install a pre-commit hook to automatically run `clang-format` before every commit:
```
pip3 install pre-commit
pre-commit install
@@ -267,49 +271,51 @@ locations, where the reporting of contract violations on the Antithesis
platform is either not possible or not useful.
For this reason:
* The locations where `assert` or `assert(false)` contracts should continue to be used:
* `constexpr` functions
* unit tests i.e. files under `src/test`
* unit tests-related modules (files under `beast/test` and `beast/unit_test`)
* Outside of the listed locations, do not use `assert`; use `XRPL_ASSERT` instead,
- The locations where `assert` or `assert(false)` contracts should continue to be used:
- `constexpr` functions
- unit tests i.e. files under `src/test`
- unit tests-related modules (files under `beast/test` and `beast/unit_test`)
- Outside of the listed locations, do not use `assert`; use `XRPL_ASSERT` instead,
giving it unique name, with the short description of the contract.
* Outside of the listed locations, do not use `assert(false)`; use
- Outside of the listed locations, do not use `assert(false)`; use
`UNREACHABLE` instead, giving it unique name, with the description of the
condition being violated
* The contract name should start with a full name (including scope) of the
function, optionally a named lambda, followed by a colon ` : ` and a brief
- The contract name should start with a full name (including scope) of the
function, optionally a named lambda, followed by a colon `:` and a brief
(typically at most five words) description. `UNREACHABLE` contracts
can use slightly longer descriptions. If there are multiple overloads of the
function, use common sense to balance both brevity and unambiguity of the
function name. NOTE: the purpose of name is to provide stable means of
unique identification of every contract; for this reason try to avoid elements
which can change in some obvious refactors or when reinforcing the condition.
* Contract description typically (except for `UNREACHABLE`) should describe the
- Contract description typically (except for `UNREACHABLE`) should describe the
_expected_ condition, as in "I assert that _expected_ is true".
* Contract description for `UNREACHABLE` should describe the _unexpected_
- Contract description for `UNREACHABLE` should describe the _unexpected_
situation which caused the line to have been reached.
* Example good name for an
- Example good name for an
`UNREACHABLE` macro `"Json::operator==(Value, Value) : invalid type"`; example
good name for an `XRPL_ASSERT` macro `"Json::Value::asCString : valid type"`.
* Example **bad** name
- Example **bad** name
`"RFC1751::insert(char* s, int x, int start, int length) : length is greater than or equal zero"`
(missing namespace, unnecessary full function signature, description too verbose).
Good name: `"ripple::RFC1751::insert : minimum length"`.
* In **few** well-justified cases a non-standard name can be used, in which case a
- In **few** well-justified cases a non-standard name can be used, in which case a
comment should be placed to explain the rationale (example in `contract.cpp`)
* Do **not** rename a contract without a good reason (e.g. the name no longer
- Do **not** rename a contract without a good reason (e.g. the name no longer
reflects the location or the condition being checked)
* Do not use `std::unreachable`
* Do not put contracts where they can be violated by an external condition
- Do not use `std::unreachable`
- Do not put contracts where they can be violated by an external condition
(e.g. timing, data payload before mandatory validation etc.) as this creates
bogus bug reports (and causes crashes of Debug builds)
## Unit Tests
To execute all unit tests:
```rippled --unittest --unittest-jobs=<number of cores>```
`rippled --unittest --unittest-jobs=<number of cores>`
(Note: Using multiple cores on a Mac M1 can cause spurious test failures. The
(Note: Using multiple cores on a Mac M1 can cause spurious test failures. The
cause is still under investigation. If you observe this problem, try specifying fewer jobs.)
To run a specific set of test suites:
@@ -317,10 +323,11 @@ To run a specific set of test suites:
```
rippled --unittest TestSuiteName
```
Note: In this example, all tests with prefix `TestSuiteName` will be run, so if
`TestSuiteName1` and `TestSuiteName2` both exist, then both tests will run.
Alternatively, if the unit test name finds an exact match, it will stop
doing partial matches, i.e. if a unit test with a title of `TestSuiteName`
`TestSuiteName1` and `TestSuiteName2` both exist, then both tests will run.
Alternatively, if the unit test name finds an exact match, it will stop
doing partial matches, i.e. if a unit test with a title of `TestSuiteName`
exists, then no other unit test will be executed, apart from `TestSuiteName`.
## Avoid
@@ -336,7 +343,6 @@ exists, then no other unit test will be executed, apart from `TestSuiteName`.
explanatory comments.
8. Importing new libraries unless there is a very good reason to do so.
## Seek to
9. Extend functionality of existing code rather than creating new code.
@@ -351,14 +357,12 @@ exists, then no other unit test will be executed, apart from `TestSuiteName`.
14. Provide as many comments as you feel that a competent programmer
would need to understand what your code does.
# Maintainers
Maintainers are ecosystem participants with elevated access to the repository.
They are able to push new code, make decisions on when a release should be
made, etc.
## Adding and removing
New maintainers can be proposed by two existing maintainers, subject to a vote
@@ -373,47 +377,41 @@ A minimum of 60% agreement and 50% participation are required.
The XRP Ledger Foundation will have the ability, for cause, to remove an
existing maintainer without a vote.
## Current Maintainers
Maintainers are users with maintain or admin access to the repo.
* [bthomee](https://github.com/bthomee) (Ripple)
* [intelliot](https://github.com/intelliot) (Ripple)
* [JoelKatz](https://github.com/JoelKatz) (Ripple)
* [nixer89](https://github.com/nixer89) (XRP Ledger Foundation)
* [RichardAH](https://github.com/RichardAH) (XRP Ledger Foundation)
* [Silkjaer](https://github.com/Silkjaer) (XRP Ledger Foundation)
* [WietseWind](https://github.com/WietseWind) (XRPL Labs + XRP Ledger Foundation)
* [ximinez](https://github.com/ximinez) (Ripple)
- [bthomee](https://github.com/bthomee) (Ripple)
- [intelliot](https://github.com/intelliot) (Ripple)
- [JoelKatz](https://github.com/JoelKatz) (Ripple)
- [legleux](https://github.com/legleux) (Ripple)
- [mankins](https://github.com/mankins) (XRP Ledger Foundation)
- [WietseWind](https://github.com/WietseWind) (XRPL Labs + XRP Ledger Foundation)
- [ximinez](https://github.com/ximinez) (Ripple)
## Current Code Reviewers
Code Reviewers are developers who have the ability to review, approve, and
in some cases merge source code changes.
* [HowardHinnant](https://github.com/HowardHinnant) (Ripple)
* [scottschurr](https://github.com/scottschurr) (Ripple)
* [seelabs](https://github.com/seelabs) (Ripple)
* [Ed Hennis](https://github.com/ximinez) (Ripple)
* [mvadari](https://github.com/mvadari) (Ripple)
* [thejohnfreeman](https://github.com/thejohnfreeman) (Ripple)
* [Bronek](https://github.com/Bronek) (Ripple)
* [manojsdoshi](https://github.com/manojsdoshi) (Ripple)
* [godexsoft](https://github.com/godexsoft) (Ripple)
* [mDuo13](https://github.com/mDuo13) (Ripple)
* [ckniffen](https://github.com/ckniffen) (Ripple)
* [arihantkothari](https://github.com/arihantkothari) (Ripple)
* [pwang200](https://github.com/pwang200) (Ripple)
* [sophiax851](https://github.com/sophiax851) (Ripple)
* [shawnxie999](https://github.com/shawnxie999) (Ripple)
* [gregtatcam](https://github.com/gregtatcam) (Ripple)
* [mtrippled](https://github.com/mtrippled) (Ripple)
* [ckeshava](https://github.com/ckeshava) (Ripple)
* [nbougalis](https://github.com/nbougalis) None
* [RichardAH](https://github.com/RichardAH) (XRPL Labs + XRP Ledger Foundation)
* [dangell7](https://github.com/dangell7) (XRPL Labs)
- [a1q123456](https://github.com/a1q123456) (Ripple)
- [Bronek](https://github.com/Bronek) (Ripple)
- [bthomee](https://github.com/bthomee) (Ripple)
- [ckeshava](https://github.com/ckeshava) (Ripple)
- [dangell7](https://github.com/dangell7) (XRPL Labs)
- [godexsoft](https://github.com/godexsoft) (Ripple)
- [gregtatcam](https://github.com/gregtatcam) (Ripple)
- [kuznetsss](https://github.com/kuznetsss) (Ripple)
- [lmaisons](https://github.com/lmaisons) (Ripple)
- [mathbunnyru](https://github.com/mathbunnyru) (Ripple)
- [mvadari](https://github.com/mvadari) (Ripple)
- [oleks-rip](https://github.com/oleks-rip) (Ripple)
- [PeterChen13579](https://github.com/PeterChen13579) (Ripple)
- [pwang200](https://github.com/pwang200) (Ripple)
- [q73zhao](https://github.com/q73zhao) (Ripple)
- [shawnxie999](https://github.com/shawnxie999) (Ripple)
- [Tapanito](https://github.com/Tapanito) (Ripple)
- [ximinez](https://github.com/ximinez) (Ripple)
Developers not on this list are able and encouraged to submit feedback
on pending code changes (open pull requests).
@@ -423,6 +421,7 @@ on pending code changes (open pull requests).
These instructions assume you have your git upstream remotes configured
to avoid accidental pushes to the main repo, and a remote group
specifying both of them. e.g.
```
$ git remote -v | grep upstream
upstream https://github.com/XRPLF/rippled.git (fetch)
@@ -437,6 +436,7 @@ upstream upstream-push
You can use the [setup-upstreams] script to set this up.
It also assumes you have a default gpg signing key set up in git. e.g.
```
$ git config user.signingkey
968479A1AFF927E37D1A566BB5690EEEBB952194
@@ -461,8 +461,8 @@ the suggested commit message, or modify it as needed.
#### Slightly more complicated pull requests
Some pull requests need to be pushed to `develop` as more than one
commit. A PR author may *request* to merge as separate commits. They
must *justify* why separate commits are needed, and *specify* how they
commit. A PR author may _request_ to merge as separate commits. They
must _justify_ why separate commits are needed, and _specify_ how they
would like the commits to be merged. If you disagree with the author,
discuss it with them directly.
@@ -471,20 +471,22 @@ fast forward only merge (`--ff-only`) on the command line and push to
`develop`.
Some examples of when separate commits are worthwhile are:
1. PRs where source files are reorganized in multiple steps.
2. PRs where the commits are mostly independent and *could* be separate
2. PRs where the commits are mostly independent and _could_ be separate
PRs, but are pulled together into one PR under a commit theme or
issue.
3. PRs that are complicated enough that `git bisect` would not be much
help if it determined this PR introduced a problem.
Either way, check that:
* The commits are based on the current tip of `develop`.
* The commits are clean: No merge commits (except when reverse
- The commits are based on the current tip of `develop`.
- The commits are clean: No merge commits (except when reverse
merging), no "[FOLD]" or "fixup!" messages.
* All commits are signed. If the commits are not signed by the author, use
- All commits are signed. If the commits are not signed by the author, use
`git commit --amend -S` to sign them yourself.
* At least one (but preferably all) of the commits has the PR number
- At least one (but preferably all) of the commits has the PR number
in the commit message.
The "Create a merge commit" and "Rebase and merge" options should be
@@ -502,13 +504,13 @@ Rippled uses a linear workflow model that can be summarized as:
1. In between releases, developers work against the `develop` branch.
2. Periodically, a maintainer will build and tag a beta version from
`develop`, which is pushed to `release`.
* Betas are usually released every two to three weeks, though that
- Betas are usually released every two to three weeks, though that
schedule can vary depending on progress, availability, and other
factors.
3. When the changes in `develop` are considered stable and mature enough
to be ready to release, a release candidate (RC) is built and tagged
from `develop`, and merged to `release`.
* Further development for that release (primarily fixes) then
- Further development for that release (primarily fixes) then
continues against `release`, while other development continues on
`develop`. Effectively, `release` is forked from `develop`. Changes
to `release` must be reverse merged to `develop`.
@@ -543,6 +545,7 @@ Rippled uses a linear workflow model that can be summarized as:
the version number, etc.
The workflow may look something like:
```
git fetch --multiple upstreams user1 user2 user3 [...]
git checkout -B release-next --no-track upstream/develop
@@ -581,8 +584,9 @@ This includes, betas, and the first release candidate (RC).
1. If you didn't create one [preparing the `develop`
branch](#preparing-the-develop-branch), Ensure there is no old
`release-next` branch hanging around. Then make a `release-next`
`release-next` branch hanging around. Then make a `release-next`
branch that only changes the version number. e.g.
```
git fetch upstreams
@@ -603,25 +607,30 @@ git push upstream-push
git fetch upstreams
git branch --set-upstream-to=upstream/release-next
```
You can also use the [update-version] script.
2. Create a Pull Request for `release-next` with **`develop`** as
the base branch.
1. Use the title "[TRIVIAL] Set version to X.X.X-bX".
2. Instead of the default description template, use the following:
You can also use the [update-version] script. 2. Create a Pull Request for `release-next` with **`develop`** as
the base branch.
1. Use the title "[TRIVIAL] Set version to X.X.X-bX".
2. Instead of the default description template, use the following:
```
## High Level Overview of Change
This PR only changes the version number. It will be merged as
soon as Github CI actions successfully complete.
```
3. Wait for CI to successfully complete, and get someone to approve
the PR. (It is safe to ignore known CI issues.)
4. Push the updated `develop` branch using your `release-next`
branch. **Do not use the Github UI. It's important to preserve
commit IDs.**
```
git push upstream-push release-next:develop
```
5. In the unlikely event that the push fails because someone has merged
something else in the meantime, rebase your branch onto the updated
`develop` branch, push again, and go back to step 3.
@@ -630,22 +639,25 @@ git push upstream-push release-next:develop
7. Once this is done, forward progress on `develop` can continue
(other PRs may be merged).
8. Now create a Pull Request for `release-next` with **`release`** as
the base branch. Instead of the default template, reuse and update
the base branch. Instead of the default template, reuse and update
the message from the previous release. Include the following verbiage
somewhere in the description:
```
The base branch is `release`. [All releases (including
betas)](https://github.com/XRPLF/rippled/blob/develop/CONTRIBUTING.md#before-you-start)
go in `release`. This PR branch will be pushed directly to `release` (not
squashed or rebased, and not using the GitHub UI).
```
7. Sign-offs for the three platforms (Linux, Mac, Windows) usually occur
offline, but at least one approval will be needed on the PR.
* If issues are discovered during testing, simply abandon the
release. It's easy to start a new release, it should be easy to
- If issues are discovered during testing, simply abandon the
release. It's easy to start a new release, it should be easy to
abandon one. **DO NOT REUSE THE VERSION NUMBER.** e.g. If you
abandon 2.4.0-b1, the next attempt will be 2.4.0-b2.
8. Once everything is ready to go, push to `release`.
```
git fetch upstreams
@@ -666,23 +678,28 @@ git log -1 --oneline
# Other branches, including some from upstream-push, may also be
# present.
```
9. Tag the release, too.
```
git tag <version number>
git push upstream-push <version number>
```
10. Delete the `release-next` branch on the repo. Use the Github UI or:
```
git push --delete upstream-push release-next
```
11. Finally [create a new release on
Github](https://github.com/XRPLF/rippled/releases).
#### Release candidates after the first
Once the first release candidate is [merged into
release](#making-the-release), then `release` and `develop` *are allowed
to diverge*.
release](#making-the-release), then `release` and `develop` _are allowed
to diverge_.
If a bug or issue is discovered in a version that has a release
candidate being tested, any fix and new version will need to be applied
@@ -690,7 +707,7 @@ against `release`, then reverse-merged to `develop`. This helps keep git
history as linear as possible.
A `release-next` branch will be created from `release`, and any further
work for that release must be based on `release-next`. Specifically,
work for that release must be based on `release-next`. Specifically,
PRs must use `release-next` as the base, and those PRs will be merged
directly to `release-next` when approved. Changes should be restricted
to bug fixes, but other changes may be necessary from time to time.
@@ -713,17 +730,21 @@ Once the RC is merged and tagged, it needs to be reverse merged into
1. Create a branch, based on `upstream/develop`.
The branch name is not important, but could include "mergeNNNrcN".
E.g. For release A.B.C-rcD, use `mergeABCrcD`.
```
git fetch upstreams
git checkout --no-track -b mergeABCrcD upstream/develop
```
2. Merge `release` into your branch.
```
# I like the "--edit --log --verbose" parameters, but they are
# not required.
git merge upstream/release
```
3. `BuildInfo.cpp` will have a conflict with the version number.
Resolve it with the version from `develop` - the higher version.
4. Push your branch to your repo (or `upstream` if you have permission),
@@ -731,22 +752,27 @@ git merge upstream/release
simply indicate that this is a merge of the RC. The "Context" should
summarize the changes from the RC. Include the following text
prominently:
```
This PR must be merged manually using a push. Do not use the Github UI.
```
5. Depending on the complexity of the changes, and/or merge conflicts,
the PR may need a thorough review, or just a sign-off that the
merge was done correctly.
6. If `develop` is updated before this PR is merged, do not merge
`develop` back into your branch. Instead rebase preserving merges,
or do the merge again. (See also the `rerere` git config setting.)
```
git rebase --rebase-merges upstream/develop
# OR
git reset --hard upstream/develop
git merge upstream/release
```
7. When the PR is ready, push it to `develop`.
```
git fetch upstreams
@@ -757,8 +783,8 @@ git push upstream-push mergeABCrcD:develop
git fetch upstreams
```
Development on `develop` can proceed as normal.
Development on `develop` can proceed as normal.
#### Final releases
@@ -773,7 +799,7 @@ internally as if they were RCs (at minimum, ensuring unit tests pass,
and the app starts, syncs, and stops cleanly across all three
platforms.)
*If in doubt, make an RC first.*
_If in doubt, make an RC first._
The process for building a final release is very similar to [the process
for building a beta](#making-the-release), except the code will be
@@ -785,20 +811,23 @@ moving from `release` to `master` instead of from `develop` to
number. As above, or using the
[update-version] script.
2. Create a Pull Request for `master-next` with **`master`** as
the base branch. Instead of the default template, reuse and update
the base branch. Instead of the default template, reuse and update
the message from the previous final release. Include the following verbiage
somewhere in the description:
```
The base branch is `master`. This PR branch will be pushed directly to
`release` and `master` (not squashed or rebased, and not using the
GitHub UI).
```
7. Sign-offs for the three platforms (Linux, Mac, Windows) usually occur
offline, but at least one approval will be needed on the PR.
* If issues are discovered during testing, close the PR, delete
- If issues are discovered during testing, close the PR, delete
`master-next`, and move development back to `release`, [issuing
more RCs as necessary](#release-candidates-after-the-first)
8. Once everything is ready to go, push to `release` and `master`.
```
git fetch upstreams
@@ -821,15 +850,20 @@ git log -1 --oneline
# Other branches, including some from upstream-push, may also be
# present.
```
9. Tag the release, too.
```
git tag <version number>
git push upstream-push <version number>
```
10. Delete the `master-next` branch on the repo. Use the Github UI or:
```
git push --delete upstream-push master-next
```
11. [Create a new release on
Github](https://github.com/XRPLF/rippled/releases). Be sure that
"Set as the latest release" is checked.
@@ -856,11 +890,13 @@ any branch. When it's ready to merge, jump to step 3 using your branch
instead of `master-next`.
1. Create a `master-next` branch from `master`.
```
git checkout --no-track -b master-next upstream/master
git push upstream-push
git fetch upstreams
```
2. Open any PRs for the pending hotfix using `master-next` as the base,
so they can be merged directly in to it. Unlike `develop`, though,
`master-next` can be thrown away and recreated if necessary.
@@ -868,19 +904,22 @@ git fetch upstreams
steps as above, or use the
[update-version] script.
4. Create a Pull Request for `master-next` with **`master`** as
the base branch. Instead of the default template, reuse and update
the base branch. Instead of the default template, reuse and update
the message from the previous final release. Include the following verbiage
somewhere in the description:
```
The base branch is `master`. This PR branch will be pushed directly to
`master` (not squashed or rebased, and not using the GitHub UI).
```
7. Sign-offs for the three platforms (Linux, Mac, Windows) usually occur
offline, but at least one approval will be needed on the PR.
* If issues are discovered during testing, update `master-next` as
- If issues are discovered during testing, update `master-next` as
needed, but ensure that the changes are properly squashed, and the
version setting commit remains last
8. Once everything is ready to go, push to `master` **only**.
```
git fetch upstreams
@@ -901,15 +940,20 @@ git log -1 --oneline
# Other branches, including some from upstream-push, may also be
# present.
```
9. Tag the release, too.
```
git tag <version number>
git push upstream-push <version number>
```
9. Delete the `master-next` branch on the repo.
```
git push --delete upstream-push master-next
```
10. [Create a new release on
Github](https://github.com/XRPLF/rippled/releases). Be sure that
"Set as the latest release" is checked.
@@ -921,17 +965,21 @@ Once the hotfix is released, it needs to be reverse merged into
1. Create a branch in your own repo, based on `upstream/develop`.
The branch name is not important, but could include "mergeNNN".
E.g. For release 2.2.3, use `merge223`.
```
git fetch upstreams
git checkout --no-track -b merge223 upstream/develop
```
2. Merge master into your branch.
```
# I like the "--edit --log --verbose" parameters, but they are
# not required.
git merge upstream/master
```
3. `BuildInfo.cpp` will have a conflict with the version number.
Resolve it with the version from `develop` - the higher version.
4. Push your branch to your repo, and open a normal PR against
@@ -939,22 +987,27 @@ git merge upstream/master
is a merge of the hotfix version. The "Context" should summarize
the changes from the hotfix. Include the following text
prominently:
```
This PR must be merged manually using a --ff-only merge. Do not use the Github UI.
```
5. Depending on the complexity of the hotfix, and/or merge conflicts,
the PR may need a thorough review, or just a sign-off that the
merge was done correctly.
6. If `develop` is updated before this PR is merged, do not merge
`develop` back into your branch. Instead rebase preserving merges,
or do the merge again. (See also the `rerere` git config setting.)
```
git rebase --rebase-merges upstream/develop
# OR
git reset --hard upstream/develop
git merge upstream/master
```
7. When the PR is ready, push it to `develop`.
```
git fetch upstreams
@@ -963,6 +1016,7 @@ git log --show-signature "upstream/develop..HEAD"
git push upstream-push HEAD:develop
```
Development on `develop` can proceed as normal. It is recommended to
create a beta (or RC) immediately to ensure that everything worked as
expected.
@@ -977,12 +1031,13 @@ a significant fraction of users, which would necessitate a hotfix / point
release to that version as well as any later versions.
This scenario would follow the same basic procedure as above,
except that *none* of `develop`, `release`, or `master`
except that _none_ of `develop`, `release`, or `master`
would be touched during the release process.
In this example, consider if version 2.1.1 needed to be patched.
1. Create two branches in the main (`upstream`) repo.
```
git fetch upstreams
@@ -996,6 +1051,7 @@ git push upstream-push
git fetch upstreams
```
2. Work continues as above, except using `master-2.1.2`as
the base branch for any merging, packaging, etc.
3. After the release is tagged and packages are built, you could

View File

@@ -1,4 +1,4 @@
ISC License
ISC License
Copyright (c) 2011, Arthur Britto, David Schwartz, Jed McCaleb, Vinnie Falco, Bob Way, Eric Lombrozo, Nikolaos D. Bougalis, Howard Hinnant.
Copyright (c) 2012-2020, the XRP Ledger developers.
@@ -14,4 +14,3 @@ ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

View File

@@ -1,51 +1,54 @@
[![codecov](https://codecov.io/gh/XRPLF/rippled/graph/badge.svg?token=WyFr5ajq3O)](https://codecov.io/gh/XRPLF/rippled)
# The XRP Ledger
The [XRP Ledger](https://xrpl.org/) is a decentralized cryptographic ledger powered by a network of peer-to-peer nodes. The XRP Ledger uses a novel Byzantine Fault Tolerant consensus algorithm to settle and record transactions in a secure distributed database without a central operator.
## XRP
[XRP](https://xrpl.org/xrp.html) is a public, counterparty-free asset native to the XRP Ledger, and is designed to bridge the many different currencies in use worldwide. XRP is traded on the open-market and is available for anyone to access. The XRP Ledger was created in 2012 with a finite supply of 100 billion units of XRP.
[XRP](https://xrpl.org/xrp.html) is a public, counterparty-free crypto-asset native to the XRP Ledger, and is designed as a gas token for network services and to bridge different currencies. XRP is traded on the open-market and is available for anyone to access. The XRP Ledger was created in 2012 with a finite supply of 100 billion units of XRP.
## rippled
The server software that powers the XRP Ledger is called `rippled` and is available in this repository under the permissive [ISC open-source license](LICENSE.md). The `rippled` server software is written primarily in C++ and runs on a variety of platforms. The `rippled` server software can run in several modes depending on its [configuration](https://xrpl.org/rippled-server-modes.html).
If you are interested in running an **API Server** (including a **Full History Server**), take a look at [Clio](https://github.com/XRPLF/clio). (rippled Reporting Mode has been replaced by Clio.)
### Build from Source
* [Read the build instructions in `BUILD.md`](BUILD.md)
* If you encounter any issues, please [open an issue](https://github.com/XRPLF/rippled/issues)
- [Read the build instructions in `BUILD.md`](BUILD.md)
- If you encounter any issues, please [open an issue](https://github.com/XRPLF/rippled/issues)
## Key Features of the XRP Ledger
- **[Censorship-Resistant Transaction Processing][]:** No single party decides which transactions succeed or fail, and no one can "roll back" a transaction after it completes. As long as those who choose to participate in the network keep it healthy, they can settle transactions in seconds.
- **[Fast, Efficient Consensus Algorithm][]:** The XRP Ledger's consensus algorithm settles transactions in 4 to 5 seconds, processing at a throughput of up to 1500 transactions per second. These properties put XRP at least an order of magnitude ahead of other top digital assets.
- **[Finite XRP Supply][]:** When the XRP Ledger began, 100 billion XRP were created, and no more XRP will ever be created. The available supply of XRP decreases slowly over time as small amounts are destroyed to pay transaction costs.
- **[Responsible Software Governance][]:** A team of full-time, world-class developers at Ripple maintain and continually improve the XRP Ledger's underlying software with contributions from the open-source community. Ripple acts as a steward for the technology and an advocate for its interests, and builds constructive relationships with governments and financial institutions worldwide.
- **[Finite XRP Supply][]:** When the XRP Ledger began, 100 billion XRP were created, and no more XRP will ever be created. The available supply of XRP decreases slowly over time as small amounts are destroyed to pay transaction fees.
- **[Responsible Software Governance][]:** A team of full-time developers at Ripple & other organizations maintain and continually improve the XRP Ledger's underlying software with contributions from the open-source community. Ripple acts as a steward for the technology and an advocate for its interests.
- **[Secure, Adaptable Cryptography][]:** The XRP Ledger relies on industry standard digital signature systems like ECDSA (the same scheme used by Bitcoin) but also supports modern, efficient algorithms like Ed25519. The extensible nature of the XRP Ledger's software makes it possible to add and disable algorithms as the state of the art in cryptography advances.
- **[Modern Features for Smart Contracts][]:** Features like Escrow, Checks, and Payment Channels support cutting-edge financial applications including the [Interledger Protocol](https://interledger.org/). This toolbox of advanced features comes with safety features like a process for amending the network and separate checks against invariant constraints.
- **[Modern Features][]:** Features like Escrow, Checks, and Payment Channels support financial applications atop of the XRP Ledger. This toolbox of advanced features comes with safety features like a process for amending the network and separate checks against invariant constraints.
- **[On-Ledger Decentralized Exchange][]:** In addition to all the features that make XRP useful on its own, the XRP Ledger also has a fully-functional accounting system for tracking and trading obligations denominated in any way users want, and an exchange built into the protocol. The XRP Ledger can settle long, cross-currency payment paths and exchanges of multiple currencies in atomic transactions, bridging gaps of trust with XRP.
[Censorship-Resistant Transaction Processing]: https://xrpl.org/xrp-ledger-overview.html#censorship-resistant-transaction-processing
[Fast, Efficient Consensus Algorithm]: https://xrpl.org/xrp-ledger-overview.html#fast-efficient-consensus-algorithm
[Finite XRP Supply]: https://xrpl.org/xrp-ledger-overview.html#finite-xrp-supply
[Responsible Software Governance]: https://xrpl.org/xrp-ledger-overview.html#responsible-software-governance
[Secure, Adaptable Cryptography]: https://xrpl.org/xrp-ledger-overview.html#secure-adaptable-cryptography
[Modern Features for Smart Contracts]: https://xrpl.org/xrp-ledger-overview.html#modern-features-for-smart-contracts
[On-Ledger Decentralized Exchange]: https://xrpl.org/xrp-ledger-overview.html#on-ledger-decentralized-exchange
[Censorship-Resistant Transaction Processing]: https://xrpl.org/transaction-censorship-detection.html#transaction-censorship-detection
[Fast, Efficient Consensus Algorithm]: https://xrpl.org/consensus-research.html#consensus-research
[Finite XRP Supply]: https://xrpl.org/what-is-xrp.html
[Responsible Software Governance]: https://xrpl.org/contribute-code.html#contribute-code-to-the-xrp-ledger
[Secure, Adaptable Cryptography]: https://xrpl.org/cryptographic-keys.html#cryptographic-keys
[Modern Features]: https://xrpl.org/use-specialized-payment-types.html
[On-Ledger Decentralized Exchange]: https://xrpl.org/decentralized-exchange.html#decentralized-exchange
## Source Code
Here are some good places to start learning the source code:
- Read the markdown files in the source tree: `src/ripple/**/*.md`.
- Read [the levelization document](./Builds/levelization) to get an idea of the internal dependency graph.
- Read [the levelization document](.github/scripts/levelization) to get an idea of the internal dependency graph.
- In the big picture, the `main` function constructs an `ApplicationImp` object, which implements the `Application` virtual interface. Almost every component in the application takes an `Application&` parameter in its constructor, typically named `app` and stored as a member variable `app_`. This allows most components to depend on any other component.
### Repository Contents
| Folder | Contents |
|:-----------|:-------------------------------------------------|
| :--------- | :----------------------------------------------- |
| `./bin` | Scripts and data files for Ripple integrators. |
| `./Builds` | Platform-specific guides for building `rippled`. |
| `./docs` | Source documentation files and doxygen config. |
@@ -55,15 +58,14 @@ Here are some good places to start learning the source code:
Some of the directories under `src` are external repositories included using
git-subtree. See those directories' README files for more details.
## Additional Documentation
* [XRP Ledger Dev Portal](https://xrpl.org/)
* [Setup and Installation](https://xrpl.org/install-rippled.html)
* [Source Documentation (Doxygen)](https://xrplf.github.io/rippled/)
- [XRP Ledger Dev Portal](https://xrpl.org/)
- [Setup and Installation](https://xrpl.org/install-rippled.html)
- [Source Documentation (Doxygen)](https://xrplf.github.io/rippled/)
## See Also
* [Clio API Server for the XRP Ledger](https://github.com/XRPLF/clio)
* [Mailing List for Release Announcements](https://groups.google.com/g/ripple-server)
* [Learn more about the XRP Ledger (YouTube)](https://www.youtube.com/playlist?list=PLJQ55Tj1hIVZtJ_JdTvSum2qMTsedWkNi)
- [Clio API Server for the XRP Ledger](https://github.com/XRPLF/clio)
- [Mailing List for Release Announcements](https://groups.google.com/g/ripple-server)
- [Learn more about the XRP Ledger (YouTube)](https://www.youtube.com/playlist?list=PLJQ55Tj1hIVZtJ_JdTvSum2qMTsedWkNi)

File diff suppressed because it is too large Load Diff

View File

@@ -2,7 +2,6 @@
For more details on operating an XRP Ledger server securely, please visit https://xrpl.org/manage-the-rippled-server.html.
# Security Policy
## Supported Versions
@@ -77,13 +76,14 @@ The amount paid varies dramatically. Vulnerabilities that are harmless on their
To report a qualifying bug, please send a detailed report to:
|Email Address|bugs@ripple.com |
|:-----------:|:----------------------------------------------------|
|Short Key ID | `0xC57929BE` |
|Long Key ID | `0xCD49A0AFC57929BE` |
|Fingerprint | `24E6 3B02 37E0 FA9C 5E96 8974 CD49 A0AF C579 29BE` |
| Email Address | bugs@ripple.com |
| :-----------: | :-------------------------------------------------- |
| Short Key ID | `0xC57929BE` |
| Long Key ID | `0xCD49A0AFC57929BE` |
| Fingerprint | `24E6 3B02 37E0 FA9C 5E96 8974 CD49 A0AF C579 29BE` |
The full PGP key for this address, which is also available on several key servers (e.g. on [keyserver.ubuntu.com](https://keyserver.ubuntu.com)), is:
The full PGP key for this address, which is also available on several key servers (e.g. on [keys.gnupg.net](https://keys.gnupg.net)), is:
```
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBFUwGHYBEAC0wpGpBPkd8W1UdQjg9+cEFzeIEJRaoZoeuJD8mofwI5Ejnjdt

View File

@@ -1,470 +0,0 @@
#!/usr/bin/node
//
// ledger?l=L
// transaction?h=H
// ledger_entry?l=L&h=H
// account?l=L&a=A
// directory?l=L&dir_root=H&i=I
// directory?l=L&o=A&i=I // owner directory
// offer?l=L&offer=H
// offer?l=L&account=A&i=I
// ripple_state=l=L&a=A&b=A&c=C
// account_lines?l=L&a=A
//
// A=address
// C=currency 3 letter code
// H=hash
// I=index
// L=current | closed | validated | index | hash
//
var async = require("async");
var extend = require("extend");
var http = require("http");
var url = require("url");
var Remote = require("ripple-lib").Remote;
var program = process.argv[1];
var httpd_response = function (res, opts) {
var self=this;
res.statusCode = opts.statusCode;
res.end(
"<HTML>"
+ "<HEAD><TITLE>Title</TITLE></HEAD>"
+ "<BODY BACKGROUND=\"#FFFFFF\">"
+ "State:" + self.state
+ "<UL>"
+ "<LI><A HREF=\"/\">home</A>"
+ "<LI>" + html_link('r4EM4gBQfr1QgQLXSPF4r7h84qE9mb6iCC')
// + "<LI><A HREF=\""+test+"\">rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh</A>"
+ "<LI><A HREF=\"/ledger\">ledger</A>"
+ "</UL>"
+ (opts.body || '')
+ '<HR><PRE>'
+ (opts.url || '')
+ '</PRE>'
+ "</BODY>"
+ "</HTML>"
);
};
var html_link = function (generic) {
return '<A HREF="' + build_uri({ type: 'account', account: generic}) + '">' + generic + '</A>';
};
// Build a link to a type.
var build_uri = function (params, opts) {
var c;
if (params.type === 'account') {
c = {
pathname: 'account',
query: {
l: params.ledger,
a: params.account,
},
};
} else if (params.type === 'ledger') {
c = {
pathname: 'ledger',
query: {
l: params.ledger,
},
};
} else if (params.type === 'transaction') {
c = {
pathname: 'transaction',
query: {
h: params.hash,
},
};
} else {
c = {};
}
opts = opts || {};
c.protocol = "http";
c.hostname = opts.hostname || self.base.hostname;
c.port = opts.port || self.base.port;
return url.format(c);
};
var build_link = function (item, link) {
console.log(link);
return "<A HREF=" + link + ">" + item + "</A>";
};
var rewrite_field = function (type, obj, field, opts) {
if (field in obj) {
obj[field] = rewrite_type(type, obj[field], opts);
}
};
var rewrite_type = function (type, obj, opts) {
if ('amount' === type) {
if ('string' === typeof obj) {
// XRP.
return '<B>' + obj + '</B>';
} else {
rewrite_field('address', obj, 'issuer', opts);
return obj;
}
return build_link(
obj,
build_uri({
type: 'account',
account: obj
}, opts)
);
}
if ('address' === type) {
return build_link(
obj,
build_uri({
type: 'account',
account: obj
}, opts)
);
}
else if ('ledger' === type) {
return build_link(
obj,
build_uri({
type: 'ledger',
ledger: obj,
}, opts)
);
}
else if ('node' === type) {
// A node
if ('PreviousTxnID' in obj)
obj.PreviousTxnID = rewrite_type('transaction', obj.PreviousTxnID, opts);
if ('Offer' === obj.LedgerEntryType) {
if ('NewFields' in obj) {
if ('TakerGets' in obj.NewFields)
obj.NewFields.TakerGets = rewrite_type('amount', obj.NewFields.TakerGets, opts);
if ('TakerPays' in obj.NewFields)
obj.NewFields.TakerPays = rewrite_type('amount', obj.NewFields.TakerPays, opts);
}
}
obj.LedgerEntryType = '<B>' + obj.LedgerEntryType + '</B>';
return obj;
}
else if ('transaction' === type) {
// Reference to a transaction.
return build_link(
obj,
build_uri({
type: 'transaction',
hash: obj
}, opts)
);
}
return 'ERROR: ' + type;
};
var rewrite_object = function (obj, opts) {
var out = extend({}, obj);
rewrite_field('address', out, 'Account', opts);
rewrite_field('ledger', out, 'parent_hash', opts);
rewrite_field('ledger', out, 'ledger_index', opts);
rewrite_field('ledger', out, 'ledger_current_index', opts);
rewrite_field('ledger', out, 'ledger_hash', opts);
if ('ledger' in obj) {
// It's a ledger header.
out.ledger = rewrite_object(out.ledger, opts);
if ('ledger_hash' in out.ledger)
out.ledger.ledger_hash = '<B>' + out.ledger.ledger_hash + '</B>';
delete out.ledger.hash;
delete out.ledger.totalCoins;
}
if ('TransactionType' in obj) {
// It's a transaction.
out.TransactionType = '<B>' + obj.TransactionType + '</B>';
rewrite_field('amount', out, 'TakerGets', opts);
rewrite_field('amount', out, 'TakerPays', opts);
rewrite_field('ledger', out, 'inLedger', opts);
out.meta.AffectedNodes = out.meta.AffectedNodes.map(function (node) {
var kind = 'CreatedNode' in node
? 'CreatedNode'
: 'ModifiedNode' in node
? 'ModifiedNode'
: 'DeletedNode' in node
? 'DeletedNode'
: undefined;
if (kind) {
node[kind] = rewrite_type('node', node[kind], opts);
}
return node;
});
}
else if ('node' in obj && 'LedgerEntryType' in obj.node) {
// Its a ledger entry.
if (obj.node.LedgerEntryType === 'AccountRoot') {
rewrite_field('address', out.node, 'Account', opts);
rewrite_field('transaction', out.node, 'PreviousTxnID', opts);
rewrite_field('ledger', out.node, 'PreviousTxnLgrSeq', opts);
}
out.node.LedgerEntryType = '<B>' + out.node.LedgerEntryType + '</B>';
}
return out;
};
var augment_object = function (obj, opts, done) {
if (obj.node.LedgerEntryType == 'AccountRoot') {
var tx_hash = obj.node.PreviousTxnID;
var tx_ledger = obj.node.PreviousTxnLgrSeq;
obj.history = [];
async.whilst(
function () { return tx_hash; },
function (callback) {
// console.log("augment_object: request: %s %s", tx_hash, tx_ledger);
opts.remote.request_tx(tx_hash)
.on('success', function (m) {
tx_hash = undefined;
tx_ledger = undefined;
//console.log("augment_object: ", JSON.stringify(m));
m.meta.AffectedNodes.filter(function(n) {
// console.log("augment_object: ", JSON.stringify(n));
// if (n.ModifiedNode)
// console.log("augment_object: %s %s %s %s %s %s/%s", 'ModifiedNode' in n, n.ModifiedNode && (n.ModifiedNode.LedgerEntryType === 'AccountRoot'), n.ModifiedNode && n.ModifiedNode.FinalFields && (n.ModifiedNode.FinalFields.Account === obj.node.Account), Object.keys(n)[0], n.ModifiedNode && (n.ModifiedNode.LedgerEntryType), obj.node.Account, n.ModifiedNode && n.ModifiedNode.FinalFields && n.ModifiedNode.FinalFields.Account);
// if ('ModifiedNode' in n && n.ModifiedNode.LedgerEntryType === 'AccountRoot')
// {
// console.log("***: ", JSON.stringify(m));
// console.log("***: ", JSON.stringify(n));
// }
return 'ModifiedNode' in n
&& n.ModifiedNode.LedgerEntryType === 'AccountRoot'
&& n.ModifiedNode.FinalFields
&& n.ModifiedNode.FinalFields.Account === obj.node.Account;
})
.forEach(function (n) {
tx_hash = n.ModifiedNode.PreviousTxnID;
tx_ledger = n.ModifiedNode.PreviousTxnLgrSeq;
obj.history.push({
tx_hash: tx_hash,
tx_ledger: tx_ledger
});
console.log("augment_object: next: %s %s", tx_hash, tx_ledger);
});
callback();
})
.on('error', function (m) {
callback(m);
})
.request();
},
function (err) {
if (err) {
done();
}
else {
async.forEach(obj.history, function (o, callback) {
opts.remote.request_account_info(obj.node.Account)
.ledger_index(o.tx_ledger)
.on('success', function (m) {
//console.log("augment_object: ", JSON.stringify(m));
o.Balance = m.account_data.Balance;
// o.account_data = m.account_data;
callback();
})
.on('error', function (m) {
o.error = m;
callback();
})
.request();
},
function (err) {
done(err);
});
}
});
}
else {
done();
}
};
if (process.argv.length < 4 || process.argv.length > 7) {
console.log("Usage: %s ws_ip ws_port [<ip> [<port> [<start>]]]", program);
}
else {
var ws_ip = process.argv[2];
var ws_port = process.argv[3];
var ip = process.argv.length > 4 ? process.argv[4] : "127.0.0.1";
var port = process.argv.length > 5 ? process.argv[5] : "8080";
// console.log("START");
var self = this;
var remote = (new Remote({
websocket_ip: ws_ip,
websocket_port: ws_port,
trace: false
}))
.on('state', function (m) {
console.log("STATE: %s", m);
self.state = m;
})
// .once('ledger_closed', callback)
.connect()
;
self.base = {
hostname: ip,
port: port,
remote: remote,
};
// console.log("SERVE");
var server = http.createServer(function (req, res) {
var input = "";
req.setEncoding();
req.on('data', function (buffer) {
// console.log("DATA: %s", buffer);
input = input + buffer;
});
req.on('end', function () {
// console.log("URL: %s", req.url);
// console.log("HEADERS: %s", JSON.stringify(req.headers, undefined, 2));
var _parsed = url.parse(req.url, true);
var _url = JSON.stringify(_parsed, undefined, 2);
// console.log("HEADERS: %s", JSON.stringify(_parsed, undefined, 2));
if (_parsed.pathname === "/account") {
var request = remote
.request_ledger_entry('account_root')
.ledger_index(-1)
.account_root(_parsed.query.a)
.on('success', function (m) {
// console.log("account_root: %s", JSON.stringify(m, undefined, 2));
augment_object(m, self.base, function() {
httpd_response(res,
{
statusCode: 200,
url: _url,
body: "<PRE>"
+ JSON.stringify(rewrite_object(m, self.base), undefined, 2)
+ "</PRE>"
});
});
})
.request();
} else if (_parsed.pathname === "/ledger") {
var request = remote
.request_ledger(undefined, { expand: true, transactions: true })
.on('success', function (m) {
// console.log("Ledger: %s", JSON.stringify(m, undefined, 2));
httpd_response(res,
{
statusCode: 200,
url: _url,
body: "<PRE>"
+ JSON.stringify(rewrite_object(m, self.base), undefined, 2)
+"</PRE>"
});
})
if (_parsed.query.l && _parsed.query.l.length === 64) {
request.ledger_hash(_parsed.query.l);
}
else if (_parsed.query.l) {
request.ledger_index(Number(_parsed.query.l));
}
else {
request.ledger_index(-1);
}
request.request();
} else if (_parsed.pathname === "/transaction") {
var request = remote
.request_tx(_parsed.query.h)
// .request_transaction_entry(_parsed.query.h)
// .ledger_select(_parsed.query.l)
.on('success', function (m) {
// console.log("transaction: %s", JSON.stringify(m, undefined, 2));
httpd_response(res,
{
statusCode: 200,
url: _url,
body: "<PRE>"
+ JSON.stringify(rewrite_object(m, self.base), undefined, 2)
+"</PRE>"
});
})
.on('error', function (m) {
httpd_response(res,
{
statusCode: 200,
url: _url,
body: "<PRE>"
+ 'ERROR: ' + JSON.stringify(m, undefined, 2)
+"</PRE>"
});
})
.request();
} else {
var test = build_uri({
type: 'account',
ledger: 'closed',
account: 'rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh',
}, self.base);
httpd_response(res,
{
statusCode: req.url === "/" ? 200 : 404,
url: _url,
});
}
});
});
server.listen(port, ip, undefined,
function () {
console.log("Listening at: http://%s:%s", ip, port);
});
}
// vim:sw=2:sts=2:ts=8:et

View File

@@ -1,64 +0,0 @@
var ripple = require('ripple-lib');
var v = {
seed: "snoPBrXtMeMyMHUVTgbuqAfg1SUTb",
addr: "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh"
};
var remote = ripple.Remote.from_config({
"trusted" : true,
"websocket_ip" : "127.0.0.1",
"websocket_port" : 5006,
"websocket_ssl" : false,
"local_signing" : true
});
var tx_json = {
"Account" : v.addr,
"Amount" : "10000000",
"Destination" : "rEu2ULPiEQm1BAL8pYzmXnNX1aFX9sCks",
"Fee" : "10",
"Flags" : 0,
"Sequence" : 3,
"TransactionType" : "Payment"
//"SigningPubKey": '0396941B22791A448E5877A44CE98434DB217D6FB97D63F0DAD23BE49ED45173C9'
};
remote.on('connected', function () {
var req = remote.request_sign(v.seed, tx_json);
req.message.debug_signing = true;
req.on('success', function (result) {
console.log("SERVER RESULT");
console.log(result);
var sim = {};
var tx = remote.transaction();
tx.tx_json = tx_json;
tx._secret = v.seed;
tx.complete();
var unsigned = tx.serialize().to_hex();
tx.sign();
sim.tx_blob = tx.serialize().to_hex();
sim.tx_json = tx.tx_json;
sim.tx_signing_hash = tx.signing_hash().to_hex();
sim.tx_unsigned = unsigned;
console.log("\nLOCAL RESULT");
console.log(sim);
remote.connect(false);
});
req.on('error', function (err) {
if (err.error === "remoteError" && err.remote.error === "srcActNotFound") {
console.log("Please fund account "+v.addr+" to run this test.");
} else {
console.log('error', err);
}
remote.connect(false);
});
req.request();
});
remote.connect();

View File

@@ -1,18 +0,0 @@
#!/usr/bin/node
//
// Returns a Gravatar style hash as per: http://en.gravatar.com/site/implement/hash/
//
if (3 != process.argv.length) {
process.stderr.write("Usage: " + process.argv[1] + " email_address\n\nReturns gravatar style hash.\n");
process.exit(1);
} else {
var md5 = require('crypto').createHash('md5');
md5.update(process.argv[2].trim().toLowerCase());
process.stdout.write(md5.digest('hex') + "\n");
}
// vim:sw=2:sts=2:ts=8:et

View File

@@ -1,31 +0,0 @@
#!/usr/bin/node
//
// This program allows IE 9 ripple-clients to make websocket connections to
// rippled using flash. As IE 9 does not have websocket support, this required
// if you wish to support IE 9 ripple-clients.
//
// http://www.lightsphere.com/dev/articles/flash_socket_policy.html
//
// For better security, be sure to set the Port below to the port of your
// [websocket_public_port].
//
var net = require("net"),
port = "*",
domains = ["*:"+port]; // Domain:Port
net.createServer(
function(socket) {
socket.write("<?xml version='1.0' ?>\n");
socket.write("<!DOCTYPE cross-domain-policy SYSTEM 'http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd'>\n");
socket.write("<cross-domain-policy>\n");
domains.forEach(
function(domain) {
var parts = domain.split(':');
socket.write("\t<allow-access-from domain='" + parts[0] + "' to-ports='" + parts[1] + "' />\n");
}
);
socket.write("</cross-domain-policy>\n");
socket.end();
}
).listen(843);

View File

@@ -1,150 +0,0 @@
#!/usr/bin/env bash
# This script generates information about your rippled installation
# and system. It can be used to help debug issues that you may face
# in your installation. While this script endeavors to not display any
# sensitive information, it is recommended that you read the output
# before sharing with any third parties.
rippled_exe=/opt/ripple/bin/rippled
conf_file=/etc/opt/ripple/rippled.cfg
while getopts ":e:c:" opt; do
case $opt in
e)
rippled_exe=${OPTARG}
;;
c)
conf_file=${OPTARG}
;;
\?)
echo "Invalid option: -$OPTARG"
exit -1
esac
done
tmp_loc=$(mktemp -d --tmpdir ripple_info.XXXXX)
chmod 751 ${tmp_loc}
awk_prog=${tmp_loc}/cfg.awk
summary_out=${tmp_loc}/rippled_info.md
printf "# rippled report info\n\n> generated at %s\n" "$(date -R)" > ${summary_out}
function log_section {
printf "\n## %s\n" "$*" >> ${summary_out}
while read -r l; do
echo " $l" >> ${summary_out}
done </dev/stdin
}
function join_by {
local IFS="$1"; shift; echo "$*";
}
if [[ -f ${conf_file} ]] ; then
exclude=( ips ips_fixed node_seed validation_seed validator_token )
cleaned_conf=${tmp_loc}/cleaned_rippled_cfg.txt
cat << 'EOP' >> ${awk_prog}
BEGIN {FS="[[:space:]]*=[[:space:]]*"; skip=0; db_path=""; print > OUT_FILE; split(exl,exa,"|")}
/^#/ {next}
save==2 && /^[[:space:]]*$/ {next}
/^\[.+\]$/ {
section=tolower(gensub(/^\[[[:space:]]*([a-zA-Z_]+)[[:space:]]*\]$/, "\\1", "g"))
skip = 0
for (i in exa) {
if (section == exa[i])
skip = 1
}
if (section == "database_path")
save = 1
}
skip==1 {next}
save==2 {save=0; db_path=$0}
save==1 {save=2}
$1 ~ /password/ {$0=$1"=<redacted>"}
{print >> OUT_FILE}
END {print db_path}
EOP
db=$(\
sed -r -e 's/\<s[[:alnum:]]{28}\>/<redactedsecret>/g;s/^[[:space:]]*//;s/[[:space:]]*$//' ${conf_file} |\
awk -v OUT_FILE=${cleaned_conf} -v exl="$(join_by '|' "${exclude[@]}")" -f ${awk_prog})
rm ${awk_prog}
cat ${cleaned_conf} | log_section "cleaned config file"
rm ${cleaned_conf}
echo "${db}" | log_section "database path"
df ${db} | log_section "df: database"
fi
# Send output from this script to a log file
## this captures any messages
## or errors from the script itself
log_file=${tmp_loc}/get_info.log
exec 3>&1 1>>${log_file} 2>&1
## Send all stdout files to /tmp
if [[ -x ${rippled_exe} ]] ; then
pgrep rippled && \
${rippled_exe} --conf ${conf_file} \
-- server_info | log_section "server info"
fi
cat /proc/meminfo | log_section "meminfo"
cat /proc/swaps | log_section "swap space"
ulimit -a | log_section "ulimit"
if command -v lshw >/dev/null 2>&1 ; then
lshw 2>/dev/null | log_section "hardware info"
else
lscpu > ${tmp_loc}/hw_info.txt
hwinfo >> ${tmp_loc}/hw_info.txt
lspci >> ${tmp_loc}/hw_info.txt
lsblk >> ${tmp_loc}/hw_info.txt
cat ${tmp_loc}/hw_info.txt | log_section "hardware info"
rm ${tmp_loc}/hw_info.txt
fi
if command -v iostat >/dev/null 2>&1 ; then
iostat -t -d -x 2 6 | log_section "iostat"
fi
df -h | log_section "free disk space"
drives=($(df | awk '$1 ~ /^\/dev\// {print $1}' | xargs -n 1 basename))
block_devs=($(ls /sys/block/))
for d in "${drives[@]}"; do
for dev in "${block_devs[@]}"; do
#echo "D: [$d], DEV: [$dev]"
if [[ $d =~ $dev ]]; then
# this file (if exists) has 0 for SSD and 1 for HDD
if [[ "$(cat /sys/block/${dev}/queue/rotational 2>/dev/null)" == 0 ]] ; then
echo "${d} : SSD" >> ${tmp_loc}/is_ssd.txt
else
echo "${d} : NO SSD" >> ${tmp_loc}/is_ssd.txt
fi
fi
done
done
if [[ -f ${tmp_loc}/is_ssd.txt ]] ; then
cat ${tmp_loc}/is_ssd.txt | log_section "SSD"
rm ${tmp_loc}/is_ssd.txt
fi
cat ${log_file} | log_section "script log"
cat << MSG | tee /dev/fd/3
####################################################
rippled info has been gathered. Please copy the
contents of ${summary_out}
to a github gist at https://gist.github.com/
PLEASE REVIEW THIS FILE FOR ANY SENSITIVE DATA
BEFORE POSTING! We have tried our best to omit
any sensitive information from this file, but you
should verify before posting.
####################################################
MSG

View File

@@ -5,7 +5,7 @@ then
name=$( basename $0 )
cat <<- USAGE
Usage: $name <username>
Where <username> is the Github username of the upstream repo. e.g. XRPLF
USAGE
exit 0
@@ -83,4 +83,3 @@ fi
_run git fetch --jobs=$(nproc) upstreams
exit 0

View File

@@ -5,7 +5,7 @@ then
name=$( basename $0 )
cat <<- USAGE
Usage: $name workbranch base/branch user/branch [user/branch [...]]
* workbranch will be created locally from base/branch
* base/branch and user/branch may be specified as user:branch to allow
easy copying from Github PRs
@@ -66,4 +66,3 @@ git push $push HEAD:$b
git fetch $repo
-------------------------------------------------------------------
PUSH

View File

@@ -1,23 +0,0 @@
#!/usr/bin/node
//
// Returns hex of lowercasing a string.
//
var stringToHex = function (s) {
return Array.prototype.map.call(s, function (c) {
var b = c.charCodeAt(0);
return b < 16 ? "0" + b.toString(16) : b.toString(16);
}).join("");
};
if (3 != process.argv.length) {
process.stderr.write("Usage: " + process.argv[1] + " string\n\nReturns hex of lowercasing string.\n");
process.exit(1);
} else {
process.stdout.write(stringToHex(process.argv[2].toLowerCase()) + "\n");
}
// vim:sw=2:sts=2:ts=8:et

View File

@@ -1,42 +0,0 @@
#!/usr/bin/node
//
// This is a tool to issue JSON-RPC requests from the command line.
//
// This can be used to test a JSON-RPC server.
//
// Requires: npm simple-jsonrpc
//
var jsonrpc = require('simple-jsonrpc');
var program = process.argv[1];
if (5 !== process.argv.length) {
console.log("Usage: %s <URL> <method> <json>", program);
}
else {
var url = process.argv[2];
var method = process.argv[3];
var json_raw = process.argv[4];
var json;
try {
json = JSON.parse(json_raw);
}
catch (e) {
console.log("JSON parse error: %s", e.message);
throw e;
}
var client = jsonrpc.client(url);
client.call(method, json,
function (result) {
console.log(JSON.stringify(result, undefined, 2));
},
function (error) {
console.log(JSON.stringify(error, undefined, 2));
});
}
// vim:sw=2:sts=2:ts=8:et

View File

@@ -1,68 +0,0 @@
#!/usr/bin/node
//
// This is a tool to listen for JSON-RPC requests at an IP and port.
//
// This will report the request to console and echo back the request as the response.
//
var http = require("http");
var program = process.argv[1];
if (4 !== process.argv.length) {
console.log("Usage: %s <ip> <port>", program);
}
else {
var ip = process.argv[2];
var port = process.argv[3];
var server = http.createServer(function (req, res) {
console.log("CONNECT");
var input = "";
req.setEncoding();
req.on('data', function (buffer) {
// console.log("DATA: %s", buffer);
input = input + buffer;
});
req.on('end', function () {
// console.log("END");
var json_req;
console.log("URL: %s", req.url);
console.log("HEADERS: %s", JSON.stringify(req.headers, undefined, 2));
try {
json_req = JSON.parse(input);
console.log("REQ: %s", JSON.stringify(json_req, undefined, 2));
}
catch (e) {
console.log("BAD JSON: %s", e.message);
json_req = { error : e.message }
}
res.statusCode = 200;
res.end(JSON.stringify({
jsonrpc: "2.0",
result: { request : json_req },
id: req.id
}));
});
req.on('close', function () {
console.log("CLOSE");
});
});
server.listen(port, ip, undefined,
function () {
console.log("Listening at: %s:%s", ip, port);
});
}
// vim:sw=2:sts=2:ts=8:et

View File

@@ -1,218 +0,0 @@
#!/bin/bash
set -o errexit
marker_base=985c80fbc6131f3a8cedd0da7e8af98dfceb13c7
marker_commit=${1:-${marker_base}}
if [ $(git merge-base ${marker_commit} ${marker_base}) != ${marker_base} ]; then
echo "first marker commit not an ancestor: ${marker_commit}"
exit 1
fi
if [ $(git merge-base ${marker_commit} HEAD) != $(git rev-parse --verify ${marker_commit}) ]; then
echo "given marker commit not an ancestor: ${marker_commit}"
exit 1
fi
if [ -e Builds/CMake ]; then
echo move CMake
git mv Builds/CMake cmake
git add --update .
git commit -m 'Move CMake directory' --author 'Pretty Printer <cpp@ripple.com>'
fi
if [ -e src/ripple ]; then
echo move protocol buffers
mkdir -p include/xrpl
if [ -e src/ripple/proto ]; then
git mv src/ripple/proto include/xrpl
fi
extract_list() {
git show ${marker_commit}:Builds/CMake/RippledCore.cmake | \
awk "/END ${1}/ { p = 0 } p && /src\/ripple/; /BEGIN ${1}/ { p = 1 }" | \
sed -e 's#src/ripple/##' -e 's#[^a-z]\+$##'
}
move_files() {
oldroot="$1"; shift
newroot="$1"; shift
detail="$1"; shift
files=("$@")
for file in ${files[@]}; do
if [ ! -e ${oldroot}/${file} ]; then
continue
fi
dir=$(dirname ${file})
if [ $(basename ${dir}) == 'details' ]; then
dir=$(dirname ${dir})
fi
if [ $(basename ${dir}) == 'impl' ]; then
dir="$(dirname ${dir})/${detail}"
fi
mkdir -p ${newroot}/${dir}
git mv ${oldroot}/${file} ${newroot}/${dir}
done
}
echo move libxrpl headers
files=$(extract_list 'LIBXRPL HEADERS')
files+=(
basics/SlabAllocator.h
beast/asio/io_latency_probe.h
beast/container/aged_container.h
beast/container/aged_container_utility.h
beast/container/aged_map.h
beast/container/aged_multimap.h
beast/container/aged_multiset.h
beast/container/aged_set.h
beast/container/aged_unordered_map.h
beast/container/aged_unordered_multimap.h
beast/container/aged_unordered_multiset.h
beast/container/aged_unordered_set.h
beast/container/detail/aged_associative_container.h
beast/container/detail/aged_container_iterator.h
beast/container/detail/aged_ordered_container.h
beast/container/detail/aged_unordered_container.h
beast/container/detail/empty_base_optimization.h
beast/core/LockFreeStack.h
beast/insight/Collector.h
beast/insight/Counter.h
beast/insight/CounterImpl.h
beast/insight/Event.h
beast/insight/EventImpl.h
beast/insight/Gauge.h
beast/insight/GaugeImpl.h
beast/insight/Group.h
beast/insight/Groups.h
beast/insight/Hook.h
beast/insight/HookImpl.h
beast/insight/Insight.h
beast/insight/Meter.h
beast/insight/MeterImpl.h
beast/insight/NullCollector.h
beast/insight/StatsDCollector.h
beast/test/fail_counter.h
beast/test/fail_stream.h
beast/test/pipe_stream.h
beast/test/sig_wait.h
beast/test/string_iostream.h
beast/test/string_istream.h
beast/test/string_ostream.h
beast/test/test_allocator.h
beast/test/yield_to.h
beast/utility/hash_pair.h
beast/utility/maybe_const.h
beast/utility/temp_dir.h
# included by only json/impl/json_assert.h
json/json_errors.h
protocol/PayChan.h
protocol/RippleLedgerHash.h
protocol/messages.h
protocol/st.h
)
files+=(
basics/README.md
crypto/README.md
json/README.md
protocol/README.md
resource/README.md
)
move_files src/ripple include/xrpl detail ${files[@]}
echo move libxrpl sources
files=$(extract_list 'LIBXRPL SOURCES')
move_files src/ripple src/libxrpl "" ${files[@]}
echo check leftovers
dirs=$(cd include/xrpl; ls -d */)
dirs=$(cd src/ripple; ls -d ${dirs} 2>/dev/null || true)
files="$(cd src/ripple; find ${dirs} -type f)"
if [ -n "${files}" ]; then
echo "leftover files:"
echo ${files}
exit
fi
echo remove empty directories
empty_dirs="$(cd src/ripple; find ${dirs} -depth -type d)"
for dir in ${empty_dirs[@]}; do
if [ -e ${dir} ]; then
rmdir ${dir}
fi
done
echo move xrpld sources
files=$(
extract_list 'XRPLD SOURCES'
cd src/ripple
find * -regex '.*\.\(h\|ipp\|md\|pu\|uml\|png\)'
)
move_files src/ripple src/xrpld detail ${files[@]}
files="$(cd src/ripple; find . -type f)"
if [ -n "${files}" ]; then
echo "leftover files:"
echo ${files}
exit
fi
fi
rm -rf src/ripple
echo rename .hpp to .h
find include src -name '*.hpp' -exec bash -c 'f="{}"; git mv "${f}" "${f%hpp}h"' \;
echo move PerfLog.h
if [ -e include/xrpl/basics/PerfLog.h ]; then
git mv include/xrpl/basics/PerfLog.h src/xrpld/perflog
fi
# Make sure all protobuf includes have the correct prefix.
protobuf_replace='s:^#include\s*["<].*org/xrpl\([^">]\+\)[">]:#include <xrpl/proto/org/xrpl\1>:'
# Make sure first-party includes use angle brackets and .h extension.
ripple_replace='s:include\s*["<]ripple/\(.*\)\.h\(pp\)\?[">]:include <ripple/\1.h>:'
beast_replace='s:include\s*<beast/:include <xrpl/beast/:'
# Rename impl directories to detail.
impl_rename='s:\(<xrpl.*\)/impl\(/details\)\?/:\1/detail/:'
echo rewrite includes in libxrpl
find include/xrpl src/libxrpl -type f -exec sed -i \
-e "${protobuf_replace}" \
-e "${ripple_replace}" \
-e "${beast_replace}" \
-e 's:^#include <ripple/:#include <xrpl/:' \
-e "${impl_rename}" \
{} +
echo rewrite includes in xrpld
# # https://www.baeldung.com/linux/join-multiple-lines
libxrpl_dirs="$(cd include/xrpl; ls -d1 */ | sed 's:/$::')"
# libxrpl_dirs='a\nb\nc\n'
readarray -t libxrpl_dirs <<< "${libxrpl_dirs}"
# libxrpl_dirs=(a b c)
libxrpl_dirs=$(printf -v txt '%s\\|' "${libxrpl_dirs[@]}"; echo "${txt%\\|}")
# libxrpl_dirs='a\|b\|c'
find src/xrpld src/test -type f -exec sed -i \
-e "${protobuf_replace}" \
-e "${ripple_replace}" \
-e "${beast_replace}" \
-e "s:^#include <ripple/basics/PerfLog.h>:#include <xrpld/perflog/PerfLog.h>:" \
-e "s:^#include <ripple/\(${libxrpl_dirs}\)/:#include <xrpl/\1/:" \
-e 's:^#include <ripple/:#include <xrpld/:' \
-e "${impl_rename}" \
{} +
git commit -m 'Rearrange sources' --author 'Pretty Printer <cpp@ripple.com>'
find include src -type f \( -name '*.cpp' -o -name '*.h' -o -name '*.ipp' \) -exec clang-format-10 -i {} +
git add --update .
git commit -m 'Rewrite includes' --author 'Pretty Printer <cpp@ripple.com>'
./Builds/levelization/levelization.sh
git add --update .
git commit -m 'Recompute loops' --author 'Pretty Printer <cpp@ripple.com>'

View File

@@ -1,252 +0,0 @@
#!/usr/bin/node
var async = require('async');
var Remote = require('ripple-lib').Remote;
var Transaction = require('ripple-lib').Transaction;
var UInt160 = require('ripple-lib').UInt160;
var Amount = require('ripple-lib').Amount;
var book_key = function (book) {
return book.taker_pays.currency
+ ":" + book.taker_pays.issuer
+ ":" + book.taker_gets.currency
+ ":" + book.taker_gets.issuer;
};
var book_key_cross = function (book) {
return book.taker_gets.currency
+ ":" + book.taker_gets.issuer
+ ":" + book.taker_pays.currency
+ ":" + book.taker_pays.issuer;
};
var ledger_verify = function (ledger) {
var dir_nodes = ledger.accountState.filter(function (entry) {
return entry.LedgerEntryType === 'DirectoryNode' // Only directories
&& entry.index === entry.RootIndex // Only root nodes
&& 'TakerGetsCurrency' in entry; // Only offer directories
});
var books = {};
dir_nodes.forEach(function (node) {
var book = {
taker_gets: {
currency: UInt160.from_generic(node.TakerGetsCurrency).to_json(),
issuer: UInt160.from_generic(node.TakerGetsIssuer).to_json()
},
taker_pays: {
currency: UInt160.from_generic(node.TakerPaysCurrency).to_json(),
issuer: UInt160.from_generic(node.TakerPaysIssuer).to_json()
},
quality: Amount.from_quality(node.RootIndex),
index: node.RootIndex
};
books[book_key(book)] = book;
// console.log(JSON.stringify(node, undefined, 2));
});
// console.log(JSON.stringify(dir_entry, undefined, 2));
console.log("#%s books: %s", ledger.ledger_index, Object.keys(books).length);
Object.keys(books).forEach(function (key) {
var book = books[key];
var key_cross = book_key_cross(book);
var book_cross = books[key_cross];
if (book && book_cross && !book_cross.done)
{
var book_cross_quality_inverted = Amount.from_json("1.0/1/1").divide(book_cross.quality);
if (book_cross_quality_inverted.compareTo(book.quality) >= 0)
{
// Crossing books
console.log("crossing: #%s :: %s :: %s :: %s :: %s :: %s :: %s", ledger.ledger_index, key, book.quality.to_text(), book_cross.quality.to_text(), book_cross_quality_inverted.to_text(),
book.index, book_cross.index);
}
book_cross.done = true;
}
});
var ripple_selfs = {};
var accounts = {};
var counts = {};
ledger.accountState.forEach(function (entry) {
if (entry.LedgerEntryType === 'Offer')
{
counts[entry.Account] = (counts[entry.Account] || 0) + 1;
}
else if (entry.LedgerEntryType === 'RippleState')
{
if (entry.Flags & (0x10000 | 0x40000))
{
counts[entry.LowLimit.issuer] = (counts[entry.LowLimit.issuer] || 0) + 1;
}
if (entry.Flags & (0x20000 | 0x80000))
{
counts[entry.HighLimit.issuer] = (counts[entry.HighLimit.issuer] || 0) + 1;
}
if (entry.HighLimit.issuer === entry.LowLimit.issuer)
ripple_selfs[entry.Account] = entry;
}
else if (entry.LedgerEntryType == 'AccountRoot')
{
accounts[entry.Account] = entry;
}
});
var low = 0; // Accounts with too low a count.
var high = 0;
var missing_accounts = 0; // Objects with no referencing account.
var missing_objects = 0; // Accounts specifying an object but having none.
Object.keys(counts).forEach(function (account) {
if (account in accounts)
{
if (counts[account] !== accounts[account].OwnerCount)
{
if (counts[account] < accounts[account].OwnerCount)
{
high += 1;
console.log("%s: high count %s/%s", account, counts[account], accounts[account].OwnerCount);
}
else
{
low += 1;
console.log("%s: low count %s/%s", account, counts[account], accounts[account].OwnerCount);
}
}
}
else
{
missing_accounts += 1;
console.log("%s: missing : count %s", account, counts[account]);
}
});
Object.keys(accounts).forEach(function (account) {
if (!('OwnerCount' in accounts[account]))
{
console.log("%s: bad entry : %s", account, JSON.stringify(accounts[account], undefined, 2));
}
else if (!(account in counts) && accounts[account].OwnerCount)
{
missing_objects += 1;
console.log("%s: no objects : %s/%s", account, 0, accounts[account].OwnerCount);
}
});
if (low)
console.log("counts too low = %s", low);
if (high)
console.log("counts too high = %s", high);
if (missing_objects)
console.log("missing_objects = %s", missing_objects);
if (missing_accounts)
console.log("missing_accounts = %s", missing_accounts);
if (Object.keys(ripple_selfs).length)
console.log("RippleState selfs = %s", Object.keys(ripple_selfs).length);
};
var ledger_request = function (remote, ledger_index, done) {
remote.request_ledger(undefined, {
accounts: true,
expand: true,
})
.ledger_index(ledger_index)
.on('success', function (m) {
// console.log("ledger: ", ledger_index);
// console.log("ledger: ", JSON.stringify(m, undefined, 2));
done(m.ledger);
})
.on('error', function (m) {
console.log("error");
done();
})
.request();
};
var usage = function () {
console.log("rlint.js _websocket_ip_ _websocket_port_ ");
};
var finish = function (remote) {
remote.disconnect();
// XXX Because remote.disconnect() doesn't work:
process.exit();
};
console.log("args: ", process.argv.length);
console.log("args: ", process.argv);
if (process.argv.length < 4) {
usage();
}
else {
var remote = Remote.from_config({
websocket_ip: process.argv[2],
websocket_port: process.argv[3],
})
.once('ledger_closed', function (m) {
console.log("ledger_closed: ", JSON.stringify(m, undefined, 2));
if (process.argv.length === 5) {
var ledger_index = process.argv[4];
ledger_request(remote, ledger_index, function (l) {
if (l) {
ledger_verify(l);
}
finish(remote);
});
} else if (process.argv.length === 6) {
var ledger_start = Number(process.argv[4]);
var ledger_end = Number(process.argv[5]);
var ledger_cursor = ledger_end;
async.whilst(
function () {
return ledger_start <= ledger_cursor && ledger_cursor <=ledger_end;
},
function (callback) {
// console.log(ledger_cursor);
ledger_request(remote, ledger_cursor, function (l) {
if (l) {
ledger_verify(l);
}
--ledger_cursor;
callback();
});
},
function (error) {
finish(remote);
});
} else {
finish(remote);
}
})
.connect();
}
// vim:sw=2:sts=2:ts=8:et

View File

@@ -1,51 +0,0 @@
#!/usr/bin/env bash
set -exu
: ${TRAVIS_BUILD_DIR:=""}
: ${VCPKG_DIR:=".vcpkg"}
export VCPKG_ROOT=${VCPKG_DIR}
: ${VCPKG_DEFAULT_TRIPLET:="x64-windows-static"}
export VCPKG_DEFAULT_TRIPLET
EXE="vcpkg"
if [[ -z ${COMSPEC:-} ]]; then
EXE="${EXE}.exe"
fi
if [[ -d "${VCPKG_DIR}" && -x "${VCPKG_DIR}/${EXE}" && -d "${VCPKG_DIR}/installed" ]] ; then
echo "Using cached vcpkg at ${VCPKG_DIR}"
${VCPKG_DIR}/${EXE} list
else
if [[ -d "${VCPKG_DIR}" ]] ; then
rm -rf "${VCPKG_DIR}"
fi
git clone --branch 2021.04.30 https://github.com/Microsoft/vcpkg.git ${VCPKG_DIR}
pushd ${VCPKG_DIR}
BSARGS=()
if [[ "$(uname)" == "Darwin" ]] ; then
BSARGS+=(--allowAppleClang)
fi
if [[ -z ${COMSPEC:-} ]]; then
chmod +x ./bootstrap-vcpkg.sh
time ./bootstrap-vcpkg.sh "${BSARGS[@]}"
else
time ./bootstrap-vcpkg.bat
fi
popd
fi
# TODO: bring boost in this way as well ?
# NOTE: can pin specific ports to a commit/version like this:
# git checkout <SOME COMMIT HASH> ports/boost
if [ $# -eq 0 ]; then
echo "No extra packages specified..."
PKGS=()
else
PKGS=( "$@" )
fi
for LIB in "${PKGS[@]}"; do
time ${VCPKG_DIR}/${EXE} --clean-after-build install ${LIB}
done

View File

@@ -1,40 +0,0 @@
# NOTE: must be sourced from a shell so it can export vars
cat << BATCH > ./getenv.bat
CALL %*
ENV
BATCH
while read line ; do
IFS='"' read x path arg <<<"${line}"
if [ -f "${path}" ] ; then
echo "FOUND: $path"
export VCINSTALLDIR=$(./getenv.bat "${path}" ${arg} | grep "^VCINSTALLDIR=" | sed -E "s/^VCINSTALLDIR=//g")
if [ "${VCINSTALLDIR}" != "" ] ; then
echo "USING ${VCINSTALLDIR}"
export LIB=$(./getenv.bat "${path}" ${arg} | grep "^LIB=" | sed -E "s/^LIB=//g")
export LIBPATH=$(./getenv.bat "${path}" ${arg} | grep "^LIBPATH=" | sed -E "s/^LIBPATH=//g")
export INCLUDE=$(./getenv.bat "${path}" ${arg} | grep "^INCLUDE=" | sed -E "s/^INCLUDE=//g")
ADDPATH=$(./getenv.bat "${path}" ${arg} | grep "^PATH=" | sed -E "s/^PATH=//g")
export PATH="${ADDPATH}:${PATH}"
break
fi
fi
done <<EOL
"C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Auxiliary/Build/vcvarsall.bat" x86_amd64
"C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Auxiliary/Build/vcvarsall.bat" x86_amd64
"C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/VC/Auxiliary/Build/vcvarsall.bat" x86_amd64
"C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Auxiliary/Build/vcvarsall.bat" x86_amd64
"C:/Program Files (x86)/Microsoft Visual Studio 15.0/VC/vcvarsall.bat" amd64
"C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/vcvarsall.bat" amd64
"C:/Program Files (x86)/Microsoft Visual Studio 13.0/VC/vcvarsall.bat" amd64
"C:/Program Files (x86)/Microsoft Visual Studio 12.0/VC/vcvarsall.bat" amd64
EOL
# TODO: update the list above as needed to support newer versions of msvc tools
rm -f getenv.bat
if [ "${VCINSTALLDIR}" = "" ] ; then
echo "No compatible visual studio found!"
fi

View File

@@ -1,246 +0,0 @@
#!/usr/bin/env python
"""A script to test rippled in an infinite loop of start-sync-stop.
- Requires Python 3.7+.
- Can be stopped with SIGINT.
- Has no dependencies outside the standard library.
"""
import sys
assert sys.version_info.major == 3 and sys.version_info.minor >= 7
import argparse
import asyncio
import configparser
import contextlib
import json
import logging
import os
from pathlib import Path
import platform
import subprocess
import time
import urllib.error
import urllib.request
# Enable asynchronous subprocesses on Windows. The default changed in 3.8.
# https://docs.python.org/3.7/library/asyncio-platforms.html#subprocess-support-on-windows
if (platform.system() == 'Windows' and sys.version_info.major == 3
and sys.version_info.minor < 8):
asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy())
DEFAULT_EXE = 'rippled'
DEFAULT_CONFIGURATION_FILE = 'rippled.cfg'
# Number of seconds to wait before forcefully terminating.
PATIENCE = 120
# Number of contiguous seconds in a sync state to be considered synced.
DEFAULT_SYNC_DURATION = 60
# Number of seconds between polls of state.
DEFAULT_POLL_INTERVAL = 5
SYNC_STATES = ('full', 'validating', 'proposing')
def read_config(config_file):
# strict = False: Allow duplicate keys, e.g. [rpc_startup].
# allow_no_value = True: Allow keys with no values. Generally, these
# instances use the "key" as the value, and the section name is the key,
# e.g. [debug_logfile].
# delimiters = ('='): Allow ':' as a character in Windows paths. Some of
# our "keys" are actually values, and we don't want to split them on ':'.
config = configparser.ConfigParser(
strict=False,
allow_no_value=True,
delimiters=('='),
)
config.read(config_file)
return config
def to_list(value, separator=','):
"""Parse a list from a delimited string value."""
return [s.strip() for s in value.split(separator) if s]
def find_log_file(config_file):
"""Try to figure out what log file the user has chosen. Raises all kinds
of exceptions if there is any possibility of ambiguity."""
config = read_config(config_file)
values = list(config['debug_logfile'].keys())
if len(values) < 1:
raise ValueError(
f'no [debug_logfile] in configuration file: {config_file}')
if len(values) > 1:
raise ValueError(
f'too many [debug_logfile] in configuration file: {config_file}')
return values[0]
def find_http_port(config_file):
config = read_config(config_file)
names = list(config['server'].keys())
for name in names:
server = config[name]
if 'http' in to_list(server.get('protocol', '')):
return int(server['port'])
raise ValueError(f'no server in [server] for "http" protocol')
@contextlib.asynccontextmanager
async def rippled(exe=DEFAULT_EXE, config_file=DEFAULT_CONFIGURATION_FILE):
"""A context manager for a rippled process."""
# Start the server.
process = await asyncio.create_subprocess_exec(
str(exe),
'--conf',
str(config_file),
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
logging.info(f'rippled started with pid {process.pid}')
try:
yield process
finally:
# Ask it to stop.
logging.info(f'asking rippled (pid: {process.pid}) to stop')
start = time.time()
process.terminate()
# Wait nicely.
try:
await asyncio.wait_for(process.wait(), PATIENCE)
except asyncio.TimeoutError:
# Ask the operating system to kill it.
logging.warning(f'killing rippled ({process.pid})')
try:
process.kill()
except ProcessLookupError:
pass
code = await process.wait()
end = time.time()
logging.info(
f'rippled stopped after {end - start:.1f} seconds with code {code}'
)
async def sync(
port,
*,
duration=DEFAULT_SYNC_DURATION,
interval=DEFAULT_POLL_INTERVAL,
):
"""Poll rippled on an interval until it has been synced for a duration."""
start = time.perf_counter()
while (time.perf_counter() - start) < duration:
await asyncio.sleep(interval)
request = urllib.request.Request(
f'http://127.0.0.1:{port}',
data=json.dumps({
'method': 'server_state'
}).encode(),
headers={'Content-Type': 'application/json'},
)
with urllib.request.urlopen(request) as response:
try:
body = json.loads(response.read())
except urllib.error.HTTPError as cause:
logging.warning(f'server_state returned not JSON: {cause}')
start = time.perf_counter()
continue
try:
state = body['result']['state']['server_state']
except KeyError as cause:
logging.warning(f'server_state response missing key: {cause.key}')
start = time.perf_counter()
continue
logging.info(f'server_state: {state}')
if state not in SYNC_STATES:
# Require a contiguous sync state.
start = time.perf_counter()
async def loop(test,
*,
exe=DEFAULT_EXE,
config_file=DEFAULT_CONFIGURATION_FILE):
"""
Start-test-stop rippled in an infinite loop.
Moves log to a different file after each iteration.
"""
log_file = find_log_file(config_file)
id = 0
while True:
logging.info(f'iteration: {id}')
async with rippled(exe, config_file) as process:
start = time.perf_counter()
exited = asyncio.create_task(process.wait())
tested = asyncio.create_task(test())
# Try to sync as long as the process is running.
done, pending = await asyncio.wait(
{exited, tested},
return_when=asyncio.FIRST_COMPLETED,
)
if done == {exited}:
code = exited.result()
logging.warning(
f'server halted for unknown reason with code {code}')
else:
assert done == {tested}
assert tested.exception() is None
end = time.perf_counter()
logging.info(f'synced after {end - start:.0f} seconds')
os.replace(log_file, f'debug.{id}.log')
id += 1
logging.basicConfig(
format='%(asctime)s %(levelname)-8s %(message)s',
level=logging.INFO,
datefmt='%Y-%m-%d %H:%M:%S',
)
parser = argparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument(
'rippled',
type=Path,
nargs='?',
default=DEFAULT_EXE,
help='Path to rippled.',
)
parser.add_argument(
'--conf',
type=Path,
default=DEFAULT_CONFIGURATION_FILE,
help='Path to configuration file.',
)
parser.add_argument(
'--duration',
type=int,
default=DEFAULT_SYNC_DURATION,
help='Number of contiguous seconds required in a synchronized state.',
)
parser.add_argument(
'--interval',
type=int,
default=DEFAULT_POLL_INTERVAL,
help='Number of seconds to wait between polls of state.',
)
args = parser.parse_args()
port = find_http_port(args.conf)
def test():
return sync(port, duration=args.duration, interval=args.interval)
try:
asyncio.run(loop(test, exe=args.rippled, config_file=args.conf))
except KeyboardInterrupt:
# Squelch the message. This is a normal mode of exit.
pass

View File

@@ -1,133 +0,0 @@
/* -------------------------------- REQUIRES -------------------------------- */
var child = require("child_process");
var assert = require("assert");
/* --------------------------------- CONFIG --------------------------------- */
if (process.argv[2] == null) {
[
'Usage: ',
'',
' `node bin/stop-test.js i,j [rippled_path] [rippled_conf]`',
'',
' Launch rippled and stop it after n seconds for all n in [i, j}',
' For all even values of n launch rippled with `--fg`',
' For values of n where n % 3 == 0 launch rippled with `--fg`\n',
'Examples: ',
'',
' $ node bin/stop-test.js 5,10',
(' $ node bin/stop-test.js 1,4 ' +
'build/clang.debug/rippled $HOME/.confs/rippled.cfg')
]
.forEach(function(l){console.log(l)});
process.exit();
} else {
var testRange = process.argv[2].split(',').map(Number);
var rippledPath = process.argv[3] || 'build/rippled'
var rippledConf = process.argv[4] || 'rippled.cfg'
}
var options = {
env: process.env,
stdio: 'ignore' // we could dump the child io when it fails abnormally
};
// default args
var conf_args = ['--conf='+rippledConf];
var start_args = conf_args.concat([/*'--net'*/])
var stop_args = conf_args.concat(['stop']);
/* --------------------------------- HELPERS -------------------------------- */
function start(args) {
return child.spawn(rippledPath, args, options);
}
function stop(rippled) { child.execFile(rippledPath, stop_args, options)}
function secs_l8r(ms, f) {setTimeout(f, ms * 1000); }
function show_results_and_exit(results) {
console.log(JSON.stringify(results, undefined, 2));
process.exit();
}
var timeTakes = function (range) {
function sumRange(n) {return (n+1) * n /2}
var ret = sumRange(range[1]);
if (range[0] > 1) {
ret = ret - sumRange(range[0] - 1)
}
var stopping = (range[1] - range[0]) * 0.5;
return ret + stopping;
}
/* ---------------------------------- TEST ---------------------------------- */
console.log("Test will take ~%s seconds", timeTakes(testRange));
(function oneTest(n /* seconds */, results) {
if (n >= testRange[1]) {
// show_results_and_exit(results);
console.log(JSON.stringify(results, undefined, 2));
oneTest(testRange[0], []);
return;
}
var args = start_args;
if (n % 2 == 0) {args = args.concat(['--fg'])}
if (n % 3 == 0) {args = args.concat(['--net'])}
var result = {args: args, alive_for: n};
results.push(result);
console.log("\nLaunching `%s` with `%s` for %d seconds",
rippledPath, JSON.stringify(args), n);
rippled = start(args);
console.log("Rippled pid: %d", rippled.pid);
// defaults
var b4StopSent = false;
var stopSent = false;
var stop_took = null;
rippled.once('exit', function(){
if (!stopSent && !b4StopSent) {
console.warn('\nRippled exited itself b4 stop issued');
process.exit();
};
// The io handles close AFTER exit, may have implications for
// `stdio:'inherit'` option to `child.spawn`.
rippled.once('close', function() {
result.stop_took = (+new Date() - stop_took) / 1000; // seconds
console.log("Stopping after %d seconds took %s seconds",
n, result.stop_took);
oneTest(n+1, results);
});
});
secs_l8r(n, function(){
console.log("Stopping rippled after %d seconds", n);
// possible race here ?
// seems highly unlikely, but I was having issues at one point
b4StopSent=true;
stop_took = (+new Date());
// when does `exit` actually get sent?
stop();
stopSent=true;
// Sometimes we want to attach with a debugger.
if (process.env.ABORT_TESTS_ON_STALL != null) {
// We wait 30 seconds, and if it hasn't stopped, we abort the process
secs_l8r(30, function() {
if (result.stop_took == null) {
console.log("rippled has stalled");
process.exit();
};
});
}
})
}(testRange[0], []));

View File

@@ -1,119 +0,0 @@
/**
* bin/update_bintypes.js
*
* This unholy abomination of a script generates the JavaScript file
* src/js/bintypes.js from various parts of the C++ source code.
*
* This should *NOT* be part of any automatic build process unless the C++
* source data are brought into a more easily parseable format. Until then,
* simply run this script manually and fix as needed.
*/
// XXX: Process LedgerFormats.(h|cpp) as well.
var filenameProto = __dirname + '/../src/cpp/ripple/SerializeProto.h',
filenameTxFormatsH = __dirname + '/../src/cpp/ripple/TransactionFormats.h',
filenameTxFormats = __dirname + '/../src/cpp/ripple/TransactionFormats.cpp';
var fs = require('fs');
var output = [];
// Stage 1: Get the field types and codes from SerializeProto.h
var types = {},
fields = {};
String(fs.readFileSync(filenameProto)).split('\n').forEach(function (line) {
line = line.replace(/^\s+|\s+$/g, '').replace(/\s+/g, '');
if (!line.length || line.slice(0, 2) === '//' || line.slice(-1) !== ')') return;
var tmp = line.slice(0, -1).split('('),
type = tmp[0],
opts = tmp[1].split(',');
if (type === 'TYPE') types[opts[1]] = [opts[0], +opts[2]];
else if (type === 'FIELD') fields[opts[0]] = [types[opts[1]][0], +opts[2]];
});
output.push('var ST = require("./serializedtypes");');
output.push('');
output.push('var REQUIRED = exports.REQUIRED = 0,');
output.push(' OPTIONAL = exports.OPTIONAL = 1,');
output.push(' DEFAULT = exports.DEFAULT = 2;');
output.push('');
function pad(s, n) { while (s.length < n) s += ' '; return s; }
function padl(s, n) { while (s.length < n) s = ' '+s; return s; }
Object.keys(types).forEach(function (type) {
output.push(pad('ST.'+types[type][0]+'.id', 25) + ' = '+types[type][1]+';');
});
output.push('');
// Stage 2: Get the transaction type IDs from TransactionFormats.h
var ttConsts = {};
String(fs.readFileSync(filenameTxFormatsH)).split('\n').forEach(function (line) {
var regex = /tt([A-Z_]+)\s+=\s+([0-9-]+)/;
var match = line.match(regex);
if (match) ttConsts[match[1]] = +match[2];
});
// Stage 3: Get the transaction formats from TransactionFormats.cpp
var base = [],
sections = [],
current = base;
String(fs.readFileSync(filenameTxFormats)).split('\n').forEach(function (line) {
line = line.replace(/^\s+|\s+$/g, '').replace(/\s+/g, '');
var d_regex = /DECLARE_TF\(([A-Za-z]+),tt([A-Z_]+)/;
var d_match = line.match(d_regex);
var s_regex = /SOElement\(sf([a-z]+),SOE_(REQUIRED|OPTIONAL|DEFAULT)/i;
var s_match = line.match(s_regex);
if (d_match) sections.push(current = [d_match[1], ttConsts[d_match[2]]]);
else if (s_match) current.push([s_match[1], s_match[2]]);
});
function removeFinalComma(arr) {
arr[arr.length-1] = arr[arr.length-1].slice(0, -1);
}
output.push('var base = [');
base.forEach(function (field) {
var spec = fields[field[0]];
output.push(' [ '+
pad("'"+field[0]+"'", 21)+', '+
pad(field[1], 8)+', '+
padl(""+spec[1], 2)+', '+
'ST.'+pad(spec[0], 3)+
' ],');
});
removeFinalComma(output);
output.push('];');
output.push('');
output.push('exports.tx = {');
sections.forEach(function (section) {
var name = section.shift(),
ttid = section.shift();
output.push(' '+name+': ['+ttid+'].concat(base, [');
section.forEach(function (field) {
var spec = fields[field[0]];
output.push(' [ '+
pad("'"+field[0]+"'", 21)+', '+
pad(field[1], 8)+', '+
padl(""+spec[1], 2)+', '+
'ST.'+pad(spec[0], 3)+
' ],');
});
removeFinalComma(output);
output.push(' ]),');
});
removeFinalComma(output);
output.push('};');
output.push('');
console.log(output.join('\n'));

View File

@@ -396,8 +396,8 @@
# true - enables compression
# false - disables compression [default].
#
# The rippled server can save bandwidth by compressing its peer-to-peer communications,
# at a cost of greater CPU usage. If you enable link compression,
# The rippled server can save bandwidth by compressing its peer-to-peer communications,
# at a cost of greater CPU usage. If you enable link compression,
# the server automatically compresses communications with peer servers
# that also have link compression enabled.
# https://xrpl.org/enable-link-compression.html
@@ -1011,7 +1011,7 @@
# that rippled is still in sync with the network,
# and that the validated ledger is less than
# 'age_threshold_seconds' old. If not, then continue
# sleeping for this number of seconds and
# sleeping for this number of seconds and
# checking until healthy.
# Default is 5.
#
@@ -1113,7 +1113,7 @@
# page_size Valid values: integer (MUST be power of 2 between 512 and 65536)
# The default is 4096 bytes. This setting determines
# the size of a page in the transaction.db file.
# See https://www.sqlite.org/pragma.html#pragma_page_size
# See https://www.sqlite.org/pragma.html#pragma_page_size
# for more details about the available options.
#
# journal_size_limit Valid values: integer

View File

@@ -26,7 +26,7 @@
#
# Examples:
# https://vl.ripple.com
# https://vl.xrplf.org
# https://unl.xrplf.org
# http://127.0.0.1:8000
# file:///etc/opt/ripple/vl.txt
#

View File

@@ -98,6 +98,17 @@
# 2024-04-03, Bronek Kozicki
# - add support for output formats: jacoco, clover, lcov
#
# 2025-05-12, Jingchen Wu
# - add -fprofile-update=atomic to ensure atomic profile generation
#
# 2025-08-28, Bronek Kozicki
# - fix "At least one COMMAND must be given" CMake warning from policy CMP0175
#
# 2025-09-03, Jingchen Wu
# - remove the unused function append_coverage_compiler_flags and append_coverage_compiler_flags_to_target
# - add a new function add_code_coverage_to_target
# - remove some unused code
#
# USAGE:
#
# 1. Copy this file into your cmake modules path.
@@ -106,10 +117,8 @@
# using a CMake option() to enable it just optionally):
# include(CodeCoverage)
#
# 3. Append necessary compiler flags for all supported source files:
# append_coverage_compiler_flags()
# Or for specific target:
# append_coverage_compiler_flags_to_target(YOUR_TARGET_NAME)
# 3. Append necessary compiler flags and linker flags for all supported source files:
# add_code_coverage_to_target(<target> <PRIVATE|PUBLIC|INTERFACE>)
#
# 3.a (OPTIONAL) Set appropriate optimization flags, e.g. -O0, -O1 or -Og
#
@@ -198,55 +207,69 @@ endforeach()
set(COVERAGE_COMPILER_FLAGS "-g --coverage"
CACHE INTERNAL "")
set(COVERAGE_CXX_COMPILER_FLAGS "")
set(COVERAGE_C_COMPILER_FLAGS "")
set(COVERAGE_CXX_LINKER_FLAGS "")
set(COVERAGE_C_LINKER_FLAGS "")
if(CMAKE_CXX_COMPILER_ID MATCHES "(GNU|Clang)")
include(CheckCXXCompilerFlag)
include(CheckCCompilerFlag)
include(CheckLinkerFlag)
set(COVERAGE_CXX_COMPILER_FLAGS ${COVERAGE_COMPILER_FLAGS})
set(COVERAGE_C_COMPILER_FLAGS ${COVERAGE_COMPILER_FLAGS})
set(COVERAGE_CXX_LINKER_FLAGS ${COVERAGE_COMPILER_FLAGS})
set(COVERAGE_C_LINKER_FLAGS ${COVERAGE_COMPILER_FLAGS})
check_cxx_compiler_flag(-fprofile-abs-path HAVE_cxx_fprofile_abs_path)
if(HAVE_cxx_fprofile_abs_path)
set(COVERAGE_CXX_COMPILER_FLAGS "${COVERAGE_COMPILER_FLAGS} -fprofile-abs-path")
set(COVERAGE_CXX_COMPILER_FLAGS "${COVERAGE_CXX_COMPILER_FLAGS} -fprofile-abs-path")
endif()
include(CheckCCompilerFlag)
check_c_compiler_flag(-fprofile-abs-path HAVE_c_fprofile_abs_path)
if(HAVE_c_fprofile_abs_path)
set(COVERAGE_C_COMPILER_FLAGS "${COVERAGE_COMPILER_FLAGS} -fprofile-abs-path")
set(COVERAGE_C_COMPILER_FLAGS "${COVERAGE_C_COMPILER_FLAGS} -fprofile-abs-path")
endif()
endif()
set(CMAKE_Fortran_FLAGS_COVERAGE
${COVERAGE_COMPILER_FLAGS}
CACHE STRING "Flags used by the Fortran compiler during coverage builds."
FORCE )
set(CMAKE_CXX_FLAGS_COVERAGE
${COVERAGE_COMPILER_FLAGS}
CACHE STRING "Flags used by the C++ compiler during coverage builds."
FORCE )
set(CMAKE_C_FLAGS_COVERAGE
${COVERAGE_COMPILER_FLAGS}
CACHE STRING "Flags used by the C compiler during coverage builds."
FORCE )
set(CMAKE_EXE_LINKER_FLAGS_COVERAGE
""
CACHE STRING "Flags used for linking binaries during coverage builds."
FORCE )
set(CMAKE_SHARED_LINKER_FLAGS_COVERAGE
""
CACHE STRING "Flags used by the shared libraries linker during coverage builds."
FORCE )
mark_as_advanced(
CMAKE_Fortran_FLAGS_COVERAGE
CMAKE_CXX_FLAGS_COVERAGE
CMAKE_C_FLAGS_COVERAGE
CMAKE_EXE_LINKER_FLAGS_COVERAGE
CMAKE_SHARED_LINKER_FLAGS_COVERAGE )
check_linker_flag(CXX -fprofile-abs-path HAVE_cxx_linker_fprofile_abs_path)
if(HAVE_cxx_linker_fprofile_abs_path)
set(COVERAGE_CXX_LINKER_FLAGS "${COVERAGE_CXX_LINKER_FLAGS} -fprofile-abs-path")
endif()
check_linker_flag(C -fprofile-abs-path HAVE_c_linker_fprofile_abs_path)
if(HAVE_c_linker_fprofile_abs_path)
set(COVERAGE_C_LINKER_FLAGS "${COVERAGE_C_LINKER_FLAGS} -fprofile-abs-path")
endif()
check_cxx_compiler_flag(-fprofile-update=atomic HAVE_cxx_fprofile_update)
if(HAVE_cxx_fprofile_update)
set(COVERAGE_CXX_COMPILER_FLAGS "${COVERAGE_CXX_COMPILER_FLAGS} -fprofile-update=atomic")
endif()
check_c_compiler_flag(-fprofile-update=atomic HAVE_c_fprofile_update)
if(HAVE_c_fprofile_update)
set(COVERAGE_C_COMPILER_FLAGS "${COVERAGE_C_COMPILER_FLAGS} -fprofile-update=atomic")
endif()
check_linker_flag(CXX -fprofile-update=atomic HAVE_cxx_linker_fprofile_update)
if(HAVE_cxx_linker_fprofile_update)
set(COVERAGE_CXX_LINKER_FLAGS "${COVERAGE_CXX_LINKER_FLAGS} -fprofile-update=atomic")
endif()
check_linker_flag(C -fprofile-update=atomic HAVE_c_linker_fprofile_update)
if(HAVE_c_linker_fprofile_update)
set(COVERAGE_C_LINKER_FLAGS "${COVERAGE_C_LINKER_FLAGS} -fprofile-update=atomic")
endif()
endif()
get_property(GENERATOR_IS_MULTI_CONFIG GLOBAL PROPERTY GENERATOR_IS_MULTI_CONFIG)
if(NOT (CMAKE_BUILD_TYPE STREQUAL "Debug" OR GENERATOR_IS_MULTI_CONFIG))
message(WARNING "Code coverage results with an optimised (non-Debug) build may be misleading")
endif() # NOT (CMAKE_BUILD_TYPE STREQUAL "Debug" OR GENERATOR_IS_MULTI_CONFIG)
if(CMAKE_C_COMPILER_ID STREQUAL "GNU" OR CMAKE_Fortran_COMPILER_ID STREQUAL "GNU")
link_libraries(gcov)
endif()
# Defines a target for running and collection code coverage information
# Builds dependencies, runs the given executable and outputs reports.
# NOTE! The executable should always have a ZERO as exit code otherwise
@@ -431,23 +454,24 @@ function(setup_target_for_coverage_gcovr)
# Show info where to find the report
add_custom_command(TARGET ${Coverage_NAME} POST_BUILD
COMMAND ;
COMMAND echo
COMMENT "Code coverage report saved in ${GCOVR_OUTPUT_FILE} formatted as ${Coverage_FORMAT}"
)
endfunction() # setup_target_for_coverage_gcovr
function(append_coverage_compiler_flags)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${COVERAGE_COMPILER_FLAGS}" PARENT_SCOPE)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${COVERAGE_COMPILER_FLAGS}" PARENT_SCOPE)
set(CMAKE_Fortran_FLAGS "${CMAKE_Fortran_FLAGS} ${COVERAGE_COMPILER_FLAGS}" PARENT_SCOPE)
message(STATUS "Appending code coverage compiler flags: ${COVERAGE_COMPILER_FLAGS}")
endfunction() # append_coverage_compiler_flags
function(add_code_coverage_to_target name scope)
separate_arguments(COVERAGE_CXX_COMPILER_FLAGS NATIVE_COMMAND "${COVERAGE_CXX_COMPILER_FLAGS}")
separate_arguments(COVERAGE_C_COMPILER_FLAGS NATIVE_COMMAND "${COVERAGE_C_COMPILER_FLAGS}")
separate_arguments(COVERAGE_CXX_LINKER_FLAGS NATIVE_COMMAND "${COVERAGE_CXX_LINKER_FLAGS}")
separate_arguments(COVERAGE_C_LINKER_FLAGS NATIVE_COMMAND "${COVERAGE_C_LINKER_FLAGS}")
# Setup coverage for specific library
function(append_coverage_compiler_flags_to_target name)
separate_arguments(_flag_list NATIVE_COMMAND "${COVERAGE_COMPILER_FLAGS}")
target_compile_options(${name} PRIVATE ${_flag_list})
if(CMAKE_C_COMPILER_ID STREQUAL "GNU" OR CMAKE_Fortran_COMPILER_ID STREQUAL "GNU")
target_link_libraries(${name} PRIVATE gcov)
endif()
endfunction()
# Add compiler options to the target
target_compile_options(${name} ${scope}
$<$<COMPILE_LANGUAGE:CXX>:${COVERAGE_CXX_COMPILER_FLAGS}>
$<$<COMPILE_LANGUAGE:C>:${COVERAGE_C_COMPILER_FLAGS}>)
target_link_libraries (${name} ${scope}
$<$<LINK_LANGUAGE:CXX>:${COVERAGE_CXX_LINKER_FLAGS} gcov>
$<$<LINK_LANGUAGE:C>:${COVERAGE_C_LINKER_FLAGS} gcov>
)
endfunction() # add_code_coverage_to_target

View File

@@ -16,13 +16,16 @@ set(CMAKE_CXX_EXTENSIONS OFF)
target_compile_definitions (common
INTERFACE
$<$<CONFIG:Debug>:DEBUG _DEBUG>
$<$<AND:$<BOOL:${profile}>,$<NOT:$<BOOL:${assert}>>>:NDEBUG>)
# ^^^^ NOTE: CMAKE release builds already have NDEBUG
# defined, so no need to add it explicitly except for
# this special case of (profile ON) and (assert OFF)
# -- presumably this is because we don't want profile
# builds asserting unless asserts were specifically
# requested
#[===[
NOTE: CMAKE release builds already have NDEBUG defined, so no need to add it
explicitly except for the special case of (profile ON) and (assert OFF).
Presumably this is because we don't want profile builds asserting unless
asserts were specifically requested.
]===]
$<$<AND:$<BOOL:${profile}>,$<NOT:$<BOOL:${assert}>>>:NDEBUG>
# TODO: Remove once we have migrated functions from OpenSSL 1.x to 3.x.
OPENSSL_SUPPRESS_DEPRECATED
)
if (MSVC)
# remove existing exception flag since we set it to -EHa
@@ -90,28 +93,16 @@ if (MSVC)
-errorreport:none
-machine:X64)
else ()
# HACK : because these need to come first, before any warning demotion
string (APPEND CMAKE_CXX_FLAGS " -Wall -Wdeprecated")
if (wextra)
string (APPEND CMAKE_CXX_FLAGS " -Wextra -Wno-unused-parameter")
endif ()
# not MSVC
target_compile_options (common
INTERFACE
-Wall
-Wdeprecated
$<$<BOOL:${is_clang}>:-Wno-deprecated-declarations>
$<$<BOOL:${wextra}>:-Wextra -Wno-unused-parameter>
$<$<BOOL:${werr}>:-Werror>
$<$<COMPILE_LANGUAGE:CXX>:
-frtti
-Wnon-virtual-dtor
>
-Wno-sign-compare
-Wno-char-subscripts
-Wno-format
-Wno-unused-local-typedefs
-fstack-protector
$<$<BOOL:${is_gcc}>:
-Wno-unused-but-set-variable
-Wno-deprecated
>
-Wno-sign-compare
-Wno-unused-but-set-variable
$<$<NOT:$<CONFIG:Debug>>:-fno-strict-aliasing>
# tweak gcc optimization for debug
$<$<AND:$<BOOL:${is_gcc}>,$<CONFIG:Debug>>:-O0>

View File

@@ -99,9 +99,24 @@ target_link_libraries(xrpl.libxrpl.protocol PUBLIC
add_module(xrpl resource)
target_link_libraries(xrpl.libxrpl.resource PUBLIC xrpl.libxrpl.protocol)
# Level 06
add_module(xrpl net)
target_link_libraries(xrpl.libxrpl.net PUBLIC
xrpl.libxrpl.basics
xrpl.libxrpl.json
xrpl.libxrpl.protocol
xrpl.libxrpl.resource
)
add_module(xrpl server)
target_link_libraries(xrpl.libxrpl.server PUBLIC xrpl.libxrpl.protocol)
add_module(xrpl ledger)
target_link_libraries(xrpl.libxrpl.ledger PUBLIC
xrpl.libxrpl.basics
xrpl.libxrpl.json
xrpl.libxrpl.protocol
)
add_library(xrpl.libxrpl)
set_target_properties(xrpl.libxrpl PROPERTIES OUTPUT_NAME xrpl)
@@ -121,6 +136,8 @@ target_link_modules(xrpl PUBLIC
protocol
resource
server
net
ledger
)
# All headers in libxrpl are in modules.

View File

@@ -33,6 +33,8 @@ setup_target_for_coverage_gcovr(
FORMAT ${coverage_format}
EXECUTABLE rippled
EXECUTABLE_ARGS --unittest$<$<BOOL:${coverage_test}>:=${coverage_test}> --unittest-jobs ${coverage_test_parallelism} --quiet --unittest-log
EXCLUDE "src/test" "include/xrpl/beast/test" "include/xrpl/beast/unit_test" "${CMAKE_BINARY_DIR}/pb-xrpl.libpb"
EXCLUDE "src/test" "src/tests" "include/xrpl/beast/test" "include/xrpl/beast/unit_test" "${CMAKE_BINARY_DIR}/pb-xrpl.libpb"
DEPENDENCIES rippled
)
add_code_coverage_to_target(opts INTERFACE)

View File

@@ -53,9 +53,9 @@ set(download_script "${CMAKE_BINARY_DIR}/docs/download-cppreference.cmake")
file(WRITE
"${download_script}"
"file(DOWNLOAD \
http://upload.cppreference.com/mwiki/images/b/b2/html_book_20190607.zip \
https://github.com/PeterFeicht/cppreference-doc/releases/download/v20250209/html-book-20250209.zip \
${CMAKE_BINARY_DIR}/docs/cppreference.zip \
EXPECTED_HASH MD5=82b3a612d7d35a83e3cb1195a63689ab \
EXPECTED_HASH MD5=bda585f72fbca4b817b29a3d5746567b \
)\n \
execute_process( \
COMMAND \"${CMAKE_COMMAND}\" -E tar -xf cppreference.zip \

View File

@@ -18,7 +18,9 @@ install (
xrpl.libxrpl.json
xrpl.libxrpl.protocol
xrpl.libxrpl.resource
xrpl.libxrpl.ledger
xrpl.libxrpl.server
xrpl.libxrpl.net
xrpl.libxrpl
antithesis-sdk-cpp
EXPORT RippleExports

View File

@@ -28,15 +28,11 @@ target_compile_options (opts
$<$<AND:$<BOOL:${is_gcc}>,$<COMPILE_LANGUAGE:CXX>>:-Wsuggest-override>
$<$<BOOL:${is_gcc}>:-Wno-maybe-uninitialized>
$<$<BOOL:${perf}>:-fno-omit-frame-pointer>
$<$<AND:$<BOOL:${is_gcc}>,$<BOOL:${coverage}>>:-g --coverage -fprofile-abs-path>
$<$<AND:$<BOOL:${is_clang}>,$<BOOL:${coverage}>>:-g --coverage>
$<$<BOOL:${profile}>:-pg>
$<$<AND:$<BOOL:${is_gcc}>,$<BOOL:${profile}>>:-p>)
target_link_libraries (opts
INTERFACE
$<$<AND:$<BOOL:${is_gcc}>,$<BOOL:${coverage}>>:-g --coverage -fprofile-abs-path>
$<$<AND:$<BOOL:${is_clang}>,$<BOOL:${coverage}>>:-g --coverage>
$<$<BOOL:${profile}>:-pg>
$<$<AND:$<BOOL:${is_gcc}>,$<BOOL:${profile}>>:-p>)

View File

@@ -2,16 +2,6 @@
convenience variables and sanity checks
#]===================================================================]
include(ProcessorCount)
if (NOT ep_procs)
ProcessorCount(ep_procs)
if (ep_procs GREATER 1)
# never use more than half of cores for EP builds
math (EXPR ep_procs "${ep_procs} / 2")
message (STATUS "Using ${ep_procs} cores for ExternalProject builds.")
endif ()
endif ()
get_property(is_multiconfig GLOBAL PROPERTY GENERATOR_IS_MULTI_CONFIG)
set (CMAKE_CONFIGURATION_TYPES "Debug;Release" CACHE STRING "" FORCE)

View File

@@ -18,7 +18,7 @@ if(tests)
endif()
endif()
option(unity "Creates a build using UNITY support in cmake. This is the default" ON)
option(unity "Creates a build using UNITY support in cmake." OFF)
if(unity)
if(NOT is_ci)
set(CMAKE_UNITY_BUILD_BATCH_SIZE 15 CACHE STRING "")
@@ -118,7 +118,7 @@ option(beast_no_unit_test_inline
"Prevents unit test definitions from being inserted into global table"
OFF)
option(single_io_service_thread
"Restricts the number of threads calling io_service::run to one. \
"Restricts the number of threads calling io_context::run to one. \
This can be useful when debugging."
OFF)
option(boost_show_deprecated

View File

@@ -2,7 +2,6 @@ find_package(Boost 1.82 REQUIRED
COMPONENTS
chrono
container
context
coroutine
date_time
filesystem
@@ -15,22 +14,17 @@ find_package(Boost 1.82 REQUIRED
add_library(ripple_boost INTERFACE)
add_library(Ripple::boost ALIAS ripple_boost)
if(XCODE)
target_include_directories(ripple_boost BEFORE INTERFACE ${Boost_INCLUDE_DIRS})
target_compile_options(ripple_boost INTERFACE --system-header-prefix="boost/")
else()
target_include_directories(ripple_boost SYSTEM BEFORE INTERFACE ${Boost_INCLUDE_DIRS})
endif()
target_link_libraries(ripple_boost
INTERFACE
Boost::boost
Boost::headers
Boost::chrono
Boost::container
Boost::coroutine
Boost::date_time
Boost::filesystem
Boost::json
Boost::process
Boost::program_options
Boost::regex
Boost::system

41
cmake/xrpl_add_test.cmake Normal file
View File

@@ -0,0 +1,41 @@
include(isolate_headers)
function(xrpl_add_test name)
set(target ${PROJECT_NAME}.test.${name})
file(GLOB_RECURSE sources CONFIGURE_DEPENDS
"${CMAKE_CURRENT_SOURCE_DIR}/${name}/*.cpp"
"${CMAKE_CURRENT_SOURCE_DIR}/${name}.cpp"
)
add_executable(${target} EXCLUDE_FROM_ALL ${ARGN} ${sources})
isolate_headers(
${target}
"${CMAKE_SOURCE_DIR}"
"${CMAKE_SOURCE_DIR}/tests/${name}"
PRIVATE
)
# Make sure the test isn't optimized away in unity builds
set_target_properties(${target} PROPERTIES
UNITY_BUILD_MODE GROUP
UNITY_BUILD_BATCH_SIZE 0) # Adjust as needed
add_test(NAME ${target} COMMAND ${target})
set_tests_properties(
${target} PROPERTIES
FIXTURES_REQUIRED ${target}_fixture
)
add_test(
NAME ${target}.build
COMMAND
${CMAKE_COMMAND}
--build ${CMAKE_BINARY_DIR}
--config $<CONFIG>
--target ${target}
)
set_tests_properties(${target}.build PROPERTIES
FIXTURES_SETUP ${target}_fixture
)
endfunction()

56
conan.lock Normal file
View File

@@ -0,0 +1,56 @@
{
"version": "0.5",
"requires": [
"zlib/1.3.1#b8bc2603263cf7eccbd6e17e66b0ed76%1756234269.497",
"xxhash/0.8.3#681d36a0a6111fc56e5e45ea182c19cc%1756234289.683",
"sqlite3/3.49.1#8631739a4c9b93bd3d6b753bac548a63%1756234266.869",
"soci/4.0.3#a9f8d773cd33e356b5879a4b0564f287%1756234262.318",
"snappy/1.1.10#968fef506ff261592ec30c574d4a7809%1756234314.246",
"rocksdb/10.0.1#85537f46e538974d67da0c3977de48ac%1756234304.347",
"re2/20230301#dfd6e2bf050eb90ddd8729cfb4c844a4%1756234257.976",
"protobuf/3.21.12#d927114e28de9f4691a6bbcdd9a529d1%1756234251.614",
"openssl/3.6.0#89e8af1d4a21afcac0557079d23d8890%1759746682.365",
"nudb/2.0.9#c62cfd501e57055a7e0d8ee3d5e5427d%1756234237.107",
"lz4/1.10.0#59fc63cac7f10fbe8e05c7e62c2f3504%1756234228.999",
"libiconv/1.17#1e65319e945f2d31941a9d28cc13c058%1756223727.64",
"libbacktrace/cci.20210118#a7691bfccd8caaf66309df196790a5a1%1756230911.03",
"libarchive/3.8.1#5cf685686322e906cb42706ab7e099a8%1756234256.696",
"jemalloc/5.3.0#e951da9cf599e956cebc117880d2d9f8%1729241615.244",
"grpc/1.50.1#02291451d1e17200293a409410d1c4e1%1756234248.958",
"doctest/2.4.11#a4211dfc329a16ba9f280f9574025659%1756234220.819",
"date/3.0.4#f74bbba5a08fa388256688743136cb6f%1756234217.493",
"c-ares/1.34.5#b78b91e7cfb1f11ce777a285bbf169c6%1756234217.915",
"bzip2/1.0.8#00b4a4658791c1f06914e087f0e792f5%1756234261.716",
"boost/1.88.0#8852c0b72ce8271fb8ff7c53456d4983%1756223752.326",
"abseil/20230802.1#f0f91485b111dc9837a68972cb19ca7b%1756234220.907"
],
"build_requires": [
"zlib/1.3.1#b8bc2603263cf7eccbd6e17e66b0ed76%1756234269.497",
"strawberryperl/5.32.1.1#707032463aa0620fa17ec0d887f5fe41%1756234281.733",
"protobuf/3.21.12#d927114e28de9f4691a6bbcdd9a529d1%1756234251.614",
"nasm/2.16.01#31e26f2ee3c4346ecd347911bd126904%1756234232.901",
"msys2/cci.latest#5b73b10144f73cc5bfe0572ed9be39e1%1751977009.857",
"m4/1.4.19#b38ced39a01e31fef5435bc634461fd2%1700758725.451",
"cmake/3.31.8#dde3bde00bb843687e55aea5afa0e220%1756234232.89",
"b2/5.3.3#107c15377719889654eb9a162a673975%1756234226.28",
"automake/1.16.5#b91b7c384c3deaa9d535be02da14d04f%1755524470.56",
"autoconf/2.71#51077f068e61700d65bb05541ea1e4b0%1731054366.86"
],
"python_requires": [],
"overrides": {
"protobuf/3.21.12": [
null,
"protobuf/3.21.12"
],
"lz4/1.9.4": [
"lz4/1.10.0"
],
"boost/1.83.0": [
"boost/1.88.0"
],
"sqlite3/3.44.2": [
"sqlite3/3.49.1"
]
},
"config_requires": []
}

9
conan/global.conf Normal file
View File

@@ -0,0 +1,9 @@
# Global configuration for Conan. This is used to set the number of parallel
# downloads, uploads, and build jobs. The verbosity is set to verbose to
# provide more information during the build process.
core:non_interactive=True
core.download:parallel={{ os.cpu_count() }}
core.upload:parallel={{ os.cpu_count() }}
tools.build:jobs={{ (os.cpu_count() * 4/5) | int }}
tools.build:verbosity=verbose
tools.compilation:verbosity=verbose

34
conan/profiles/default Normal file
View File

@@ -0,0 +1,34 @@
{% set os = detect_api.detect_os() %}
{% set arch = detect_api.detect_arch() %}
{% set compiler, version, compiler_exe = detect_api.detect_default_compiler() %}
{% set compiler_version = version %}
{% if os == "Linux" %}
{% set compiler_version = detect_api.default_compiler_version(compiler, version) %}
{% endif %}
[settings]
os={{ os }}
arch={{ arch }}
build_type=Debug
compiler={{compiler}}
compiler.version={{ compiler_version }}
compiler.cppstd=20
{% if os == "Windows" %}
compiler.runtime=static
{% else %}
compiler.libcxx={{detect_api.detect_libcxx(compiler, version, compiler_exe)}}
{% endif %}
[conf]
{% if compiler == "clang" and compiler_version >= 19 %}
tools.build:cxxflags=['-Wno-missing-template-arg-list-after-template-kw']
{% endif %}
{% if compiler == "apple-clang" and compiler_version >= 17 %}
tools.build:cxxflags=['-Wno-missing-template-arg-list-after-template-kw']
{% endif %}
{% if compiler == "gcc" and compiler_version < 13 %}
tools.build:cxxflags=['-Wno-restrict']
{% endif %}
[tool_requires]
!cmake/*: cmake/[>=3 <4]

View File

@@ -1,4 +1,4 @@
from conan import ConanFile
from conan import ConanFile, __version__ as conan_version
from conan.tools.cmake import CMake, CMakeToolchain, cmake_layout
import re
@@ -24,18 +24,20 @@ class Xrpl(ConanFile):
}
requires = [
'date/3.0.3',
'grpc/1.50.1',
'libarchive/3.7.6',
'nudb/2.0.8',
'openssl/1.1.1v',
'libarchive/3.8.1',
'nudb/2.0.9',
'openssl/3.6.0',
'soci/4.0.3',
'xxhash/0.8.2',
'zlib/1.3.1',
]
test_requires = [
'doctest/2.4.11',
]
tool_requires = [
'protobuf/3.21.9',
'protobuf/3.21.12',
]
default_options = {
@@ -87,30 +89,36 @@ class Xrpl(ConanFile):
}
def set_version(self):
path = f'{self.recipe_folder}/src/libxrpl/protocol/BuildInfo.cpp'
regex = r'versionString\s?=\s?\"(.*)\"'
with open(path, 'r') as file:
matches = (re.search(regex, line) for line in file)
match = next(m for m in matches if m)
self.version = match.group(1)
if self.version is None:
path = f'{self.recipe_folder}/src/libxrpl/protocol/BuildInfo.cpp'
regex = r'versionString\s?=\s?\"(.*)\"'
with open(path, encoding='utf-8') as file:
matches = (re.search(regex, line) for line in file)
match = next(m for m in matches if m)
self.version = match.group(1)
def configure(self):
if self.settings.compiler == 'apple-clang':
self.options['boost'].visibility = 'global'
if self.settings.compiler in ['clang', 'gcc']:
self.options['boost'].without_cobalt = True
def requirements(self):
self.requires('boost/1.83.0', force=True)
# Conan 2 requires transitive headers to be specified
transitive_headers_opt = {'transitive_headers': True} if conan_version.split('.')[0] == '2' else {}
self.requires('boost/1.88.0', force=True, **transitive_headers_opt)
self.requires('date/3.0.4', **transitive_headers_opt)
self.requires('lz4/1.10.0', force=True)
self.requires('protobuf/3.21.9', force=True)
self.requires('sqlite3/3.47.0', force=True)
self.requires('protobuf/3.21.12', force=True)
self.requires('sqlite3/3.49.1', force=True)
if self.options.jemalloc:
self.requires('jemalloc/5.3.0')
if self.options.rocksdb:
self.requires('rocksdb/9.7.3')
self.requires('rocksdb/10.0.1')
self.requires('xxhash/0.8.3', **transitive_headers_opt)
exports_sources = (
'CMakeLists.txt',
'bin/getRippledInfo',
'cfg/*',
'cmake/*',
'external/*',
@@ -161,7 +169,18 @@ class Xrpl(ConanFile):
# `include/`, not `include/ripple/proto/`.
libxrpl.includedirs = ['include', 'include/ripple/proto']
libxrpl.requires = [
'boost::boost',
'boost::headers',
'boost::chrono',
'boost::container',
'boost::coroutine',
'boost::date_time',
'boost::filesystem',
'boost::json',
'boost::program_options',
'boost::process',
'boost::regex',
'boost::system',
'boost::thread',
'date::date',
'grpc::grpc++',
'libarchive::libarchive',

View File

@@ -30,7 +30,7 @@ the ledger (so the entire network has the same view). This will help the network
see which validators are **currently** unreliable, and adjust their quorum
calculation accordingly.
*Improving the liveness of the network is the main motivation for the negative UNL.*
_Improving the liveness of the network is the main motivation for the negative UNL._
### Targeted Faults
@@ -53,16 +53,17 @@ even if the number of remaining validators gets to 60%. Say we have a network
with 10 validators on the UNL and everything is operating correctly. The quorum
required for this network would be 8 (80% of 10). When validators fail, the
quorum required would be as low as 6 (60% of 10), which is the absolute
***minimum quorum***. We need the absolute minimum quorum to be strictly greater
**_minimum quorum_**. We need the absolute minimum quorum to be strictly greater
than 50% of the original UNL so that there cannot be two partitions of
well-behaved nodes headed in different directions. We arbitrarily choose 60% as
the minimum quorum to give a margin of safety.
Consider these events in the absence of negative UNL:
1. 1:00pm - validator1 fails, votes vs. quorum: 9 >= 8, we have quorum
1. 3:00pm - validator2 fails, votes vs. quorum: 8 >= 8, we have quorum
1. 5:00pm - validator3 fails, votes vs. quorum: 7 < 8, we dont have quorum
* **network cannot validate new ledgers with 3 failed validators**
- **network cannot validate new ledgers with 3 failed validators**
We're below 80% agreement, so new ledgers cannot be validated. This is how the
XRP Ledger operates today, but if the negative UNL was enabled, the events would
@@ -70,18 +71,20 @@ happen as follows. (Please note that the events below are from a simplified
version of our protocol.)
1. 1:00pm - validator1 fails, votes vs. quorum: 9 >= 8, we have quorum
1. 1:40pm - network adds validator1 to negative UNL, quorum changes to ceil(9 * 0.8), or 8
1. 1:40pm - network adds validator1 to negative UNL, quorum changes to ceil(9 \* 0.8), or 8
1. 3:00pm - validator2 fails, votes vs. quorum: 8 >= 8, we have quorum
1. 3:40pm - network adds validator2 to negative UNL, quorum changes to ceil(8 * 0.8), or 7
1. 3:40pm - network adds validator2 to negative UNL, quorum changes to ceil(8 \* 0.8), or 7
1. 5:00pm - validator3 fails, votes vs. quorum: 7 >= 7, we have quorum
1. 5:40pm - network adds validator3 to negative UNL, quorum changes to ceil(7 * 0.8), or 6
1. 5:40pm - network adds validator3 to negative UNL, quorum changes to ceil(7 \* 0.8), or 6
1. 7:00pm - validator4 fails, votes vs. quorum: 6 >= 6, we have quorum
* **network can still validate new ledgers with 4 failed validators**
- **network can still validate new ledgers with 4 failed validators**
## External Interactions
### Message Format Changes
This proposal will:
1. add a new pseudo-transaction type
1. add the negative UNL to the ledger data structure.
@@ -89,19 +92,20 @@ Any tools or systems that rely on the format of this data will have to be
updated.
### Amendment
This feature **will** need an amendment to activate.
## Design
This section discusses the following topics about the Negative UNL design:
* [Negative UNL protocol overview](#Negative-UNL-Protocol-Overview)
* [Validator reliability measurement](#Validator-Reliability-Measurement)
* [Format Changes](#Format-Changes)
* [Negative UNL maintenance](#Negative-UNL-Maintenance)
* [Quorum size calculation](#Quorum-Size-Calculation)
* [Filter validation messages](#Filter-Validation-Messages)
* [High level sequence diagram of code
- [Negative UNL protocol overview](#Negative-UNL-Protocol-Overview)
- [Validator reliability measurement](#Validator-Reliability-Measurement)
- [Format Changes](#Format-Changes)
- [Negative UNL maintenance](#Negative-UNL-Maintenance)
- [Quorum size calculation](#Quorum-Size-Calculation)
- [Filter validation messages](#Filter-Validation-Messages)
- [High level sequence diagram of code
changes](#High-Level-Sequence-Diagram-of-Code-Changes)
### Negative UNL Protocol Overview
@@ -114,9 +118,9 @@ with V in their UNL adjust the quorum and Vs validation message is not counte
when verifying if a ledger is fully validated. Vs flow of messages and network
interactions, however, will remain the same.
We define the ***effective UNL** = original UNL - negative UNL*, and the
***effective quorum*** as the quorum of the *effective UNL*. And we set
*effective quorum = Ceiling(80% * effective UNL)*.
We define the **\*effective UNL** = original UNL - negative UNL\*, and the
**_effective quorum_** as the quorum of the _effective UNL_. And we set
_effective quorum = Ceiling(80% _ effective UNL)\*.
### Validator Reliability Measurement
@@ -126,16 +130,16 @@ measure about its validators, but we have chosen ledger validation messages.
This is because every validator shall send one and only one signed validation
message per ledger. This keeps the measurement simple and removes
timing/clock-sync issues. A node will measure the percentage of agreeing
validation messages (*PAV*) received from each validator on the node's UNL. Note
validation messages (_PAV_) received from each validator on the node's UNL. Note
that the node will only count the validation messages that agree with its own
validations.
We define the **PAV** as the **P**ercentage of **A**greed **V**alidation
messages received for the last N ledgers, where N = 256 by default.
When the PAV drops below the ***low-water mark***, the validator is considered
When the PAV drops below the **_low-water mark_**, the validator is considered
unreliable, and is a candidate to be disabled by being added to the negative
UNL. A validator must have a PAV higher than the ***high-water mark*** to be
UNL. A validator must have a PAV higher than the **_high-water mark_** to be
re-enabled. The validator is re-enabled by removing it from the negative UNL. In
the implementation, we plan to set the low-water mark as 50% and the high-water
mark as 80%.
@@ -143,22 +147,24 @@ mark as 80%.
### Format Changes
The negative UNL component in a ledger contains three fields.
* ***NegativeUNL***: The current negative UNL, a list of unreliable validators.
* ***ToDisable***: The validator to be added to the negative UNL on the next
- **_NegativeUNL_**: The current negative UNL, a list of unreliable validators.
- **_ToDisable_**: The validator to be added to the negative UNL on the next
flag ledger.
* ***ToReEnable***: The validator to be removed from the negative UNL on the
- **_ToReEnable_**: The validator to be removed from the negative UNL on the
next flag ledger.
All three fields are optional. When the *ToReEnable* field exists, the
*NegativeUNL* field cannot be empty.
All three fields are optional. When the _ToReEnable_ field exists, the
_NegativeUNL_ field cannot be empty.
A new pseudo-transaction, ***UNLModify***, is added. It has three fields
* ***Disabling***: A flag indicating whether the modification is to disable or
A new pseudo-transaction, **_UNLModify_**, is added. It has three fields
- **_Disabling_**: A flag indicating whether the modification is to disable or
to re-enable a validator.
* ***Seq***: The ledger sequence number.
* ***Validator***: The validator to be disabled or re-enabled.
- **_Seq_**: The ledger sequence number.
- **_Validator_**: The validator to be disabled or re-enabled.
There would be at most one *disable* `UNLModify` and one *re-enable* `UNLModify`
There would be at most one _disable_ `UNLModify` and one _re-enable_ `UNLModify`
transaction per flag ledger. The full machinery is described further on.
### Negative UNL Maintenance
@@ -167,19 +173,19 @@ The negative UNL can only be modified on the flag ledgers. If a validator's
reliability status changes, it takes two flag ledgers to modify the negative
UNL. Let's see an example of the algorithm:
* Ledger seq = 100: A validator V goes offline.
* Ledger seq = 256: This is a flag ledger, and V's reliability measurement *PAV*
- Ledger seq = 100: A validator V goes offline.
- Ledger seq = 256: This is a flag ledger, and V's reliability measurement _PAV_
is lower than the low-water mark. Other validators add `UNLModify`
pseudo-transactions `{true, 256, V}` to the transaction set which goes through
the consensus. Then the pseudo-transaction is applied to the negative UNL
ledger component by setting `ToDisable = V`.
* Ledger seq = 257 ~ 511: The negative UNL ledger component is copied from the
- Ledger seq = 257 ~ 511: The negative UNL ledger component is copied from the
parent ledger.
* Ledger seq=512: This is a flag ledger, and the negative UNL is updated
- Ledger seq=512: This is a flag ledger, and the negative UNL is updated
`NegativeUNL = NegativeUNL + ToDisable`.
The negative UNL may have up to `MaxNegativeListed = floor(original UNL * 25%)`
validators. The 25% is because of 75% * 80% = 60%, where 75% = 100% - 25%, 80%
validators. The 25% is because of 75% \* 80% = 60%, where 75% = 100% - 25%, 80%
is the quorum of the effective UNL, and 60% is the absolute minimum quorum of
the original UNL. Adding more than 25% validators to the negative UNL does not
improve the liveness of the network, because adding more validators to the
@@ -187,52 +193,43 @@ negative UNL cannot lower the effective quorum.
The following is the detailed algorithm:
* **If** the ledger seq = x is a flag ledger
- **If** the ledger seq = x is a flag ledger
1. Compute `NegativeUNL = NegativeUNL + ToDisable - ToReEnable` if they
exist in the parent ledger
1. Compute `NegativeUNL = NegativeUNL + ToDisable - ToReEnable` if they
exist in the parent ledger
1. Try to find a candidate to disable if `sizeof NegativeUNL < MaxNegativeListed`
1. Try to find a candidate to disable if `sizeof NegativeUNL < MaxNegativeListed`
1. Find a validator V that has a _PAV_ lower than the low-water
mark, but is not in `NegativeUNL`.
1. Find a validator V that has a *PAV* lower than the low-water
mark, but is not in `NegativeUNL`.
1. If two or more are found, their public keys are XORed with the hash
of the parent ledger and the one with the lowest XOR result is chosen.
1. If V is found, create a `UNLModify` pseudo-transaction
`TxDisableValidator = {true, x, V}`
1. Try to find a candidate to re-enable if `sizeof NegativeUNL > 0`:
1. Find a validator U that is in `NegativeUNL` and has a _PAV_ higher
than the high-water mark.
1. If U is not found, try to find one in `NegativeUNL` but not in the
local _UNL_.
1. If two or more are found, their public keys are XORed with the hash
of the parent ledger and the one with the lowest XOR result is chosen.
1. If U is found, create a `UNLModify` pseudo-transaction
`TxReEnableValidator = {false, x, U}`
1. If two or more are found, their public keys are XORed with the hash
of the parent ledger and the one with the lowest XOR result is chosen.
1. If V is found, create a `UNLModify` pseudo-transaction
`TxDisableValidator = {true, x, V}`
1. Try to find a candidate to re-enable if `sizeof NegativeUNL > 0`:
1. Find a validator U that is in `NegativeUNL` and has a *PAV* higher
than the high-water mark.
1. If U is not found, try to find one in `NegativeUNL` but not in the
local *UNL*.
1. If two or more are found, their public keys are XORed with the hash
of the parent ledger and the one with the lowest XOR result is chosen.
1. If U is found, create a `UNLModify` pseudo-transaction
`TxReEnableValidator = {false, x, U}`
1. If any `UNLModify` pseudo-transactions are created, add them to the
transaction set. The transaction set goes through the consensus algorithm.
1. If have enough support, the `UNLModify` pseudo-transactions remain in the
transaction set agreed by the validators. Then the pseudo-transactions are
applied to the ledger:
1. If have `TxDisableValidator`, set `ToDisable=TxDisableValidator.V`.
Else clear `ToDisable`.
1. If have `TxReEnableValidator`, set
`ToReEnable=TxReEnableValidator.U`. Else clear `ToReEnable`.
* **Else** (not a flag ledger)
1. If any `UNLModify` pseudo-transactions are created, add them to the
transaction set. The transaction set goes through the consensus algorithm.
1. If have enough support, the `UNLModify` pseudo-transactions remain in the
transaction set agreed by the validators. Then the pseudo-transactions are
applied to the ledger:
1. Copy the negative UNL ledger component from the parent ledger
1. If have `TxDisableValidator`, set `ToDisable=TxDisableValidator.V`.
Else clear `ToDisable`.
1. If have `TxReEnableValidator`, set
`ToReEnable=TxReEnableValidator.U`. Else clear `ToReEnable`.
- **Else** (not a flag ledger)
1. Copy the negative UNL ledger component from the parent ledger
The negative UNL is stored on each ledger because we don't know when a validator
may reconnect to the network. If the negative UNL was stored only on every flag
@@ -273,31 +270,26 @@ not counted when checking if the ledger is fully validated.
The diagram below is the sequence of one round of consensus. Classes and
components with non-trivial changes are colored green.
* The `ValidatorList` class is modified to compute the quorum of the effective
- The `ValidatorList` class is modified to compute the quorum of the effective
UNL.
* The `Validations` class provides an interface for querying the validation
- The `Validations` class provides an interface for querying the validation
messages from trusted validators.
* The `ConsensusAdaptor` component:
* The `RCLConsensus::Adaptor` class is modified for creating `UNLModify`
Pseudo-Transactions.
* The `Change` class is modified for applying `UNLModify`
Pseudo-Transactions.
* The `Ledger` class is modified for creating and adjusting the negative UNL
ledger component.
* The `LedgerMaster` class is modified for filtering out validation messages
from negative UNL validators when verifying if a ledger is fully
validated.
- The `ConsensusAdaptor` component:
- The `RCLConsensus::Adaptor` class is modified for creating `UNLModify`
Pseudo-Transactions.
- The `Change` class is modified for applying `UNLModify`
Pseudo-Transactions.
- The `Ledger` class is modified for creating and adjusting the negative UNL
ledger component.
- The `LedgerMaster` class is modified for filtering out validation messages
from negative UNL validators when verifying if a ledger is fully
validated.
![Sequence diagram](./negativeUNL_highLevel_sequence.png?raw=true "Negative UNL
Changes")
## Roads Not Taken
### Use a Mechanism Like Fee Voting to Process UNLModify Pseudo-Transactions
@@ -311,7 +303,7 @@ and different quorums for the same ledger. As a result, the network's safety is
impacted.
This updated version does not impact safety though operates a bit more slowly.
The negative UNL modifications in the *UNLModify* pseudo-transaction approved by
The negative UNL modifications in the _UNLModify_ pseudo-transaction approved by
the consensus will take effect at the next flag ledger. The extra time of the
256 ledgers should be enough for nodes to be in sync of the negative UNL
modifications.
@@ -334,29 +326,28 @@ expiration approach cannot be simply applied.
### Validator Reliability Measurement and Flag Ledger Frequency
If the ledger time is about 4.5 seconds and the low-water mark is 50%, then in
the worst case, it takes 48 minutes *((0.5 * 256 + 256 + 256) * 4.5 / 60 = 48)*
the worst case, it takes 48 minutes _((0.5 _ 256 + 256 + 256) _ 4.5 / 60 = 48)_
to put an offline validator on the negative UNL. We considered lowering the flag
ledger frequency so that the negative UNL can be more responsive. We also
considered decoupling the reliability measurement and flag ledger frequency to
be more flexible. In practice, however, their benefits are not clear.
## New Attack Vectors
A group of malicious validators may try to frame a reliable validator and put it
on the negative UNL. But they cannot succeed. Because:
1. A reliable validator sends a signed validation message every ledger. A
sufficient peer-to-peer network will propagate the validation messages to other
validators. The validators will decide if another validator is reliable or not
only by its local observation of the validation messages received. So an honest
validators vote on another validators reliability is accurate.
sufficient peer-to-peer network will propagate the validation messages to other
validators. The validators will decide if another validator is reliable or not
only by its local observation of the validation messages received. So an honest
validators vote on another validators reliability is accurate.
1. Given the votes are accurate, and one vote per validator, an honest validator
will not create a UNLModify transaction of a reliable validator.
will not create a UNLModify transaction of a reliable validator.
1. A validator can be added to a negative UNL only through a UNLModify
transaction.
transaction.
Assuming the group of malicious validators is less than the quorum, they cannot
frame a reliable validator.
@@ -365,32 +356,32 @@ frame a reliable validator.
The bullet points below briefly summarize the current proposal:
* The motivation of the negative UNL is to improve the liveness of the network.
- The motivation of the negative UNL is to improve the liveness of the network.
* The targeted faults are the ones frequently observed in the production
- The targeted faults are the ones frequently observed in the production
network.
* Validators propose negative UNL candidates based on their local measurements.
- Validators propose negative UNL candidates based on their local measurements.
* The absolute minimum quorum is 60% of the original UNL.
- The absolute minimum quorum is 60% of the original UNL.
* The format of the ledger is changed, and a new *UNLModify* pseudo-transaction
- The format of the ledger is changed, and a new _UNLModify_ pseudo-transaction
is added. Any tools or systems that rely on the format of these data will have
to be updated.
* The negative UNL can only be modified on the flag ledgers.
- The negative UNL can only be modified on the flag ledgers.
* At most one validator can be added to the negative UNL at a flag ledger.
- At most one validator can be added to the negative UNL at a flag ledger.
* At most one validator can be removed from the negative UNL at a flag ledger.
- At most one validator can be removed from the negative UNL at a flag ledger.
* If a validator's reliability status changes, it takes two flag ledgers to
- If a validator's reliability status changes, it takes two flag ledgers to
modify the negative UNL.
* The quorum is the larger of 80% of the effective UNL and 60% of the original
- The quorum is the larger of 80% of the effective UNL and 60% of the original
UNL.
* If a validator is on the negative UNL, its validation messages are ignored
- If a validator is on the negative UNL, its validation messages are ignored
when the local node verifies if a ledger is fully validated.
## FAQ
@@ -415,7 +406,7 @@ lower quorum size while keeping the network safe.
validator removed from the negative UNL? </h3>
A validators reliability is measured by other validators. If a validator
becomes unreliable, at a flag ledger, other validators propose *UNLModify*
becomes unreliable, at a flag ledger, other validators propose _UNLModify_
pseudo-transactions which vote the validator to add to the negative UNL during
the consensus session. If agreed, the validator is added to the negative UNL at
the next flag ledger. The mechanism of removing a validator from the negative
@@ -423,32 +414,32 @@ UNL is the same.
### Question: Given a negative UNL, what happens if the UNL changes?
Answer: Lets consider the cases:
Answer: Lets consider the cases:
1. A validator is added to the UNL, and it is already in the negative UNL. This
case could happen when not all the nodes have the same UNL. Note that the
negative UNL on the ledger lists unreliable nodes that are not necessarily the
validators for everyone.
1. A validator is added to the UNL, and it is already in the negative UNL. This
case could happen when not all the nodes have the same UNL. Note that the
negative UNL on the ledger lists unreliable nodes that are not necessarily the
validators for everyone.
In this case, the liveness is affected negatively. Because the minimum
quorum could be larger but the usable validators are not increased.
In this case, the liveness is affected negatively. Because the minimum
quorum could be larger but the usable validators are not increased.
1. A validator is removed from the UNL, and it is in the negative UNL.
1. A validator is removed from the UNL, and it is in the negative UNL.
In this case, the liveness is affected positively. Because the quorum could
be smaller but the usable validators are not reduced.
1. A validator is added to the UNL, and it is not in the negative UNL.
1. A validator is removed from the UNL, and it is not in the negative UNL.
1. A validator is added to the UNL, and it is not in the negative UNL.
1. A validator is removed from the UNL, and it is not in the negative UNL.
Case 3 and 4 are not affected by the negative UNL protocol.
### Question: Can we simply lower the quorum to 60% without the negative UNL?
### Question: Can we simply lower the quorum to 60% without the negative UNL?
Answer: No, because the negative UNL approach is safer.
First lets compare the two approaches intuitively, (1) the *negative UNL*
approach, and (2) *lower quorum*: simply lowering the quorum from 80% to 60%
First lets compare the two approaches intuitively, (1) the _negative UNL_
approach, and (2) _lower quorum_: simply lowering the quorum from 80% to 60%
without the negative UNL. The negative UNL approach uses consensus to come up
with a list of unreliable validators, which are then removed from the effective
UNL temporarily. With this approach, the list of unreliable validators is agreed
@@ -462,75 +453,75 @@ Next we compare the two approaches quantitatively with examples, and apply
Theorem 8 of [Analysis of the XRP Ledger Consensus
Protocol](https://arxiv.org/abs/1802.07242) paper:
*XRP LCP guarantees fork safety if **O<sub>i,j</sub> > n<sub>j</sub> / 2 +
_XRP LCP guarantees fork safety if **O<sub>i,j</sub> > n<sub>j</sub> / 2 +
n<sub>i</sub> q<sub>i</sub> + t<sub>i,j</sub>** for every pair of nodes
P<sub>i</sub>, P<sub>j</sub>,*
P<sub>i</sub>, P<sub>j</sub>,_
where *O<sub>i,j</sub>* is the overlapping requirement, n<sub>j</sub> and
where _O<sub>i,j</sub>_ is the overlapping requirement, n<sub>j</sub> and
n<sub>i</sub> are UNL sizes, q<sub>i</sub> is the quorum size of P<sub>i</sub>,
*t<sub>i,j</sub> = min(t<sub>i</sub>, t<sub>j</sub>, O<sub>i,j</sub>)*, and
_t<sub>i,j</sub> = min(t<sub>i</sub>, t<sub>j</sub>, O<sub>i,j</sub>)_, and
t<sub>i</sub> and t<sub>j</sub> are the number of faults can be tolerated by
P<sub>i</sub> and P<sub>j</sub>.
We denote *UNL<sub>i</sub>* as *P<sub>i</sub>'s UNL*, and *|UNL<sub>i</sub>|* as
the size of *P<sub>i</sub>'s UNL*.
We denote _UNL<sub>i</sub>_ as _P<sub>i</sub>'s UNL_, and _|UNL<sub>i</sub>|_ as
the size of _P<sub>i</sub>'s UNL_.
Assuming *|UNL<sub>i</sub>| = |UNL<sub>j</sub>|*, let's consider the following
Assuming _|UNL<sub>i</sub>| = |UNL<sub>j</sub>|_, let's consider the following
three cases:
1. With 80% quorum and 20% faults, *O<sub>i,j</sub> > 100% / 2 + 100% - 80% +
20% = 90%*. I.e. fork safety requires > 90% UNL overlaps. This is one of the
results in the analysis paper.
1. With 80% quorum and 20% faults, _O<sub>i,j</sub> > 100% / 2 + 100% - 80% +
20% = 90%_. I.e. fork safety requires > 90% UNL overlaps. This is one of the
results in the analysis paper.
1. If the quorum is 60%, the relationship between the overlapping requirement
and the faults that can be tolerated is *O<sub>i,j</sub> > 90% +
t<sub>i,j</sub>*. Under the same overlapping condition (i.e. 90%), to guarantee
the fork safety, the network cannot tolerate any faults. So under the same
overlapping condition, if the quorum is simply lowered, the network can tolerate
fewer faults.
1. If the quorum is 60%, the relationship between the overlapping requirement
and the faults that can be tolerated is _O<sub>i,j</sub> > 90% +
t<sub>i,j</sub>_. Under the same overlapping condition (i.e. 90%), to guarantee
the fork safety, the network cannot tolerate any faults. So under the same
overlapping condition, if the quorum is simply lowered, the network can tolerate
fewer faults.
1. With the negative UNL approach, we want to argue that the inequation
*O<sub>i,j</sub> > n<sub>j</sub> / 2 + n<sub>i</sub> q<sub>i</sub> +
t<sub>i,j</sub>* is always true to guarantee fork safety, while the negative UNL
protocol runs, i.e. the effective quorum is lowered without weakening the
network's fault tolerance. To make the discussion easier, we rewrite the
inequation as *O<sub>i,j</sub> > n<sub>j</sub> / 2 + (n<sub>i</sub>
q<sub>i</sub>) + min(t<sub>i</sub>, t<sub>j</sub>)*, where O<sub>i,j</sub> is
dropped from the definition of t<sub>i,j</sub> because *O<sub>i,j</sub> >
min(t<sub>i</sub>, t<sub>j</sub>)* always holds under the parameters we will
use. Assuming a validator V is added to the negative UNL, now let's consider the
4 cases:
1. With the negative UNL approach, we want to argue that the inequation
_O<sub>i,j</sub> > n<sub>j</sub> / 2 + n<sub>i</sub> q<sub>i</sub> +
t<sub>i,j</sub>_ is always true to guarantee fork safety, while the negative UNL
protocol runs, i.e. the effective quorum is lowered without weakening the
network's fault tolerance. To make the discussion easier, we rewrite the
inequation as _O<sub>i,j</sub> > n<sub>j</sub> / 2 + (n<sub>i</sub>
q<sub>i</sub>) + min(t<sub>i</sub>, t<sub>j</sub>)_, where O<sub>i,j</sub> is
dropped from the definition of t<sub>i,j</sub> because _O<sub>i,j</sub> >
min(t<sub>i</sub>, t<sub>j</sub>)_ always holds under the parameters we will
use. Assuming a validator V is added to the negative UNL, now let's consider the
4 cases:
1. V is not on UNL<sub>i</sub> nor UNL<sub>j</sub>
1. V is not on UNL<sub>i</sub> nor UNL<sub>j</sub>
The inequation holds because none of the variables change.
The inequation holds because none of the variables change.
1. V is on UNL<sub>i</sub> but not on UNL<sub>j</sub>
1. V is on UNL<sub>i</sub> but not on UNL<sub>j</sub>
The value of *(n<sub>i</sub> q<sub>i</sub>)* is smaller. The value of
*min(t<sub>i</sub>, t<sub>j</sub>)* could be smaller too. Other
variables do not change. Overall, the left side of the inequation does
not change, but the right side is smaller. So the inequation holds.
1. V is not on UNL<sub>i</sub> but on UNL<sub>j</sub>
The value of *(n<sub>i</sub> q<sub>i</sub>)* is smaller. The value of
*min(t<sub>i</sub>, t<sub>j</sub>)* could be smaller too. Other
variables do not change. Overall, the left side of the inequation does
not change, but the right side is smaller. So the inequation holds.
The value of *n<sub>j</sub> / 2* is smaller. The value of
*min(t<sub>i</sub>, t<sub>j</sub>)* could be smaller too. Other
variables do not change. Overall, the left side of the inequation does
not change, but the right side is smaller. So the inequation holds.
1. V is on both UNL<sub>i</sub> and UNL<sub>j</sub>
1. V is not on UNL<sub>i</sub> but on UNL<sub>j</sub>
The value of *O<sub>i,j</sub>* is reduced by 1. The values of
*n<sub>j</sub> / 2*, *(n<sub>i</sub> q<sub>i</sub>)*, and
*min(t<sub>i</sub>, t<sub>j</sub>)* are reduced by 0.5, 0.2, and 1
respectively. The right side is reduced by 1.7. Overall, the left side
of the inequation is reduced by 1, and the right side is reduced by 1.7.
So the inequation holds.
The value of *n<sub>j</sub> / 2* is smaller. The value of
*min(t<sub>i</sub>, t<sub>j</sub>)* could be smaller too. Other
variables do not change. Overall, the left side of the inequation does
not change, but the right side is smaller. So the inequation holds.
The inequation holds for all the cases. So with the negative UNL approach,
the network's fork safety is preserved, while the quorum is lowered that
increases the network's liveness.
1. V is on both UNL<sub>i</sub> and UNL<sub>j</sub>
The value of *O<sub>i,j</sub>* is reduced by 1. The values of
*n<sub>j</sub> / 2*, *(n<sub>i</sub> q<sub>i</sub>)*, and
*min(t<sub>i</sub>, t<sub>j</sub>)* are reduced by 0.5, 0.2, and 1
respectively. The right side is reduced by 1.7. Overall, the left side
of the inequation is reduced by 1, and the right side is reduced by 1.7.
So the inequation holds.
The inequation holds for all the cases. So with the negative UNL approach,
the network's fork safety is preserved, while the quorum is lowered that
increases the network's liveness.
<h3> Question: We have observed that occasionally a validator wanders off on its
own chain. How is this case handled by the negative UNL algorithm? </h3>
@@ -565,11 +556,11 @@ will be used after that. We want to see the test cases still pass with real
network delay. A test case specifies:
1. a UNL with different number of validators for different test cases,
1. a network with zero or more non-validator nodes,
1. a network with zero or more non-validator nodes,
1. a sequence of validator reliability change events (by killing/restarting
nodes, or by running modified rippled that does not send all validation
messages),
1. the correct outcomes.
1. the correct outcomes.
For all the test cases, the correct outcomes are verified by examining logs. We
will grep the log to see if the correct negative UNLs are generated, and whether
@@ -579,6 +570,7 @@ timing parameters of rippled will be changed to have faster ledger time. Most if
not all test cases do not need client transactions.
For example, the test cases for the prototype:
1. A 10-validator UNL.
1. The network does not have other nodes.
1. The validators will be started from the genesis. Once they start to produce
@@ -587,11 +579,11 @@ For example, the test cases for the prototype:
1. A sequence of events (or the lack of events) such as a killed validator is
added to the negative UNL.
#### Roads Not Taken: Test with Extended CSF
#### Roads Not Taken: Test with Extended CSF
We considered testing with the current unit test framework, specifically the
[Consensus Simulation
Framework](https://github.com/ripple/rippled/blob/develop/src/test/csf/README.md)
(CSF). However, the CSF currently can only test the generic consensus algorithm
as in the paper: [Analysis of the XRP Ledger Consensus
Protocol](https://arxiv.org/abs/1802.07242).
Protocol](https://arxiv.org/abs/1802.07242).

View File

@@ -5,8 +5,8 @@ skinparam roundcorner 20
skinparam maxmessagesize 160
actor "Rippled Start" as RS
participant "Timer" as T
participant "NetworkOPs" as NOP
participant "Timer" as T
participant "NetworkOPs" as NOP
participant "ValidatorList" as VL #lightgreen
participant "Consensus" as GC
participant "ConsensusAdaptor" as CA #lightgreen
@@ -20,7 +20,7 @@ VL -> NOP
NOP -> VL: update trusted validators
activate VL
VL -> VL: re-calculate quorum
hnote over VL#lightgreen: ignore negative listed validators\nwhen calculate quorum
hnote over VL#lightgreen: ignore negative listed validators\nwhen calculate quorum
VL -> NOP
deactivate VL
NOP -> GC: start round
@@ -36,14 +36,14 @@ activate GC
end
alt phase == OPEN
alt should close ledger
alt should close ledger
GC -> GC: phase = ESTABLISH
GC -> CA: onClose
activate CA
alt sqn%256==0
alt sqn%256==0
CA -[#green]> RM: <font color=green>getValidations
CA -[#green]> CA: <font color=green>create UNLModify Tx
hnote over CA#lightgreen: use validatations of the last 256 ledgers\nto figure out UNLModify Tx candidates.\nIf any, create UNLModify Tx, and add to TxSet.
CA -[#green]> CA: <font color=green>create UNLModify Tx
hnote over CA#lightgreen: use validatations of the last 256 ledgers\nto figure out UNLModify Tx candidates.\nIf any, create UNLModify Tx, and add to TxSet.
end
CA -> GC
GC -> CA: propose
@@ -61,14 +61,14 @@ else phase == ESTABLISH
CA -> CA : build LCL
hnote over CA #lightgreen: copy negative UNL from parent ledger
alt sqn%256==0
CA -[#green]> CA: <font color=green>Adjust negative UNL
CA -[#green]> CA: <font color=green>Adjust negative UNL
CA -[#green]> CA: <font color=green>apply UNLModify Tx
end
CA -> CA : validate and send validation message
activate NOP
CA -> NOP : end consensus and\n<b>begin next consensus round
deactivate NOP
deactivate CA
deactivate CA
hnote over RM: receive validations
end
else phase == ACCEPTED
@@ -76,4 +76,4 @@ else phase == ACCEPTED
end
deactivate GC
@enduml
@enduml

View File

@@ -82,7 +82,9 @@ pattern and the way coroutines are implemented, where every yield saves the spot
in the code where it left off and every resume jumps back to that spot.
### Sequence Diagram
![Sequence diagram](./ledger_replay_sequence.png?raw=true "A successful ledger replay")
### Class Diagram
![Class diagram](./ledger_replay_classes.png?raw=true "Ledger replay classes")

View File

@@ -4,7 +4,7 @@ class TimeoutCounter {
#app_ : Application&
}
TimeoutCounter o-- "1" Application
TimeoutCounter o-- "1" Application
': app_
Stoppable <.. Application
@@ -14,13 +14,13 @@ class Application {
-m_inboundLedgers : uptr<InboundLedgers>
}
Application *-- "1" LedgerReplayer
Application *-- "1" LedgerReplayer
': m_ledgerReplayer
Application *-- "1" InboundLedgers
Application *-- "1" InboundLedgers
': m_inboundLedgers
Stoppable <.. InboundLedgers
Application "1" --o InboundLedgers
Application "1" --o InboundLedgers
': app_
class InboundLedgers {
@@ -28,9 +28,9 @@ class InboundLedgers {
}
Stoppable <.. LedgerReplayer
InboundLedgers "1" --o LedgerReplayer
InboundLedgers "1" --o LedgerReplayer
': inboundLedgers_
Application "1" --o LedgerReplayer
Application "1" --o LedgerReplayer
': app_
class LedgerReplayer {
@@ -42,17 +42,17 @@ class LedgerReplayer {
-skipLists_ : hash_map<u256, wptr<SkipListAcquire>>
}
LedgerReplayer *-- LedgerReplayTask
LedgerReplayer *-- LedgerReplayTask
': tasks_
LedgerReplayer o-- LedgerDeltaAcquire
LedgerReplayer o-- LedgerDeltaAcquire
': deltas_
LedgerReplayer o-- SkipListAcquire
LedgerReplayer o-- SkipListAcquire
': skipLists_
TimeoutCounter <.. LedgerReplayTask
InboundLedgers "1" --o LedgerReplayTask
InboundLedgers "1" --o LedgerReplayTask
': inboundLedgers_
LedgerReplayer "1" --o LedgerReplayTask
LedgerReplayer "1" --o LedgerReplayTask
': replayer_
class LedgerReplayTask {
@@ -63,15 +63,15 @@ class LedgerReplayTask {
+addDelta(sptr<LedgerDeltaAcquire>)
}
LedgerReplayTask *-- "1" SkipListAcquire
LedgerReplayTask *-- "1" SkipListAcquire
': skipListAcquirer_
LedgerReplayTask *-- LedgerDeltaAcquire
LedgerReplayTask *-- LedgerDeltaAcquire
': deltas_
TimeoutCounter <.. SkipListAcquire
InboundLedgers "1" --o SkipListAcquire
InboundLedgers "1" --o SkipListAcquire
': inboundLedgers_
LedgerReplayer "1" --o SkipListAcquire
LedgerReplayer "1" --o SkipListAcquire
': replayer_
LedgerReplayTask --o SkipListAcquire : implicit via callback
@@ -83,9 +83,9 @@ class SkipListAcquire {
}
TimeoutCounter <.. LedgerDeltaAcquire
InboundLedgers "1" --o LedgerDeltaAcquire
InboundLedgers "1" --o LedgerDeltaAcquire
': inboundLedgers_
LedgerReplayer "1" --o LedgerDeltaAcquire
LedgerReplayer "1" --o LedgerDeltaAcquire
': replayer_
LedgerReplayTask --o LedgerDeltaAcquire : implicit via callback
@@ -95,4 +95,4 @@ class LedgerDeltaAcquire {
-replayer_ : LedgerReplayer&
-dataReadyCallbacks_ : vector<callback>
}
@enduml
@enduml

View File

@@ -38,7 +38,7 @@ deactivate lr
loop
lr -> lda : make_shared(ledgerId, ledgerSeq)
return delta
lr -> lrt : addDelta(delta)
lr -> lrt : addDelta(delta)
lrt -> lda : addDataCallback(callback)
return
return
@@ -62,7 +62,7 @@ deactivate peer
lr -> lda : processData(ledgerHeader, txns)
lda -> lda : notify()
note over lda: call the callbacks added by\naddDataCallback(callback).
lda -> lrt : callback(ledgerId)
lda -> lrt : callback(ledgerId)
lrt -> lrt : deltaReady(ledgerId)
lrt -> lrt : tryAdvance()
loop as long as child can be built
@@ -82,4 +82,4 @@ deactivate peer
deactivate peer
@enduml
@enduml

View File

@@ -16,5 +16,5 @@
## Function
- Minimize external dependencies
* Pass options in the ctor instead of using theConfig
* Use as few other classes as possible
- Pass options in the ctor instead of using theConfig
- Use as few other classes as possible

View File

@@ -1,18 +1,18 @@
# Coding Standards
Coding standards used here gradually evolve and propagate through
Coding standards used here gradually evolve and propagate through
code reviews. Some aspects are enforced more strictly than others.
## Rules
These rules only apply to our own code. We can't enforce any sort of
These rules only apply to our own code. We can't enforce any sort of
style on the external repositories and libraries we include. The best
guideline is to maintain the standards that are used in those libraries.
* Tab inserts 4 spaces. No tab characters.
* Braces are indented in the [Allman style][1].
* Modern C++ principles. No naked ```new``` or ```delete```.
* Line lengths limited to 80 characters. Exceptions limited to data and tables.
- Tab inserts 4 spaces. No tab characters.
- Braces are indented in the [Allman style][1].
- Modern C++ principles. No naked `new` or `delete`.
- Line lengths limited to 80 characters. Exceptions limited to data and tables.
## Guidelines
@@ -21,17 +21,17 @@ why you're doing it. Think, use common sense, and consider that this
your changes will probably need to be maintained long after you've
moved on to other projects.
* Use white space and blank lines to guide the eye and keep your intent clear.
* Put private data members at the top of a class, and the 6 public special
members immediately after, in the following order:
* Destructor
* Default constructor
* Copy constructor
* Copy assignment
* Move constructor
* Move assignment
* Don't over-inline by defining large functions within the class
declaration, not even for template classes.
- Use white space and blank lines to guide the eye and keep your intent clear.
- Put private data members at the top of a class, and the 6 public special
members immediately after, in the following order:
- Destructor
- Default constructor
- Copy constructor
- Copy assignment
- Move constructor
- Move assignment
- Don't over-inline by defining large functions within the class
declaration, not even for template classes.
## Formatting
@@ -39,44 +39,44 @@ The goal of source code formatting should always be to make things as easy to
read as possible. White space is used to guide the eye so that details are not
overlooked. Blank lines are used to separate code into "paragraphs."
* Always place a space before and after all binary operators,
- Always place a space before and after all binary operators,
especially assignments (`operator=`).
* The `!` operator should be preceded by a space, but not followed by one.
* The `~` operator should be preceded by a space, but not followed by one.
* The `++` and `--` operators should have no spaces between the operator and
- The `!` operator should be preceded by a space, but not followed by one.
- The `~` operator should be preceded by a space, but not followed by one.
- The `++` and `--` operators should have no spaces between the operator and
the operand.
* A space never appears before a comma, and always appears after a comma.
* Don't put spaces after a parenthesis. A typical member function call might
- A space never appears before a comma, and always appears after a comma.
- Don't put spaces after a parenthesis. A typical member function call might
look like this: `foobar (1, 2, 3);`
* In general, leave a blank line before an `if` statement.
* In general, leave a blank line after a closing brace `}`.
* Do not place code on the same line as any opening or
- In general, leave a blank line before an `if` statement.
- In general, leave a blank line after a closing brace `}`.
- Do not place code on the same line as any opening or
closing brace.
* Do not write `if` statements all-on-one-line. The exception to this is when
- Do not write `if` statements all-on-one-line. The exception to this is when
you've got a sequence of similar `if` statements, and are aligning them all
vertically to highlight their similarities.
* In an `if-else` statement, if you surround one half of the statement with
- In an `if-else` statement, if you surround one half of the statement with
braces, you also need to put braces around the other half, to match.
* When writing a pointer type, use this spacing: `SomeObject* myObject`.
- When writing a pointer type, use this spacing: `SomeObject* myObject`.
Technically, a more correct spacing would be `SomeObject *myObject`, but
it makes more sense for the asterisk to be grouped with the type name,
since being a pointer is part of the type, not the variable name. The only
time that this can lead to any problems is when you're declaring multiple
pointers of the same type in the same statement - which leads on to the next
rule:
* When declaring multiple pointers, never do so in a single statement, e.g.
- When declaring multiple pointers, never do so in a single statement, e.g.
`SomeObject* p1, *p2;` - instead, always split them out onto separate lines
and write the type name again, to make it quite clear what's going on, and
avoid the danger of missing out any vital asterisks.
* The previous point also applies to references, so always put the `&` next to
- The previous point also applies to references, so always put the `&` next to
the type rather than the variable, e.g. `void foo (Thing const& thing)`. And
don't put a space on both sides of the `*` or `&` - always put a space after
it, but never before it.
* The word `const` should be placed to the right of the thing that it modifies,
- The word `const` should be placed to the right of the thing that it modifies,
for consistency. For example `int const` refers to an int which is const.
`int const*` is a pointer to an int which is const. `int *const` is a const
pointer to an int.
* Always place a space in between the template angle brackets and the type
- Always place a space in between the template angle brackets and the type
name. Template code is already hard enough to read!
[1]: http://en.wikipedia.org/wiki/Indent_style#Allman_style

View File

@@ -31,7 +31,7 @@ and header under /opt/local/include:
$ scons clang profile-jemalloc=/opt/local
----------------------
---
## Using the jemalloc library from within the code
@@ -60,4 +60,3 @@ Linking against the jemalloc library will override
the system's default `malloc()` and related functions with jemalloc's
implementation. This is the case even if the code is not instrumented
to use jemalloc's specific API.

View File

@@ -7,7 +7,6 @@ Install these dependencies:
- [Doxygen](http://www.doxygen.nl): All major platforms have [official binary
distributions](http://www.doxygen.nl/download.html#srcbin), or you can
build from [source](http://www.doxygen.nl/download.html#srcbin).
- MacOS: We recommend installing via Homebrew: `brew install doxygen`.
The executable will be installed in `/usr/local/bin` which is already
in the default `PATH`.
@@ -21,18 +20,15 @@ Install these dependencies:
$ ln -s /Applications/Doxygen.app/Contents/Resources/doxygen /usr/local/bin/doxygen
```
- [PlantUML](http://plantuml.com):
- [PlantUML](http://plantuml.com):
1. Install a functioning Java runtime, if you don't already have one.
2. Download [`plantuml.jar`](http://sourceforge.net/projects/plantuml/files/plantuml.jar/download).
- [Graphviz](https://www.graphviz.org):
- Linux: Install from your package manager.
- Windows: Use an [official installer](https://graphviz.gitlab.io/_pages/Download/Download_windows.html).
- MacOS: Install via Homebrew: `brew install graphviz`.
## Docker
Instead of installing the above dependencies locally, you can use the official
@@ -40,14 +36,16 @@ build environment Docker image, which has all of them installed already.
1. Install [Docker](https://docs.docker.com/engine/installation/)
2. Pull the image:
```
sudo docker pull rippleci/rippled-ci-builder:2944b78d22db
```
3. Run the image from the project folder:
```
sudo docker run -v $PWD:/opt/rippled --rm rippleci/rippled-ci-builder:2944b78d22db
```
```
sudo docker pull rippleci/rippled-ci-builder:2944b78d22db
```
3. Run the image from the project folder:
```
sudo docker run -v $PWD:/opt/rippled --rm rippleci/rippled-ci-builder:2944b78d22db
```
## Build

14
docs/build/conan.md vendored
View File

@@ -5,7 +5,6 @@ we should first understand _why_ we use Conan,
and to understand that,
we need to understand how we use CMake.
### CMake
Technically, you don't need CMake to build this project.
@@ -33,9 +32,9 @@ Parameters include:
- where to find the compiler and linker
- where to find dependencies, e.g. libraries and headers
- how to link dependencies, e.g. any special compiler or linker flags that
need to be used with them, including preprocessor definitions
need to be used with them, including preprocessor definitions
- how to compile translation units, e.g. with optimizations, debug symbols,
position-independent code, etc.
position-independent code, etc.
- on Windows, which runtime library to link with
For some of these parameters, like the build system and compiler,
@@ -54,7 +53,6 @@ Most humans prefer to put them into a configuration file, once, that
CMake can read every time it is configured.
For CMake, that file is a [toolchain file][toolchain].
### Conan
These next few paragraphs on Conan are going to read much like the ones above
@@ -79,10 +77,10 @@ Those files include:
- A single toolchain file.
- For every dependency, a CMake [package configuration file][pcf],
[package version file][pvf], and for every build type, a package
targets file.
Together, these files implement version checking and define `IMPORTED`
targets for the dependencies.
[package version file][pvf], and for every build type, a package
targets file.
Together, these files implement version checking and define `IMPORTED`
targets for the dependencies.
The toolchain file itself amends the search path
([`CMAKE_PREFIX_PATH`][prefix_path]) so that [`find_package()`][find_package]

View File

@@ -2,8 +2,7 @@ We recommend two different methods to depend on libxrpl in your own [CMake][]
project.
Both methods add a CMake library target named `xrpl::libxrpl`.
## Conan requirement
## Conan requirement
The first method adds libxrpl as a [Conan][] requirement.
With this method, there is no need for a Git [submodule][].
@@ -48,7 +47,6 @@ cmake \
cmake --build . --parallel
```
## CMake subdirectory
The second method adds the [rippled][] project as a CMake
@@ -90,7 +88,6 @@ cmake \
cmake --build . --parallel
```
[add_subdirectory]: https://cmake.org/cmake/help/latest/command/add_subdirectory.html
[submodule]: https://git-scm.com/book/en/v2/Git-Tools-Submodules
[rippled]: https://github.com/ripple/rippled

View File

@@ -5,41 +5,39 @@ platforms: Linux, macOS, or Windows.
[BUILD.md]: ../../BUILD.md
## Linux
Package ecosystems vary across Linux distributions,
so there is no one set of instructions that will work for every Linux user.
These instructions are written for Ubuntu 22.04.
They are largely copied from the [script][1] used to configure our Docker
container for continuous integration.
That script handles many more responsibilities.
These instructions are just the bare minimum to build one configuration of
rippled.
You can check that codebase for other Linux distributions and versions.
If you cannot find yours there,
then we hope that these instructions can at least guide you in the right
direction.
The instructions below are written for Debian 12 (Bookworm).
```
apt update
apt install --yes curl git libssl-dev python3.10-dev python3-pip make g++-11 libprotobuf-dev protobuf-compiler
export GCC_RELEASE=12
sudo apt update
sudo apt install --yes gcc-${GCC_RELEASE} g++-${GCC_RELEASE} python3-pip \
python-is-python3 python3-venv python3-dev curl wget ca-certificates \
git build-essential cmake ninja-build libc6-dev
sudo pip install --break-system-packages conan
curl --location --remote-name \
"https://github.com/Kitware/CMake/releases/download/v3.25.1/cmake-3.25.1.tar.gz"
tar -xzf cmake-3.25.1.tar.gz
rm cmake-3.25.1.tar.gz
cd cmake-3.25.1
./bootstrap --parallel=$(nproc)
make --jobs $(nproc)
make install
cd ..
pip3 install 'conan<2'
sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-${GCC_RELEASE} 999
sudo update-alternatives --install \
/usr/bin/gcc gcc /usr/bin/gcc-${GCC_RELEASE} 100 \
--slave /usr/bin/g++ g++ /usr/bin/g++-${GCC_RELEASE} \
--slave /usr/bin/gcc-ar gcc-ar /usr/bin/gcc-ar-${GCC_RELEASE} \
--slave /usr/bin/gcc-nm gcc-nm /usr/bin/gcc-nm-${GCC_RELEASE} \
--slave /usr/bin/gcc-ranlib gcc-ranlib /usr/bin/gcc-ranlib-${GCC_RELEASE} \
--slave /usr/bin/gcov gcov /usr/bin/gcov-${GCC_RELEASE} \
--slave /usr/bin/gcov-tool gcov-tool /usr/bin/gcov-tool-${GCC_RELEASE} \
--slave /usr/bin/gcov-dump gcov-dump /usr/bin/gcov-dump-${GCC_RELEASE} \
--slave /usr/bin/lto-dump lto-dump /usr/bin/lto-dump-${GCC_RELEASE}
sudo update-alternatives --auto cc
sudo update-alternatives --auto gcc
```
[1]: https://github.com/thejohnfreeman/rippled-docker/blob/master/ubuntu-22.04/install.sh
If you use different Linux distribution, hope the instruction above can guide
you in the right direction. We try to maintain compatibility with all recent
compiler releases, so if you use a rolling distribution like e.g. Arch or CentOS
then there is a chance that everything will "just work".
## macOS
@@ -52,6 +50,33 @@ minimum required (see [BUILD.md][]).
clang --version
```
### Install Xcode Specific Version (Optional)
If you develop other applications using XCode you might be consistently updating to the newest version of Apple Clang.
This will likely cause issues building rippled. You may want to install a specific version of Xcode:
1. **Download Xcode**
- Visit [Apple Developer Downloads](https://developer.apple.com/download/more/)
- Sign in with your Apple Developer account
- Search for an Xcode version that includes **Apple Clang (Expected Version)**
- Download the `.xip` file
2. **Install and Configure Xcode**
```bash
# Extract the .xip file and rename for version management
# Example: Xcode_16.2.app
# Move to Applications directory
sudo mv Xcode_16.2.app /Applications/
# Set as default toolchain (persistent)
sudo xcode-select -s /Applications/Xcode_16.2.app/Contents/Developer
# Set as environment variable (temporary)
export DEVELOPER_DIR=/Applications/Xcode_16.2.app/Contents/Developer
```
The command line developer tools should include Git too:
```
@@ -71,10 +96,10 @@ and use it to install Conan:
brew update
brew install xz
brew install pyenv
pyenv install 3.10-dev
pyenv global 3.10-dev
pyenv install 3.11
pyenv global 3.11
eval "$(pyenv init -)"
pip install 'conan<2'
pip install 'conan'
```
Install CMake with Homebrew too:

42
docs/build/install.md vendored
View File

@@ -6,7 +6,6 @@ like CentOS.
Installing from source is an option for all platforms,
and the only supported option for installing custom builds.
## From source
From a source build, you can install rippled and libxrpl using CMake's
@@ -21,25 +20,23 @@ The default [prefix][1] is typically `/usr/local` on Linux and macOS and
[1]: https://cmake.org/cmake/help/latest/variable/CMAKE_INSTALL_PREFIX.html
## With the APT package manager
1. Update repositories:
1. Update repositories:
sudo apt update -y
2. Install utilities:
2. Install utilities:
sudo apt install -y apt-transport-https ca-certificates wget gnupg
3. Add Ripple's package-signing GPG key to your list of trusted keys:
3. Add Ripple's package-signing GPG key to your list of trusted keys:
sudo mkdir /usr/local/share/keyrings/
wget -q -O - "https://repos.ripple.com/repos/api/gpg/key/public" | gpg --dearmor > ripple-key.gpg
sudo mv ripple-key.gpg /usr/local/share/keyrings
4. Check the fingerprint of the newly-added key:
4. Check the fingerprint of the newly-added key:
gpg /usr/local/share/keyrings/ripple-key.gpg
@@ -51,37 +48,34 @@ The default [prefix][1] is typically `/usr/local` on Linux and macOS and
uid TechOps Team at Ripple <techops+rippled@ripple.com>
sub rsa3072 2019-02-14 [E] [expires: 2026-02-17]
In particular, make sure that the fingerprint matches. (In the above example, the fingerprint is on the third line, starting with `C001`.)
4. Add the appropriate Ripple repository for your operating system version:
5. Add the appropriate Ripple repository for your operating system version:
echo "deb [signed-by=/usr/local/share/keyrings/ripple-key.gpg] https://repos.ripple.com/repos/rippled-deb focal stable" | \
sudo tee -a /etc/apt/sources.list.d/ripple.list
The above example is appropriate for **Ubuntu 20.04 Focal Fossa**. For other operating systems, replace the word `focal` with one of the following:
- `jammy` for **Ubuntu 22.04 Jammy Jellyfish**
- `bionic` for **Ubuntu 18.04 Bionic Beaver**
- `bullseye` for **Debian 11 Bullseye**
- `buster` for **Debian 10 Buster**
If you want access to development or pre-release versions of `rippled`, use one of the following instead of `stable`:
- `unstable` - Pre-release builds ([`release` branch](https://github.com/ripple/rippled/tree/release))
- `nightly` - Experimental/development builds ([`develop` branch](https://github.com/ripple/rippled/tree/develop))
**Warning:** Unstable and nightly builds may be broken at any time. Do not use these builds for production servers.
5. Fetch the Ripple repository.
6. Fetch the Ripple repository.
sudo apt -y update
6. Install the `rippled` software package:
7. Install the `rippled` software package:
sudo apt -y install rippled
7. Check the status of the `rippled` service:
8. Check the status of the `rippled` service:
systemctl status rippled.service
@@ -89,24 +83,22 @@ The default [prefix][1] is typically `/usr/local` on Linux and macOS and
sudo systemctl start rippled.service
8. Optional: allow `rippled` to bind to privileged ports.
9. Optional: allow `rippled` to bind to privileged ports.
This allows you to serve incoming API requests on port 80 or 443. (If you want to do so, you must also update the config file's port settings.)
sudo setcap 'cap_net_bind_service=+ep' /opt/ripple/bin/rippled
## With the YUM package manager
1. Install the Ripple RPM repository:
1. Install the Ripple RPM repository:
Choose the appropriate RPM repository for the stability of releases you want:
- `stable` for the latest production release (`master` branch)
- `unstable` for pre-release builds (`release` branch)
- `nightly` for experimental/development builds (`develop` branch)
*Stable*
_Stable_
cat << REPOFILE | sudo tee /etc/yum.repos.d/ripple.repo
[ripple-stable]
@@ -118,7 +110,7 @@ The default [prefix][1] is typically `/usr/local` on Linux and macOS and
gpgkey=https://repos.ripple.com/repos/rippled-rpm/stable/repodata/repomd.xml.key
REPOFILE
*Unstable*
_Unstable_
cat << REPOFILE | sudo tee /etc/yum.repos.d/ripple.repo
[ripple-unstable]
@@ -130,7 +122,7 @@ The default [prefix][1] is typically `/usr/local` on Linux and macOS and
gpgkey=https://repos.ripple.com/repos/rippled-rpm/unstable/repodata/repomd.xml.key
REPOFILE
*Nightly*
_Nightly_
cat << REPOFILE | sudo tee /etc/yum.repos.d/ripple.repo
[ripple-nightly]
@@ -142,18 +134,18 @@ The default [prefix][1] is typically `/usr/local` on Linux and macOS and
gpgkey=https://repos.ripple.com/repos/rippled-rpm/nightly/repodata/repomd.xml.key
REPOFILE
2. Fetch the latest repo updates:
2. Fetch the latest repo updates:
sudo yum -y update
3. Install the new `rippled` package:
3. Install the new `rippled` package:
sudo yum install -y rippled
4. Configure the `rippled` service to start on boot:
4. Configure the `rippled` service to start on boot:
sudo systemctl enable rippled.service
5. Start the `rippled` service:
5. Start the `rippled` service:
sudo systemctl start rippled.service

View File

@@ -3,7 +3,7 @@
**This section is a work in progress!!**
Consensus is the task of reaching agreement within a distributed system in the
presence of faulty or even malicious participants. This document outlines the
presence of faulty or even malicious participants. This document outlines the
[XRP Ledger Consensus Algorithm](https://arxiv.org/abs/1802.07242)
as implemented in [rippled](https://github.com/ripple/rippled), but
focuses on its utility as a generic consensus algorithm independent of the
@@ -15,38 +15,38 @@ collectively trusted subnetworks.
## Distributed Agreement
A challenge for distributed systems is reaching agreement on changes in shared
state. For the Ripple network, the shared state is the current ledger--account
information, account balances, order books and other financial data. We will
state. For the Ripple network, the shared state is the current ledger--account
information, account balances, order books and other financial data. We will
refer to shared distributed state as a /ledger/ throughout the remainder of this
document.
![Ledger Chain](images/consensus/ledger_chain.png "Ledger Chain")
As shown above, new ledgers are made by applying a set of transactions to the
prior ledger. For the Ripple network, transactions include payments,
prior ledger. For the Ripple network, transactions include payments,
modification of account settings, updates to offers and more.
In a centralized system, generating the next ledger is trivial since there is a
single unique arbiter of which transactions to include and how to apply them to
a ledger. For decentralized systems, participants must resolve disagreements on
a ledger. For decentralized systems, participants must resolve disagreements on
the set of transactions to include, the order to apply those transactions, and
even the resulting ledger after applying the transactions. This is even more
even the resulting ledger after applying the transactions. This is even more
difficult when some participants are faulty or malicious.
The Ripple network is a decentralized and **trust-full** network. Anyone is free
The Ripple network is a decentralized and **trust-full** network. Anyone is free
to join and participants are free to choose a subset of peers that are
collectively trusted to not collude in an attempt to defraud the participant.
Leveraging this network of trust, the Ripple algorithm has two main components.
* *Consensus* in which network participants agree on the transactions to apply
- _Consensus_ in which network participants agree on the transactions to apply
to a prior ledger, based on the positions of their chosen peers.
* *Validation* in which network participants agree on what ledger was
- _Validation_ in which network participants agree on what ledger was
generated, based on the ledgers generated by chosen peers.
These phases are continually repeated to process transactions submitted to the
network, generating successive ledgers and giving rise to the blockchain ledger
history depicted below. In this diagram, time is flowing to the right, but
links between ledgers point backward to the parent. Also note the alternate
history depicted below. In this diagram, time is flowing to the right, but
links between ledgers point backward to the parent. Also note the alternate
Ledger 2 that was generated by some participants, but which failed validation
and was abandoned.
@@ -54,7 +54,7 @@ and was abandoned.
The remainder of this section describes the Consensus and Validation algorithms
in more detail and is meant as a companion guide to understanding the generic
implementation in `rippled`. The document **does not** discuss correctness,
implementation in `rippled`. The document **does not** discuss correctness,
fault-tolerance or liveness properties of the algorithms or the full details of
how they integrate within `rippled` to support the Ripple Consensus Ledger.
@@ -62,76 +62,76 @@ how they integrate within `rippled` to support the Ripple Consensus Ledger.
### Definitions
* The *ledger* is the shared distributed state. Each ledger has a unique ID to
distinguish it from all other ledgers. During consensus, the *previous*,
*prior* or *last-closed* ledger is the most recent ledger seen by consensus
- The _ledger_ is the shared distributed state. Each ledger has a unique ID to
distinguish it from all other ledgers. During consensus, the _previous_,
_prior_ or _last-closed_ ledger is the most recent ledger seen by consensus
and is the basis upon which it will build the next ledger.
* A *transaction* is an instruction for an atomic change in the ledger state. A
- A _transaction_ is an instruction for an atomic change in the ledger state. A
unique ID distinguishes a transaction from other transactions.
* A *transaction set* is a set of transactions under consideration by consensus.
The goal of consensus is to reach agreement on this set. The generic
- A _transaction set_ is a set of transactions under consideration by consensus.
The goal of consensus is to reach agreement on this set. The generic
consensus algorithm does not rely on an ordering of transactions within the
set, nor does it specify how to apply a transaction set to a ledger to
generate a new ledger. A unique ID distinguishes a set of transactions from
generate a new ledger. A unique ID distinguishes a set of transactions from
all other sets of transactions.
* A *node* is one of the distributed actors running the consensus algorithm. It
- A _node_ is one of the distributed actors running the consensus algorithm. It
has a unique ID to distinguish it from all other nodes.
* A *peer* of a node is another node that it has chosen to follow and which it
believes will not collude with other chosen peers. The choice of peers is not
- A _peer_ of a node is another node that it has chosen to follow and which it
believes will not collude with other chosen peers. The choice of peers is not
symmetric, since participants can decide on their chosen sets independently.
* A /position/ is the current belief of the next ledger's transaction set and
- A /position/ is the current belief of the next ledger's transaction set and
close time. Position can refer to the node's own position or the position of a
peer.
* A *proposal* is one of a sequence of positions a node shares during consensus.
- A _proposal_ is one of a sequence of positions a node shares during consensus.
An initial proposal contains the starting position taken by a node before it
considers any peer positions. If a node subsequently updates its position in
response to its peers, it will issue an updated proposal. A proposal is
considers any peer positions. If a node subsequently updates its position in
response to its peers, it will issue an updated proposal. A proposal is
uniquely identified by the ID of the proposing node, the ID of the position
taken, the ID of the prior ledger the proposal is for, and the sequence number
of the proposal.
* A *dispute* is a transaction that is either not part of a node's position or
- A _dispute_ is a transaction that is either not part of a node's position or
not in a peer's position. During consensus, the node will add or remove
disputed transactions from its position based on that transaction's support
amongst its peers.
Note that most types have an ID as a lightweight identifier of instances of that
type. Consensus often operates on the IDs directly since the underlying type is
potentially expensive to share over the network. For example, proposal's only
contain the ID of the position of a peer. Since many peers likely have the same
type. Consensus often operates on the IDs directly since the underlying type is
potentially expensive to share over the network. For example, proposal's only
contain the ID of the position of a peer. Since many peers likely have the same
position, this reduces the need to send the full transaction set multiple times.
Instead, a node can request the transaction set from the network if necessary.
### Overview
### Overview
![Consensus Overview](images/consensus/consensus_overview.png "Consensus Overview")
The diagram above is an overview of the consensus process from the perspective
of a single participant. Recall that during a single consensus round, a node is
of a single participant. Recall that during a single consensus round, a node is
trying to agree with its peers on which transactions to apply to its prior
ledger when generating the next ledger. It also attempts to agree on the
[network time when the ledger closed](#effective_close_time). There are
ledger when generating the next ledger. It also attempts to agree on the
[network time when the ledger closed](#effective_close_time). There are
3 main phases to a consensus round:
* A call to `startRound` places the node in the `Open` phase. In this phase,
the node is waiting for transactions to include in its open ledger.
* At some point, the node will `Close` the open ledger and transition to the
`Establish` phase. In this phase, the node shares/receives peer proposals on
which transactions should be accepted in the closed ledger.
* At some point, the node determines it has reached consensus with its peers on
which transactions to include. It transitions to the `Accept` phase. In this
phase, the node works on applying the transactions to the prior ledger to
generate a new closed ledger. Once the new ledger is completed, the node shares
the validated ledger hash with the network and makes a call to `startRound` to
start the cycle again for the next ledger.
- A call to `startRound` places the node in the `Open` phase. In this phase,
the node is waiting for transactions to include in its open ledger.
- At some point, the node will `Close` the open ledger and transition to the
`Establish` phase. In this phase, the node shares/receives peer proposals on
which transactions should be accepted in the closed ledger.
- At some point, the node determines it has reached consensus with its peers on
which transactions to include. It transitions to the `Accept` phase. In this
phase, the node works on applying the transactions to the prior ledger to
generate a new closed ledger. Once the new ledger is completed, the node shares
the validated ledger hash with the network and makes a call to `startRound` to
start the cycle again for the next ledger.
Throughout, a heartbeat timer calls `timerEntry` at a regular frequency to drive
the process forward. Although the `startRound` call occurs at arbitrary times
based on when the initial round began and the time it takes to apply
transactions, the transitions from `Open` to `Establish` and `Establish` to
`Accept` only occur during calls to `timerEntry`. Similarly, transactions can
`Accept` only occur during calls to `timerEntry`. Similarly, transactions can
arrive at arbitrary times, independent of the heartbeat timer. Transactions
received after the `Open` to `Close` transition and not part of peer proposals
won't be considered until the next consensus round. They are represented above
won't be considered until the next consensus round. They are represented above
by the light green triangles.
Peer proposals are issued by a node during a `timerEntry` call, but since peers
@@ -139,16 +139,16 @@ do not synchronize `timerEntry` calls, they are received by other peers at
arbitrary times. Peer proposals are only considered if received prior to the
`Establish` to `Accept` transition, and only if the peer is working on the same
prior ledger. Peer proposals received after consensus is reached will not be
meaningful and are represented above by the circle with the X in it. Only
meaningful and are represented above by the circle with the X in it. Only
proposals from chosen peers are considered.
### Effective Close Time ### {#effective_close_time}
### Effective Close Time ### {#effective_close_time}
In addition to agreeing on a transaction set, each consensus round tries to
agree on the time the ledger closed. Each node calculates its own close time
when it closes the open ledger. This exact close time is rounded to the nearest
multiple of the current *effective close time resolution*. It is this
*effective close time* that nodes seek to agree on. This allows servers to
agree on the time the ledger closed. Each node calculates its own close time
when it closes the open ledger. This exact close time is rounded to the nearest
multiple of the current _effective close time resolution_. It is this
_effective close time_ that nodes seek to agree on. This allows servers to
derive a common time for a ledger without the need for perfectly synchronized
clocks. As depicted below, the 3 pink arrows represent exact close times from 3
consensus nodes that round to the same effective close time given the current
@@ -158,9 +158,9 @@ different effective close time given the current resolution.
![Effective Close Time](images/consensus/EffCloseTime.png "Effective Close Time")
The effective close time is part of the node's position and is shared with peers
in its proposals. Just like the position on the consensus transaction set, a
in its proposals. Just like the position on the consensus transaction set, a
node will update its close time position in response to its peers' effective
close time positions. Peers can agree to disagree on the close time, in which
close time positions. Peers can agree to disagree on the close time, in which
case the effective close time is taken as 1 second past the prior close.
The close time resolution is itself dynamic, decreasing (coarser) resolution in
@@ -173,12 +173,12 @@ reach close time consensus.
Internally, a node operates under one of the following consensus modes. Either
of the first two modes may be chosen when a consensus round starts.
* *Proposing* indicates the node is a full-fledged consensus participant. It
- _Proposing_ indicates the node is a full-fledged consensus participant. It
takes on positions and sends proposals to its peers.
* *Observing* indicates the node is a passive consensus participant. It
- _Observing_ indicates the node is a passive consensus participant. It
maintains a position internally, but does not propose that position to its
peers. Instead, it receives peer proposals and updates its position
to track the majority of its peers. This may be preferred if the node is only
to track the majority of its peers. This may be preferred if the node is only
being used to track the state of the network or during a start-up phase while
it is still synchronizing with the network.
@@ -186,14 +186,14 @@ The other two modes are set internally during the consensus round when the node
believes it is no longer working on the dominant ledger chain based on peer
validations. It checks this on every call to `timerEntry`.
* *Wrong Ledger* indicates the node is not working on the correct prior ledger
and does not have it available. It requests that ledger from the network, but
continues to work towards consensus this round while waiting. If it had been
*proposing*, it will send a special "bowout" proposal to its peers to indicate
- _Wrong Ledger_ indicates the node is not working on the correct prior ledger
and does not have it available. It requests that ledger from the network, but
continues to work towards consensus this round while waiting. If it had been
_proposing_, it will send a special "bowout" proposal to its peers to indicate
its change in mode for the rest of this round. For the duration of the round,
it defers to peer positions for determining the consensus outcome as if it
were just *observing*.
* *Switch Ledger* indicates that the node has acquired the correct prior ledger
were just _observing_.
- _Switch Ledger_ indicates that the node has acquired the correct prior ledger
from the network. Although it now has the correct prior ledger, the fact that
it had the wrong one at some point during this round means it is likely behind
and should defer to peer positions for determining the consensus outcome.
@@ -201,7 +201,7 @@ validations. It checks this on every call to `timerEntry`.
![Consensus Modes](images/consensus/consensus_modes.png "Consensus Modes")
Once either wrong ledger or switch ledger are reached, the node cannot
return to proposing or observing until the next consensus round. However,
return to proposing or observing until the next consensus round. However,
the node could change its view of the correct prior ledger, so going from
switch ledger to wrong ledger and back again is possible.
@@ -215,16 +215,16 @@ decide how best to generate the next ledger once it declares consensus.
### Phases
As depicted in the overview diagram, consensus is best viewed as a progression
through 3 phases. There are 4 public methods of the generic consensus algorithm
through 3 phases. There are 4 public methods of the generic consensus algorithm
that determine this progression
* `startRound` begins a consensus round.
* `timerEntry` is called at a regular frequency (`LEDGER_MIN_CLOSE`) and is the
only call to consensus that can change the phase from `Open` to `Establish`
- `startRound` begins a consensus round.
- `timerEntry` is called at a regular frequency (`LEDGER_MIN_CLOSE`) and is the
only call to consensus that can change the phase from `Open` to `Establish`
or `Accept`.
* `peerProposal` is called whenever a peer proposal is received and is what
- `peerProposal` is called whenever a peer proposal is received and is what
allows a node to update its position in a subsequent `timerEntry` call.
* `gotTxSet` is called when a transaction set is received from the network. This
- `gotTxSet` is called when a transaction set is received from the network. This
is typically in response to a prior request from the node to acquire the
transaction set corresponding to a disagreeing peer's position.
@@ -234,13 +234,13 @@ actions are taken in response to these calls.
#### Open
The `Open` phase is a quiescent period to allow transactions to build up in the
node's open ledger. The duration is a trade-off between latency and throughput.
node's open ledger. The duration is a trade-off between latency and throughput.
A shorter window reduces the latency to generating the next ledger, but also
reduces transaction throughput due to fewer transactions accepted into the
ledger.
A call to `startRound` would forcibly begin the next consensus round, skipping
completion of the current round. This is not expected during normal operation.
completion of the current round. This is not expected during normal operation.
Calls to `peerProposal` or `gotTxSet` simply store the proposal or transaction
set for use in the coming `Establish` phase.
@@ -254,28 +254,27 @@ the ledger.
Under normal circumstances, the open ledger period ends when one of the following
is true
* if there are transactions in the open ledger and more than `LEDGER_MIN_CLOSE`
have elapsed. This is the typical behavior.
* if there are no open transactions and a suitably longer idle interval has
elapsed. This increases the opportunity to get some transaction into
- if there are transactions in the open ledger and more than `LEDGER_MIN_CLOSE`
have elapsed. This is the typical behavior.
- if there are no open transactions and a suitably longer idle interval has
elapsed. This increases the opportunity to get some transaction into
the next ledger and avoids doing useless work closing an empty ledger.
* if more than half the number of prior round peers have already closed or finished
- if more than half the number of prior round peers have already closed or finished
this round. This indicates the node is falling behind and needs to catch up.
When closing the ledger, the node takes its initial position based on the
transactions in the open ledger and uses the current time as
its initial close time estimate. If in the proposing mode, the node shares its
initial position with peers. Now that the node has taken a position, it will
consider any peer positions for this round that arrived earlier. The node
its initial close time estimate. If in the proposing mode, the node shares its
initial position with peers. Now that the node has taken a position, it will
consider any peer positions for this round that arrived earlier. The node
generates disputed transactions for each transaction not in common with a peer's
position. The node also records the vote of each peer for each disputed
position. The node also records the vote of each peer for each disputed
transaction.
In the example below, we suppose our node has closed with transactions 1,2 and 3. It creates disputes
In the example below, we suppose our node has closed with transactions 1,2 and 3. It creates disputes
for transactions 2,3 and 4, since at least one peer position differs on each.
##### disputes ##### {#disputes_image}
##### disputes ##### {#disputes_image}
![Disputes](images/consensus/disputes.png "Disputes")
@@ -286,22 +285,22 @@ exchanges proposals with peers in an attempt to reach agreement on the consensus
transactions and effective close time.
A call to `startRound` would forcibly begin the next consensus round, skipping
completion of the current round. This is not expected during normal operation.
completion of the current round. This is not expected during normal operation.
Calls to `peerProposal` or `gotTxSet` that reflect new positions will generate
disputed transactions for any new disagreements and will update the peer's vote
for all disputed transactions.
A call to `timerEntry` first checks that the node is working from the correct
prior ledger. If not, the node will update the mode and request the correct
ledger. Otherwise, the node updates the node's position and considers whether
to switch to the `Accepted` phase and declare consensus reached. However, at
least `LEDGER_MIN_CONSENSUS` time must have elapsed before doing either. This
prior ledger. If not, the node will update the mode and request the correct
ledger. Otherwise, the node updates the node's position and considers whether
to switch to the `Accepted` phase and declare consensus reached. However, at
least `LEDGER_MIN_CONSENSUS` time must have elapsed before doing either. This
allows peers an opportunity to take an initial position and share it.
##### Update Position
In order to achieve consensus, the node is looking for a transaction set that is
supported by a super-majority of peers. The node works towards this set by
supported by a super-majority of peers. The node works towards this set by
adding or removing disputed transactions from its position based on an
increasing threshold for inclusion.
@@ -310,23 +309,23 @@ increasing threshold for inclusion.
By starting with a lower threshold, a node initially allows a wide set of
transactions into its position. If the establish round continues and the node is
"stuck", a higher threshold can focus on accepting transactions with the most
support. The constants that define the thresholds and durations at which the
support. The constants that define the thresholds and durations at which the
thresholds change are given by `AV_XXX_CONSENSUS_PCT` and
`AV_XXX_CONSENSUS_TIME` respectively, where `XXX` is `INIT`,`MID`,`LATE` and
`STUCK`. The effective close time position is updated using the same
`STUCK`. The effective close time position is updated using the same
thresholds.
Given the [example disputes above](#disputes_image) and an initial threshold
of 50%, our node would retain its position since transaction 1 was not in
dispute and transactions 2 and 3 have 75% support. Since its position did not
change, it would not need to send a new proposal to peers. Peer C would not
dispute and transactions 2 and 3 have 75% support. Since its position did not
change, it would not need to send a new proposal to peers. Peer C would not
change either. Peer A would add transaction 3 to its position and Peer B would
remove transaction 4 from its position; both would then send an updated
position.
Conversely, if the diagram reflected a later call to =timerEntry= that occurs in
the stuck region with a threshold of say 95%, our node would remove transactions
2 and 3 from its candidate set and send an updated position. Likewise, all the
2 and 3 from its candidate set and send an updated position. Likewise, all the
other peers would end up with only transaction 1 in their position.
Lastly, if our node were not in the proposing mode, it would not include its own
@@ -336,7 +335,7 @@ our node would maintain its position of transactions 1, 2 and 3.
##### Checking Consensus
After updating its position, the node checks for supermajority agreement with
its peers on its current position. This agreement is of the exact transaction
its peers on its current position. This agreement is of the exact transaction
set, not just the support of individual transactions. That is, if our position
is a subset of a peer's position, that counts as a disagreement. Also recall
that effective close time agreement allows a supermajority of participants
@@ -344,10 +343,10 @@ agreeing to disagree.
Consensus is declared when the following 3 clauses are true:
* `LEDGER_MIN_CONSENSUS` time has elapsed in the establish phase
* At least 75% of the prior round proposers have proposed OR this establish
- `LEDGER_MIN_CONSENSUS` time has elapsed in the establish phase
- At least 75% of the prior round proposers have proposed OR this establish
phase is `LEDGER_MIN_CONSENSUS` longer than the last round's establish phase
* `minimumConsensusPercentage` of ourself and our peers share the same position
- `minimumConsensusPercentage` of ourself and our peers share the same position
The middle condition ensures slower peers have a chance to share positions, but
prevents waiting too long on peers that have disconnected. Additionally, a node
@@ -364,22 +363,22 @@ logic.
Once consensus is reached (or moved on), the node switches to the `Accept` phase
and signals to the implementing code that the round is complete. That code is
responsible for using the consensus transaction set to generate the next ledger
and calling `startRound` to begin the next round. The implementation has total
and calling `startRound` to begin the next round. The implementation has total
freedom on ordering transactions, deciding what to do if consensus moved on,
determining whether to retry or abandon local transactions that did not make the
consensus set and updating any internal state based on the consensus progress.
#### Accept
The `Accept` phase is the terminal phase of the consensus algorithm. Calls to
The `Accept` phase is the terminal phase of the consensus algorithm. Calls to
`timerEntry`, `peerProposal` and `gotTxSet` will not change the internal
consensus state while in the accept phase. The expectation is that the
consensus state while in the accept phase. The expectation is that the
application specific code is working to generate the new ledger based on the
consensus outcome. Once complete, that code should make a call to `startRound`
to kick off the next consensus round. The `startRound` call includes the new
prior ledger, prior ledger ID and whether the round should begin in the
proposing or observing mode. After setting some initial state, the phase
transitions to `Open`. The node will also check if the provided prior ledger
proposing or observing mode. After setting some initial state, the phase
transitions to `Open`. The node will also check if the provided prior ledger
and ID are correct, updating the mode and requesting the proper ledger from the
network if necessary.
@@ -448,9 +447,9 @@ struct TxSet
### Ledger
The `Ledger` type represents the state shared amongst the
distributed participants. Notice that the details of how the next ledger is
distributed participants. Notice that the details of how the next ledger is
generated from the prior ledger and the consensus accepted transaction set is
not part of the interface. Within the generic code, this type is primarily used
not part of the interface. Within the generic code, this type is primarily used
to know that peers are working on the same tip of the ledger chain and to
provide some basic timing data for consensus.
@@ -626,7 +625,7 @@ struct Adaptor
// Called when consensus operating mode changes
void onModeChange(ConsensuMode before, ConsensusMode after);
// Called when ledger closes. Implementation should generate an initial Result
// with position based on the current open ledger's transactions.
ConsensusResult onClose(Ledger const &, Ledger const & prev, ConsensusMode mode);
@@ -657,27 +656,24 @@ struct Adaptor
The implementing class hides many details of the peer communication
model from the generic code.
* The `share` member functions are responsible for sharing the given type with a
- The `share` member functions are responsible for sharing the given type with a
node's peers, but are agnostic to the mechanism. Ideally, messages are delivered
faster than `LEDGER_GRANULARITY`.
* The generic code does not specify how transactions are submitted by clients,
faster than `LEDGER_GRANULARITY`.
- The generic code does not specify how transactions are submitted by clients,
propagated through the network or stored in the open ledger. Indeed, the open
ledger is only conceptual from the perspective of the generic code---the
initial position and transaction set are opaquely generated in a
`Consensus::Result` instance returned from the `onClose` callback.
* The calls to `acquireLedger` and `acquireTxSet` only have non-trivial return
if the ledger or transaction set of interest is available. The implementing
- The calls to `acquireLedger` and `acquireTxSet` only have non-trivial return
if the ledger or transaction set of interest is available. The implementing
class is free to block while acquiring, or return the empty option while
servicing the request asynchronously. Due to legacy reasons, the two calls
servicing the request asynchronously. Due to legacy reasons, the two calls
are not symmetric. `acquireTxSet` requires the host application to call
`gotTxSet` when an asynchronous `acquire` completes. Conversely,
`acquireLedger` will be called again later by the consensus code if it still
desires the ledger with the hope that the asynchronous acquisition is
complete.
## Validation
Coming Soon!

View File

@@ -1,3 +1,3 @@
---
DisableFormat: true
SortIncludes: false
SortIncludes: Never

Some files were not shown because too many files have changed in this diff Show More