Compare commits

...

104 Commits

Author SHA1 Message Date
JCW
4b5ed9c219 Fix formatting
Signed-off-by: JCW <a1q123456@users.noreply.github.com>
2025-08-04 21:49:52 +01:00
JCW
5dd2020902 Revert unnecessary changes
Signed-off-by: JCW <a1q123456@users.noreply.github.com>
2025-08-04 21:46:14 +01:00
JCW
ae95aff7aa Demonstrate how strongly typed ledger objects work
Signed-off-by: JCW <a1q123456@users.noreply.github.com>

# Conflicts:
#	include/xrpl/protocol/Keylet.h
#	src/libxrpl/protocol/Indexes.cpp
#	src/xrpld/rpc/handlers/VaultInfo.cpp
2025-08-04 21:43:22 +01:00
JCW
4f5fb6b298 Fix build error
Signed-off-by: JCW <a1q123456@users.noreply.github.com>
2025-08-04 21:38:45 +01:00
JCW
c742a94e5f Fix formatting
Signed-off-by: JCW <a1q123456@users.noreply.github.com>
2025-08-04 21:34:27 +01:00
JCW
abf240f6f3 Make Keylet strongly typed so that we can get strongly typed ledger obejcts from views
Signed-off-by: JCW <a1q123456@users.noreply.github.com>
2025-08-04 21:32:56 +01:00
JCW
3f8fed9628 Fix formatting
Signed-off-by: JCW <a1q123456@users.noreply.github.com>
2025-08-04 21:27:40 +01:00
JCW
e2e82e1c96 Support const ledger entries
Signed-off-by: JCW <a1q123456@users.noreply.github.com>
2025-08-04 21:24:03 +01:00
JCW
a7a39d62d3 Fix formatting 2025-07-11 17:19:27 +01:00
JCW
b875ea9f37 Add comments 2025-07-11 17:09:15 +01:00
JCW
a25f2ee9dd Fix formatting 2025-07-11 11:10:59 +01:00
JCW
4f295d4856 Fix build error 2025-07-11 10:40:17 +01:00
JCW
2e5f69f8f8 Fix test coverage 2025-07-10 20:18:27 +01:00
JCW
02da6b2508 Strongly typed ledger objects support 2025-07-10 17:19:38 +01:00
Bronek Kozicki
b113190563 Downgrade required CMake version for Antithesis SDK (#5548)
The current version was copied from `antithesis-sdk-cpp` but there is no logical reason to require this specific version of CMake. This change downgrades the version to make the project build with older CMake versions.
2025-07-10 11:46:02 -04:00
Ayaz Salikhov
358b7f50a7 fix: Link with boost libraries explicitly (#5546)
Having `boost::boost` in `self.requires` makes clio link with all boost libraries. There are additionally several Boost stacktrace backends that are both linked with, which violate ODR.
This change fixes the problem.
2025-07-10 06:14:27 -04:00
Bronek Kozicki
f47e2f4e82 chore: Fix compilation error with clang-20 and cleanup (#5543)
Removes clutter for old compilers, defaults to non-unity builds in cmake to match conanfile.py, and workaround for clang-20 compilation errors.
2025-07-09 17:47:34 +00:00
Bronek Kozicki
a7eea9546f test: Remove circular jtx.h dependencies (#5544)
Circular includes in header files can yield unpredictable results.
2025-07-09 08:43:11 -04:00
Jingchen
9874d47d7f Decouple CredentialHelpers from xrpld/app/tx (#5487)
This PR refactors `CredentialHelpers` and removes some unnecessary dependencies as a step of modularization.

The ledger component is almost independent except that it references `MPTokenAuthorize` and `CredentialHelpers.h`, and the latter further references `Transactor.h`. This PR partially clears the path to modularizing the ledger component and decouples `CredentialHelpers` from xrpld.
2025-07-03 14:27:37 +00:00
Mayukha Vadari
c2f3e2e263 fix: crash when trace-logging in tests (#5529)
This PR fixes a crash in tests when the test `Env is run at trace/debug log level.

This issue only affects tests, and only if logging at trace/debug level, so really only relevant during rippled development, and does not affect production servers.
2025-07-02 19:10:25 +00:00
Vlad
e18f27f5f7 test: switch some unit tests to doctest (#5383)
This change moves some tests from the current unit tests that are compiled into the rippled binary to using the doctest framework.
2025-06-26 19:35:31 +00:00
Jingchen
df6daf0d8f Add XRPL_ABANDON and use it to abandon OwnerPaysFee (#5510) 2025-06-26 12:09:05 -04:00
Jingchen
e9d46f0bfc Remove OwnerPaysFee as it's never fully supported (#5435)
The OwnerPaysFee amendment was never fully supported, and this change removes the feature to the extent possible.
2025-06-24 18:56:58 +00:00
Bart
42fd74b77b Removes release notes from codebase (#5508) 2025-06-24 13:10:00 -04:00
tequ
c55ea56c5e Add nftoken_id, nftoken_ids, offer_id to meta for transaction stream (#5230) 2025-06-24 09:02:22 -04:00
Michael Legleux
1e01cd34f7 Set version to 2.5.0 2025-06-23 10:13:01 -07:00
Alex Kremer
e2fa5c1b7c chore: Change libXRPL check conan remote to dev (#5482)
This change aligns the Conan remote used by the libXRPL Clio compatibility check workflow with the recent changes applied to Clio.
2025-06-20 17:02:16 +00:00
Ed Hennis
fc0984d286 Require a message on "Application::signalStop" (#5255)
This change adds a message parameter to Application::signalStop for extra context.
2025-06-20 16:24:34 +00:00
Valentin Balaschenko
8b3dcd41f7 refactor: Change getNodeFat Missing Node State Tree error into warning (#5455) 2025-06-20 15:44:42 +00:00
Denis Angell
8f2f5310e2 Fix: Improve error handling in Batch RPC response (#5503) 2025-06-18 17:46:45 -04:00
Michael Legleux
edb4f0342c Set version to 2.5.0-rc2 2025-06-11 17:10:45 -07:00
yinyiqian1
ea17abb92a fix: Ensure delegate tests do not silently fail with batch (#5476)
The tests that ensure `tfInnerBatchTxn` won't block delegated transactions silently fail in `Delegate_test.cpp`. This change removes these cases from that file and adds them to `Batch_test.cpp` instead where they do not silently fail, because there the batch delegate results are explicitly checked. Moving them to that file further avoids refactoring many helper functions.
2025-06-11 13:21:24 +08:00
Mayukha Vadari
35a40a8e62 fix: Improve multi-sign usage of simulate (#5479)
This change allows users to submit simulate requests from a multi-sign account without needing to specify the accounts that are doing the multi-signing, and fixes an error with simulate that allowed double-"signed" (both single-sign and multi-sign public keys are provided) transactions.
2025-06-10 14:47:27 +08:00
Ed Hennis
d494bf45b2 refactor: Collapse some split log messages into one (#5347)
Multi-line log messages are hard to work with. Writing these handful of related messages as one message should make the log a tiny bit easier to manage.
2025-06-06 16:01:02 +00:00
Vlad
8bf4a5cbff chore: Remove external project build cores division (#5475)
The CMake statements that make it seem as if the number of cores used to build external project dependencies is halved don't actually do anything. This change removes these statements.
2025-06-05 13:37:30 +00:00
Denis Angell
58c2c82a30 fix: Amendment-guard TokenEscrow preclaim and expand tests (#5473)
This change amendment-guards the preclaim for `TokenEscrow`, as well as expands tests to increase code coverage.
2025-06-05 12:54:45 +00:00
Michael Legleux
11edaa441d Set version to 2.5.0-rc1 (#5472) 2025-06-04 17:55:23 +00:00
yinyiqian1
a5e953b191 fix: Add tecNO_DELEGATE_PERMISSION and fix flags (#5465)
* Adds `tecNO_DELEGATE_PERMISSION` for unauthorized transactions sent by a delegated account.
* Returns `tecNO_TARGET` instead of `terNO_ACCOUNT` for the `DelegateSet` transaction if the delegated account does not exist.
* Fixes `tfFullyCanonicalSig` and `tfInnerBatchTxn` blocking transactions issue by adding `tfUniversal` in the permission related masks in `txFlags.h`
2025-06-03 22:20:29 +00:00
Mark Travis
506ae12a8c Increase network i/o capacity (#5464)
The change increases the default network I/O worker thread pool size from 2 to 6. This will improve stability, as worker thread saturation correlates to desyncs, particularly on high-traffic peers, such as hubs.
2025-06-03 21:33:09 +00:00
Ayaz Salikhov
0310c5cbe0 fix: Specify transitive_headers when building with Conan 2 (#5462)
To be able to consume `rippled` in Conan 2, the recipe should specify transitive_headers for external libraries that are present in the exported header files. This change remains compatibility with Conan 1, where this flag was not present.
2025-06-03 17:33:32 +00:00
Denis Angell
053e1af7ff Add support for XLS-85 Token Escrow (#5185)
- Specification: https://github.com/XRPLF/XRPL-Standards/pull/272
- Amendment: `TokenEscrow`
- Enables escrowing of IOU and MPT tokens in addition to native XRP.
- Allows accounts to lock issued tokens (IOU/MPT) in escrow objects, with support for freeze, authorization, and transfer rates.
- Adds new ledger fields (`sfLockedAmount`, `sfIssuerNode`, etc.) to track locked balances for IOU and MPT escrows.
- Updates EscrowCreate, EscrowFinish, and EscrowCancel transaction logic to support IOU and MPT assets, including proper handling of trustlines and MPT authorization, transfer rates, and locked balances.
- Enforces invariant checks for escrowed IOU/MPT amounts.
- Extends GatewayBalances RPC to report locked (escrowed) balances.
2025-06-03 12:51:55 -04:00
Vlad
7e24adbdd0 fix: Address NFT interactions with trustlines (#5297)
The changes are focused on fixing NFT transactions bypassing the trustline authorization requirement and potential invariant violation when interacting with deep frozen trustlines.
2025-06-02 16:13:20 +00:00
Gregory Tsipenyuk
621df422a7 fix: Add AMMv1_3 amendment (#5203)
* Add AMM bid/create/deposit/swap/withdraw/vote invariants:
  - Deposit, Withdrawal invariants: `sqrt(asset1Balance * asset2Balance) >= LPTokens`.
  - Bid: `sqrt(asset1Balance * asset2Balance) > LPTokens` and the pool balances don't change.
  - Create: `sqrt(asset1Balance * assetBalance2) == LPTokens`.
  - Swap: `asset1BalanceAfter * asset2BalanceAfter >= asset1BalanceBefore * asset2BalanceBefore`
     and `LPTokens` don't change.
  - Vote: `LPTokens` and pool balances don't change.
  - All AMM and swap transactions: amounts and tokens are greater than zero, except on withdrawal if all tokens
    are withdrawn.
* Add AMM deposit and withdraw rounding to ensure AMM invariant:
  - On deposit, tokens out are rounded downward and deposit amount is rounded upward.
  - On withdrawal, tokens in are rounded upward and withdrawal amount is rounded downward.
* Add Order Book Offer invariant to verify consumed amounts. Consumed amounts are less than the offer.
* Fix Bid validation. `AuthAccount` can't have duplicate accounts or the submitter account.
2025-06-02 09:52:10 -04:00
Shawn Xie
0a34b5c691 Add support for XLS-81 Permissioned DEX (#5404)
Modified transactions:
- OfferCreate
- Payment

Modified RPCs:
- book_changes
- subscribe
- book_offers
- ripple_path_find
- path_find

Spec: https://github.com/XRPLF/XRPL-Standards/pull/281
2025-05-30 13:24:48 -04:00
Matt Mankins
e0bc3dd51f docs: update example keyserver host in SECURITY.md (#5460) 2025-05-30 08:46:08 -04:00
Bronek Kozicki
dacecd24ba Fix unit build error (#5459)
This change fixes the issue that there is a `using namespace` statement inside a namespace scope.
2025-05-29 20:53:31 +00:00
Mayukha Vadari
05105743e9 chore[tests]: improve env.meta usage (#5457)
This commit changes the ledger close in env.meta to be conditional on if it hasn't already been closed (i.e. the current ledger doesn't have any transactions in it). This change will make it a bit easier to use, as it will still work if you close the ledger outside of this usage. Previously, if you accidentally closed the ledger outside of the meta function, it would segfault and it was incredibly difficult to debug.
2025-05-29 16:28:09 +00:00
Bronek Kozicki
9e1fe9a85e Fix: Improve handling of expired credentials in VaultDeposit (#5452)
This change returns `tecEXPIRED` from VaultDeposit to allow the Transactor to remove the expired credentials.
2025-05-28 10:28:18 -04:00
Vito Tumas
d71ce51901 feat: improve squelching configuration (#5438)
This commit introduces the following changes:
* Renames `vp_enable config` option to `vp_base_squelch_enable` to enable squelching for validators.
* Removes `vp_squelch` config option which was used to configure whether to send squelch messages to peers or not. With this flag removed, if squelching is enabled, squelch messages will be sent. This was an option used for debugging.
* Introduces a temporary `vp_base_squelch_max_trusted_peers` config option to change the max number of peers who are selected as validator message sources. This is a temporary option, which will be removed once a good value is found.
* Adds a traffic counter to count the number of times peers ignored squelch messages and kept sending messages for squelched validators.
* Moves the decision whether squelching is enabled and ready into Slot.h.
2025-05-28 06:30:03 -04:00
Michael Legleux
be668ee26d chore: Update CPP ref source (#5453) 2025-05-27 20:46:25 +00:00
Bart
cae5294b4e chore: Rename docs job (#5398) 2025-05-27 20:03:23 +00:00
Elliot.
cd777f79ef docs: add -j $(nproc) to BUILD.md (#5288)
This improves build times.
2025-05-27 19:11:13 +00:00
Valentin Balaschenko
8b9e21e3f5 docs: Update build instructions for Ubuntu 22.04+ (#5292) 2025-05-27 18:32:25 +00:00
Denis Angell
2a61aee562 Add Batch feature (XLS-56) (#5060)
- Specification: [XRPLF/XRPL-Standards 56](https://github.com/XRPLF/XRPL-Standards/blob/master/XLS-0056d-batch/README.md)
- Amendment: `Batch`
- Implements execution of multiple transactions within a single batch transaction with four execution modes: `tfAllOrNothing`, `tfOnlyOne`, `tfUntilFailure`, and `tfIndependent`.
- Enables atomic multi-party transactions where multiple accounts can participate in a single batch, with up to 8 inner transactions and 8 batch signers per batch transaction.
- Inner transactions use `tfInnerBatchTxn` flag with zero fees, no signature, and empty signing public key.
- Inner transactions are applied after the outer batch succeeds via the `applyBatchTransactions` function in apply.cpp.
- Network layer prevents relay of transactions with `tfInnerBatchTxn` flag - each peer applies inner transactions locally from the batch.
- Batch transactions are excluded from AccountDelegate permissions but inner transactions retain full delegation support.
- Metadata includes `ParentBatchID` linking inner transactions to their containing batch for traceability and auditing.
- Extended STTx with batch-specific signature verification methods and added protocol structures (`sfRawTransactions`, `sfBatchSigners`).
2025-05-23 19:53:53 +00:00
Bronek Kozicki
40ce8a8833 fix: Fix pseudo-account ID calculation (#5447)
Before #5224, the pseudoaccount ID was calculated using prefix expressed in `std::uint16_t`. The refactoring to move the pseudoaccount ID calculation to View.cpp had accidentally changed the prefix type to `int` (derived from `auto i = 0`) which in turn changed the length of the input to `sha512Half` from 2 bytes to 4, altering the result.

This resulted in a different ID of the pseudoaccount calculated from the function after the refactoring, breaking the ledger. This impacts AMMCreate, even when the `SingleAssetVault` amendment is not active. This change restores the prefix type to `std::uint16_t`.
2025-05-23 14:05:36 +00:00
Bronek Kozicki
7713ff8c5c Add codecov badge, raise .codecov.yml thresholds (#5428) 2025-05-22 14:43:41 +00:00
Olek
70371a4344 Fix initializer list initialization for GCC-15 (#5443) 2025-05-21 13:28:18 -04:00
Bronek Kozicki
e514de76ed Add single asset vault (XLS-65d) (#5224)
- Specification: XRPLF/XRPL-Standards#239
- Amendment: `SingleAssetVault`
- Implements a vault feature used to store a fungible asset (XRP, IOU, or MPT, but not NFT) and to receive shares in the vault (an MPT) in exchange.
- A vault can be private or public.
- A private vault can use permissioned domains, subject to the `PermissionedDomains` amendment.
- Shares can be exchanged back into asset with `VaultWithdraw`.
- Permissions on the asset in the vault are transitively applied on shares in the vault.
- Issuer of the asset in the vault can clawback with `VaultClawback`.
- Extended `MPTokenIssuance` with `DomainID`, used by the permissioned domain on the vault shares.

Co-authored-by: John Freeman <jfreeman08@gmail.com>
2025-05-20 14:06:41 -04:00
Bart
dd62cfcc22 fix: Update path in CODEOWNERS (#5440) 2025-05-20 15:24:07 +00:00
Michael Legleux
09690f1b38 Set version to 2.5.0-b1 2025-05-18 20:39:18 +01:00
Valentin Balaschenko
380ba9f1c1 Fix: Resolve slow test on macOS pipeline (#5392)
Using std::barrier performs extremely poorly (~1 hour vs ~1 minute to run the test suite) in certain macOS environments.
To unblock our macOS CI pipeline, std::barrier has been replaced with a custom mutex-based barrier (Barrier) that significantly improves performance without compromising correctness.
2025-05-16 10:31:51 +00:00
brettmollin
c3e9380fb4 fix: Update validators-example.txt fix xrplf example URL (#5384) 2025-05-16 09:49:14 +00:00
Jingchen
e3ebc253fa fix: Ensure that coverage file generation is atomic. (#5426)
Running unit tests in parallel and multiple threads can write into one file can corrupt output files, and then gcovr won't be able to parse the corrupted file. This change adds -fprofile-update=atomic as instructed by https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68080.
2025-05-12 14:54:01 +00:00
Bart
c6c7c84355 Configure CODEOWNERS for changes to RPC code (#5266)
To ensure changes to any RPC-related code are compatible with other services, such as Clio, the RPC team will be required to review them.
2025-05-12 12:42:03 +00:00
yinyiqian1
28f50cb7cf fix: enable LedgerStateFix for delegation (#5427) 2025-05-10 10:36:11 -04:00
Vito Tumas
3e152fec74 refactor: use east const convention (#5409)
This change refactors the codebase to use the "east const convention", and adds a clang-format rule to follow this convention.
2025-05-08 11:00:42 +00:00
yinyiqian1
2db2791805 Add PermissionDelegation feature (#5354)
This change implements the account permission delegation described in XLS-75d, see https://github.com/XRPLF/XRPL-Standards/pull/257.

* Introduces transaction-level and granular permissions that can be delegated to other accounts.
* Adds `DelegateSet` transaction to grant specified permissions to another account.
* Adds `ltDelegate` ledger object to maintain the permission list for delegating/delegated account pair.
* Adds an optional `Delegate` field in common fields, allowing a delegated account to send transactions on behalf of the delegating account within the granted permission scope. The `Account` field remains the delegating account; the `Delegate` field specifies the delegated account. The transaction is signed by the delegated account.
2025-05-08 06:14:02 -04:00
Vito Tumas
9ec2d7f8ff Enable passive squelching (#5358)
This change updates the squelching logic to accept squelch messages for untrusted validators. As a result, servers will also squelch untrusted validator messages reducing duplicate traffic they generate.

In particular:
* Updates squelch message handling logic to squelch messages for all validators, not only trusted ones.
* Updates the logic to send squelch messages to peers that don't squelch themselves
* Increases the threshold for the number of messages that a peer has to deliver to consider it as a candidate for validator messages.
2025-05-02 11:01:45 -04:00
Ed Hennis
4a084ce34c Improve transaction relay logic (#4985)
Combines four related changes:
1. "Decrease `shouldRelay` limit to 30s." Pretty self-explanatory. Currently, the limit is 5 minutes, by which point the `HashRouter` entry could have expired, making this transaction look brand new (and thus causing it to be relayed back to peers which have sent it to us recently).
2.  "Give a transaction more chances to be retried." Will put a transaction into `LedgerMaster`'s held transactions if the transaction gets a `ter`, `tel`, or `tef` result. Old behavior was just `ter`.
     * Additionally, to prevent a transaction from being repeatedly held indefinitely, it must meet some extra conditions. (Documented in a comment in the code.)
3. "Pop all transactions with sequential sequences, or tickets." When a transaction is processed successfully, currently, one held transaction for the same account (if any) will be popped out of the held transactions list, and queued up for the next transaction batch. This change pops all transactions for the account, but only if they have sequential sequences (for non-ticket transactions) or use a ticket. This issue was identified from interactions with @mtrippled's #4504, which was merged, but unfortunately reverted later by #4852. When the batches were spaced out, it could potentially take a very long time for a large number of held transactions for an account to get processed through. However, whether batched or not, this change will help get held transactions cleared out, particularly if a missing earlier transaction is what held them up.
4. "Process held transactions through existing NetworkOPs batching." In the current processing, at the end of each consensus round, all held transactions are directly applied to the open ledger, then the held list is reset. This bypasses all of the logic in `NetworkOPs::apply` which, among other things, broadcasts successful transactions to peers. This means that the transaction may not get broadcast to peers for a really long time (5 minutes in the current implementation, or 30 seconds with this first commit). If the node is a bottleneck (either due to network configuration, or because the transaction was submitted locally), the transaction may not be seen by any other nodes or validators before it expires or causes other problems.
2025-05-01 13:58:18 -04:00
Vito Tumas
3502df2174 fix: Replaces random endpoint resolution with sequential (#5365)
This change addresses an issue where `rippled` attempts to connect to an IPv6 address, even when the local network lacks IPv6 support, resulting in a "Network is unreachable" error.

The fix replaces the custom endpoint selection logic with `boost::async_connect`, which sequentially attempts to connect to available endpoints until one succeeds or all fail.
2025-04-28 15:38:55 -04:00
Vlad
fa1e25abef chore: Small clarification to lsfDefaultRipple comment (#5410) 2025-04-25 15:21:27 +00:00
Denis Angell
217ba8dd4d fix: CTID to use correct ledger_index (#5408) 2025-04-24 10:24:10 -04:00
Ed Hennis
405f4613d8 chore: Run CI on PRs that are Ready or have the "DraftRunCI" label (#5400)
- Avoids costly overhead for idle PRs where the CI results don't add any
  value.
2025-04-11 22:20:59 +00:00
Mayukha Vadari
cba512068b refactor: Clean up test logging to make it easier to search (#5396)
This PR replaces the word `failed` with `failure` in any test names and renames some test files to fix MSVC warnings, so that it is easier to search through the test output to find tests that failed.
2025-04-11 09:07:42 +00:00
Valentin Balaschenko
1c99ea23d1 Temporary disable automatic triggering macOS pipeline (#5397)
We temporarily disable running unit tests on macOS on the CI pipeline while we are investigating the delays.
2025-04-10 21:58:29 +02:00
Denis Angell
c4308b216f fix: Adds CTID to RPC tx and updates error (#4738)
This change fixes a number of issues involved with CTID:
* CTID is not present on all RPC tx transactions.
* rpcWRONG_NETWORK is missing in the ErrorCodes.cpp
2025-04-10 12:38:52 +00:00
Wietse Wind
aafd2d8525 Fix: admin RPC webhook queue limit removal and timeout reduction (#5163)
When using subscribe at admin RPC port to send webhooks for the transaction stream to a backend, on large(r) ledgers the endpoint receives fewer HTTP POSTs with TX information than the amount of transactions in a ledger. This change removes the hardcoded queue length to avoid dropping TX notifications for the admin-only command. In addition, the per-request TTL for outgoing RPC HTTP calls has been reduced from 10 minutes to 30 seconds.
2025-04-10 06:37:24 +00:00
Denis Angell
a574ec6023 fix: fixPayChanV1 (#4717)
This change introduces a new fix amendment (`fixPayChanV1`) that prevents the creation of new `PaymentChannelCreate` transaction with a `CancelAfter` time less than the current ledger time. It piggy backs off of fix1571.

Once the amendment is activated, creating a new `PaymentChannel` will require that if you specify the `CancelAfter` time/value, that value must be greater than or equal to the current ledger time.

Currently users can create a payment channel where the `CancelAfter` time is before the current ledger time. This results in the payment channel being immediately closed on the next PaymentChannel transaction.
2025-04-09 22:08:44 +00:00
Mayukha Vadari
e429455f4d refactor(trivial): reorganize ledger entry tests and helper functions (#5376)
This PR splits out `ledger_entry` tests into its own file (`LedgerEntry_test.cpp`) and alphabetizes the helper functions in `LedgerEntry.cpp`. These commits were split out of #5237 to make that PR a little more manageable, since these basic trivial changes are most of the diff. There is no code change, just moving code around.
2025-04-09 17:02:03 +00:00
Vito Tumas
7692eeb9a0 Instrument proposal, validation and transaction messages (#5348)
Adds metric counters for the following P2P message types:

* Untrusted proposal and validation messages
* Duplicate proposal, validation and transaction messages
2025-04-09 15:33:17 +02:00
Bronek Kozicki
a099f5a804 Remove UNREACHABLE from NetworkOPsImp::processTrustedProposal (#5387)
It’s possible for this to happen legitimately if a set of peers, including a validator, are connected in a cycle, and the latency and message processing time between those peers is significantly less than the latency between the validator and the last peer. It’s unlikely in the real world, but obviously easy to simulate with Antithesis.
2025-04-08 14:43:34 +00:00
Michael Legleux
ca0bc767fe fix: Use the build image from ghcr.io (#5390)
The ci pipelines are constantly hitting Docker Hub's public rate limiting since increasing the number of jobs we're running. This change switches over to images hosted in GitHub's registry.
2025-04-05 02:24:31 +00:00
Mayukha Vadari
4ba9288935 fix: disable channel_authorize when signing_support is disabled (#5385) 2025-04-05 01:08:34 +00:00
Valentin Balaschenko
e923ec6d36 Fix to correct memory ordering for compare_exchange_weak and wait in the intrusive reference counting logic (#5381)
This change addresses a memory ordering assertion failure observed on one of the Windows test machines during the IntrusiveShared_test suite.
2025-04-04 18:21:17 +00:00
Vlad
851d99d99e fix: uint128 ambiguousness breaking macos unity build (#5386) 2025-04-04 08:28:33 -04:00
Bart
f608e653ca Fix undefined uint128_t type on Windows non-unity builds (#5377)
As part of import optimization, a transitive include had been removed that defined `BOOST_COMP_MSVC` on Windows. In unity builds, this definition was pulled in, but in non-unity builds it was not - causing a compilation error. An inspection of the Boost code revealed that we can just gate the statements by `_MS_VER` instead. A `#pragma message` is added to verify that the statement is only printed on Windows builds.
2025-04-01 11:21:59 -04:00
Vlad
72e076b694 test: enable compile time param to change reference fee value (#5159)
Adds an extra CI pipeline to perform unit tests using different values for fees.
2025-03-27 23:40:36 +00:00
Bart
6cf37c4abe refactor: Move integration tests from 'examples/' into 'tests/' (#5367)
This change moves `examples/example` into `tests/conan` to make it clear it is an integration test, and adjusts the `conan` CI job accordingly
2025-03-27 14:49:09 +00:00
Valentin Balaschenko
fc204773d6 Intrusive SHAMap smart pointers for efficient memory use and lock-free synchronization (#5152)
The main goal of this optimisation is memory reduction in SHAMapTreeNodes by introducing intrusive pointers instead of standard std::shared_ptr and std::weak_ptr.
2025-03-25 18:40:25 +00:00
Vlad
2bc5cb240f test: enable unit tests to work with variable reference fee (#5145)
Fix remaining unit tests to be able to process reference fee values other than 10.
2025-03-25 10:31:25 -04:00
Vlad
67028d6ea6 test: enable TxQ unit tests work with variable reference fee (#5118)
In preparation for a potential reference fee change we would like to verify that fee change works as expected. The first step is to fix all unit tests to be able to work with different reference fee values.
2025-03-24 14:56:19 -04:00
Ed Hennis
d22a5057b9 Prevent consensus from getting stuck in the establish phase (#5277)
- Detects if the consensus process is "stalled". If it is, then we can declare a 
  consensus and end successfully even if we do not have 80% agreement on
  our proposal.
  - "Stalled" is defined as:
    - We have a close time consensus
    - Each disputed transaction is individually stalled:
      - It has been in the final "stuck" 95% requirement for at least 2
        (avMIN_ROUNDS) "inner rounds" of phaseEstablish,
      - and either all of the other trusted proposers or this validator, if proposing,
        have had the same vote(s) for at least 4 (avSTALLED_ROUNDS) "inner
        rounds", and at least 80% of the validators (including this one, if
        appropriate) agree about the vote (whether yes or no).
- If we have been in the establish phase for more than 10x the previous
  consensus establish phase's time, then consensus is considered "expired",
  and we will leave the round, which sends a partial validation (indicating
  that the node is moving on without validating). Two restrictions avoid
  prematurely exiting, or having an extended exit in extreme situations.
  - The 10x time is clamped to be within a range of 15s
    (ledgerMAX_CONSENSUS) to 120s (ledgerABANDON_CONSENSUS).
  - If consensus has not had an opportunity to walk through all avalanche
    states (defined as not going through 8 "inner rounds" of phaseEstablish),
    then ConsensusState::Expired is treated as ConsensusState::No.
- When enough nodes leave the round, any remaining nodes will see they've
  fallen behind, and move on, too, generally before hitting the timeout. Any
  validations or partial validations sent during this time will help the
  consensus process bring the nodes back together.
2025-03-20 12:41:44 -04:00
Alex Kremer
75a20194c5 chore: Update link to ripple-binary-codec (#5355)
The link to ripple-binary-codec's definitions.json appears to be outdated. The updated link is also documented here: https://xrpl.org/docs/references/protocol/binary-format#definitions-file
2025-03-19 17:33:23 -04:00
Alex Kremer
7fe81fe62e chore: Add PR number to payload (#5310)
This PR adds one more payload field to the libXRPL compatibility check workflow - the PR number itself.
2025-03-18 17:26:08 +00:00
Bronek Kozicki
345ddc7234 fix: Remove null pointer deref, just do abort (#5338)
This change removes the existing undefined behavior from `LogicError`, so we can be certain that there will be always a stacktrace.

De-referencing a null pointer is an old trick to generate `SIGSEGV`, which would typically also create a stacktrace. However it is also an undefined behaviour and compilers can do something else. A more robust way to create a stacktrace while crashing the program is to use `std::abort`, which we have also used in this location for a long time. If we combine the two, we might not get the expected behaviour - namely, the nullpointer deref followed by `std::abort`, as handled in certain compiler versions may not immediately cause a crash. We have observed stacktrace being wiped instead, and thread put in indeterminate state, then stacktrace created without any useful information.
2025-03-18 12:45:25 -04:00
Bart
d167d4864f refactor: Updates Conan dependencies: RocksDB (#5335)
Updates RocksDB to version 9.7.3, the latest version supported in Conan 1.x. A patch for 9.7.4 that fixes a memory leak is included.
2025-03-18 11:25:48 -04:00
Vlad
bf504912a4 fix: trust line RPC no ripple flag (#5345)
The Trustline RPC `no_ripple` flag gets set depending on `lsfDefaultRipple` flag, which is not a flag of a trustline but of the account root. The `lsfDefaultRipple` flag does not provide any insight if this particular trust line has `lsfLowNoRipple` or `lsfHighNoRipple` flag set, so it should not be used here at all. This change simplifies the logic.
2025-03-18 09:03:03 -04:00
cyan317
a7fb8ae915 fix: Handle invalid marker parameter in grpc call (#5317)
The `end_marker` is used to limit the range of ledger entries to fetch. If `end_marker` is less than `marker`, a crash can occur. This change adds an additional check.
2025-03-18 08:21:33 -04:00
Sergey Kuznetsov
d9b7a2688f fix: Error message for ledger_entry rpc (#5344)
Changes the error to `malformedAddress` for `permissioned_domain` in the `ledger_entry` rpc, when the account is not a string. This change makes it more clear to a user what is wrong with their request.
2025-03-17 09:14:49 -04:00
Darius Tumas
c0299dba88 Adds hub.xrpl-commons.org as a new Bootstrap Cluster (#5263) 2025-03-17 07:04:46 -04:00
Bronek Kozicki
c3ecdb4746 Rename "deadlock" to "stall" in LoadManager (#5341)
What the LoadManager class does is stall detection, which is not the same as deadlock detection. In the condition of severe CPU starvation, LoadManager will currently intentionally crash rippled reporting `LogicError: Deadlock detected`. This error message is misleading as the condition being detected is not a deadlock. This change fixes and refactors the code in response.
2025-03-14 16:15:09 -04:00
Ed Hennis
c17676a9be refactor: Improve ordering of headers with clang-format (#5343)
Removes all manual header groupings from source and header files by leveraging clang-format options.
2025-03-12 18:33:21 -04:00
Ed Hennis
ed8e32cc92 refactor: Calculate numFeatures automatically (#5324)
Requiring manual updates of numFeatures is an annoying manual process that is easily forgotten, and leads to frequent merge conflicts. This change takes advantage of the `XRPL_FEATURE` and `XRPL_FIX` macros, and adds a new `XRPL_RETIRE` macro to automatically set `numFeatures`.
2025-03-12 17:34:06 -04:00
Bart
2406b28e64 refactor: Remove unused and add missing includes (#5293)
The codebase is filled with includes that are unused, and which thus can be removed. At the same time, the files often do not include all headers that contain the definitions used in those files. This change uses clang-format and clang-tidy to clean up the includes, with minor manual intervention to ensure the code compiles on all platforms.
2025-03-11 14:16:45 -04:00
1150 changed files with 49844 additions and 16593 deletions

View File

@@ -44,6 +44,7 @@ DerivePointerAlignment: false
DisableFormat: false
ExperimentalAutoDetectBinPacking: false
ForEachMacros: [ Q_FOREACH, BOOST_FOREACH ]
IncludeBlocks: Regroup
IncludeCategories:
- Regex: '^<(test)/'
Priority: 0
@@ -53,8 +54,12 @@ IncludeCategories:
Priority: 2
- Regex: '^<(boost)/'
Priority: 3
- Regex: '.*'
- Regex: '^.*/'
Priority: 4
- Regex: '^.*\.h'
Priority: 5
- Regex: '.*'
Priority: 6
IncludeIsMainRegex: '$'
IndentCaseLabels: true
IndentFunctionDeclarationAfterType: false
@@ -89,3 +94,4 @@ SpacesInSquareBrackets: false
Standard: Cpp11
TabWidth: 8
UseTab: Never
QualifierAlignment: Right

View File

@@ -7,13 +7,13 @@ comment:
show_carryforward_flags: false
coverage:
range: "60..80"
range: "70..85"
precision: 1
round: nearest
status:
project:
default:
target: 60%
target: 75%
threshold: 2%
patch:
default:

8
.github/CODEOWNERS vendored Normal file
View File

@@ -0,0 +1,8 @@
# Allow anyone to review any change by default.
*
# Require the rpc-reviewers team to review changes to the rpc code.
include/xrpl/protocol/ @xrplf/rpc-reviewers
src/libxrpl/protocol/ @xrplf/rpc-reviewers
src/xrpld/rpc/ @xrplf/rpc-reviewers
src/xrpld/app/misc/ @xrplf/rpc-reviewers

View File

@@ -14,7 +14,7 @@ runs:
run: |
conan config set general.revisions_enabled=1
conan export external/snappy snappy/1.1.10@
conan export external/rocksdb rocksdb/6.29.5@
conan export external/rocksdb rocksdb/9.7.3@
conan export external/soci soci/4.0.3@
conan export external/nudb nudb/2.0.8@
- name: add Ripple Conan remote
@@ -55,7 +55,3 @@ runs:
--options xrpld=True \
--settings build_type=${{ inputs.configuration }} \
..
- name: upload dependencies to remote
if: (steps.binaries.outputs.missing != '[]') && (steps.remote.outputs.outcome == 'success')
shell: bash
run: conan upload --remote ripple '*' --all --parallel --confirm

View File

@@ -1,9 +1,13 @@
name: clang-format
on: [push, pull_request]
on:
push:
pull_request:
types: [opened, reopened, synchronize, ready_for_review]
jobs:
check:
if: ${{ github.event_name == 'push' || github.event.pull_request.draft != true || contains(github.event.pull_request.labels.*.name, 'DraftRunCI') }}
runs-on: ubuntu-24.04
env:
CLANG_VERSION: 18
@@ -20,7 +24,7 @@ jobs:
sudo apt-get update
sudo apt-get install clang-format-${CLANG_VERSION}
- name: Format first-party sources
run: find include src -type f \( -name '*.cpp' -o -name '*.hpp' -o -name '*.h' -o -name '*.ipp' \) -exec clang-format-${CLANG_VERSION} -i {} +
run: find include src tests -type f \( -name '*.cpp' -o -name '*.hpp' -o -name '*.h' -o -name '*.ipp' \) -exec clang-format-${CLANG_VERSION} -i {} +
- name: Check for differences
id: assert
run: |

View File

@@ -10,11 +10,11 @@ concurrency:
cancel-in-progress: true
jobs:
job:
documentation:
runs-on: ubuntu-latest
permissions:
contents: write
container: rippleci/rippled-build-ubuntu:aaf5e3e
container: ghcr.io/xrplf/rippled-build-ubuntu:aaf5e3e
steps:
- name: checkout
uses: actions/checkout@v4

View File

@@ -1,9 +1,13 @@
name: levelization
on: [push, pull_request]
on:
push:
pull_request:
types: [opened, reopened, synchronize, ready_for_review]
jobs:
check:
if: ${{ github.event_name == 'push' || github.event.pull_request.draft != true || contains(github.event.pull_request.labels.*.name, 'DraftRunCI') }}
runs-on: ubuntu-latest
env:
CLANG_VERSION: 10

View File

@@ -1,6 +1,6 @@
name: Check libXRPL compatibility with Clio
env:
CONAN_URL: http://18.143.149.228:8081/artifactory/api/conan/conan-non-prod
CONAN_URL: http://18.143.149.228:8081/artifactory/api/conan/dev
CONAN_LOGIN_USERNAME_RIPPLE: ${{ secrets.CONAN_USERNAME }}
CONAN_PASSWORD_RIPPLE: ${{ secrets.CONAN_TOKEN }}
on:
@@ -8,19 +8,21 @@ on:
paths:
- 'src/libxrpl/protocol/BuildInfo.cpp'
- '.github/workflows/libxrpl.yml'
types: [opened, reopened, synchronize, ready_for_review]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
publish:
if: ${{ github.event_name == 'push' || github.event.pull_request.draft != true || contains(github.event.pull_request.labels.*.name, 'DraftRunCI') }}
name: Publish libXRPL
outputs:
outcome: ${{ steps.upload.outputs.outcome }}
version: ${{ steps.version.outputs.version }}
channel: ${{ steps.channel.outputs.channel }}
runs-on: [self-hosted, heavy]
container: rippleci/rippled-build-ubuntu:aaf5e3e
container: ghcr.io/xrplf/rippled-build-ubuntu:aaf5e3e
steps:
- name: Wait for essential checks to succeed
uses: lewagon/wait-on-check-action@v1.3.4
@@ -85,4 +87,5 @@ jobs:
run: |
gh api --method POST -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" \
/repos/xrplf/clio/dispatches -f "event_type=check_libxrpl" \
-F "client_payload[version]=${{ needs.publish.outputs.version }}@${{ needs.publish.outputs.channel }}"
-F "client_payload[version]=${{ needs.publish.outputs.version }}@${{ needs.publish.outputs.channel }}" \
-F "client_payload[pr]=${{ github.event.pull_request.number }}"

View File

@@ -1,6 +1,7 @@
name: macos
on:
pull_request:
types: [opened, reopened, synchronize, ready_for_review]
push:
# If the branches list is ever changed, be sure to change it on all
# build/test jobs (nix, macos, windows, instrumentation)
@@ -18,6 +19,7 @@ concurrency:
jobs:
test:
if: ${{ github.event_name == 'push' || github.event.pull_request.draft != true || contains(github.event.pull_request.labels.*.name, 'DraftRunCI') }}
strategy:
matrix:
platform:
@@ -69,11 +71,13 @@ jobs:
nproc --version
echo -n "nproc returns: "
nproc
system_profiler SPHardwareDataType
sysctl -n hw.logicalcpu
clang --version
- name: configure Conan
run : |
conan profile new default --detect || true
conan profile update settings.compiler.cppstd=20 default
conan profile update 'conf.tools.build:cxxflags+=["-DBOOST_ASIO_DISABLE_CONCEPTS"]' default
- name: build dependencies
uses: ./.github/actions/dependencies
env:
@@ -92,4 +96,7 @@ jobs:
run: |
n=$(nproc)
echo "Using $n test jobs"
${build_dir}/rippled --unittest --unittest-jobs $n
cd ${build_dir}
./rippled --unittest --unittest-jobs $n
ctest -j $n --output-on-failure

View File

@@ -1,6 +1,7 @@
name: nix
on:
pull_request:
types: [opened, reopened, synchronize, ready_for_review]
push:
# If the branches list is ever changed, be sure to change it on all
# build/test jobs (nix, macos, windows)
@@ -39,6 +40,7 @@ concurrency:
jobs:
dependencies:
if: ${{ github.event_name == 'push' || github.event.pull_request.draft != true || contains(github.event.pull_request.labels.*.name, 'DraftRunCI') }}
strategy:
fail-fast: false
matrix:
@@ -62,7 +64,7 @@ jobs:
cc: /usr/bin/clang-14
cxx: /usr/bin/clang++-14
runs-on: [self-hosted, heavy]
container: rippleci/rippled-build-ubuntu:aaf5e3e
container: ghcr.io/xrplf/rippled-build-ubuntu:aaf5e3e
env:
build_dir: .build
steps:
@@ -124,7 +126,7 @@ jobs:
- "-Dunity=ON"
needs: dependencies
runs-on: [self-hosted, heavy]
container: rippleci/rippled-build-ubuntu:aaf5e3e
container: ghcr.io/xrplf/rippled-build-ubuntu:aaf5e3e
env:
build_dir: .build
steps:
@@ -161,8 +163,65 @@ jobs:
cmake-args: "-Dassert=TRUE -Dwerr=TRUE ${{ matrix.cmake-args }}"
- name: test
run: |
${build_dir}/rippled --unittest --unittest-jobs $(nproc)
cd ${build_dir}
./rippled --unittest --unittest-jobs $(nproc)
ctest -j $(nproc) --output-on-failure
reference-fee-test:
strategy:
fail-fast: false
matrix:
platform:
- linux
compiler:
- gcc
configuration:
- Debug
cmake-args:
- "-DUNIT_TEST_REFERENCE_FEE=200"
- "-DUNIT_TEST_REFERENCE_FEE=1000"
needs: dependencies
runs-on: [self-hosted, heavy]
container: ghcr.io/xrplf/rippled-build-ubuntu:aaf5e3e
env:
build_dir: .build
steps:
- name: upgrade conan
run: |
pip install --upgrade "conan<2"
- name: download cache
uses: actions/download-artifact@v4
with:
name: ${{ matrix.platform }}-${{ matrix.compiler }}-${{ matrix.configuration }}
- name: extract cache
run: |
mkdir -p ~/.conan
tar -xzf conan.tar -C ~/.conan
- name: check environment
run: |
env | sort
echo ${PATH} | tr ':' '\n'
conan --version
cmake --version
- name: checkout
uses: actions/checkout@v4
- name: dependencies
uses: ./.github/actions/dependencies
env:
CONAN_URL: http://18.143.149.228:8081/artifactory/api/conan/conan-non-prod
with:
configuration: ${{ matrix.configuration }}
- name: build
uses: ./.github/actions/build
with:
generator: Ninja
configuration: ${{ matrix.configuration }}
cmake-args: "-Dassert=TRUE -Dwerr=TRUE ${{ matrix.cmake-args }}"
- name: test
run: |
cd ${build_dir}
./rippled --unittest --unittest-jobs $(nproc)
ctest -j $(nproc) --output-on-failure
coverage:
strategy:
fail-fast: false
@@ -175,7 +234,7 @@ jobs:
- Debug
needs: dependencies
runs-on: [self-hosted, heavy]
container: rippleci/rippled-build-ubuntu:aaf5e3e
container: ghcr.io/xrplf/rippled-build-ubuntu:aaf5e3e
env:
build_dir: .build
steps:
@@ -191,7 +250,7 @@ jobs:
mkdir -p ~/.conan
tar -xzf conan.tar -C ~/.conan
- name: install gcovr
run: pip install "gcovr>=7,<8"
run: pip install "gcovr>=7,<9"
- name: check environment
run: |
echo ${PATH} | tr ':' '\n'
@@ -249,7 +308,7 @@ jobs:
conan:
needs: dependencies
runs-on: [self-hosted, heavy]
container: rippleci/rippled-build-ubuntu:aaf5e3e
container: ghcr.io/xrplf/rippled-build-ubuntu:aaf5e3e
env:
build_dir: .build
configuration: Release
@@ -288,7 +347,7 @@ jobs:
echo "reference=${reference}" >> "${GITHUB_ENV}"
- name: build
run: |
cd examples/example
cd tests/conan
mkdir ${build_dir}
cd ${build_dir}
conan install .. --output-folder . \
@@ -304,6 +363,7 @@ jobs:
# later
instrumentation-build:
if: ${{ github.event_name == 'push' || github.event.pull_request.draft != true || contains(github.event.pull_request.labels.*.name, 'DraftRunCI') }}
env:
CLANG_RELEASE: 16
strategy:
@@ -384,3 +444,4 @@ jobs:
run: |
cd ${BUILD_DIR}
./rippled -u --unittest-jobs $(( $(nproc)/4 ))
ctest -j $(nproc) --output-on-failure

View File

@@ -2,6 +2,7 @@ name: windows
on:
pull_request:
types: [opened, reopened, synchronize, ready_for_review]
push:
# If the branches list is ever changed, be sure to change it on all
# build/test jobs (nix, macos, windows, instrumentation)
@@ -21,6 +22,7 @@ concurrency:
jobs:
test:
if: ${{ github.event_name == 'push' || github.event.pull_request.draft != true || contains(github.event.pull_request.labels.*.name, 'DraftRunCI') }}
strategy:
fail-fast: false
matrix:
@@ -93,5 +95,6 @@ jobs:
shell: bash
if: ${{ matrix.configuration.tests }}
run: |
${build_dir}/${{ matrix.configuration.type }}/rippled --unittest \
--unittest-jobs $(nproc)
cd ${build_dir}/${{ matrix.configuration.type }}
./rippled --unittest --unittest-jobs $(nproc)
ctest -j $(nproc) --output-on-failure

View File

@@ -83,9 +83,17 @@ The [commandline](https://xrpl.org/docs/references/http-websocket-apis/api-conve
The `network_id` field was added in the `server_info` response in version 1.5.0 (2019), but it is not returned in [reporting mode](https://xrpl.org/rippled-server-modes.html#reporting-mode). However, use of reporting mode is now discouraged, in favor of using [Clio](https://github.com/XRPLF/clio) instead.
## XRP Ledger server version 2.5.0
As of 2025-04-04, version 2.5.0 is in development. You can use a pre-release version by building from source or [using the `nightly` package](https://xrpl.org/docs/infrastructure/installation/install-rippled-on-ubuntu).
### Additions and bugfixes in 2.5.0
- `channel_authorize`: If `signing_support` is not enabled in the config, the RPC is disabled.
## XRP Ledger server version 2.4.0
As of 2025-01-28, version 2.4.0 is in development. You can use a pre-release version by building from source or [using the `nightly` package](https://xrpl.org/docs/infrastructure/installation/install-rippled-on-ubuntu).
[Version 2.4.0](https://github.com/XRPLF/rippled/releases/tag/2.4.0) was released on March 4, 2025.
### Additions and bugfixes in 2.4.0

View File

@@ -178,9 +178,9 @@ It does not override paths to dependencies when building with Visual Studio.
```
# Conan 1.x
conan export external/rocksdb rocksdb/6.29.5@
conan export external/rocksdb rocksdb/9.7.3@
# Conan 2.x
conan export --version 6.29.5 external/rocksdb
conan export --version 9.7.3 external/rocksdb
```
Export our [Conan recipe for SOCI](./external/soci).
@@ -288,7 +288,7 @@ It fixes some source files to add missing `#include`s.
Single-config generators:
```
cmake --build .
cmake --build . -j $(nproc)
```
Multi-config generators:

View File

@@ -14,10 +14,10 @@ Loop: xrpld.app xrpld.net
xrpld.app > xrpld.net
Loop: xrpld.app xrpld.overlay
xrpld.overlay == xrpld.app
xrpld.overlay > xrpld.app
Loop: xrpld.app xrpld.peerfinder
xrpld.app > xrpld.peerfinder
xrpld.peerfinder ~= xrpld.app
Loop: xrpld.app xrpld.rpc
xrpld.rpc > xrpld.app

View File

@@ -6,6 +6,7 @@ libxrpl.protocol > xrpl.basics
libxrpl.protocol > xrpl.json
libxrpl.protocol > xrpl.protocol
libxrpl.resource > xrpl.basics
libxrpl.resource > xrpl.json
libxrpl.resource > xrpl.resource
libxrpl.server > xrpl.basics
libxrpl.server > xrpl.json
@@ -42,6 +43,7 @@ test.consensus > xrpl.basics
test.consensus > xrpld.app
test.consensus > xrpld.consensus
test.consensus > xrpld.ledger
test.consensus > xrpl.json
test.core > test.jtx
test.core > test.toplevel
test.core > test.unit_test
@@ -58,7 +60,6 @@ test.json > test.jtx
test.json > xrpl.json
test.jtx > xrpl.basics
test.jtx > xrpld.app
test.jtx > xrpld.consensus
test.jtx > xrpld.core
test.jtx > xrpld.ledger
test.jtx > xrpld.net
@@ -131,6 +132,7 @@ test.shamap > xrpl.protocol
test.toplevel > test.csf
test.toplevel > xrpl.json
test.unit_test > xrpl.basics
tests.libxrpl > xrpl.basics
xrpl.json > xrpl.basics
xrpl.protocol > xrpl.basics
xrpl.protocol > xrpl.json
@@ -158,7 +160,6 @@ xrpld.core > xrpl.basics
xrpld.core > xrpl.json
xrpld.core > xrpl.protocol
xrpld.ledger > xrpl.basics
xrpld.ledger > xrpld.core
xrpld.ledger > xrpl.json
xrpld.ledger > xrpl.protocol
xrpld.net > xrpl.basics
@@ -183,7 +184,6 @@ xrpld.peerfinder > xrpld.core
xrpld.peerfinder > xrpl.protocol
xrpld.perflog > xrpl.basics
xrpld.perflog > xrpl.json
xrpld.perflog > xrpl.protocol
xrpld.rpc > xrpl.basics
xrpld.rpc > xrpld.core
xrpld.rpc > xrpld.ledger

View File

@@ -16,6 +16,18 @@ set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
if(CMAKE_CXX_COMPILER_ID MATCHES "GNU")
# GCC-specific fixes
add_compile_options(-Wno-unknown-pragmas -Wno-subobject-linkage)
# -Wno-subobject-linkage can be removed when we upgrade GCC version to at least 13.3
elseif(CMAKE_CXX_COMPILER_ID MATCHES "Clang")
# Clang-specific fixes
add_compile_options(-Wno-unknown-warning-option) # Ignore unknown warning options
elseif(MSVC)
# MSVC-specific fixes
add_compile_options(/wd4068) # Ignore unknown pragmas
endif()
# make GIT_COMMIT_HASH define available to all sources
find_package(Git)
if(Git_FOUND)
@@ -78,6 +90,11 @@ set_target_properties(OpenSSL::SSL PROPERTIES
INTERFACE_COMPILE_DEFINITIONS OPENSSL_NO_SSL2
)
set(SECP256K1_INSTALL TRUE)
set(SECP256K1_BUILD_BENCHMARK FALSE)
set(SECP256K1_BUILD_TESTS FALSE)
set(SECP256K1_BUILD_EXHAUSTIVE_TESTS FALSE)
set(SECP256K1_BUILD_CTIME_TESTS FALSE)
set(SECP256K1_BUILD_EXAMPLES FALSE)
add_subdirectory(external/secp256k1)
add_library(secp256k1::secp256k1 ALIAS secp256k1)
add_subdirectory(external/ed25519-donna)
@@ -132,3 +149,8 @@ set(PROJECT_EXPORT_SET RippleExports)
include(RippledCore)
include(RippledInstall)
include(RippledValidatorKeys)
if(tests)
include(CTest)
add_subdirectory(src/tests/libxrpl)
endif()

View File

@@ -1,3 +1,5 @@
[![codecov](https://codecov.io/gh/XRPLF/rippled/graph/badge.svg?token=WyFr5ajq3O)](https://codecov.io/gh/XRPLF/rippled)
# The XRP Ledger
The [XRP Ledger](https://xrpl.org/) is a decentralized cryptographic ledger powered by a network of peer-to-peer nodes. The XRP Ledger uses a novel Byzantine Fault Tolerant consensus algorithm to settle and record transactions in a secure distributed database without a central operator.

File diff suppressed because it is too large Load Diff

View File

@@ -83,7 +83,7 @@ To report a qualifying bug, please send a detailed report to:
|Long Key ID | `0xCD49A0AFC57929BE` |
|Fingerprint | `24E6 3B02 37E0 FA9C 5E96 8974 CD49 A0AF C579 29BE` |
The full PGP key for this address, which is also available on several key servers (e.g. on [keys.gnupg.net](https://keys.gnupg.net)), is:
The full PGP key for this address, which is also available on several key servers (e.g. on [keyserver.ubuntu.com](https://keyserver.ubuntu.com)), is:
```
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBFUwGHYBEAC0wpGpBPkd8W1UdQjg9+cEFzeIEJRaoZoeuJD8mofwI5Ejnjdt

View File

@@ -420,6 +420,7 @@
# - r.ripple.com 51235
# - sahyadri.isrdc.in 51235
# - hubs.xrpkuwait.com 51235
# - hub.xrpl-commons.org 51235
#
# Examples:
#

View File

@@ -26,7 +26,7 @@
#
# Examples:
# https://vl.ripple.com
# https://vl.xrplf.org
# https://unl.xrplf.org
# http://127.0.0.1:8000
# file:///etc/opt/ripple/vl.txt
#

View File

@@ -98,6 +98,9 @@
# 2024-04-03, Bronek Kozicki
# - add support for output formats: jacoco, clover, lcov
#
# 2025-05-12, Jingchen Wu
# - add -fprofile-update=atomic to ensure atomic profile generation
#
# USAGE:
#
# 1. Copy this file into your cmake modules path.
@@ -200,15 +203,27 @@ set(COVERAGE_COMPILER_FLAGS "-g --coverage"
CACHE INTERNAL "")
if(CMAKE_CXX_COMPILER_ID MATCHES "(GNU|Clang)")
include(CheckCXXCompilerFlag)
include(CheckCCompilerFlag)
check_cxx_compiler_flag(-fprofile-abs-path HAVE_cxx_fprofile_abs_path)
if(HAVE_cxx_fprofile_abs_path)
set(COVERAGE_CXX_COMPILER_FLAGS "${COVERAGE_COMPILER_FLAGS} -fprofile-abs-path")
endif()
include(CheckCCompilerFlag)
check_c_compiler_flag(-fprofile-abs-path HAVE_c_fprofile_abs_path)
if(HAVE_c_fprofile_abs_path)
set(COVERAGE_C_COMPILER_FLAGS "${COVERAGE_COMPILER_FLAGS} -fprofile-abs-path")
endif()
check_cxx_compiler_flag(-fprofile-update HAVE_cxx_fprofile_update)
if(HAVE_cxx_fprofile_update)
set(COVERAGE_CXX_COMPILER_FLAGS "${COVERAGE_COMPILER_FLAGS} -fprofile-update=atomic")
endif()
check_c_compiler_flag(-fprofile-update HAVE_c_fprofile_update)
if(HAVE_c_fprofile_update)
set(COVERAGE_C_COMPILER_FLAGS "${COVERAGE_COMPILER_FLAGS} -fprofile-update=atomic")
endif()
endif()
set(CMAKE_Fortran_FLAGS_COVERAGE

View File

@@ -136,6 +136,9 @@ if(xrpld)
add_executable(rippled)
if(tests)
target_compile_definitions(rippled PUBLIC ENABLE_TESTS)
target_compile_definitions(rippled PRIVATE
UNIT_TEST_REFERENCE_FEE=${UNIT_TEST_REFERENCE_FEE}
)
endif()
target_include_directories(rippled
PRIVATE

View File

@@ -53,9 +53,9 @@ set(download_script "${CMAKE_BINARY_DIR}/docs/download-cppreference.cmake")
file(WRITE
"${download_script}"
"file(DOWNLOAD \
http://upload.cppreference.com/mwiki/images/b/b2/html_book_20190607.zip \
https://github.com/PeterFeicht/cppreference-doc/releases/download/v20250209/html-book-20250209.zip \
${CMAKE_BINARY_DIR}/docs/cppreference.zip \
EXPECTED_HASH MD5=82b3a612d7d35a83e3cb1195a63689ab \
EXPECTED_HASH MD5=bda585f72fbca4b817b29a3d5746567b \
)\n \
execute_process( \
COMMAND \"${CMAKE_COMMAND}\" -E tar -xf cppreference.zip \

View File

@@ -2,16 +2,6 @@
convenience variables and sanity checks
#]===================================================================]
include(ProcessorCount)
if (NOT ep_procs)
ProcessorCount(ep_procs)
if (ep_procs GREATER 1)
# never use more than half of cores for EP builds
math (EXPR ep_procs "${ep_procs} / 2")
message (STATUS "Using ${ep_procs} cores for ExternalProject builds.")
endif ()
endif ()
get_property(is_multiconfig GLOBAL PROPERTY GENERATOR_IS_MULTI_CONFIG)
set (CMAKE_CONFIGURATION_TYPES "Debug;Release" CACHE STRING "" FORCE)

View File

@@ -11,8 +11,14 @@ option(assert "Enables asserts, even in release builds" OFF)
option(xrpld "Build xrpld" ON)
option(tests "Build tests" ON)
if(tests)
# This setting allows making a separate workflow to test fees other than default 10
if(NOT UNIT_TEST_REFERENCE_FEE)
set(UNIT_TEST_REFERENCE_FEE "10" CACHE STRING "")
endif()
endif()
option(unity "Creates a build using UNITY support in cmake. This is the default" ON)
option(unity "Creates a build using UNITY support in cmake." OFF)
if(unity)
if(NOT is_ci)
set(CMAKE_UNITY_BUILD_BATCH_SIZE 15 CACHE STRING "")

View File

@@ -2,7 +2,6 @@ find_package(Boost 1.82 REQUIRED
COMPONENTS
chrono
container
context
coroutine
date_time
filesystem
@@ -24,7 +23,7 @@ endif()
target_link_libraries(ripple_boost
INTERFACE
Boost::boost
Boost::headers
Boost::chrono
Boost::container
Boost::coroutine

41
cmake/xrpl_add_test.cmake Normal file
View File

@@ -0,0 +1,41 @@
include(isolate_headers)
function(xrpl_add_test name)
set(target ${PROJECT_NAME}.test.${name})
file(GLOB_RECURSE sources CONFIGURE_DEPENDS
"${CMAKE_CURRENT_SOURCE_DIR}/${name}/*.cpp"
"${CMAKE_CURRENT_SOURCE_DIR}/${name}.cpp"
)
add_executable(${target} EXCLUDE_FROM_ALL ${ARGN} ${sources})
isolate_headers(
${target}
"${CMAKE_SOURCE_DIR}"
"${CMAKE_SOURCE_DIR}/tests/${name}"
PRIVATE
)
# Make sure the test isn't optimized away in unity builds
set_target_properties(${target} PROPERTIES
UNITY_BUILD_MODE GROUP
UNITY_BUILD_BATCH_SIZE 0) # Adjust as needed
add_test(NAME ${target} COMMAND ${target})
set_tests_properties(
${target} PROPERTIES
FIXTURES_REQUIRED ${target}_fixture
)
add_test(
NAME ${target}.build
COMMAND
${CMAKE_COMMAND}
--build ${CMAKE_BINARY_DIR}
--config $<CONFIG>
--target ${target}
)
set_tests_properties(${target}.build PROPERTIES
FIXTURES_SETUP ${target}_fixture
)
endfunction()

View File

@@ -1,4 +1,4 @@
from conan import ConanFile
from conan import ConanFile, __version__ as conan_version
from conan.tools.cmake import CMake, CMakeToolchain, cmake_layout
import re
@@ -24,13 +24,12 @@ class Xrpl(ConanFile):
}
requires = [
'date/3.0.3',
'doctest/2.4.11',
'grpc/1.50.1',
'libarchive/3.7.6',
'nudb/2.0.8',
'openssl/1.1.1v',
'soci/4.0.3',
'xxhash/0.8.2',
'zlib/1.3.1',
]
@@ -99,14 +98,18 @@ class Xrpl(ConanFile):
self.options['boost'].visibility = 'global'
def requirements(self):
self.requires('boost/1.83.0', force=True)
# Conan 2 requires transitive headers to be specified
transitive_headers_opt = {'transitive_headers': True} if conan_version.split('.')[0] == '2' else {}
self.requires('boost/1.83.0', force=True, **transitive_headers_opt)
self.requires('date/3.0.3', **transitive_headers_opt)
self.requires('lz4/1.10.0', force=True)
self.requires('protobuf/3.21.9', force=True)
self.requires('sqlite3/3.47.0', force=True)
if self.options.jemalloc:
self.requires('jemalloc/5.3.0')
if self.options.rocksdb:
self.requires('rocksdb/6.29.5')
self.requires('rocksdb/9.7.3')
self.requires('xxhash/0.8.2', **transitive_headers_opt)
exports_sources = (
'CMakeLists.txt',
@@ -161,7 +164,17 @@ class Xrpl(ConanFile):
# `include/`, not `include/ripple/proto/`.
libxrpl.includedirs = ['include', 'include/ripple/proto']
libxrpl.requires = [
'boost::boost',
'boost::headers',
'boost::chrono',
'boost::container',
'boost::coroutine',
'boost::date_time',
'boost::filesystem',
'boost::json',
'boost::program_options',
'boost::regex',
'boost::system',
'boost::thread',
'date::date',
'grpc::grpc++',
'libarchive::libarchive',

View File

@@ -23,7 +23,7 @@ direction.
```
apt update
apt install --yes curl git libssl-dev python3.10-dev python3-pip make g++-11 libprotobuf-dev protobuf-compiler
apt install --yes curl git libssl-dev pipx python3.10-dev python3-pip make g++-11 libprotobuf-dev protobuf-compiler
curl --location --remote-name \
"https://github.com/Kitware/CMake/releases/download/v3.25.1/cmake-3.25.1.tar.gz"
@@ -35,7 +35,8 @@ make --jobs $(nproc)
make install
cd ..
pip3 install 'conan<2'
pipx install 'conan<2'
pipx ensurepath
```
[1]: https://github.com/thejohnfreeman/rippled-docker/blob/master/ubuntu-22.04/install.sh

View File

@@ -558,7 +558,7 @@ struct ConsensusResult
ConsensusTimer roundTime;
// Indicates state in which consensus ended. Once in the accept phase
// will be either Yes or MovedOn
// will be either Yes or MovedOn or Expired
ConsensusState state = ConsensusState::No;
};

View File

@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 3.25)
cmake_minimum_required(VERSION 3.18)
# Note, version set explicitly by rippled project
project(antithesis-sdk-cpp VERSION 0.4.4 LANGUAGES CXX)

View File

@@ -1,27 +1,12 @@
sources:
"6.29.5":
url: "https://github.com/facebook/rocksdb/archive/refs/tags/v6.29.5.tar.gz"
sha256: "ddbf84791f0980c0bbce3902feb93a2c7006f6f53bfd798926143e31d4d756f0"
"6.27.3":
url: "https://github.com/facebook/rocksdb/archive/refs/tags/v6.27.3.tar.gz"
sha256: "ee29901749b9132692b26f0a6c1d693f47d1a9ed8e3771e60556afe80282bf58"
"6.20.3":
url: "https://github.com/facebook/rocksdb/archive/refs/tags/v6.20.3.tar.gz"
sha256: "c6502c7aae641b7e20fafa6c2b92273d935d2b7b2707135ebd9a67b092169dca"
"8.8.1":
url: "https://github.com/facebook/rocksdb/archive/refs/tags/v8.8.1.tar.gz"
sha256: "056c7e21ad8ae36b026ac3b94b9d6e0fcc60e1d937fc80330921e4181be5c36e"
"9.7.3":
url: "https://github.com/facebook/rocksdb/archive/refs/tags/v9.7.3.tar.gz"
sha256: "acfabb989cbfb5b5c4d23214819b059638193ec33dad2d88373c46448d16d38b"
patches:
"6.29.5":
- patch_file: "patches/6.29.5-0001-add-include-cstdint-for-gcc-13.patch"
patch_description: "Fix build with gcc 13 by including cstdint"
patch_type: "portability"
patch_source: "https://github.com/facebook/rocksdb/pull/11118"
- patch_file: "patches/6.29.5-0002-exclude-thirdparty.patch"
"9.7.3":
- patch_file: "patches/9.x.x-0001-exclude-thirdparty.patch"
patch_description: "Do not include thirdparty.inc"
patch_type: "portability"
"6.27.3":
- patch_file: "patches/6.27.3-0001-add-include-cstdint-for-gcc-13.patch"
patch_description: "Fix build with gcc 13 by including cstdint"
- patch_file: "patches/9.7.3-0001-memory-leak.patch"
patch_description: "Fix a leak of obsolete blob files left open until DB::Close()"
patch_type: "portability"
patch_source: "https://github.com/facebook/rocksdb/pull/11118"

View File

@@ -15,10 +15,10 @@ required_conan_version = ">=1.53.0"
class RocksDBConan(ConanFile):
name = "rocksdb"
homepage = "https://github.com/facebook/rocksdb"
description = "A library that provides an embeddable, persistent key-value store for fast storage"
license = ("GPL-2.0-only", "Apache-2.0")
url = "https://github.com/conan-io/conan-center-index"
description = "A library that provides an embeddable, persistent key-value store for fast storage"
homepage = "https://github.com/facebook/rocksdb"
topics = ("database", "leveldb", "facebook", "key-value")
package_type = "library"
settings = "os", "arch", "compiler", "build_type"
@@ -58,12 +58,12 @@ class RocksDBConan(ConanFile):
@property
def _compilers_minimum_version(self):
return {} if self._min_cppstd == "11" else {
"apple-clang": "10",
"clang": "7",
"gcc": "7",
"msvc": "191",
"Visual Studio": "15",
}
"apple-clang": "10",
"clang": "7",
"gcc": "7",
"msvc": "191",
"Visual Studio": "15",
}
def export_sources(self):
export_conandata_patches(self)
@@ -115,9 +115,9 @@ class RocksDBConan(ConanFile):
check_min_vs(self, "191")
if self.version == "6.20.3" and \
self.settings.os == "Linux" and \
self.settings.compiler == "gcc" and \
Version(self.settings.compiler.version) < "5":
self.settings.os == "Linux" and \
self.settings.compiler == "gcc" and \
Version(self.settings.compiler.version) < "5":
raise ConanInvalidConfiguration("Rocksdb 6.20.3 is not compilable with gcc <5.") # See https://github.com/facebook/rocksdb/issues/3522
def source(self):
@@ -163,6 +163,8 @@ class RocksDBConan(ConanFile):
if self.options.with_jemalloc:
deps.set_property("jemalloc", "cmake_file_name", "JeMalloc")
deps.set_property("jemalloc", "cmake_target_name", "JeMalloc::JeMalloc")
if self.options.with_zstd:
deps.set_property("zstd", "cmake_target_name", "zstd::zstd")
deps.generate()
def build(self):

View File

@@ -1,30 +0,0 @@
--- a/include/rocksdb/utilities/checkpoint.h
+++ b/include/rocksdb/utilities/checkpoint.h
@@ -8,6 +8,7 @@
#pragma once
#ifndef ROCKSDB_LITE
+#include <cstdint>
#include <string>
#include <vector>
#include "rocksdb/status.h"
--- a/table/block_based/data_block_hash_index.h
+++ b/table/block_based/data_block_hash_index.h
@@ -5,6 +5,7 @@
#pragma once
+#include <cstdint>
#include <string>
#include <vector>
--- a/util/string_util.h
+++ b/util/string_util.h
@@ -6,6 +6,7 @@
#pragma once
+#include <cstdint>
#include <sstream>
#include <string>
#include <unordered_map>

View File

@@ -1,16 +0,0 @@
diff --git a/CMakeLists.txt b/CMakeLists.txt
index ec59d4491..35577c998 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -101 +100,0 @@ if(MSVC)
- option(WITH_GFLAGS "build with GFlags" OFF)
@@ -103,2 +102,2 @@ if(MSVC)
- include(${CMAKE_CURRENT_SOURCE_DIR}/thirdparty.inc)
-else()
+endif()
+
@@ -117 +116 @@ else()
- if(MINGW)
+ if(MINGW OR MSVC)
@@ -183 +181,0 @@ else()
-endif()

View File

@@ -0,0 +1,319 @@
diff --git a/HISTORY.md b/HISTORY.md
index 36d472229..05ad1a202 100644
--- a/HISTORY.md
+++ b/HISTORY.md
@@ -1,6 +1,10 @@
# Rocksdb Change Log
> NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt`
+## 9.7.4 (10/31/2024)
+### Bug Fixes
+* Fix a leak of obsolete blob files left open until DB::Close(). This bug was introduced in version 9.4.0.
+
## 9.7.3 (10/16/2024)
### Behavior Changes
* OPTIONS file to be loaded by remote worker is now preserved so that it does not get purged by the primary host. A similar technique as how we are preserving new SST files from getting purged is used for this. min_options_file_numbers_ is tracked like pending_outputs_ is tracked.
diff --git a/db/blob/blob_file_cache.cc b/db/blob/blob_file_cache.cc
index 5f340aadf..1b9faa238 100644
--- a/db/blob/blob_file_cache.cc
+++ b/db/blob/blob_file_cache.cc
@@ -42,6 +42,7 @@ Status BlobFileCache::GetBlobFileReader(
assert(blob_file_reader);
assert(blob_file_reader->IsEmpty());
+ // NOTE: sharing same Cache with table_cache
const Slice key = GetSliceForKey(&blob_file_number);
assert(cache_);
@@ -98,4 +99,13 @@ Status BlobFileCache::GetBlobFileReader(
return Status::OK();
}
+void BlobFileCache::Evict(uint64_t blob_file_number) {
+ // NOTE: sharing same Cache with table_cache
+ const Slice key = GetSliceForKey(&blob_file_number);
+
+ assert(cache_);
+
+ cache_.get()->Erase(key);
+}
+
} // namespace ROCKSDB_NAMESPACE
diff --git a/db/blob/blob_file_cache.h b/db/blob/blob_file_cache.h
index 740e67ada..6858d012b 100644
--- a/db/blob/blob_file_cache.h
+++ b/db/blob/blob_file_cache.h
@@ -36,6 +36,15 @@ class BlobFileCache {
uint64_t blob_file_number,
CacheHandleGuard<BlobFileReader>* blob_file_reader);
+ // Called when a blob file is obsolete to ensure it is removed from the cache
+ // to avoid effectively leaking the open file and assicated memory
+ void Evict(uint64_t blob_file_number);
+
+ // Used to identify cache entries for blob files (not normally useful)
+ static const Cache::CacheItemHelper* GetHelper() {
+ return CacheInterface::GetBasicHelper();
+ }
+
private:
using CacheInterface =
BasicTypedCacheInterface<BlobFileReader, CacheEntryRole::kMisc>;
diff --git a/db/column_family.h b/db/column_family.h
index e4b7adde8..86637736a 100644
--- a/db/column_family.h
+++ b/db/column_family.h
@@ -401,6 +401,7 @@ class ColumnFamilyData {
SequenceNumber earliest_seq);
TableCache* table_cache() const { return table_cache_.get(); }
+ BlobFileCache* blob_file_cache() const { return blob_file_cache_.get(); }
BlobSource* blob_source() const { return blob_source_.get(); }
// See documentation in compaction_picker.h
diff --git a/db/db_impl/db_impl.cc b/db/db_impl/db_impl.cc
index 261593423..06573ac2e 100644
--- a/db/db_impl/db_impl.cc
+++ b/db/db_impl/db_impl.cc
@@ -659,8 +659,9 @@ Status DBImpl::CloseHelper() {
// We need to release them before the block cache is destroyed. The block
// cache may be destroyed inside versions_.reset(), when column family data
// list is destroyed, so leaving handles in table cache after
- // versions_.reset() may cause issues.
- // Here we clean all unreferenced handles in table cache.
+ // versions_.reset() may cause issues. Here we clean all unreferenced handles
+ // in table cache, and (for certain builds/conditions) assert that no obsolete
+ // files are hanging around unreferenced (leak) in the table/blob file cache.
// Now we assume all user queries have finished, so only version set itself
// can possibly hold the blocks from block cache. After releasing unreferenced
// handles here, only handles held by version set left and inside
@@ -668,6 +669,9 @@ Status DBImpl::CloseHelper() {
// time a handle is released, we erase it from the cache too. By doing that,
// we can guarantee that after versions_.reset(), table cache is empty
// so the cache can be safely destroyed.
+#ifndef NDEBUG
+ TEST_VerifyNoObsoleteFilesCached(/*db_mutex_already_held=*/true);
+#endif // !NDEBUG
table_cache_->EraseUnRefEntries();
for (auto& txn_entry : recovered_transactions_) {
@@ -3227,6 +3231,8 @@ Status DBImpl::MultiGetImpl(
s = Status::Aborted();
break;
}
+ // This could be a long-running operation
+ ROCKSDB_THREAD_YIELD_HOOK();
}
// Post processing (decrement reference counts and record statistics)
diff --git a/db/db_impl/db_impl.h b/db/db_impl/db_impl.h
index 5e4fa310b..ccc0abfa7 100644
--- a/db/db_impl/db_impl.h
+++ b/db/db_impl/db_impl.h
@@ -1241,9 +1241,14 @@ class DBImpl : public DB {
static Status TEST_ValidateOptions(const DBOptions& db_options) {
return ValidateOptions(db_options);
}
-
#endif // NDEBUG
+ // In certain configurations, verify that the table/blob file cache only
+ // contains entries for live files, to check for effective leaks of open
+ // files. This can only be called when purging of obsolete files has
+ // "settled," such as during parts of DB Close().
+ void TEST_VerifyNoObsoleteFilesCached(bool db_mutex_already_held) const;
+
// persist stats to column family "_persistent_stats"
void PersistStats();
diff --git a/db/db_impl/db_impl_debug.cc b/db/db_impl/db_impl_debug.cc
index 790a50d7a..67f5b4aaf 100644
--- a/db/db_impl/db_impl_debug.cc
+++ b/db/db_impl/db_impl_debug.cc
@@ -9,6 +9,7 @@
#ifndef NDEBUG
+#include "db/blob/blob_file_cache.h"
#include "db/column_family.h"
#include "db/db_impl/db_impl.h"
#include "db/error_handler.h"
@@ -328,5 +329,49 @@ size_t DBImpl::TEST_EstimateInMemoryStatsHistorySize() const {
InstrumentedMutexLock l(&const_cast<DBImpl*>(this)->stats_history_mutex_);
return EstimateInMemoryStatsHistorySize();
}
+
+void DBImpl::TEST_VerifyNoObsoleteFilesCached(
+ bool db_mutex_already_held) const {
+ // This check is somewhat expensive and obscure to make a part of every
+ // unit test in every build variety. Thus, we only enable it for ASAN builds.
+ if (!kMustFreeHeapAllocations) {
+ return;
+ }
+
+ std::optional<InstrumentedMutexLock> l;
+ if (db_mutex_already_held) {
+ mutex_.AssertHeld();
+ } else {
+ l.emplace(&mutex_);
+ }
+
+ std::vector<uint64_t> live_files;
+ for (auto cfd : *versions_->GetColumnFamilySet()) {
+ if (cfd->IsDropped()) {
+ continue;
+ }
+ // Sneakily add both SST and blob files to the same list
+ cfd->current()->AddLiveFiles(&live_files, &live_files);
+ }
+ std::sort(live_files.begin(), live_files.end());
+
+ auto fn = [&live_files](const Slice& key, Cache::ObjectPtr, size_t,
+ const Cache::CacheItemHelper* helper) {
+ if (helper != BlobFileCache::GetHelper()) {
+ // Skip non-blob files for now
+ // FIXME: diagnose and fix the leaks of obsolete SST files revealed in
+ // unit tests.
+ return;
+ }
+ // See TableCache and BlobFileCache
+ assert(key.size() == sizeof(uint64_t));
+ uint64_t file_number;
+ GetUnaligned(reinterpret_cast<const uint64_t*>(key.data()), &file_number);
+ // Assert file is in sorted live_files
+ assert(
+ std::binary_search(live_files.begin(), live_files.end(), file_number));
+ };
+ table_cache_->ApplyToAllEntries(fn, {});
+}
} // namespace ROCKSDB_NAMESPACE
#endif // NDEBUG
diff --git a/db/db_iter.cc b/db/db_iter.cc
index e02586377..bf4749eb9 100644
--- a/db/db_iter.cc
+++ b/db/db_iter.cc
@@ -540,6 +540,8 @@ bool DBIter::FindNextUserEntryInternal(bool skipping_saved_key,
} else {
iter_.Next();
}
+ // This could be a long-running operation due to tombstones, etc.
+ ROCKSDB_THREAD_YIELD_HOOK();
} while (iter_.Valid());
valid_ = false;
diff --git a/db/table_cache.cc b/db/table_cache.cc
index 71fc29c32..8a5be75e8 100644
--- a/db/table_cache.cc
+++ b/db/table_cache.cc
@@ -164,6 +164,7 @@ Status TableCache::GetTableReader(
}
Cache::Handle* TableCache::Lookup(Cache* cache, uint64_t file_number) {
+ // NOTE: sharing same Cache with BlobFileCache
Slice key = GetSliceForFileNumber(&file_number);
return cache->Lookup(key);
}
@@ -179,6 +180,7 @@ Status TableCache::FindTable(
size_t max_file_size_for_l0_meta_pin, Temperature file_temperature) {
PERF_TIMER_GUARD_WITH_CLOCK(find_table_nanos, ioptions_.clock);
uint64_t number = file_meta.fd.GetNumber();
+ // NOTE: sharing same Cache with BlobFileCache
Slice key = GetSliceForFileNumber(&number);
*handle = cache_.Lookup(key);
TEST_SYNC_POINT_CALLBACK("TableCache::FindTable:0",
diff --git a/db/version_builder.cc b/db/version_builder.cc
index ed8ab8214..c98f53f42 100644
--- a/db/version_builder.cc
+++ b/db/version_builder.cc
@@ -24,6 +24,7 @@
#include <vector>
#include "cache/cache_reservation_manager.h"
+#include "db/blob/blob_file_cache.h"
#include "db/blob/blob_file_meta.h"
#include "db/dbformat.h"
#include "db/internal_stats.h"
@@ -744,12 +745,9 @@ class VersionBuilder::Rep {
return Status::Corruption("VersionBuilder", oss.str());
}
- // Note: we use C++11 for now but in C++14, this could be done in a more
- // elegant way using generalized lambda capture.
- VersionSet* const vs = version_set_;
- const ImmutableCFOptions* const ioptions = ioptions_;
-
- auto deleter = [vs, ioptions](SharedBlobFileMetaData* shared_meta) {
+ auto deleter = [vs = version_set_, ioptions = ioptions_,
+ bc = cfd_ ? cfd_->blob_file_cache()
+ : nullptr](SharedBlobFileMetaData* shared_meta) {
if (vs) {
assert(ioptions);
assert(!ioptions->cf_paths.empty());
@@ -758,6 +756,9 @@ class VersionBuilder::Rep {
vs->AddObsoleteBlobFile(shared_meta->GetBlobFileNumber(),
ioptions->cf_paths.front().path);
}
+ if (bc) {
+ bc->Evict(shared_meta->GetBlobFileNumber());
+ }
delete shared_meta;
};
@@ -766,7 +767,7 @@ class VersionBuilder::Rep {
blob_file_number, blob_file_addition.GetTotalBlobCount(),
blob_file_addition.GetTotalBlobBytes(),
blob_file_addition.GetChecksumMethod(),
- blob_file_addition.GetChecksumValue(), deleter);
+ blob_file_addition.GetChecksumValue(), std::move(deleter));
mutable_blob_file_metas_.emplace(
blob_file_number, MutableBlobFileMetaData(std::move(shared_meta)));
diff --git a/db/version_set.h b/db/version_set.h
index 9336782b1..024f869e7 100644
--- a/db/version_set.h
+++ b/db/version_set.h
@@ -1514,7 +1514,6 @@ class VersionSet {
void GetLiveFilesMetaData(std::vector<LiveFileMetaData>* metadata);
void AddObsoleteBlobFile(uint64_t blob_file_number, std::string path) {
- // TODO: Erase file from BlobFileCache?
obsolete_blob_files_.emplace_back(blob_file_number, std::move(path));
}
diff --git a/include/rocksdb/version.h b/include/rocksdb/version.h
index 2a19796b8..0afa2cab1 100644
--- a/include/rocksdb/version.h
+++ b/include/rocksdb/version.h
@@ -13,7 +13,7 @@
// minor or major version number planned for release.
#define ROCKSDB_MAJOR 9
#define ROCKSDB_MINOR 7
-#define ROCKSDB_PATCH 3
+#define ROCKSDB_PATCH 4
// Do not use these. We made the mistake of declaring macros starting with
// double underscore. Now we have to live with our choice. We'll deprecate these
diff --git a/port/port.h b/port/port.h
index 13aa56d47..141716e5b 100644
--- a/port/port.h
+++ b/port/port.h
@@ -19,3 +19,19 @@
#elif defined(OS_WIN)
#include "port/win/port_win.h"
#endif
+
+#ifdef OS_LINUX
+// A temporary hook into long-running RocksDB threads to support modifying their
+// priority etc. This should become a public API hook once the requirements
+// are better understood.
+extern "C" void RocksDbThreadYield() __attribute__((__weak__));
+#define ROCKSDB_THREAD_YIELD_HOOK() \
+ { \
+ if (RocksDbThreadYield) { \
+ RocksDbThreadYield(); \
+ } \
+ }
+#else
+#define ROCKSDB_THREAD_YIELD_HOOK() \
+ {}
+#endif

View File

@@ -0,0 +1,30 @@
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 93b884d..b715cb6 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -106,14 +106,9 @@ endif()
include(CMakeDependentOption)
if(MSVC)
- option(WITH_GFLAGS "build with GFlags" OFF)
option(WITH_XPRESS "build with windows built in compression" OFF)
- option(ROCKSDB_SKIP_THIRDPARTY "skip thirdparty.inc" OFF)
-
- if(NOT ROCKSDB_SKIP_THIRDPARTY)
- include(${CMAKE_CURRENT_SOURCE_DIR}/thirdparty.inc)
- endif()
-else()
+endif()
+if(TRUE)
if(CMAKE_SYSTEM_NAME MATCHES "FreeBSD" AND NOT CMAKE_SYSTEM_NAME MATCHES "kFreeBSD")
# FreeBSD has jemalloc as default malloc
# but it does not have all the jemalloc files in include/...
@@ -126,7 +121,7 @@ else()
endif()
endif()
- if(MINGW)
+ if(MSVC OR MINGW)
option(WITH_GFLAGS "build with GFlags" OFF)
else()
option(WITH_GFLAGS "build with GFlags" ON)

View File

@@ -27,7 +27,6 @@
#include <algorithm>
#include <optional>
#include <ostream>
#include <string>
#include <unordered_map>
#include <vector>
@@ -368,7 +367,7 @@ get(Section const& section,
}
inline std::string
get(Section const& section, std::string const& name, const char* defaultValue)
get(Section const& section, std::string const& name, char const* defaultValue)
{
try
{

View File

@@ -22,10 +22,9 @@
#include <xrpl/basics/Slice.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <cstdint>
#include <cstring>
#include <memory>
#include <utility>
namespace ripple {

View File

@@ -21,9 +21,11 @@
#define RIPPLED_COMPRESSIONALGORITHMS_H_INCLUDED
#include <xrpl/basics/contract.h>
#include <lz4.h>
#include <algorithm>
#include <cstdint>
#include <lz4.h>
#include <stdexcept>
#include <vector>
@@ -53,7 +55,7 @@ lz4Compress(void const* in, std::size_t inSize, BufferFactory&& bf)
auto compressed = bf(outCapacity);
auto compressedSize = LZ4_compress_default(
reinterpret_cast<const char*>(in),
reinterpret_cast<char const*>(in),
reinterpret_cast<char*>(compressed),
inSize,
outCapacity);
@@ -87,7 +89,7 @@ lz4Decompress(
Throw<std::runtime_error>("lz4Decompress: integer overflow (output)");
if (LZ4_decompress_safe(
reinterpret_cast<const char*>(in),
reinterpret_cast<char const*>(in),
reinterpret_cast<char*>(decompressed),
inSize,
decompressedSize) != decompressedSize)

View File

@@ -21,6 +21,7 @@
#define RIPPLE_BASICS_COUNTEDOBJECT_H_INCLUDED
#include <xrpl/beast/type_name.h>
#include <atomic>
#include <string>
#include <utility>

View File

@@ -22,11 +22,19 @@
#include <xrpl/basics/contract.h>
#if defined(__clang__)
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wdeprecated"
#pragma clang diagnostic ignored "-Wdeprecated-declarations"
#endif
#include <boost/outcome.hpp>
#include <concepts>
#if defined(__clang__)
#pragma clang diagnostic pop
#endif
#include <stdexcept>
#include <type_traits>
namespace ripple {
@@ -95,7 +103,7 @@ public:
{
}
constexpr const E&
constexpr E const&
value() const&
{
return val_;
@@ -113,7 +121,7 @@ public:
return std::move(val_);
}
constexpr const E&&
constexpr E const&&
value() const&&
{
return std::move(val_);

View File

@@ -0,0 +1,515 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2023 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_BASICS_INTRUSIVEPOINTER_H_INCLUDED
#define RIPPLE_BASICS_INTRUSIVEPOINTER_H_INCLUDED
#include <concepts>
#include <cstdint>
#include <type_traits>
#include <utility>
namespace ripple {
//------------------------------------------------------------------------------
/** Tag to create an intrusive pointer from another intrusive pointer by using a
static cast. This is useful to create an intrusive pointer to a derived
class from an intrusive pointer to a base class.
*/
struct StaticCastTagSharedIntrusive
{
};
/** Tag to create an intrusive pointer from another intrusive pointer by using a
dynamic cast. This is useful to create an intrusive pointer to a derived
class from an intrusive pointer to a base class. If the cast fails an empty
(null) intrusive pointer is created.
*/
struct DynamicCastTagSharedIntrusive
{
};
/** When creating or adopting a raw pointer, controls whether the strong count
is incremented or not. Use this tag to increment the strong count.
*/
struct SharedIntrusiveAdoptIncrementStrongTag
{
};
/** When creating or adopting a raw pointer, controls whether the strong count
is incremented or not. Use this tag to leave the strong count unchanged.
*/
struct SharedIntrusiveAdoptNoIncrementTag
{
};
//------------------------------------------------------------------------------
//
template <class T>
concept CAdoptTag = std::is_same_v<T, SharedIntrusiveAdoptIncrementStrongTag> ||
std::is_same_v<T, SharedIntrusiveAdoptNoIncrementTag>;
//------------------------------------------------------------------------------
/** A shared intrusive pointer class that supports weak pointers.
This is meant to be used for SHAMapInnerNodes, but may be useful for other
cases. Since the reference counts are stored on the pointee, the pointee is
not destroyed until both the strong _and_ weak pointer counts go to zero.
When the strong pointer count goes to zero, the "partialDestructor" is
called. This can be used to destroy as much of the object as possible while
still retaining the reference counts. For example, for SHAMapInnerNodes the
children may be reset in that function. Note that std::shared_poiner WILL
run the destructor when the strong count reaches zero, but may not free the
memory used by the object until the weak count reaches zero. In rippled, we
typically allocate shared pointers with the `make_shared` function. When
that is used, the memory is not reclaimed until the weak count reaches zero.
*/
template <class T>
class SharedIntrusive
{
public:
SharedIntrusive() = default;
template <CAdoptTag TAdoptTag>
SharedIntrusive(T* p, TAdoptTag) noexcept;
SharedIntrusive(SharedIntrusive const& rhs);
template <class TT>
// TODO: convertible_to isn't quite right. That include a static castable.
// Find the right concept.
requires std::convertible_to<TT*, T*>
SharedIntrusive(SharedIntrusive<TT> const& rhs);
SharedIntrusive(SharedIntrusive&& rhs);
template <class TT>
requires std::convertible_to<TT*, T*>
SharedIntrusive(SharedIntrusive<TT>&& rhs);
SharedIntrusive&
operator=(SharedIntrusive const& rhs);
bool
operator!=(std::nullptr_t) const;
bool
operator==(std::nullptr_t) const;
template <class TT>
requires std::convertible_to<TT*, T*>
SharedIntrusive&
operator=(SharedIntrusive<TT> const& rhs);
SharedIntrusive&
operator=(SharedIntrusive&& rhs);
template <class TT>
requires std::convertible_to<TT*, T*>
SharedIntrusive&
operator=(SharedIntrusive<TT>&& rhs);
/** Adopt the raw pointer. The strong reference may or may not be
incremented, depending on the TAdoptTag
*/
template <CAdoptTag TAdoptTag = SharedIntrusiveAdoptIncrementStrongTag>
void
adopt(T* p);
~SharedIntrusive();
/** Create a new SharedIntrusive by statically casting the pointer
controlled by the rhs param.
*/
template <class TT>
SharedIntrusive(
StaticCastTagSharedIntrusive,
SharedIntrusive<TT> const& rhs);
/** Create a new SharedIntrusive by statically casting the pointer
controlled by the rhs param.
*/
template <class TT>
SharedIntrusive(StaticCastTagSharedIntrusive, SharedIntrusive<TT>&& rhs);
/** Create a new SharedIntrusive by dynamically casting the pointer
controlled by the rhs param.
*/
template <class TT>
SharedIntrusive(
DynamicCastTagSharedIntrusive,
SharedIntrusive<TT> const& rhs);
/** Create a new SharedIntrusive by dynamically casting the pointer
controlled by the rhs param.
*/
template <class TT>
SharedIntrusive(DynamicCastTagSharedIntrusive, SharedIntrusive<TT>&& rhs);
T&
operator*() const noexcept;
T*
operator->() const noexcept;
explicit
operator bool() const noexcept;
/** Set the pointer to null, decrement the strong count, and run the
appropriate release action.
*/
void
reset();
/** Get the raw pointer */
T*
get() const;
/** Return the strong count */
std::size_t
use_count() const;
template <class TT, class... Args>
friend SharedIntrusive<TT>
make_SharedIntrusive(Args&&... args);
template <class TT>
friend class SharedIntrusive;
template <class TT>
friend class SharedWeakUnion;
template <class TT>
friend class WeakIntrusive;
private:
/** Return the raw pointer held by this object. */
T*
unsafeGetRawPtr() const;
/** Exchange the current raw pointer held by this object with the given
pointer. Decrement the strong count of the raw pointer previously held
by this object and run the appropriate release action.
*/
void
unsafeReleaseAndStore(T* next);
/** Set the raw pointer directly. This is wrapped in a function so the class
can support both atomic and non-atomic pointers in a future patch.
*/
void
unsafeSetRawPtr(T* p);
/** Exchange the raw pointer directly.
This sets the raw pointer to the given value and returns the previous
value. This is wrapped in a function so the class can support both
atomic and non-atomic pointers in a future patch.
*/
T*
unsafeExchange(T* p);
/** pointer to the type with an intrusive count */
T* ptr_{nullptr};
};
//------------------------------------------------------------------------------
/** A weak intrusive pointer class for the SharedIntrusive pointer class.
Note that this weak pointer class asks differently from normal weak pointer
classes. When the strong pointer count goes to zero, the "partialDestructor"
is called. See the comment on SharedIntrusive for a fuller explanation.
*/
template <class T>
class WeakIntrusive
{
public:
WeakIntrusive() = default;
WeakIntrusive(WeakIntrusive const& rhs);
WeakIntrusive(WeakIntrusive&& rhs);
WeakIntrusive(SharedIntrusive<T> const& rhs);
// There is no move constructor from a strong intrusive ptr because
// moving would be move expensive than copying in this case (the strong
// ref would need to be decremented)
WeakIntrusive(SharedIntrusive<T> const&& rhs) = delete;
// Since there are no current use cases for copy assignment in
// WeakIntrusive, we delete this operator to simplify the implementation. If
// a need arises in the future, we can reintroduce it with proper
// consideration."
WeakIntrusive&
operator=(WeakIntrusive const&) = delete;
template <class TT>
requires std::convertible_to<TT*, T*>
WeakIntrusive&
operator=(SharedIntrusive<TT> const& rhs);
/** Adopt the raw pointer and increment the weak count. */
void
adopt(T* ptr);
~WeakIntrusive();
/** Get a strong pointer from the weak pointer, if possible. This will
only return a seated pointer if the strong count on the raw pointer
is non-zero before locking.
*/
SharedIntrusive<T>
lock() const;
/** Return true if the strong count is zero. */
bool
expired() const;
/** Set the pointer to null and decrement the weak count.
Note: This may run the destructor if the strong count is zero.
*/
void
reset();
private:
T* ptr_ = nullptr;
/** Decrement the weak count. This does _not_ set the raw pointer to
null.
Note: This may run the destructor if the strong count is zero.
*/
void
unsafeReleaseNoStore();
};
//------------------------------------------------------------------------------
/** A combination of a strong and a weak intrusive pointer stored in the
space of a single pointer.
This class is similar to a `std::variant<SharedIntrusive,WeakIntrusive>`
with some optimizations. In particular, it uses a low-order bit to
determine if the raw pointer represents a strong pointer or a weak
pointer. It can also be quickly switched between its strong pointer and
weak pointer representations. This class is useful for storing intrusive
pointers in tagged caches.
*/
template <class T>
class SharedWeakUnion
{
// Tagged pointer. Low bit determines if this is a strong or a weak
// pointer. The low bit must be masked to zero when converting back to a
// pointer. If the low bit is '1', this is a weak pointer.
static_assert(
alignof(T) >= 2,
"Bad alignment: Combo pointer requires low bit to be zero");
public:
SharedWeakUnion() = default;
SharedWeakUnion(SharedWeakUnion const& rhs);
template <class TT>
requires std::convertible_to<TT*, T*>
SharedWeakUnion(SharedIntrusive<TT> const& rhs);
SharedWeakUnion(SharedWeakUnion&& rhs);
template <class TT>
requires std::convertible_to<TT*, T*>
SharedWeakUnion(SharedIntrusive<TT>&& rhs);
SharedWeakUnion&
operator=(SharedWeakUnion const& rhs);
template <class TT>
requires std::convertible_to<TT*, T*>
SharedWeakUnion&
operator=(SharedIntrusive<TT> const& rhs);
template <class TT>
requires std::convertible_to<TT*, T*>
SharedWeakUnion&
operator=(SharedIntrusive<TT>&& rhs);
~SharedWeakUnion();
/** Return a strong pointer if this is already a strong pointer (i.e.
don't lock the weak pointer. Use the `lock` method if that's what's
needed)
*/
SharedIntrusive<T>
getStrong() const;
/** Return true if this is a strong pointer and the strong pointer is
seated.
*/
explicit
operator bool() const noexcept;
/** Set the pointer to null, decrement the appropriate ref count, and
run the appropriate release action.
*/
void
reset();
/** If this is a strong pointer, return the raw pointer. Otherwise
return null.
*/
T*
get() const;
/** If this is a strong pointer, return the strong count. Otherwise
* return 0
*/
std::size_t
use_count() const;
/** Return true if there is a non-zero strong count. */
bool
expired() const;
/** If this is a strong pointer, return the strong pointer. Otherwise
attempt to lock the weak pointer.
*/
SharedIntrusive<T>
lock() const;
/** Return true is this represents a strong pointer. */
bool
isStrong() const;
/** Return true is this represents a weak pointer. */
bool
isWeak() const;
/** If this is a weak pointer, attempt to convert it to a strong
pointer.
@return true if successfully converted to a strong pointer (or was
already a strong pointer). Otherwise false.
*/
bool
convertToStrong();
/** If this is a strong pointer, attempt to convert it to a weak
pointer.
@return false if the pointer is null. Otherwise return true.
*/
bool
convertToWeak();
private:
// Tagged pointer. Low bit determines if this is a strong or a weak
// pointer. The low bit must be masked to zero when converting back to a
// pointer. If the low bit is '1', this is a weak pointer.
std::uintptr_t tp_{0};
static constexpr std::uintptr_t tagMask = 1;
static constexpr std::uintptr_t ptrMask = ~tagMask;
private:
/** Return the raw pointer held by this object.
*/
T*
unsafeGetRawPtr() const;
enum class RefStrength { strong, weak };
/** Set the raw pointer and tag bit directly.
*/
void
unsafeSetRawPtr(T* p, RefStrength rs);
/** Set the raw pointer and tag bit to all zeros (strong null pointer).
*/
void unsafeSetRawPtr(std::nullptr_t);
/** Decrement the appropriate ref count, and run the appropriate release
action. Note: this does _not_ set the raw pointer to null.
*/
void
unsafeReleaseNoStore();
};
//------------------------------------------------------------------------------
/** Create a shared intrusive pointer.
Note: unlike std::shared_ptr, where there is an advantage of allocating
the pointer and control block together, there is no benefit for intrusive
pointers.
*/
template <class TT, class... Args>
SharedIntrusive<TT>
make_SharedIntrusive(Args&&... args)
{
auto p = new TT(std::forward<Args>(args)...);
static_assert(
noexcept(SharedIntrusive<TT>(
std::declval<TT*>(),
std::declval<SharedIntrusiveAdoptNoIncrementTag>())),
"SharedIntrusive constructor should not throw or this can leak "
"memory");
return SharedIntrusive<TT>(p, SharedIntrusiveAdoptNoIncrementTag{});
}
//------------------------------------------------------------------------------
namespace intr_ptr {
template <class T>
using SharedPtr = SharedIntrusive<T>;
template <class T>
using WeakPtr = WeakIntrusive<T>;
template <class T>
using SharedWeakUnionPtr = SharedWeakUnion<T>;
template <class T, class... A>
SharedPtr<T>
make_shared(A&&... args)
{
return make_SharedIntrusive<T>(std::forward<A>(args)...);
}
template <class T, class TT>
SharedPtr<T>
static_pointer_cast(TT const& v)
{
return SharedPtr<T>(StaticCastTagSharedIntrusive{}, v);
}
template <class T, class TT>
SharedPtr<T>
dynamic_pointer_cast(TT const& v)
{
return SharedPtr<T>(DynamicCastTagSharedIntrusive{}, v);
}
} // namespace intr_ptr
} // namespace ripple
#endif

View File

@@ -0,0 +1,740 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2023 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_BASICS_INTRUSIVEPOINTER_IPP_INCLUDED
#define RIPPLE_BASICS_INTRUSIVEPOINTER_IPP_INCLUDED
#include <xrpl/basics/IntrusivePointer.h>
#include <xrpl/basics/IntrusiveRefCounts.h>
#include <utility>
namespace ripple {
template <class T>
template <CAdoptTag TAdoptTag>
SharedIntrusive<T>::SharedIntrusive(T* p, TAdoptTag) noexcept : ptr_{p}
{
if constexpr (std::is_same_v<
TAdoptTag,
SharedIntrusiveAdoptIncrementStrongTag>)
{
if (p)
p->addStrongRef();
}
}
template <class T>
SharedIntrusive<T>::SharedIntrusive(SharedIntrusive const& rhs)
: ptr_{[&] {
auto p = rhs.unsafeGetRawPtr();
if (p)
p->addStrongRef();
return p;
}()}
{
}
template <class T>
template <class TT>
requires std::convertible_to<TT*, T*>
SharedIntrusive<T>::SharedIntrusive(SharedIntrusive<TT> const& rhs)
: ptr_{[&] {
auto p = rhs.unsafeGetRawPtr();
if (p)
p->addStrongRef();
return p;
}()}
{
}
template <class T>
SharedIntrusive<T>::SharedIntrusive(SharedIntrusive&& rhs)
: ptr_{rhs.unsafeExchange(nullptr)}
{
}
template <class T>
template <class TT>
requires std::convertible_to<TT*, T*>
SharedIntrusive<T>::SharedIntrusive(SharedIntrusive<TT>&& rhs)
: ptr_{rhs.unsafeExchange(nullptr)}
{
}
template <class T>
SharedIntrusive<T>&
SharedIntrusive<T>::operator=(SharedIntrusive const& rhs)
{
if (this == &rhs)
return *this;
auto p = rhs.unsafeGetRawPtr();
if (p)
p->addStrongRef();
unsafeReleaseAndStore(p);
return *this;
}
template <class T>
template <class TT>
// clang-format off
requires std::convertible_to<TT*, T*>
// clang-format on
SharedIntrusive<T>&
SharedIntrusive<T>::operator=(SharedIntrusive<TT> const& rhs)
{
if constexpr (std::is_same_v<T, TT>)
{
// This case should never be hit. The operator above will run instead.
// (The normal operator= is needed or it will be marked `deleted`)
if (this == &rhs)
return *this;
}
auto p = rhs.unsafeGetRawPtr();
if (p)
p->addStrongRef();
unsafeReleaseAndStore(p);
return *this;
}
template <class T>
SharedIntrusive<T>&
SharedIntrusive<T>::operator=(SharedIntrusive&& rhs)
{
if (this == &rhs)
return *this;
unsafeReleaseAndStore(rhs.unsafeExchange(nullptr));
return *this;
}
template <class T>
template <class TT>
// clang-format off
requires std::convertible_to<TT*, T*>
// clang-format on
SharedIntrusive<T>&
SharedIntrusive<T>::operator=(SharedIntrusive<TT>&& rhs)
{
static_assert(
!std::is_same_v<T, TT>,
"This overload should not be instantiated for T == TT");
unsafeReleaseAndStore(rhs.unsafeExchange(nullptr));
return *this;
}
template <class T>
bool
SharedIntrusive<T>::operator!=(std::nullptr_t) const
{
return this->get() != nullptr;
}
template <class T>
bool
SharedIntrusive<T>::operator==(std::nullptr_t) const
{
return this->get() == nullptr;
}
template <class T>
template <CAdoptTag TAdoptTag>
void
SharedIntrusive<T>::adopt(T* p)
{
if constexpr (std::is_same_v<
TAdoptTag,
SharedIntrusiveAdoptIncrementStrongTag>)
{
if (p)
p->addStrongRef();
}
unsafeReleaseAndStore(p);
}
template <class T>
SharedIntrusive<T>::~SharedIntrusive()
{
unsafeReleaseAndStore(nullptr);
};
template <class T>
template <class TT>
SharedIntrusive<T>::SharedIntrusive(
StaticCastTagSharedIntrusive,
SharedIntrusive<TT> const& rhs)
: ptr_{[&] {
auto p = static_cast<T*>(rhs.unsafeGetRawPtr());
if (p)
p->addStrongRef();
return p;
}()}
{
}
template <class T>
template <class TT>
SharedIntrusive<T>::SharedIntrusive(
StaticCastTagSharedIntrusive,
SharedIntrusive<TT>&& rhs)
: ptr_{static_cast<T*>(rhs.unsafeExchange(nullptr))}
{
}
template <class T>
template <class TT>
SharedIntrusive<T>::SharedIntrusive(
DynamicCastTagSharedIntrusive,
SharedIntrusive<TT> const& rhs)
: ptr_{[&] {
auto p = dynamic_cast<T*>(rhs.unsafeGetRawPtr());
if (p)
p->addStrongRef();
return p;
}()}
{
}
template <class T>
template <class TT>
SharedIntrusive<T>::SharedIntrusive(
DynamicCastTagSharedIntrusive,
SharedIntrusive<TT>&& rhs)
{
// This can be simplified without the `exchange`, but the `exchange` is kept
// in anticipation of supporting atomic operations.
auto toSet = rhs.unsafeExchange(nullptr);
if (toSet)
{
ptr_ = dynamic_cast<T*>(toSet);
if (!ptr_)
// need to set the pointer back or will leak
rhs.unsafeExchange(toSet);
}
}
template <class T>
T&
SharedIntrusive<T>::operator*() const noexcept
{
return *unsafeGetRawPtr();
}
template <class T>
T*
SharedIntrusive<T>::operator->() const noexcept
{
return unsafeGetRawPtr();
}
template <class T>
SharedIntrusive<T>::operator bool() const noexcept
{
return bool(unsafeGetRawPtr());
}
template <class T>
void
SharedIntrusive<T>::reset()
{
unsafeReleaseAndStore(nullptr);
}
template <class T>
T*
SharedIntrusive<T>::get() const
{
return unsafeGetRawPtr();
}
template <class T>
std::size_t
SharedIntrusive<T>::use_count() const
{
if (auto p = unsafeGetRawPtr())
return p->use_count();
return 0;
}
template <class T>
T*
SharedIntrusive<T>::unsafeGetRawPtr() const
{
return ptr_;
}
template <class T>
void
SharedIntrusive<T>::unsafeSetRawPtr(T* p)
{
ptr_ = p;
}
template <class T>
T*
SharedIntrusive<T>::unsafeExchange(T* p)
{
return std::exchange(ptr_, p);
}
template <class T>
void
SharedIntrusive<T>::unsafeReleaseAndStore(T* next)
{
auto prev = unsafeExchange(next);
if (!prev)
return;
using enum ReleaseStrongRefAction;
auto action = prev->releaseStrongRef();
switch (action)
{
case noop:
break;
case destroy:
delete prev;
break;
case partialDestroy:
prev->partialDestructor();
partialDestructorFinished(&prev);
// prev is null and may no longer be used
break;
}
}
//------------------------------------------------------------------------------
template <class T>
WeakIntrusive<T>::WeakIntrusive(WeakIntrusive const& rhs) : ptr_{rhs.ptr_}
{
if (ptr_)
ptr_->addWeakRef();
}
template <class T>
WeakIntrusive<T>::WeakIntrusive(WeakIntrusive&& rhs) : ptr_{rhs.ptr_}
{
rhs.ptr_ = nullptr;
}
template <class T>
WeakIntrusive<T>::WeakIntrusive(SharedIntrusive<T> const& rhs)
: ptr_{rhs.unsafeGetRawPtr()}
{
if (ptr_)
ptr_->addWeakRef();
}
template <class T>
template <class TT>
// clang-format off
requires std::convertible_to<TT*, T*>
// clang-format on
WeakIntrusive<T>&
WeakIntrusive<T>::operator=(SharedIntrusive<TT> const& rhs)
{
unsafeReleaseNoStore();
auto p = rhs.unsafeGetRawPtr();
if (p)
p->addWeakRef();
return *this;
}
template <class T>
void
WeakIntrusive<T>::adopt(T* ptr)
{
unsafeReleaseNoStore();
if (ptr)
ptr->addWeakRef();
ptr_ = ptr;
}
template <class T>
WeakIntrusive<T>::~WeakIntrusive()
{
unsafeReleaseNoStore();
}
template <class T>
SharedIntrusive<T>
WeakIntrusive<T>::lock() const
{
if (ptr_ && ptr_->checkoutStrongRefFromWeak())
{
return SharedIntrusive<T>{ptr_, SharedIntrusiveAdoptNoIncrementTag{}};
}
return {};
}
template <class T>
bool
WeakIntrusive<T>::expired() const
{
return (!ptr_ || ptr_->expired());
}
template <class T>
void
WeakIntrusive<T>::reset()
{
unsafeReleaseNoStore();
ptr_ = nullptr;
}
template <class T>
void
WeakIntrusive<T>::unsafeReleaseNoStore()
{
if (!ptr_)
return;
using enum ReleaseWeakRefAction;
auto action = ptr_->releaseWeakRef();
switch (action)
{
case noop:
break;
case destroy:
delete ptr_;
break;
}
}
//------------------------------------------------------------------------------
template <class T>
SharedWeakUnion<T>::SharedWeakUnion(SharedWeakUnion const& rhs) : tp_{rhs.tp_}
{
auto p = rhs.unsafeGetRawPtr();
if (!p)
return;
if (rhs.isStrong())
p->addStrongRef();
else
p->addWeakRef();
}
template <class T>
template <class TT>
requires std::convertible_to<TT*, T*>
SharedWeakUnion<T>::SharedWeakUnion(SharedIntrusive<TT> const& rhs)
{
auto p = rhs.unsafeGetRawPtr();
if (p)
p->addStrongRef();
unsafeSetRawPtr(p, RefStrength::strong);
}
template <class T>
SharedWeakUnion<T>::SharedWeakUnion(SharedWeakUnion&& rhs) : tp_{rhs.tp_}
{
rhs.unsafeSetRawPtr(nullptr);
}
template <class T>
template <class TT>
requires std::convertible_to<TT*, T*>
SharedWeakUnion<T>::SharedWeakUnion(SharedIntrusive<TT>&& rhs)
{
auto p = rhs.unsafeGetRawPtr();
if (p)
unsafeSetRawPtr(p, RefStrength::strong);
rhs.unsafeSetRawPtr(nullptr);
}
template <class T>
SharedWeakUnion<T>&
SharedWeakUnion<T>::operator=(SharedWeakUnion const& rhs)
{
if (this == &rhs)
return *this;
unsafeReleaseNoStore();
if (auto p = rhs.unsafeGetRawPtr())
{
if (rhs.isStrong())
{
p->addStrongRef();
unsafeSetRawPtr(p, RefStrength::strong);
}
else
{
p->addWeakRef();
unsafeSetRawPtr(p, RefStrength::weak);
}
}
else
{
unsafeSetRawPtr(nullptr);
}
return *this;
}
template <class T>
template <class TT>
// clang-format off
requires std::convertible_to<TT*, T*>
// clang-format on
SharedWeakUnion<T>&
SharedWeakUnion<T>::operator=(SharedIntrusive<TT> const& rhs)
{
unsafeReleaseNoStore();
auto p = rhs.unsafeGetRawPtr();
if (p)
p->addStrongRef();
unsafeSetRawPtr(p, RefStrength::strong);
return *this;
}
template <class T>
template <class TT>
// clang-format off
requires std::convertible_to<TT*, T*>
// clang-format on
SharedWeakUnion<T>&
SharedWeakUnion<T>::operator=(SharedIntrusive<TT>&& rhs)
{
unsafeReleaseNoStore();
unsafeSetRawPtr(rhs.unsafeGetRawPtr(), RefStrength::strong);
rhs.unsafeSetRawPtr(nullptr);
return *this;
}
template <class T>
SharedWeakUnion<T>::~SharedWeakUnion()
{
unsafeReleaseNoStore();
};
// Return a strong pointer if this is already a strong pointer (i.e. don't
// lock the weak pointer. Use the `lock` method if that's what's needed)
template <class T>
SharedIntrusive<T>
SharedWeakUnion<T>::getStrong() const
{
SharedIntrusive<T> result;
auto p = unsafeGetRawPtr();
if (p && isStrong())
{
result.template adopt<SharedIntrusiveAdoptIncrementStrongTag>(p);
}
return result;
}
template <class T>
SharedWeakUnion<T>::operator bool() const noexcept
{
return bool(get());
}
template <class T>
void
SharedWeakUnion<T>::reset()
{
unsafeReleaseNoStore();
unsafeSetRawPtr(nullptr);
}
template <class T>
T*
SharedWeakUnion<T>::get() const
{
return isStrong() ? unsafeGetRawPtr() : nullptr;
}
template <class T>
std::size_t
SharedWeakUnion<T>::use_count() const
{
if (auto p = get())
return p->use_count();
return 0;
}
template <class T>
bool
SharedWeakUnion<T>::expired() const
{
auto p = unsafeGetRawPtr();
return (!p || p->expired());
}
template <class T>
SharedIntrusive<T>
SharedWeakUnion<T>::lock() const
{
SharedIntrusive<T> result;
auto p = unsafeGetRawPtr();
if (!p)
return result;
if (isStrong())
{
result.template adopt<SharedIntrusiveAdoptIncrementStrongTag>(p);
return result;
}
if (p->checkoutStrongRefFromWeak())
{
result.template adopt<SharedIntrusiveAdoptNoIncrementTag>(p);
return result;
}
return result;
}
template <class T>
bool
SharedWeakUnion<T>::isStrong() const
{
return !(tp_ & tagMask);
}
template <class T>
bool
SharedWeakUnion<T>::isWeak() const
{
return tp_ & tagMask;
}
template <class T>
bool
SharedWeakUnion<T>::convertToStrong()
{
if (isStrong())
return true;
auto p = unsafeGetRawPtr();
if (p && p->checkoutStrongRefFromWeak())
{
[[maybe_unused]] auto action = p->releaseWeakRef();
XRPL_ASSERT(
(action == ReleaseWeakRefAction::noop),
"ripple::SharedWeakUnion::convertToStrong : "
"action is noop");
unsafeSetRawPtr(p, RefStrength::strong);
return true;
}
return false;
}
template <class T>
bool
SharedWeakUnion<T>::convertToWeak()
{
if (isWeak())
return true;
auto p = unsafeGetRawPtr();
if (!p)
return false;
using enum ReleaseStrongRefAction;
auto action = p->addWeakReleaseStrongRef();
switch (action)
{
case noop:
break;
case destroy:
// We just added a weak ref. How could we destroy?
UNREACHABLE(
"ripple::SharedWeakUnion::convertToWeak : destroying freshly "
"added ref");
delete p;
unsafeSetRawPtr(nullptr);
return true; // Should never happen
case partialDestroy:
// This is a weird case. We just converted the last strong
// pointer to a weak pointer.
p->partialDestructor();
partialDestructorFinished(&p);
// p is null and may no longer be used
break;
}
unsafeSetRawPtr(p, RefStrength::weak);
return true;
}
template <class T>
T*
SharedWeakUnion<T>::unsafeGetRawPtr() const
{
return reinterpret_cast<T*>(tp_ & ptrMask);
}
template <class T>
void
SharedWeakUnion<T>::unsafeSetRawPtr(T* p, RefStrength rs)
{
tp_ = reinterpret_cast<std::uintptr_t>(p);
if (tp_ && rs == RefStrength::weak)
tp_ |= tagMask;
}
template <class T>
void
SharedWeakUnion<T>::unsafeSetRawPtr(std::nullptr_t)
{
tp_ = 0;
}
template <class T>
void
SharedWeakUnion<T>::unsafeReleaseNoStore()
{
auto p = unsafeGetRawPtr();
if (!p)
return;
if (isStrong())
{
using enum ReleaseStrongRefAction;
auto strongAction = p->releaseStrongRef();
switch (strongAction)
{
case noop:
break;
case destroy:
delete p;
break;
case partialDestroy:
p->partialDestructor();
partialDestructorFinished(&p);
// p is null and may no longer be used
break;
}
}
else
{
using enum ReleaseWeakRefAction;
auto weakAction = p->releaseWeakRef();
switch (weakAction)
{
case noop:
break;
case destroy:
delete p;
break;
}
}
}
} // namespace ripple
#endif

View File

@@ -0,0 +1,502 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2023 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_BASICS_INTRUSIVEREFCOUNTS_H_INCLUDED
#define RIPPLE_BASICS_INTRUSIVEREFCOUNTS_H_INCLUDED
#include <xrpl/beast/utility/instrumentation.h>
#include <atomic>
#include <cstdint>
namespace ripple {
/** Action to perform when releasing a strong pointer.
noop: Do nothing. For example, a `noop` action will occur when a count is
decremented to a non-zero value.
partialDestroy: Run the `partialDestructor`. This action will happen when a
strong count is decremented to zero and the weak count is non-zero.
destroy: Run the destructor. This action will occur when either the strong
count or weak count is decremented and the other count is also zero.
*/
enum class ReleaseStrongRefAction { noop, partialDestroy, destroy };
/** Action to perform when releasing a weak pointer.
noop: Do nothing. For example, a `noop` action will occur when a count is
decremented to a non-zero value.
destroy: Run the destructor. This action will occur when either the strong
count or weak count is decremented and the other count is also zero.
*/
enum class ReleaseWeakRefAction { noop, destroy };
/** Implement the strong count, weak count, and bit flags for an intrusive
pointer.
A class can satisfy the requirements of a ripple::IntrusivePointer by
inheriting from this class.
*/
struct IntrusiveRefCounts
{
virtual ~IntrusiveRefCounts() noexcept;
// This must be `noexcept` or the make_SharedIntrusive function could leak
// memory.
void
addStrongRef() const noexcept;
void
addWeakRef() const noexcept;
ReleaseStrongRefAction
releaseStrongRef() const;
// Same as:
// {
// addWeakRef();
// return releaseStrongRef;
// }
// done as one atomic operation
ReleaseStrongRefAction
addWeakReleaseStrongRef() const;
ReleaseWeakRefAction
releaseWeakRef() const;
// Returns true is able to checkout a strong ref. False otherwise
bool
checkoutStrongRefFromWeak() const noexcept;
bool
expired() const noexcept;
std::size_t
use_count() const noexcept;
// This function MUST be called after a partial destructor finishes running.
// Calling this function may cause other threads to delete the object
// pointed to by `o`, so `o` should never be used after calling this
// function. The parameter will be set to a `nullptr` after calling this
// function to emphasize that it should not be used.
// Note: This is intentionally NOT called at the end of `partialDestructor`.
// The reason for this is if new classes are written to support this smart
// pointer class, they need to write their own `partialDestructor` function
// and ensure `partialDestructorFinished` is called at the end. Putting this
// call inside the smart pointer class itself is expected to be less error
// prone.
// Note: The "two-star" programming is intentional. It emphasizes that `o`
// may be deleted and the unergonomic API is meant to signal the special
// nature of this function call to callers.
// Note: This is a template to support incompletely defined classes.
template <class T>
friend void
partialDestructorFinished(T** o);
private:
// TODO: We may need to use a uint64_t for both counts. This will reduce the
// memory savings. We need to audit the code to make sure 16 bit counts are
// enough for strong pointers and 14 bit counts are enough for weak
// pointers. Use type aliases to make it easy to switch types.
using CountType = std::uint16_t;
static constexpr size_t StrongCountNumBits = sizeof(CountType) * 8;
static constexpr size_t WeakCountNumBits = StrongCountNumBits - 2;
using FieldType = std::uint32_t;
static constexpr size_t FieldTypeBits = sizeof(FieldType) * 8;
static constexpr FieldType one = 1;
/** `refCounts` consists of four fields that are treated atomically:
1. Strong count. This is a count of the number of shared pointers that
hold a reference to this object. When the strong counts goes to zero,
if the weak count is zero, the destructor is run. If the weak count is
non-zero when the strong count goes to zero then the partialDestructor
is run.
2. Weak count. This is a count of the number of weak pointer that hold
a reference to this object. When the weak count goes to zero and the
strong count is also zero, then the destructor is run.
3. Partial destroy started bit. This bit is set if the
`partialDestructor` function has been started (or is about to be
started). This is used to prevent the destructor from running
concurrently with the partial destructor. This can easily happen when
the last strong pointer release its reference in one thread and starts
the partialDestructor, while in another thread the last weak pointer
goes out of scope and starts the destructor while the partialDestructor
is still running. Both a start and finished bit is needed to handle a
corner-case where the last strong pointer goes out of scope, then then
last `weakPointer` goes out of scope, but this happens before the
`partialDestructor` bit is set. It would be possible to use a single
bit if it could also be set atomically when the strong count goes to
zero and the weak count is non-zero, but that would add complexity (and
likely slow down common cases as well).
4. Partial destroy finished bit. This bit is set when the
`partialDestructor` has finished running. See (3) above for more
information.
*/
mutable std::atomic<FieldType> refCounts{strongDelta};
/** Amount to change the strong count when adding or releasing a reference
Note: The strong count is stored in the low `StrongCountNumBits` bits
of refCounts
*/
static constexpr FieldType strongDelta = 1;
/** Amount to change the weak count when adding or releasing a reference
Note: The weak count is stored in the high `WeakCountNumBits` bits of
refCounts
*/
static constexpr FieldType weakDelta = (one << StrongCountNumBits);
/** Flag that is set when the partialDestroy function has started running
(or is about to start running).
See description of the `refCounts` field for a fuller description of
this field.
*/
static constexpr FieldType partialDestroyStartedMask =
(one << (FieldTypeBits - 1));
/** Flag that is set when the partialDestroy function has finished running
See description of the `refCounts` field for a fuller description of
this field.
*/
static constexpr FieldType partialDestroyFinishedMask =
(one << (FieldTypeBits - 2));
/** Mask that will zero out all the `count` bits and leave the tag bits
unchanged.
*/
static constexpr FieldType tagMask =
partialDestroyStartedMask | partialDestroyFinishedMask;
/** Mask that will zero out the `tag` bits and leave the count bits
unchanged.
*/
static constexpr FieldType valueMask = ~tagMask;
/** Mask that will zero out everything except the strong count.
*/
static constexpr FieldType strongMask =
((one << StrongCountNumBits) - 1) & valueMask;
/** Mask that will zero out everything except the weak count.
*/
static constexpr FieldType weakMask =
(((one << WeakCountNumBits) - 1) << StrongCountNumBits) & valueMask;
/** Unpack the count and tag fields from the packed atomic integer form. */
struct RefCountPair
{
CountType strong;
CountType weak;
/** The `partialDestroyStartedBit` is set to on when the partial
destroy function is started. It is not a boolean; it is a uint32
with all bits zero with the possible exception of the
`partialDestroyStartedMask` bit. This is done so it can be directly
masked into the `combinedValue`.
*/
FieldType partialDestroyStartedBit{0};
/** The `partialDestroyFinishedBit` is set to on when the partial
destroy function has finished.
*/
FieldType partialDestroyFinishedBit{0};
RefCountPair(FieldType v) noexcept;
RefCountPair(CountType s, CountType w) noexcept;
/** Convert back to the packed integer form. */
FieldType
combinedValue() const noexcept;
static constexpr CountType maxStrongValue =
static_cast<CountType>((one << StrongCountNumBits) - 1);
static constexpr CountType maxWeakValue =
static_cast<CountType>((one << WeakCountNumBits) - 1);
/** Put an extra margin to detect when running up against limits.
This is only used in debug code, and is useful if we reduce the
number of bits in the strong and weak counts (to 16 and 14 bits).
*/
static constexpr CountType checkStrongMaxValue = maxStrongValue - 32;
static constexpr CountType checkWeakMaxValue = maxWeakValue - 32;
};
};
inline void
IntrusiveRefCounts::addStrongRef() const noexcept
{
refCounts.fetch_add(strongDelta, std::memory_order_acq_rel);
}
inline void
IntrusiveRefCounts::addWeakRef() const noexcept
{
refCounts.fetch_add(weakDelta, std::memory_order_acq_rel);
}
inline ReleaseStrongRefAction
IntrusiveRefCounts::releaseStrongRef() const
{
// Subtract `strongDelta` from refCounts. If this releases the last strong
// ref, set the `partialDestroyStarted` bit. It is important that the ref
// count and the `partialDestroyStartedBit` are changed atomically (hence
// the loop and `compare_exchange` op). If this didn't need to be done
// atomically, the loop could be replaced with a `fetch_sub` and a
// conditional `fetch_or`. This loop will almost always run once.
using enum ReleaseStrongRefAction;
auto prevIntVal = refCounts.load(std::memory_order_acquire);
while (1)
{
RefCountPair const prevVal{prevIntVal};
XRPL_ASSERT(
(prevVal.strong >= strongDelta),
"ripple::IntrusiveRefCounts::releaseStrongRef : previous ref "
"higher than new");
auto nextIntVal = prevIntVal - strongDelta;
ReleaseStrongRefAction action = noop;
if (prevVal.strong == 1)
{
if (prevVal.weak == 0)
{
action = destroy;
}
else
{
nextIntVal |= partialDestroyStartedMask;
action = partialDestroy;
}
}
if (refCounts.compare_exchange_weak(
prevIntVal, nextIntVal, std::memory_order_acq_rel))
{
// Can't be in partial destroy because only decrementing the strong
// count to zero can start a partial destroy, and that can't happen
// twice.
XRPL_ASSERT(
(action == noop) || !(prevIntVal & partialDestroyStartedMask),
"ripple::IntrusiveRefCounts::releaseStrongRef : not in partial "
"destroy");
return action;
}
}
}
inline ReleaseStrongRefAction
IntrusiveRefCounts::addWeakReleaseStrongRef() const
{
using enum ReleaseStrongRefAction;
static_assert(weakDelta > strongDelta);
auto constexpr delta = weakDelta - strongDelta;
auto prevIntVal = refCounts.load(std::memory_order_acquire);
// This loop will almost always run once. The loop is needed to atomically
// change the counts and flags (the count could be atomically changed, but
// the flags depend on the current value of the counts).
//
// Note: If this becomes a perf bottleneck, the `partialDestoryStartedMask`
// may be able to be set non-atomically. But it is easier to reason about
// the code if the flag is set atomically.
while (1)
{
RefCountPair const prevVal{prevIntVal};
// Converted the last strong pointer to a weak pointer.
//
// Can't be in partial destroy because only decrementing the
// strong count to zero can start a partial destroy, and that
// can't happen twice.
XRPL_ASSERT(
(!prevVal.partialDestroyStartedBit),
"ripple::IntrusiveRefCounts::addWeakReleaseStrongRef : not in "
"partial destroy");
auto nextIntVal = prevIntVal + delta;
ReleaseStrongRefAction action = noop;
if (prevVal.strong == 1)
{
if (prevVal.weak == 0)
{
action = noop;
}
else
{
nextIntVal |= partialDestroyStartedMask;
action = partialDestroy;
}
}
if (refCounts.compare_exchange_weak(
prevIntVal, nextIntVal, std::memory_order_acq_rel))
{
XRPL_ASSERT(
(!(prevIntVal & partialDestroyStartedMask)),
"ripple::IntrusiveRefCounts::addWeakReleaseStrongRef : not "
"started partial destroy");
return action;
}
}
}
inline ReleaseWeakRefAction
IntrusiveRefCounts::releaseWeakRef() const
{
auto prevIntVal = refCounts.fetch_sub(weakDelta, std::memory_order_acq_rel);
RefCountPair prev = prevIntVal;
if (prev.weak == 1 && prev.strong == 0)
{
if (!prev.partialDestroyStartedBit)
{
// This case should only be hit if the partialDestroyStartedBit is
// set non-atomically (and even then very rarely). The code is kept
// in case we need to set the flag non-atomically for perf reasons.
refCounts.wait(prevIntVal, std::memory_order_acquire);
prevIntVal = refCounts.load(std::memory_order_acquire);
prev = RefCountPair{prevIntVal};
}
if (!prev.partialDestroyFinishedBit)
{
// partial destroy MUST finish before running a full destroy (when
// using weak pointers)
refCounts.wait(prevIntVal - weakDelta, std::memory_order_acquire);
}
return ReleaseWeakRefAction::destroy;
}
return ReleaseWeakRefAction::noop;
}
inline bool
IntrusiveRefCounts::checkoutStrongRefFromWeak() const noexcept
{
auto curValue = RefCountPair{1, 1}.combinedValue();
auto desiredValue = RefCountPair{2, 1}.combinedValue();
while (!refCounts.compare_exchange_weak(
curValue, desiredValue, std::memory_order_acq_rel))
{
RefCountPair const prev{curValue};
if (!prev.strong)
return false;
desiredValue = curValue + strongDelta;
}
return true;
}
inline bool
IntrusiveRefCounts::expired() const noexcept
{
RefCountPair const val = refCounts.load(std::memory_order_acquire);
return val.strong == 0;
}
inline std::size_t
IntrusiveRefCounts::use_count() const noexcept
{
RefCountPair const val = refCounts.load(std::memory_order_acquire);
return val.strong;
}
inline IntrusiveRefCounts::~IntrusiveRefCounts() noexcept
{
#ifndef NDEBUG
auto v = refCounts.load(std::memory_order_acquire);
XRPL_ASSERT(
(!(v & valueMask)),
"ripple::IntrusiveRefCounts::~IntrusiveRefCounts : count must be zero");
auto t = v & tagMask;
XRPL_ASSERT(
(!t || t == tagMask),
"ripple::IntrusiveRefCounts::~IntrusiveRefCounts : valid tag");
#endif
}
//------------------------------------------------------------------------------
inline IntrusiveRefCounts::RefCountPair::RefCountPair(
IntrusiveRefCounts::FieldType v) noexcept
: strong{static_cast<CountType>(v & strongMask)}
, weak{static_cast<CountType>((v & weakMask) >> StrongCountNumBits)}
, partialDestroyStartedBit{v & partialDestroyStartedMask}
, partialDestroyFinishedBit{v & partialDestroyFinishedMask}
{
XRPL_ASSERT(
(strong < checkStrongMaxValue && weak < checkWeakMaxValue),
"ripple::IntrusiveRefCounts::RefCountPair(FieldType) : inputs inside "
"range");
}
inline IntrusiveRefCounts::RefCountPair::RefCountPair(
IntrusiveRefCounts::CountType s,
IntrusiveRefCounts::CountType w) noexcept
: strong{s}, weak{w}
{
XRPL_ASSERT(
(strong < checkStrongMaxValue && weak < checkWeakMaxValue),
"ripple::IntrusiveRefCounts::RefCountPair(CountType, CountType) : "
"inputs inside range");
}
inline IntrusiveRefCounts::FieldType
IntrusiveRefCounts::RefCountPair::combinedValue() const noexcept
{
XRPL_ASSERT(
(strong < checkStrongMaxValue && weak < checkWeakMaxValue),
"ripple::IntrusiveRefCounts::RefCountPair::combinedValue : inputs "
"inside range");
return (static_cast<IntrusiveRefCounts::FieldType>(weak)
<< IntrusiveRefCounts::StrongCountNumBits) |
static_cast<IntrusiveRefCounts::FieldType>(strong) |
partialDestroyStartedBit | partialDestroyFinishedBit;
}
template <class T>
inline void
partialDestructorFinished(T** o)
{
T& self = **o;
IntrusiveRefCounts::RefCountPair p =
self.refCounts.fetch_or(IntrusiveRefCounts::partialDestroyFinishedMask);
XRPL_ASSERT(
(!p.partialDestroyFinishedBit && p.partialDestroyStartedBit &&
!p.strong),
"ripple::partialDestructorFinished : not a weak ref");
if (!p.weak)
{
// There was a weak count before the partial destructor ran (or we would
// have run the full destructor) and now there isn't a weak count. Some
// thread is waiting to run the destructor.
self.refCounts.notify_one();
}
// Set the pointer to null to emphasize that the object shouldn't be used
// after calling this function as it may be destroyed in another thread.
*o = nullptr;
}
//------------------------------------------------------------------------------
} // namespace ripple
#endif

View File

@@ -21,6 +21,7 @@
#define RIPPLE_BASICS_LOCALVALUE_H_INCLUDED
#include <boost/thread/tss.hpp>
#include <memory>
#include <unordered_map>

View File

@@ -22,8 +22,10 @@
#include <xrpl/basics/UnorderedContainers.h>
#include <xrpl/beast/utility/Journal.h>
#include <boost/beast/core/string.hpp>
#include <boost/filesystem.hpp>
#include <map>
#include <memory>
#include <mutex>

View File

@@ -20,11 +20,11 @@
#ifndef RIPPLE_BASICS_RESOLVER_H_INCLUDED
#define RIPPLE_BASICS_RESOLVER_H_INCLUDED
#include <xrpl/beast/net/IPEndpoint.h>
#include <functional>
#include <vector>
#include <xrpl/beast/net/IPEndpoint.h>
namespace ripple {
class Resolver

View File

@@ -22,6 +22,7 @@
#include <xrpl/basics/Resolver.h>
#include <xrpl/beast/utility/Journal.h>
#include <boost/asio/io_service.hpp>
namespace ripple {

View File

@@ -0,0 +1,135 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2023 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_BASICS_SHAREDWEAKCACHEPOINTER_H_INCLUDED
#define RIPPLE_BASICS_SHAREDWEAKCACHEPOINTER_H_INCLUDED
#include <memory>
#include <variant>
namespace ripple {
/** A combination of a std::shared_ptr and a std::weak_pointer.
This class is a wrapper to a `std::variant<std::shared_ptr,std::weak_ptr>`
This class is useful for storing intrusive pointers in tagged caches using less
memory than storing both pointers directly.
*/
template <class T>
class SharedWeakCachePointer
{
public:
SharedWeakCachePointer() = default;
SharedWeakCachePointer(SharedWeakCachePointer const& rhs);
template <class TT>
requires std::convertible_to<TT*, T*>
SharedWeakCachePointer(std::shared_ptr<TT> const& rhs);
SharedWeakCachePointer(SharedWeakCachePointer&& rhs);
template <class TT>
requires std::convertible_to<TT*, T*>
SharedWeakCachePointer(std::shared_ptr<TT>&& rhs);
SharedWeakCachePointer&
operator=(SharedWeakCachePointer const& rhs);
template <class TT>
requires std::convertible_to<TT*, T*>
SharedWeakCachePointer&
operator=(std::shared_ptr<TT> const& rhs);
template <class TT>
requires std::convertible_to<TT*, T*>
SharedWeakCachePointer&
operator=(std::shared_ptr<TT>&& rhs);
~SharedWeakCachePointer();
/** Return a strong pointer if this is already a strong pointer (i.e. don't
lock the weak pointer. Use the `lock` method if that's what's needed)
*/
std::shared_ptr<T> const&
getStrong() const;
/** Return true if this is a strong pointer and the strong pointer is
seated.
*/
explicit
operator bool() const noexcept;
/** Set the pointer to null, decrement the appropriate ref count, and run
the appropriate release action.
*/
void
reset();
/** If this is a strong pointer, return the raw pointer. Otherwise return
null.
*/
T*
get() const;
/** If this is a strong pointer, return the strong count. Otherwise return 0
*/
std::size_t
use_count() const;
/** Return true if there is a non-zero strong count. */
bool
expired() const;
/** If this is a strong pointer, return the strong pointer. Otherwise
attempt to lock the weak pointer.
*/
std::shared_ptr<T>
lock() const;
/** Return true is this represents a strong pointer. */
bool
isStrong() const;
/** Return true is this represents a weak pointer. */
bool
isWeak() const;
/** If this is a weak pointer, attempt to convert it to a strong pointer.
@return true if successfully converted to a strong pointer (or was
already a strong pointer). Otherwise false.
*/
bool
convertToStrong();
/** If this is a strong pointer, attempt to convert it to a weak pointer.
@return false if the pointer is null. Otherwise return true.
*/
bool
convertToWeak();
private:
std::variant<std::shared_ptr<T>, std::weak_ptr<T>> combo_;
};
} // namespace ripple
#endif

View File

@@ -0,0 +1,192 @@
//------------------------------------------------------------------------------
/*
This file is part of rippled: https://github.com/ripple/rippled
Copyright (c) 2023 Ripple Labs Inc.
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL , DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
//==============================================================================
#ifndef RIPPLE_BASICS_SHAREDWEAKCACHEPOINTER_IPP_INCLUDED
#define RIPPLE_BASICS_SHAREDWEAKCACHEPOINTER_IPP_INCLUDED
#include <xrpl/basics/SharedWeakCachePointer.h>
namespace ripple {
template <class T>
SharedWeakCachePointer<T>::SharedWeakCachePointer(
SharedWeakCachePointer const& rhs) = default;
template <class T>
template <class TT>
requires std::convertible_to<TT*, T*>
SharedWeakCachePointer<T>::SharedWeakCachePointer(
std::shared_ptr<TT> const& rhs)
: combo_{rhs}
{
}
template <class T>
SharedWeakCachePointer<T>::SharedWeakCachePointer(
SharedWeakCachePointer&& rhs) = default;
template <class T>
template <class TT>
requires std::convertible_to<TT*, T*>
SharedWeakCachePointer<T>::SharedWeakCachePointer(std::shared_ptr<TT>&& rhs)
: combo_{std::move(rhs)}
{
}
template <class T>
SharedWeakCachePointer<T>&
SharedWeakCachePointer<T>::operator=(SharedWeakCachePointer const& rhs) =
default;
template <class T>
template <class TT>
requires std::convertible_to<TT*, T*>
SharedWeakCachePointer<T>&
SharedWeakCachePointer<T>::operator=(std::shared_ptr<TT> const& rhs)
{
combo_ = rhs;
return *this;
}
template <class T>
template <class TT>
requires std::convertible_to<TT*, T*>
SharedWeakCachePointer<T>&
SharedWeakCachePointer<T>::operator=(std::shared_ptr<TT>&& rhs)
{
combo_ = std::move(rhs);
return *this;
}
template <class T>
SharedWeakCachePointer<T>::~SharedWeakCachePointer() = default;
// Return a strong pointer if this is already a strong pointer (i.e. don't
// lock the weak pointer. Use the `lock` method if that's what's needed)
template <class T>
std::shared_ptr<T> const&
SharedWeakCachePointer<T>::getStrong() const
{
static std::shared_ptr<T> const empty;
if (auto p = std::get_if<std::shared_ptr<T>>(&combo_))
return *p;
return empty;
}
template <class T>
SharedWeakCachePointer<T>::operator bool() const noexcept
{
return !!std::get_if<std::shared_ptr<T>>(&combo_);
}
template <class T>
void
SharedWeakCachePointer<T>::reset()
{
combo_ = std::shared_ptr<T>{};
}
template <class T>
T*
SharedWeakCachePointer<T>::get() const
{
return std::get_if<std::shared_ptr<T>>(&combo_).get();
}
template <class T>
std::size_t
SharedWeakCachePointer<T>::use_count() const
{
if (auto p = std::get_if<std::shared_ptr<T>>(&combo_))
return p->use_count();
return 0;
}
template <class T>
bool
SharedWeakCachePointer<T>::expired() const
{
if (auto p = std::get_if<std::weak_ptr<T>>(&combo_))
return p->expired();
return !std::get_if<std::shared_ptr<T>>(&combo_);
}
template <class T>
std::shared_ptr<T>
SharedWeakCachePointer<T>::lock() const
{
if (auto p = std::get_if<std::shared_ptr<T>>(&combo_))
return *p;
if (auto p = std::get_if<std::weak_ptr<T>>(&combo_))
return p->lock();
return {};
}
template <class T>
bool
SharedWeakCachePointer<T>::isStrong() const
{
if (auto p = std::get_if<std::shared_ptr<T>>(&combo_))
return !!p->get();
return false;
}
template <class T>
bool
SharedWeakCachePointer<T>::isWeak() const
{
return !isStrong();
}
template <class T>
bool
SharedWeakCachePointer<T>::convertToStrong()
{
if (isStrong())
return true;
if (auto p = std::get_if<std::weak_ptr<T>>(&combo_))
{
if (auto s = p->lock())
{
combo_ = std::move(s);
return true;
}
}
return false;
}
template <class T>
bool
SharedWeakCachePointer<T>::convertToWeak()
{
if (isWeak())
return true;
if (auto p = std::get_if<std::shared_ptr<T>>(&combo_))
{
combo_ = std::weak_ptr<T>(*p);
return true;
}
return false;
}
} // namespace ripple
#endif

View File

@@ -23,6 +23,7 @@
#include <xrpl/basics/contract.h>
#include <xrpl/basics/strHex.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <algorithm>
#include <array>
#include <cstdint>

View File

@@ -20,11 +20,14 @@
#ifndef RIPPLE_BASICS_TAGGEDCACHE_H_INCLUDED
#define RIPPLE_BASICS_TAGGEDCACHE_H_INCLUDED
#include <xrpl/basics/IntrusivePointer.h>
#include <xrpl/basics/Log.h>
#include <xrpl/basics/SharedWeakCachePointer.ipp>
#include <xrpl/basics/UnorderedContainers.h>
#include <xrpl/basics/hardened_hash.h>
#include <xrpl/beast/clock/abstract_clock.h>
#include <xrpl/beast/insight/Insight.h>
#include <atomic>
#include <functional>
#include <mutex>
@@ -50,6 +53,8 @@ template <
class Key,
class T,
bool IsKeyCache = false,
class SharedWeakUnionPointerType = SharedWeakCachePointer<T>,
class SharedPointerType = std::shared_ptr<T>,
class Hash = hardened_hash<>,
class KeyEqual = std::equal_to<Key>,
class Mutex = std::recursive_mutex>
@@ -60,6 +65,8 @@ public:
using key_type = Key;
using mapped_type = T;
using clock_type = beast::abstract_clock<std::chrono::steady_clock>;
using shared_weak_combo_pointer_type = SharedWeakUnionPointerType;
using shared_pointer_type = SharedPointerType;
public:
TaggedCache(
@@ -69,231 +76,48 @@ public:
clock_type& clock,
beast::Journal journal,
beast::insight::Collector::ptr const& collector =
beast::insight::NullCollector::New())
: m_journal(journal)
, m_clock(clock)
, m_stats(
name,
std::bind(&TaggedCache::collect_metrics, this),
collector)
, m_name(name)
, m_target_size(size)
, m_target_age(expiration)
, m_cache_count(0)
, m_hits(0)
, m_misses(0)
{
}
beast::insight::NullCollector::New());
public:
/** Return the clock associated with the cache. */
clock_type&
clock()
{
return m_clock;
}
clock();
/** Returns the number of items in the container. */
std::size_t
size() const
{
std::lock_guard lock(m_mutex);
return m_cache.size();
}
void
setTargetSize(int s)
{
std::lock_guard lock(m_mutex);
m_target_size = s;
if (s > 0)
{
for (auto& partition : m_cache.map())
{
partition.rehash(static_cast<std::size_t>(
(s + (s >> 2)) /
(partition.max_load_factor() * m_cache.partitions()) +
1));
}
}
JLOG(m_journal.debug()) << m_name << " target size set to " << s;
}
clock_type::duration
getTargetAge() const
{
std::lock_guard lock(m_mutex);
return m_target_age;
}
void
setTargetAge(clock_type::duration s)
{
std::lock_guard lock(m_mutex);
m_target_age = s;
JLOG(m_journal.debug())
<< m_name << " target age set to " << m_target_age.count();
}
size() const;
int
getCacheSize() const
{
std::lock_guard lock(m_mutex);
return m_cache_count;
}
getCacheSize() const;
int
getTrackSize() const
{
std::lock_guard lock(m_mutex);
return m_cache.size();
}
getTrackSize() const;
float
getHitRate()
{
std::lock_guard lock(m_mutex);
auto const total = static_cast<float>(m_hits + m_misses);
return m_hits * (100.0f / std::max(1.0f, total));
}
getHitRate();
void
clear()
{
std::lock_guard lock(m_mutex);
m_cache.clear();
m_cache_count = 0;
}
clear();
void
reset()
{
std::lock_guard lock(m_mutex);
m_cache.clear();
m_cache_count = 0;
m_hits = 0;
m_misses = 0;
}
reset();
/** Refresh the last access time on a key if present.
@return `true` If the key was found.
*/
template <class KeyComparable>
bool
touch_if_exists(KeyComparable const& key)
{
std::lock_guard lock(m_mutex);
auto const iter(m_cache.find(key));
if (iter == m_cache.end())
{
++m_stats.misses;
return false;
}
iter->second.touch(m_clock.now());
++m_stats.hits;
return true;
}
touch_if_exists(KeyComparable const& key);
using SweptPointersVector = std::pair<
std::vector<std::shared_ptr<mapped_type>>,
std::vector<std::weak_ptr<mapped_type>>>;
using SweptPointersVector = std::vector<SharedWeakUnionPointerType>;
void
sweep()
{
// Keep references to all the stuff we sweep
// For performance, each worker thread should exit before the swept data
// is destroyed but still within the main cache lock.
std::vector<SweptPointersVector> allStuffToSweep(m_cache.partitions());
clock_type::time_point const now(m_clock.now());
clock_type::time_point when_expire;
auto const start = std::chrono::steady_clock::now();
{
std::lock_guard lock(m_mutex);
if (m_target_size == 0 ||
(static_cast<int>(m_cache.size()) <= m_target_size))
{
when_expire = now - m_target_age;
}
else
{
when_expire =
now - m_target_age * m_target_size / m_cache.size();
clock_type::duration const minimumAge(std::chrono::seconds(1));
if (when_expire > (now - minimumAge))
when_expire = now - minimumAge;
JLOG(m_journal.trace())
<< m_name << " is growing fast " << m_cache.size() << " of "
<< m_target_size << " aging at "
<< (now - when_expire).count() << " of "
<< m_target_age.count();
}
std::vector<std::thread> workers;
workers.reserve(m_cache.partitions());
std::atomic<int> allRemovals = 0;
for (std::size_t p = 0; p < m_cache.partitions(); ++p)
{
workers.push_back(sweepHelper(
when_expire,
now,
m_cache.map()[p],
allStuffToSweep[p],
allRemovals,
lock));
}
for (std::thread& worker : workers)
worker.join();
m_cache_count -= allRemovals;
}
// At this point allStuffToSweep will go out of scope outside the lock
// and decrement the reference count on each strong pointer.
JLOG(m_journal.debug())
<< m_name << " TaggedCache sweep lock duration "
<< std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now() - start)
.count()
<< "ms";
}
sweep();
bool
del(const key_type& key, bool valid)
{
// Remove from cache, if !valid, remove from map too. Returns true if
// removed from cache
std::lock_guard lock(m_mutex);
auto cit = m_cache.find(key);
if (cit == m_cache.end())
return false;
Entry& entry = cit->second;
bool ret = false;
if (entry.isCached())
{
--m_cache_count;
entry.ptr.reset();
ret = true;
}
if (!valid || entry.isExpired())
m_cache.erase(cit);
return ret;
}
del(key_type const& key, bool valid);
public:
/** Replace aliased objects with originals.
Due to concurrency it is possible for two separate objects with
@@ -307,100 +131,23 @@ public:
@return `true` If the key already existed.
*/
public:
template <class R>
bool
canonicalize(
const key_type& key,
std::shared_ptr<T>& data,
std::function<bool(std::shared_ptr<T> const&)>&& replace)
{
// Return canonical value, store if needed, refresh in cache
// Return values: true=we had the data already
std::lock_guard lock(m_mutex);
auto cit = m_cache.find(key);
if (cit == m_cache.end())
{
m_cache.emplace(
std::piecewise_construct,
std::forward_as_tuple(key),
std::forward_as_tuple(m_clock.now(), data));
++m_cache_count;
return false;
}
Entry& entry = cit->second;
entry.touch(m_clock.now());
if (entry.isCached())
{
if (replace(entry.ptr))
{
entry.ptr = data;
entry.weak_ptr = data;
}
else
{
data = entry.ptr;
}
return true;
}
auto cachedData = entry.lock();
if (cachedData)
{
if (replace(entry.ptr))
{
entry.ptr = data;
entry.weak_ptr = data;
}
else
{
entry.ptr = cachedData;
data = cachedData;
}
++m_cache_count;
return true;
}
entry.ptr = data;
entry.weak_ptr = data;
++m_cache_count;
return false;
}
key_type const& key,
SharedPointerType& data,
R&& replaceCallback);
bool
canonicalize_replace_cache(
const key_type& key,
std::shared_ptr<T> const& data)
{
return canonicalize(
key,
const_cast<std::shared_ptr<T>&>(data),
[](std::shared_ptr<T> const&) { return true; });
}
key_type const& key,
SharedPointerType const& data);
bool
canonicalize_replace_client(const key_type& key, std::shared_ptr<T>& data)
{
return canonicalize(
key, data, [](std::shared_ptr<T> const&) { return false; });
}
canonicalize_replace_client(key_type const& key, SharedPointerType& data);
std::shared_ptr<T>
fetch(const key_type& key)
{
std::lock_guard<mutex_type> l(m_mutex);
auto ret = initialFetch(key, l);
if (!ret)
++m_misses;
return ret;
}
SharedPointerType
fetch(key_type const& key);
/** Insert the element into the container.
If the key already exists, nothing happens.
@@ -409,26 +156,11 @@ public:
template <class ReturnType = bool>
auto
insert(key_type const& key, T const& value)
-> std::enable_if_t<!IsKeyCache, ReturnType>
{
auto p = std::make_shared<T>(std::cref(value));
return canonicalize_replace_client(key, p);
}
-> std::enable_if_t<!IsKeyCache, ReturnType>;
template <class ReturnType = bool>
auto
insert(key_type const& key) -> std::enable_if_t<IsKeyCache, ReturnType>
{
std::lock_guard lock(m_mutex);
clock_type::time_point const now(m_clock.now());
auto [it, inserted] = m_cache.emplace(
std::piecewise_construct,
std::forward_as_tuple(key),
std::forward_as_tuple(now));
if (!inserted)
it->second.last_access = now;
return inserted;
}
insert(key_type const& key) -> std::enable_if_t<IsKeyCache, ReturnType>;
// VFALCO NOTE It looks like this returns a copy of the data in
// the output parameter 'data'. This could be expensive.
@@ -436,50 +168,18 @@ public:
// simply return an iterator.
//
bool
retrieve(const key_type& key, T& data)
{
// retrieve the value of the stored data
auto entry = fetch(key);
if (!entry)
return false;
data = *entry;
return true;
}
retrieve(key_type const& key, T& data);
mutex_type&
peekMutex()
{
return m_mutex;
}
peekMutex();
std::vector<key_type>
getKeys() const
{
std::vector<key_type> v;
{
std::lock_guard lock(m_mutex);
v.reserve(m_cache.size());
for (auto const& _ : m_cache)
v.push_back(_.first);
}
return v;
}
getKeys() const;
// CachedSLEs functions.
/** Returns the fraction of cache hits. */
double
rate() const
{
std::lock_guard lock(m_mutex);
auto const tot = m_hits + m_misses;
if (tot == 0)
return 0;
return double(m_hits) / tot;
}
rate() const;
/** Fetch an item from the cache.
If the digest was not found, Handler
@@ -487,73 +187,16 @@ public:
std::shared_ptr<SLE const>(void)
*/
template <class Handler>
std::shared_ptr<T>
fetch(key_type const& digest, Handler const& h)
{
{
std::lock_guard l(m_mutex);
if (auto ret = initialFetch(digest, l))
return ret;
}
auto sle = h();
if (!sle)
return {};
std::lock_guard l(m_mutex);
++m_misses;
auto const [it, inserted] =
m_cache.emplace(digest, Entry(m_clock.now(), std::move(sle)));
if (!inserted)
it->second.touch(m_clock.now());
return it->second.ptr;
}
SharedPointerType
fetch(key_type const& digest, Handler const& h);
// End CachedSLEs functions.
private:
std::shared_ptr<T>
initialFetch(key_type const& key, std::lock_guard<mutex_type> const& l)
{
auto cit = m_cache.find(key);
if (cit == m_cache.end())
return {};
Entry& entry = cit->second;
if (entry.isCached())
{
++m_hits;
entry.touch(m_clock.now());
return entry.ptr;
}
entry.ptr = entry.lock();
if (entry.isCached())
{
// independent of cache size, so not counted as a hit
++m_cache_count;
entry.touch(m_clock.now());
return entry.ptr;
}
m_cache.erase(cit);
return {};
}
SharedPointerType
initialFetch(key_type const& key, std::lock_guard<mutex_type> const& l);
void
collect_metrics()
{
m_stats.size.set(getCacheSize());
{
beast::insight::Gauge::value_type hit_rate(0);
{
std::lock_guard lock(m_mutex);
auto const total(m_hits + m_misses);
if (total != 0)
hit_rate = (m_hits * 100) / total;
}
m_stats.hit_rate.set(hit_rate);
}
}
collect_metrics();
private:
struct Stats
@@ -599,36 +242,37 @@ private:
class ValueEntry
{
public:
std::shared_ptr<mapped_type> ptr;
std::weak_ptr<mapped_type> weak_ptr;
shared_weak_combo_pointer_type ptr;
clock_type::time_point last_access;
ValueEntry(
clock_type::time_point const& last_access_,
std::shared_ptr<mapped_type> const& ptr_)
: ptr(ptr_), weak_ptr(ptr_), last_access(last_access_)
shared_pointer_type const& ptr_)
: ptr(ptr_), last_access(last_access_)
{
}
bool
isWeak() const
{
return ptr == nullptr;
if (!ptr)
return true;
return ptr.isWeak();
}
bool
isCached() const
{
return ptr != nullptr;
return ptr && ptr.isStrong();
}
bool
isExpired() const
{
return weak_ptr.expired();
return ptr.expired();
}
std::shared_ptr<mapped_type>
SharedPointerType
lock()
{
return weak_ptr.lock();
return ptr.lock();
}
void
touch(clock_type::time_point const& now)
@@ -657,72 +301,7 @@ private:
typename KeyValueCacheType::map_type& partition,
SweptPointersVector& stuffToSweep,
std::atomic<int>& allRemovals,
std::lock_guard<std::recursive_mutex> const&)
{
return std::thread([&, this]() {
int cacheRemovals = 0;
int mapRemovals = 0;
// Keep references to all the stuff we sweep
// so that we can destroy them outside the lock.
stuffToSweep.first.reserve(partition.size());
stuffToSweep.second.reserve(partition.size());
{
auto cit = partition.begin();
while (cit != partition.end())
{
if (cit->second.isWeak())
{
// weak
if (cit->second.isExpired())
{
stuffToSweep.second.push_back(
std::move(cit->second.weak_ptr));
++mapRemovals;
cit = partition.erase(cit);
}
else
{
++cit;
}
}
else if (cit->second.last_access <= when_expire)
{
// strong, expired
++cacheRemovals;
if (cit->second.ptr.use_count() == 1)
{
stuffToSweep.first.push_back(
std::move(cit->second.ptr));
++mapRemovals;
cit = partition.erase(cit);
}
else
{
// remains weakly cached
cit->second.ptr.reset();
++cit;
}
}
else
{
// strong, not expired
++cit;
}
}
}
if (mapRemovals || cacheRemovals)
{
JLOG(m_journal.debug())
<< "TaggedCache partition sweep " << m_name
<< ": cache = " << partition.size() << "-" << cacheRemovals
<< ", map-=" << mapRemovals;
}
allRemovals += cacheRemovals;
});
}
std::lock_guard<std::recursive_mutex> const&);
[[nodiscard]] std::thread
sweepHelper(
@@ -731,45 +310,7 @@ private:
typename KeyOnlyCacheType::map_type& partition,
SweptPointersVector&,
std::atomic<int>& allRemovals,
std::lock_guard<std::recursive_mutex> const&)
{
return std::thread([&, this]() {
int cacheRemovals = 0;
int mapRemovals = 0;
// Keep references to all the stuff we sweep
// so that we can destroy them outside the lock.
{
auto cit = partition.begin();
while (cit != partition.end())
{
if (cit->second.last_access > now)
{
cit->second.last_access = now;
++cit;
}
else if (cit->second.last_access <= when_expire)
{
cit = partition.erase(cit);
}
else
{
++cit;
}
}
}
if (mapRemovals || cacheRemovals)
{
JLOG(m_journal.debug())
<< "TaggedCache partition sweep " << m_name
<< ": cache = " << partition.size() << "-" << cacheRemovals
<< ", map-=" << mapRemovals;
}
allRemovals += cacheRemovals;
});
};
std::lock_guard<std::recursive_mutex> const&);
beast::Journal m_journal;
clock_type& m_clock;
@@ -781,10 +322,10 @@ private:
std::string m_name;
// Desired number of cache entries (0 = ignore)
int m_target_size;
int const m_target_size;
// Desired maximum cache age
clock_type::duration m_target_age;
clock_type::duration const m_target_age;
// Number of items cached
int m_cache_count;

File diff suppressed because it is too large Load Diff

View File

@@ -25,6 +25,7 @@
#include <xrpl/beast/hash/hash_append.h>
#include <xrpl/beast/hash/uhash.h>
#include <xrpl/beast/hash/xxhasher.h>
#include <unordered_map>
#include <unordered_set>

View File

@@ -33,12 +33,13 @@
#include <xrpl/basics/strHex.h>
#include <xrpl/beast/utility/Zero.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <boost/endian/conversion.hpp>
#include <boost/functional/hash.hpp>
#include <algorithm>
#include <array>
#include <cstring>
#include <functional>
#include <type_traits>
namespace ripple {
@@ -373,7 +374,7 @@ public:
}
base_uint&
operator^=(const base_uint& b)
operator^=(base_uint const& b)
{
for (int i = 0; i < WIDTH; i++)
data_[i] ^= b.data_[i];
@@ -382,7 +383,7 @@ public:
}
base_uint&
operator&=(const base_uint& b)
operator&=(base_uint const& b)
{
for (int i = 0; i < WIDTH; i++)
data_[i] &= b.data_[i];
@@ -391,7 +392,7 @@ public:
}
base_uint&
operator|=(const base_uint& b)
operator|=(base_uint const& b)
{
for (int i = 0; i < WIDTH; i++)
data_[i] |= b.data_[i];
@@ -414,11 +415,11 @@ public:
return *this;
}
const base_uint
base_uint const
operator++(int)
{
// postfix operator
const base_uint ret = *this;
base_uint const ret = *this;
++(*this);
return ret;
@@ -440,11 +441,11 @@ public:
return *this;
}
const base_uint
base_uint const
operator--(int)
{
// postfix operator
const base_uint ret = *this;
base_uint const ret = *this;
--(*this);
return ret;
@@ -465,7 +466,7 @@ public:
}
base_uint&
operator+=(const base_uint& b)
operator+=(base_uint const& b)
{
std::uint64_t carry = 0;
@@ -510,7 +511,7 @@ public:
}
[[nodiscard]] constexpr bool
parseHex(const char* str)
parseHex(char const* str)
{
return parseHex(std::string_view{str});
}

View File

@@ -20,17 +20,16 @@
#ifndef RIPPLE_BASICS_CHRONO_H_INCLUDED
#define RIPPLE_BASICS_CHRONO_H_INCLUDED
#include <date/date.h>
#include <xrpl/beast/clock/abstract_clock.h>
#include <xrpl/beast/clock/basic_seconds_clock.h>
#include <xrpl/beast/clock/manual_clock.h>
#include <date/date.h>
#include <chrono>
#include <cstdint>
#include <ratio>
#include <string>
#include <type_traits>
namespace ripple {

View File

@@ -43,7 +43,7 @@ struct less
using result_type = bool;
constexpr bool
operator()(const T& left, const T& right) const
operator()(T const& left, T const& right) const
{
return std::less<T>()(left, right);
}
@@ -55,7 +55,7 @@ struct equal_to
using result_type = bool;
constexpr bool
operator()(const T& left, const T& right) const
operator()(T const& left, T const& right) const
{
return std::equal_to<T>()(left, right);
}

View File

@@ -21,9 +21,9 @@
#define RIPPLE_BASICS_CONTRACT_H_INCLUDED
#include <xrpl/beast/type_name.h>
#include <exception>
#include <string>
#include <typeinfo>
#include <utility>
namespace ripple {

View File

@@ -21,6 +21,7 @@
#define RIPPLE_BASICS_MAKE_SSLCONTEXT_H_INCLUDED
#include <boost/asio/ssl/context.hpp>
#include <string>
namespace ripple {

View File

@@ -22,6 +22,7 @@
#include <xrpl/beast/hash/uhash.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <functional>
#include <optional>
#include <string>
@@ -51,7 +52,7 @@ template <
typename Value,
typename Hash,
typename Pred = std::equal_to<Key>,
typename Alloc = std::allocator<std::pair<const Key, Value>>>
typename Alloc = std::allocator<std::pair<Key const, Value>>>
class partitioned_unordered_map
{
std::size_t partitions_;

View File

@@ -22,9 +22,9 @@
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/beast/xor_shift_engine.h>
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <limits>
#include <mutex>
#include <random>

View File

@@ -19,6 +19,7 @@
#define RIPPLE_BASICS_SPINLOCK_H_INCLUDED
#include <xrpl/beast/utility/instrumentation.h>
#include <atomic>
#include <limits>
#include <type_traits>

View File

@@ -21,7 +21,9 @@
#define BEAST_UTILITY_TAGGED_INTEGER_H_INCLUDED
#include <xrpl/beast/hash/hash_append.h>
#include <boost/operators.hpp>
#include <functional>
#include <iostream>
#include <type_traits>
@@ -74,13 +76,13 @@ public:
}
bool
operator<(const tagged_integer& rhs) const noexcept
operator<(tagged_integer const& rhs) const noexcept
{
return m_value < rhs.m_value;
}
bool
operator==(const tagged_integer& rhs) const noexcept
operator==(tagged_integer const& rhs) const noexcept
{
return m_value == rhs.m_value;
}
@@ -142,14 +144,14 @@ public:
}
tagged_integer&
operator<<=(const tagged_integer& rhs) noexcept
operator<<=(tagged_integer const& rhs) noexcept
{
m_value <<= rhs.m_value;
return *this;
}
tagged_integer&
operator>>=(const tagged_integer& rhs) noexcept
operator>>=(tagged_integer const& rhs) noexcept
{
m_value >>= rhs.m_value;
return *this;

View File

@@ -21,6 +21,7 @@
#define BEAST_ASIO_IO_LATENCY_PROBE_H_INCLUDED
#include <xrpl/beast/utility/instrumentation.h>
#include <boost/asio/basic_waitable_timer.hpp>
#include <boost/asio/io_service.hpp>

View File

@@ -25,9 +25,11 @@
#include <xrpl/beast/container/detail/aged_associative_container.h>
#include <xrpl/beast/container/detail/aged_container_iterator.h>
#include <xrpl/beast/container/detail/empty_base_optimization.h>
#include <boost/intrusive/list.hpp>
#include <boost/intrusive/set.hpp>
#include <boost/version.hpp>
#include <algorithm>
#include <functional>
#include <initializer_list>

View File

@@ -25,8 +25,10 @@
#include <xrpl/beast/container/detail/aged_associative_container.h>
#include <xrpl/beast/container/detail/aged_container_iterator.h>
#include <xrpl/beast/container/detail/empty_base_optimization.h>
#include <boost/intrusive/list.hpp>
#include <boost/intrusive/unordered_set.hpp>
#include <algorithm>
#include <cmath>
#include <functional>

View File

@@ -11,6 +11,7 @@
#define BEAST_CONTAINER_DETAIL_EMPTY_BASE_OPTIMIZATION_H_INCLUDED
#include <boost/type_traits/is_final.hpp>
#include <type_traits>
#include <utility>

View File

@@ -23,6 +23,7 @@
#include <xrpl/beast/utility/instrumentation.h>
#include <boost/core/detail/string_view.hpp>
#include <algorithm>
#include <cerrno>
#include <charconv>

View File

@@ -21,7 +21,6 @@
#define BEAST_INTRUSIVE_LIST_H_INCLUDED
#include <iterator>
#include <type_traits>
namespace beast {

View File

@@ -23,13 +23,39 @@
#include <boost/container/flat_set.hpp>
#include <boost/endian/conversion.hpp>
/*
Workaround for overzealous clang warning, which trips on libstdc++ headers
In file included from
/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_algo.h:61:
/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_tempbuf.h:263:8:
error: 'get_temporary_buffer<std::pair<ripple::Quality, const
std::vector<std::unique_ptr<ripple::Step>> *>>' is deprecated
[-Werror,-Wdeprecated-declarations] 263 |
std::get_temporary_buffer<value_type>(_M_original_len));
^
*/
#if defined(__clang__)
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wdeprecated"
#pragma clang diagnostic ignored "-Wdeprecated-declarations"
#endif
#include <functional>
#include <memory>
#if defined(__clang__)
#pragma clang diagnostic pop
#endif
#include <array>
#include <chrono>
#include <cstdint>
#include <cstring>
#include <functional>
#include <map>
#include <memory>
#include <set>
#include <string>
#include <system_error>

View File

@@ -30,7 +30,7 @@ namespace beast {
template <class Hasher = xxhasher>
struct uhash
{
explicit uhash() = default;
uhash() = default;
using result_type = typename Hasher::result_type;

View File

@@ -21,6 +21,7 @@
#define BEAST_HASH_XXHASHER_H_INCLUDED
#include <boost/endian/conversion.hpp>
#include <xxhash.h>
#include <cstddef>

View File

@@ -20,10 +20,10 @@
#ifndef BEAST_INSIGHT_METER_H_INCLUDED
#define BEAST_INSIGHT_METER_H_INCLUDED
#include <memory>
#include <xrpl/beast/insight/MeterImpl.h>
#include <memory>
namespace beast {
namespace insight {

View File

@@ -21,7 +21,6 @@
#define BEAST_INSIGHT_STATSDCOLLECTOR_H_INCLUDED
#include <xrpl/beast/insight/Collector.h>
#include <xrpl/beast/net/IPEndpoint.h>
#include <xrpl/beast/utility/Journal.h>

View File

@@ -25,8 +25,10 @@
#include <xrpl/beast/net/IPAddressV4.h>
#include <xrpl/beast/net/IPAddressV6.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <boost/asio/ip/address.hpp>
#include <boost/functional/hash.hpp>
#include <cstdint>
#include <ios>
#include <sstream>

View File

@@ -22,8 +22,6 @@
#include <xrpl/beast/net/IPEndpoint.h>
#include <sstream>
#include <boost/asio.hpp>
namespace beast {

View File

@@ -21,7 +21,9 @@
#define BEAST_NET_IPADDRESSV4_H_INCLUDED
#include <xrpl/beast/hash/hash_append.h>
#include <boost/asio/ip/address_v4.hpp>
#include <cstdint>
#include <functional>
#include <ios>

View File

@@ -21,7 +21,9 @@
#define BEAST_NET_IPADDRESSV6_H_INCLUDED
#include <xrpl/beast/utility/instrumentation.h>
#include <boost/asio/ip/address_v6.hpp>
#include <cstdint>
#include <functional>
#include <ios>

View File

@@ -215,7 +215,7 @@ namespace std {
template <>
struct hash<::beast::IP::Endpoint>
{
explicit hash() = default;
hash() = default;
std::size_t
operator()(::beast::IP::Endpoint const& endpoint) const
@@ -230,7 +230,7 @@ namespace boost {
template <>
struct hash<::beast::IP::Endpoint>
{
explicit hash() = default;
hash() = default;
std::size_t
operator()(::beast::IP::Endpoint const& endpoint) const

View File

@@ -20,10 +20,6 @@
#ifndef BEAST_RFC2616_HPP
#define BEAST_RFC2616_HPP
// TODO: This include <cstdint> is a workaround for beast compilation bug.
// Remove when fix https://github.com/boostorg/beast/pull/2682/ is available.
#include <cstdint>
#include <boost/beast/http/message.hpp>
#include <boost/beast/http/rfc7230.hpp>
#include <boost/range/algorithm/equal.hpp>
@@ -32,6 +28,7 @@
#include <algorithm>
#include <cctype>
#include <cstdint>
#include <iterator>
#include <string>
#include <utility>

View File

@@ -11,6 +11,7 @@
#include <boost/asio/io_service.hpp>
#include <boost/asio/spawn.hpp>
#include <boost/optional.hpp>
#include <condition_variable>
#include <functional>
#include <mutex>

View File

@@ -9,6 +9,7 @@
#define BEAST_UNIT_TEST_MATCH_HPP
#include <xrpl/beast/unit_test/suite_info.h>
#include <string>
namespace beast {

View File

@@ -10,8 +10,10 @@
#include <xrpl/beast/unit_test/amount.h>
#include <xrpl/beast/unit_test/recorder.h>
#include <boost/lexical_cast.hpp>
#include <boost/optional.hpp>
#include <algorithm>
#include <chrono>
#include <functional>

View File

@@ -9,7 +9,9 @@
#define BEAST_UNIT_TEST_RUNNER_H_INCLUDED
#include <xrpl/beast/unit_test/suite_info.h>
#include <boost/assert.hpp>
#include <mutex>
#include <ostream>
#include <string>

View File

@@ -9,9 +9,11 @@
#define BEAST_UNIT_TEST_SUITE_HPP
#include <xrpl/beast/unit_test/runner.h>
#include <boost/filesystem.hpp>
#include <boost/lexical_cast.hpp>
#include <boost/throw_exception.hpp>
#include <ostream>
#include <sstream>
#include <string>

View File

@@ -10,7 +10,9 @@
#include <xrpl/beast/unit_test/detail/const_container.h>
#include <xrpl/beast/unit_test/suite_info.h>
#include <boost/assert.hpp>
#include <set>
#include <typeindex>
#include <unordered_set>

View File

@@ -9,6 +9,7 @@
#define BEAST_UNIT_TEST_THREAD_HPP
#include <xrpl/beast/unit_test/suite.h>
#include <functional>
#include <thread>
#include <utility>

View File

@@ -21,6 +21,7 @@
#define BEAST_UTILITY_JOURNAL_H_INCLUDED
#include <xrpl/beast/utility/instrumentation.h>
#include <sstream>
namespace beast {

View File

@@ -22,11 +22,9 @@
#include <xrpl/beast/core/List.h>
#include <cstdint>
#include <mutex>
#include <sstream>
#include <string>
#include <utility>
namespace beast {

View File

@@ -21,6 +21,7 @@
#define BEAST_RANDOM_RNGFILL_H_INCLUDED
#include <xrpl/beast/utility/instrumentation.h>
#include <array>
#include <cstdint>
#include <cstring>

View File

@@ -21,6 +21,7 @@
#define BEAST_UTILITY_TEMP_DIR_H_INCLUDED
#include <boost/filesystem.hpp>
#include <string>
namespace beast {
@@ -36,9 +37,9 @@ class temp_dir
public:
#if !GENERATING_DOCS
temp_dir(const temp_dir&) = delete;
temp_dir(temp_dir const&) = delete;
temp_dir&
operator=(const temp_dir&) = delete;
operator=(temp_dir const&) = delete;
#endif
/// Construct a temporary directory.

View File

@@ -21,8 +21,6 @@
#define RIPPLE_CRYPTO_RANDOM_H_INCLUDED
#include <mutex>
#include <string>
#include <type_traits>
namespace ripple {

View File

@@ -21,6 +21,7 @@
#define RIPPLE_JSON_OBJECT_H_INCLUDED
#include <xrpl/json/Writer.h>
#include <memory>
namespace Json {

View File

@@ -21,6 +21,7 @@
#define RIPPLE_JSON_OUTPUT_H_INCLUDED
#include <boost/beast/core/string.hpp>
#include <functional>
#include <string>

View File

@@ -24,6 +24,7 @@
#include <xrpl/basics/contract.h>
#include <xrpl/json/Output.h>
#include <xrpl/json/json_value.h>
#include <memory>
namespace Json {

View File

@@ -20,6 +20,7 @@
#ifndef RIPPLE_JSON_JSON_ASSERT_H_INCLUDED
#define RIPPLE_JSON_JSON_ASSERT_H_INCLUDED
#include <xrpl/basics/contract.h>
#include <xrpl/json/json_errors.h>
#define JSON_ASSERT_MESSAGE(condition, message) \

View File

@@ -24,7 +24,9 @@
#include <xrpl/json/json_forwards.h>
#include <xrpl/json/json_value.h>
#include <boost/asio/buffer.hpp>
#include <stack>
namespace Json {
@@ -37,7 +39,7 @@ class Reader
{
public:
using Char = char;
using Location = const Char*;
using Location = Char const*;
/** \brief Constructs a Reader allowing all features
* for parsing.
@@ -62,7 +64,7 @@ public:
* error occurred.
*/
bool
parse(const char* beginDoc, const char* endDoc, Value& root);
parse(char const* beginDoc, char const* endDoc, Value& root);
/// \brief Parse from input stream.
/// \see Json::operator>>(std::istream&, Json::Value&).
@@ -131,7 +133,7 @@ private:
using Errors = std::deque<ErrorInfo>;
bool
expectToken(TokenType type, Token& token, const char* message);
expectToken(TokenType type, Token& token, char const* message);
bool
readToken(Token& token);
void

View File

@@ -20,10 +20,13 @@
#ifndef RIPPLE_JSON_JSON_VALUE_H_INCLUDED
#define RIPPLE_JSON_JSON_VALUE_H_INCLUDED
#include <xrpl/basics/Number.h>
#include <xrpl/json/json_forwards.h>
#include <cstring>
#include <map>
#include <string>
#include <utility>
#include <vector>
/** \brief JSON (JavaScript Object Notation).
@@ -60,24 +63,24 @@ enum ValueType {
class StaticString
{
public:
constexpr explicit StaticString(const char* czstring) : str_(czstring)
constexpr explicit StaticString(char const* czstring) : str_(czstring)
{
}
constexpr
operator const char*() const
operator char const*() const
{
return str_;
}
constexpr const char*
constexpr char const*
c_str() const
{
return str_;
}
private:
const char* str_;
char const* str_;
};
inline bool
@@ -155,10 +158,10 @@ public:
using Int = Json::Int;
using ArrayIndex = UInt;
static const Value null;
static const Int minInt;
static const Int maxInt;
static const UInt maxUInt;
static Value const null;
static Int const minInt;
static Int const maxInt;
static UInt const maxUInt;
private:
class CZString
@@ -170,24 +173,24 @@ private:
duplicateOnCopy
};
CZString(int index);
CZString(const char* cstr, DuplicationPolicy allocate);
CZString(const CZString& other);
CZString(char const* cstr, DuplicationPolicy allocate);
CZString(CZString const& other);
~CZString();
CZString&
operator=(const CZString& other) = delete;
operator=(CZString const& other) = delete;
bool
operator<(const CZString& other) const;
operator<(CZString const& other) const;
bool
operator==(const CZString& other) const;
operator==(CZString const& other) const;
int
index() const;
const char*
char const*
c_str() const;
bool
isStaticString() const;
private:
const char* cstr_;
char const* cstr_;
int index_;
};
@@ -214,7 +217,8 @@ public:
Value(Int value);
Value(UInt value);
Value(double value);
Value(const char* value);
Value(char const* value);
Value(ripple::Number const& value);
/** \brief Constructs a value from a static string.
* Like other value string constructor but do not duplicate the string for
@@ -226,10 +230,10 @@ public:
* Json::Value aValue( StaticString("some text") );
* \endcode
*/
Value(const StaticString& value);
Value(StaticString const& value);
Value(std::string const& value);
Value(bool value);
Value(const Value& other);
Value(Value const& other);
~Value();
Value&
@@ -246,7 +250,7 @@ public:
ValueType
type() const;
const char*
char const*
asCString() const;
/** Returns the unquoted string value. */
std::string
@@ -316,12 +320,12 @@ public:
/// Access an array element (zero based index )
/// (You may need to say 'value[0u]' to get your compiler to distinguish
/// this from the operator[] which takes a string.)
const Value&
Value const&
operator[](UInt index) const;
/// If the array contains at least index+1 elements, returns the element
/// value, otherwise returns defaultValue.
Value
get(UInt index, const Value& defaultValue) const;
get(UInt index, Value const& defaultValue) const;
/// Return true if index < size().
bool
isValidIndex(UInt index) const;
@@ -329,25 +333,25 @@ public:
///
/// Equivalent to jsonvalue[jsonvalue.size()] = value;
Value&
append(const Value& value);
append(Value const& value);
Value&
append(Value&& value);
/// Access an object value by name, create a null member if it does not
/// exist.
Value&
operator[](const char* key);
operator[](char const* key);
/// Access an object value by name, returns null if there is no member with
/// that name.
const Value&
operator[](const char* key) const;
Value const&
operator[](char const* key) const;
/// Access an object value by name, create a null member if it does not
/// exist.
Value&
operator[](std::string const& key);
/// Access an object value by name, returns null if there is no member with
/// that name.
const Value&
Value const&
operator[](std::string const& key) const;
/** \brief Access an object value by name, create a null member if it does
not exist.
@@ -363,14 +367,16 @@ public:
* \endcode
*/
Value&
operator[](const StaticString& key);
operator[](StaticString const& key);
Value const&
operator[](StaticString const& key) const;
/// Return the member named key if it exist, defaultValue otherwise.
Value
get(const char* key, const Value& defaultValue) const;
get(char const* key, Value const& defaultValue) const;
/// Return the member named key if it exist, defaultValue otherwise.
Value
get(std::string const& key, const Value& defaultValue) const;
get(std::string const& key, Value const& defaultValue) const;
/// \brief Remove and return the named member.
///
@@ -379,14 +385,14 @@ public:
/// \pre type() is objectValue or nullValue
/// \post type() is unchanged
Value
removeMember(const char* key);
removeMember(char const* key);
/// Same as removeMember(const char*)
Value
removeMember(std::string const& key);
/// Return true if the object has a member named key.
bool
isMember(const char* key) const;
isMember(char const* key) const;
/// Return true if the object has a member named key.
bool
isMember(std::string const& key) const;
@@ -413,13 +419,13 @@ public:
end();
friend bool
operator==(const Value&, const Value&);
operator==(Value const&, Value const&);
friend bool
operator<(const Value&, const Value&);
operator<(Value const&, Value const&);
private:
Value&
resolveReference(const char* key, bool isStatic);
resolveReference(char const* key, bool isStatic);
private:
union ValueHolder
@@ -435,32 +441,38 @@ private:
int allocated_ : 1; // Notes: if declared as bool, bitfield is useless.
};
inline Value
to_json(ripple::Number const& number)
{
return to_string(number);
}
bool
operator==(const Value&, const Value&);
operator==(Value const&, Value const&);
inline bool
operator!=(const Value& x, const Value& y)
operator!=(Value const& x, Value const& y)
{
return !(x == y);
}
bool
operator<(const Value&, const Value&);
operator<(Value const&, Value const&);
inline bool
operator<=(const Value& x, const Value& y)
operator<=(Value const& x, Value const& y)
{
return !(y < x);
}
inline bool
operator>(const Value& x, const Value& y)
operator>(Value const& x, Value const& y)
{
return y < x;
}
inline bool
operator>=(const Value& x, const Value& y)
operator>=(Value const& x, Value const& y)
{
return !(x < y);
}
@@ -481,11 +493,11 @@ public:
virtual ~ValueAllocator() = default;
virtual char*
makeMemberName(const char* memberName) = 0;
makeMemberName(char const* memberName) = 0;
virtual void
releaseMemberName(char* memberName) = 0;
virtual char*
duplicateStringValue(const char* value, unsigned int length = unknown) = 0;
duplicateStringValue(char const* value, unsigned int length = unknown) = 0;
virtual void
releaseStringValue(char* value) = 0;
};
@@ -502,16 +514,16 @@ public:
ValueIteratorBase();
explicit ValueIteratorBase(const Value::ObjectValues::iterator& current);
explicit ValueIteratorBase(Value::ObjectValues::iterator const& current);
bool
operator==(const SelfType& other) const
operator==(SelfType const& other) const
{
return isEqual(other);
}
bool
operator!=(const SelfType& other) const
operator!=(SelfType const& other) const
{
return !isEqual(other);
}
@@ -527,7 +539,7 @@ public:
/// Return the member name of the referenced Value. "" if it is not an
/// objectValue.
const char*
char const*
memberName() const;
protected:
@@ -541,13 +553,13 @@ protected:
decrement();
difference_type
computeDistance(const SelfType& other) const;
computeDistance(SelfType const& other) const;
bool
isEqual(const SelfType& other) const;
isEqual(SelfType const& other) const;
void
copy(const SelfType& other);
copy(SelfType const& other);
private:
Value::ObjectValues::iterator current_;
@@ -565,8 +577,8 @@ class ValueConstIterator : public ValueIteratorBase
public:
using size_t = unsigned int;
using difference_type = int;
using reference = const Value&;
using pointer = const Value*;
using reference = Value const&;
using pointer = Value const*;
using SelfType = ValueConstIterator;
ValueConstIterator() = default;
@@ -574,11 +586,11 @@ public:
private:
/*! \internal Use by Value to create an iterator.
*/
explicit ValueConstIterator(const Value::ObjectValues::iterator& current);
explicit ValueConstIterator(Value::ObjectValues::iterator const& current);
public:
SelfType&
operator=(const ValueIteratorBase& other);
operator=(ValueIteratorBase const& other);
SelfType
operator++(int)
@@ -631,17 +643,17 @@ public:
using SelfType = ValueIterator;
ValueIterator() = default;
ValueIterator(const ValueConstIterator& other);
ValueIterator(const ValueIterator& other);
ValueIterator(ValueConstIterator const& other);
ValueIterator(ValueIterator const& other);
private:
/*! \internal Use by Value to create an iterator.
*/
explicit ValueIterator(const Value::ObjectValues::iterator& current);
explicit ValueIterator(Value::ObjectValues::iterator const& current);
public:
SelfType&
operator=(const SelfType& other);
operator=(SelfType const& other);
SelfType
operator++(int)

Some files were not shown because too many files have changed in this diff Show More