* XRPLF/develop: (99 commits)
refactor: Clean up tec object deletion logic (6588)
fix: Move AMMInvariant weakInvariantCheck logic into the transaction (7032)
fix: Improve ValidAMM invariant (7295)
feat: Remove clear mutable flags for DynamicMPT XLS-94 (7439)
ci: Build and push docker images in forks too (7588)
ci: [DEPENDABOT] bump actions/checkout from 6.0.3 to 7.0.0 (7585)
fix: Ensure xrpld service directories exist at startup (7565)
fix: Use template for granular delegation permissions (6613)
docs: Fix some comments to improve readability (7405)
ci: Disable assertions on Release builds (7443)
build: Add graphviz to Nix images (7566)
refactor: Rerevert "Explicitly trim the heap after cache sweeps (6022)"
release: Bump version to 3.3.0-b0
ci: Make clang-tidy workflow adjustments to stay in sync with Clio (7563)
build: Add git-lfs to Nix images (7561)
build: Add zip to Nix images (7551)
docs: Rewrite build environment docs (7533)
release: Bump version to 3.2.0
ci: [DEPENDABOT] bump codecov/codecov-action from 6.0.1 to 7.0.0 (7426)
release: Bump version to 3.2.0-rc6
...
* XRPLF/develop: (30 commits)
chore: Pin Python packages for codegen using uv (7329)
style: Use shfmt instead of bashate (7326)
fix: Fix edge-case where vault-depositor may get stuck (7139)
fix: Fix `VaultInvariant` and `VaultDeposit` precision bugs at IOU scale boundaries (7272)
ci: Add clang to nix images (7308)
fix: Include management-fee delta in doOverpayment assertion (7039)
fix: Fix clang-tidy pre-commit hook to locate compile_commands.json from repo root (7325)
fix: Use consistent scale for `debtTotal` (7093)
fix: Skip deleted book directories and non-root modifications in `ValidBookDirectory` invariant (7312)
fix: Address review feedback on FD/handle guarding (5823 follow-up) (7310)
fix: Fix non-canonical MPT amount (7117)
release: Bump version to 3.2.0-b7 (7316)
fix: Check if the MPT first loss cover can be sent to the broker before deleting the broker (7125)
fix: Fix RPM prerelease ordering and start xrpld on DEB install (7313)
ci: Re-enable full nproc for Linux (7315)
fix: Add assorted MPT/DEX fixes (7040)
refactor: Remove dead `fetchBatch` code (7309)
release: Bump version to 3.2.0-b6 (7311)
chore: Revert graceful peer disconnection and follow-up fix (7296)
fix: Fix IOU precision issues in LoanBrokerCover transactions (7274)
...
* XRPLF/develop:
refactor: Clean up comments post-clang-tidy changes (7283)
release: Set version to 3.3.0-b0 (7280)
refactor: Rename static constants (7120)
refactor: Use `isFlag` where possible instead of bitwise math (7278)
ci: Update XRPLF/actions (7281)
Improve job queue collision checks and logging
- Improve logging related to ledger acquisition and operating mode
changes
- Class "CanProcess" to keep track of processing of distinct items
- Drop duplicate outgoing TMGetLedger messages per peer
- Allow a retry after 30s in case of peer or network congestion.
- Addresses RIPD-1870
- (Changes levelization. That is not desirable, and will need to be fixed.)
- Drop duplicate incoming TMGetLedger messages per peer
- Allow a retry after 15s in case of peer or network congestion.
- The requestCookie is ignored when computing the hash, thus increasing
the chances of detecting duplicate messages.
- With duplicate messages, keep track of the different requestCookies
(or lack of cookie). When work is finally done for a given request,
send the response to all the peers that are waiting on the request,
sending one message per peer, including all the cookies and
a "directResponse" flag indicating the data is intended for the
sender, too.
- Addresses RIPD-1871
- Drop duplicate incoming TMLedgerData messages
- Addresses RIPD-1869
Subscribe tests have a problem that there is no way to synchronize application running in background threads and test threads. Threads are communicating via websocket messages. When the code is compiled in debug mode with code coverage enabled it executes quite slow, so receiving websocket messages by the client in subscribe tests may time out.
This change does 2 things to fix the problem:
* Increases timeout for receiving a websocket message.
* Decreases the number of tests running in parallel.
While testing the fix for subscribe test another flaky test in ledger replay was found, which has also been addressed.
This change renames all occurrences of `namespace ripple` and `ripple::` to `namespace xrpl` and `xrpl::`, respectively, as well as the names of test suites. It also provides a script to allow developers to replicate the changes in their local branch or fork to avoid conflicts.
This change renames all the `info()` functions to `header()`, since they return `LedgerHeader` structs. It also renames the underlying variables from `info_` to `header_`.
Per XLS-0095, we are taking steps to rename ripple(d) to xrpl(d).
This change specifically removes all copyright notices referencing Ripple, XRPLF, and certain affiliated contributors upon mutual agreement, so the notice in the LICENSE.md file applies throughout. Copyright notices referencing external contributions remain as-is. Duplicate verbiage is also removed.
To protect the identity of UNL validators, the IP addresses are redacted from the log messages sent to the common Grafana instance. However, without such identifying information it is challenging to debug issues. This change adds a node's public key to logs to improve our ability to debug issues.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
This updates Boost to 1.88, which is needed because Clio wants to move to 1.88 as that fixes several ASAN false positives around coroutine usage. In order for Clio to move to newer boost, libXRPL needs to move too. Hence the changes in this PR. A lot has changed between 1.83 and 1.88 so there are lots of changes in the diff, especially in regards to Boost.Asio and coroutines in particular.
- Drop duplicate outgoing TMGetLedger messages per peer
- Allow a retry after 30s in case of peer or network congestion.
- Addresses RIPD-1870
- (Changes levelization. That is not desirable, and will need to be fixed.)
- Drop duplicate incoming TMGetLedger messages per peer
- Allow a retry after 15s in case of peer or network congestion.
- The requestCookie is ignored when computing the hash, thus increasing
the chances of detecting duplicate messages.
- With duplicate messages, keep track of the different requestCookies
(or lack of cookie). When work is finally done for a given request,
send the response to all the peers that are waiting on the request,
sending one message per peer, including all the cookies and
a "directResponse" flag indicating the data is intended for the
sender, too.
- Addresses RIPD-1871
- Drop duplicate incoming TMLedgerData messages
- Addresses RIPD-1869
- Improve logging related to ledger acquisition
- Class "CanProcess" to keep track of processing of distinct items
---------
Co-authored-by: Valentin Balaschenko <13349202+vlntb@users.noreply.github.com>
- Fix an erroneous high fee penalty that peers could incur for sending
older transactions.
- Update to the fees charged for imposing a load on the server.
- Prevent the relaying of internal pseudo-transactions.
- Before: Pseudo-transactions received from a peer will fail the signature
check, even if they were requested (using TMGetObjectByHash), because
they have no signature. This causes the peer to be charge for an
invalid signature.
- After: Pseudo-transactions, are put into the global cache
(TransactionMaster) only. If the transaction is not part of
a TMTransactions batch, the peer is charged an unwanted data fee.
These fees will not be a problem in the normal course of operations,
but should dissuade peers from behaving badly by sending a bunch of
junk.
- Improve logging: include the reason for fees charged to a peer.
Co-authored-by: Ed Hennis <ed@ripple.com>
* upstream/master:
Set version to 2.2.2
Allow only 1 job queue slot for each validation ledger check
Allow only 1 job queue slot for acquiring inbound ledger.
Track latencies of certain code blocks, and log if they take too long
* Log when duplicate concurrent inbound ledger are filtered.
* RAII for containers that track concurrent inbound ledger.
* Comment on when to asynchronously acquire inbound ledgers, which
is possible to be always OK, but should have further review.
* Other small logging changes
Co-authored-by: Ed Hennis <ed@ripple.com>
* It is now an invariant that all constructed Public Keys are valid,
non-empty and contain 33 bytes of data.
* Additionally, the memory footprint of the PublicKey class is reduced.
The size_ data member is declared as static.
* Distinguish and identify the PublisherList retrieved from the local
config file, versus the ones obtained from other validators.
* Fixes#2942
* Add logging for Application.cpp sweep()
* Improve lifetime management of ledger objects (`SLE`s)
* Only store SLE digest in CachedView; get SLEs from CachedSLEs
* Also force release of last ledger used for path finding if there are
no path finding requests to process
* Count more ST objects (derive from `CountedObject`)
* Track CachedView stats in CountedObjects
* Rename the CachedView counters
* Fix the scope of the digest lookup lock
Before this patch, if you asked "is it caching?" It was always caching.
The `SHAMapItem` class contains a variable-sized buffer that
holds the serialized data associated with a particular item
inside a `SHAMap`.
Prior to this commit, the buffer for the serialized data was
allocated separately. Coupled with the fact that most instances
of `SHAMapItem` were wrapped around a `std::shared_ptr` meant
that an instantiation might result in up to three separate
memory allocations.
This commit switches away from `std::shared_ptr` for `SHAMapItem`
and uses `boost::intrusive_ptr` instead, allowing the reference
count for an instance to live inside the instance itself. Coupled
with using a slab-based allocator to optimize memory allocation
for the most commonly sized buffers, the net result is significant
memory savings. In testing, the reduction in memory usage hovers
between 400MB and 650MB. Other scenarios might result in larger
savings.
In performance testing with NFTs, this commit reduces memory size by
about 15% sustained over long duration.
Commit 2 of 3 in #4218.
A few unit tests have historically generated a lot of noise
to the console from log writes. This noise was not useful
and made it harder to locate actual test failures.
By changing the log level of these tests from
- severities::kError to
- severities::kDisabled
it was possible to remove that noise coming from the logs.
The existing logic involves every server sending every transaction
that it receives to all its peers (except the one that it received
a transaction from).
This commit instead uses a randomized algorithm, where a node will
randomly select peers to relay a given transaction to, caching the
list of transaction hashes that are not relayed and forwading them
to peers once every second. Peers can then determine whether there
are transactions that they have not seen and can request them from
the node which has them.
It is expected that this feature will further reduce the bandwidth
needed to operate a server.
