This change makes the `read` function call in `handleConnection` async, adds a new class `TestSink` to help debugging, and adds a new target `xrpl.tests.helpers` to put the helper class in.
To allow developers to consume the latest unstable and (near-)stable versions of our `xrpl` Conan recipe, we should export and upload it whenever a push occurs to the corresponding branch or a release tag has been created. This way, developers do not have to figure out themselves what the most recent shortened commit hash was to determine the latest unstable recipe version (e.g. `3.2.0-b0+a1b2c3d`) or what the most recent release (candidate) was to determine the latest (near-)stable recipe version (e.g. `3.1.0-rc2`).
Now, pushes to the `develop` branch will produce the `develop` recipe version, pushes to the `release` branch will produce the `rc` recipe version, and creation of versioned tags will produce the `release` recipe version.
This change replaces the mutex `stoppingMutex_`, the `atomic_bool` variable `isTimeToStop`, and the conditional variable `stoppingCondition_` with an `atomic_flag` variable.
When `xrpld` is running the embedded tests as a child process, it has a control thread (the app bundle thread) that starts the application, and an application thread (the thread that executes `app_->run()`). Due to the relaxed memory ordering on ARM, it's not guaranteed that the application thread can see the change of the value resulting from the `isTimeToStop.exchange(true)` call before it is notified by `stoppingCondition_.notify_all()`, even though they do happen in the right order in the app bundle thread in `ApplicationImp::signalStop`. We therefore often get into the situation where `isTimeToStop` is `true`, but the application thread is waiting for `stoppingCondition_` to notify, because the app bundle thread may have already notified before the application thread actually starts waiting.
Switching to a single `atomic_flag` variable makes sure that there's only one synchronisation object and then the memory order guarantee provided by c++ can make sure that `notify_all` gets synchronised after `test_and_set` does.
Fixing this issue will stop the unit tests hanging forever and then we should see less (or hopefully no) time out errors in daily github action runs
Since the minimum Clang version we support is 16, the checks for version < 15 are no longer necessary. This change therefore removes the macros checking if the clang version is < 15 and simplifies uses of `std::source_location`.
The `upload-conan-deps` workflow that's triggered on push is supposed to upload the Conan dependencies to our remote, so future PR commits can pull those dependencies from the remote. However, as the `sanitize` argument is missing, it was building different dependencies than what the PRs are building for the asan/tsan/ubsan job, so the latter would not find anything in the remote that they could use. This change sets the missing `sanitizers` input variable when running the `build-deps` action.
Separately, the `setup-conan` action showed the default profile, while we are using the `ci` profile. To ensure the profile is correctly printed when sanitizers are enabled, the environment variable the profile uses is set before calling the action.
The export and upload steps were initially in a separate action, where GitHub Actions does not support the `secrets` keyword, but only `inputs` for the credentials. After they were moved to a reusable workflow, only part of the references to the credentials were updated. This change correctly references to the Conan credentials via `secrets` instead of `inputs`.
By default the Conan recipe extracts the version from `BuildInfo.cpp`, but in some of the cases we want to upload a recipe with a suffix derived from the commit hash. This currently then results in the uploading to fail, since there is a version mismatch.
Here we explicitly set the version, and then simplify the steps in the upload workflow since we now need the recipe name (embedded within the conanfile.py but also needed when uploading), the recipe version, and the recipe ref (name/version).
Conan recipes use semantic versioning, and since our version already contains a hyphen the second hyphen causes Conan to ignore it. The plus sign is a valid separator we can use instead, so this change uses a `+` to separate a version suffix (commit hash) instead of a `-`.
There were a few uninitialized variables in CMake files. This change will make sure we always check if a variable has been initialized before using them, or in come cases initialize them by default. This change will raise an error on CI if a developer introduced an uninitialized variable in CMake files.
This change continues the thread naming work from #5691 and #5758, which enables more useful lock contention profiling by ensuring threads/jobs have short, stable, human-readable names (rather than being truncated/failing due to OS limits). This changes diagnostic naming only (thread names and job/load-event labels), not behavior.
Specific modifications are:
* Shortens all thread/job names used with `beast::setCurrentThreadName`, so the effective Linux thread name stays within the 15-character limit.
* Removes per-ledger sequence numbers from job/thread names to avoid long labels. This improves aggregation in lock contention profiling for short-lived job executions.
The Ripple Bug Bounty program recently changed the public keys that security researchers can use to encrypt vulnerabilities and messages for submission to the program. This information was updated on https://ripple.com/legal/bug-bounty/ and this PR updates the `SECURITY.md` to align.
During several iterations of development of https://github.com/XRPLF/rippled/pull/6235, the commit hash was supposed to be moved into the `run:` statement, but it slipped through the cracks and did not get added. This change adds the commit hash as suffix to the Conan recipe version.
The `Number.h` header file now has `std::reference_wrapper` from `<functional>`, but the include is missing, causing downstream build problems. This change adds the header.
This change uploads the `libxrpl` library as a Conan recipe to our remote when (i) merging into the `develop` branch, (ii) committing to a PR that targets a `release*` branch, and (iii) a versioned tag is applied. Clio is only notified in the second case. The user and channel are no longer used when uploading the recipe.
Specific changes are:
* A `generate-version` action is added, which extracts the build version from `BuildInfo.cpp` and appends the short 7-character commit hash to it for merges into the `develop` branch and for commits to a PR that targets a `release*` branch. When a tag is applied, however, the tag itself is used as the version. This functionality has been turned into a separate action as we will use the same versioning logic for creating .rpm and .deb packages, as well as Docker images.
* An `upload-recipe` action is added, which calls the `generate-version` action and further handles the uploading of the recipe to Conan.
* This action is called by both the `on-pr` and `on-trigger` workflows, and a new `on-tag` workflow.
The reason for this change is that we have downstream uses for the `libxrpl` library, but currently only upload the recipe to check for compatibility with Clio when making commits to a PR that targets the release branch.
`PeerImp` processes `TMGetObjectByHash` queries with an unbounded per-request loop, which performs a `NodeStore` fetch and then appends retrieved data to the reply for each queried object without a local count cap or reply-byte budget. However, the `Nodestore` fetches are expensive when high in numbers, which might slow down the process overall. Hence this code change adds an upper cap on the response size.
This change removes the `master` branch as a trigger for the CI pipelines, and updates comments accordingly. It also fixes the pre-commit workflow, so it will run on all release branches.
These "fixed location" objects can be found in multiple ways:
1. The lookup parameters use the same format as other ledger objects, but the only valid value is true or the valid index of the object:
- Amendments: "amendments" : true
- FeeSettings: "fee" : true
- NegativeUNL: "nunl" : true
- LedgerHashes: "hashes" : true (For the "short" list. See below.)
2. With RPC API >= 3, using special case values to "index", such as "index" : "amendments". Uses the same names as above. Note that for "hashes", this option will only return the recent ledger hashes / "short" skip list.
3. LedgerHashes has two types: "short", which stores recent ledger hashes, and "long", which stores the flag ledger hashes for a particular ledger range.
- To find a "long" LedgerHashes object, request '"hashes" : <ledger sequence>'. <ledger sequence> must be a number that evaluates to an unsigned integer.
- To find the "short" LedgerHashes object, request "hashes": true as with the other fixed objects.
The following queries are all functionally equivalent:
- "amendments" : true
- "index" : "amendments" (API >=3 only)
- "amendments" : "7DB0788C020F02780A673DC74757F23823FA3014C1866E72CC4CD8B226CD6EF4"
- "index" : "7DB0788C020F02780A673DC74757F23823FA3014C1866E72CC4CD8B226CD6EF4"
Finally, whether the object is found or not, if a valid index is computed, that index will be returned. This can be used to confirm the query was valid, or to save the index for future use.
This change adds support for sanitizer build options in CI builds workflow. Currently `asan+ubsan` is enabled, while `tsan+ubsan` is left disabled as more changes are required.
The word `failed` in the test case makes it hard to search through the test logs when an actual test failure occurs, so this change renames the word to just `fail` instead.
- Refactor Number internals away from int64 to uint64 & a sign flag
- ctors and accessors use `rep`. Very few things expose
`internalrep`.
- An exception is "unchecked" and the new "normalized", which explicitly
take an internalrep. But with those special control flags, it's easier
to distinguish and control when they are used.
- For now, skip the larger mantissas in AMM transactions and tests
- Remove trailing zeros from scientific notation Number strings
- Update tests. This has the happy side effect of making some of the string
representations _more_ consistent between the small and large
mantissa ranges.
- Add semi-automatic rounding of STNumbers based on Asset types
- Create a new SField metadata enum, sMD_NeedsAsset, which indicates
the field should be associated with an Asset so it can be rounded.
- Add a new STTakesAsset intermediate class to handle the Asset
association to a derived ST class. Currently only used in STNumber,
but could be used by other types in the future.
- Add "associateAsset" which takes an SLE and an Asset, finds the
sMD_NeedsAsset fields, and associates the Asset to them. In the case
of STNumber, that both stores the Asset, and rounds the value
immediately.
- Transactors only need to add a call to associateAsset _after_ all of
the STNumbers have been set. Unfortunately, the inner workings of
STObject do not do the association correctly with uninitialized
fields.
- When serializing an STNumber that has an Asset, round it before
serializing.
- Add an override of roundToAsset, which rounds a Number value in place
to an Asset, but without any additional scale.
- Update and fix a bunch of Loan-related tests to accommodate the
expanded Number class.
---------
Co-authored-by: Vito <5780819+Tapanito@users.noreply.github.com>
- Spec: XLS-66
Fix overpayment asserts (#6084)
MPTTester::operator() parameter should be std::int64_t
- Originally defined as uint64_t, but the testIssuerLoan() test called
it with a negative number, causing an overflow to a very large number
that in some circumstances could be silently cast back to an int64_t,
but might not be. I believe this is UB, and we don't want to rely on
that.
Review feedback from @Tapanito: overpayment value change
- In overpayment results, the management fee was being calculated twice:
once as part of the value change, and as part of the fees paid.
Exclude it from the value change.
Fix Overpayment Calculation (#6087)
- Adds additional unit tests to cover math calculations.
- Removes unused methods.
Review feedback from @shawnxie999: even more rounding
- Round the initial total value computation upward, unless there is
0-interest.
- Rename getVaultScale to getAssetsTotalScale, and convert one incorrect
computation to use it.
- Use adjustImpreciseNumber for LossUnrealized.
- Add some logging to computeLoanProperties.
Fix LoanBrokerSet debtMaximum limits (#6116)
Fix some minor bugs in Lending Protocol (#6101)
- add nodiscard to unimpairLoan, and check result in LoanPay
- add a check to verify that issuer exists
- improve LoanManage error code for dust amounts
Check permissions in LoanSet and LoanPay (#6108)
Disallow pseudo accounts to be Destination for LoanBrokerCoverWithdraw (#6106)
Ensure vault asset cap is not exceeded (#6124)
Fix Overpayment ValueChange calculation in Lending Protocol (#6114)
- Adds loan state to LoanProperties.
- Cleans up computeLoanProperties.
- Fixes missing management fee from overpayment.
fix: Enable LP Deposits when the broker is the asset issuer (#6119)
* Replace accountHolds with accountSpendable when checking
for account funds in VaultDeposit and LoanBrokerCoverDeposit
Add a few minor changes (#6158)
- Updates or fixes a couple of things I noticed while reviewing changes
to the spec.
- Rename sfPreviousPaymentDate to sfPreviousPaymentDueDate.
- Make the vault asset cap check added in #6124 a little more robust:
1. Check in preflight if the vault is _already_ over the limit.
2. Prevent overflow when checking with the loan value. (Subtract
instead of adding, in case the values are near maxint. Both return
the same result. Also add a unit test so each case is covered.
Add minimum grace period validation (#6133)
Fix bugs: frozen pseudo-account, and FLC cutoff (#6170)
refactor: Rename raw state to theoretical state (#6187)
Check if a withdrawal amount exceeds any applicable receiving limit. (#6117)
Fix overpayment result calculation (#6195)
Address review feedback from Lending Protocol re-review (#6161)
---------
Co-authored-by: Gregory Tsipenyuk <gregtatcam@users.noreply.github.com>
Co-authored-by: Bronek Kozicki <brok@incorrekt.com>
Co-authored-by: Vito Tumas <5780819+Tapanito@users.noreply.github.com>
Co-authored-by: Shawn Xie <35279399+shawnxie999@users.noreply.github.com>
Co-authored-by: Jingchen <a1q123456@users.noreply.github.com>
This change removes unnecessary version numbers in the OpenSSL and Boost `find_package` CMake statements. An unnecessary OpenSSL definition is removed, while Conan options for SSL are updated to disable insecure ciphers. Moreover, the statements are now ordered alphabetically and more logically.
- Introduces amendment `fixBatchInnerSigs`
- Update Batch unit tests
- Fix all the Env instantiations to _use_ the "features" parameter.
- testInnerSubmitRPC runs with Batch enabled and disabled.
- Add a test to testInnerSubmitRPC for a correctly signed tx incorrectly
using the tfInnerBatchTxn flag.
- Generalize the submitAndValidate lambda in testInnerSubmitRPC.
- With the fix amendment, a transaction never reaches the transaction
engine (Transactor and derived classes.)
- Test submitting a pseudo-transaction. Stopped before reaching the
transaction engine, but with different errors.
- The tests verify that without the amendment, a transaction with
tfInnerBatchTxn is immediately rejected. Without the amendment, things
are safe. The amendment just makes things safer and more future-proof.
- Adds a mechanism for the vault owner to burn user shares when the vault is stuck. If the Vault has 0 AssetsAvailable and Total, the owner may submit a VaultClawback to reclaim the worthless fees, and thus allow the Vault to be deleted. The Amount must be left off (unless the owner is the asset issuer), specified as 0 Shares, or specified as the number of Shares held.
This change:
* Truncates thread names if more than 15 chars with `snprintf`.
* Adds warnings for truncated thread names if `-DTRUNCATED_THREAD_NAME_LOGS=ON`.
* Add a static assert for string literals to stop compiling if > 15 chars.
* Shortens `Resource::Manager` to `Resource::Mngr` to fix the static assert failure.
* Updates `CurrentThreadName_test` unit test specifically for Linux to verify truncation.
This change updates the XRPLF pre-commit workflow and prepare-runner action to their latest versions. For naming consistency the prepare-runner action changed the disable_ccache variable into enable_ccache, which matches our naming.
This change fixes the last of the spelling issues, and enables the pre-commit (and CI) check for spelling. There are no functionality changes, but it does rename some enum values.
