Merge develop into ripple/smart-escrow (#5357)

* Set version to 2.4.0

* refactor: Remove unused and add missing includes (#5293)

The codebase is filled with includes that are unused, and which thus can be removed. At the same time, the files often do not include all headers that contain the definitions used in those files. This change uses clang-format and clang-tidy to clean up the includes, with minor manual intervention to ensure the code compiles on all platforms.

* refactor: Calculate numFeatures automatically (#5324)

Requiring manual updates of numFeatures is an annoying manual process that is easily forgotten, and leads to frequent merge conflicts. This change takes advantage of the `XRPL_FEATURE` and `XRPL_FIX` macros, and adds a new `XRPL_RETIRE` macro to automatically set `numFeatures`.

* refactor: Improve ordering of headers with clang-format (#5343)

Removes all manual header groupings from source and header files by leveraging clang-format options.

* Rename "deadlock" to "stall" in `LoadManager` (#5341)

What the LoadManager class does is stall detection, which is not the same as deadlock detection. In the condition of severe CPU starvation, LoadManager will currently intentionally crash rippled reporting `LogicError: Deadlock detected`. This error message is misleading as the condition being detected is not a deadlock. This change fixes and refactors the code in response.

* Adds hub.xrpl-commons.org as a new Bootstrap Cluster (#5263)

* fix: Error message for ledger_entry rpc (#5344)

Changes the error to `malformedAddress` for `permissioned_domain` in the `ledger_entry` rpc, when the account is not a string. This change makes it more clear to a user what is wrong with their request.

* fix: Handle invalid marker parameter in grpc call (#5317)

The `end_marker` is used to limit the range of ledger entries to fetch. If `end_marker` is less than `marker`, a crash can occur. This change adds an additional check.

* fix: trust line RPC no ripple flag (#5345)

The Trustline RPC `no_ripple` flag gets set depending on `lsfDefaultRipple` flag, which is not a flag of a trustline but of the account root. The `lsfDefaultRipple` flag does not provide any insight if this particular trust line has `lsfLowNoRipple` or `lsfHighNoRipple` flag set, so it should not be used here at all. This change simplifies the logic.

* refactor: Updates Conan dependencies: RocksDB (#5335)

Updates RocksDB to version 9.7.3, the latest version supported in Conan 1.x. A patch for 9.7.4 that fixes a memory leak is included.

* fix: Remove null pointer deref, just do abort (#5338)

This change removes the existing undefined behavior from `LogicError`, so we can be certain that there will be always a stacktrace.

De-referencing a null pointer is an old trick to generate `SIGSEGV`, which would typically also create a stacktrace. However it is also an undefined behaviour and compilers can do something else. A more robust way to create a stacktrace while crashing the program is to use `std::abort`, which we have also used in this location for a long time. If we combine the two, we might not get the expected behaviour - namely, the nullpointer deref followed by `std::abort`, as handled in certain compiler versions may not immediately cause a crash. We have observed stacktrace being wiped instead, and thread put in indeterminate state, then stacktrace created without any useful information.

* chore: Add PR number to payload (#5310)

This PR adds one more payload field to the libXRPL compatibility check workflow - the PR number itself.

* chore: Update link to ripple-binary-codec (#5355)

The link to ripple-binary-codec's definitions.json appears to be outdated. The updated link is also documented here: https://xrpl.org/docs/references/protocol/binary-format#definitions-file

* Prevent consensus from getting stuck in the establish phase (#5277)

- Detects if the consensus process is "stalled". If it is, then we can declare a 
  consensus and end successfully even if we do not have 80% agreement on
  our proposal.
  - "Stalled" is defined as:
    - We have a close time consensus
    - Each disputed transaction is individually stalled:
      - It has been in the final "stuck" 95% requirement for at least 2
        (avMIN_ROUNDS) "inner rounds" of phaseEstablish,
      - and either all of the other trusted proposers or this validator, if proposing,
        have had the same vote(s) for at least 4 (avSTALLED_ROUNDS) "inner
        rounds", and at least 80% of the validators (including this one, if
        appropriate) agree about the vote (whether yes or no).
- If we have been in the establish phase for more than 10x the previous
  consensus establish phase's time, then consensus is considered "expired",
  and we will leave the round, which sends a partial validation (indicating
  that the node is moving on without validating). Two restrictions avoid
  prematurely exiting, or having an extended exit in extreme situations.
  - The 10x time is clamped to be within a range of 15s
    (ledgerMAX_CONSENSUS) to 120s (ledgerABANDON_CONSENSUS).
  - If consensus has not had an opportunity to walk through all avalanche
    states (defined as not going through 8 "inner rounds" of phaseEstablish),
    then ConsensusState::Expired is treated as ConsensusState::No.
- When enough nodes leave the round, any remaining nodes will see they've
  fallen behind, and move on, too, generally before hitting the timeout. Any
  validations or partial validations sent during this time will help the
  consensus process bring the nodes back together.

---------

Co-authored-by: Michael Legleux <mlegleux@ripple.com>
Co-authored-by: Bart <bthomee@users.noreply.github.com>
Co-authored-by: Ed Hennis <ed@ripple.com>
Co-authored-by: Bronek Kozicki <brok@incorrekt.com>
Co-authored-by: Darius Tumas <Tokeiito@users.noreply.github.com>
Co-authored-by: Sergey Kuznetsov <skuznetsov@ripple.com>
Co-authored-by: cyan317 <120398799+cindyyan317@users.noreply.github.com>
Co-authored-by: Vlad <129996061+vvysokikh1@users.noreply.github.com>
Co-authored-by: Alex Kremer <akremer@ripple.com>
This commit is contained in:
Mayukha Vadari
2025-03-20 16:47:14 -04:00
committed by GitHub
parent b6a95f9970
commit 911c0466c0
983 changed files with 3542 additions and 1397 deletions

View File

@@ -44,6 +44,7 @@ DerivePointerAlignment: false
DisableFormat: false
ExperimentalAutoDetectBinPacking: false
ForEachMacros: [ Q_FOREACH, BOOST_FOREACH ]
IncludeBlocks: Regroup
IncludeCategories:
- Regex: '^<(test)/'
Priority: 0
@@ -53,8 +54,12 @@ IncludeCategories:
Priority: 2
- Regex: '^<(boost)/'
Priority: 3
- Regex: '.*'
- Regex: '^.*/'
Priority: 4
- Regex: '^.*\.h'
Priority: 5
- Regex: '.*'
Priority: 6
IncludeIsMainRegex: '$'
IndentCaseLabels: true
IndentFunctionDeclarationAfterType: false

View File

@@ -14,7 +14,7 @@ runs:
run: |
conan config set general.revisions_enabled=1
conan export external/snappy snappy/1.1.10@
conan export external/rocksdb rocksdb/6.29.5@
conan export external/rocksdb rocksdb/9.7.3@
conan export external/soci soci/4.0.3@
conan export external/nudb nudb/2.0.8@
- name: add Ripple Conan remote
@@ -55,7 +55,3 @@ runs:
--options xrpld=True \
--settings build_type=${{ inputs.configuration }} \
..
- name: upload dependencies to remote
if: (steps.binaries.outputs.missing != '[]') && (steps.remote.outputs.outcome == 'success')
shell: bash
run: conan upload --remote ripple '*' --all --parallel --confirm

View File

@@ -85,4 +85,5 @@ jobs:
run: |
gh api --method POST -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" \
/repos/xrplf/clio/dispatches -f "event_type=check_libxrpl" \
-F "client_payload[version]=${{ needs.publish.outputs.version }}@${{ needs.publish.outputs.channel }}"
-F "client_payload[version]=${{ needs.publish.outputs.version }}@${{ needs.publish.outputs.channel }}" \
-F "client_payload[pr]=${{ github.event.pull_request.number }}"

View File

@@ -73,7 +73,6 @@ jobs:
run : |
conan profile new default --detect || true
conan profile update settings.compiler.cppstd=20 default
conan profile update 'conf.tools.build:cxxflags+=["-DBOOST_ASIO_DISABLE_CONCEPTS"]' default
- name: build dependencies
uses: ./.github/actions/dependencies
env:

View File

@@ -178,9 +178,9 @@ It does not override paths to dependencies when building with Visual Studio.
```
# Conan 1.x
conan export external/rocksdb rocksdb/6.29.5@
conan export external/rocksdb rocksdb/9.7.3@
# Conan 2.x
conan export --version 6.29.5 external/rocksdb
conan export --version 9.7.3 external/rocksdb
```
Export our [Conan recipe for SOCI](./external/soci).

View File

@@ -14,10 +14,10 @@ Loop: xrpld.app xrpld.net
xrpld.app > xrpld.net
Loop: xrpld.app xrpld.overlay
xrpld.overlay == xrpld.app
xrpld.overlay > xrpld.app
Loop: xrpld.app xrpld.peerfinder
xrpld.app > xrpld.peerfinder
xrpld.peerfinder ~= xrpld.app
Loop: xrpld.app xrpld.rpc
xrpld.rpc > xrpld.app

View File

@@ -6,6 +6,7 @@ libxrpl.protocol > xrpl.basics
libxrpl.protocol > xrpl.json
libxrpl.protocol > xrpl.protocol
libxrpl.resource > xrpl.basics
libxrpl.resource > xrpl.json
libxrpl.resource > xrpl.resource
libxrpl.server > xrpl.basics
libxrpl.server > xrpl.json
@@ -42,6 +43,7 @@ test.consensus > xrpl.basics
test.consensus > xrpld.app
test.consensus > xrpld.consensus
test.consensus > xrpld.ledger
test.consensus > xrpl.json
test.core > test.jtx
test.core > test.toplevel
test.core > test.unit_test
@@ -58,7 +60,6 @@ test.json > test.jtx
test.json > xrpl.json
test.jtx > xrpl.basics
test.jtx > xrpld.app
test.jtx > xrpld.consensus
test.jtx > xrpld.core
test.jtx > xrpld.ledger
test.jtx > xrpld.net
@@ -158,7 +159,6 @@ xrpld.core > xrpl.basics
xrpld.core > xrpl.json
xrpld.core > xrpl.protocol
xrpld.ledger > xrpl.basics
xrpld.ledger > xrpld.core
xrpld.ledger > xrpl.json
xrpld.ledger > xrpl.protocol
xrpld.net > xrpl.basics
@@ -183,7 +183,6 @@ xrpld.peerfinder > xrpld.core
xrpld.peerfinder > xrpl.protocol
xrpld.perflog > xrpl.basics
xrpld.perflog > xrpl.json
xrpld.perflog > xrpl.protocol
xrpld.rpc > xrpl.basics
xrpld.rpc > xrpld.core
xrpld.rpc > xrpld.ledger

View File

@@ -5,6 +5,108 @@
This document contains the release notes for `rippled`, the reference server implementation of the XRP Ledger protocol. To learn more about how to build, run or update a `rippled` server, visit https://xrpl.org/install-rippled.html
Have new ideas? Need help with setting up your node? [Please open an issue here](https://github.com/xrplf/rippled/issues/new/choose).
## Full Changelog
### Amendments
The following amendments are open for voting with this release:
- **DynamicNFT (XLS-46)** - Adds the ability to mint mutable `NFToken` objects whose URI can be changed. ([#5048](https://github.com/XRPLF/rippled/pull/5048))
- **PermissionedDomains (XLS-80)** - Adds Permissioned Domains, which act as part of broader systems on the XRP Ledger to restrict access to satisfy compliance rules. ([#5161](https://github.com/XRPLF/rippled/pull/5161))
- **DeepFreeze (XLS-77)** - Adds the ability to deep freeze trust lines, enabling token issuers to block the transfer of assets for holders who have been deep frozen. ([#5187](https://github.com/XRPLF/rippled/pull/5187))
- **fixFrozenLPTokenTransfer** - Prohibits the transfer of LP tokens when the associated liquidity pool contains at least one frozen asset. ([#5227](https://github.com/XRPLF/rippled/pull/5227))
- **fixInvalidTxFlags** - Adds transaction flag checking for `CredentialCreate`, `CredentialAccept`, and `CredentialDelete` transactions. ([#5250](https://github.com/XRPLF/rippled/pull/5250))
### New Features
- Added a new `simulate` API method to execute dry runs of transactions and see the simulated metadata. ([#5069](https://github.com/XRPLF/rippled/pull/5069), [#5265](https://github.com/XRPLF/rippled/pull/5265))
- Added the ability to specify MPTs when defining assets in transactions. ([#5200](https://github.com/XRPLF/rippled/pull/5200))
- Added a `state` alias for `ripple_state` in the `ledger_entry` API method. Also refactored `LedgerEntry.cpp` to make it easier to read. ([#5199](https://github.com/XRPLF/rippled/pull/5199))
- Improved UNL security by enabling validators to set a minimum number of UNL publishers to agree on validators. ([#5112](https://github.com/XRPLF/rippled/pull/5112))
- Updated the XRPL Foundation UNL keys. ([#5289](https://github.com/XRPLF/rippled/pull/5289))
- Added a new XRPL Foundation subdomain to enable a staged migration without modifying the key for the current UNL list. ([#5326](https://github.com/XRPLF/rippled/pull/5326))
- Added support to filter ledger entry types by their canonical names in the `ledger`, `ledger_data`, and `account_objects` API methods. ([#5271](https://github.com/XRPLF/rippled/pull/5271))
- Added detailed logging for each validation and proposal received from the network. ([#5291](https://github.com/XRPLF/rippled/pull/5291))
- Improved git commit hash lookups when checking the version of a `rippled` debug build. Also added git commit hash info when using the `server_info` API method on an admin connection. ([#5225](https://github.com/XRPLF/rippled/pull/5225))
### Bug fixes
- Fixed an issue with overlapping data types in the `Expected` class. ([#5218](https://github.com/XRPLF/rippled/pull/5218))
- Fixed an issue that prevented `rippled` from building on Windows with VS2022. ([#5197](https://github.com/XRPLF/rippled/pull/5197))
- Fixed `server_definitions` prefixes. ([#5231](https://github.com/XRPLF/rippled/pull/5231))
- Added missing dependency installations for generic MasOS runners. ([#5233](https://github.com/XRPLF/rippled/pull/5233))
- Updated deprecated Github actions. ([#5241](https://github.com/XRPLF/rippled/pull/5241))
- Fixed a failing assert scenario when submitting the `connect` admin RPC. ([#5235](https://github.com/XRPLF/rippled/pull/5235))
- Fixed the levelization script to ignore single-line comments during dependency analysis. ([#5194](https://github.com/XRPLF/rippled/pull/5194))
- Fixed the assert name used in `PermissionedDomainDelete`. ([#5245](https://github.com/XRPLF/rippled/pull/5245))
- Fixed macOS unit tests. ([#5196](https://github.com/XRPLF/rippled/pull/5196))
- Fixed an issue with validators not accurately reflecting amendment votes. Also added debug logging of amendment votes. ([#5173](https://github.com/XRPLF/rippled/pull/5173), [#5312](https://github.com/XRPLF/rippled/pull/5312))
- Fixed a potential issue with double-charging fees. ([#5269](https://github.com/XRPLF/rippled/pull/5269))
- Removed the `new parent hash` assert and replaced it with a log message. ([#5313](https://github.com/XRPLF/rippled/pull/5313))
- Fixed an issue that prevented previously-failed inbound ledgers to not be acquired if a new trusted proposal arrived. ([#5318](https://github.com/XRPLF/rippled/pull/5318))
### Other Improvements
- Added unit tests for `AccountID` handling. ([#5174](https://github.com/XRPLF/rippled/pull/5174))
- Added enforced levelization in `libxrpl` with CMake. ([#5199](https://github.com/XRPLF/rippled/pull/5111))
- Updated `libxrpl` and all submodules to use the same compiler options. ([#5228](https://github.com/XRPLF/rippled/pull/5228))
- Added Antithesis instrumentation. ([#5042](https://github.com/XRPLF/rippled/pull/5042), [#5213](https://github.com/XRPLF/rippled/pull/5213))
- Added `rpcName` to the `LEDGER_ENTRY` macro to help prevent future bugs. ([#5202](https://github.com/XRPLF/rippled/pull/5202))
- Updated the contribution guidelines to introduce a new workflow that avoids code freezes. Also added scripts that can be used by maintainers in branch management, and a CI job to check that code is consistent across the three main branches: `master`, `release`, and `develop`. ([#5215](https://github.com/XRPLF/rippled/pull/5215))
- Added unit tests to check for caching issues fixed in `rippled 2.3.0`. ([#5242](https://github.com/XRPLF/rippled/pull/5242))
- Cleaned up the API changelog. ([#5207](https://github.com/XRPLF/rippled/pull/5207))
- Improved logs readability. ([#5251](https://github.com/XRPLF/rippled/pull/5251))
- Updated Visual Studio CI to VS 2022, and added VS Debug builds. ([#5240](https://github.com/XRPLF/rippled/pull/5240))
- Updated the `secp256k1` library to version 0.6.0. ([#5254](https://github.com/XRPLF/rippled/pull/5254))
- Changed the `[port_peer]` parameter in `rippled` example config back to `51235`; also added the recommendation to use the default port of `2459` for new deployments. ([#5290](https://github.com/XRPLF/rippled/pull/5290), [#5299](https://github.com/XRPLF/rippled/pull/5299))
- Improved CI management. ([#5268](https://github.com/XRPLF/rippled/pull/5268))
- Updated the git commit message rules for contributors. ([#5283](https://github.com/XRPLF/rippled/pull/5283))
- Fixed unnecessary `setCurrentThreadName` calls. ([#5280](https://github.com/XRPLF/rippled/pull/5280))
- Added a check to prevent permissioned domains from being created in the event the Permissioned Domains amendement is enabled before the Credentials amendement. ([#5275](https://github.com/XRPLF/rippled/pull/5275))
- Updated Conan dependencies. ([#5256](https://github.com/XRPLF/rippled/pull/5256))
- Fixed minor typos in code comments. ([#5279](https://github.com/XRPLF/rippled/pull/5279))
- Fixed incorrect build instructions. ([#5274](https://github.com/XRPLF/rippled/pull/5274))
- Refactored `rotateWithLock()` to not hold a lock during callbacks. ([#5276](https://github.com/XRPLF/rippled/pull/5276))
- Cleaned up debug logging by combining multiple data points into a single message. ([#5302](https://github.com/XRPLF/rippled/pull/5302))
- Updated build flags to fix performance regressions. ([#5325](https://github.com/XRPLF/rippled/pull/5325))
## Credits
The following people contributed directly to this release:
- Aanchal Malhotra <amalhotra@ripple.com>
- Bart Thomee <11445373+bthomee@users.noreply.github.com>
- Bronek Kozicki <brok@incorrekt.com>
- code0xff <ian.jungyong.um@gmail.com>
- Darius Tumas <tokeiito@tokeiito.eu>
- David Fuelling <fuelling@ripple.com>
- Donovan Hide <donovanhide@gmail.com>
- Ed Hennis <ed@ripple.com>
- Elliot Lee <github.public@intelliot.com>
- Javier Romero <jromero@ripple.com>
- Kenny Lei <klei@ripple.com>
- Mark Travis <7728157+mtrippled@users.noreply.github.com>
- Mayukha Vadari <mvadari@gmail.com>
- Michael Legleux <mlegleux@ripple.com>
- Oleksandr <115580134+oleks-rip@users.noreply.github.com>
- Qi Zhao <qzhao@ripple.com>
- Ramkumar Srirengaram Gunasegharan <rgunasegharan@ripple.com>
- Shae Wang <swang@ripple.com>
- Shawn Xie <shawnxie920@gmail.com>
- Sophia Xie <sxie@ripple.com>
- Vijay Khanna Raviraj <vraviraj@ripple.com>
- Vladislav Vysokikh <vvysokikh@gmail.com>
- Xun Zhao <xzhao@ripple.com>
## Bug Bounties and Responsible Disclosures
We welcome reviews of the `rippled` code and urge researchers to responsibly disclose any issues they may find.
To report a bug, please send a detailed report to: <bugs@xrpl.org>
# Version 2.3.1

View File

@@ -420,6 +420,7 @@
# - r.ripple.com 51235
# - sahyadri.isrdc.in 51235
# - hubs.xrpkuwait.com 51235
# - hub.xrpl-commons.org 51235
#
# Examples:
#

View File

@@ -108,7 +108,7 @@ class Xrpl(ConanFile):
if self.options.jemalloc:
self.requires('jemalloc/5.3.0')
if self.options.rocksdb:
self.requires('rocksdb/6.29.5')
self.requires('rocksdb/9.7.3')
exports_sources = (
'CMakeLists.txt',

View File

@@ -558,7 +558,7 @@ struct ConsensusResult
ConsensusTimer roundTime;
// Indicates state in which consensus ended. Once in the accept phase
// will be either Yes or MovedOn
// will be either Yes or MovedOn or Expired
ConsensusState state = ConsensusState::No;
};

View File

@@ -1,7 +1,7 @@
#include <cstdio>
#include <xrpl/protocol/BuildInfo.h>
#include <cstdio>
int main(int argc, char const** argv) {
std::printf("%s\n", ripple::BuildInfo::getVersionString().c_str());
return 0;

View File

@@ -1,27 +1,12 @@
sources:
"6.29.5":
url: "https://github.com/facebook/rocksdb/archive/refs/tags/v6.29.5.tar.gz"
sha256: "ddbf84791f0980c0bbce3902feb93a2c7006f6f53bfd798926143e31d4d756f0"
"6.27.3":
url: "https://github.com/facebook/rocksdb/archive/refs/tags/v6.27.3.tar.gz"
sha256: "ee29901749b9132692b26f0a6c1d693f47d1a9ed8e3771e60556afe80282bf58"
"6.20.3":
url: "https://github.com/facebook/rocksdb/archive/refs/tags/v6.20.3.tar.gz"
sha256: "c6502c7aae641b7e20fafa6c2b92273d935d2b7b2707135ebd9a67b092169dca"
"8.8.1":
url: "https://github.com/facebook/rocksdb/archive/refs/tags/v8.8.1.tar.gz"
sha256: "056c7e21ad8ae36b026ac3b94b9d6e0fcc60e1d937fc80330921e4181be5c36e"
"9.7.3":
url: "https://github.com/facebook/rocksdb/archive/refs/tags/v9.7.3.tar.gz"
sha256: "acfabb989cbfb5b5c4d23214819b059638193ec33dad2d88373c46448d16d38b"
patches:
"6.29.5":
- patch_file: "patches/6.29.5-0001-add-include-cstdint-for-gcc-13.patch"
patch_description: "Fix build with gcc 13 by including cstdint"
patch_type: "portability"
patch_source: "https://github.com/facebook/rocksdb/pull/11118"
- patch_file: "patches/6.29.5-0002-exclude-thirdparty.patch"
"9.7.3":
- patch_file: "patches/9.x.x-0001-exclude-thirdparty.patch"
patch_description: "Do not include thirdparty.inc"
patch_type: "portability"
"6.27.3":
- patch_file: "patches/6.27.3-0001-add-include-cstdint-for-gcc-13.patch"
patch_description: "Fix build with gcc 13 by including cstdint"
- patch_file: "patches/9.7.3-0001-memory-leak.patch"
patch_description: "Fix a leak of obsolete blob files left open until DB::Close()"
patch_type: "portability"
patch_source: "https://github.com/facebook/rocksdb/pull/11118"

View File

@@ -15,10 +15,10 @@ required_conan_version = ">=1.53.0"
class RocksDBConan(ConanFile):
name = "rocksdb"
homepage = "https://github.com/facebook/rocksdb"
description = "A library that provides an embeddable, persistent key-value store for fast storage"
license = ("GPL-2.0-only", "Apache-2.0")
url = "https://github.com/conan-io/conan-center-index"
description = "A library that provides an embeddable, persistent key-value store for fast storage"
homepage = "https://github.com/facebook/rocksdb"
topics = ("database", "leveldb", "facebook", "key-value")
package_type = "library"
settings = "os", "arch", "compiler", "build_type"
@@ -58,12 +58,12 @@ class RocksDBConan(ConanFile):
@property
def _compilers_minimum_version(self):
return {} if self._min_cppstd == "11" else {
"apple-clang": "10",
"clang": "7",
"gcc": "7",
"msvc": "191",
"Visual Studio": "15",
}
"apple-clang": "10",
"clang": "7",
"gcc": "7",
"msvc": "191",
"Visual Studio": "15",
}
def export_sources(self):
export_conandata_patches(self)
@@ -115,9 +115,9 @@ class RocksDBConan(ConanFile):
check_min_vs(self, "191")
if self.version == "6.20.3" and \
self.settings.os == "Linux" and \
self.settings.compiler == "gcc" and \
Version(self.settings.compiler.version) < "5":
self.settings.os == "Linux" and \
self.settings.compiler == "gcc" and \
Version(self.settings.compiler.version) < "5":
raise ConanInvalidConfiguration("Rocksdb 6.20.3 is not compilable with gcc <5.") # See https://github.com/facebook/rocksdb/issues/3522
def source(self):
@@ -163,6 +163,8 @@ class RocksDBConan(ConanFile):
if self.options.with_jemalloc:
deps.set_property("jemalloc", "cmake_file_name", "JeMalloc")
deps.set_property("jemalloc", "cmake_target_name", "JeMalloc::JeMalloc")
if self.options.with_zstd:
deps.set_property("zstd", "cmake_target_name", "zstd::zstd")
deps.generate()
def build(self):

View File

@@ -1,30 +0,0 @@
--- a/include/rocksdb/utilities/checkpoint.h
+++ b/include/rocksdb/utilities/checkpoint.h
@@ -8,6 +8,7 @@
#pragma once
#ifndef ROCKSDB_LITE
+#include <cstdint>
#include <string>
#include <vector>
#include "rocksdb/status.h"
--- a/table/block_based/data_block_hash_index.h
+++ b/table/block_based/data_block_hash_index.h
@@ -5,6 +5,7 @@
#pragma once
+#include <cstdint>
#include <string>
#include <vector>
--- a/util/string_util.h
+++ b/util/string_util.h
@@ -6,6 +6,7 @@
#pragma once
+#include <cstdint>
#include <sstream>
#include <string>
#include <unordered_map>

View File

@@ -1,16 +0,0 @@
diff --git a/CMakeLists.txt b/CMakeLists.txt
index ec59d4491..35577c998 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -101 +100,0 @@ if(MSVC)
- option(WITH_GFLAGS "build with GFlags" OFF)
@@ -103,2 +102,2 @@ if(MSVC)
- include(${CMAKE_CURRENT_SOURCE_DIR}/thirdparty.inc)
-else()
+endif()
+
@@ -117 +116 @@ else()
- if(MINGW)
+ if(MINGW OR MSVC)
@@ -183 +181,0 @@ else()
-endif()

View File

@@ -0,0 +1,319 @@
diff --git a/HISTORY.md b/HISTORY.md
index 36d472229..05ad1a202 100644
--- a/HISTORY.md
+++ b/HISTORY.md
@@ -1,6 +1,10 @@
# Rocksdb Change Log
> NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt`
+## 9.7.4 (10/31/2024)
+### Bug Fixes
+* Fix a leak of obsolete blob files left open until DB::Close(). This bug was introduced in version 9.4.0.
+
## 9.7.3 (10/16/2024)
### Behavior Changes
* OPTIONS file to be loaded by remote worker is now preserved so that it does not get purged by the primary host. A similar technique as how we are preserving new SST files from getting purged is used for this. min_options_file_numbers_ is tracked like pending_outputs_ is tracked.
diff --git a/db/blob/blob_file_cache.cc b/db/blob/blob_file_cache.cc
index 5f340aadf..1b9faa238 100644
--- a/db/blob/blob_file_cache.cc
+++ b/db/blob/blob_file_cache.cc
@@ -42,6 +42,7 @@ Status BlobFileCache::GetBlobFileReader(
assert(blob_file_reader);
assert(blob_file_reader->IsEmpty());
+ // NOTE: sharing same Cache with table_cache
const Slice key = GetSliceForKey(&blob_file_number);
assert(cache_);
@@ -98,4 +99,13 @@ Status BlobFileCache::GetBlobFileReader(
return Status::OK();
}
+void BlobFileCache::Evict(uint64_t blob_file_number) {
+ // NOTE: sharing same Cache with table_cache
+ const Slice key = GetSliceForKey(&blob_file_number);
+
+ assert(cache_);
+
+ cache_.get()->Erase(key);
+}
+
} // namespace ROCKSDB_NAMESPACE
diff --git a/db/blob/blob_file_cache.h b/db/blob/blob_file_cache.h
index 740e67ada..6858d012b 100644
--- a/db/blob/blob_file_cache.h
+++ b/db/blob/blob_file_cache.h
@@ -36,6 +36,15 @@ class BlobFileCache {
uint64_t blob_file_number,
CacheHandleGuard<BlobFileReader>* blob_file_reader);
+ // Called when a blob file is obsolete to ensure it is removed from the cache
+ // to avoid effectively leaking the open file and assicated memory
+ void Evict(uint64_t blob_file_number);
+
+ // Used to identify cache entries for blob files (not normally useful)
+ static const Cache::CacheItemHelper* GetHelper() {
+ return CacheInterface::GetBasicHelper();
+ }
+
private:
using CacheInterface =
BasicTypedCacheInterface<BlobFileReader, CacheEntryRole::kMisc>;
diff --git a/db/column_family.h b/db/column_family.h
index e4b7adde8..86637736a 100644
--- a/db/column_family.h
+++ b/db/column_family.h
@@ -401,6 +401,7 @@ class ColumnFamilyData {
SequenceNumber earliest_seq);
TableCache* table_cache() const { return table_cache_.get(); }
+ BlobFileCache* blob_file_cache() const { return blob_file_cache_.get(); }
BlobSource* blob_source() const { return blob_source_.get(); }
// See documentation in compaction_picker.h
diff --git a/db/db_impl/db_impl.cc b/db/db_impl/db_impl.cc
index 261593423..06573ac2e 100644
--- a/db/db_impl/db_impl.cc
+++ b/db/db_impl/db_impl.cc
@@ -659,8 +659,9 @@ Status DBImpl::CloseHelper() {
// We need to release them before the block cache is destroyed. The block
// cache may be destroyed inside versions_.reset(), when column family data
// list is destroyed, so leaving handles in table cache after
- // versions_.reset() may cause issues.
- // Here we clean all unreferenced handles in table cache.
+ // versions_.reset() may cause issues. Here we clean all unreferenced handles
+ // in table cache, and (for certain builds/conditions) assert that no obsolete
+ // files are hanging around unreferenced (leak) in the table/blob file cache.
// Now we assume all user queries have finished, so only version set itself
// can possibly hold the blocks from block cache. After releasing unreferenced
// handles here, only handles held by version set left and inside
@@ -668,6 +669,9 @@ Status DBImpl::CloseHelper() {
// time a handle is released, we erase it from the cache too. By doing that,
// we can guarantee that after versions_.reset(), table cache is empty
// so the cache can be safely destroyed.
+#ifndef NDEBUG
+ TEST_VerifyNoObsoleteFilesCached(/*db_mutex_already_held=*/true);
+#endif // !NDEBUG
table_cache_->EraseUnRefEntries();
for (auto& txn_entry : recovered_transactions_) {
@@ -3227,6 +3231,8 @@ Status DBImpl::MultiGetImpl(
s = Status::Aborted();
break;
}
+ // This could be a long-running operation
+ ROCKSDB_THREAD_YIELD_HOOK();
}
// Post processing (decrement reference counts and record statistics)
diff --git a/db/db_impl/db_impl.h b/db/db_impl/db_impl.h
index 5e4fa310b..ccc0abfa7 100644
--- a/db/db_impl/db_impl.h
+++ b/db/db_impl/db_impl.h
@@ -1241,9 +1241,14 @@ class DBImpl : public DB {
static Status TEST_ValidateOptions(const DBOptions& db_options) {
return ValidateOptions(db_options);
}
-
#endif // NDEBUG
+ // In certain configurations, verify that the table/blob file cache only
+ // contains entries for live files, to check for effective leaks of open
+ // files. This can only be called when purging of obsolete files has
+ // "settled," such as during parts of DB Close().
+ void TEST_VerifyNoObsoleteFilesCached(bool db_mutex_already_held) const;
+
// persist stats to column family "_persistent_stats"
void PersistStats();
diff --git a/db/db_impl/db_impl_debug.cc b/db/db_impl/db_impl_debug.cc
index 790a50d7a..67f5b4aaf 100644
--- a/db/db_impl/db_impl_debug.cc
+++ b/db/db_impl/db_impl_debug.cc
@@ -9,6 +9,7 @@
#ifndef NDEBUG
+#include "db/blob/blob_file_cache.h"
#include "db/column_family.h"
#include "db/db_impl/db_impl.h"
#include "db/error_handler.h"
@@ -328,5 +329,49 @@ size_t DBImpl::TEST_EstimateInMemoryStatsHistorySize() const {
InstrumentedMutexLock l(&const_cast<DBImpl*>(this)->stats_history_mutex_);
return EstimateInMemoryStatsHistorySize();
}
+
+void DBImpl::TEST_VerifyNoObsoleteFilesCached(
+ bool db_mutex_already_held) const {
+ // This check is somewhat expensive and obscure to make a part of every
+ // unit test in every build variety. Thus, we only enable it for ASAN builds.
+ if (!kMustFreeHeapAllocations) {
+ return;
+ }
+
+ std::optional<InstrumentedMutexLock> l;
+ if (db_mutex_already_held) {
+ mutex_.AssertHeld();
+ } else {
+ l.emplace(&mutex_);
+ }
+
+ std::vector<uint64_t> live_files;
+ for (auto cfd : *versions_->GetColumnFamilySet()) {
+ if (cfd->IsDropped()) {
+ continue;
+ }
+ // Sneakily add both SST and blob files to the same list
+ cfd->current()->AddLiveFiles(&live_files, &live_files);
+ }
+ std::sort(live_files.begin(), live_files.end());
+
+ auto fn = [&live_files](const Slice& key, Cache::ObjectPtr, size_t,
+ const Cache::CacheItemHelper* helper) {
+ if (helper != BlobFileCache::GetHelper()) {
+ // Skip non-blob files for now
+ // FIXME: diagnose and fix the leaks of obsolete SST files revealed in
+ // unit tests.
+ return;
+ }
+ // See TableCache and BlobFileCache
+ assert(key.size() == sizeof(uint64_t));
+ uint64_t file_number;
+ GetUnaligned(reinterpret_cast<const uint64_t*>(key.data()), &file_number);
+ // Assert file is in sorted live_files
+ assert(
+ std::binary_search(live_files.begin(), live_files.end(), file_number));
+ };
+ table_cache_->ApplyToAllEntries(fn, {});
+}
} // namespace ROCKSDB_NAMESPACE
#endif // NDEBUG
diff --git a/db/db_iter.cc b/db/db_iter.cc
index e02586377..bf4749eb9 100644
--- a/db/db_iter.cc
+++ b/db/db_iter.cc
@@ -540,6 +540,8 @@ bool DBIter::FindNextUserEntryInternal(bool skipping_saved_key,
} else {
iter_.Next();
}
+ // This could be a long-running operation due to tombstones, etc.
+ ROCKSDB_THREAD_YIELD_HOOK();
} while (iter_.Valid());
valid_ = false;
diff --git a/db/table_cache.cc b/db/table_cache.cc
index 71fc29c32..8a5be75e8 100644
--- a/db/table_cache.cc
+++ b/db/table_cache.cc
@@ -164,6 +164,7 @@ Status TableCache::GetTableReader(
}
Cache::Handle* TableCache::Lookup(Cache* cache, uint64_t file_number) {
+ // NOTE: sharing same Cache with BlobFileCache
Slice key = GetSliceForFileNumber(&file_number);
return cache->Lookup(key);
}
@@ -179,6 +180,7 @@ Status TableCache::FindTable(
size_t max_file_size_for_l0_meta_pin, Temperature file_temperature) {
PERF_TIMER_GUARD_WITH_CLOCK(find_table_nanos, ioptions_.clock);
uint64_t number = file_meta.fd.GetNumber();
+ // NOTE: sharing same Cache with BlobFileCache
Slice key = GetSliceForFileNumber(&number);
*handle = cache_.Lookup(key);
TEST_SYNC_POINT_CALLBACK("TableCache::FindTable:0",
diff --git a/db/version_builder.cc b/db/version_builder.cc
index ed8ab8214..c98f53f42 100644
--- a/db/version_builder.cc
+++ b/db/version_builder.cc
@@ -24,6 +24,7 @@
#include <vector>
#include "cache/cache_reservation_manager.h"
+#include "db/blob/blob_file_cache.h"
#include "db/blob/blob_file_meta.h"
#include "db/dbformat.h"
#include "db/internal_stats.h"
@@ -744,12 +745,9 @@ class VersionBuilder::Rep {
return Status::Corruption("VersionBuilder", oss.str());
}
- // Note: we use C++11 for now but in C++14, this could be done in a more
- // elegant way using generalized lambda capture.
- VersionSet* const vs = version_set_;
- const ImmutableCFOptions* const ioptions = ioptions_;
-
- auto deleter = [vs, ioptions](SharedBlobFileMetaData* shared_meta) {
+ auto deleter = [vs = version_set_, ioptions = ioptions_,
+ bc = cfd_ ? cfd_->blob_file_cache()
+ : nullptr](SharedBlobFileMetaData* shared_meta) {
if (vs) {
assert(ioptions);
assert(!ioptions->cf_paths.empty());
@@ -758,6 +756,9 @@ class VersionBuilder::Rep {
vs->AddObsoleteBlobFile(shared_meta->GetBlobFileNumber(),
ioptions->cf_paths.front().path);
}
+ if (bc) {
+ bc->Evict(shared_meta->GetBlobFileNumber());
+ }
delete shared_meta;
};
@@ -766,7 +767,7 @@ class VersionBuilder::Rep {
blob_file_number, blob_file_addition.GetTotalBlobCount(),
blob_file_addition.GetTotalBlobBytes(),
blob_file_addition.GetChecksumMethod(),
- blob_file_addition.GetChecksumValue(), deleter);
+ blob_file_addition.GetChecksumValue(), std::move(deleter));
mutable_blob_file_metas_.emplace(
blob_file_number, MutableBlobFileMetaData(std::move(shared_meta)));
diff --git a/db/version_set.h b/db/version_set.h
index 9336782b1..024f869e7 100644
--- a/db/version_set.h
+++ b/db/version_set.h
@@ -1514,7 +1514,6 @@ class VersionSet {
void GetLiveFilesMetaData(std::vector<LiveFileMetaData>* metadata);
void AddObsoleteBlobFile(uint64_t blob_file_number, std::string path) {
- // TODO: Erase file from BlobFileCache?
obsolete_blob_files_.emplace_back(blob_file_number, std::move(path));
}
diff --git a/include/rocksdb/version.h b/include/rocksdb/version.h
index 2a19796b8..0afa2cab1 100644
--- a/include/rocksdb/version.h
+++ b/include/rocksdb/version.h
@@ -13,7 +13,7 @@
// minor or major version number planned for release.
#define ROCKSDB_MAJOR 9
#define ROCKSDB_MINOR 7
-#define ROCKSDB_PATCH 3
+#define ROCKSDB_PATCH 4
// Do not use these. We made the mistake of declaring macros starting with
// double underscore. Now we have to live with our choice. We'll deprecate these
diff --git a/port/port.h b/port/port.h
index 13aa56d47..141716e5b 100644
--- a/port/port.h
+++ b/port/port.h
@@ -19,3 +19,19 @@
#elif defined(OS_WIN)
#include "port/win/port_win.h"
#endif
+
+#ifdef OS_LINUX
+// A temporary hook into long-running RocksDB threads to support modifying their
+// priority etc. This should become a public API hook once the requirements
+// are better understood.
+extern "C" void RocksDbThreadYield() __attribute__((__weak__));
+#define ROCKSDB_THREAD_YIELD_HOOK() \
+ { \
+ if (RocksDbThreadYield) { \
+ RocksDbThreadYield(); \
+ } \
+ }
+#else
+#define ROCKSDB_THREAD_YIELD_HOOK() \
+ {}
+#endif

View File

@@ -0,0 +1,30 @@
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 93b884d..b715cb6 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -106,14 +106,9 @@ endif()
include(CMakeDependentOption)
if(MSVC)
- option(WITH_GFLAGS "build with GFlags" OFF)
option(WITH_XPRESS "build with windows built in compression" OFF)
- option(ROCKSDB_SKIP_THIRDPARTY "skip thirdparty.inc" OFF)
-
- if(NOT ROCKSDB_SKIP_THIRDPARTY)
- include(${CMAKE_CURRENT_SOURCE_DIR}/thirdparty.inc)
- endif()
-else()
+endif()
+if(TRUE)
if(CMAKE_SYSTEM_NAME MATCHES "FreeBSD" AND NOT CMAKE_SYSTEM_NAME MATCHES "kFreeBSD")
# FreeBSD has jemalloc as default malloc
# but it does not have all the jemalloc files in include/...
@@ -126,7 +121,7 @@ else()
endif()
endif()
- if(MINGW)
+ if(MSVC OR MINGW)
option(WITH_GFLAGS "build with GFlags" OFF)
else()
option(WITH_GFLAGS "build with GFlags" ON)

View File

@@ -27,7 +27,6 @@
#include <algorithm>
#include <optional>
#include <ostream>
#include <string>
#include <unordered_map>
#include <vector>

View File

@@ -22,10 +22,9 @@
#include <xrpl/basics/Slice.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <cstdint>
#include <cstring>
#include <memory>
#include <utility>
namespace ripple {

View File

@@ -21,9 +21,11 @@
#define RIPPLED_COMPRESSIONALGORITHMS_H_INCLUDED
#include <xrpl/basics/contract.h>
#include <lz4.h>
#include <algorithm>
#include <cstdint>
#include <lz4.h>
#include <stdexcept>
#include <vector>

View File

@@ -21,6 +21,7 @@
#define RIPPLE_BASICS_COUNTEDOBJECT_H_INCLUDED
#include <xrpl/beast/type_name.h>
#include <atomic>
#include <string>
#include <utility>

View File

@@ -24,9 +24,7 @@
#include <boost/outcome.hpp>
#include <concepts>
#include <stdexcept>
#include <type_traits>
namespace ripple {

View File

@@ -21,6 +21,7 @@
#define RIPPLE_BASICS_LOCALVALUE_H_INCLUDED
#include <boost/thread/tss.hpp>
#include <memory>
#include <unordered_map>

View File

@@ -22,8 +22,10 @@
#include <xrpl/basics/UnorderedContainers.h>
#include <xrpl/beast/utility/Journal.h>
#include <boost/beast/core/string.hpp>
#include <boost/filesystem.hpp>
#include <map>
#include <memory>
#include <mutex>

View File

@@ -20,11 +20,11 @@
#ifndef RIPPLE_BASICS_RESOLVER_H_INCLUDED
#define RIPPLE_BASICS_RESOLVER_H_INCLUDED
#include <xrpl/beast/net/IPEndpoint.h>
#include <functional>
#include <vector>
#include <xrpl/beast/net/IPEndpoint.h>
namespace ripple {
class Resolver

View File

@@ -22,6 +22,7 @@
#include <xrpl/basics/Resolver.h>
#include <xrpl/beast/utility/Journal.h>
#include <boost/asio/io_service.hpp>
namespace ripple {

View File

@@ -23,6 +23,7 @@
#include <xrpl/basics/contract.h>
#include <xrpl/basics/strHex.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <algorithm>
#include <array>
#include <cstdint>

View File

@@ -25,6 +25,7 @@
#include <xrpl/basics/hardened_hash.h>
#include <xrpl/beast/clock/abstract_clock.h>
#include <xrpl/beast/insight/Insight.h>
#include <atomic>
#include <functional>
#include <mutex>

View File

@@ -25,6 +25,7 @@
#include <xrpl/beast/hash/hash_append.h>
#include <xrpl/beast/hash/uhash.h>
#include <xrpl/beast/hash/xxhasher.h>
#include <unordered_map>
#include <unordered_set>

View File

@@ -33,12 +33,13 @@
#include <xrpl/basics/strHex.h>
#include <xrpl/beast/utility/Zero.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <boost/endian/conversion.hpp>
#include <boost/functional/hash.hpp>
#include <algorithm>
#include <array>
#include <cstring>
#include <functional>
#include <type_traits>
namespace ripple {

View File

@@ -20,17 +20,16 @@
#ifndef RIPPLE_BASICS_CHRONO_H_INCLUDED
#define RIPPLE_BASICS_CHRONO_H_INCLUDED
#include <date/date.h>
#include <xrpl/beast/clock/abstract_clock.h>
#include <xrpl/beast/clock/basic_seconds_clock.h>
#include <xrpl/beast/clock/manual_clock.h>
#include <date/date.h>
#include <chrono>
#include <cstdint>
#include <ratio>
#include <string>
#include <type_traits>
namespace ripple {

View File

@@ -21,9 +21,9 @@
#define RIPPLE_BASICS_CONTRACT_H_INCLUDED
#include <xrpl/beast/type_name.h>
#include <exception>
#include <string>
#include <typeinfo>
#include <utility>
namespace ripple {

View File

@@ -21,6 +21,7 @@
#define RIPPLE_BASICS_MAKE_SSLCONTEXT_H_INCLUDED
#include <boost/asio/ssl/context.hpp>
#include <string>
namespace ripple {

View File

@@ -22,6 +22,7 @@
#include <xrpl/beast/hash/uhash.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <functional>
#include <optional>
#include <string>

View File

@@ -22,9 +22,9 @@
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/beast/xor_shift_engine.h>
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <limits>
#include <mutex>
#include <random>

View File

@@ -19,6 +19,7 @@
#define RIPPLE_BASICS_SPINLOCK_H_INCLUDED
#include <xrpl/beast/utility/instrumentation.h>
#include <atomic>
#include <limits>
#include <type_traits>

View File

@@ -21,7 +21,9 @@
#define BEAST_UTILITY_TAGGED_INTEGER_H_INCLUDED
#include <xrpl/beast/hash/hash_append.h>
#include <boost/operators.hpp>
#include <functional>
#include <iostream>
#include <type_traits>

View File

@@ -21,6 +21,7 @@
#define BEAST_ASIO_IO_LATENCY_PROBE_H_INCLUDED
#include <xrpl/beast/utility/instrumentation.h>
#include <boost/asio/basic_waitable_timer.hpp>
#include <boost/asio/io_service.hpp>

View File

@@ -25,9 +25,11 @@
#include <xrpl/beast/container/detail/aged_associative_container.h>
#include <xrpl/beast/container/detail/aged_container_iterator.h>
#include <xrpl/beast/container/detail/empty_base_optimization.h>
#include <boost/intrusive/list.hpp>
#include <boost/intrusive/set.hpp>
#include <boost/version.hpp>
#include <algorithm>
#include <functional>
#include <initializer_list>

View File

@@ -25,8 +25,10 @@
#include <xrpl/beast/container/detail/aged_associative_container.h>
#include <xrpl/beast/container/detail/aged_container_iterator.h>
#include <xrpl/beast/container/detail/empty_base_optimization.h>
#include <boost/intrusive/list.hpp>
#include <boost/intrusive/unordered_set.hpp>
#include <algorithm>
#include <cmath>
#include <functional>

View File

@@ -11,6 +11,7 @@
#define BEAST_CONTAINER_DETAIL_EMPTY_BASE_OPTIMIZATION_H_INCLUDED
#include <boost/type_traits/is_final.hpp>
#include <type_traits>
#include <utility>

View File

@@ -23,6 +23,7 @@
#include <xrpl/beast/utility/instrumentation.h>
#include <boost/core/detail/string_view.hpp>
#include <algorithm>
#include <cerrno>
#include <charconv>

View File

@@ -21,7 +21,6 @@
#define BEAST_INTRUSIVE_LIST_H_INCLUDED
#include <iterator>
#include <type_traits>
namespace beast {

View File

@@ -23,6 +23,7 @@
#include <boost/container/flat_set.hpp>
#include <boost/endian/conversion.hpp>
#include <array>
#include <chrono>
#include <cstdint>

View File

@@ -21,6 +21,7 @@
#define BEAST_HASH_XXHASHER_H_INCLUDED
#include <boost/endian/conversion.hpp>
#include <xxhash.h>
#include <cstddef>

View File

@@ -20,10 +20,10 @@
#ifndef BEAST_INSIGHT_METER_H_INCLUDED
#define BEAST_INSIGHT_METER_H_INCLUDED
#include <memory>
#include <xrpl/beast/insight/MeterImpl.h>
#include <memory>
namespace beast {
namespace insight {

View File

@@ -21,7 +21,6 @@
#define BEAST_INSIGHT_STATSDCOLLECTOR_H_INCLUDED
#include <xrpl/beast/insight/Collector.h>
#include <xrpl/beast/net/IPEndpoint.h>
#include <xrpl/beast/utility/Journal.h>

View File

@@ -25,8 +25,10 @@
#include <xrpl/beast/net/IPAddressV4.h>
#include <xrpl/beast/net/IPAddressV6.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <boost/asio/ip/address.hpp>
#include <boost/functional/hash.hpp>
#include <cstdint>
#include <ios>
#include <sstream>

View File

@@ -22,8 +22,6 @@
#include <xrpl/beast/net/IPEndpoint.h>
#include <sstream>
#include <boost/asio.hpp>
namespace beast {

View File

@@ -21,7 +21,9 @@
#define BEAST_NET_IPADDRESSV4_H_INCLUDED
#include <xrpl/beast/hash/hash_append.h>
#include <boost/asio/ip/address_v4.hpp>
#include <cstdint>
#include <functional>
#include <ios>

View File

@@ -21,7 +21,9 @@
#define BEAST_NET_IPADDRESSV6_H_INCLUDED
#include <xrpl/beast/utility/instrumentation.h>
#include <boost/asio/ip/address_v6.hpp>
#include <cstdint>
#include <functional>
#include <ios>

View File

@@ -20,10 +20,6 @@
#ifndef BEAST_RFC2616_HPP
#define BEAST_RFC2616_HPP
// TODO: This include <cstdint> is a workaround for beast compilation bug.
// Remove when fix https://github.com/boostorg/beast/pull/2682/ is available.
#include <cstdint>
#include <boost/beast/http/message.hpp>
#include <boost/beast/http/rfc7230.hpp>
#include <boost/range/algorithm/equal.hpp>
@@ -32,6 +28,7 @@
#include <algorithm>
#include <cctype>
#include <cstdint>
#include <iterator>
#include <string>
#include <utility>

View File

@@ -11,6 +11,7 @@
#include <boost/asio/io_service.hpp>
#include <boost/asio/spawn.hpp>
#include <boost/optional.hpp>
#include <condition_variable>
#include <functional>
#include <mutex>

View File

@@ -9,6 +9,7 @@
#define BEAST_UNIT_TEST_MATCH_HPP
#include <xrpl/beast/unit_test/suite_info.h>
#include <string>
namespace beast {

View File

@@ -10,8 +10,10 @@
#include <xrpl/beast/unit_test/amount.h>
#include <xrpl/beast/unit_test/recorder.h>
#include <boost/lexical_cast.hpp>
#include <boost/optional.hpp>
#include <algorithm>
#include <chrono>
#include <functional>

View File

@@ -9,7 +9,9 @@
#define BEAST_UNIT_TEST_RUNNER_H_INCLUDED
#include <xrpl/beast/unit_test/suite_info.h>
#include <boost/assert.hpp>
#include <mutex>
#include <ostream>
#include <string>

View File

@@ -9,9 +9,11 @@
#define BEAST_UNIT_TEST_SUITE_HPP
#include <xrpl/beast/unit_test/runner.h>
#include <boost/filesystem.hpp>
#include <boost/lexical_cast.hpp>
#include <boost/throw_exception.hpp>
#include <ostream>
#include <sstream>
#include <string>

View File

@@ -10,7 +10,9 @@
#include <xrpl/beast/unit_test/detail/const_container.h>
#include <xrpl/beast/unit_test/suite_info.h>
#include <boost/assert.hpp>
#include <set>
#include <typeindex>
#include <unordered_set>

View File

@@ -9,6 +9,7 @@
#define BEAST_UNIT_TEST_THREAD_HPP
#include <xrpl/beast/unit_test/suite.h>
#include <functional>
#include <thread>
#include <utility>

View File

@@ -21,6 +21,7 @@
#define BEAST_UTILITY_JOURNAL_H_INCLUDED
#include <xrpl/beast/utility/instrumentation.h>
#include <sstream>
namespace beast {

View File

@@ -22,11 +22,9 @@
#include <xrpl/beast/core/List.h>
#include <cstdint>
#include <mutex>
#include <sstream>
#include <string>
#include <utility>
namespace beast {

View File

@@ -21,6 +21,7 @@
#define BEAST_RANDOM_RNGFILL_H_INCLUDED
#include <xrpl/beast/utility/instrumentation.h>
#include <array>
#include <cstdint>
#include <cstring>

View File

@@ -21,6 +21,7 @@
#define BEAST_UTILITY_TEMP_DIR_H_INCLUDED
#include <boost/filesystem.hpp>
#include <string>
namespace beast {

View File

@@ -21,8 +21,6 @@
#define RIPPLE_CRYPTO_RANDOM_H_INCLUDED
#include <mutex>
#include <string>
#include <type_traits>
namespace ripple {

View File

@@ -21,6 +21,7 @@
#define RIPPLE_JSON_OBJECT_H_INCLUDED
#include <xrpl/json/Writer.h>
#include <memory>
namespace Json {

View File

@@ -21,6 +21,7 @@
#define RIPPLE_JSON_OUTPUT_H_INCLUDED
#include <boost/beast/core/string.hpp>
#include <functional>
#include <string>

View File

@@ -24,6 +24,7 @@
#include <xrpl/basics/contract.h>
#include <xrpl/json/Output.h>
#include <xrpl/json/json_value.h>
#include <memory>
namespace Json {

View File

@@ -20,6 +20,7 @@
#ifndef RIPPLE_JSON_JSON_ASSERT_H_INCLUDED
#define RIPPLE_JSON_JSON_ASSERT_H_INCLUDED
#include <xrpl/basics/contract.h>
#include <xrpl/json/json_errors.h>
#define JSON_ASSERT_MESSAGE(condition, message) \

View File

@@ -24,7 +24,9 @@
#include <xrpl/json/json_forwards.h>
#include <xrpl/json/json_value.h>
#include <boost/asio/buffer.hpp>
#include <stack>
namespace Json {

View File

@@ -21,6 +21,7 @@
#define RIPPLE_JSON_JSON_VALUE_H_INCLUDED
#include <xrpl/json/json_forwards.h>
#include <cstring>
#include <map>
#include <string>

View File

@@ -22,6 +22,7 @@
#include <xrpl/json/json_forwards.h>
#include <xrpl/json/json_value.h>
#include <ostream>
#include <vector>

View File

@@ -22,6 +22,7 @@
#include <xrpl/basics/CountedObject.h>
#include <xrpl/protocol/Issue.h>
#include <boost/utility/base_from_member.hpp>
namespace ripple {

View File

@@ -21,7 +21,9 @@
#define RIPPLE_PROTOCOL_FEATURE_H_INCLUDED
#include <xrpl/basics/base_uint.h>
#include <boost/container/flat_map.hpp>
#include <array>
#include <bitset>
#include <map>
@@ -33,35 +35,39 @@
*
* Steps required to add new features to the code:
*
* 1) In this file, increment `numFeatures` and add a uint256 declaration
* for the feature at the bottom
* 2) Add a uint256 definition for the feature to the corresponding source
* file (Feature.cpp). Use `registerFeature` to create the feature with
* the feature's name, `Supported::no`, and `VoteBehavior::DefaultNo`. This
* should be the only place the feature's name appears in code as a string.
* 3) Use the uint256 as the parameter to `view.rules.enabled()` to
* control flow into new code that this feature limits.
* 4) If the feature development is COMPLETE, and the feature is ready to be
* SUPPORTED, change the `registerFeature` parameter to Supported::yes.
* 5) When the feature is ready to be ENABLED, change the `registerFeature`
* parameter to `VoteBehavior::DefaultYes`.
* In general, any newly supported amendments (`Supported::yes`) should have
* a `VoteBehavior::DefaultNo` for at least one full release cycle. High
* priority bug fixes can be an exception to this rule of thumb.
* 1) Add the appropriate XRPL_FEATURE or XRPL_FIX macro definition for the
* feature to features.macro with the feature's name, `Supported::no`, and
* `VoteBehavior::DefaultNo`.
*
* 2) Use the generated variable name as the parameter to `view.rules.enabled()`
* to control flow into new code that this feature limits. (featureName or
* fixName)
*
* 3) If the feature development is COMPLETE, and the feature is ready to be
* SUPPORTED, change the macro parameter in features.macro to Supported::yes.
*
* 4) In general, any newly supported amendments (`Supported::yes`) should have
* a `VoteBehavior::DefaultNo` indefinitely so that external governance can
* make the decision on when to activate it. High priority bug fixes can be
* an exception to this rule. In such cases, ensure the fix has been
* clearly communicated to the community using appropriate channels,
* then change the macro parameter in features.macro to
* `VoteBehavior::DefaultYes`. The communication process is beyond
* the scope of these instructions.
*
*
* When a feature has been enabled for several years, the conditional code
* may be removed, and the feature "retired". To retire a feature:
* 1) Remove the uint256 declaration from this file.
* 2) MOVE the uint256 definition in Feature.cpp to the "retired features"
* section at the end of the file.
* 3) CHANGE the name of the variable to start with "retired".
* 4) CHANGE the parameters of the `registerFeature` call to `Supported::yes`
* and `VoteBehavior::DefaultNo`.
*
* 1) MOVE the macro definition in features.macro to the "retired features"
* section at the end of the file, and change the macro to XRPL_RETIRE.
*
* The feature must remain registered and supported indefinitely because it
* still exists in the ledger, but there is no need to vote for it because
* there's nothing to vote for. If it is removed completely from the code, any
* instances running that code will get amendment blocked. Removing the
* feature from the ledger is beyond the scope of these instructions.
* may exist in the Amendments object on ledger. There is no need to vote
* for it because there's nothing to vote for. If the feature definition is
* removed completely from the code, any instances running that code will get
* amendment blocked. Removing the feature from the ledger is beyond the scope
* of these instructions.
*
*/
@@ -76,11 +82,32 @@ allAmendments();
namespace detail {
#pragma push_macro("XRPL_FEATURE")
#undef XRPL_FEATURE
#pragma push_macro("XRPL_FIX")
#undef XRPL_FIX
#pragma push_macro("XRPL_RETIRE")
#undef XRPL_RETIRE
#define XRPL_FEATURE(name, supported, vote) +1
#define XRPL_FIX(name, supported, vote) +1
#define XRPL_RETIRE(name) +1
// This value SHOULD be equal to the number of amendments registered in
// Feature.cpp. Because it's only used to reserve storage, and determine how
// large to make the FeatureBitset, it MAY be larger. It MUST NOT be less than
// the actual number of amendments. A LogicError on startup will verify this.
static constexpr std::size_t numFeatures = 89;
static constexpr std::size_t numFeatures =
(0 +
#include <xrpl/protocol/detail/features.macro>
);
#undef XRPL_RETIRE
#pragma pop_macro("XRPL_RETIRE")
#undef XRPL_FIX
#pragma pop_macro("XRPL_FIX")
#undef XRPL_FEATURE
#pragma pop_macro("XRPL_FEATURE")
/** Amendments that this server supports and the default voting behavior.
Whether they are enabled depends on the Rules defined in the validated
@@ -320,12 +347,17 @@ foreachFeature(FeatureBitset bs, F&& f)
#undef XRPL_FEATURE
#pragma push_macro("XRPL_FIX")
#undef XRPL_FIX
#pragma push_macro("XRPL_RETIRE")
#undef XRPL_RETIRE
#define XRPL_FEATURE(name, supported, vote) extern uint256 const feature##name;
#define XRPL_FIX(name, supported, vote) extern uint256 const fix##name;
#define XRPL_RETIRE(name)
#include <xrpl/protocol/detail/features.macro>
#undef XRPL_RETIRE
#pragma pop_macro("XRPL_RETIRE")
#undef XRPL_FIX
#pragma pop_macro("XRPL_FIX")
#undef XRPL_FEATURE

View File

@@ -23,6 +23,7 @@
#include <xrpl/beast/utility/Zero.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/json/json_value.h>
#include <boost/multiprecision/cpp_int.hpp>
#include <boost/operators.hpp>

View File

@@ -21,6 +21,7 @@
#define RIPPLE_PROTOCOL_HASHPREFIX_H_INCLUDED
#include <xrpl/beast/hash/hash_append.h>
#include <cstdint>
namespace ripple {

View File

@@ -23,7 +23,9 @@
#include <xrpl/basics/LocalValue.h>
#include <xrpl/basics/Number.h>
#include <xrpl/beast/utility/Zero.h>
#include <boost/operators.hpp>
#include <cstdint>
#include <string>
#include <utility>

View File

@@ -23,7 +23,9 @@
#include <xrpl/basics/contract.h>
#include <xrpl/beast/type_name.h>
#include <xrpl/protocol/SOTemplate.h>
#include <boost/container/flat_map.hpp>
#include <algorithm>
#include <forward_list>

View File

@@ -20,10 +20,10 @@
#ifndef RIPPLE_JSON_MULTIAPIJSON_H_INCLUDED
#define RIPPLE_JSON_MULTIAPIJSON_H_INCLUDED
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/json/json_value.h>
#include <xrpl/protocol/ApiVersion.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <array>
#include <concepts>
#include <cstdlib>

View File

@@ -23,6 +23,7 @@
#include <xrpl/basics/ByteUtilities.h>
#include <xrpl/basics/base_uint.h>
#include <xrpl/basics/partitioned_unordered_map.h>
#include <cstdint>
namespace ripple {

View File

@@ -22,7 +22,9 @@
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/protocol/STAmount.h>
#include <boost/operators.hpp>
#include <cstdint>
#include <ostream>

View File

@@ -22,6 +22,7 @@
#include <xrpl/basics/contract.h>
#include <xrpl/protocol/SField.h>
#include <functional>
#include <initializer_list>
#include <memory>

View File

@@ -23,12 +23,14 @@
#include <xrpl/basics/contract.h>
#include <xrpl/protocol/SField.h>
#include <xrpl/protocol/Serializer.h>
#include <memory>
#include <ostream>
#include <string>
#include <type_traits>
#include <typeinfo>
#include <utility>
namespace ripple {
/// Note, should be treated as flags that can be | and &

View File

@@ -28,6 +28,7 @@
#include <xrpl/protocol/STBlob.h>
#include <xrpl/protocol/STInteger.h>
#include <xrpl/protocol/STObject.h>
#include <memory>
#include <optional>
#include <stdexcept>

View File

@@ -124,4 +124,4 @@ STLedgerEntry::getType() const
} // namespace ripple
#endif
#endif

View File

@@ -35,7 +35,9 @@
#include <xrpl/protocol/STPathSet.h>
#include <xrpl/protocol/STVector256.h>
#include <xrpl/protocol/detail/STVar.h>
#include <boost/iterator/transform_iterator.hpp>
#include <optional>
#include <stdexcept>
#include <type_traits>

View File

@@ -21,6 +21,7 @@
#define RIPPLE_PROTOCOL_STPARSEDJSON_H_INCLUDED
#include <xrpl/protocol/STArray.h>
#include <optional>
namespace ripple {

View File

@@ -26,6 +26,7 @@
#include <xrpl/protocol/SField.h>
#include <xrpl/protocol/STBase.h>
#include <xrpl/protocol/UintTypes.h>
#include <cstddef>
#include <optional>

View File

@@ -28,6 +28,7 @@
#include <xrpl/protocol/SecretKey.h>
#include <xrpl/protocol/SeqProxy.h>
#include <xrpl/protocol/TxFormats.h>
#include <boost/container/flat_set.hpp>
#include <functional>

View File

@@ -26,6 +26,7 @@
#include <xrpl/protocol/PublicKey.h>
#include <xrpl/protocol/STObject.h>
#include <xrpl/protocol/SecretKey.h>
#include <cstdint>
#include <functional>
#include <memory>
@@ -146,8 +147,7 @@ public:
render() const
{
std::stringstream ss;
ss << "validation: "
<< " ledger_hash: " << getLedgerHash()
ss << "validation: " << " ledger_hash: " << getLedgerHash()
<< " consensus_hash: " << getConsensusHash()
<< " sign_time: " << to_string(getSignTime())
<< " seen_time: " << to_string(getSeenTime())

View File

@@ -26,6 +26,7 @@
#include <xrpl/protocol/PublicKey.h>
#include <xrpl/protocol/Seed.h>
#include <xrpl/protocol/tokens.h>
#include <array>
#include <cstring>
#include <string>

View File

@@ -23,6 +23,7 @@
#include <xrpl/basics/Slice.h>
#include <xrpl/basics/base_uint.h>
#include <xrpl/protocol/tokens.h>
#include <array>
#include <optional>

View File

@@ -30,6 +30,7 @@
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/protocol/HashPrefix.h>
#include <xrpl/protocol/SField.h>
#include <cstdint>
#include <cstring>
#include <iomanip>

View File

@@ -24,6 +24,7 @@
#include <xrpl/protocol/PublicKey.h>
#include <xrpl/protocol/STObject.h>
#include <xrpl/protocol/SecretKey.h>
#include <utility>
namespace ripple {

View File

@@ -22,6 +22,7 @@
#include <xrpl/basics/chrono.h>
#include <xrpl/protocol/XRPAmount.h>
#include <cstdint>
#include <string>

View File

@@ -42,7 +42,7 @@ using TERUnderlyingType = int;
enum TELcodes : TERUnderlyingType {
// Note: Range is stable.
// Exact numbers are used in ripple-binary-codec:
// https://github.com/ripple/ripple-binary-codec/blob/master/src/enums/definitions.json
// https://github.com/XRPLF/xrpl.js/blob/main/packages/ripple-binary-codec/src/enums/definitions.json
// Use tokens.
// -399 .. -300: L Local error (transaction fee inadequate, exceeds local
@@ -73,7 +73,7 @@ enum TELcodes : TERUnderlyingType {
enum TEMcodes : TERUnderlyingType {
// Note: Range is stable.
// Exact numbers are used in ripple-binary-codec:
// https://github.com/ripple/ripple-binary-codec/blob/master/src/enums/definitions.json
// https://github.com/XRPLF/xrpl.js/blob/main/packages/ripple-binary-codec/src/enums/definitions.json
// Use tokens.
// -299 .. -200: M Malformed (bad signature)
@@ -148,7 +148,7 @@ enum TEMcodes : TERUnderlyingType {
enum TEFcodes : TERUnderlyingType {
// Note: Range is stable.
// Exact numbers are used in ripple-binary-codec:
// https://github.com/ripple/ripple-binary-codec/blob/master/src/enums/definitions.json
// https://github.com/XRPLF/xrpl.js/blob/main/packages/ripple-binary-codec/src/enums/definitions.json
// Use tokens.
// -199 .. -100: F
@@ -192,7 +192,7 @@ enum TEFcodes : TERUnderlyingType {
enum TERcodes : TERUnderlyingType {
// Note: Range is stable.
// Exact numbers are used in ripple-binary-codec:
// https://github.com/ripple/ripple-binary-codec/blob/master/src/enums/definitions.json
// https://github.com/XRPLF/xrpl.js/blob/main/packages/ripple-binary-codec/src/enums/definitions.json
// Use tokens.
// -99 .. -1: R Retry

View File

@@ -20,10 +20,10 @@
#ifndef RIPPLE_PROTOCOL_TXFLAGS_H_INCLUDED
#define RIPPLE_PROTOCOL_TXFLAGS_H_INCLUDED
#include <cstdint>
#include <xrpl/protocol/LedgerFormats.h>
#include <cstdint>
namespace ripple {
/** Transaction flags.

View File

@@ -24,7 +24,9 @@
#include <xrpl/protocol/STArray.h>
#include <xrpl/protocol/STLedgerEntry.h>
#include <xrpl/protocol/TER.h>
#include <boost/container/flat_set.hpp>
#include <optional>
namespace ripple {

View File

@@ -23,12 +23,9 @@
#include <xrpl/protocol/SField.h>
#include <xrpl/protocol/STBase.h>
#include <xrpl/protocol/Serializer.h>
#include <concepts>
#include <cstddef>
#include <cstdint>
#include <type_traits>
#include <typeinfo>
#include <utility>
namespace ripple {
namespace detail {

View File

@@ -23,6 +23,9 @@
#if !defined(XRPL_FIX)
#error "undefined macro: XRPL_FIX"
#endif
#if !defined(XRPL_RETIRE)
#error "undefined macro: XRPL_RETIRE"
#endif
// Add new amendments to the top of this list.
// Keep it sorted in reverse chronological order.
@@ -121,3 +124,22 @@ XRPL_FIX (NFTokenDirV1, Supported::yes, VoteBehavior::Obsolete)
XRPL_FEATURE(NonFungibleTokensV1, Supported::yes, VoteBehavior::Obsolete)
XRPL_FEATURE(CryptoConditionsSuite, Supported::yes, VoteBehavior::Obsolete)
// The following amendments have been active for at least two years. Their
// pre-amendment code has been removed and the identifiers are deprecated.
// All known amendments and amendments that may appear in a validated
// ledger must be registered either here or above with the "active" amendments
XRPL_RETIRE(MultiSign)
XRPL_RETIRE(TrustSetAuth)
XRPL_RETIRE(FeeEscalation)
XRPL_RETIRE(PayChan)
XRPL_RETIRE(CryptoConditions)
XRPL_RETIRE(TickSize)
XRPL_RETIRE(fix1368)
XRPL_RETIRE(Escrow)
XRPL_RETIRE(fix1373)
XRPL_RETIRE(EnforceInvariants)
XRPL_RETIRE(SortedDirectories)
XRPL_RETIRE(fix1201)
XRPL_RETIRE(fix1512)
XRPL_RETIRE(fix1523)
XRPL_RETIRE(fix1528)

Some files were not shown because too many files have changed in this diff Show More