Merge branch 'develop' into ximinez/fix/validator-cache

- If any [validator_list_keys] are not available after all
  [validator_list_sites] have had a chance to be queried, then fall
  back to loading cache files. Currently, cache files are only used if
  no sites are defined, or the request to one of them has an error. It
  does not include cases where not enough sites are defined, or if a
  site returns an invalid VL (or something else entirely).
- Resolves #5320

.claude/CLAUDE.md

@@ -1,54 +0,0 @@
-# Workflow Orchestration
-
-## 1. Plan Mode Default
-- Enter plan mode for ANY non-trivial task (3+ steps or architectural decisions)
-- If something goes sideways, STOP and re-plan immediately - don't keep pushing
-- Use plan mode for verification steps, not just building
-- Write detailed specs upfront to reduce ambiguity
-
-## 2. Subagent Strategy
-- Use subagents liberally to keep main context window clean
-- Offload research, exploration, and parallel analysis to subagents
-- For complex problems, throw compute at it via parallelizing
-- One task per subagent for focused execution
-
-## 3. Self-Improvement Loop
-- After ANY correction from the user: update `tasks/lessons.md` with the pattern
-- Write rules for yourself that prevent the same mistake
-- Ruthlessly iterate on these rules until mistake drops
-- Review lessons at session start for relevant project
-
-## 4. Verification Before Done
-- Never mark a task complete without proving it works
-- Diff behavior between main and your changes when relevant
-- Ask yourself: "Would a staff engineer approve this?"
-- Run tests, check logs, demonstrate correctness
-
-## 5. Demand Elegance (Balanced)
-- For non-trivial changes: pause and ask "is there a more elegant way?"
-- If a fix feels hacky: "Know now, do later, implement the elegant solution"
-- Skip this for simple, obvious fixes - don't over-engineer
-- Challenge your own first solution
-
-## 6. Autonomous Bug Fixing
-- When given a bug report: just fix it. Don't ask for hand-holding
-- Point at logs, errors, failing tests - then resolve them
-- Zero context switching required from the user
-- Go fix failing CI tests without being told how
-
-# Task Management
-
-1. **Plan First**: Write plan to `tasks/todo.md` with checkable items
-2. **Verify Plans**: Check in before starting implementation
-3. **Track Progress**: Mark items complete as you go
-4. **Explain Changes**: High-level summary at each step
-5. **Document Results**: Add review section to `tasks/todo.md`
-6. **Capture Lessons**: Update `tasks/lessons.md` after corrections
-
-# Core Principles
-
-**Simplicity First**: Make every change as simple as possible. Impact minimal code.
-
-**No Boilerplate**: Skip documentation. No README/docs bloat. Test only if requested.
-
-**Minimal Impact**: Changes should only touch what's necessary. Avoid introducing bugs.

.claude/skills/amendment.md

@@ -1,530 +0,0 @@
-# XRPL Amendment Creation Skill
-
-This skill guides you through creating XRPL amendments, whether for brand new features or fixes/extensions to existing functionality.
-
-## Amendment Types
-
-There are two main types of amendments:
-
-1. **New Feature Amendment** (`feature{Name}`) - Entirely new functionality
-2. **Fix/Extension Amendment** (`fix{Name}`) - Modifications to existing functionality
-
-## Step 1: Determine Amendment Type
-
-Ask the user:
-- Is this a **brand new feature** (new transaction type, ledger entry, or capability)?
-- Or is this a **fix or extension** to existing functionality?
-
----
-
-## For NEW FEATURE Amendments
-
-### Checklist:
-
-#### 1. Feature Definition in features.macro
-**ONLY FILE TO EDIT:** `include/xrpl/protocol/detail/features.macro`
-
-- [ ] Add to TOP of features.macro (reverse chronological order):
-```
-XRPL_FEATURE(YourFeatureName, Supported::no, VoteBehavior::DefaultNo)
-```
-- [ ] This creates the variable `featureYourFeatureName` automatically
-- [ ] Follow naming convention: Use the feature name WITHOUT the "feature" prefix
-- [ ] Examples: `Batch` → `featureBatch`, `LendingProtocol` → `featureLendingProtocol`
-
-#### 2. Support Status
-- [ ] Start with `Supported::no` during development
-- [ ] Change to `Supported::yes` when ready for network voting
-- [ ] Use `VoteBehavior::DefaultNo` (validators must explicitly vote for it)
-
-#### 3. Code Implementation
-- [ ] Implement new functionality (transaction type, ledger entry, etc.)
-- [ ] Add feature gate check in preflight:
-```cpp
-if (!env.current()->rules().enabled(feature{Name}))
-{
-    return temDISABLED;
-}
-```
-
-#### 4. Disable Route Handling
-- [ ] Ensure transaction returns `temDISABLED` when amendment is disabled
-- [ ] Implement early rejection in preflight/preclaim phase
-- [ ] Add appropriate error messages
-
-#### 5. Test Implementation
-Create comprehensive test suite with this structure:
-
-```cpp
-class {FeatureName}_test : public beast::unit_test::suite
-{
-public:
-    void testEnable(FeatureBitset features)
-    {
-        testcase("enabled");
-
-        // Test with feature DISABLED
-        {
-            auto const amendNoFeature = features - feature{Name};
-            Env env{*this, amendNoFeature};
-
-            env(transaction, ter(temDISABLED));
-        }
-
-        // Test with feature ENABLED
-        {
-            Env env{*this, features};
-
-            env(transaction, ter(tesSUCCESS));
-            // Validate new functionality works
-        }
-    }
-
-    void testPreflight(FeatureBitset features)
-    {
-        testcase("preflight");
-        // Test malformed transaction validation
-    }
-
-    void testPreclaim(FeatureBitset features)
-    {
-        testcase("preclaim");
-        // Test signature and claim phase validation
-    }
-
-    void testWithFeats(FeatureBitset features)
-    {
-        testEnable(features);
-        testPreflight(features);
-        testPreclaim(features);
-        // Add feature-specific tests
-    }
-
-    void run() override
-    {
-        using namespace test::jtx;
-        auto const all = supported_amendments();
-        testWithFeats(all);
-    }
-};
-```
-
-#### 6. Test Coverage Requirements
-- [ ] Test amendment DISABLED state (returns `temDISABLED`)
-- [ ] Test amendment ENABLED state (returns `tesSUCCESS`)
-- [ ] Test malformed transactions
-- [ ] Test signature validation
-- [ ] Test edge cases specific to feature
-- [ ] Test amendment transition behavior
-
-#### 7. Documentation
-- [ ] Create specification document (XLS_{NAME}.md)
-- [ ] Document new transaction types, ledger entries, or capabilities
-- [ ] Create test plan document
-- [ ] Document expected behavior when enabled/disabled
-
----
-
-## For FIX/EXTENSION Amendments
-
-### Checklist:
-
-#### 1. Fix Definition in features.macro
-**ONLY FILE TO EDIT:** `include/xrpl/protocol/detail/features.macro`
-
-- [ ] Add to TOP of features.macro (reverse chronological order):
-```
-XRPL_FIX(YourFixName, Supported::no, VoteBehavior::DefaultNo)
-```
-- [ ] This creates the variable `fixYourFixName` automatically
-- [ ] Follow naming convention: Use the fix name WITHOUT the "fix" prefix (it's added automatically)
-- [ ] Examples: `TokenEscrowV1` → `fixTokenEscrowV1`, `DirectoryLimit` → `fixDirectoryLimit`
-- [ ] Start with `Supported::no` during development, change to `Supported::yes` when ready
-
-#### 2. Backward Compatibility Implementation
-**Critical**: Use enable() with if/else to preserve existing functionality
-
-```cpp
-// Check if fix is enabled
-bool const fix{Name} = env.current()->rules().enabled(fix{Name});
-
-// Conditional logic based on amendment state
-if (fix{Name})
-{
-    // NEW behavior with fix applied
-    // This is the corrected/improved logic
-}
-else
-{
-    // OLD behavior (legacy path)
-    // Preserve original functionality for backward compatibility
-}
-```
-
-**Alternative pattern with ternary operator:**
-```cpp
-auto& viewToUse = sb.rules().enabled(fix{Name}) ? sb : legacyView;
-```
-
-#### 3. Multiple Fix Versions Pattern
-For iterative fixes, use version checking:
-
-```cpp
-bool const fixV1 = rv.rules().enabled(fixXahauV1);
-bool const fixV2 = rv.rules().enabled(fixXahauV2);
-
-switch (transactionType)
-{
-    case TYPE_1:
-        if (fixV1) {
-            // Behavior with V1 fix
-        } else {
-            // Legacy behavior
-        }
-        break;
-
-    case TYPE_2:
-        if (fixV2) {
-            // Behavior with V2 fix
-        } else if (fixV1) {
-            // Behavior with only V1
-        } else {
-            // Legacy behavior
-        }
-        break;
-}
-```
-
-#### 4. Test Both Paths
-Always test BOTH enabled and disabled states:
-
-```cpp
-void testFix(FeatureBitset features)
-{
-    testcase("fix behavior");
-
-    for (bool withFix : {false, true})
-    {
-        auto const amend = withFix ? features : features - fix{Name};
-        Env env{*this, amend};
-
-        // Setup test scenario
-        env.fund(XRP(1000), alice);
-        env.close();
-
-        if (!withFix)
-        {
-            // Test OLD behavior (before fix)
-            env(operation, ter(expectedErrorWithoutFix));
-            // Verify old behavior is preserved
-        }
-        else
-        {
-            // Test NEW behavior (after fix)
-            env(operation, ter(expectedErrorWithFix));
-            // Verify fix works correctly
-        }
-    }
-}
-```
-
-#### 5. Security Fix Pattern
-For security-critical fixes (like fixBatchInnerSigs):
-
-```cpp
-// Test vulnerability exists WITHOUT fix
-{
-    auto const amendNoFix = features - fix{Name};
-    Env env{*this, amendNoFix};
-
-    // Demonstrate vulnerability
-    // Expected: Validity::Valid (WRONG - vulnerable!)
-    BEAST_EXPECT(result == Validity::Valid);
-}
-
-// Test vulnerability is FIXED WITH amendment
-{
-    Env env{*this, features};
-
-    // Demonstrate fix
-    // Expected: Validity::SigBad (CORRECT - protected!)
-    BEAST_EXPECT(result == Validity::SigBad);
-}
-```
-
-#### 6. Test Coverage Requirements
-- [ ] Test fix DISABLED (legacy behavior preserved)
-- [ ] Test fix ENABLED (new behavior applied)
-- [ ] Test amendment transition
-- [ ] For security fixes: demonstrate vulnerability without fix
-- [ ] For security fixes: demonstrate protection with fix
-- [ ] Test edge cases that triggered the fix
-- [ ] Test combinations with other amendments
-
-#### 7. Documentation
-- [ ] Document what was broken/suboptimal
-- [ ] Document the fix applied
-- [ ] Document backward compatibility behavior
-- [ ] Create test summary showing both paths
-
----
-
-## Best Practices for All Amendments
-
-### 1. Naming Conventions
-- New features: `feature{DescriptiveName}` (e.g., `featureBatch`, `featureHooks`)
-- Fixes: `fix{IssueDescription}` (e.g., `fixBatchInnerSigs`, `fixNSDelete`)
-- Use CamelCase without underscores
-
-### 2. Feature Flag Checking
-```cpp
-// At the point where behavior diverges:
-bool const amendmentEnabled = env.current()->rules().enabled(feature{Name});
-
-// Or in view/rules context:
-if (!rv.rules().enabled(feature{Name}))
-    return {};  // or legacy behavior
-```
-
-### 3. Error Codes
-- New features when disabled: `temDISABLED`
-- Fixes may return different validation errors based on state
-- Document all error code changes
-
-### 4. Test Structure Template
-```cpp
-class Amendment_test : public beast::unit_test::suite
-{
-public:
-    // Core tests
-    void testEnable(FeatureBitset features);      // Enable/disable states
-    void testPreflight(FeatureBitset features);   // Validation
-    void testPreclaim(FeatureBitset features);    // Claim phase
-
-    // Feature-specific tests
-    void test{SpecificScenario1}(FeatureBitset features);
-    void test{SpecificScenario2}(FeatureBitset features);
-
-    // Master orchestrator
-    void testWithFeats(FeatureBitset features)
-    {
-        testEnable(features);
-        testPreflight(features);
-        testPreclaim(features);
-        test{SpecificScenario1}(features);
-        test{SpecificScenario2}(features);
-    }
-
-    void run() override
-    {
-        using namespace test::jtx;
-        auto const all = supported_amendments();
-        testWithFeats(all);
-    }
-};
-
-BEAST_DEFINE_TESTSUITE(Amendment, app, ripple);
-```
-
-### 5. Documentation Files
-Create these files:
-- **Specification**: `XLS_{FEATURE_NAME}.md` - Technical specification
-- **Test Plan**: `{FEATURE}_COMPREHENSIVE_TEST_PLAN.md` - Test strategy
-- **Test Summary**: `{FEATURE}_TEST_IMPLEMENTATION_SUMMARY.md` - Test results
-- **Review Findings**: `{FEATURE}_REVIEW_FINDINGS.md` (if applicable)
-
-### 6. Amendment Transition Testing
-Test the moment an amendment activates:
-
-```cpp
-void testAmendmentTransition(FeatureBitset features)
-{
-    testcase("amendment transition");
-
-    // Start with amendment disabled
-    auto const amendNoFeature = features - feature{Name};
-    Env env{*this, amendNoFeature};
-
-    // Perform operations in disabled state
-    env(operation1, ter(temDISABLED));
-
-    // Enable amendment mid-test (if testing mechanism supports it)
-    // Verify state transitions correctly
-
-    // Perform operations in enabled state
-    env(operation2, ter(tesSUCCESS));
-}
-```
-
-### 7. Cache Behavior
-If amendment affects caching (like fixBatchInnerSigs):
-- [ ] Test cache behavior without fix
-- [ ] Test cache behavior with fix
-- [ ] Document cache invalidation requirements
-
-### 8. Multi-Amendment Combinations
-Test interactions with other amendments:
-
-```cpp
-void testMultipleAmendments(FeatureBitset features)
-{
-    // Test all combinations
-    for (bool withFeature1 : {false, true})
-    for (bool withFeature2 : {false, true})
-    {
-        auto amend = features;
-        if (!withFeature1) amend -= feature1;
-        if (!withFeature2) amend -= feature2;
-
-        Env env{*this, amend};
-        // Test interaction behavior
-    }
-}
-```
-
-### 9. Performance Considerations
-- [ ] Minimize runtime checks (cache `rules().enabled()` result if used multiple times)
-- [ ] Avoid nested feature checks where possible
-- [ ] Document performance impact
-
-### 10. Code Review Checklist
-- [ ] Both enabled/disabled paths are tested
-- [ ] Backward compatibility is preserved (for fixes)
-- [ ] Error codes are appropriate
-- [ ] Documentation is complete
-- [ ] Security implications are considered
-- [ ] Cache behavior is correct
-- [ ] Edge cases are covered
-
----
-
-## Common Patterns Reference
-
-### Pattern: New Transaction Type
-```cpp
-// In transactor code:
-TER doApply() override
-{
-    if (!ctx_.view().rules().enabled(feature{Name}))
-        return temDISABLED;
-
-    // New transaction logic here
-    return tesSUCCESS;
-}
-```
-
-### Pattern: New Ledger Entry Type
-```cpp
-// In ledger entry creation:
-if (!view.rules().enabled(feature{Name}))
-    return temDISABLED;
-
-auto const sle = std::make_shared<SLE>(ltNEW_TYPE, keylet);
-view.insert(sle);
-```
-
-### Pattern: Behavioral Fix
-```cpp
-// At decision point:
-bool const useFix = view.rules().enabled(fix{Name});
-
-if (useFix)
-{
-    // Corrected behavior
-    return performCorrectValidation();
-}
-else
-{
-    // Legacy behavior (preserved for compatibility)
-    return performLegacyValidation();
-}
-```
-
-### Pattern: View Selection
-```cpp
-// Select which view to use based on amendment:
-auto& applyView = sb.rules().enabled(feature{Name}) ? newView : legacyView;
-```
-
----
-
-## Example Workflows
-
-### Workflow 1: Creating a New Feature Amendment
-
-1. User requests: "Add a new ClaimReward transaction type"
-2. Skill asks: "What should the amendment be called? (e.g., BalanceRewards - without 'feature' prefix)"
-3. Add to features.macro:
-```
-XRPL_FEATURE(BalanceRewards, Supported::no, VoteBehavior::DefaultNo)
-```
-4. Implement transaction with `temDISABLED` gate using `featureBalanceRewards`
-5. Create test suite with testEnable, testPreflight, testPreclaim
-6. Run tests with amendment enabled and disabled
-7. When ready, update to `Supported::yes` in features.macro
-8. Create specification document
-9. Review checklist
-
-### Workflow 2: Creating a Fix Amendment
-
-1. User requests: "Fix the signature validation bug in batch transactions"
-2. Skill asks: "What should the fix be called? (e.g., BatchInnerSigs - without 'fix' prefix)"
-3. Add to features.macro:
-```
-XRPL_FIX(BatchInnerSigs, Supported::no, VoteBehavior::DefaultNo)
-```
-4. Implement fix with if/else using `fixBatchInnerSigs` to preserve old behavior
-5. Create test demonstrating vulnerability without fix
-6. Create test showing fix works when enabled
-7. When ready, update to `Supported::yes` in features.macro
-8. Document both code paths
-9. Review checklist
-
----
-
-## Quick Reference: File Locations
-
-- **Amendment definitions (ONLY place to add)**: `include/xrpl/protocol/detail/features.macro`
-- **Feature.h (auto-generated, DO NOT EDIT)**: `include/xrpl/protocol/Feature.h`
-- **Feature.cpp (auto-generated, DO NOT EDIT)**: `src/libxrpl/protocol/Feature.cpp`
-- **Test files**: `src/test/app/` or `src/test/protocol/`
-- **Specifications**: Project root (e.g., `XLS_SMART_CONTRACTS.md`)
-- **Test plans**: Project root (e.g., `BATCH_COMPREHENSIVE_TEST_PLAN.md`)
-
-## How the Macro System Works
-
-The amendment system uses C preprocessor macros:
-
-1. **features.macro** - Single source of truth (ONLY file you edit):
-```
-XRPL_FEATURE(Batch, Supported::yes, VoteBehavior::DefaultNo)
-XRPL_FIX(TokenEscrowV1, Supported::yes, VoteBehavior::DefaultNo)
-```
-
-2. **Feature.h** - Auto-generated declarations from macro:
-```cpp
-extern uint256 const featureBatch;
-extern uint256 const fixTokenEscrowV1;
-```
-
-3. **Feature.cpp** - Auto-generated registrations from macro:
-```cpp
-uint256 const featureBatch = registerFeature("Batch", ...);
-uint256 const fixTokenEscrowV1 = registerFeature("fixTokenEscrowV1", ...);
-```
-
-**DO NOT** modify Feature.h or Feature.cpp directly - they process features.macro automatically.
-
----
-
-## When to Use This Skill
-
-Invoke this skill when:
-- Creating a new XRPL amendment
-- Adding a new transaction type or ledger entry
-- Fixing existing XRPL functionality
-- Need guidance on amendment best practices
-- Setting up amendment tests
-- Reviewing amendment implementation
-
-The skill will guide you through the appropriate workflow based on amendment type.

.github/CODEOWNERS

@@ -0,0 +1,8 @@
+# Allow anyone to review any change by default.
+*
+
+# Require the rpc-reviewers team to review changes to the rpc code.
+include/xrpl/protocol/ @xrplf/rpc-reviewers
+src/libxrpl/protocol/ @xrplf/rpc-reviewers
+src/xrpld/rpc/ @xrplf/rpc-reviewers
+src/xrpld/app/misc/ @xrplf/rpc-reviewers

.github/scripts/strategy-matrix/generate.py

@@ -196,22 +196,11 @@ def generate_strategy_matrix(all: bool, config: Config) -> list:
         # Enable code coverage for Debian Bookworm using GCC 15 in Debug on
         # linux/amd64
         if (
-            f"{os['distro_name']}-{os['distro_version']}" == "debian-bookworm"
-            and f"{os['compiler_name']}-{os['compiler_version']}" == "gcc-15"
+            f"{os['compiler_name']}-{os['compiler_version']}" == "gcc-15"
             and build_type == "Debug"
             and architecture["platform"] == "linux/amd64"
         ):
-            cmake_args = f"{cmake_args} -Dcoverage=ON -Dcoverage_format=xml -DCODE_COVERAGE_VERBOSE=ON -DCMAKE_C_FLAGS=-O0 -DCMAKE_CXX_FLAGS=-O0"
-
-        # Enable unity build for Ubuntu Jammy using GCC 12 in Debug on
-        # linux/amd64.
-        if (
-            f"{os['distro_name']}-{os['distro_version']}" == "ubuntu-jammy"
-            and f"{os['compiler_name']}-{os['compiler_version']}" == "gcc-12"
-            and build_type == "Debug"
-            and architecture["platform"] == "linux/amd64"
-        ):
-            cmake_args = f"{cmake_args} -Dunity=ON"
+            cmake_args = f"-Dcoverage=ON -Dcoverage_format=xml -DCODE_COVERAGE_VERBOSE=ON -DCMAKE_C_FLAGS=-O0 -DCMAKE_CXX_FLAGS=-O0 {cmake_args}"
 
         # Generate a unique name for the configuration, e.g. macos-arm64-debug
         # or debian-bookworm-gcc-12-amd64-release.
@@ -228,8 +217,6 @@ def generate_strategy_matrix(all: bool, config: Config) -> list:
         config_name += f"-{build_type.lower()}"
         if "-Dcoverage=ON" in cmake_args:
             config_name += "-coverage"
-        if "-Dunity=ON" in cmake_args:
-            config_name += "-unity"
 
         # Add the configuration to the list, with the most unique fields first,
         # so that they are easier to identify in the GitHub Actions UI, as long

.gitignore

@@ -69,3 +69,5 @@ DerivedData
 
 # AI tools.
 /.augment
+/.claude
+/CLAUDE.md

BUILD.md

@@ -575,16 +575,10 @@ See [Sanitizers docs](./docs/build/sanitizers.md) for more details.
 | `assert`   | OFF           | Enable assertions.                                             |
 | `coverage` | OFF           | Prepare the coverage report.                                   |
 | `tests`    | OFF           | Build tests.                                                   |
-| `unity`    | OFF           | Configure a unity build.                                       |
 | `xrpld`    | OFF           | Build the xrpld application, and not just the libxrpl library. |
 | `werr`     | OFF           | Treat compilation warnings as errors                           |
 | `wextra`   | OFF           | Enable additional compilation warnings                         |
 
-[Unity builds][5] may be faster for the first build (at the cost of much more
-memory) since they concatenate sources into fewer translation units. Non-unity
-builds may be faster for incremental builds, and can be helpful for detecting
-`#include` omissions.
-
 ## Troubleshooting
 
 ### Conan
@@ -651,7 +645,6 @@ If you want to experiment with a new package, follow these steps:
 [1]: https://github.com/conan-io/conan-center-index/issues/13168
 [2]: https://en.cppreference.com/w/cpp/compiler_support/20
 [3]: https://docs.conan.io/en/latest/getting_started.html
-[5]: https://en.wikipedia.org/wiki/Unity_build
 [6]: https://github.com/boostorg/beast/issues/2648
 [7]: https://github.com/boostorg/beast/issues/2661
 [gcovr]: https://gcovr.com/en/stable/getting-started.html

cfg/xrpld-example.cfg

@@ -940,7 +940,23 @@
 #
 #       path                Location to store the database
 #
-#   Optional keys for NuDB and RocksDB:
+#   Optional keys
+#
+#       cache_size          Size of cache for database records. Default is 16384.
+#                           Setting this value to 0 will use the default value.
+#
+#       cache_age           Length of time in minutes to keep database records
+#                           cached. Default is 5 minutes. Setting this value to
+#                           0 will use the default value.
+#
+#                           Note: if neither cache_size nor cache_age is
+#                           specified, the cache for database records will not
+#                           be created. If only one of cache_size or cache_age
+#                           is specified, the cache will be created using the
+#                           default value for the unspecified parameter.
+#
+#                           Note: the cache will not be created if online_delete
+#                           is specified.
 #
 #       fast_load           Boolean. If set, load the last persisted ledger
 #                           from disk upon process start before syncing to
@@ -948,6 +964,8 @@
 #                           if sufficient IOPS capacity is available.
 #                           Default 0.
 #
+#   Optional keys for NuDB or RocksDB:
+#
 #       earliest_seq        The default is 32570 to match the XRP ledger
 #                           network's earliest allowed sequence. Alternate
 #                           networks may set this value. Minimum value of 1.

cmake/XrplCore.cmake

@@ -4,12 +4,7 @@
 
 include(target_protobuf_sources)
 
-# Protocol buffers cannot participate in a unity build,
-# because all the generated sources
-# define a bunch of `static const` variables with the same names,
-# so we just build them as a separate library.
 add_library(xrpl.libpb)
-set_target_properties(xrpl.libpb PROPERTIES UNITY_BUILD OFF)
 target_protobuf_sources(xrpl.libpb xrpl/proto LANGUAGE cpp IMPORT_DIRS include/xrpl/proto
                         PROTOS include/xrpl/proto/xrpl.proto)
 

cmake/XrplSettings.cmake

@@ -30,14 +30,6 @@ if (tests)
     endif ()
 endif ()
 
-option(unity "Creates a build using UNITY support in cmake." OFF)
-if (unity)
-    if (NOT is_ci)
-        set(CMAKE_UNITY_BUILD_BATCH_SIZE 15 CACHE STRING "")
-    endif ()
-    set(CMAKE_UNITY_BUILD ON CACHE BOOL "Do a unity build")
-endif ()
-
 if (is_clang AND is_linux)
     option(voidstar "Enable Antithesis instrumentation." OFF)
 endif ()

conan.lock

@@ -6,7 +6,7 @@
         "sqlite3/3.49.1#8631739a4c9b93bd3d6b753bac548a63%1765850149.926",
         "soci/4.0.3#a9f8d773cd33e356b5879a4b0564f287%1765850149.46",
         "snappy/1.1.10#968fef506ff261592ec30c574d4a7809%1765850147.878",
-        "secp256k1/0.7.1#3a61e95e220062ef32c48d019e9c81f7%1770306721.686",
+        "secp256k1/0.7.0#0fda78daa3b864deb8a2fbc083398356%1770226294.524",
         "rocksdb/10.5.1#4a197eca381a3e5ae8adf8cffa5aacd0%1765850186.86",
         "re2/20230301#ca3b241baec15bd31ea9187150e0b333%1765850148.103",
         "protobuf/6.32.1#f481fd276fc23a33b85a3ed1e898b693%1765850161.038",

conanfile.py

@@ -23,7 +23,6 @@ class Xrpl(ConanFile):
         "shared": [True, False],
         "static": [True, False],
         "tests": [True, False],
-        "unity": [True, False],
         "xrpld": [True, False],
     }
 
@@ -33,7 +32,7 @@ class Xrpl(ConanFile):
         "libarchive/3.8.1",
         "nudb/2.0.9",
         "openssl/3.5.5",
-        "secp256k1/0.7.1",
+        "secp256k1/0.7.0",
         "soci/4.0.3",
         "zlib/1.3.1",
     ]
@@ -55,7 +54,6 @@ class Xrpl(ConanFile):
         "shared": False,
         "static": True,
         "tests": False,
-        "unity": False,
         "xrpld": False,
         "date/*:header_only": True,
         "ed25519/*:shared": False,
@@ -168,7 +166,6 @@ class Xrpl(ConanFile):
         tc.variables["rocksdb"] = self.options.rocksdb
         tc.variables["BUILD_SHARED_LIBS"] = self.options.shared
         tc.variables["static"] = self.options.static
-        tc.variables["unity"] = self.options.unity
         tc.variables["xrpld"] = self.options.xrpld
         tc.generate()
 

include/xrpl/nodestore/Database.h

@@ -133,6 +133,10 @@ public:
         std::uint32_t ledgerSeq,
         std::function<void(std::shared_ptr<NodeObject> const&)>&& callback);
 
+    /** Remove expired entries from the positive and negative caches. */
+    virtual void
+    sweep() = 0;
+
     /** Gather statistics pertaining to read and write activities.
      *
      * @param obj Json object reference into which to place counters.

include/xrpl/nodestore/detail/DatabaseNodeImp.h

@@ -23,6 +23,32 @@ public:
         beast::Journal j)
         : Database(scheduler, readThreads, config, j), backend_(std::move(backend))
     {
+        std::optional<int> cacheSize, cacheAge;
+
+        if (config.exists("cache_size"))
+        {
+            cacheSize = get<int>(config, "cache_size");
+            if (cacheSize.value() < 0)
+            {
+                Throw<std::runtime_error>("Specified negative value for cache_size");
+            }
+        }
+
+        if (config.exists("cache_age"))
+        {
+            cacheAge = get<int>(config, "cache_age");
+            if (cacheAge.value() < 0)
+            {
+                Throw<std::runtime_error>("Specified negative value for cache_age");
+            }
+        }
+
+        if (cacheSize != 0 || cacheAge != 0)
+        {
+            cache_ = std::make_shared<TaggedCache<uint256, NodeObject>>(
+                "DatabaseNodeImp", cacheSize.value_or(0), std::chrono::minutes(cacheAge.value_or(0)), stopwatch(), j);
+        }
+
         XRPL_ASSERT(
             backend_,
             "xrpl::NodeStore::DatabaseNodeImp::DatabaseNodeImp : non-null "
@@ -77,7 +103,13 @@ public:
         std::uint32_t ledgerSeq,
         std::function<void(std::shared_ptr<NodeObject> const&)>&& callback) override;
 
+    void
+    sweep() override;
+
 private:
+    // Cache for database objects. This cache is not always initialized. Check
+    // for null before using.
+    std::shared_ptr<TaggedCache<uint256, NodeObject>> cache_;
     // Persistent key/value storage
     std::shared_ptr<Backend> backend_;
 

include/xrpl/nodestore/detail/DatabaseRotatingImp.h

@@ -55,6 +55,9 @@ public:
     void
     sync() override;
 
+    void
+    sweep() override;
+
 private:
     std::shared_ptr<Backend> writableBackend_;
     std::shared_ptr<Backend> archiveBackend_;

src/libxrpl/net/RegisterSSLCerts.cpp

@@ -85,7 +85,8 @@ registerSSLCerts(boost::asio::ssl::context& ctx, boost::system::error_code& ec,
 // There is a very unpleasant interaction between <wincrypt> and
 // openssl x509 types (namely the former has macros that stomp
 // on the latter), these undefs allow this TU to be safely used in
-// unity builds without messing up subsequent TUs.
+// unity builds without messing up subsequent TUs. Although we
+// no longer use unity builds, leaving the undefs here does no harm.
 #if BOOST_OS_WINDOWS
 #undef X509_NAME
 #undef X509_EXTENSIONS

src/libxrpl/nodestore/DatabaseNodeImp.cpp

@@ -10,6 +10,11 @@ DatabaseNodeImp::store(NodeObjectType type, Blob&& data, uint256 const& hash, st
 
     auto obj = NodeObject::createObject(type, std::move(data), hash);
     backend_->store(obj);
+    if (cache_)
+    {
+        // After the store, replace a negative cache entry if there is one
+        cache_->canonicalize(hash, obj, [](std::shared_ptr<NodeObject> const& n) { return n->getType() == hotDUMMY; });
+    }
 }
 
 void
@@ -18,36 +23,77 @@ DatabaseNodeImp::asyncFetch(
     std::uint32_t ledgerSeq,
     std::function<void(std::shared_ptr<NodeObject> const&)>&& callback)
 {
+    if (cache_)
+    {
+        std::shared_ptr<NodeObject> obj = cache_->fetch(hash);
+        if (obj)
+        {
+            callback(obj->getType() == hotDUMMY ? nullptr : obj);
+            return;
+        }
+    }
     Database::asyncFetch(hash, ledgerSeq, std::move(callback));
 }
 
+void
+DatabaseNodeImp::sweep()
+{
+    if (cache_)
+        cache_->sweep();
+}
+
 std::shared_ptr<NodeObject>
 DatabaseNodeImp::fetchNodeObject(uint256 const& hash, std::uint32_t, FetchReport& fetchReport, bool duplicate)
 {
-    std::shared_ptr<NodeObject> nodeObject = nullptr;
-    Status status;
+    std::shared_ptr<NodeObject> nodeObject = cache_ ? cache_->fetch(hash) : nullptr;
 
-    try
+    if (!nodeObject)
     {
-        status = backend_->fetch(hash.data(), &nodeObject);
-    }
-    catch (std::exception const& e)
-    {
-        JLOG(j_.fatal()) << "fetchNodeObject " << hash << ": Exception fetching from backend: " << e.what();
-        Rethrow();
-    }
+        JLOG(j_.trace()) << "fetchNodeObject " << hash << ": record not " << (cache_ ? "cached" : "found");
 
-    switch (status)
+        Status status;
+
+        try
+        {
+            status = backend_->fetch(hash.data(), &nodeObject);
+        }
+        catch (std::exception const& e)
+        {
+            JLOG(j_.fatal()) << "fetchNodeObject " << hash << ": Exception fetching from backend: " << e.what();
+            Rethrow();
+        }
+
+        switch (status)
+        {
+            case ok:
+                if (cache_)
+                {
+                    if (nodeObject)
+                        cache_->canonicalize_replace_client(hash, nodeObject);
+                    else
+                    {
+                        auto notFound = NodeObject::createObject(hotDUMMY, {}, hash);
+                        cache_->canonicalize_replace_client(hash, notFound);
+                        if (notFound->getType() != hotDUMMY)
+                            nodeObject = notFound;
+                    }
+                }
+                break;
+            case notFound:
+                break;
+            case dataCorrupt:
+                JLOG(j_.fatal()) << "fetchNodeObject " << hash << ": nodestore data is corrupted";
+                break;
+            default:
+                JLOG(j_.warn()) << "fetchNodeObject " << hash << ": backend returns unknown result " << status;
+                break;
+        }
+    }
+    else
     {
-        case ok:
-        case notFound:
-            break;
-        case dataCorrupt:
-            JLOG(j_.fatal()) << "fetchNodeObject " << hash << ": nodestore data is corrupted";
-            break;
-        default:
-            JLOG(j_.warn()) << "fetchNodeObject " << hash << ": backend returns unknown result " << status;
-            break;
+        JLOG(j_.trace()) << "fetchNodeObject " << hash << ": record found in cache";
+        if (nodeObject->getType() == hotDUMMY)
+            nodeObject.reset();
     }
 
     if (nodeObject)
@@ -59,36 +105,66 @@ DatabaseNodeImp::fetchNodeObject(uint256 const& hash, std::uint32_t, FetchReport
 std::vector<std::shared_ptr<NodeObject>>
 DatabaseNodeImp::fetchBatch(std::vector<uint256> const& hashes)
 {
+    std::vector<std::shared_ptr<NodeObject>> results{hashes.size()};
     using namespace std::chrono;
     auto const before = steady_clock::now();
-
-    std::vector<uint256 const*> batch{};
-    batch.reserve(hashes.size());
+    std::unordered_map<uint256 const*, size_t> indexMap;
+    std::vector<uint256 const*> cacheMisses;
+    uint64_t hits = 0;
+    uint64_t fetches = 0;
     for (size_t i = 0; i < hashes.size(); ++i)
     {
         auto const& hash = hashes[i];
-        batch.push_back(&hash);
-    }
-
-    // Get the node objects that match the hashes from the backend. To protect
-    // against the backends returning fewer or more results than expected, the
-    // container is resized to the number of hashes.
-    auto results = backend_->fetchBatch(batch).first;
-    XRPL_ASSERT(
-        results.size() == hashes.size() || results.empty(),
-        "number of output objects either matches number of input hashes or is empty");
-    results.resize(hashes.size());
-    for (size_t i = 0; i < results.size(); ++i)
-    {
-        if (!results[i])
+        // See if the object already exists in the cache
+        auto nObj = cache_ ? cache_->fetch(hash) : nullptr;
+        ++fetches;
+        if (!nObj)
         {
-            JLOG(j_.error()) << "fetchBatch - "
-                             << "record not found in db. hash = " << strHex(hashes[i]);
+            // Try the database
+            indexMap[&hash] = i;
+            cacheMisses.push_back(&hash);
+        }
+        else
+        {
+            results[i] = nObj->getType() == hotDUMMY ? nullptr : nObj;
+            // It was in the cache.
+            ++hits;
         }
     }
 
+    JLOG(j_.debug()) << "fetchBatch - cache hits = " << (hashes.size() - cacheMisses.size())
+                     << " - cache misses = " << cacheMisses.size();
+    auto dbResults = backend_->fetchBatch(cacheMisses).first;
+
+    for (size_t i = 0; i < dbResults.size(); ++i)
+    {
+        auto nObj = std::move(dbResults[i]);
+        size_t index = indexMap[cacheMisses[i]];
+        auto const& hash = hashes[index];
+
+        if (nObj)
+        {
+            // Ensure all threads get the same object
+            if (cache_)
+                cache_->canonicalize_replace_client(hash, nObj);
+        }
+        else
+        {
+            JLOG(j_.error()) << "fetchBatch - "
+                             << "record not found in db or cache. hash = " << strHex(hash);
+            if (cache_)
+            {
+                auto notFound = NodeObject::createObject(hotDUMMY, {}, hash);
+                cache_->canonicalize_replace_client(hash, notFound);
+                if (notFound->getType() != hotDUMMY)
+                    nObj = std::move(notFound);
+            }
+        }
+        results[index] = std::move(nObj);
+    }
+
     auto fetchDurationUs = std::chrono::duration_cast<std::chrono::microseconds>(steady_clock::now() - before).count();
-    updateFetchMetrics(hashes.size(), 0, fetchDurationUs);
+    updateFetchMetrics(fetches, hits, fetchDurationUs);
     return results;
 }
 

src/libxrpl/nodestore/DatabaseRotatingImp.cpp

@@ -93,6 +93,12 @@ DatabaseRotatingImp::store(NodeObjectType type, Blob&& data, uint256 const& hash
     storeStats(1, nObj->getData().size());
 }
 
+void
+DatabaseRotatingImp::sweep()
+{
+    // nothing to do
+}
+
 std::shared_ptr<NodeObject>
 DatabaseRotatingImp::fetchNodeObject(uint256 const& hash, std::uint32_t, FetchReport& fetchReport, bool duplicate)
 {

src/test/app/LendingHelpers_test.cpp

@@ -592,18 +592,20 @@ class LendingHelpers_test : public beast::unit_test::suite
         auto const periodicRate = loanPeriodicRate(loanInterestRate, paymentInterval);
         Number const overpaymentAmount{50};
 
-        auto const overpaymentComponents = computeOverpaymentComponents(
+        ExtendedPaymentComponents const overpaymentComponents = computeOverpaymentComponents(
             asset, loanScale, overpaymentAmount, TenthBips32(0), TenthBips32(0), managementFeeRate);
 
-        auto const loanProperties = computeLoanProperties(
+        auto const loanProperites = computeLoanProperties(
             asset, loanPrincipal, loanInterestRate, paymentInterval, paymentsRemaining, managementFeeRate, loanScale);
 
+        Number const periodicPayment = loanProperites.periodicPayment;
+
         auto const ret = tryOverpayment(
             asset,
             loanScale,
             overpaymentComponents,
-            loanProperties.loanState,
-            loanProperties.periodicPayment,
+            loanProperites.loanState,
+            periodicPayment,
             periodicRate,
             paymentsRemaining,
             managementFeeRate,
@@ -634,20 +636,20 @@ class LendingHelpers_test : public beast::unit_test::suite
 
         // =========== VALIDATE STATE CHANGES ===========
         BEAST_EXPECTS(
-            loanProperties.loanState.interestDue - newState.interestDue == 0,
+            loanProperites.loanState.interestDue - newState.interestDue == 0,
             " interest change mismatch: expected 0, got " +
-                to_string(loanProperties.loanState.interestDue - newState.interestDue));
+                to_string(loanProperites.loanState.interestDue - newState.interestDue));
 
         BEAST_EXPECTS(
-            loanProperties.loanState.managementFeeDue - newState.managementFeeDue == 0,
+            loanProperites.loanState.managementFeeDue - newState.managementFeeDue == 0,
             " management fee change mismatch: expected 0, got " +
-                to_string(loanProperties.loanState.managementFeeDue - newState.managementFeeDue));
+                to_string(loanProperites.loanState.managementFeeDue - newState.managementFeeDue));
 
         BEAST_EXPECTS(
             actualPaymentParts.principalPaid ==
-                loanProperties.loanState.principalOutstanding - newState.principalOutstanding,
+                loanProperites.loanState.principalOutstanding - newState.principalOutstanding,
             " principalPaid mismatch: expected " +
-                to_string(loanProperties.loanState.principalOutstanding - newState.principalOutstanding) + ", got " +
+                to_string(loanProperites.loanState.principalOutstanding - newState.principalOutstanding) + ", got " +
                 to_string(actualPaymentParts.principalPaid));
     }
 
@@ -670,7 +672,7 @@ class LendingHelpers_test : public beast::unit_test::suite
         std::uint32_t const paymentsRemaining = 10;
         auto const periodicRate = loanPeriodicRate(loanInterestRate, paymentInterval);
 
-        auto const overpaymentComponents = computeOverpaymentComponents(
+        ExtendedPaymentComponents const overpaymentComponents = computeOverpaymentComponents(
             asset,
             loanScale,
             Number{50, 0},
@@ -678,15 +680,17 @@ class LendingHelpers_test : public beast::unit_test::suite
             TenthBips32(10'000),  // 10% overpayment fee
             managementFeeRate);
 
-        auto const loanProperties = computeLoanProperties(
+        auto const loanProperites = computeLoanProperties(
             asset, loanPrincipal, loanInterestRate, paymentInterval, paymentsRemaining, managementFeeRate, loanScale);
 
+        Number const periodicPayment = loanProperites.periodicPayment;
+
         auto const ret = tryOverpayment(
             asset,
             loanScale,
             overpaymentComponents,
-            loanProperties.loanState,
-            loanProperties.periodicPayment,
+            loanProperites.loanState,
+            periodicPayment,
             periodicRate,
             paymentsRemaining,
             managementFeeRate,
@@ -717,21 +721,21 @@ class LendingHelpers_test : public beast::unit_test::suite
         // =========== VALIDATE STATE CHANGES ===========
         // With no Loan interest, interest outstanding should not change
         BEAST_EXPECTS(
-            loanProperties.loanState.interestDue - newState.interestDue == 0,
+            loanProperites.loanState.interestDue - newState.interestDue == 0,
             " interest change mismatch: expected 0, got " +
-                to_string(loanProperties.loanState.interestDue - newState.interestDue));
+                to_string(loanProperites.loanState.interestDue - newState.interestDue));
 
         // With no Loan management fee, management fee due should not change
         BEAST_EXPECTS(
-            loanProperties.loanState.managementFeeDue - newState.managementFeeDue == 0,
+            loanProperites.loanState.managementFeeDue - newState.managementFeeDue == 0,
             " management fee change mismatch: expected 0, got " +
-                to_string(loanProperties.loanState.managementFeeDue - newState.managementFeeDue));
+                to_string(loanProperites.loanState.managementFeeDue - newState.managementFeeDue));
 
         BEAST_EXPECTS(
             actualPaymentParts.principalPaid ==
-                loanProperties.loanState.principalOutstanding - newState.principalOutstanding,
+                loanProperites.loanState.principalOutstanding - newState.principalOutstanding,
             " principalPaid mismatch: expected " +
-                to_string(loanProperties.loanState.principalOutstanding - newState.principalOutstanding) + ", got " +
+                to_string(loanProperites.loanState.principalOutstanding - newState.principalOutstanding) + ", got " +
                 to_string(actualPaymentParts.principalPaid));
     }
 
@@ -754,7 +758,7 @@ class LendingHelpers_test : public beast::unit_test::suite
         std::uint32_t const paymentsRemaining = 10;
         auto const periodicRate = loanPeriodicRate(loanInterestRate, paymentInterval);
 
-        auto const overpaymentComponents = computeOverpaymentComponents(
+        ExtendedPaymentComponents const overpaymentComponents = computeOverpaymentComponents(
             asset,
             loanScale,
             Number{50, 0},
@@ -762,15 +766,17 @@ class LendingHelpers_test : public beast::unit_test::suite
             TenthBips32(0),  // 0% overpayment fee
             managementFeeRate);
 
-        auto const loanProperties = computeLoanProperties(
+        auto const loanProperites = computeLoanProperties(
             asset, loanPrincipal, loanInterestRate, paymentInterval, paymentsRemaining, managementFeeRate, loanScale);
 
+        Number const periodicPayment = loanProperites.periodicPayment;
+
         auto const ret = tryOverpayment(
             asset,
             loanScale,
             overpaymentComponents,
-            loanProperties.loanState,
-            loanProperties.periodicPayment,
+            loanProperites.loanState,
+            periodicPayment,
             periodicRate,
             paymentsRemaining,
             managementFeeRate,
@@ -806,22 +812,22 @@ class LendingHelpers_test : public beast::unit_test::suite
         // =========== VALIDATE STATE CHANGES ===========
         BEAST_EXPECTS(
             actualPaymentParts.principalPaid ==
-                loanProperties.loanState.principalOutstanding - newState.principalOutstanding,
+                loanProperites.loanState.principalOutstanding - newState.principalOutstanding,
             " principalPaid mismatch: expected " +
-                to_string(loanProperties.loanState.principalOutstanding - newState.principalOutstanding) + ", got " +
+                to_string(loanProperites.loanState.principalOutstanding - newState.principalOutstanding) + ", got " +
                 to_string(actualPaymentParts.principalPaid));
 
         BEAST_EXPECTS(
-            actualPaymentParts.valueChange == newState.interestDue - loanProperties.loanState.interestDue,
+            actualPaymentParts.valueChange == newState.interestDue - loanProperites.loanState.interestDue,
             " valueChange mismatch: expected " +
-                to_string(newState.interestDue - loanProperties.loanState.interestDue) + ", got " +
+                to_string(newState.interestDue - loanProperites.loanState.interestDue) + ", got " +
                 to_string(actualPaymentParts.valueChange));
 
         // With no Loan management fee, management fee due should not change
         BEAST_EXPECTS(
-            loanProperties.loanState.managementFeeDue - newState.managementFeeDue == 0,
+            loanProperites.loanState.managementFeeDue - newState.managementFeeDue == 0,
             " management fee change mismatch: expected 0, got " +
-                to_string(loanProperties.loanState.managementFeeDue - newState.managementFeeDue));
+                to_string(loanProperites.loanState.managementFeeDue - newState.managementFeeDue));
     }
 
     void
@@ -843,7 +849,7 @@ class LendingHelpers_test : public beast::unit_test::suite
         std::uint32_t const paymentsRemaining = 10;
         auto const periodicRate = loanPeriodicRate(loanInterestRate, paymentInterval);
 
-        auto const overpaymentComponents = computeOverpaymentComponents(
+        ExtendedPaymentComponents const overpaymentComponents = computeOverpaymentComponents(
             asset,
             loanScale,
             Number{50, 0},
@@ -851,15 +857,17 @@ class LendingHelpers_test : public beast::unit_test::suite
             TenthBips32(0),       // 0% overpayment fee
             managementFeeRate);
 
-        auto const loanProperties = computeLoanProperties(
+        auto const loanProperites = computeLoanProperties(
             asset, loanPrincipal, loanInterestRate, paymentInterval, paymentsRemaining, managementFeeRate, loanScale);
 
+        Number const periodicPayment = loanProperites.periodicPayment;
+
         auto const ret = tryOverpayment(
             asset,
             loanScale,
             overpaymentComponents,
-            loanProperties.loanState,
-            loanProperties.periodicPayment,
+            loanProperites.loanState,
+            periodicPayment,
             periodicRate,
             paymentsRemaining,
             managementFeeRate,
@@ -896,26 +904,26 @@ class LendingHelpers_test : public beast::unit_test::suite
         // =========== VALIDATE STATE CHANGES ===========
         BEAST_EXPECTS(
             actualPaymentParts.principalPaid ==
-                loanProperties.loanState.principalOutstanding - newState.principalOutstanding,
+                loanProperites.loanState.principalOutstanding - newState.principalOutstanding,
             " principalPaid mismatch: expected " +
-                to_string(loanProperties.loanState.principalOutstanding - newState.principalOutstanding) + ", got " +
+                to_string(loanProperites.loanState.principalOutstanding - newState.principalOutstanding) + ", got " +
                 to_string(actualPaymentParts.principalPaid));
 
         // The change in interest is equal to the value change sans the
         // overpayment interest
         BEAST_EXPECTS(
             actualPaymentParts.valueChange - actualPaymentParts.interestPaid ==
-                newState.interestDue - loanProperties.loanState.interestDue,
+                newState.interestDue - loanProperites.loanState.interestDue,
             " valueChange mismatch: expected " +
                 to_string(
-                    newState.interestDue - loanProperties.loanState.interestDue + actualPaymentParts.interestPaid) +
+                    newState.interestDue - loanProperites.loanState.interestDue + actualPaymentParts.interestPaid) +
                 ", got " + to_string(actualPaymentParts.valueChange));
 
         // With no Loan management fee, management fee due should not change
         BEAST_EXPECTS(
-            loanProperties.loanState.managementFeeDue - newState.managementFeeDue == 0,
+            loanProperites.loanState.managementFeeDue - newState.managementFeeDue == 0,
             " management fee change mismatch: expected 0, got " +
-                to_string(loanProperties.loanState.managementFeeDue - newState.managementFeeDue));
+                to_string(loanProperites.loanState.managementFeeDue - newState.managementFeeDue));
     }
 
     void
@@ -939,7 +947,7 @@ class LendingHelpers_test : public beast::unit_test::suite
         std::uint32_t const paymentsRemaining = 10;
         auto const periodicRate = loanPeriodicRate(loanInterestRate, paymentInterval);
 
-        auto const overpaymentComponents = computeOverpaymentComponents(
+        ExtendedPaymentComponents const overpaymentComponents = computeOverpaymentComponents(
             asset,
             loanScale,
             Number{50, 0},
@@ -947,15 +955,17 @@ class LendingHelpers_test : public beast::unit_test::suite
             TenthBips32(0),       // 0% overpayment fee
             managementFeeRate);
 
-        auto const loanProperties = computeLoanProperties(
+        auto const loanProperites = computeLoanProperties(
             asset, loanPrincipal, loanInterestRate, paymentInterval, paymentsRemaining, managementFeeRate, loanScale);
 
+        Number const periodicPayment = loanProperites.periodicPayment;
+
         auto const ret = tryOverpayment(
             asset,
             loanScale,
             overpaymentComponents,
-            loanProperties.loanState,
-            loanProperties.periodicPayment,
+            loanProperites.loanState,
+            periodicPayment,
             periodicRate,
             paymentsRemaining,
             managementFeeRate,
@@ -994,23 +1004,23 @@ class LendingHelpers_test : public beast::unit_test::suite
         // =========== VALIDATE STATE CHANGES ===========
         BEAST_EXPECTS(
             actualPaymentParts.principalPaid ==
-                loanProperties.loanState.principalOutstanding - newState.principalOutstanding,
+                loanProperites.loanState.principalOutstanding - newState.principalOutstanding,
             " principalPaid mismatch: expected " +
-                to_string(loanProperties.loanState.principalOutstanding - newState.principalOutstanding) + ", got " +
+                to_string(loanProperites.loanState.principalOutstanding - newState.principalOutstanding) + ", got " +
                 to_string(actualPaymentParts.principalPaid));
 
         // Note that the management fee value change is not captured, as this
         // value is not needed to correctly update the Vault state.
         BEAST_EXPECTS(
-            (newState.managementFeeDue - loanProperties.loanState.managementFeeDue == Number{-20592, -5}),
+            (newState.managementFeeDue - loanProperites.loanState.managementFeeDue == Number{-20592, -5}),
             " management fee change mismatch: expected " + to_string(Number{-20592, -5}) + ", got " +
-                to_string(newState.managementFeeDue - loanProperties.loanState.managementFeeDue));
+                to_string(newState.managementFeeDue - loanProperites.loanState.managementFeeDue));
 
         BEAST_EXPECTS(
             actualPaymentParts.valueChange - actualPaymentParts.interestPaid ==
-                newState.interestDue - loanProperties.loanState.interestDue,
+                newState.interestDue - loanProperites.loanState.interestDue,
             " valueChange mismatch: expected " +
-                to_string(newState.interestDue - loanProperties.loanState.interestDue) + ", got " +
+                to_string(newState.interestDue - loanProperites.loanState.interestDue) + ", got " +
                 to_string(actualPaymentParts.valueChange - actualPaymentParts.interestPaid));
     }
 
@@ -1033,7 +1043,7 @@ class LendingHelpers_test : public beast::unit_test::suite
         std::uint32_t const paymentsRemaining = 10;
         auto const periodicRate = loanPeriodicRate(loanInterestRate, paymentInterval);
 
-        auto const overpaymentComponents = computeOverpaymentComponents(
+        ExtendedPaymentComponents const overpaymentComponents = computeOverpaymentComponents(
             asset,
             loanScale,
             Number{50, 0},
@@ -1041,15 +1051,17 @@ class LendingHelpers_test : public beast::unit_test::suite
             TenthBips32(10'000),  // 10% overpayment fee
             managementFeeRate);
 
-        auto const loanProperties = computeLoanProperties(
+        auto const loanProperites = computeLoanProperties(
             asset, loanPrincipal, loanInterestRate, paymentInterval, paymentsRemaining, managementFeeRate, loanScale);
 
+        Number const periodicPayment = loanProperites.periodicPayment;
+
         auto const ret = tryOverpayment(
             asset,
             loanScale,
             overpaymentComponents,
-            loanProperties.loanState,
-            loanProperties.periodicPayment,
+            loanProperites.loanState,
+            periodicPayment,
             periodicRate,
             paymentsRemaining,
             managementFeeRate,
@@ -1089,23 +1101,23 @@ class LendingHelpers_test : public beast::unit_test::suite
 
         BEAST_EXPECTS(
             actualPaymentParts.principalPaid ==
-                loanProperties.loanState.principalOutstanding - newState.principalOutstanding,
+                loanProperites.loanState.principalOutstanding - newState.principalOutstanding,
             " principalPaid mismatch: expected " +
-                to_string(loanProperties.loanState.principalOutstanding - newState.principalOutstanding) + ", got " +
+                to_string(loanProperites.loanState.principalOutstanding - newState.principalOutstanding) + ", got " +
                 to_string(actualPaymentParts.principalPaid));
 
         // Note that the management fee value change is not captured, as this
         // value is not needed to correctly update the Vault state.
         BEAST_EXPECTS(
-            (newState.managementFeeDue - loanProperties.loanState.managementFeeDue == Number{-18304, -5}),
+            (newState.managementFeeDue - loanProperites.loanState.managementFeeDue == Number{-18304, -5}),
             " management fee change mismatch: expected " + to_string(Number{-18304, -5}) + ", got " +
-                to_string(newState.managementFeeDue - loanProperties.loanState.managementFeeDue));
+                to_string(newState.managementFeeDue - loanProperites.loanState.managementFeeDue));
 
         BEAST_EXPECTS(
             actualPaymentParts.valueChange - actualPaymentParts.interestPaid ==
-                newState.interestDue - loanProperties.loanState.interestDue,
+                newState.interestDue - loanProperites.loanState.interestDue,
             " valueChange mismatch: expected " +
-                to_string(newState.interestDue - loanProperties.loanState.interestDue) + ", got " +
+                to_string(newState.interestDue - loanProperites.loanState.interestDue) + ", got " +
                 to_string(actualPaymentParts.valueChange - actualPaymentParts.interestPaid));
     }
 

src/test/app/Manifest_test.cpp

@@ -827,13 +827,8 @@ public:
 
             // applyManifest should accept new manifests with
             // higher sequence numbers
-            auto const seq0 = cache.sequence();
             BEAST_EXPECT(cache.applyManifest(clone(s_a0)) == ManifestDisposition::accepted);
-            BEAST_EXPECT(cache.sequence() > seq0);
-
-            auto const seq1 = cache.sequence();
             BEAST_EXPECT(cache.applyManifest(clone(s_a0)) == ManifestDisposition::stale);
-            BEAST_EXPECT(cache.sequence() == seq1);
 
             BEAST_EXPECT(cache.applyManifest(clone(s_a1)) == ManifestDisposition::accepted);
             BEAST_EXPECT(cache.applyManifest(clone(s_a1)) == ManifestDisposition::stale);

src/test/app/SHAMapStore_test.cpp

@@ -490,8 +490,19 @@ public:
         Env env(*this, envconfig(onlineDelete));
 
         /////////////////////////////////////////////////////////////
-        // Create NodeStore with two backends to allow online deletion of data.
-        // Normally, SHAMapStoreImp handles all these details.
+        // Create the backend. Normally, SHAMapStoreImp handles all these
+        // details
+        auto nscfg = env.app().config().section(ConfigSection::nodeDatabase());
+
+        // Provide default values:
+        if (!nscfg.exists("cache_size"))
+            nscfg.set(
+                "cache_size", std::to_string(env.app().config().getValueFor(SizedItem::treeCacheSize, std::nullopt)));
+
+        if (!nscfg.exists("cache_age"))
+            nscfg.set(
+                "cache_age", std::to_string(env.app().config().getValueFor(SizedItem::treeCacheAge, std::nullopt)));
+
         NodeStoreScheduler scheduler(env.app().getJobQueue());
 
         std::string const writableDb = "write";
@@ -499,8 +510,9 @@ public:
         auto writableBackend = makeBackendRotating(env, scheduler, writableDb);
         auto archiveBackend = makeBackendRotating(env, scheduler, archiveDb);
 
+        // Create NodeStore with two backends to allow online deletion of
+        // data
         constexpr int readThreads = 4;
-        auto nscfg = env.app().config().section(ConfigSection::nodeDatabase());
         auto dbr = std::make_unique<NodeStore::DatabaseRotatingImp>(
             scheduler,
             readThreads,

src/xrpld/app/main/Application.cpp

@@ -908,6 +908,10 @@ public:
             JLOG(m_journal.debug()) << "MasterTransaction sweep.  Size before: " << oldMasterTxSize
                                     << "; size after: " << masterTxCache.size();
         }
+        {
+            // Does not appear to have an associated cache.
+            getNodeStore().sweep();
+        }
         {
             std::size_t const oldLedgerMasterCacheSize = getLedgerMaster().getFetchPackCacheSize();
 

src/xrpld/app/misc/SHAMapStoreImp.cpp

@@ -130,6 +130,14 @@ std::unique_ptr<NodeStore::Database>
 SHAMapStoreImp::makeNodeStore(int readThreads)
 {
     auto nscfg = app_.config().section(ConfigSection::nodeDatabase());
+
+    // Provide default values:
+    if (!nscfg.exists("cache_size"))
+        nscfg.set("cache_size", std::to_string(app_.config().getValueFor(SizedItem::treeCacheSize, std::nullopt)));
+
+    if (!nscfg.exists("cache_age"))
+        nscfg.set("cache_age", std::to_string(app_.config().getValueFor(SizedItem::treeCacheAge, std::nullopt)));
+
     std::unique_ptr<NodeStore::Database> db;
 
     if (deleteInterval_)
@@ -218,6 +226,8 @@ SHAMapStoreImp::run()
     LedgerIndex lastRotated = state_db_.getState().lastRotated;
     netOPs_ = &app_.getOPs();
     ledgerMaster_ = &app_.getLedgerMaster();
+    fullBelowCache_ = &(*app_.getNodeFamily().getFullBelowCache());
+    treeNodeCache_ = &(*app_.getNodeFamily().getTreeNodeCache());
 
     if (advisoryDelete_)
         canDelete_ = state_db_.getCanDelete();
@@ -480,19 +490,16 @@ void
 SHAMapStoreImp::clearCaches(LedgerIndex validatedSeq)
 {
     ledgerMaster_->clearLedgerCachePrior(validatedSeq);
-    // Also clear the FullBelowCache so its generation counter is bumped.
-    // This prevents stale "full below" markers from persisting across
-    // backend rotation/online deletion and interfering with SHAMap sync.
-    app_.getNodeFamily().getFullBelowCache()->clear();
+    fullBelowCache_->clear();
 }
 
 void
 SHAMapStoreImp::freshenCaches()
 {
-    if (freshenCache(*app_.getNodeFamily().getTreeNodeCache()))
+    if (freshenCache(*treeNodeCache_))
+        return;
+    if (freshenCache(app_.getMasterTransaction().getCache()))
         return;
-
-    freshenCache(app_.getMasterTransaction().getCache());
 }
 
 void

src/xrpld/app/misc/SHAMapStoreImp.h

@@ -93,6 +93,8 @@ private:
     // as of run() or before
     NetworkOPs* netOPs_ = nullptr;
     LedgerMaster* ledgerMaster_ = nullptr;
+    FullBelowCache* fullBelowCache_ = nullptr;
+    TreeNodeCache* treeNodeCache_ = nullptr;
 
     static constexpr auto nodeStoreName_ = "NodeStore";
 

src/xrpld/app/misc/detail/Manifest.cpp

@@ -459,10 +459,6 @@ ManifestCache::applyManifest(Manifest m)
 
         auto masterKey = m.masterKey;
         map_.emplace(std::move(masterKey), std::move(m));
-
-        // Something has changed. Keep track of it.
-        seq_++;
-
         return ManifestDisposition::accepted;
     }
 

src/xrpld/app/misc/detail/ValidatorSite.cpp

@@ -124,7 +124,11 @@ ValidatorSite::load(std::vector<std::string> const& siteURIs, std::lock_guard<st
     {
         try
         {
-            sites_.emplace_back(uri);
+            // This is not super efficient, but it doesn't happen often.
+            bool found =
+                std::ranges::any_of(sites_, [&uri](auto const& site) { return site.loadedResource->uri == uri; });
+            if (!found)
+                sites_.emplace_back(uri);
         }
         catch (std::exception const& e)
         {
@@ -183,6 +187,15 @@ ValidatorSite::stop()
 void
 ValidatorSite::setTimer(std::lock_guard<std::mutex> const& site_lock, std::lock_guard<std::mutex> const& state_lock)
 {
+    if (!sites_.empty() &&  //
+        std::ranges::all_of(sites_, [](auto const& site) { return site.lastRefreshStatus.has_value(); }))
+    {
+        // If all of the sites have been handled at least once (including
+        // errors and timeouts), call missingSite, which will load the cache
+        // files for any lists that are still unavailable.
+        missingSite(site_lock);
+    }
+
     auto next = std::min_element(
         sites_.begin(), sites_.end(), [](Site const& a, Site const& b) { return a.nextRefresh < b.nextRefresh; });
 
@@ -285,12 +298,14 @@ ValidatorSite::onRequestTimeout(std::size_t siteIdx, error_code const& ec)
         // processes a network error. Usually, this function runs first,
         // but on extremely rare occasions, the response handler can run
         // first, which will leave activeResource empty.
-        auto const& site = sites_[siteIdx];
+        auto& site = sites_[siteIdx];
         if (site.activeResource)
             JLOG(j_.warn()) << "Request for " << site.activeResource->uri << " took too long";
         else
             JLOG(j_.error()) << "Request took too long, but a response has "
                                 "already been processed";
+        if (!site.lastRefreshStatus)
+            site.lastRefreshStatus.emplace(Site::Status{clock_type::now(), ListDisposition::invalid, "timeout"});
     }
 
     std::lock_guard lock_state{state_mutex_};