90 Commits

Author SHA1 Message Date
Alex Kremer
55657fd1e9 fix: Amendment block initial ledger load and cache load (#3149) 2026-07-22 16:13:47 +01:00
Ayaz Salikhov
0b15620d87 chore: Use libxrpl 3.3.0-rc1 (#3144) 2026-07-20 20:19:16 +01:00
Bryan
52d28f8389 feat: Add ETL Indexing for mpt_issuance_history (#3104)
## Summary

Adds live ETL indexing for mptoken_issuance_history by extracting MPT
issuance references from transaction metadata and transaction fields,
then writing both MPT transaction index shapes during the existing MPT
ETL pass. Successful transactions are indexed from affected MPT ledger
objects; failed transactions are indexed only when they carry an
attached MPT issuance reference.

## Changes

- Adds `getMPTokenIssuanceTxsFromTx` to extract issuance transaction
index records from `tesSUCCESS` transactions.
- Scans `AffectedNodes` generically for `MPToken` and `MPTokenIssuance`
ledger objects instead of relying on a transaction-type allowlist.
- Reconstructs issuance IDs for `MPTokenIssuance` objects with
`makeMptID(sequence, issuer)`, avoiding the hashed ledger-key trap.
- Deduplicates distinct issuances per transaction and attaches the
transaction’s affected accounts for account-level fanout.
- Extends `MPTExt` so live and initial ETL data writes:
  - existing MPT holder index data
  - `mptoken_issuance_transactions`
  - `account_mptoken_issuance_transactions`
- Adds a Prometheus counter for MPT issuance transaction index rows
written by ETL.
- Logs unexpectedly large per-transaction fanout.

## Tests

Adds unit coverage for:

- failed transactions producing no records
- `MPTokenIssuanceCreate`, `Destroy`, `Set`, and `MPTokenAuthorize`
- generic transaction-type coverage through `Payment`, `Clawback`,
`OfferCreate`, and `AMMDeposit`
- ID reconstruction from issuance nodes
- multi-issuance fanout and deduplication
- affected-account propagation
- ETL extension writes to both backend index paths while preserving
existing holder indexing

---------

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2026-07-07 08:40:06 -07:00
Sergey Kuznetsov
87f0b480bf fix: Don't cache requests with params (#3113)
`ResponseExpirationCache` is using only `method` (or `command`) as a
key, but responses might be different with different params. This may
lead to a client getting incorrect response because of caching. This PR
fixes it by caching only requests without any additional params.
2026-06-26 12:33:27 +01:00
Sergey Kuznetsov
9b04d64a0e chore: Update libxrpl to 3.2.0 (#3114) 2026-06-25 11:19:12 +01:00
Ayaz Salikhov
52af23fe77 chore: Fixes after tidy v22 update (#3108) 2026-06-17 11:33:39 +01:00
Shawn Xie
0aa7ed4919 feat: Update MPToken ETL to support all transactions types (#3102)
## Summary

Fixes incomplete MPT holder indexing in ETL.

Previously, Clio only indexed MPT holders when an `ltMPTOKEN` ledger
node was created by an `MPTokenAuthorize` transaction. However, with
future amendments, XRPL can also create `ltMPTOKEN` nodes as part of
other successful transactions, such as `Payment`, when a holder receives
an MPT. Those holders were silently skipped and therefore missing from
the `mpt_holders` RPC results.

This change makes MPT holder indexing metadata-driven instead of
transaction-type-driven. ETL now scans successful transaction metadata
for newly created `ltMPTOKEN` nodes and writes every holder it finds.

## What Changed

- MPT holder extraction now works for any successful transaction that
creates an `ltMPTOKEN`.
- Multiple MPT holders created in a single transaction are all indexed.
- The `mpt_holders` table is kept consistent with ledger metadata
instead of relying on assumptions about which transaction types can
create holder nodes.
- Added regression coverage for MPT holders created by a `Payment`
transaction.

## Testing

- `build/clio_tests '--gtest_filter=MPTExtTests.*'`
- `clang-tidy -p build src/etl/MPTHelpers.cpp src/etl/MPTHelpers.hpp
src/etl/impl/ext/MPT.cpp tests/unit/etl/ext/MPTTests.cpp`
2026-06-15 11:46:55 +01:00
Alex Kremer
dbc98894ba refactor: Rename static constants (#3073) 2026-05-13 12:09:06 +01:00
Sergey Kuznetsov
e974e1899f fix: Don't cache responses to admin's requests (#3058) 2026-05-05 17:00:59 +01:00
Alex Kremer
e96ef39857 chore: Fix clang-tidy issues after introducing new checks (#3061) 2026-05-05 15:21:54 +01:00
Alex Kremer
51244feb4a chore: Enable more clang-tidy checks (#3054) 2026-05-01 15:31:45 +01:00
Alex Kremer
21066d16e9 chore: Remove copyright from all source files (#2975) 2026-03-24 15:25:32 +00:00
Sergey Kuznetsov
2316cced64 feat: Recover from fallback writer state (#3000)
Add a timer and new `FallbackRecovery` state to try to recover from
`Fallback` writer state every hour.
Fixes #2997
2026-03-16 14:09:26 +00:00
Sergey Kuznetsov
fbdd6d6105 feat: Limit cache loading in cluster (#2985)
This PR adds an option to limit simultaneous cache loading in a cluster
to one node at a time.
Fixes #2707
2026-03-09 17:11:20 +00:00
Sergey Kuznetsov
05e52ee7a4 fix: Start without cache file (#2976)
#2830 introduced a bug that clio couldn't start without having a cache
file. This PR fixes the problem.
2026-03-06 15:59:27 +00:00
Ayaz Salikhov
6ba58f42f0 style: Set clang-format width 100 (#2953) 2026-02-20 15:56:03 +00:00
github-actions[bot]
3bb3e0b9f9 style: clang-tidy auto fixes (#2930)
Fixes #2929.
2026-01-22 11:34:04 +00:00
Sergey Kuznetsov
a72e5a180f feat: Choose writer by cluster communication (#2830)
Fixes #1974
2026-01-21 11:41:26 +00:00
github-actions[bot]
89fbcbf66a style: clang-tidy auto fixes (#2862) 2025-12-17 10:19:20 +00:00
Ayaz Salikhov
7600e740a0 revert: "refactor: Add writing command to etl::SystemState" (#2860) 2025-12-16 15:06:35 +00:00
github-actions[bot]
e9b98cf5b3 style: clang-tidy auto fixes (#2848)
Co-authored-by: godexsoft <385326+godexsoft@users.noreply.github.com>
2025-12-11 09:51:03 +00:00
Sergey Kuznetsov
3aa1854129 refactor: Add writing command to etl::SystemState (#2842) 2025-12-10 16:29:29 +00:00
Sergey Kuznetsov
346c9f9bdf feat: Read and write LedgerCache to file (#2761)
Fixes #2413.
2025-11-13 17:01:40 +00:00
Alex Kremer
2f8a704071 feat: Ledger publisher use async framework (#2756) 2025-11-05 15:26:03 +00:00
Alex Kremer
fcc5a5425e feat: New ETL by default (#2752) 2025-11-05 13:29:36 +00:00
Alex Kremer
b4e40558c9 fix: Address AmendmentBlockHandler flakiness in old ETL tests (#2694)
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2025-10-15 17:15:12 +01:00
Alex Kremer
693ed2061c fix: ASAN issue with AmendmentBlockHandler test (#2674)
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2025-10-07 15:18:01 +01:00
github-actions[bot]
420b99cfa1 style: clang-tidy auto fixes (#2635)
Co-authored-by: godexsoft <385326+godexsoft@users.noreply.github.com>
2025-09-16 14:14:55 +01:00
Ayaz Salikhov
3f2ada3439 fix: Keep spdlog loggers valid between tests (#2614) 2025-09-15 14:47:35 +01:00
Ayaz Salikhov
a172d0b7ea feat: Validate unexpected config values (#2457) 2025-08-20 12:38:51 +01:00
Ayaz Salikhov
7e42507b9a fix: Fix GCC 15 discovered bugs (#2425) 2025-08-12 15:57:14 +01:00
Ayaz Salikhov
9420c506ca feat: Use spdlog logger (#2372) 2025-08-11 18:04:04 +01:00
Alex Kremer
87565b685a fix: MPT test object producing ASAN error (#2406)
Plus some driveby cleanup of incorrect placement of `constexpr`
2025-08-07 18:57:26 +01:00
Alex Kremer
b29e2e4c88 chore: Update boost usages to match 1.88 (#2355) 2025-07-23 15:49:19 +01:00
github-actions[bot]
4919b57466 style: clang-tidy auto fixes (#2317) 2025-07-10 10:50:03 +01:00
Alex Kremer
413b823976 fix: ASAN stack-buffer-overflow in NFTHelpersTest_NFTDataFromLedgerObject (#2306) 2025-07-09 13:43:39 +01:00
github-actions[bot]
cc506fd094 style: Update pre-commit hooks (#2290)
Update versions of pre-commit hooks to latest version.

Co-authored-by: mathbunnyru <12270691+mathbunnyru@users.noreply.github.com>
2025-07-02 16:36:34 +01:00
Ayaz Salikhov
57dda8ac50 chore: Update fmt to 11.2.0 (#2281)
Seems that new clang-tidy fails with our old fmt lib
2025-07-02 14:11:35 +01:00
Ayaz Salikhov
082f2fe21e style: Put static before type (#2231) 2025-06-17 16:07:19 +01:00
Alex Kremer
743c9b92de feat: Read-write switching in ETLng (#2199)
Fixes #1597
2025-06-11 17:53:14 +01:00
Ayaz Salikhov
ecdea015b9 style: Mark JSON literal strings with R"JSON (#2169) 2025-05-30 15:50:39 +01:00
github-actions[bot]
bfa17134d2 style: clang-tidy auto fixes (#2167)
Fixes #2166. Please review and commit clang-tidy fixes.

Co-authored-by: godexsoft <385326+godexsoft@users.noreply.github.com>
2025-05-30 10:54:16 +01:00
Ayaz Salikhov
57b8ff1c49 fix: Use UniformRandomGenerator class to prevent threading issue (#2165) 2025-05-29 19:59:40 +01:00
Peter Chen
cdb5882688 refactor: remove old config (#2097)
fixes #1805

---------

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
Co-authored-by: Maria Shodunke <maria-robobug@users.noreply.github.com>
Co-authored-by: Sergey Kuznetsov <skuznetsov@ripple.com>
2025-05-15 11:53:47 -04:00
github-actions[bot]
683e2ba54b style: clang-tidy auto fixes (#2126) 2025-05-15 10:34:59 +01:00
Ayaz Salikhov
243e174f1e refactor: Load balancer forwarding metrics (#2103) 2025-05-14 11:43:31 +01:00
Ayaz Salikhov
5e43c3b099 test: Remove SetUp and TearDown methods in tests (#2086)
Fix: https://github.com/XRPLF/clio/issues/910
2025-05-07 09:41:23 +01:00
Ayaz Salikhov
af0a8af287 style: Add typos pre-commit hook (#2041) 2025-04-29 09:59:11 +01:00
Sergey Kuznetsov
46514c8fe9 feat: More efficient cache (#1997)
Fixes #1473.
2025-04-17 16:44:53 +01:00
Peter Chen
89eb962d85 fix: CTID issue (#2001)
fixes #1998
2025-04-16 19:55:12 +01:00