ARCHIVE/clio
1
0
Fork 0
mirror of https://github.com/XRPLF/clio.git synced 2025-11-04 20:05:51 +00:00
Code Issues Packages Projects Releases Wiki Activity

style: Add prettier pre-commit hook (#2031)

Browse Source 
There are 2 things to know about prettier:
- it's quite pretty most of the time
- it's not configurable
This commit is contained in:
Ayaz Salikhov
2025-04-25 16:24:45 +01:00
committed by GitHub
parent 593d7298b1
commit 1e0a2f5162
45 changed files with 632 additions and 548 deletions
Download Patch File Download Diff File Expand all files Collapse all files

33
src/migration/README.md
View File

@@ -1,35 +1,26 @@
# Clio Migration
Clio maintains the off-chain data of XRPL and multiple indexes tables to powering complex queries. To simplify the creation of index tables, this migration framework handles the process of database change and facilitates the migration of historical data seamlessly.
## Command Line Usage
Clio provides a migration command-line tool to migrate data in database.
> Note: We need a **configuration file** to run the migration tool. This configuration file has the same format as the configuration file of the Clio server, ensuring consistency and ease of use. It reads the database configuration from the same session as the server's configuration, eliminating the need for separate setup or additional configuration files. Be aware that migration-specific configuration is under `.migration` session.
### To query migration status:
./clio_server --migrate status ~/config/migrator.json
This command returns the current migration status of each migrator. The example output:
Current Migration Status:
Migrator: ExampleMigrator - Feature v1, Clio v3 - not migrated
### To start a migration:
./clio_server --migrate ExampleMigrator ~/config/migrator.json
Migration will run if the migrator has not been migrated. The migrator will be marked as migrated after the migration is completed.
## How to write a migrator
@@ -52,10 +43,10 @@ It contains:
> **Note** Each migrator is designed to work with a specific database.
- Register your migrator in MigrationManager. Currently we only support Cassandra/ScyllaDB. Migrator needs to be registered in `CassandraSupportedMigrators`.
- Register your migrator in MigrationManager. Currently we only support Cassandra/ScyllaDB. Migrator needs to be registered in `CassandraSupportedMigrators`.
## How to use full table scanner (Only for Cassandra/ScyllaDB)
Sometimes migrator isn't able to query the historical data by table's partition key. For example, migrator of transactions needs the historical transaction data without knowing each transaction hash. Full table scanner can help to get all the rows in parallel.
Most indexes are based on either ledger states or transactions. We provide the `objects` and `transactions` scanner. Developers only need to implement the callback function to receive the historical data. Please find the examples in `tests/integration/migration/cassandra/ExampleTransactionsMigrator.cpp` and `tests/integration/migration/cassandra/ExampleObjectsMigrator.cpp`.
@@ -65,15 +56,16 @@ Most indexes are based on either ledger states or transactions. We provide the `
## How to write a full table scan adapter (Only for Cassandra/ScyllaDB)
If you need to do full scan against other table, you can follow below steps:
- Describe the table which needs full scan in a struct. It has to satisfy the `TableSpec`(cassandra/Spec.hpp) concept, containing static member:
- Tuple type `Row`, it's the type of each field in a row. The order of types should match what database will return in a row. Key types should come first, followed by other field types sorted in alphabetical order.
- `kPARTITION_KEY`, it's the name of the partition key of the table.
- `kTABLE_NAME`
- Tuple type `Row`, it's the type of each field in a row. The order of types should match what database will return in a row. Key types should come first, followed by other field types sorted in alphabetical order.
- `kPARTITION_KEY`, it's the name of the partition key of the table.
- `kTABLE_NAME`
- Inherent from `FullTableScannerAdapterBase`.
- Implement `onRowRead`, its parameter is the `Row` we defined. It's the callback function when a row is read.
Please take ObjectsAdapter/TransactionsAdapter as example.
## Examples:
@@ -82,13 +74,16 @@ We have some example migrators under `tests/integration/migration/cassandra` fol
- ExampleDropTableMigrator
This migrator drops `diff` table.
This migrator drops `diff` table.
- ExampleLedgerMigrator
This migrator shows how to migrate data when we don't need to do full table scan. This migrator creates an index table `ledger_example` which maintains the map of ledger sequence and its account hash.
This migrator shows how to migrate data when we don't need to do full table scan. This migrator creates an index table `ledger_example` which maintains the map of ledger sequence and its account hash.
- ExampleObjectsMigrator
This migrator shows how to migrate ledger states related data. It uses `ObjectsScanner` to proceed the full scan in parallel. It counts the number of ACCOUNT_ROOT.
This migrator shows how to migrate ledger states related data. It uses `ObjectsScanner` to proceed the full scan in parallel. It counts the number of ACCOUNT_ROOT.
- ExampleTransactionsMigrator
This migrator shows how to migrate transactions related data. It uses `TransactionsScanner` to proceed the `transactions` table full scan in parallel. It creates an index table `tx_index_example` which tracks the transaction hash and its according transaction type.
This migrator shows how to migrate transactions related data. It uses `TransactionsScanner` to proceed the `transactions` table full scan in parallel. It creates an index table `tx_index_example` which tracks the transaction hash and its according transaction type.