Commit Graph

49 Commits

Author SHA1 Message Date
Nathan Nichols
ed58095029 add a limit 2021-05-14 09:45:31 -05:00
Nathan Nichols
d2815ec32e make keys unique after merge 2021-05-13 11:41:29 -05:00
Nathan Nichols
db92f420d9 address comments 2021-05-13 09:06:20 -05:00
Nathan Nichols
cabd9e0c45 fix message 2021-05-13 08:26:42 -05:00
Nathan Nichols
9b8e0bdf99 adjust complete query 2021-05-13 07:43:40 -05:00
Nathan Nichols
f2c7d54b0d address pr feedback 2021-05-13 06:40:23 -05:00
Nathan Nichols
945598d953 merge master into branch 2021-05-10 15:10:58 -05:00
CJ Cobb
f32af6bb2d various fixes. make shift info optional in config. Properly check book flag ledger is complete 2021-05-07 20:49:42 +00:00
Nathan Nichols
ce49de4fee impliment book offers for async etl 2021-05-06 23:07:52 -05:00
CJ Cobb
144ad5eb19 allow different shifts for books and keys. default to 10 and 20 respectively 2021-05-06 20:13:04 +00:00
CJ Cobb
20b8059151 fix up some issues with async indexer 2021-05-05 20:35:19 +00:00
CJ Cobb
736e0a675f small fixes in CassandraBackend 2021-05-03 21:19:06 +00:00
CJ Cobb
64d0c5d050 async populate caches 2021-05-03 21:18:38 +00:00
CJ Cobb
971437f456 async write to keys and books with recovery and warnings 2021-05-03 21:18:38 +00:00
CJ Cobb
4e58c76eac Fix crashing bug related to account_tx writes 2021-05-03 21:18:36 +00:00
Nathan Nichols
ee763ec1ea merge master 2021-05-03 12:13:50 -05:00
CJ Cobb
c0612e740e Many changes around ledger_data and book_offers
* populate caches on startup
* implement new algorithms for Postgres
* use one shift interval instead of two
* block etl at flag ledger until writes are finished
* write one flag ledger ahead
* abandon ledger_diff component of ledger_data
2021-04-29 21:30:05 +00:00
CJ Cobb
e747c61873 Merge branch 'ledger_data_indexer' into master_new 2021-04-28 20:12:28 +00:00
CJ Cobb
20184527af Merge branch 'postgres_partitioning' into master_new 2021-04-28 19:55:25 +00:00
CJ Cobb
5ebc515978 support for read replicas with postgres 2021-04-28 19:02:44 +00:00
Nathan Nichols
3c8ecc01f1 impliment cursor for book_offers 2021-04-26 15:35:29 -05:00
Nathan Nichols
0bcf3a4601 Order book offers by quality 2021-04-21 23:27:14 -05:00
CJ Cobb
d9a8ff5399 index during ETL. not tested 2021-04-15 14:10:28 +00:00
CJ Cobb
ed4ce93481 remove mention of keys table 2021-04-15 13:48:47 +00:00
CJ Cobb
5f9e5d03f4 speed up indexer 2021-04-05 15:22:07 +00:00
CJ Cobb
df8b6f15d9 make cassandra driver thread count configurable 2021-04-01 23:43:42 +00:00
CJ Cobb
0d11898730 indexer picks up from where it left off 2021-04-01 23:43:11 +00:00
CJ Cobb
db37c05b7b new book offers algorithm, seems to work 2021-03-31 19:22:53 +00:00
Ubuntu
d27b53e4f7 actually write to keys table 2021-03-30 19:36:55 +00:00
Ubuntu
ef555b00a7 indexer values look good. Nothing is being written yet 2021-03-30 18:35:40 +00:00
CJ Cobb
d1f47b490a iterate through diffs. don't write anything 2021-03-30 11:13:51 -04:00
CJ Cobb
58509093ac make cassandra driver thread count configurable 2021-03-26 08:42:23 -04:00
CJ Cobb
168283f0aa the best ledger_data implementation yet 2021-03-25 21:57:04 -04:00
CJ Cobb
d72867ec64 switch deserialization order 2021-03-24 21:00:49 -04:00
CJ Cobb
14fd6d24ad add ancestry check 2021-03-24 14:46:48 -04:00
CJ Cobb
6eb87bfaff proper parallel extraction 2021-03-24 05:33:27 -04:00
CJ Cobb
564d46cb5f online delete with postgres. seems to work 2021-03-23 15:55:09 -04:00
CJ Cobb
4fc44c5ac8 log parser 2021-03-18 13:05:47 -04:00
CJ Cobb
4609010967 Add database timeout exception. clear out incomplete cassandra data. add ledger_entry rpc 2021-03-12 16:37:27 -05:00
CJ Cobb
c3b7983618 refactor CassandraBackend async bookkeeping 2021-03-10 16:03:56 -05:00
CJ Cobb
f781bd6c00 ledger rpc accepts more flags. binary options for all rpcs. tests to verify ledger and account_info 2021-03-09 16:33:24 -05:00
CJ Cobb
e3a121e571 bug fixes in both backends. add ledger_range rpc. improve test scripts 2021-03-05 16:46:55 -05:00
CJ Cobb
7a6dfe5967 postgres fixes 2021-03-05 11:25:12 -05:00
CJ Cobb
6bbb5579ca Ledger handler 2021-03-04 14:34:55 -05:00
CJ Cobb
5606d4a7dd add type field to config. clean up cassandra code more 2021-03-04 10:36:42 -05:00
CJ Cobb
6d39a5494f implement second version of fetchLedgerPage 2021-03-03 16:48:00 -05:00
CJ Cobb
997e3ac2e7 refactor of CassandraBackend. compiles, not tested 2021-03-03 16:26:34 -05:00
CJ Cobb
308c585801 partial refactor of CassandraBackend 2021-03-02 16:19:14 -05:00
CJ Cobb
8691cff8df rename CassandraFlatMapBackend to CassandraBackend and rename file 2021-03-02 10:23:58 -05:00