Commit Graph

1666 Commits

Author SHA1 Message Date
CJ Cobb
d9a8ff5399 index during ETL. not tested 2021-04-15 14:10:28 +00:00
CJ Cobb
ed4ce93481 remove mention of keys table 2021-04-15 13:48:47 +00:00
CJ Cobb
d5f21ab295 fix indexes 2021-04-14 21:33:11 +00:00
CJ Cobb
23ba45ea5d remove main tx hash index 2021-04-14 14:00:28 +00:00
CJ Cobb
dde44aa3e0 table partitioning 2021-04-12 21:14:29 +00:00
CJ Cobb
990e618fe8 add accum parameter 2021-04-12 15:27:16 +00:00
CJ Cobb
67f98e8ac9 fix memory leak 2021-04-08 20:09:21 +00:00
CJ Cobb
d773fcfdb7 add finish sequence 2021-04-07 13:43:24 +00:00
CJ Cobb
5f9e5d03f4 speed up indexer 2021-04-05 15:22:07 +00:00
CJ Cobb
4f834fc25f fix memory leak 2021-04-01 23:43:54 +00:00
CJ Cobb
df8b6f15d9 make cassandra driver thread count configurable 2021-04-01 23:43:42 +00:00
CJ Cobb
0d11898730 indexer picks up from where it left off 2021-04-01 23:43:11 +00:00
CJ Cobb
db37c05b7b new book offers algorithm, seems to work 2021-03-31 19:22:53 +00:00
CJ Cobb
7f39378268 test script updates 2021-03-31 15:57:02 +00:00
Ubuntu
d27b53e4f7 actually write to keys table 2021-03-30 19:36:55 +00:00
Ubuntu
ef555b00a7 indexer values look good. Nothing is being written yet 2021-03-30 18:35:40 +00:00
CJ Cobb
d1f47b490a iterate through diffs. don't write anything 2021-03-30 11:13:51 -04:00
Ubuntu
2eca3cac96 enhance test.py 2021-03-26 19:41:52 +00:00
Ubuntu
8aaa365e46 bypass validated and hasLedger checks 2021-03-26 17:14:05 +00:00
CJ Cobb
9e1bc896e7 fix memory leak 2021-03-26 13:05:48 -04:00
CJ Cobb
58509093ac make cassandra driver thread count configurable 2021-03-26 08:42:23 -04:00
CJ Cobb
d2da6051d1 reduce etl queue size based on number of extractor threads 2021-03-25 21:58:33 -04:00
CJ Cobb
168283f0aa the best ledger_data implementation yet 2021-03-25 21:57:04 -04:00
CJ Cobb
2af6d72d7e reduce etl queue size based on number of extractor threads 2021-03-25 21:57:04 -04:00
CJ Cobb
d72867ec64 switch deserialization order 2021-03-24 21:00:49 -04:00
CJ Cobb
29424fd863 remove sparsity algorithm in cassandra ledger page 2021-03-24 20:44:39 -04:00
Ubuntu
a77d13449d update test.py and metrics.py 2021-03-25 00:42:44 +00:00
CJ Cobb
0e23a2e0b5 update metrics.py to print aggregate at end 2021-03-24 15:19:27 -04:00
CJ Cobb
fbd4a19f31 update metrics.py 2021-03-24 15:08:44 -04:00
CJ Cobb
59124e7b0a fix publish bug. check ledger range 2021-03-24 15:00:11 -04:00
CJ Cobb
14fd6d24ad add ancestry check 2021-03-24 14:46:48 -04:00
CJ Cobb
c74f9654af inc current sequence in load thread 2021-03-24 13:27:46 -04:00
CJ Cobb
3062b5e678 add avg extract time 2021-03-24 13:13:28 -04:00
Ubuntu
694111a9b7 numExtractors is config 2021-03-24 17:03:15 +00:00
CJ Cobb
6eb87bfaff proper parallel extraction 2021-03-24 05:33:27 -04:00
CJ Cobb
a7a71ec0c7 etl pipeline 2021-03-24 04:29:54 -04:00
CJ Cobb
c69fb9672e update cmake version 2021-03-24 01:54:45 -04:00
CJ Cobb
564d46cb5f online delete with postgres. seems to work 2021-03-23 15:55:09 -04:00
CJ Cobb
c6a42fa9f2 change log level of extract phase perf logging 2021-03-23 10:58:43 -04:00
CJ Cobb
2ef28ff9be enhance test.py and metrics.py 2021-03-22 11:16:10 -04:00
CJ Cobb
4fc44c5ac8 log parser 2021-03-18 13:05:47 -04:00
CJ Cobb
8d021f2019 don't cascade delete to objects and books 2021-03-18 11:34:20 -04:00
CJ Cobb
aed0f8707f read only mode. ledger_data makes sure ledger is present first 2021-03-18 11:34:20 -04:00
CJ Cobb
f476777ef5 fix book_offers postgres issue 2021-03-18 11:34:20 -04:00
CJ Cobb
6f06c60eef fix book offers paging 2021-03-18 11:34:20 -04:00
CJ Cobb
d84c593064 add ledger entry handler (should have been added in prior commit) 2021-03-12 17:31:12 -05:00
CJ Cobb
9281ffb6e6 add cursor to offers response. verify offers work 2021-03-12 17:30:50 -05:00
CJ Cobb
4609010967 Add database timeout exception. clear out incomplete cassandra data. add ledger_entry rpc 2021-03-12 16:37:27 -05:00
CJ Cobb
d2f0537f02 handle postgres timeouts 2021-03-11 16:44:43 -05:00
CJ Cobb
d62f7accfe fix race condition in Postgres backend. add cursor to account_tx response. test script verifies account_tx 2021-03-11 13:14:21 -05:00