50 Commits

Author	SHA1	Message	Date
CJ Cobb	db0595c083	update cache asynchronously	2021-05-25 20:19:09 +00:00
CJ Cobb	cadf2fa972	checkpoint	2021-05-24 18:23:36 +00:00
CJ Cobb	fa8eb67ca6	move websocket_server_async.cpp	2021-05-20 17:02:23 +00:00
CJ Cobb	24816c021d	online delete for cassandra. doesn't crash, but not sure it works	2021-05-20 16:59:50 +00:00
Nathan Nichols	945598d953	merge master into branch	2021-05-10 15:10:58 -05:00
Nathan Nichols	ce49de4fee	impliment book offers for async etl	2021-05-06 23:07:52 -05:00
CJ Cobb	144ad5eb19	allow different shifts for books and keys. default to 10 and 20 respectively	2021-05-06 20:13:04 +00:00
CJ Cobb	20b8059151	fix up some issues with async indexer	2021-05-05 20:35:19 +00:00
CJ Cobb	64d0c5d050	async populate caches	2021-05-03 21:18:38 +00:00
Nathan Nichols	ee763ec1ea	merge master	2021-05-03 12:13:50 -05:00
CJ Cobb	c0612e740e	Many changes around ledger_data and book_offers ... * populate caches on startup * implement new algorithms for Postgres * use one shift interval instead of two * block etl at flag ledger until writes are finished * write one flag ledger ahead * abandon ledger_diff component of ledger_data	2021-04-29 21:30:05 +00:00
CJ Cobb	8c6fe09590	compilation fix after merge	2021-04-28 20:16:47 +00:00
CJ Cobb	e747c61873	Merge branch 'ledger_data_indexer' into master_new	2021-04-28 20:12:28 +00:00
CJ Cobb	5ebc515978	support for read replicas with postgres	2021-04-28 19:02:44 +00:00
Nathan Nichols	0bcf3a4601	Order book offers by quality	2021-04-21 23:27:14 -05:00
CJ Cobb	d9a8ff5399	index during ETL. not tested	2021-04-15 14:10:28 +00:00
CJ Cobb	990e618fe8	add accum parameter	2021-04-12 15:27:16 +00:00
CJ Cobb	d773fcfdb7	add finish sequence	2021-04-07 13:43:24 +00:00
CJ Cobb	5f9e5d03f4	speed up indexer	2021-04-05 15:22:07 +00:00
CJ Cobb	d2da6051d1	reduce etl queue size based on number of extractor threads	2021-03-25 21:58:33 -04:00
CJ Cobb	168283f0aa	the best ledger_data implementation yet	2021-03-25 21:57:04 -04:00
CJ Cobb	2af6d72d7e	reduce etl queue size based on number of extractor threads	2021-03-25 21:57:04 -04:00
CJ Cobb	59124e7b0a	fix publish bug. check ledger range	2021-03-24 15:00:11 -04:00
CJ Cobb	14fd6d24ad	add ancestry check	2021-03-24 14:46:48 -04:00
CJ Cobb	c74f9654af	inc current sequence in load thread	2021-03-24 13:27:46 -04:00
CJ Cobb	3062b5e678	add avg extract time	2021-03-24 13:13:28 -04:00
Ubuntu	694111a9b7	numExtractors is config	2021-03-24 17:03:15 +00:00
CJ Cobb	6eb87bfaff	proper parallel extraction	2021-03-24 05:33:27 -04:00
CJ Cobb	a7a71ec0c7	etl pipeline	2021-03-24 04:29:54 -04:00
CJ Cobb	564d46cb5f	online delete with postgres. seems to work	2021-03-23 15:55:09 -04:00
CJ Cobb	c6a42fa9f2	change log level of extract phase perf logging	2021-03-23 10:58:43 -04:00
CJ Cobb	2ef28ff9be	enhance test.py and metrics.py	2021-03-22 11:16:10 -04:00
CJ Cobb	aed0f8707f	read only mode. ledger_data makes sure ledger is present first	2021-03-18 11:34:20 -04:00
CJ Cobb	f781bd6c00	ledger rpc accepts more flags. binary options for all rpcs. tests to verify ledger and account_info	2021-03-09 16:33:24 -05:00
CJ Cobb	e3a121e571	bug fixes in both backends. add ledger_range rpc. improve test scripts	2021-03-05 16:46:55 -05:00
CJ Cobb	7a6dfe5967	postgres fixes	2021-03-05 11:25:12 -05:00
CJ Cobb	694ec7bfe5	bug fixes. insert objects in batch	2021-03-04 12:50:36 -05:00
CJ Cobb	308c585801	partial refactor of CassandraBackend	2021-03-02 16:19:14 -05:00
CJ Cobb	6839538b3c	postgres writes	2021-03-01 11:53:53 -05:00
CJ Cobb	009688175e	cleanup and postgres stubs	2021-02-24 14:28:15 -05:00
CJ Cobb	78c6bde902	clean up ledger page interface. wire up account_tx	2021-02-23 16:13:48 -05:00
CJ Cobb	5534d9c75e	created backend interface. Cassandra derives from it	2021-02-23 14:28:38 -05:00
CJ Cobb	6bda1af3e6	bug fixes	2021-02-16 17:35:57 +00:00
CJ Cobb	069fd1a259	fix initial ledger load	2021-02-11 17:09:37 -05:00
CJ Cobb	bb1205fd36	everything in cassandra	2021-02-11 16:46:47 -05:00
CJ Cobb	b3731f54e0	fix books table	2021-01-27 15:39:31 -05:00
CJ Cobb	f754ff9f5a	book_offers in Cassandra	2021-01-26 12:51:49 -05:00
CJ Cobb	176f91e15c	keys table, new ledger_data algorithm	2021-01-20 14:39:32 -05:00
CJ Cobb	bd7ae08bcc	checkpoint. handles account_info	2020-12-22 13:12:26 -05:00
CJ Cobb	80a8ed8d1b	checkpoint	2020-12-18 19:40:38 -05:00