Commit Graph

31 Commits

Author SHA1 Message Date
Savinda Senevirathne
a775b0e419 Configs to support full history mode. (#252)
* Adding full history mode and max shard configs for primary and blob shards.

* Commenting validation checks for hpfs responses until bug is fixed.

* Comment update.
2021-02-19 14:45:25 +05:30
Chalith Desaman
7059f68f11 Removed legacy lcl code and dependencies (#249) 2021-02-18 17:07:50 +05:30
Savinda Senevirathne
16c5b3fae2 Introducing ledger shards and new ledger syncing logic. (#247)
- The ledgers are stored in a sqlite database in ledger file system.
- Ledgers are organized in shards inside primary folder.
- Raw inputs are saved as shards inside blob folder. No input files are created if neither inputs nor outputs are available.
- Both last primary shard hash and last blob shard hashes are subjected to consensus and necessary sync operations are performed if out of sync.
- Hpfs sync support setting single sync targets from both ends of the list. (Targets set from front are prioritized).
- Contract and ledger syncs and serving are performed independently.
2021-02-18 11:24:05 +05:30
Ravin Perera
e394497698 Used uint32 for roundtime and timeout fields. (#248) 2021-02-17 12:21:51 +05:30
Ravin Perera
9f4499653a Dynamic roundtime detection. (#244)
When consensus is unreliable detect roundtime based on roundtime reported by peers.
2021-02-16 13:34:29 +05:30
Ravin Perera
067883b778 Support external hpfs process. (#239) 2021-02-09 15:23:03 +05:30
Savinda Senevirathne
a4399544b9 Separate contract and ledger file system mounts. (#235)
- Creating two separate hpfs folders for contract and ledger file systems (contract_fs and ledger_fs).
- Added separate mounts for contract and ledger file systems.
- Added separate instances for contract serving and ledger serving.
- Added separate instances for contract syncing and ledger syncing.
- Modified cluster creating script to accompany folder name changes in contract folder.
2021-02-05 11:48:52 +05:30
Savinda Senevirathne
d08d2630f6 Refactoring hpfs, hpfs sync and hpfs serve code. (#231)
* Refactoring hpfs code to a class so it can support multiple mounts.
* Refactoring hpfs serve into a class to support mulitiple mount serving.
* Refactoring hpfs sync into class to support multiple instances.
* Code improvements in hpfs_sync.
* Taking a sync target list for hpfs syncing target set.
2021-02-02 13:17:21 +05:30
Ravin Perera
11cb57e9ee Keep hp cfg and patch cfg consistent. (#216)
* Copy from hp cfg to patch cfg upon startup.
* Persist to hp cfg whenever patch cfg is updated.
2021-01-10 17:48:12 +05:30
Ravin Perera
e8e7921ac1 Single-process hpfs integration. (#212) 2021-01-06 21:53:39 +05:30
Savinda Senevirathne
bed8205ca8 Config patch change detection. (#211)
* Applying patch file changes to hpcore runtime after patch file change detection.
* Removing unl sync functionality.
* Removing subjecting unl changeset to consensus.
2021-01-06 20:02:14 +05:30
Savinda Senevirathne
c87ae6693d Patch config and hpfs sync refactor. (#208)
* Terminology changes from state to hpfs.
* Syncing hpfs parents in priority order.
* Changing how the hash is calculate in hpfs response validate check.
2021-01-06 10:50:58 +05:30
Ravin Perera
f97d7de530 Moved h32 hash type from hpfs to util. (#204) 2020-12-23 21:58:49 +05:30
Savinda Senevirathne
e835e18d18 New config file structure. (#201) 2020-12-23 18:52:58 +05:30
Chalith Desaman
b77a3fc924 Validate peer state response content against the hash. (#190) 2020-12-10 17:32:18 +05:30
Ravin Perera
fe9e276f8d Consensus deadlock fixes and reliability improvements. (#187)
* Consensus time vote improvements.
* Skipped self proposal in Observer mode.
* Added docker cluster script params.
* Added state/lcl sync abandon threshold.
* Added pubkey display for cluster script.
* Added proposal latency log.
* Added sync completion check to prevent deadlocks.
2020-12-08 15:17:27 +05:30
Ravin Perera
d9517b6164 Reorganized util and hpws source files. (#161) 2020-11-23 23:18:42 +05:30
Ravin Perera
7a4e91b0dd Integrated hpfs in-proc sessions. (#134) 2020-10-23 10:04:45 +05:30
Ravin Perera
4d920a9219 Fixed child process signal behaviours. (#128)
* Restored signal handlers upon fork().
* Improved error handling of hpfs process kill scenario.
* Set pgid for forked processes for graceful sending of SIGINT.
2020-09-26 21:39:26 +05:30
Ravin Perera
5bfa722b24 Converted forked process logging to use stdout. (#126)
Forked processes are now logging to stdout to avoid race conditions with hpcore log file.
2020-09-24 10:40:17 +05:30
priyadharsun
91122474a1 Replaced Boost log with Plog. (#124) 2020-09-23 06:06:42 +05:30
Ravin Perera
180b1ec714 Marked fds to close on exec. (#122) 2020-09-18 11:40:03 +05:30
Ravin Perera
f622a252e6 Improved return codes of hpfs hash access interface. (#121) 2020-09-18 10:36:46 +05:30
Ravin Perera
64efc53cef Added latest hpfs binary and integrated hpfs logging. (#117) 2020-09-15 21:55:13 +05:30
ravinsp
f121fa90fb Fixed process kill waitpid cleanup issues. 2020-09-07 14:34:07 +05:30
Ravin Perera
baf5d8b14a Fixed incorrect memory access issues. (#109) 2020-08-20 15:20:11 +05:30
ravinsp
051e358f14 Fixed hpfs hash update issue. 2020-08-18 22:15:12 +05:30
Ravin Perera
611d362876 Integrated hpfs hash map cache. (#107) 2020-08-16 21:41:14 +05:30
Ravin Perera
1328b06346 Improved hpfs initialisation error handling. (#104) 2020-08-04 12:05:27 +05:30
Ravin Perera
b89dbe0a2c Rearchitected state sync with hpfs. (#96) 2020-06-10 20:51:45 +05:30
Ravin Perera
10374352c6 hpfs integration. (#94) 2020-06-04 13:51:53 +05:30