Commit Graph

44 Commits

Author SHA1 Message Date
Chalith Desaman
7059f68f11 Removed legacy lcl code and dependencies (#249) 2021-02-18 17:07:50 +05:30
Savinda Senevirathne
16c5b3fae2 Introducing ledger shards and new ledger syncing logic. (#247)
- The ledgers are stored in a sqlite database in ledger file system.
- Ledgers are organized in shards inside primary folder.
- Raw inputs are saved as shards inside blob folder. No input files are created if neither inputs nor outputs are available.
- Both last primary shard hash and last blob shard hashes are subjected to consensus and necessary sync operations are performed if out of sync.
- Hpfs sync support setting single sync targets from both ends of the list. (Targets set from front are prioritized).
- Contract and ledger syncs and serving are performed independently.
2021-02-18 11:24:05 +05:30
Ravin Perera
e394497698 Used uint32 for roundtime and timeout fields. (#248) 2021-02-17 12:21:51 +05:30
Ravin Perera
9f4499653a Dynamic roundtime detection. (#244)
When consensus is unreliable detect roundtime based on roundtime reported by peers.
2021-02-16 13:34:29 +05:30
Ravin Perera
033b5fa7bc User inputs round limit. (#240) 2021-02-14 20:19:53 +05:30
Ravin Perera
94eedcaf5b Added roundtime offset based on contract id. (#238) 2021-02-05 17:48:21 +05:30
Savinda Senevirathne
a4399544b9 Separate contract and ledger file system mounts. (#235)
- Creating two separate hpfs folders for contract and ledger file systems (contract_fs and ledger_fs).
- Added separate mounts for contract and ledger file systems.
- Added separate instances for contract serving and ledger serving.
- Added separate instances for contract syncing and ledger syncing.
- Modified cluster creating script to accompany folder name changes in contract folder.
2021-02-05 11:48:52 +05:30
Savinda Senevirathne
d08d2630f6 Refactoring hpfs, hpfs sync and hpfs serve code. (#231)
* Refactoring hpfs code to a class so it can support multiple mounts.
* Refactoring hpfs serve into a class to support mulitiple mount serving.
* Refactoring hpfs sync into class to support multiple instances.
* Code improvements in hpfs_sync.
* Taking a sync target list for hpfs syncing target set.
2021-02-02 13:17:21 +05:30
Ravin Perera
610b67cec0 Contract execute config flag. (#226)
* Added contract execute flag. Made bin_path optional.
* Updated docker cluster default config.
* Renamed contract config to hp config.
2021-01-22 14:33:07 +05:30
Savinda Senevirathne
146bb1d46c Apply patch file changes only after consensus verification. (#222) 2021-01-19 11:56:33 +05:30
Ravin Perera
11cb57e9ee Keep hp cfg and patch cfg consistent. (#216)
* Copy from hp cfg to patch cfg upon startup.
* Persist to hp cfg whenever patch cfg is updated.
2021-01-10 17:48:12 +05:30
Ravin Perera
e8e7921ac1 Single-process hpfs integration. (#212) 2021-01-06 21:53:39 +05:30
Savinda Senevirathne
bed8205ca8 Config patch change detection. (#211)
* Applying patch file changes to hpcore runtime after patch file change detection.
* Removing unl sync functionality.
* Removing subjecting unl changeset to consensus.
2021-01-06 20:02:14 +05:30
Savinda Senevirathne
c87ae6693d Patch config and hpfs sync refactor. (#208)
* Terminology changes from state to hpfs.
* Syncing hpfs parents in priority order.
* Changing how the hash is calculate in hpfs response validate check.
2021-01-06 10:50:58 +05:30
Ravin Perera
f97d7de530 Moved h32 hash type from hpfs to util. (#204) 2020-12-23 21:58:49 +05:30
Ravin Perera
c25ebe66f4 Util helper func simplification. (#203) 2020-12-23 21:49:13 +05:30
Savinda Senevirathne
e835e18d18 New config file structure. (#201) 2020-12-23 18:52:58 +05:30
Ravin Perera
1a954ad96e Implemented output signature checks. (#200) 2020-12-22 16:55:44 +05:30
Ravin Perera
bfbf41fb32 Fixed issue in ledger sync top ledger removal. (#193)
* History response ledger removal fix.
* Consensus code refactor.
2020-12-14 12:35:07 +05:30
Savinda Senevirathne
0bc53fe23c Review UNL message rules. (#188) 2020-12-09 13:34:40 +05:30
Ravin Perera
fe9e276f8d Consensus deadlock fixes and reliability improvements. (#187)
* Consensus time vote improvements.
* Skipped self proposal in Observer mode.
* Added docker cluster script params.
* Added state/lcl sync abandon threshold.
* Added pubkey display for cluster script.
* Added proposal latency log.
* Added sync completion check to prevent deadlocks.
2020-12-08 15:17:27 +05:30
Savinda Senevirathne
7bf0475b6f Subject unl list to consensus. (#186)
* Unl hash in consensus proposals.
* Unl syncing and sync request serving.
* Adding unl hash to the ledger block.
2020-12-08 15:05:38 +05:30
Chalith Desaman
a87e8a0c7e Subject UNL changeset to consensus. (#183) 2020-12-07 13:06:58 +05:30
Savinda Senevirathne
a421f13d91 Introduce consensus/npl public/private flags to config. (#182)
Npl messages are forwarded and broadcasted only to trusted nodes if npl flag is set to private. If public, messages are broadcasted to all the connected peers.
Consensus proposals are forwarded and broadcasted only to trusted nodes if consensus flag is set to private. If public, proposals are broadcasted to all the connected peers.
2020-12-03 08:12:56 +05:30
Ravin Perera
dfe42f827b Refactored node contract lib control flow. (#176) 2020-11-30 17:39:07 +05:30
Chalith Desaman
2f5d95ae7a Save raw inputs alongside the ledger (#167)
* Saving raw inputs in the ledger.
* Store full history in separate folder.

* Moved full history fb structs to a seperate fbs file

* Removed raw_inputs from the proposal

* Fixed code comments

* Saving raw user input string

* Code refactoring

* Resolved PR comments
2020-11-30 15:42:20 +05:30
Ravin Perera
80392cc995 Persisted contract UNL updates in config file. (#171)
* Refactored config loading and saving.
* Aded unl update persistance.
2020-11-27 09:06:17 +05:30
Ravin Perera
4de1bb2393 Already submitted input detection. (#170)
* Added same nonce/sig for input comparison.
* Added status response withholding for already submitted inputs.
2020-11-26 23:27:40 +05:30
Ravin Perera
d476f787a7 Introduced UNL update control message. (#169) 2020-11-26 22:26:12 +05:30
Ravin Perera
29425095e0 User nonce validation and expiration. (#164) 2020-11-25 07:10:07 +05:30
Ravin Perera
d9517b6164 Reorganized util and hpws source files. (#161) 2020-11-23 23:18:42 +05:30
Ravin Perera
86e2e1211c Introduced consensus round nonce and reintroduced stage 0. (#160)
* Reintroduced stage 0.
* Introduced round nonce for reduced lcl predictability.
2020-11-23 23:00:13 +05:30
Ravin Perera
9c1f62dbde Migrated user inputs from domain sockets to memfd. (#159) 2020-11-23 21:52:05 +05:30
priyadharsun
eb0012bc47 Change epoch milliseconds return type to uint64_t. (#155) 2020-11-19 12:54:33 +05:30
Ravin Perera
1608e9fc49 Contract execution refactor. (#153) 2020-11-18 07:16:17 +05:30
ravinsp
645f0023a0 Fixed consensus proposal time vote filter. 2020-11-14 17:20:09 +05:30
priyadharsun
8f00c5e7d4 Introduced contract control channel with 'terminate' message (#147) 2020-11-10 14:09:24 +05:30
Ravin Perera
ba0cae019d Refactored consensus into 3 rounds. (#144)
* Refactored consensus into 3 stages and removed stage 0.
* Consensus threshold calculation improvements.
* Refactored candidate user input processing.
* Renamed proposal sent timestamp field.
* Introduced comm_session display name.
2020-11-07 15:01:01 +05:30
Savinda Senevirathne
51173e37f2 Support message separation for multiple inputs from same user. (#142) 2020-11-06 10:55:40 +05:30
Ravin Perera
202a6a2715 Ledger and state sync improvements. (#143)
* Increased stage time window.
* Introduced central state tracker.
* Used shared mutex for target state update.
* Refactored lcl and state sync checks in consensus.
* lcl/state sync loop wait time improvements.
2020-11-04 16:01:10 +05:30
Ravin Perera
00069b7a43 Updated ledger creation to report current lcl. (#136) 2020-10-24 07:22:28 +05:30
Ravin Perera
7a4e91b0dd Integrated hpfs in-proc sessions. (#134) 2020-10-23 10:04:45 +05:30
Chalith Desaman
5f40aebf08 NPL message refactor. (#132)
* Implemented feeding and broadcasting npl messages from the contract execution in real-time.
* Replaced npl pipe with domain sockets.
* Refactored npl read and write in nodejs echo contract
2020-10-14 15:18:00 +05:30
Ravin Perera
cb4d0c4f59 Ledger maintenance refactor. (#130)
* Added ledger namespace.
* Thread-safe lcl access and update.
* Refactored history sync and serving into a thread.
* Restructured ledger cache item.
2020-10-08 22:25:47 +05:30