Commit Graph

93 Commits

Author SHA1 Message Date
Ravin Perera
cf695af30b Remove appbill. (#358) 2022-01-27 10:39:18 +05:30
Ravin Perera
8dabfb4a2e IP ban refactor with hpws. (#354)
* Refactored corebill.
* Integrated hpws ban tracking.
* Removed session handler helpers.
* Improved bad behaviour reporting.
2021-12-17 07:08:26 +05:30
Ravin Perera
647759501f Refactored vote status reporting. (#349)
* Refactored global sync status tracking.
* Updated streamer for vote status.
* Docker image now uses 'latest' tag.
* Updated hp version.
2021-09-22 09:28:01 +05:30
Ravin Perera
6dc0776b56 Large cluster optimizations. (#348)
* Added sync log to streamer.
* Fixed ledger closing attempt while syncing.
* Added diagnostic contract.
* Reset to stage 0 on unreliable votes.
* Reduced peer msg age threshold.
* Added health tracking.
* Weakly-connected detection improvement.
* Increased version 0.5.1.
* Improved client lib server version check.
* Added health logging support to text client.
* Added weakly connected status in status response.
* Increased max peers limits when serializing.
* Local docker cluster manual ip.
* Updated vultr script vm region order.
* Sync status reporting improvement.
* Added milliseconds to logging.
2021-09-17 11:53:49 +05:30
Savinda Senevirathne
866a617989 concurrent_read_reqeusts typo fix. (#335) 2021-07-22 19:40:29 +05:30
Ravin Perera
c1e1cd12a8 Input responses while syncing. (#323)
* Send pending input responses after syncing.
* Send input statuses on expired inputs.
* Sqlite no journal.
2021-06-12 15:14:21 +05:30
Ravin Perera
aae1617ec0 Metrics script improvements. (#322) 2021-06-09 13:22:14 +05:30
Ravin Perera
cc11ebd7b3 Event subscription infrastructure and ledger events. (#319)
- Introduced event subscription infrastructure for users.
- Added ability to subscribe to ledger events.
- Updated client lib for event subscription support.
2021-06-07 10:24:58 +05:30
Ravin Perera
a7b7f6c9de Added user message to get lcl. (#318) 2021-06-04 16:16:22 +05:30
Ravin Perera
33bd63ac64 Status tracking infrastructure. (#317)
* Added node's current status information tracker.
* Used the status tracker for responding to status messages.
* Used change-event notifications to broadcast UNL change event.
2021-06-04 15:08:10 +05:30
Ravin Perera
fded7b7917 Introduced priority queues for proposal processing. (#301)
* Introduced high priority queue for comm session message processing.
* Added high priority send for proposals.
2021-05-03 21:44:36 +05:30
Ravin Perera
370e03fe84 Switched user input nonce from string to uint64. (#286) 2021-04-11 14:51:57 +05:30
Ravin Perera
0a3183a3d6 Included output hash and ledger info in output return message. (#284) 2021-04-08 10:42:33 +05:30
Ravin Perera
d4646179c2 Returned input hash and ledger info for input submissions. (#283)
- Introduced input hash which can later be used to query the ledger.
- Returned input hash and ledger info at input submission.
- Updated js client lib to support input hash.
- Updated consensus proposal candidate expiration rules.
2021-04-07 21:22:29 +05:30
Ravin Perera
948113398c Ledger query infrastructure. (#275)
* Added json ledger query param parser.

* Added initial query response creation.

* Updated client lib.

* Implemented get ledger by seq no.

* Added ledger query execution wrappers.

* Included log record info.

* Fixed empty output hash issue.

* Added bson support.

* Added db file existance check.

* Added requesy/reply tracking for queries in client lib.

* Improved multi connection usage in client lib.

* Added genesis ledger query support.

* Updated naming convention of query result fields.

* Comments.

* Used sqlite bind() for query param.

* Used binary hashes in ledger sqlite db.

* Missing const.
2021-03-29 11:20:15 +05:30
Chalith Desaman
363116fc2a Added user input max ledger seq offset limit config (#256)
- Added new config for max input ledger offset inside contract section.
- Updated implementation of getting and updating patch config inside node and c contracts.
- Skip inputs if max ledger offset exceeds, when handling user inputs and nup messages.
2021-02-24 09:19:36 +05:30
Ravin Perera
6b8d60a404 Separated lcl string usage to sequence no and hash. (#251) 2021-02-18 18:28:38 +05:30
Chalith Desaman
7059f68f11 Removed legacy lcl code and dependencies (#249) 2021-02-18 17:07:50 +05:30
Savinda Senevirathne
16c5b3fae2 Introducing ledger shards and new ledger syncing logic. (#247)
- The ledgers are stored in a sqlite database in ledger file system.
- Ledgers are organized in shards inside primary folder.
- Raw inputs are saved as shards inside blob folder. No input files are created if neither inputs nor outputs are available.
- Both last primary shard hash and last blob shard hashes are subjected to consensus and necessary sync operations are performed if out of sync.
- Hpfs sync support setting single sync targets from both ends of the list. (Targets set from front are prioritized).
- Contract and ledger syncs and serving are performed independently.
2021-02-18 11:24:05 +05:30
Ravin Perera
9d1163c8c3 Config flag to disable peer port listening. (#245) 2021-02-16 14:27:03 +05:30
Ravin Perera
033b5fa7bc User inputs round limit. (#240) 2021-02-14 20:19:53 +05:30
Ravin Perera
e1b1382599 Read request configuration options. (#236)
* Ignores read requests when contract execution disabled.
* Added concurrent_read_reqeuests config.
* Improved metric test script.
2021-02-05 14:54:42 +05:30
Savinda Senevirathne
a4399544b9 Separate contract and ledger file system mounts. (#235)
- Creating two separate hpfs folders for contract and ledger file systems (contract_fs and ledger_fs).
- Added separate mounts for contract and ledger file systems.
- Added separate instances for contract serving and ledger serving.
- Added separate instances for contract syncing and ledger syncing.
- Modified cluster creating script to accompany folder name changes in contract folder.
2021-02-05 11:48:52 +05:30
Savinda Senevirathne
d08d2630f6 Refactoring hpfs, hpfs sync and hpfs serve code. (#231)
* Refactoring hpfs code to a class so it can support multiple mounts.
* Refactoring hpfs serve into a class to support mulitiple mount serving.
* Refactoring hpfs sync into class to support multiple instances.
* Code improvements in hpfs_sync.
* Taking a sync target list for hpfs syncing target set.
2021-02-02 13:17:21 +05:30
Ravin Perera
3d36912c25 Added configs to max in connections per remote host. (#229) 2021-01-29 13:05:39 +05:30
Chalith Desaman
73f5eea451 Unl change announcement to connected users. (#224)
Introduced json and bson messages for unl list announcement.
When the unl set is modified send a json or bson unl list message (according to the user protocol) to all the connected users.
2021-01-19 11:33:27 +05:30
Ravin Perera
e8e7921ac1 Single-process hpfs integration. (#212) 2021-01-06 21:53:39 +05:30
Savinda Senevirathne
c87ae6693d Patch config and hpfs sync refactor. (#208)
* Terminology changes from state to hpfs.
* Syncing hpfs parents in priority order.
* Changing how the hash is calculate in hpfs response validate check.
2021-01-06 10:50:58 +05:30
Chalith Desaman
a16eb39d1f User connections enable/disable config flag (#209) 2021-01-06 09:36:48 +05:30
Ravin Perera
c25ebe66f4 Util helper func simplification. (#203) 2020-12-23 21:49:13 +05:30
Savinda Senevirathne
e835e18d18 New config file structure. (#201) 2020-12-23 18:52:58 +05:30
Chalith Desaman
21482a3f8a Input nonce expiry based on max ledger sequence no. (#197) 2020-12-17 17:19:27 +05:30
Ravin Perera
f2ed9040c0 User protocol upgrade and js client lib. (#191)
* Unified js client lib for browser and nodejs.
* Client lib multiple connections support.
* Implemented server challenge response.
* Contract guid and version validation.
* Server key validation.
* User json message encoding improvements.
2020-12-11 11:02:58 +05:30
Savinda Senevirathne
f3055822ed Introduce pubkey/unl tracking to Peer sessions. (#181)
* Adding pubkey in binary to comm sessions and is_unl flag to peer sessions.
* Removing display_name overloads and populating pubkey in user session.
* Changing peer session lookup key from uniqueid to pubkey (binary).
2020-12-02 13:05:06 +05:30
Ravin Perera
332e5a4750 User input json/bson format improvements. (#174)
* Removed hex encoding in json input container.
* Refactored client lib exports.
2020-11-28 21:24:35 +05:30
Ravin Perera
4de1bb2393 Already submitted input detection. (#170)
* Added same nonce/sig for input comparison.
* Added status response withholding for already submitted inputs.
2020-11-26 23:27:40 +05:30
Ravin Perera
29425095e0 User nonce validation and expiration. (#164) 2020-11-25 07:10:07 +05:30
Chalith Desaman
459799760b Added element limits for queue and lists (#162) 2020-11-24 13:17:53 +05:30
Ravin Perera
d9517b6164 Reorganized util and hpws source files. (#161) 2020-11-23 23:18:42 +05:30
Ravin Perera
9c1f62dbde Migrated user inputs from domain sockets to memfd. (#159) 2020-11-23 21:52:05 +05:30
Savinda Senevirathne
bf710c6bde Idle connection timeout config for both peer and user connections. (#158) 2020-11-20 19:48:45 +05:30
Savinda Senevirathne
fe24ef2388 Introduce peer requirement announcement. (#156) 2020-11-20 16:01:46 +05:30
Savinda Senevirathne
9160b2ee15 Improve corebill and graylisting. (#154) 2020-11-19 08:40:15 +05:30
Ravin Perera
3ea0299964 Comm session re-architecture and self comm channel. (#145)
* Introduced self comm channel instead of loopback websocket.
* Introduced comm_session and comm_server inheritance hierarchy.
* Separated peer session and user session classes.
2020-11-08 22:14:08 +05:30
Ravin Perera
ba0cae019d Refactored consensus into 3 rounds. (#144)
* Refactored consensus into 3 stages and removed stage 0.
* Consensus threshold calculation improvements.
* Refactored candidate user input processing.
* Renamed proposal sent timestamp field.
* Introduced comm_session display name.
2020-11-07 15:01:01 +05:30
Ravin Perera
7183383ab7 Replace websocketd/websocat with hpws. (#131) 2020-10-15 17:02:06 +05:30
Ravin Perera
cb4d0c4f59 Ledger maintenance refactor. (#130)
* Added ledger namespace.
* Thread-safe lcl access and update.
* Refactored history sync and serving into a thread.
* Restructured ledger cache item.
2020-10-08 22:25:47 +05:30
ravinsp
ef1a650f91 Added TLS on/off flag for user port. 2020-09-24 11:05:42 +05:30
priyadharsun
91122474a1 Replaced Boost log with Plog. (#124) 2020-09-23 06:06:42 +05:30
Savinda Senevirathne
38ef3cc1b1 Replacing lock_guard from scoped_lock. (#120) 2020-09-18 10:29:34 +05:30