Commit Graph

24 Commits

Author SHA1 Message Date
CJ Cobb
e7ce3909d2 gRPC support for account_tx and tx
- Add support for all transaction types and ledger object types to gRPC
  implementation of tx and account_tx.

- Create common handlers for tx and account_tx.

- Remove mutex and abort() from gRPC server. JobQueue is stopped before
  gRPC server, with all coroutines executed to completion, so no need for
  synchronization.
2020-02-25 19:21:55 -08:00
Edward Hennis
2c71802e38 Propagate validator lists (VLs or UNLs) over the peer network:
* Whenever a node downloads a new VL, send it to all peers that
  haven't already sent or received it. It also saves it to the
  database_dir as a Json text file named "cache." plus the public key of
  the list signer. Any files that exist for public keys provided in
  [validator_list_keys] will be loaded and processed if any download
  from [validator_list_sites] fails or no [validator_list_sites] are
  configured.
* Whenever a node receives a broadcast VL message, it treats it as if
  it had downloaded it on it's own, broadcasting to other peers as
  described above.
* Because nodes normally download the VL once every 5 minutes, a single
  node downloading a VL with an updated sequence number could
  potentially propagate across a large part of a well-connected network
  before any other nodes attempt to download, decreasing the amount of
  time that different parts of the network are using different VLs.
* Send all of our current valid VLs to new peers on connection.
  This is probably the "noisiest" part of this change, but will give
  poorly connected or poorly networked nodes the best chance of syncing
  quickly. Nodes which have no http(s) access configured or available
  can get a VL with no extra effort.
* Requests on the peer port to the /vl/<pubkey> endpoint will return
  that VL in the same JSON format as is used to download now, IF the
  node trusts and has a valid instance of that VL.
* Upgrade protocol version to 2.1. VLs will only be sent to 2.1 and
  higher nodes.
* Resolves #2953
2020-02-12 10:19:23 -08:00
CJ Cobb
7d867b806d Add gRPC support (#3127):
* add support for AccountInfo, Fee and Submit RPCs

* add partial support for Tx RPC (only supports Payments)
2020-01-10 12:31:24 -08:00
Nik Bougalis
f6916bfd42 Improve protocol-level handshaking protocol:
This commit restructures the HTTP based protocol negotiation that `rippled`
executes and introduces support for negotiation of compression for peer
links which, if implemented, should result in significant bandwidth savings
for some server roles.

This commit also introduces the new `[network_id]` configuration option
that administrators can use to specify which network the server is part of
and intends to join. This makes it possible for servers from different
networks to drop the link early.

The changeset also improves the log messages generated when negotiation
of a peer link upgrade fails. In the past, no useful information would
be logged, making it more difficult for admins to troubleshoot errors.

This commit also fixes RIPD-237 and RIPD-451
2019-11-28 09:46:17 -08:00
Miguel Portilla
2e26377e7c Use public key when routing shard crawl requests 2019-04-15 12:39:08 -07:00
Miguel Portilla
3661dc88fe Add RPC command shard crawl (RIPD-1663) 2018-10-10 12:16:01 -04:00
Mike Ellery
08382d866b Support ipv6 for peer and RPC comms:
Fixes: RIPD-1574

Alias beast address classes to the asio equivalents. Adjust users of
address classes accordingly. Fix resolver class so that it can support
ipv6 addresses. Make unit tests use ipv6 localhost network. Extend
endpoint peer message to support string endpoint
representations while also supporting the existing fields (both are
optional/repeated types). Expand test for Livecache and Endpoint.
Workaround some false positive ipaddr tests on windows (asio bug?)
Replaced usage of address::from_string(deprecated) with free function
make_address. Identified a remaining use of v4 address type and
replaced with the more appropriate IPEndpoint type (rpc_ip cmdline
option). Add CLI flag for using ipv4 with unit tests.

Release Notes
-------------

The optional rpc_port command line flag is deprecated. The rpc_ip
parameter now works as documented and accepts ip and port combined.
2018-06-19 09:32:54 -07:00
Miguel Portilla
718d217158 Implement Shards 2018-01-17 13:43:54 -08:00
MarkusTeufelberger
af79c9007e Specify syntax version for ripple.proto file:
This change eliminates a warning about unspecified syntax version when using
a newer proto3 compiler.
2017-03-01 13:18:29 -05:00
wilsonianb
cdf470e68d Forward manifests from new peer (RIPD-1325):
Previously, manifests sent to new peers were marked as history so that
they would not be forwarded. However, this prevented a starting up
node's new manifest from being forwarded beyond its directly connected
peers. Stale or invalid manifests are still not forwarded.
2016-12-23 14:36:10 -08:00
JoelKatz
0c05bd3def Improve transport security:
* Add fields for local and remote IP addresses in hello.
* Add configuration for known local public IP address
* Set fields appropriately
* Check the fields
* Disallow self connection by key
2015-09-28 17:24:05 -07:00
JoelKatz
36a62f110c Switch some Ledger instances to ReadView instances
* Remove ltCURRENT
* Change getOwnerInfo
* Use ReadView in TransactionSign
* Change AcceptedLedger and ProposedTransaction to use ReadView
* Change RPC::accounts
2015-07-29 11:46:01 -04:00
Nik Bougalis
dd902292ed Consensus cleanup:
* Inject dependencies, make functions free and levelize
* Add comments to document the intent behind the code
* Reduce class public interfaces
* Remove support for proposals without ledger hashes
2015-05-29 13:44:44 -04:00
seelabs
1b4e0f5f48 Tidying & Selectively forward manifests to peers:
* Do not forward manifests to peers that already know that manifest
* Do not forward historical manifests to peers
* Save/Load ValidatorManifests from a database
* Python test for setting ephmeral keys
* Cleanup manifest interface
2015-05-28 08:16:56 -07:00
Josh Juran
0dd6b95ac2 Add Validator Manifests (RIPD-772):
A Validator Manifest allows validators to use a generated ed25519
secret key as a master key for generating new validator public/secret
key pairs. Using this mechanism, rippled instances trust the master
ed25519 public key instead of the now-ephemeral validator public key.

Through a new message and propagation scheme, this lets a validator
change its ephemeral public key without requiring that all rippled
instances on the network restart after maintaining the configuration
file.
2015-05-28 07:22:32 -07:00
JoelKatz
b1881e798b Control query depth based on latency:
This changes TMGetLedger protocol in a backward-compatible way to include
a "query depth" parameter - the number of extra levels in the SHAMap tree
that a server should return in the corresponding TMLedgerData. Depending
on the value or absence of the field, a server may adjust the amount of
returned data based on the observed latency of the requestor: higher
latencies will return larger data sets (to compensate for greater
request/response turnaround times).
2015-05-13 12:40:16 -07:00
Vinnie Falco
90bb53af20 Structured Overlay support for TTL limited messages:
When the [overlay] configuration key "expire" is set to 1, proposals
and validations will include a hops field. The hops is incremented with
each relay. Messages with a hop count will be dropped when they exceed
the TTL (Time to Live). Messages containing a hops field will not be
relayed or broadcast to older versions of rippled that don't understand
the field.

This change will not affect normal operation of the network or rippled
instances that do not set "expire" to 1.
2015-04-29 14:34:54 -04:00
Vinnie Falco
08a81a0ab9 Tidy up the structure of sources in protocol/:
Split out and rename STValidation
Split out and rename STBlob
Split out and rename STAccount
Split out STPathSet
Split STVector256 and move UintTypes to protocol/
Rename to STBase
Rename to STLedgerEntry
Rename to SOTemplate
Rename to STTx
Remove obsolete AgedHistory
Remove types.h and add missing includes
Remove unnecessary includes in app.h
Remove unnecessary includes in app.h
Remove include app.h from app1.cpp
2014-11-20 20:15:29 -08:00
JoelKatz
65125eac87 Add "deferred" flag to transaction relay message
If we receive a deferred transaction from a server in our
cluster, treat it as if it wasn't received from a server
in our cluster.

This currently has no effect but is needed for server to
interoperate with future code that will relay deferred
transactions.
2014-10-22 19:29:28 -07:00
Vinnie Falco
5f59282ba1 Clean up Overlay and PeerFinder sources:
* Tidy up identifiers and declarations
* Merge PeerFinder headers into one file
* Merge handout classes and functions into one file
2014-10-10 15:04:36 -07:00
Vinnie Falco
d6066183b9 Refactor Overlay for Structured Network support:
* Move overlay up one directory
* Add abstract_protocol_handler, message_stream
* Add peer_protocol_detector
* Tidy up some declarations
* Use strand::running_in_this_thread instead of bool
* Update README.md
* Replace protocol message read loop:
  - Process data in arbitrary size chunks
  - message_stream extracts individual messages
  - peer_protocol_detector identifies the handshake
  - abstract_protocol_handler used for dispatching messages
* Remove unused protocol message types:
  - mtACCOUNT
  - mtCONTACT
  - mtERROR
  - mtGET_ACCOUNT
  - mtGET_CONTACTS
  - mtGET_VALIDATIONS
  - mtSEARCH_TRANSACTION
  - mtUNUSED_FIELD

Conflicts:
	src/ripple/module/app/main/Application.cpp
	src/ripple/module/app/misc/NetworkOPs.cpp
	src/ripple/module/app/peers/PeerSet.cpp
2014-06-18 15:17:18 -07:00
Vinnie Falco
4f1d1d2a8a Reorganize source file hierarchy:
* Rename unity files
* Move some modules to new subdirectories
* Remove obsolete Visual Studio project files
* Remove obsolete coding style and TODO list
2014-06-03 21:43:59 -07:00
Vinnie Falco
97c9d02c43 Improved SConstruct:
* Automatic source list built via directory iteration
* Build multiple toolchains and flavors simulaneously:
    - Toolchains: gcc, clang, msvc
    - Flavors: debug, release
* Documentation on aliases (top of the SConstruct file)
2014-05-27 15:23:50 -07:00
Vinnie Falco
a10c48182f Move .proto and generated sources 2014-05-01 09:53:42 -07:00