Commit Graph

165 Commits

Author SHA1 Message Date
Nik Bougalis
dd902292ed Consensus cleanup:
* Inject dependencies, make functions free and levelize
* Add comments to document the intent behind the code
* Reduce class public interfaces
* Remove support for proposals without ledger hashes
2015-05-29 13:44:44 -04:00
Josh Juran
0dd6b95ac2 Add Validator Manifests (RIPD-772):
A Validator Manifest allows validators to use a generated ed25519
secret key as a master key for generating new validator public/secret
key pairs. Using this mechanism, rippled instances trust the master
ed25519 public key instead of the now-ephemeral validator public key.

Through a new message and propagation scheme, this lets a validator
change its ephemeral public key without requiring that all rippled
instances on the network restart after maintaining the configuration
file.
2015-05-28 07:22:32 -07:00
Josh Juran
7d75041fb1 Fixes to cryptographic containers:
* Include Sign.cpp in unity builds
* Fix AnySecretKey publicKey and sign
* Make AnyPublicKey copyable
* Define op != for Slice
* Overload op << for Slice
* Overload op << for KeyType
2015-05-27 13:25:20 -07:00
Howard Hinnant
155fcdbcd0 Change typedef to using.
Conflicts:
	src/ripple/app/TODO.md
	src/ripple/app/ledger/Ledger.h
	src/ripple/protocol/Protocol.h
2015-05-22 11:09:50 -07:00
Vinnie Falco
52f298f150 Tidy up app unity sources 2015-05-22 10:58:16 -07:00
Vinnie Falco
860ab3523c Refactor Ledger:
* Consolidate constructors
* Use pointer/size instead of Serializer
* Factor out PendingSaves
* Make some members free functions
* Reduce size of constructor initializer lists
2015-05-22 10:58:04 -07:00
Scott Schurr
d6ef66646f 2-level transaction multi-signatures (RIPD-182):
This commit provides support for 2-level multi-signing of
transactions.  The ability is usually compiled out, since other
aspects of multi-signing are not yet complete.

Here are the missing parts:

 o Full support for Tickets in transactions.
 o Variable fees based on the number of signers,
 o Multiple SignerLists with access control flags on accounts,
 o Enable / disable operations based on access control flags,
 o Enable / disable all of multi-signing based on an amendment,
 o Integration tests, and
 o Documentation.
2015-05-20 13:24:06 -07:00
Scott Schurr
cf1638e6de sign_for RPC command (RIPD-182):
The sign_for RPC command returns a field suitable for
signing a multi-signed transaction.  Actual multi-signed
transactions are in the next commit.
2015-05-20 13:23:59 -07:00
Scott Schurr
64ebd64d2b SignerListSet txn and InnerObjectFormats (RIPD-182):
Add support for the SignerListSet transaction as a step toward
multi-sign support.

As part of the SignerListSet implementation, add InnerObjectFormat
templates (similar to TxFormats and LedgerFormats) and enforce them
in STObject, STArray, and STParsedJSON.
2015-05-20 13:23:52 -07:00
Vinnie Falco
2f3834359e Modularize app/tx:
* Move LedgerView declaration
* Move CrossType
* Move Clock declaration
* Move Quality to protocol/
* Move Amounts to protocol/
* Move book to tx/impl
* Remove 0.27 legacy support
* Remove unused AmountType
* Remove core namespace
* Use STAmount
2015-05-20 12:34:19 -07:00
Vinnie Falco
18299c3f7a Tidy up PeerSet:
* Move PeerSet to overlay/
* Remove unused functions
* Make some public members private
* Rename some functions
* Add comments
2015-05-11 12:06:14 -07:00
Miguel Portilla
ca07a1230b Add filtering to Account Objects (RIPD-868) 2015-05-11 11:58:35 -07:00
Josh Juran
daf4f8fcde Remove wallet_accounts and generator maps (RIPD-804):
* Remove the deprecated wallet_accounts command.
 * Remove dead code for generator maps.
 * Remove the help for the obsolete wallet_add and wallet_claim commands
   (which have already been removed).
2015-05-06 13:11:24 -07:00
seelabs
8377f2516b Cache and apply account credits after payment processing (RIPD-821):
Credits made to any account during the processing of a payment are delayed until
the payment completes, enforcing a new invariant: liquidity for any paths
during a payment's execution may never increase. This eliminates the need for special
code to handle a variety of corner cases where consuming liquidity in one path
increases liquidity in others.
2015-04-29 14:34:52 -04:00
seelabs
a14f29f84f Remove obsolete code 2015-04-23 16:47:23 -04:00
Nik Bougalis
67c666b033 Clean up LedgerEntrySet and TransactionEngine:
* Reduce public interfaces
* Remove wrapper functions
* Remove freeze timed cutover code
* Return results directly instead of via ref parameters
2015-04-23 16:47:19 -04:00
Vinnie Falco
99c2fac143 STVar: optimized storage for STObject (RIPD-825):
This introduces the STVar container, capable of holding any STBase-derived
class and implementing a "small string" optimization. STObject is changed
to store std::vector<STVar> instead of boost::ptr_vector<STBase>. This
eliminates a significant number of needless dynamic memory allocations and
deallocations during transaction processing when ledger entries are
deserialized. It comes at the expense of larger overall storage requirements
for STObject.
2015-04-20 15:54:26 -07:00
Miguel Portilla
4d0ed3d857 RPC account_objects (RIPD-777)
General RPC command that can retrieve objects in the account root.
  * Add account objects integration test.
  * Support tickets.

* Add removeElement in Json::Value
2015-04-20 15:54:09 -07:00
Torrie Fischer
a90bb53cd2 Drop nexmo SMS support. Reverts 58b3cc1d. 2015-04-10 19:11:27 -07:00
Nik Bougalis
b2a9c79de5 Fix transaction enumeration in account_tx (RIPD-734):
In some corner cases, an incorrect resume marker could be
returned, preventing the complete enumeration of account
transactions.

* Robust markers via improved paging support
* New unit tests
* Cleanup
2015-04-10 19:00:22 -07:00
Nik Bougalis
a7efdb4e52 Improve version switchover semantics:
* Support PreviousTxnID until the switchover
* Implement "No Ripple" for issue_iou and redeem_iou.
* Do not utilize issue_iou and redeem_iou from legacy code
* Rename 0.27.x legacy files to account for VS build process
* Misc. cleanups
2015-04-10 18:56:52 -07:00
David Schwartz
1fedede771 Remove transaction set acquire logic from consensus object
This creates a new InboundTransactions object that handles transaction sets,
removing this responsibility from the consensus object. The main benefit is
that many inbound transaction operations no longer require the master lock.

Improve logic to decide which peers to query, when to add more peers, and
when to re-query existing peers.
2015-03-31 21:50:38 -04:00
Edward Hennis
35a8ce2349 Pathfinding unit tests:
* Refactor ripple path find to be more testable.
* Reimplements the first 4 tests from `tests\path-test.js`
* Verify balances in Ledger test.
2015-03-26 12:38:33 -04:00
JoelKatz
185b1a3d36 Add noripple_check RPC command
To help gateways make the changes needed to adjust to the
"default ripple" flag, we've added the "noripple_check"
RPC command. This command tells gateways what they need
to do to set this flag and fix any trust lines created
before they set the flag.

Once your server is running and synchronized, you can run
the tool from the command line with a command like:
  rippled json noripple_check '
  {
    "account" : "<gateway_trusted_address_here>",
    "role" : "gateway",
    "transactions" : "true"
  }'

The server will respond with a list of "problems" that it
sees with the configuration of the account and its trust
lines. It will also return a "transactions" array suggesting
the transactions needed to fix the problems it found.
2015-03-19 07:41:57 -07:00
seelabs
97623d20c5 Use soci in more places:
* Validator, peerfinder, SHAMapStore,
  RpcDB, TxnDB, LedgerDB, WalletDB use soci backend.
2015-03-18 19:39:26 -07:00
seelabs
d37802a42f Remove SqliteFactory. 2015-03-18 19:37:09 -07:00
seelabs
9b837a24aa Remove beast's sqdb module.
Conflicts:
	src/beast/beast/module/sqdb/api/backend.h
	src/beast/beast/module/sqdb/api/blob.h
	src/beast/beast/module/sqdb/api/into.h
	src/beast/beast/module/sqdb/api/session.h
	src/beast/beast/module/sqdb/api/statement.h
	src/beast/beast/module/sqdb/api/transaction.h
	src/beast/beast/module/sqdb/api/type_conversion_traits.h
	src/beast/beast/module/sqdb/api/use.h
	src/beast/beast/module/sqdb/detail/error_codes.h
	src/beast/beast/module/sqdb/detail/exchange_traits.h
	src/beast/beast/module/sqdb/detail/into_type.h
	src/beast/beast/module/sqdb/detail/once_temp_type.h
	src/beast/beast/module/sqdb/detail/prepare_temp_type.h
	src/beast/beast/module/sqdb/detail/ref_counted_prepare_info.h
	src/beast/beast/module/sqdb/detail/ref_counted_statement.h
	src/beast/beast/module/sqdb/detail/statement_imp.h
	src/beast/beast/module/sqdb/detail/type_conversion.h
	src/beast/beast/module/sqdb/detail/type_ptr.h
	src/beast/beast/module/sqdb/detail/use_type.h
	src/beast/beast/module/sqdb/sqdb.h
2015-03-18 19:37:08 -07:00
seelabs
d0ef2f7dd8 Use soci in some places:
* Brings the soci subtree into rippled.
* Validator, peerfinder, and SHAMapStore use new soci backend.
* Optional postgresql backend for soci (if POSTGRESQL_ROOT env var is set).
2015-03-18 19:37:08 -07:00
Nik Bougalis
8f88d915ba Support switchover from 0.27 to 0.28 processing semantics based on time:
Changes made to support autobridging and improve the offer-crossing and
pathfinding logic result in transaction-breaking changes which cause
incompatibilities between 0.27 and 0.28 builds of RippleD.

This patch simplifies deployment of 0.28 on the Ripple network by allowing
RippleD to emulate the 0.27 semantics while the last closed ledger closed
before March 30, 2015 at 13:00:00 PDT, after which time the new 0.28
semantics will become active.

The transaction-breaking changes addressed in this commit are:
    3ccbd7c9b2
    b203db27a4
2015-03-16 20:54:12 -04:00
Josh Juran
d082a0696d Support Ed25519 keys and signatures:
Recognize a new JSON parameter `key_type` in handlers for wallet_propose
and sign/submit.  In addition to letting the caller to specify either of
secp256k1 or ed25519, its presence prohibits the (now-deprecated) use of
heuristically polymorphic parameters for secret data -- the `passphrase`
parameter to wallet_propose will be not be considered as an encoded seed
value (for which `seed` and `seed_hex` should be used), and the `secret`
parameter to sign and submit will be obsoleted entirely by the same trio
above.

* Use constants instead of literals for JSON parameter names.
* Move KeyType to its own unit and add string conversions.
* RippleAddress
  * Pass the entire message, rather than a hash, to accountPrivateSign()
    and accountPublicVerify().
  * Recognize a 33-byte value beginning with 0xED as an Ed25519 key when
    signing and verifying (for accounts only).
  * Add keyFromSeed() to generate an Ed25519 secret key from a seed.
  * Add getSeedFromRPC() to extract the seed from JSON parameters for an
    RPC call.
  * Add generateKeysFromSeed() to produce a key pair of either type from
    a seed.
* Extend Ledger tests to cover both key types.
2015-03-12 21:53:59 -07:00
Tom Ritchford
e9b7003cf5 Move streaming Json objects to ripple/json. 2015-03-02 16:49:56 -05:00
Tom Ritchford
9f64ad8d89 Add WebSocket 04 interface.
* New WebSocket04 traits class implements strategies.
* New "websocket_version" configuration setting selects between 0.2 and 0.4.
2015-02-28 14:57:38 -05:00
Tom Ritchford
9c3522cb70 Isolate WebSocket 0.2-specific code.
* Hide implementation details of the WebSocket server from clients.
* Extract a generic traits class.
2015-02-28 14:57:38 -05:00
Tom Ritchford
c66fc2f656 Rename all WebSocket code into one directory. 2015-02-27 12:11:54 -05:00
Nicholas Dudfield
91871b418b Changes to Universal Port:
* Add tests
* Introduce requestRole helper
* Always honor admin=no
* Welcome guests anywhere admin privileges aren't required
2015-02-25 19:46:56 -05:00
Vinnie Falco
a2acffdfa3 New serialized object, public key, and private key interfaces
This introduces functions get and set, and a family of specialized
structs called STExchange. These interfaces allow efficient and
seamless interchange between serialized object fields and user
defined types, especially variable length objects.

A new base class template TypedField is mixed into existing SField
declarations to encode information on the field, allowing template
metaprograms to both customize interchange based on the type and
detect misuse at compile-time.

New types AnyPublicKey and AnySecretKey are introduced. These are
intended to replace the corresponding functionality in the deprecated
class RippleAddress. Specializations of STExchange for these types
are provided to allow interchange. New free functions verify and sign
allow signature verification and signature generation for serialized
objects.

* Add Buffer and Slice primitives
* Add TypedField and modify some SField
* Add STExchange and specializations for STBlob and STInteger
* Improve STBlob and STInteger to support STExchange
* Expose raw data in RippleAddress and Serializer
2015-02-24 20:28:43 -05:00
Nik Bougalis
558c6b621b Clean up JSON code:
* Remove obsolete files
* Remove obsolete preprocessor configuration tags and decorations
* Remove arcane functionality (YAML compatibility, comment support)
* Enforce strict mode (single root)
* Improve parsing of numerical types
* Misc. cleanups
2015-02-23 14:36:34 -05:00
Edward Hennis
6d91d02c62 Unit test simulated ledgers can do signature verification. 2015-02-23 14:34:37 -05:00
Josh Juran
436ded68b7 Add unit tests for wallet keypair generation:
* Allow `passphrase` to be a seed encoded in any of three formats or a
    literal passphrase.
  * Recognize the absence of `passphrase` as requesting a random seed.

Extract walletPropose() and keypairForSignature() as separately factored
functions (from doWalletPropose() and transactionSign() respectively) to
facilitate unit testing.
2015-02-23 14:34:35 -05:00
Vinnie Falco
8bda9487c6 Add shamap::Family injection bundle 2015-02-18 13:31:18 -05:00
seelabs
617d84c0ef BasicConfig support for legacy values:
* A legacy value is a config section with a single-line.
* These values may be read from the BasicConfig interface so
  the deprecated Config class does not need to be exposed to
  clients.
* Made Config class more testable.
2015-02-18 13:31:18 -05:00
Josh Juran
7a6d533014 Refactor GenerateDeterministicKey and its call sites:
Remove the use of ec_key parameters and return values from ECDSA crypto
prototypes.  Don't store key data into an ec_key variable only to fetch
it back into the original type again.  Use uint256 and Blob explicitly.

Pass private keys as uint256, and pass public keys as either pointer and
length or Blob in calls to ECDSA{Sign,Verify}() and {en,de}cryptECIES().

Replace GenerateRootDeterministicKey() with separate functions returning
either the public or private key, since no caller needs both at once.

Simplify the use of GenerateDeterministicKey within RippleAddress.  Call
a single routine rather than pass the result of one as input to another.

Add openssl unit with RAII classes for bignum, bn_ctx, and ec_point plus
free utility functions.

Rewrite the functions in GenerateDeterministicKey.cpp to use RAII rather
than explicit cleanup code:
  * factor out secp256k1_group and secp256k1_order for reuse rather than
    computing them each time
  * replace getPublicKey() with serialize_ec_point(), which makes, sets,
    and destroys an ec_key internally (sparing the caller those details)
    and calls i2o_ECPublicKey() directly
  * return bignum rather than ec_key from GenerateRootDeterministicKey()
  * return ec_point rather than EC_KEY* from GenerateRootPubKey()

Move ECDSA{Private,Public}Key() to a new ECDSAKey unit.
Move ec_key.h into impl/ since it's no longer used outside crypto/.

Remove now-unused member functions from ec_key.

Change tabs to spaces; trim trailing whitespace (including blank lines).
2015-02-11 20:42:38 -05:00
Vinnie Falco
2a201f9525 Add RocksDB to nudb import tool (RIPD-781,785):
This custom tool is specifically designed for very fast import of
RocksDB nodestore databases into NuDB.
2015-02-11 20:14:44 -05:00
Vinnie Falco
feaa0871ac Add lz4 2015-02-07 06:38:37 -08:00
Vinnie Falco
eade9f8f2b Revert RocksDB backend settings:
This reverts the change that makes RocksDBQuick the default settings for
node_db "type=rocksdb". The quick settings can be obtained by setting
"type=rocksdbquick".

RocksDBQuick settings are implicated in memory over-utilization problems
seen recently.
2015-02-03 16:24:34 -08:00
Tom Ritchford
c3ae4da83a Fix include guards in rippled. 2015-02-02 17:01:17 -08:00
Tom Ritchford
c3809ece67 New RPC method "version". 2015-02-02 17:01:17 -08:00
Vinnie Falco
f3c1f63444 Remove LevelDB and HyperLevelDB backends:
The LevelDB and HyperLevelDB are removed from the backend choices. Neither
were recommended for production environments. As RocksDB is not available
on Windows platforms yet, the recommended backend choice for Windows is NuDB.
2015-01-28 13:43:00 -08:00
Miguel Portilla
0e4de42be8 Add RPC metrics (RIPD-705)
Add metrics to record the number of RPC requests received. Record the number of
node store fetches performed per request. Additionally record the byte size of
each request response and measure the response time of each request in
milliseconds.

A new class, ScopedMetrics, uses the Boost Thead Local Storage mechanism to
efficiently record NodeStore metrics within the same thread.
2015-01-26 19:25:38 -08:00
Vinnie Falco
890bf3cce1 Add PeerFinder Logic backoff unit test 2015-01-26 19:13:40 -08:00