Add binary formats and SQLite database layouts.

Google's protocol buffers cannot be used for the inner portions of
signed/hashed objects.

This SQL database layout will facilitate a cache-based transaction
processing architecture that will scale to billions of accounts and dozens
of transactions per second.

BinaryFormats.txt

@@ -0,0 +1,123 @@
+
+All signed or hashed objects must have well-defined binary formats at the
+byte level. These formats do not have to be the same as the network wire
+formats or the forms used for efficient storage or human display. However,
+it must always be possible to precisely re-create, byte for byte, any signed
+or hashed object. Otherwise, the signatures or hashes cannot be validated.
+
+All 32-byte hashes are the first 32-bytes of a SHA512 hash. This is chosen
+because SHA512 is 33% faster than SHA256 on modern hardware.
+
+Note that it is not required the objects be stored or transmitted in these
+formats. They are only required to be put in these formats when computing or
+validating hashes or signatures.
+
+Sadly, Google's protocol buffers are not suitable for the inner portions of
+signed or hashed objects.
+
+
+1) Account ID
+
+An account starts as an ECDSA public key (X,Y)
+
+Form a 65-byte quantity consisting of a single byte with the value 4, the
+32-byte X of the public key followed by the 32-byte Y of the public key.
+Apply SHA256 to this 65-byte quantity. Apply RIPEMD160 to the result.
+
+The resulting 20-byte value is the account ID.
+
+
+
+2) Transaction (source/signed format)
+
+Fields:
+1) 64-byte source public key
+2) 20-byte destination account
+3) 8-byte amount, unsigned BE integer
+4) 4-byte source account sequence number, unsigned BE integer
+5) 4-byte source ledger index, unsigned BE integer
+6) 4-byte arbitrary source tag, unsigned BE integer
+7) Signature of the 104-byte contents of fields 1-6
+
+
+3) Transaction (ledger format)
+
+Fields:
+1) Transaction in signed format
+2) 16-byte fees held, unsigned BE integer
+
+
+
+4) Ledger (signed format)
+
+Fields:
+1) 4-byte ledger index, unsigned BE integer
+2) 8-byte fees held, unsigned BE integer
+3) 32-byte hash of previous ledger
+4) 32-byte hash of root of the transaction tree for this ledger
+5) 32-byte hash of root of the account tree for this ledger
+[remaining fields only in proposed ledger
+6) 8-byte timestamp when ledger is proposed
+7) 4-byte confidence, unsigned BE integer x/255
+
+
+5) Account status (ledger format)
+
+Fields:
+1) 20-byte Account ID
+2) 8-byte account balance, unsigned BE integer
+3) 4-byte account sequence, unsigned BE integer
+
+
+
+6) Non-Leaf Tree Node
+
+Contains 32 hashes, each 20-bytes. They correspond to hashes of the nodes
+for the 32 possible values of the *first* 5 bits of the *next* byte of the
+RIPEMD160 hash. By convention, an empty node has a hash of zero.
+
+
+
+7) Leaf Node
+
+Contains every item in this node, sorted in order of increasing raw binary
+order. (Elements that start with a zero byte come first.) In practice, this
+will be exactly one item. You'd need a quadrillion entries before two
+wound up in the same leaf.
+
+By convention, an empty leaf node has a hash of all zero bytes.
+
+
+
+8) Contact block
+
+These are used to pass node contact information around the network and are
+signed so nodes can prove they are operational.
+
+Fields:
+1) 64-byte node public key
+2) 4-byte software version
+3) 4-byte minimum supported version
+4) 8-byte node flags (to be defined)
+5) 8-byte timestamp (seconds since 1/1/70)
+6) Node Info Block
+7) Signature of fields 1-6 above.
+
+The node info block consists of one or more node info elements. Each element
+consists of a 1-byte element type, a 3-byte element length, and the element
+data. Four zero bytes marks the end of the node info block. The following
+element types are defined:
+
+0 = IPv4 Contact Information (4-byte address, 2-byte port)
+1 = IPv6 Contact Information (16-byte address, 2-byte port)
+2 = Hanko URL
+3 = Node Name
+4 = Organization Name
+5 = URL
+6 = Admin Email
+7 = Node Policy URL
+8 = Transitive Trust (20-byte node hanko, 1-byte trust type)
+
+A node *must* ignore unknown element types. Fixed-length element types may
+contain more than one value in a single element. For example, you can put
+two IP/port combos in a single type 0 element of length 4+12.

SQLiteDatabases.sql

@@ -0,0 +1,95 @@
+
+
+CREATE TABLE Transactions (  -- trans in all state
+	Hanko		BLOB PRIMARY KEY,
+	NodeHash	BLOB,
+	Source		BLOB,
+	FromSeq		BIGINT UNSIGNED,
+	Dest		BLOB,
+	Ident		BIGINT,
+	SourceLedger	BIGINT UNSIGNED,
+	Signature	BLOB,
+	LedgerCommited	BIGINT UNSIGNED,	-- 0 if none
+	Status		VARCHAR(12) NOT NULL
+);
+
+CREATE INDEX TransactionHashSet -- needed to fetch hash groups
+ON Transactions(LedgerCommited, NodeHash);
+
+
+CREATE TABLE PubKeys ( -- holds pub keys for nodes and accounts
+	Hash		BLOB PRIMARY KEY,
+	PubKey		BLOB NOT NULL
+);
+
+
+CREATE TABLE AccountStatus ( -- holds balances and sequence numbers
+	Row		INTEGER PRIMARY KEY,
+	Hanko		BLOB,
+	Balance		BIGINT UNSIGNED,
+	Sequence	BIGINT UNSIGNED,
+	FirstLedger	BIGINT UNSIGNED,
+	LastLedger	BIGINT UNSIGNED		-- 2^60 if still valid
+);
+
+CREATE TABLE Ledgers ( -- closed ledgers
+	Hash		BLOB PRIMARY KEY,
+	LedgerSeq	BIGINT UNSIGNED,
+	FeeHeld		BIGINT UNSIGNED,
+	PrevHash	BLOB,
+	AccountHash	BLOB,
+	TrasactionHash	BLOB,
+	FullyStored	VARCHAR(1),
+	Status		VARCHAR(1)
+);
+
+
+CREATE TABLE LedgerHashNodes (
+	NodeID		BLOB,
+	LedgerSeq	BIGINT UNSIGNED,
+	Hashes		BLOB
+);
+
+CREATE TABLE TransactionHashNodes (
+	NodeID		BLOB,
+	LedgerSeq	BIGINT UNSIGNED,
+	Hashes		BLOB
+);
+
+
+CREATE TABLE LedgerConfirmations (
+	LedgerSeq	BIGINT UNSIGNED,
+	LedgerHash	BLOB,
+	Hanko		BLOB,
+	Signature	BLOB
+);
+
+CREATE TABLE TrustedNodes (
+	Hanko		BLOB PRIMARY KEY,
+	Trust		SMALLINT.
+	Comment		TEXT
+);
+
+CREATE TABLE KnownNodes (
+	Hanko		BLOB PRIMARY KEY,
+	LastSeen	TEXT,			-- YYYY-MM-DD HH:MM:SS.SSS
+	LastSigned	TEXT,
+	LastIP		BLOB,
+	LastPort	BIGINT UNSIGNED,
+	ContactObject	BLOB
+);
+
+CREATE TABLE ByHash (				-- used to synch nodes
+	Hash		BLOB PRIMARY KEY,
+	Type		VARCHAR(12) NOT NULL,
+	LedgerIndex	BIGINT UNSIGNED,	-- 2^60 if valid now, 0 if none
+	Object		BLOB
+);
+
+CREATE TABLE LocalAccounts (
+	Hash		BLOB PRIMARY KEY,
+	CurrentBalance	BIGINT UNSIGNED,
+	KeyFormat	TEXT,
+	PrivateKey	BLOB
+);
+