mirror of
https://github.com/XRPLF/rippled.git
synced 2026-06-03 00:36:48 +00:00
add doc-agent
This commit is contained in:
126
docs/skills/index.md
Normal file
126
docs/skills/index.md
Normal file
@@ -0,0 +1,126 @@
|
||||
# xrpld Codebase Skills Index
|
||||
|
||||
## Description
|
||||
This is the top-level guide for all best-practices skills in this repository. Use this to understand the codebase organization and find the right skill for any task.
|
||||
|
||||
## When to Use Skills
|
||||
Reference a skill whenever you are:
|
||||
- **Writing new code** in a module - check the skill first for established patterns
|
||||
- **Modifying existing code** - verify your changes follow module conventions
|
||||
- **Adding a new transaction type** - see `libxrpl/tx/transactors.md` for the full template
|
||||
- **Debugging** - skills list key files and common pitfalls per module
|
||||
- **Reviewing code** - skills document what "correct" looks like for each module
|
||||
|
||||
## Codebase Architecture
|
||||
|
||||
The codebase is split into two main areas:
|
||||
|
||||
### `src/libxrpl/` — The Library (skills in `.claude/skills/libxrpl/`)
|
||||
Reusable library code: data types, serialization, cryptography, ledger state, transaction processing, and storage. This is the **protocol layer**.
|
||||
|
||||
| Module | Responsibility |
|
||||
|--------|---------------|
|
||||
| `basics` | Foundational types: Buffer, Slice, base_uint, Number, logging, error contracts |
|
||||
| `beast` | Support layer: Journal logging, test framework, instrumentation, IP types |
|
||||
| `conditions` | Crypto-conditions (RFC): fulfillment validation, DER encoding |
|
||||
| `core` | Job queue, load monitoring, hash-based message dedup |
|
||||
| `crypto` | CSPRNG, secure erasure, RFC1751 encoding |
|
||||
| `json` | Json::Value, parsing, serialization, StaticString optimization |
|
||||
| `ledger` | ReadView/ApplyView, state tables, payment sandbox, credit ops |
|
||||
| `net` | HTTP/HTTPS client, SSL certs, async I/O |
|
||||
| `nodestore` | Persistent node storage: RocksDB, NuDB, Memory backends |
|
||||
| `protocol` | STObject hierarchy, SField, Serializer, TER codes, Features, Keylets |
|
||||
| `proto` | Protocol Buffer generated headers (gRPC API definitions) |
|
||||
| `rdb` | SOCI database wrapper, checkpointing |
|
||||
| `resource` | Rate limiting, endpoint tracking, abuse prevention |
|
||||
| `server` | Port config, SSL/TLS, WebSocket, admin networks |
|
||||
| `shamap` | SHA-256 Merkle radix tree (16-way branching, COW) |
|
||||
| `tx` | Transaction pipeline: Transactor base, preflight/preclaim/doApply |
|
||||
|
||||
### `src/xrpld/` — The Server Application (skills in `.claude/skills/xrpld/`)
|
||||
The running rippled server: application lifecycle, consensus, networking, RPC, and peer management. This is the **application layer**.
|
||||
|
||||
| Module | Responsibility |
|
||||
|--------|---------------|
|
||||
| `app` | Application singleton, ledger management, consensus adapters, services |
|
||||
| `app/main` | Application initialization and lifecycle |
|
||||
| `app/ledger` | Ledger storage, retrieval, immutable state management |
|
||||
| `app/consensus` | RCL consensus adapters (bridges generic algorithm to rippled) |
|
||||
| `app/misc` | Fee voting, amendments, SHAMapStore, TxQ, validators, NetworkOPs |
|
||||
| `app/paths` | Payment path finding algorithm, trust line caching |
|
||||
| `app/rdb` | Application-level database operations |
|
||||
| `app/tx` | Application-level transaction handling |
|
||||
| `consensus` | Generic consensus algorithm (CRTP-based, app-independent) |
|
||||
| `core` | Configuration (Config.h), time keeping, network ID |
|
||||
| `overlay` | P2P networking: peer connections, protocol buffers, clustering |
|
||||
| `peerfinder` | Network discovery: bootcache, livecache, slot management |
|
||||
| `perflog` | Performance logging and instrumentation |
|
||||
| `rpc` | RPC handler dispatch, coroutine suspension, 40+ command handlers |
|
||||
| `shamap` | Application-level SHAMap operations (NodeFamily) |
|
||||
|
||||
### `include/xrpl/` — Header Files
|
||||
Headers live in `include/xrpl/` and mirror the `src/libxrpl/` structure. Each skill already references its corresponding headers in the "Key Files" section.
|
||||
|
||||
## Cross-Cutting Conventions
|
||||
|
||||
### Error Handling
|
||||
- **Transaction errors**: Return `TER` enum (tesSUCCESS, tecFROZEN, temBAD_AMOUNT, etc.)
|
||||
- **Logic errors**: `Throw<std::runtime_error>()`, `LogicError()`, `XRPL_ASSERT()`
|
||||
- **I/O errors**: `Status` enum or `boost::system::error_code`
|
||||
- **RPC errors**: Inject via `context.params`
|
||||
|
||||
### Assertions
|
||||
```cpp
|
||||
XRPL_ASSERT(condition, "ClassName::method : description"); // Debug only
|
||||
XRPL_VERIFY(condition, "ClassName::method : description"); // Always enabled
|
||||
```
|
||||
|
||||
### Logging
|
||||
```cpp
|
||||
JLOG(j_.warn()) << "Message"; // Always wrap in JLOG macro
|
||||
```
|
||||
|
||||
### Memory Management
|
||||
- `IntrusiveRefCounts` + `SharedIntrusive` for shared ownership in libxrpl
|
||||
- `std::shared_ptr` for shared ownership in xrpld
|
||||
- `std::unique_ptr` for exclusive ownership
|
||||
- `CountedObject<T>` mixin for instance tracking
|
||||
|
||||
### Feature Gating
|
||||
```cpp
|
||||
if (ctx.rules.enabled(featureMyFeature)) { /* new behavior */ }
|
||||
```
|
||||
|
||||
### Code Organization
|
||||
- Headers in `include/xrpl/`, implementations in `src/libxrpl/` or `src/xrpld/`
|
||||
- `#pragma once` (never `#ifndef` guards)
|
||||
- `namespace xrpl { }` for all code
|
||||
- `detail/` namespace for internal helpers
|
||||
- Factory functions: `make_*()` returning `unique_ptr` or `shared_ptr`
|
||||
|
||||
## Subsystem Skills (soul/)
|
||||
|
||||
Concise invariants, bug patterns, and review checklists for each subsystem:
|
||||
|
||||
| Subsystem | File | Focus |
|
||||
|-----------|------|-------|
|
||||
| Consensus | `soul/consensus.md` | State machine, amendments, UNL, validations, TX ordering |
|
||||
| Cryptography | `soul/cryptography.md` | Key types, signing, hashing, handshake |
|
||||
| Ledger | `soul/ledger.md` | Immutability, acquisition, entry types |
|
||||
| NodeStore | `soul/nodestore.md` | Backends, caching, online deletion |
|
||||
| Peering | `soul/peering.md` | Overlay, connections, squelching |
|
||||
| Protocol | `soul/protocol.md` | Macros, serialization format, field ordering |
|
||||
| RPC | `soul/rpc.md` | Handler dispatch, roles, subscriptions |
|
||||
| SHAMap | `soul/shamap.md` | COW, node types, sync, proofs |
|
||||
| SQL | `soul/sql.md` | Schema, config, checkpointing |
|
||||
| Testing | `soul/test.md` | JTx framework, Env setup, TER expectations, amendment gating |
|
||||
| Transactors | `soul/transactors.md` | Pipeline, new TX types, signing, transactor template |
|
||||
| WebSockets | `soul/websockets.md` | Session lifecycle, flow control |
|
||||
|
||||
### Workflow & Processes
|
||||
| Skill | File |
|
||||
|-------|------|
|
||||
| Workflow Orchestration | `workflow.md` |
|
||||
| Task Management | `task-management.md` |
|
||||
| Amendment Creation | `amendment.md` |
|
||||
| Merge Conflicts | `merge-conflicts.md` |
|
||||
86
docs/skills/soul/consensus.md
Normal file
86
docs/skills/soul/consensus.md
Normal file
@@ -0,0 +1,86 @@
|
||||
# Consensus
|
||||
|
||||
Template-based state machine in `Consensus.h` parameterized by an Adaptor (`RCLConsensus`). Three phases: open -> establish -> accepted. Modes: proposing, observing, wrongLedger, switchedLedger.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- A ledger cannot close until the previous ledger reaches consensus AND (has transactions OR close time reached)
|
||||
- Proposals must have strictly increasing sequence numbers per peer; stale proposals are silently dropped
|
||||
- The Avalanche state machine progressively lowers consensus thresholds over time (init -> mid -> late -> stuck) to prevent livelock
|
||||
- `minCONSENSUS_PCT = 80` is the baseline; timing params: `ledgerMIN_CONSENSUS = 1950ms`, `ledgerMAX_CONSENSUS = 15s`
|
||||
- Dead nodes (`deadNodes_`) are permanently excluded for the round once they bow out
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- Proposals referencing a stale `prevLedgerID_` after a ledger switch cause split-brain; always check `newPeerProp.prevLedger() != prevLedgerID_` before processing
|
||||
- Resetting the consensus timer during `establish` phase causes re-convergence and potential split; timer must only reset on phase transitions
|
||||
- `DisputedTx::updateVote` changes local vote based on peer pressure; bugs here cause determinism failures across nodes
|
||||
- `createDisputes()` deduplicates via `compares` set; missing this check creates duplicate disputes that skew vote counts
|
||||
- The `peerUnchangedCounter_` is reset to 0 when any vote changes; bugs in this counter cause premature consensus declaration
|
||||
|
||||
## Amendments
|
||||
|
||||
- 80% validator support for 2 weeks to enable; tracked via `AmendmentTable` with `amendmentMap_`
|
||||
- New amendments: add to `features.macro` with `XRPL_FEATURE`/`XRPL_FIX`, increment `numFeatures` in `Feature.h`
|
||||
- Unsupported enabled amendment blocks the server (`setAmendmentBlocked`); no mechanism to disable/revoke
|
||||
- Voting happens each consensus round in `doVoting`; votes are persisted in `FeatureVotes` SQLite table
|
||||
- `fixAmendmentMajorityCalc` changed the threshold calculation; check which calculation applies
|
||||
|
||||
## UNL and Negative UNL
|
||||
|
||||
- Negative UNL temporarily disables unreliable validators (max 25% of UNL: `negativeUNLMaxListed = 0.25`)
|
||||
- Scoring uses `buildScoreTable` over recent ledger history; low watermark (50%) = disable candidate, high watermark (80%) = re-enable candidate
|
||||
- Candidate selection is deterministic via previous ledger hash as randomizing pad
|
||||
- `newValidatorDisableSkip = FLAG_LEDGER_INTERVAL * 2` prevents disabling newly joined validators prematurely
|
||||
|
||||
## Validations
|
||||
|
||||
- `ValidationParms` defines freshness windows: CURRENT_WALL=5min, CURRENT_LOCAL=3min, SET_EXPIRES=10min, FRESHNESS=20s
|
||||
- `SeqEnforcer` rejects validations with regressed or duplicate sequence numbers (`ValStatus::badSeq`)
|
||||
- Conflicting validations (same seq, different hash) are logged as byzantine behavior
|
||||
- `handleNewValidation` is the entry point: checks trust, adds to `Validations` set, triggers `checkAccept` if current+trusted
|
||||
|
||||
## Transaction Ordering
|
||||
|
||||
- `CanonicalTXSet` orders by: salted account key (XOR with random salt) -> sequence proxy -> transaction ID
|
||||
- Salt prevents manipulation of ordering by account selection
|
||||
- `TxQ` uses `OrderCandidates`: higher fee level first, then `txID XOR parentHash` as tiebreaker
|
||||
- Per-account limit: `maximumTxnPerAccount`; blocked transactions held until blocker resolves
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Proposal Validation (prevents split-brain)
|
||||
```cpp
|
||||
// REQUIRED: reject proposals referencing stale previous ledger
|
||||
if (newPeerProp.prevLedger() != prevLedgerID_)
|
||||
{
|
||||
JLOG(j_.debug()) << "Got proposal for " << newPeerProp.prevLedger()
|
||||
<< " but we are on " << prevLedgerID_;
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
### Complete Bow-Out Handling
|
||||
```cpp
|
||||
// REQUIRED: all three steps — unvote, erase position, mark dead
|
||||
if (newPeerProp.isBowOut())
|
||||
{
|
||||
if (result_)
|
||||
for (auto& it : result_->disputes)
|
||||
it.second.unVote(peerID);
|
||||
if (currPeerPositions_.find(peerID) != currPeerPositions_.end())
|
||||
currPeerPositions_.erase(peerID);
|
||||
deadNodes_.insert(peerID); // permanently excluded this round
|
||||
}
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/xrpld/consensus/Consensus.h` - state machine
|
||||
- `src/xrpld/consensus/ConsensusParms.h` - timing/threshold params
|
||||
- `src/xrpld/app/consensus/RCLConsensus.cpp` - XRPL adaptor
|
||||
- `src/xrpld/consensus/DisputedTx.h` - dispute tracking
|
||||
- `src/xrpld/app/misc/detail/AmendmentTable.cpp` - amendment logic
|
||||
- `src/xrpld/app/misc/NegativeUNLVote.cpp` - N-UNL voting
|
||||
- `src/xrpld/consensus/Validations.h` - validation tracking
|
||||
- `src/xrpld/app/misc/CanonicalTXSet.h` - TX ordering
|
||||
59
docs/skills/soul/cryptography.md
Normal file
59
docs/skills/soul/cryptography.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# Cryptography
|
||||
|
||||
XRPL supports secp256k1 (ECDSA) and ed25519 key types. All crypto uses OpenSSL + dedicated libs (libsecp256k1, ed25519-donna).
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- `SecretKey` destructor calls `secure_erase` on internal buffer; any code handling secret keys must follow this pattern
|
||||
- ed25519 public keys are prefixed with `0xED` (33 bytes total); secp256k1 keys are 33-byte compressed
|
||||
- `sha512Half` (first 32 bytes of SHA-512) is the standard hash used throughout XRPL for node hashing, signing, etc.
|
||||
- `RIPEMD-160(SHA-256(x))` is used for account ID derivation (`ripesha_hasher`)
|
||||
- Base58 encoding includes a type byte prefix and 4-byte checksum (double SHA-256)
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- Mixing up key types: secp256k1 signing hashes the message with sha512Half first, ed25519 signs the raw message
|
||||
- `signDigest` only works with secp256k1; calling it with ed25519 throws a logic error
|
||||
- Signature canonicality: ed25519 `verify` checks signature canonicality before calling `ed25519_sign_open`; non-canonical signatures are rejected
|
||||
- Overlay handshake uses `signDigest` to sign the session fingerprint (`sharedValue`); the signature binds the TLS session to the node identity
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- New crypto code must use `crypto_prng()` singleton for randomness, never raw `rand()`
|
||||
- Secret key buffers must be `secure_erase`d after use
|
||||
- Verify that key type dispatch handles both secp256k1 and ed25519 (or explicitly rejects one with a clear error)
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Secure Erasure
|
||||
```cpp
|
||||
// REQUIRED: destructor must erase secret material
|
||||
SecretKey::~SecretKey()
|
||||
{
|
||||
secure_erase(buf_, sizeof(buf_));
|
||||
}
|
||||
|
||||
// REQUIRED: erase intermediate buffers after use
|
||||
beast::rngfill(buf, sizeof(buf), crypto_prng());
|
||||
SecretKey sk(Slice{buf, sizeof(buf)});
|
||||
secure_erase(buf, sizeof(buf)); // MUST erase raw buffer
|
||||
```
|
||||
|
||||
### Key Type Dispatch
|
||||
```cpp
|
||||
// REQUIRED: handle both key types or explicitly reject
|
||||
if (type == KeyType::ed25519)
|
||||
{ /* ed25519 path */ }
|
||||
else if (type == KeyType::secp256k1)
|
||||
{ /* secp256k1 path */ }
|
||||
else
|
||||
LogicError("unknown key type"); // MUST NOT fall through silently
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `include/xrpl/protocol/SecretKey.h` / `PublicKey.h` - key types
|
||||
- `src/libxrpl/protocol/SecretKey.cpp` - signing, key generation
|
||||
- `src/libxrpl/protocol/PublicKey.cpp` - verification
|
||||
- `include/xrpl/protocol/digest.h` - hash functions
|
||||
- `src/xrpld/overlay/detail/Handshake.cpp` - overlay handshake crypto
|
||||
63
docs/skills/soul/ledger.md
Normal file
63
docs/skills/soul/ledger.md
Normal file
@@ -0,0 +1,63 @@
|
||||
# Ledger
|
||||
|
||||
Each ledger is an immutable snapshot: header (seq, hashes, close time) + state SHAMap + transaction SHAMap. `LedgerMaster` is the central coordinator.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- Once `setImmutable()` is called, the ledger and its SHAMaps cannot change; only immutable ledgers can be set in `LedgerHolder`
|
||||
- Every server always has an open ledger; the open ledger cannot close until previous consensus completes
|
||||
- Ledger header hashes to the ledger's identity hash; includes state root, tx root, parent hash, total coins, close time
|
||||
- `LedgerMaster` tracks: `mPubLedger` (last published), `mValidLedger` (last validated), `mLedgerHistory` (cache)
|
||||
- Validation requires minimum trusted validations (`minVal`); filtered by Negative UNL
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- Modifying a ledger after `setImmutable()` corrupts shared state; always check `mImmutable` before mutation
|
||||
- Gap detection: if ledgers 603 and 600 exist but 601-602 are missing, `LedgerMaster` requests 602 first, then backfills 601
|
||||
- `InboundLedger::gotData()` queues data for processing; calling `done()` before all data arrives creates incomplete ledgers
|
||||
- `checkAccept` won't accept a ledger that isn't ahead of the last validated ledger; stale validations are silently ignored
|
||||
|
||||
## Ledger Entry Types
|
||||
|
||||
- Defined in `ledger_entries.macro` using `LEDGER_ENTRY(type, code, class, name, fields)`
|
||||
- Each entry has an `SOTemplate` defining required/optional fields
|
||||
- Key computation: `Indexes.cpp` computes unique keys (keylets) for each ledger object type
|
||||
- `STLedgerEntry` wraps the serialized data with type-safe field access
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- New ledger entry types: add to `ledger_entries.macro`, implement keylet in `Indexes.cpp`
|
||||
- Verify `LedgerCleaner` can handle the new entry type for repair
|
||||
- Check that acquisition code handles the entry in both `InboundLedger` and `LedgerMaster`
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Immutability Guard
|
||||
```cpp
|
||||
// After this, no mutations allowed on the ledger or its SHAMaps
|
||||
inline void SHAMap::setImmutable()
|
||||
{
|
||||
XRPL_ASSERT(state_ != SHAMapState::Invalid, "...");
|
||||
state_ = SHAMapState::Immutable;
|
||||
}
|
||||
// VERIFY: code never calls peek()/insert()/erase() after setImmutable()
|
||||
```
|
||||
|
||||
### New Ledger Entry Keylet
|
||||
```cpp
|
||||
// REQUIRED: every new ledger entry type needs unique keylet computation
|
||||
Keylet keylet::myEntry(AccountID const& id)
|
||||
{
|
||||
return {ltMY_ENTRY,
|
||||
sha512Half(std::uint16_t(spaceMyEntry), id)};
|
||||
}
|
||||
// Also add to ledger_entries.macro and Indexes.cpp
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/xrpld/app/ledger/Ledger.h` - ledger class
|
||||
- `src/xrpld/app/ledger/detail/LedgerMaster.cpp` - central coordinator
|
||||
- `src/xrpld/app/ledger/detail/InboundLedger.cpp` - ledger acquisition
|
||||
- `include/xrpl/protocol/detail/ledger_entries.macro` - entry type definitions
|
||||
- `src/libxrpl/protocol/Indexes.cpp` - keylet computation
|
||||
52
docs/skills/soul/nodestore.md
Normal file
52
docs/skills/soul/nodestore.md
Normal file
@@ -0,0 +1,52 @@
|
||||
# NodeStore
|
||||
|
||||
Persistent key-value store for `NodeObject`s (ledger entries). All ledger state is stored here between launches. Keys are 256-bit hashes.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- `NodeObject` types: `hotLEDGER` (1), `hotACCOUNT_NODE` (3), `hotTRANSACTION_NODE` (4), `hotDUMMY` (512, cache marker for missing entries)
|
||||
- Preferred backends: NuDB (append-only) and RocksDB; LevelDB/HyperLevelDB are deprecated
|
||||
- `TaggedCache` evicts by both `cache_size` (max items) and `cache_age` (max minutes)
|
||||
- `DatabaseRotatingImp` uses two backends (writable + archive) for online deletion; rotation moves writable to archive, creates new writable, deletes old archive
|
||||
- Corrupt data triggers fatal logging; unknown/backend errors logged with appropriate severity
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- `fetchNodeObject` with `duplicate=true` copies from archive to writable backend; forgetting this in rotating mode means objects disappear after rotation
|
||||
- `hotDUMMY` objects in cache mark missing entries; code that checks cache hits must distinguish real objects from dummies
|
||||
- Batch write limit is 65536 objects; exceeding this silently truncates or fails depending on backend
|
||||
- `fdRequired()` must be called during resource planning; running out of file descriptors causes silent backend failures
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- Config changes: verify `[node_db]` section has valid `type`, `path`, and `compression` settings
|
||||
- Online deletion: ensure `SHAMapStoreImp` coordinates rotation with the application lifecycle
|
||||
- New backend types: implement the full `Backend` interface including `fdRequired()`
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Cache Lookup — Distinguish Real vs Dummy
|
||||
```cpp
|
||||
// REQUIRED: hotDUMMY marks "confirmed missing" — not a real object
|
||||
auto obj = cache_.fetch(hash);
|
||||
if (obj && obj->getType() == hotDUMMY)
|
||||
return nullptr; // not found, just cached as missing
|
||||
return obj;
|
||||
```
|
||||
|
||||
### Backend File Descriptor Reporting
|
||||
```cpp
|
||||
// REQUIRED: every backend must accurately report FD needs
|
||||
int fdRequired() const override
|
||||
{
|
||||
return fdLimit_; // inaccurate values cause silent failures
|
||||
}
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `include/xrpl/nodestore/NodeObject.h` - object types
|
||||
- `include/xrpl/nodestore/Backend.h` - backend interface
|
||||
- `include/xrpl/nodestore/detail/DatabaseNodeImp.h` - standard implementation
|
||||
- `src/libxrpl/nodestore/DatabaseRotatingImp.cpp` - rotating/online deletion
|
||||
- `src/xrpld/app/misc/SHAMapStoreImp.cpp` - lifecycle management
|
||||
60
docs/skills/soul/peering.md
Normal file
60
docs/skills/soul/peering.md
Normal file
@@ -0,0 +1,60 @@
|
||||
# Overlay Peering
|
||||
|
||||
P2P network using persistent TCP/IP connections. Messages serialized via Protocol Buffers. `OverlayImpl` manages connections; `PeerImp` handles per-peer logic.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- Connection preference order: Fixed Peers -> Livecache -> Bootcache
|
||||
- Cluster connections do NOT count toward connection limits (unlimited)
|
||||
- Protobuf message changes MUST maintain wire compatibility or risk network partitioning
|
||||
- Squelching: after enough peers relay a validator's messages, a subset is "Selected" and the rest are temporarily muted to reduce bandwidth
|
||||
- Handshake binds TLS session to node identity via `signDigest` of the session fingerprint
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- PeerFinder slot exhaustion: if `maxPeers` is reached, new outbound connections silently fail; check slot availability before connecting
|
||||
- `HashRouter::shouldRelay` prevents duplicate relay; bypassing it causes message storms
|
||||
- `ConnectAttempt::processResponse` on HTTP 503 parses "peer-ips" for alternatives; malformed responses here can crash with bad IP parsing
|
||||
- `PeerImp::close` must run on the strand; calling from wrong thread causes race conditions on socket and timer state
|
||||
- Destructor chain: `~PeerImp` -> `deletePeer` -> `onPeerDeactivate` -> `on_closed` -> `remove`; interrupting this chain leaks slots
|
||||
|
||||
## Connection Lifecycle
|
||||
|
||||
1. `OverlayImpl::connect` -> check resource limits -> allocate PeerFinder slot -> create `ConnectAttempt`
|
||||
2. Async TCP connect -> TLS handshake -> HTTP upgrade with identity headers
|
||||
3. `processResponse` -> verify handshake -> create `PeerImp` -> `add_active` -> `run()`
|
||||
4. `doProtocolStart` -> start async message receive loop -> exchange validator lists and manifests
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- Verify resource manager checks on both inbound and outbound connections
|
||||
- New protocol messages: update protobuf definitions AND verify wire compatibility
|
||||
- Squelch changes: test with high peer counts; incorrect squelch logic can silence validators
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Strand Execution
|
||||
```cpp
|
||||
// REQUIRED: socket operations must run on the strand
|
||||
if (!strand_.running_in_this_thread())
|
||||
return post(strand_, std::bind(
|
||||
&PeerImp::close, shared_from_this()));
|
||||
// Calling socket ops from wrong thread causes races on state
|
||||
```
|
||||
|
||||
### Duplicate Relay Prevention
|
||||
```cpp
|
||||
// REQUIRED: check HashRouter before relaying
|
||||
if (!hashRouter_.shouldRelay(hash))
|
||||
return; // already relayed — suppress duplicate
|
||||
overlay_.relay(message, hash);
|
||||
// Bypassing this causes message storms across the network
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/xrpld/overlay/detail/OverlayImpl.cpp` - main overlay manager
|
||||
- `src/xrpld/overlay/detail/PeerImp.cpp` - per-peer logic
|
||||
- `src/xrpld/overlay/detail/ConnectAttempt.cpp` - outbound connection
|
||||
- `src/xrpld/overlay/Slot.h` - squelch state machine
|
||||
- `src/xrpld/overlay/detail/Handshake.cpp` - handshake crypto
|
||||
64
docs/skills/soul/protocol.md
Normal file
64
docs/skills/soul/protocol.md
Normal file
@@ -0,0 +1,64 @@
|
||||
# Protocol and Serialization
|
||||
|
||||
Macro-driven system for defining features, transactions, ledger entries, and serialized fields. Canonical binary format is required for signatures and consensus.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- Fields are sorted by (type code, field code) for canonical serialization; sorting by Field ID bytes produces WRONG results
|
||||
- Field ID encoding: 1-3 bytes depending on type/field code values (both < 16 = 1 byte)
|
||||
- Signing hash prefix: `0x53545800` for single-signing, `0x534D5400` for multi-signing
|
||||
- `STObject[sfFoo]` returns value or default; `STObject[~sfFoo]` returns optional (nothing if absent)
|
||||
- All ST types inherit from `STBase`; `STVar` provides type-erased storage with stack/heap allocation
|
||||
|
||||
## Macro System
|
||||
|
||||
- `XRPL_FEATURE(name, supported, vote)` / `XRPL_FIX` / `XRPL_RETIRE` in `features.macro`
|
||||
- `TRANSACTION(tag, value, class, delegation, fields)` in `transactions.macro`
|
||||
- `LEDGER_ENTRY(type, code, class, name, fields)` in `ledger_entries.macro`
|
||||
- `TYPED_SFIELD(name, TYPE, code)` in `sfields.macro`
|
||||
- Adding any new definition requires updating the count in the corresponding header
|
||||
|
||||
## Serialization Format
|
||||
|
||||
- XRP Amount: 8 bytes, MSB=0, next bit=1 for positive, remaining 62 bits = value
|
||||
- Token Amount: 8 bytes mantissa/exponent + 20 bytes currency + 20 bytes issuer
|
||||
- AccountID: 20 bytes, length-prefixed when top-level
|
||||
- STArray: elements between start (`0xf0`) and end (`0xf1`) markers
|
||||
- STObject: fields in canonical order between start (`0xe0`) and end (`0xe1`) markers
|
||||
- Length prefixing: 1 byte (0-192), 2 bytes (193-12480), 3 bytes (12481-918744)
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- Adding a field to `transactions.macro` without adding it to `sfields.macro` causes silent serialization failures
|
||||
- Forgetting to increment `numFeatures` after adding to `features.macro` causes out-of-bounds access
|
||||
- Non-canonical field ordering in hand-built binary blobs causes signature verification failures
|
||||
- `soeMPTSupported` flag on amount fields enables MPT token support; omitting it silently rejects MPT payments
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Amendment Registration
|
||||
```cpp
|
||||
// In features.macro — REQUIRED format:
|
||||
XRPL_FEATURE(MyNewFeature, Supported::yes, VoteBehavior::DefaultNo)
|
||||
XRPL_FIX (MyBugFix, Supported::yes, VoteBehavior::DefaultNo)
|
||||
// MUST also increment numFeatures in Feature.h — omitting causes OOB access
|
||||
```
|
||||
|
||||
### STObject Field Access
|
||||
```cpp
|
||||
// Safe: optional access — returns std::optional, never throws
|
||||
if (auto const val = tx[~sfAmount])
|
||||
use(*val);
|
||||
|
||||
// Throws if field is absent — only safe when preflight guarantees presence
|
||||
auto const amount = tx[sfAmount];
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `include/xrpl/protocol/detail/features.macro` - amendment definitions
|
||||
- `include/xrpl/protocol/detail/transactions.macro` - transaction types
|
||||
- `include/xrpl/protocol/detail/ledger_entries.macro` - ledger objects
|
||||
- `include/xrpl/protocol/detail/sfields.macro` - field definitions
|
||||
- `include/xrpl/protocol/Feature.h` - `numFeatures` constant
|
||||
- `src/libxrpl/protocol/STObject.cpp` - core serialized object
|
||||
61
docs/skills/soul/rpc.md
Normal file
61
docs/skills/soul/rpc.md
Normal file
@@ -0,0 +1,61 @@
|
||||
# RPC
|
||||
|
||||
JSON-RPC over HTTP/WebSocket and gRPC. Central handler table dispatches by method name + API version. Roles: ADMIN, USER, IDENTIFIED, PROXY, FORBID.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- Handler table in `Handler.cpp`: each entry = `{name, function, role, condition, minApiVer, maxApiVer}`
|
||||
- `conditionMet` checks server state (e.g., `NEEDS_CURRENT_LEDGER`) before invoking handler
|
||||
- API v2.0+ errors: structured objects with `status`, `code`, `message`; earlier: flat fields in response
|
||||
- Sensitive fields (`passphrase`, `secret`, `seed`, `seed_hex`) are masked in error responses
|
||||
- Batch requests: `"method": "batch"` with `"params"` array; each sub-request processed independently
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- New handler without entry in `Handler.cpp` static array = handler silently unreachable
|
||||
- Wrong `role_` on handler: USER-level handler with admin-only data leaks; ADMIN handler accessible to users = security hole
|
||||
- `conditionMet` returning false causes a generic error; ensure new conditions are documented
|
||||
- Resource charging: each request gets a fee via `Resource::Consumer`; missing charge allows DoS
|
||||
- `maxRequestSize` (RPC::Tuning) rejection happens before JSON parsing; oversized requests get no error detail
|
||||
|
||||
## Adding New RPC Handler
|
||||
|
||||
1. Declare in `Handlers.h`: `Json::Value doMyCommand(RPC::JsonContext&);`
|
||||
2. Implement in new file under `src/xrpld/rpc/handlers/`
|
||||
3. Register in `Handler.cpp` static array with role, condition, version range
|
||||
4. For gRPC: define in `xrp_ledger.proto`, add `CallData` in `GRPCServerImpl::setupListeners()`
|
||||
|
||||
## Subscriptions
|
||||
|
||||
- WebSocket clients can subscribe to: `server`, `ledger`, `book_changes`, `transactions`, `validations`, `manifests`, `peer_status` (admin), `consensus`
|
||||
- `WSInfoSub` delivers events via weak pointer to `WSSession`; dead sessions are automatically cleaned up
|
||||
- `RPCSub` delivers to remote URL endpoints with auth and SSL support
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Handler Table Registration
|
||||
```cpp
|
||||
// In Handler.cpp handlerArray[] — REQUIRED for every new handler:
|
||||
{"my_command", byRef(&doMyCommand), Role::USER, NO_CONDITION},
|
||||
// role MUST match security requirements:
|
||||
// Role::ADMIN for internal-only, Role::USER for public API
|
||||
// condition: NEEDS_CURRENT_LEDGER, NEEDS_NETWORK_CONNECTION, or NO_CONDITION
|
||||
```
|
||||
|
||||
### Version-Ranged Handler
|
||||
```cpp
|
||||
// New-style handler with API version range
|
||||
template <> Handler handlerFrom<MyCommandHandler>()
|
||||
{ return {MyCommandHandler::name, &handle<Json::Value, MyCommandHandler>,
|
||||
MyCommandHandler::role, MyCommandHandler::condition,
|
||||
MyCommandHandler::minApiVer, MyCommandHandler::maxApiVer};
|
||||
}
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/xrpld/rpc/handlers/Handlers.h` - authoritative handler list
|
||||
- `src/xrpld/rpc/detail/Handler.cpp` - handler table and dispatch
|
||||
- `src/xrpld/rpc/detail/RPCHandler.cpp` - request processing pipeline
|
||||
- `src/xrpld/rpc/detail/ServerHandler.cpp` - HTTP/WS entry points
|
||||
- `include/xrpl/protocol/ErrorCodes.h` - error code definitions
|
||||
61
docs/skills/soul/shamap.md
Normal file
61
docs/skills/soul/shamap.md
Normal file
@@ -0,0 +1,61 @@
|
||||
# SHAMap
|
||||
|
||||
Merkle radix trie (radix 16) enabling O(1) subtree comparison via hash. Used for both state tree and transaction tree. Root is always a `SHAMapInnerNode`.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- Mutable SHAMaps have non-zero `cowid`; immutable have `cowid=0`. Once immutable, nodes persist for the map's lifetime with NO mechanism to remove them
|
||||
- Copy-on-write: `unshareNode` must be called before mutating any node in a mutable SHAMap; failing this corrupts shared snapshots
|
||||
- Inner nodes have up to 16 children; hash is computed from children's hashes. Leaf hash is computed from data + type-specific prefix
|
||||
- `canonicalize` ensures only one instance per hash in the cache; prevents races between threads
|
||||
- `SHAMapInnerNode` uses atomic operations + locking (`std::atomic<std::uint16_t> lock_`) for concurrent child access
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- Modifying a node without calling `unshareNode` first corrupts the snapshot that shares it; this is the #1 SHAMap bug class
|
||||
- `getMissingNodes` uses deferred async reads; processing completions out of order causes incorrect "full below" marking
|
||||
- Inner node serialization has two formats (compressed vs full) chosen by branch count; mismatched deserializer causes corruption
|
||||
- `addKnownNode` traverses toward target; if branch is empty or hash mismatches, returns "invalid" -- callers must handle this gracefully
|
||||
- Proof path verification walks leaf-to-root; incorrect key at any level causes false negative
|
||||
|
||||
## Serialization Formats
|
||||
|
||||
- **Compressed**: only non-empty branches serialized (saves space for sparse nodes)
|
||||
- **Full**: all 16 branches including empty ones (used for dense nodes)
|
||||
- Choice is automatic in `serializeForWire` based on branch count
|
||||
|
||||
## Leaf Node Types
|
||||
|
||||
- `SHAMapAccountStateLeafNode` - account state entries
|
||||
- `SHAMapTxLeafNode` - transactions
|
||||
- `SHAMapTxPlusMetaLeafNode` - transactions with metadata
|
||||
- Each uses a different hash prefix for domain separation
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### State Machine
|
||||
```cpp
|
||||
enum class SHAMapState {
|
||||
Modifying = 0, // can add/remove objects
|
||||
Immutable = 1, // FROZEN — no changes allowed
|
||||
Synching = 2, // hash fixed, missing nodes can be added
|
||||
Invalid = 3, // corrupt — do not use
|
||||
};
|
||||
// VERIFY: no peek()/insert()/erase() calls on Immutable maps
|
||||
```
|
||||
|
||||
### COW Discipline (#1 Bug Class)
|
||||
```cpp
|
||||
// REQUIRED before mutating any shared node:
|
||||
auto node = unshareNode(branch, key); // copies if shared
|
||||
node->setChild(index, child); // now safe to modify
|
||||
// BUG: skipping unshareNode corrupts snapshots sharing the node
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `include/xrpl/shamap/SHAMap.h` - main class
|
||||
- `include/xrpl/shamap/SHAMapInnerNode.h` - inner node (COW, threading)
|
||||
- `include/xrpl/shamap/SHAMapLeafNode.h` - leaf node base
|
||||
- `src/libxrpl/shamap/SHAMapSync.cpp` - sync, missing nodes, proofs
|
||||
- `src/libxrpl/shamap/SHAMapDelta.cpp` - walkMap, parallel traversal
|
||||
60
docs/skills/soul/sql.md
Normal file
60
docs/skills/soul/sql.md
Normal file
@@ -0,0 +1,60 @@
|
||||
# SQL Database
|
||||
|
||||
SQLite via SOCI for ledger/transaction history. Only SQLite is supported; Postgres has no implementation despite interface comments.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- Two main databases: `lgrdb_` (ledger) and `txdb_` (transactions, optional via `useTxTables` config)
|
||||
- Transaction tables are optional; disabling them means no transaction history or account_tx queries
|
||||
- WAL checkpointing triggers when WAL file grows beyond threshold; scheduled via job queue
|
||||
- Database init failure is fatal (throws exception, prevents construction)
|
||||
- Free disk space < 512MB triggers fatal error on write operations
|
||||
|
||||
## Schema
|
||||
|
||||
- `Ledgers` table: seq, hash, parent hash, total coins, close time, etc. Indexed by `LedgerSeq`
|
||||
- `Transactions` table: TransID, TransType, FromAcct, FromSeq, LedgerSeq, Status, RawTxn, TxnMeta. Indexed by `LedgerSeq`
|
||||
- `AccountTransactions` table: TransID, Account, LedgerSeq, TxnSeq. Triple-indexed for account_tx queries
|
||||
- Secondary DBs: Wallet (node identity, manifests), PeerFinder (bootstrap cache), State (deletion tracking)
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- No schema migration system; `CREATE TABLE IF NOT EXISTS` means old schemas silently persist with missing columns
|
||||
- PeerFinder DB is the exception -- it has schema versioning via `SchemaVersion` table
|
||||
- `safety_level` config affects journal_mode and synchronous; "low" can lose data on crash
|
||||
- `page_size` must be power of 2 between 512-65536; invalid values cause init failure
|
||||
- Online deletion coordinates between NodeStore rotation and SQL table pruning; race conditions here lose history
|
||||
|
||||
## Configuration
|
||||
|
||||
| Option | Section | Values | Default |
|
||||
|--------|---------|--------|---------|
|
||||
| `backend` | `[relational_db]` | `sqlite` only | sqlite |
|
||||
| `page_size` | `[sqlite]` | 512-65536, power of 2 | 4096 |
|
||||
| `safety_level` | `[sqlite]` | high, medium, low | high |
|
||||
| `journal_size_limit` | `[sqlite]` | integer >= 0 | 1582080 |
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Schema Evolution Caveat
|
||||
```cpp
|
||||
// WARNING: no migration system — old databases keep old schemas
|
||||
// CREATE TABLE IF NOT EXISTS silently skips if table exists with old columns
|
||||
// New columns on existing tables require manual ALTER TABLE or
|
||||
// documentation that the column is optional and may be absent
|
||||
```
|
||||
|
||||
### Disk Space Guard
|
||||
```cpp
|
||||
// REQUIRED on write paths: < 512MB triggers fatal to prevent corruption
|
||||
if (freeDiskSpace < minDiskFree)
|
||||
Throw<std::runtime_error>("Not enough disk space for database write");
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/xrpld/app/rdb/backend/detail/SQLiteDatabase.cpp` - main implementation
|
||||
- `src/xrpld/app/main/DBInit.h` - schema definitions
|
||||
- `src/xrpld/core/detail/DatabaseCon.cpp` - connection setup and pragmas
|
||||
- `src/xrpld/app/rdb/backend/detail/Node.cpp` - ledger/tx operations
|
||||
- `src/xrpld/app/rdb/detail/State.cpp` - deletion state tracking
|
||||
75
docs/skills/soul/test.md
Normal file
75
docs/skills/soul/test.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# Testing
|
||||
|
||||
JTx framework for in-memory ledger testing. Tests live in `src/test/`, derive from `beast::unit_test::suite`, and register with `BEAST_DEFINE_TESTSUITE`.
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Test File Structure
|
||||
```cpp
|
||||
class MyFeature_test : public beast::unit_test::suite {
|
||||
void testBasic() {
|
||||
testcase("basic");
|
||||
using namespace jtx;
|
||||
Env env{*this};
|
||||
// ... test logic ...
|
||||
}
|
||||
void run() override { testBasic(); }
|
||||
};
|
||||
BEAST_DEFINE_TESTSUITE(MyFeature, app, ripple);
|
||||
```
|
||||
|
||||
### Amendment Gating
|
||||
```cpp
|
||||
// REQUIRED: test with AND without the feature amendment
|
||||
Env env{*this}; // all amendments on
|
||||
Env env{*this, testable_amendments() - featureMyFeature}; // feature disabled
|
||||
// With custom config:
|
||||
Env env{*this, envconfig([](std::unique_ptr<Config> cfg) {
|
||||
/* modify config */ return cfg;
|
||||
})};
|
||||
```
|
||||
|
||||
### Transaction Submission
|
||||
```cpp
|
||||
auto const alice = Account{"alice"};
|
||||
auto const bob = Account{"bob"};
|
||||
env.fund(XRP(10000), alice, bob);
|
||||
env.close(); // REQUIRED between setup and test transactions
|
||||
|
||||
env(pay(alice, bob, XRP(100))); // expects tesSUCCESS
|
||||
env(pay(alice, bob, XRP(999999)), ter(tecUNFUNDED_PAYMENT)); // specific TER
|
||||
env(pay(alice, bob, XRP(100)), txflags(tfPartialPayment)); // with flags
|
||||
```
|
||||
|
||||
### State Verification
|
||||
```cpp
|
||||
env.require(balance(alice, XRP(9900)));
|
||||
env.require(balance(alice, USD(1000)));
|
||||
env.require(owners(alice, 2));
|
||||
|
||||
auto const sle = env.le(keylet::account(alice.id()));
|
||||
BEAST_EXPECT(sle);
|
||||
BEAST_EXPECT((*sle)[sfBalance] == XRP(9900));
|
||||
```
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- New transaction types MUST have tests with AND without the feature amendment
|
||||
- Every error path (tem*, tec*, tef*) must have a test exercising it
|
||||
- `env.close()` required between transactions that need ledger boundaries
|
||||
- Verify state after transaction, not just TER code
|
||||
- Test class naming: `FeatureName_test`, registered with `BEAST_DEFINE_TESTSUITE`
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- Forgetting `*this` as first Env arg disconnects assertion reporting
|
||||
- Comparing XRPAmount with raw integers instead of `XRP()` or `drops()`
|
||||
- Trust lines must be established before IOU payments
|
||||
- Each Env creates a full Application — keep test methods focused
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/test/jtx/Env.h` — test environment class
|
||||
- `src/test/jtx/jtx.h` — convenience header (includes all helpers)
|
||||
- `src/test/jtx/Account.h` — test accounts
|
||||
- `src/test/jtx/amount.h` — XRP(), drops(), IOU helpers
|
||||
141
docs/skills/soul/transactors.md
Normal file
141
docs/skills/soul/transactors.md
Normal file
@@ -0,0 +1,141 @@
|
||||
# Transactors
|
||||
|
||||
Transaction processing pipeline: preflight (static validation) -> preclaim (ledger state checks) -> doApply (state mutation). Base class `Transactor` in `src/libxrpl/tx/`.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- Pipeline is strict: preflight runs WITHOUT ledger state, preclaim runs WITH read-only view, doApply runs with mutable view
|
||||
- `preflight` validates all fields exist and are well-formed; this is the ONLY place to reject malformed transactions cheaply
|
||||
- Fee is always deducted even if the transaction fails (`tecCLAIM` pattern); `payFee` runs before `doApply`
|
||||
- Sequence/ticket consumption happens in `consumeSeqProxy`; must succeed before any state changes
|
||||
- Invariant checkers run after `doApply`; they can veto the transaction post-execution
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- New transaction type missing preflight validation for new fields = malformed transactions reach doApply and corrupt state
|
||||
- Forgetting to handle `tecCLAIM` in doApply: fee is deducted but no other state changes should occur
|
||||
- Batch transactions (`Batch` type) have their own signing path (`checkBatchSign`); changes to signing must cover both paths
|
||||
- `calculateBaseFee` override without updating `minimumFee` causes fee calculation divergence between nodes
|
||||
- Missing invariant checker update for new ledger entry types = silent constraint violations
|
||||
|
||||
## Transactor Template
|
||||
|
||||
### Header (`include/xrpl/tx/transactors/MyTx.h`)
|
||||
```cpp
|
||||
#pragma once
|
||||
#include <xrpl/tx/Transactor.h>
|
||||
|
||||
namespace xrpl {
|
||||
class MyTransaction : public Transactor {
|
||||
public:
|
||||
static constexpr ConsequencesFactoryType ConsequencesFactory{Normal};
|
||||
explicit MyTransaction(ApplyContext& ctx) : Transactor(ctx) {}
|
||||
|
||||
static bool checkExtraFeatures(PreflightContext const& ctx);
|
||||
static std::uint32_t getFlagsMask(PreflightContext const& ctx);
|
||||
static NotTEC preflight(PreflightContext const& ctx); // NO ledger
|
||||
static TER preclaim(PreclaimContext const& ctx); // read-only
|
||||
TER doApply() override; // read-write
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### Implementation (`src/libxrpl/tx/transactors/MyFeature/MyTx.cpp`)
|
||||
```cpp
|
||||
bool MyTransaction::checkExtraFeatures(PreflightContext const& ctx)
|
||||
{ // REQUIRED: gate on amendment
|
||||
return ctx.rules.enabled(featureMyFeature);
|
||||
}
|
||||
|
||||
NotTEC MyTransaction::preflight(PreflightContext const& ctx)
|
||||
{ // Static validation — NO ctx.view, NO ledger access
|
||||
if (ctx.tx[sfAmount] <= beast::zero)
|
||||
return temBAD_AMOUNT;
|
||||
return tesSUCCESS;
|
||||
}
|
||||
|
||||
TER MyTransaction::preclaim(PreclaimContext const& ctx)
|
||||
{ // Read-only — ctx.view.read() only, NO peek/insert/erase
|
||||
if (!ctx.view.exists(keylet::account(ctx.tx[sfAccount])))
|
||||
return terNO_ACCOUNT;
|
||||
return tesSUCCESS;
|
||||
}
|
||||
|
||||
TER MyTransaction::doApply()
|
||||
{ // Mutable — view().peek(), view().insert(), view().update(), view().erase()
|
||||
auto sle = view().peek(keylet::account(account_));
|
||||
sle->setFieldAmount(sfBalance, newBal);
|
||||
view().update(sle); // REQUIRED after mutation
|
||||
return tesSUCCESS;
|
||||
}
|
||||
```
|
||||
|
||||
### Registration Checklist
|
||||
```cpp
|
||||
// ALL of these are REQUIRED for a new transaction type:
|
||||
// 1. transactions.macro: TRANSACTION(ttMY_TYPE, N, MyTx, delegation, fields)
|
||||
// 2. applySteps.cpp: case ttMY_TYPE: return invoke<MyTransaction>(...);
|
||||
// 3. features.macro: XRPL_FEATURE(MyFeature, Supported::yes, DefaultNo)
|
||||
// 4. Feature.h: increment numFeatures
|
||||
// 5. InvariantCheck.cpp: update if new ledger objects created
|
||||
// 6. Batch.cpp: add to disabledTxTypes if not batch-compatible
|
||||
```
|
||||
|
||||
## Transaction Lifecycle
|
||||
|
||||
1. `preflight` (static checks, no ledger) -> `PreflightResult`
|
||||
2. `preclaim` (ledger state, read-only) -> TER
|
||||
3. `operator()` orchestrates: `checkSeqProxy` -> `checkPriorTxAndLastLedger` -> `checkFee` -> `checkSign` -> `apply`
|
||||
4. `Transactor::apply()` runs `consumeSeqProxy` -> `payFee` -> `doApply` and returns a TER
|
||||
5. `operator()` inspects the TER, decides whether to commit (`ctx_.apply`) or discard (`ctx_.discard`/`reset`)
|
||||
|
||||
## State Commitment & tec* Rollback (CRITICAL for review)
|
||||
|
||||
**`doApply` mutations are NOT committed until `ctx_.apply()` is called at the end of `operator()`.** All peek/insert/update/erase during `doApply` go into an `ApplyContext` view (`view_`) layered on top of `base_`. Whether that view gets flushed to `base_` depends entirely on the TER that `doApply` returns.
|
||||
|
||||
`ApplyContext::discard()` ([src/libxrpl/tx/ApplyContext.cpp](src/libxrpl/tx/ApplyContext.cpp)) replaces `view_` with a fresh view on `base_` — **every doApply mutation is thrown away**:
|
||||
```cpp
|
||||
void ApplyContext::discard() { view_.emplace(&base_, flags_); }
|
||||
```
|
||||
|
||||
### Return-code decision table (in `Transactor::operator()`, [src/libxrpl/tx/Transactor.cpp](src/libxrpl/tx/Transactor.cpp))
|
||||
|
||||
| doApply returns | What commits to the ledger |
|
||||
|---|---|
|
||||
| `tesSUCCESS` | All doApply mutations + fee + seq (via `ctx_.apply`) |
|
||||
| `tec*` (normal, `!tapRETRY`) | `reset(fee)` calls `discard()`, then re-applies fee + seq only. **All doApply mutations reverted.** |
|
||||
| `tec*` with `tapFAIL_HARD` | `discard()` called directly, nothing committed (not even fee) |
|
||||
| `tec*` with `tapRETRY` | `applied=false`, `ctx_.apply` never called, tx re-queued |
|
||||
| `tef*` / `tem*` / `ter*` | `applied=false`, `ctx_.apply` never called |
|
||||
| `tecINVARIANT_FAILED` after invariants | reset again, commit fee only |
|
||||
|
||||
`isTecClaimHardFail(ter, flags) = isTecClaim(ter) && !(flags & tapRETRY)` ([include/xrpl/tx/applySteps.h](include/xrpl/tx/applySteps.h)) — this predicate is what drives the reset path for normal consensus application.
|
||||
|
||||
### What this means for transactor authors and reviewers
|
||||
|
||||
- **A `tec*` return from doApply acts as a full-transaction rollback.** You do NOT need to order mutations defensively so that all checks come before any state changes. If a helper called late in doApply returns `tec*`, everything mutated earlier in the same doApply is discarded via `discard()`.
|
||||
- **Orphan-state bugs of the form "we mutated X then returned tec* so X is now in an inconsistent state" are not possible at the transactor boundary.** The ApplyContext isolates the whole doApply as an atomic unit.
|
||||
- **The real failure mode is within `doApply` itself**: if you call `view().update(sle)` on a stale SLE pointer, or mutate a variable you read by value instead of peek, those are real bugs — but they are in-memory bugs, not state-commit bugs.
|
||||
- **Sandboxes inside `doApply` add nesting, not safety.** `PaymentSandbox` / nested `ApplyView` are useful when you need to conditionally commit a subset of changes *within* a single doApply (e.g., apply offers but revert if the net outcome fails). They are not needed to protect against doApply's own `tec*` return — that rollback is automatic.
|
||||
- **Only `ctx_.apply(result)` publishes to `base_`**; a doApply that `return`s early, throws, or crashes never reaches that call, so base_ stays clean.
|
||||
|
||||
### Verifying a suspected orphan-state bug
|
||||
|
||||
Before claiming "directory removed but SLE not erased because tec\*":
|
||||
1. Read the caller of `doApply` — confirm the TER path (`operator()` in Transactor.cpp).
|
||||
2. Check whether `discard()` is reached via `reset()` or the `tapFAIL_HARD` branch.
|
||||
3. If both paths call `discard()`, the mutations cannot persist on tec\*.
|
||||
4. Look instead for: missing `view().update(sle)` after mutation, stale SLE pointers, or genuine non-atomic side effects (e.g., hash router flags, which are NOT in the ApplyContext view).
|
||||
|
||||
## Permission System
|
||||
|
||||
- `checkSign` dispatches to `checkSingleSign`, `checkMultiSign`, or `checkBatchSign`
|
||||
- `checkPermission` validates delegated authority for delegatable transaction types
|
||||
- Multi-sign requires M-of-N signers matching the signer list; weight threshold must be met
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/xrpld/app/tx/detail/Transactor.cpp` - base class and pipeline
|
||||
- `include/xrpl/protocol/detail/transactions.macro` - type definitions
|
||||
- `src/xrpld/app/tx/detail/` - per-type implementations (Payment.cpp, OfferCreate.cpp, etc.)
|
||||
- `src/xrpld/app/tx/detail/InvariantCheck.cpp` - post-execution invariant checks
|
||||
62
docs/skills/soul/websockets.md
Normal file
62
docs/skills/soul/websockets.md
Normal file
@@ -0,0 +1,62 @@
|
||||
# WebSockets
|
||||
|
||||
Async WebSocket support for client RPC and real-time subscriptions. Both plain and SSL. Built on Boost.Beast + Boost.Asio.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- All async operations run on a per-session Boost.Asio strand for thread safety
|
||||
- Outgoing message queue (`wq_`) has a per-session limit (`port().ws_queue_limit`); exceeding it closes the connection with policy error "client is too slow"
|
||||
- `complete()` must be called after processing each message to resume reading; forgetting it stalls the session
|
||||
- `WSInfoSub` holds a weak pointer to `WSSession`; dead sessions are automatically skipped during event delivery
|
||||
- Message size limit: `RPC::Tuning::maxRequestSize`; oversized messages get a `jsonInvalid` error response
|
||||
|
||||
## Connection Flow
|
||||
|
||||
1. `Door` accepts TCP -> `Detector` probes for SSL vs plain -> creates `SSLHTTPPeer` or `PlainHTTPPeer`
|
||||
2. HTTP request with WebSocket upgrade -> `ServerHandler::onHandoff` -> `session.websocketUpgrade()` -> creates `PlainWSPeer` or `SSLWSPeer`
|
||||
3. `BaseWSPeer::run()` -> set permessage-deflate options -> `async_accept` handshake -> `on_ws_handshake` -> `do_read` loop
|
||||
4. `on_read` -> `ServerHandler::onWSMessage` -> validate JSON -> post to job queue -> `processSession` -> send response -> `complete()`
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- `on_read` consumes the buffer AFTER calling `onWSMessage`; if the handler accesses the buffer asynchronously after `onWSMessage` returns, it reads garbage
|
||||
- `close()` with pending messages defers the actual close; calling `send()` after `close()` but before actual close queues more messages
|
||||
- Missing `complete()` call after sending response = session never reads again = appears hung
|
||||
- Job queue shutdown: if `postCoro` returns nullptr, session must close with `going_away`; dropping this silently leaks sessions
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- Verify strand execution for all socket operations (read, write, close, timer)
|
||||
- New subscription streams: ensure `WSInfoSub::send` handles the new event type
|
||||
- Flow control: test with slow clients to verify queue limit enforcement
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### complete() Resumes Read Loop
|
||||
```cpp
|
||||
// REQUIRED after sending response — missing this = session hangs forever
|
||||
void BaseWSPeer::complete()
|
||||
{
|
||||
if (!strand_.running_in_this_thread())
|
||||
return post(strand_, std::bind(
|
||||
&BaseWSPeer::complete, impl().shared_from_this()));
|
||||
do_read(); // resume reading next message
|
||||
}
|
||||
```
|
||||
|
||||
### Queue Limit Enforcement
|
||||
```cpp
|
||||
// REQUIRED: close slow clients to prevent memory exhaustion
|
||||
if (wq_.size() >= port().ws_queue_limit)
|
||||
{
|
||||
close(boost::beast::websocket::close_code::policy_error);
|
||||
return; // do NOT queue unboundedly
|
||||
}
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `include/xrpl/server/detail/BaseWSPeer.h` - session lifecycle and message queue
|
||||
- `include/xrpl/server/detail/Door.h` - connection acceptance
|
||||
- `src/xrpld/rpc/detail/ServerHandler.cpp` - `onWSMessage` and `onHandoff`
|
||||
- `src/xrpld/rpc/detail/WSInfoSub.h` - subscription delivery
|
||||
Reference in New Issue
Block a user