mirror of
https://github.com/XRPLF/rippled.git
synced 2026-06-02 16:26:48 +00:00
add doc-agent
This commit is contained in:
7
.github/scripts/doc-agent/.gitignore
vendored
Normal file
7
.github/scripts/doc-agent/.gitignore
vendored
Normal file
@@ -0,0 +1,7 @@
|
||||
node_modules/
|
||||
dist/
|
||||
*.log
|
||||
.env
|
||||
.env.local
|
||||
doc-review-report.md
|
||||
doc-review-comments.json
|
||||
101
.github/scripts/doc-agent/README.md
vendored
Normal file
101
.github/scripts/doc-agent/README.md
vendored
Normal file
@@ -0,0 +1,101 @@
|
||||
# doc-agent
|
||||
|
||||
Automated documentation agent for the xrpld C++ codebase. Built on the
|
||||
Claude Agent SDK.
|
||||
|
||||
## What it does
|
||||
|
||||
Two modes:
|
||||
|
||||
- **document** — Add Doxygen `/** */` documentation to a C++ file or
|
||||
directory. The agent reads the file, related tests, and module skill
|
||||
context, then writes documentation comments per the project standards in
|
||||
`docs/DOCUMENTATION_STANDARDS.md`.
|
||||
- **review** — Given a git diff range, detect documentation drift. Used by
|
||||
the `doc-review` GitHub Action and locally for testing.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Node.js >= 20
|
||||
- `ANTHROPIC_API_KEY` environment variable
|
||||
- Tools the agent uses: `git`, `gh` (for `--pr`)
|
||||
|
||||
## Install
|
||||
|
||||
```sh
|
||||
cd .github/scripts/doc-agent
|
||||
npm install
|
||||
```
|
||||
|
||||
## Build and lint
|
||||
|
||||
```sh
|
||||
npm run typecheck # type check without emitting
|
||||
npm run build # compile to dist/
|
||||
npm run lint # biome lint
|
||||
npm run format # biome format --write
|
||||
npm run check # lint + format check (read-only)
|
||||
npm run check:fix # lint + format + fix
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```sh
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
|
||||
# Document a single file
|
||||
npm run document include/xrpl/basics/base_uint.h
|
||||
|
||||
# Document an entire module
|
||||
npm run document include/xrpl/basics/
|
||||
|
||||
# Review a git range
|
||||
npm run review develop..HEAD
|
||||
|
||||
# Review a PR
|
||||
npm run review -- --pr 1234
|
||||
```
|
||||
|
||||
When invoked outside the xrpld repo, set `XRPLD_ROOT` to the path of the
|
||||
checkout you want to operate on.
|
||||
|
||||
## Outputs
|
||||
|
||||
The `review` mode writes two files in the current directory:
|
||||
|
||||
- `doc-review-report.md` — markdown summary, posted as the PR comment
|
||||
- `doc-review-comments.json` — array of inline review comments, posted
|
||||
individually on the PR diff
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
doc-agent/
|
||||
├── package.json
|
||||
├── tsconfig.json
|
||||
├── biome.json
|
||||
├── prompts/
|
||||
│ ├── document-file.md # System prompt for documentation mode
|
||||
│ └── review-diff.md # System prompt for review mode
|
||||
└── src/
|
||||
├── index.ts # CLI entry point
|
||||
├── config.ts # Paths, model, module-skill map
|
||||
├── prompt-loader.ts # Loads prompts + module skill context
|
||||
├── document.ts # Document mode
|
||||
├── review.ts # Review mode
|
||||
└── types.ts # Shared types
|
||||
```
|
||||
|
||||
## Module skills
|
||||
|
||||
The agent injects per-module context from `docs/skills/soul/*.md` into its
|
||||
system prompt based on the file path being processed. The mapping lives in
|
||||
`src/config.ts` (`MODULE_SKILL_MAP`).
|
||||
|
||||
## Notes
|
||||
|
||||
- Prompts live in markdown files, not source, so they can be edited without
|
||||
touching code.
|
||||
- The `document` mode uses `permissionMode: 'acceptEdits'` so the agent
|
||||
writes directly to the target files. Run against a clean git tree so you
|
||||
can review and revert if needed.
|
||||
57
.github/scripts/doc-agent/biome.json
vendored
Normal file
57
.github/scripts/doc-agent/biome.json
vendored
Normal file
@@ -0,0 +1,57 @@
|
||||
{
|
||||
"$schema": "https://biomejs.dev/schemas/1.9.4/schema.json",
|
||||
"vcs": {
|
||||
"enabled": true,
|
||||
"clientKind": "git",
|
||||
"useIgnoreFile": true
|
||||
},
|
||||
"files": {
|
||||
"ignoreUnknown": false,
|
||||
"ignore": ["dist", "node_modules"]
|
||||
},
|
||||
"formatter": {
|
||||
"enabled": true,
|
||||
"indentStyle": "space",
|
||||
"indentWidth": 2,
|
||||
"lineWidth": 100,
|
||||
"lineEnding": "lf"
|
||||
},
|
||||
"javascript": {
|
||||
"formatter": {
|
||||
"quoteStyle": "single",
|
||||
"trailingCommas": "all",
|
||||
"semicolons": "always",
|
||||
"arrowParentheses": "always"
|
||||
}
|
||||
},
|
||||
"linter": {
|
||||
"enabled": true,
|
||||
"rules": {
|
||||
"recommended": true,
|
||||
"correctness": {
|
||||
"noUnusedVariables": "error",
|
||||
"noUnusedImports": "error",
|
||||
"useExhaustiveDependencies": "error"
|
||||
},
|
||||
"style": {
|
||||
"useConst": "error",
|
||||
"useTemplate": "error",
|
||||
"useImportType": "error",
|
||||
"useExportType": "error",
|
||||
"noNonNullAssertion": "warn"
|
||||
},
|
||||
"suspicious": {
|
||||
"noExplicitAny": "error",
|
||||
"noConsoleLog": "off"
|
||||
},
|
||||
"complexity": {
|
||||
"noUselessTypeConstraint": "error",
|
||||
"useArrowFunction": "error",
|
||||
"useLiteralKeys": "off"
|
||||
}
|
||||
}
|
||||
},
|
||||
"organizeImports": {
|
||||
"enabled": true
|
||||
}
|
||||
}
|
||||
1123
.github/scripts/doc-agent/package-lock.json
generated
vendored
Normal file
1123
.github/scripts/doc-agent/package-lock.json
generated
vendored
Normal file
File diff suppressed because it is too large
Load Diff
34
.github/scripts/doc-agent/package.json
vendored
Normal file
34
.github/scripts/doc-agent/package.json
vendored
Normal file
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"name": "xrpld-doc-agent",
|
||||
"version": "0.1.0",
|
||||
"description": "Automated documentation agent for the xrpld C++ codebase. Uses the Claude Agent SDK to generate Doxygen documentation and detect doc drift on PRs.",
|
||||
"type": "module",
|
||||
"private": true,
|
||||
"bin": {
|
||||
"doc-agent": "./dist/index.js"
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"start": "node dist/index.js",
|
||||
"dev": "tsx src/index.ts",
|
||||
"document": "tsx src/index.ts document",
|
||||
"review": "tsx src/index.ts review",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"lint": "biome lint src",
|
||||
"format": "biome format --write src",
|
||||
"check": "biome check src",
|
||||
"check:fix": "biome check --write src"
|
||||
},
|
||||
"dependencies": {
|
||||
"@anthropic-ai/claude-agent-sdk": "^0.1.10"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@biomejs/biome": "^1.9.4",
|
||||
"@types/node": "^22.10.0",
|
||||
"tsx": "^4.19.2",
|
||||
"typescript": "^5.7.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
}
|
||||
}
|
||||
63
.github/scripts/doc-agent/prompts/document-file.md
vendored
Normal file
63
.github/scripts/doc-agent/prompts/document-file.md
vendored
Normal file
@@ -0,0 +1,63 @@
|
||||
You are documenting C++ code in the xrpld (XRP Ledger daemon) codebase.
|
||||
|
||||
Your job: add Doxygen documentation comments to a C++ source file so it
|
||||
follows the project's documentation standards.
|
||||
|
||||
## Documentation Standards
|
||||
|
||||
Read `docs/DOCUMENTATION_STANDARDS.md` for the full specification. Key rules:
|
||||
|
||||
- Use `/** ... */` Javadoc-style Doxygen comments (dominant pattern in the
|
||||
codebase)
|
||||
- For multi-line comments, prefix each line with ` * ` (space, asterisk, space)
|
||||
- Document every public class, struct, function, and enum
|
||||
- Document public methods with `@param`, `@return`, `@throw`/`@throws`, `@note`
|
||||
- Continuation lines for `@param` descriptions indent 4 spaces from the `*`
|
||||
- Document `.cpp` files only where the algorithm or invariant is non-obvious
|
||||
- `JAVADOC_AUTOBRIEF = YES` — the first sentence is automatically the brief,
|
||||
so `@brief` is optional
|
||||
|
||||
## Quality Rules
|
||||
|
||||
- **Never paraphrase the signature.** `/** Returns the account ID. */` on
|
||||
`AccountID getAccountID()` is worse than no doc.
|
||||
- **Document behavior, invariants, and the WHY.** What does this function do
|
||||
in terms a developer can use? What can go wrong? What's the contract?
|
||||
- **Read the implementation before writing the doc.** Don't guess what the
|
||||
function does — read it.
|
||||
- **Cross-reference test files** to find edge cases worth documenting in
|
||||
`@note` tags.
|
||||
- **Be terse.** Target 2-5 lines for classes, 1-3 for functions, plus tag
|
||||
lines. If you need a multi-paragraph essay, the code probably needs help.
|
||||
- **Wrong docs are worse than no docs.** If you're not sure what the code
|
||||
does, say so — don't invent.
|
||||
|
||||
## Module Context
|
||||
|
||||
Before you start, read the relevant skill file in `docs/skills/soul/` for
|
||||
the module you're working on. These capture per-module conventions, key
|
||||
classes, and gotchas:
|
||||
|
||||
- `basics`, `crypto`, `json`, `beast` — foundation utilities
|
||||
- `protocol` — STObject, SField, Serializer, TER codes, Features, Keylets
|
||||
- `ledger` — ReadView/ApplyView, state tables, payment sandbox
|
||||
- `tx` / `transactors` — transaction pipeline
|
||||
- `consensus`, `peering`, `nodestore`, `shamap`, `rpc` — see `docs/skills/soul/`
|
||||
|
||||
## Process
|
||||
|
||||
1. Read the target file completely
|
||||
2. Read the corresponding skill file in `docs/skills/soul/` if one applies
|
||||
3. Identify entities that need documentation (public classes, structs,
|
||||
public methods, free functions in headers, enums)
|
||||
4. For each entity: read the implementation (and tests if helpful), then
|
||||
write a Doxygen comment that captures behavior and intent
|
||||
5. Use the Edit tool to add the comments to the file
|
||||
6. Do NOT modify code logic — only add documentation
|
||||
7. Do NOT add documentation to entities that don't need it (private members
|
||||
with obvious purpose, simple getters where the name is self-explanatory)
|
||||
|
||||
When you finish, summarize:
|
||||
- How many entities you documented
|
||||
- Any entities you skipped and why
|
||||
- Any code patterns you discovered that should be added to a skill file
|
||||
55
.github/scripts/doc-agent/prompts/review-diff.md
vendored
Normal file
55
.github/scripts/doc-agent/prompts/review-diff.md
vendored
Normal file
@@ -0,0 +1,55 @@
|
||||
You are reviewing a pull request to the xrpld (XRP Ledger daemon) codebase
|
||||
for documentation drift.
|
||||
|
||||
Your job: given a git diff, determine whether the changes invalidate
|
||||
existing Doxygen documentation comments, or introduce new public API
|
||||
surface that lacks documentation.
|
||||
|
||||
## Rules
|
||||
|
||||
- Only flag REAL semantic drift: changed behavior, new parameters, removed
|
||||
functionality, changed return values, new error conditions, changed
|
||||
invariants.
|
||||
- Do NOT flag cosmetic changes (whitespace, formatting, internal renames
|
||||
that don't change semantics).
|
||||
- Do NOT suggest docs for private implementation details unless the logic
|
||||
is genuinely non-obvious.
|
||||
- Do NOT paraphrase function signatures. Good docs explain WHY and what
|
||||
BEHAVIOR — not what the code literally does.
|
||||
- Be terse: 1-3 sentences per finding.
|
||||
|
||||
## Process
|
||||
|
||||
1. For each changed file, get the git diff and the current file content
|
||||
2. Read existing doc comments on the modified entities
|
||||
3. For each modified entity, ask:
|
||||
- Did behavior change in a way the docs miss?
|
||||
- Did parameters or return values change?
|
||||
- Are there new error conditions?
|
||||
- Did the contract / invariant change?
|
||||
- Is this a NEW public API surface with no docs?
|
||||
4. Read the module's skill file in `docs/skills/soul/` for context
|
||||
5. Read related tests if it helps you understand the change
|
||||
6. Output findings as structured JSON (see below)
|
||||
|
||||
## Output Format
|
||||
|
||||
```json
|
||||
{
|
||||
"summary": "One-paragraph summary of doc state for this PR",
|
||||
"issues": [
|
||||
{
|
||||
"file": "include/xrpl/protocol/Payment.h",
|
||||
"line": 42,
|
||||
"severity": "warning" | "suggestion",
|
||||
"message": "Brief description of the doc issue",
|
||||
"suggested_doc": "Optional: suggested doc comment text"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- `severity: warning` = doc is now incorrect / misleading
|
||||
- `severity: suggestion` = new code lacks docs, would be nice to add
|
||||
|
||||
If no issues found, return `{"summary": "Documentation is up to date.", "issues": []}`.
|
||||
77
.github/scripts/doc-agent/src/config.ts
vendored
Normal file
77
.github/scripts/doc-agent/src/config.ts
vendored
Normal file
@@ -0,0 +1,77 @@
|
||||
/**
|
||||
* Shared configuration for doc-agent.
|
||||
*
|
||||
* Paths are resolved relative to the doc-agent directory so the tool works
|
||||
* regardless of where it's invoked from.
|
||||
*/
|
||||
|
||||
import { dirname, resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
|
||||
/** Absolute path to the doc-agent root (parent of src/). */
|
||||
export const AGENT_DIR: string = resolve(__dirname, '..');
|
||||
|
||||
/** Absolute path to the prompts directory. */
|
||||
export const PROMPTS_DIR: string = resolve(AGENT_DIR, 'prompts');
|
||||
|
||||
/**
|
||||
* Absolute path to the xrpld repo root.
|
||||
*
|
||||
* Defaults to three levels up from doc-agent (which lives at
|
||||
* .github/scripts/doc-agent/). Override with the XRPLD_ROOT env var when
|
||||
* running against a different checkout.
|
||||
*/
|
||||
export const XRPLD_ROOT: string = process.env['XRPLD_ROOT'] ?? resolve(AGENT_DIR, '..', '..', '..');
|
||||
|
||||
/** Model used for documentation generation and review. */
|
||||
export const MODEL: string = process.env['DOC_AGENT_MODEL'] ?? 'claude-opus-4-7';
|
||||
|
||||
/** Absolute path to the skills directory inside the xrpld repo. */
|
||||
export const SKILLS_DIR: string = resolve(XRPLD_ROOT, 'docs', 'skills');
|
||||
|
||||
/**
|
||||
* Map module path prefixes to their skill file name in docs/skills/soul/.
|
||||
*
|
||||
* Used to inject module-specific context into the agent's system prompt
|
||||
* when documenting or reviewing code in that module.
|
||||
*/
|
||||
export const MODULE_SKILL_MAP: Readonly<Record<string, string | null>> = {
|
||||
'src/libxrpl/basics/': null,
|
||||
'src/libxrpl/crypto/': 'cryptography.md',
|
||||
'src/libxrpl/json/': null,
|
||||
'src/libxrpl/beast/': null,
|
||||
'src/libxrpl/protocol/': 'protocol.md',
|
||||
'src/libxrpl/ledger/': 'ledger.md',
|
||||
'src/libxrpl/tx/': 'transactors.md',
|
||||
'src/libxrpl/nodestore/': 'nodestore.md',
|
||||
'src/libxrpl/shamap/': 'shamap.md',
|
||||
'src/libxrpl/rdb/': 'sql.md',
|
||||
'src/xrpld/consensus/': 'consensus.md',
|
||||
'src/xrpld/overlay/': 'peering.md',
|
||||
'src/xrpld/peerfinder/': 'peering.md',
|
||||
'src/xrpld/rpc/': 'rpc.md',
|
||||
'include/xrpl/crypto/': 'cryptography.md',
|
||||
'include/xrpl/protocol/': 'protocol.md',
|
||||
'include/xrpl/ledger/': 'ledger.md',
|
||||
'include/xrpl/tx/': 'transactors.md',
|
||||
'include/xrpl/nodestore/': 'nodestore.md',
|
||||
'include/xrpl/shamap/': 'shamap.md',
|
||||
};
|
||||
|
||||
/**
|
||||
* Resolve which skill file applies to a given source path.
|
||||
*
|
||||
* @param sourcePath - Path relative to the xrpld repo root
|
||||
* @returns The skill file name, or null if no skill applies
|
||||
*/
|
||||
export function skillForPath(sourcePath: string): string | null {
|
||||
for (const [prefix, skillFile] of Object.entries(MODULE_SKILL_MAP)) {
|
||||
if (sourcePath.startsWith(prefix) || sourcePath.includes(`/${prefix}`)) {
|
||||
return skillFile;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
114
.github/scripts/doc-agent/src/document.ts
vendored
Normal file
114
.github/scripts/doc-agent/src/document.ts
vendored
Normal file
@@ -0,0 +1,114 @@
|
||||
/**
|
||||
* Document mode: add Doxygen docs to a file or all files in a directory.
|
||||
*/
|
||||
|
||||
import { existsSync, readdirSync, statSync } from 'node:fs';
|
||||
import { join, relative, resolve } from 'node:path';
|
||||
import { query } from '@anthropic-ai/claude-agent-sdk';
|
||||
import { MODEL, XRPLD_ROOT } from './config.js';
|
||||
import { loadSystemPrompt } from './prompt-loader.js';
|
||||
|
||||
const CPP_EXTENSIONS: ReadonlySet<string> = new Set(['.h', '.hpp', '.cpp']);
|
||||
|
||||
/**
|
||||
* Recursively find all C++ source files under a target path.
|
||||
*
|
||||
* @param target - File or directory path (relative to xrpld root or absolute)
|
||||
* @returns Absolute paths of all matching files
|
||||
*/
|
||||
function findCppFiles(target: string): string[] {
|
||||
const absTarget = resolve(XRPLD_ROOT, target);
|
||||
if (!existsSync(absTarget)) {
|
||||
throw new Error(`Target does not exist: ${absTarget}`);
|
||||
}
|
||||
|
||||
const stat = statSync(absTarget);
|
||||
if (stat.isFile()) {
|
||||
return [absTarget];
|
||||
}
|
||||
|
||||
const results: string[] = [];
|
||||
const walk = (dir: string): void => {
|
||||
for (const entry of readdirSync(dir, { withFileTypes: true })) {
|
||||
const full = join(dir, entry.name);
|
||||
if (entry.isDirectory()) {
|
||||
walk(full);
|
||||
} else if (entry.isFile()) {
|
||||
const dotIdx = entry.name.lastIndexOf('.');
|
||||
if (dotIdx === -1) continue;
|
||||
const ext = entry.name.slice(dotIdx);
|
||||
if (CPP_EXTENSIONS.has(ext)) {
|
||||
results.push(full);
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
walk(absTarget);
|
||||
return results;
|
||||
}
|
||||
|
||||
/**
|
||||
* Document a single file by running the documentation agent against it.
|
||||
*/
|
||||
async function documentFile(absPath: string): Promise<void> {
|
||||
const relPath = relative(XRPLD_ROOT, absPath);
|
||||
console.log(`\n=== Documenting: ${relPath} ===`);
|
||||
|
||||
const systemPrompt = await loadSystemPrompt('document-file', relPath);
|
||||
const userPrompt = `Add Doxygen documentation to: ${relPath}
|
||||
|
||||
The file is rooted at ${XRPLD_ROOT}. Use the Read tool to read it, the Edit
|
||||
tool to add documentation, and Glob/Grep to find related tests or callers
|
||||
when needed.
|
||||
|
||||
Do not modify any code logic — only add documentation comments.`;
|
||||
|
||||
const result = query({
|
||||
prompt: userPrompt,
|
||||
options: {
|
||||
model: MODEL,
|
||||
systemPrompt,
|
||||
cwd: XRPLD_ROOT,
|
||||
allowedTools: ['Read', 'Edit', 'Glob', 'Grep', 'Bash'],
|
||||
permissionMode: 'acceptEdits',
|
||||
},
|
||||
});
|
||||
|
||||
for await (const message of result) {
|
||||
if (message.type === 'assistant') {
|
||||
const content = message.message?.content;
|
||||
if (Array.isArray(content)) {
|
||||
for (const block of content) {
|
||||
if (block.type === 'text') {
|
||||
process.stdout.write(block.text);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if (message.type === 'result') {
|
||||
const cost = message.total_cost_usd?.toFixed(4) ?? '?';
|
||||
const inTok = message.usage?.['input_tokens'] ?? 0;
|
||||
const outTok = message.usage?.['output_tokens'] ?? 0;
|
||||
console.log(`\n[Cost: $${cost}, Tokens: ${inTok}/${outTok}]`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Document a file or every C++ file under a directory.
|
||||
*
|
||||
* @param target - File or directory path
|
||||
*/
|
||||
export async function documentTarget(target: string): Promise<void> {
|
||||
const files = findCppFiles(target);
|
||||
console.log(`Found ${files.length} C++ file(s) to document.`);
|
||||
|
||||
for (const file of files) {
|
||||
try {
|
||||
await documentFile(file);
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
console.error(`Failed to document ${file}: ${message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
69
.github/scripts/doc-agent/src/index.ts
vendored
Normal file
69
.github/scripts/doc-agent/src/index.ts
vendored
Normal file
@@ -0,0 +1,69 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* xrpld doc-agent CLI entry point.
|
||||
*
|
||||
* @example
|
||||
* doc-agent document src/libxrpl/basics/base_uint.h
|
||||
* doc-agent document include/xrpl/basics/
|
||||
* doc-agent review develop..HEAD
|
||||
* doc-agent review --pr 1234
|
||||
*/
|
||||
|
||||
import { documentTarget } from './document.js';
|
||||
import { reviewDiff } from './review.js';
|
||||
|
||||
const USAGE = `
|
||||
xrpld doc-agent
|
||||
|
||||
Usage:
|
||||
doc-agent document <file-or-directory> Add Doxygen documentation
|
||||
doc-agent review <base>..<head> Detect doc drift in range
|
||||
doc-agent review --pr <number> Detect doc drift for a PR
|
||||
|
||||
Environment:
|
||||
ANTHROPIC_API_KEY (required) Anthropic API key
|
||||
XRPLD_ROOT (optional) Path to xrpld repo root (default: repo root)
|
||||
DOC_AGENT_MODEL (optional) Model override (default: claude-opus-4-7)
|
||||
`;
|
||||
|
||||
function printUsageAndExit(code: number): never {
|
||||
console.error(USAGE);
|
||||
process.exit(code);
|
||||
}
|
||||
|
||||
const HELP_MODES: ReadonlySet<string> = new Set(['help', '--help', '-h']);
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const [mode, ...args] = process.argv.slice(2);
|
||||
|
||||
if (process.env['ANTHROPIC_API_KEY'] === undefined) {
|
||||
console.error('ERROR: ANTHROPIC_API_KEY environment variable is required.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
if (mode === undefined || HELP_MODES.has(mode)) {
|
||||
printUsageAndExit(0);
|
||||
}
|
||||
|
||||
if (mode === 'document') {
|
||||
const target = args[0];
|
||||
if (target === undefined) printUsageAndExit(1);
|
||||
await documentTarget(target);
|
||||
return;
|
||||
}
|
||||
|
||||
if (mode === 'review') {
|
||||
if (args.length === 0) printUsageAndExit(1);
|
||||
await reviewDiff(args);
|
||||
return;
|
||||
}
|
||||
|
||||
console.error(`Unknown mode: ${mode}`);
|
||||
printUsageAndExit(1);
|
||||
}
|
||||
|
||||
main().catch((err: unknown) => {
|
||||
const message = err instanceof Error ? (err.stack ?? err.message) : String(err);
|
||||
console.error('FATAL:', message);
|
||||
process.exit(1);
|
||||
});
|
||||
34
.github/scripts/doc-agent/src/prompt-loader.ts
vendored
Normal file
34
.github/scripts/doc-agent/src/prompt-loader.ts
vendored
Normal file
@@ -0,0 +1,34 @@
|
||||
/**
|
||||
* Loads system prompts and injects module-specific skill context.
|
||||
*/
|
||||
|
||||
import { existsSync } from 'node:fs';
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import { resolve } from 'node:path';
|
||||
import { PROMPTS_DIR, SKILLS_DIR, skillForPath } from './config.js';
|
||||
|
||||
/**
|
||||
* Load a system prompt from prompts/ and append the relevant module skill
|
||||
* if one applies to the given source path.
|
||||
*
|
||||
* @param promptName - Base name of the prompt file (without .md extension)
|
||||
* @param sourcePath - Path relative to the xrpld repo root
|
||||
* @returns The fully-assembled system prompt
|
||||
*/
|
||||
export async function loadSystemPrompt(promptName: string, sourcePath: string): Promise<string> {
|
||||
const basePromptPath = resolve(PROMPTS_DIR, `${promptName}.md`);
|
||||
const basePrompt = await readFile(basePromptPath, 'utf8');
|
||||
|
||||
const skillFile = skillForPath(sourcePath);
|
||||
if (skillFile === null) {
|
||||
return basePrompt;
|
||||
}
|
||||
|
||||
const skillPath = resolve(SKILLS_DIR, 'soul', skillFile);
|
||||
if (!existsSync(skillPath)) {
|
||||
return basePrompt;
|
||||
}
|
||||
|
||||
const skill = await readFile(skillPath, 'utf8');
|
||||
return `${basePrompt}\n\n## Module Skill (${skillFile})\n\n${skill}`;
|
||||
}
|
||||
222
.github/scripts/doc-agent/src/review.ts
vendored
Normal file
222
.github/scripts/doc-agent/src/review.ts
vendored
Normal file
@@ -0,0 +1,222 @@
|
||||
/**
|
||||
* Review mode: detect documentation drift in a git diff range.
|
||||
*
|
||||
* Used by the doc-review GitHub Action and locally for testing.
|
||||
*/
|
||||
|
||||
import { execSync } from 'node:child_process';
|
||||
import { writeFile } from 'node:fs/promises';
|
||||
import { query } from '@anthropic-ai/claude-agent-sdk';
|
||||
import { MODEL, XRPLD_ROOT } from './config.js';
|
||||
import { loadSystemPrompt } from './prompt-loader.js';
|
||||
import type { FileReviewResult, GitRange, ReviewIssue, ReviewOutput } from './types.js';
|
||||
|
||||
const MAX_DIFF_CHARS = 12_000;
|
||||
const TRACKED_PATH_PATTERN = /^(include|src\/libxrpl|src\/xrpld)\//;
|
||||
const CPP_FILE_PATTERN = /\.(h|hpp|cpp)$/;
|
||||
|
||||
/**
|
||||
* Parse the CLI arguments into a base..head git range.
|
||||
*
|
||||
* Accepts either:
|
||||
* - `base..head` (e.g. `develop..HEAD`)
|
||||
* - `--pr <number>` (resolves via `gh pr view`)
|
||||
*/
|
||||
function parseRangeArgs(args: readonly string[]): GitRange {
|
||||
const first = args[0];
|
||||
if (first === undefined) {
|
||||
throw new Error('Expected range as base..head or --pr <number>');
|
||||
}
|
||||
|
||||
if (first === '--pr') {
|
||||
const pr = args[1];
|
||||
if (pr === undefined) {
|
||||
throw new Error('--pr requires a PR number');
|
||||
}
|
||||
const base = execSync(`gh pr view ${pr} --json baseRefOid -q .baseRefOid`, {
|
||||
cwd: XRPLD_ROOT,
|
||||
})
|
||||
.toString()
|
||||
.trim();
|
||||
const head = execSync(`gh pr view ${pr} --json headRefOid -q .headRefOid`, {
|
||||
cwd: XRPLD_ROOT,
|
||||
})
|
||||
.toString()
|
||||
.trim();
|
||||
return { base, head };
|
||||
}
|
||||
|
||||
const match = first.match(/^([^.]+)\.\.([^.]+)$/);
|
||||
if (match === null || match[1] === undefined || match[2] === undefined) {
|
||||
throw new Error('Expected range as base..head or --pr <number>');
|
||||
}
|
||||
return { base: match[1], head: match[2] };
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the list of C++ source files changed in the given git range,
|
||||
* filtered to paths the doc-agent cares about.
|
||||
*/
|
||||
function getChangedCppFiles(range: GitRange): string[] {
|
||||
const out = execSync(`git diff --name-only ${range.base}...${range.head}`, {
|
||||
cwd: XRPLD_ROOT,
|
||||
}).toString();
|
||||
|
||||
return out
|
||||
.split('\n')
|
||||
.filter((line) => line.length > 0)
|
||||
.filter((file) => CPP_FILE_PATTERN.test(file))
|
||||
.filter((file) => TRACKED_PATH_PATTERN.test(file));
|
||||
}
|
||||
|
||||
/** Get the unified diff for a single file in the given range. */
|
||||
function getFileDiff(range: GitRange, file: string): string {
|
||||
return execSync(`git diff ${range.base}...${range.head} -- "${file}"`, {
|
||||
cwd: XRPLD_ROOT,
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
}).toString();
|
||||
}
|
||||
|
||||
/** Extract a JSON object from a possibly-fenced model response. */
|
||||
function extractJson(response: string): ReviewOutput | null {
|
||||
const fenced = response.match(/```json\s*([\s\S]*?)```/);
|
||||
const raw = fenced?.[1] ?? response.match(/(\{[\s\S]*\})/)?.[1];
|
||||
if (raw === undefined) return null;
|
||||
try {
|
||||
return JSON.parse(raw) as ReviewOutput;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Send one file's diff to the agent and parse the response. */
|
||||
async function reviewFile(range: GitRange, file: string): Promise<FileReviewResult | null> {
|
||||
console.log(`\n=== Reviewing: ${file} ===`);
|
||||
const diff = getFileDiff(range, file);
|
||||
if (diff.trim().length === 0) return null;
|
||||
|
||||
const systemPrompt = await loadSystemPrompt('review-diff', file);
|
||||
const userPrompt = `Review this diff for documentation drift:
|
||||
|
||||
## File: ${file}
|
||||
|
||||
## Diff
|
||||
\`\`\`
|
||||
${diff.slice(0, MAX_DIFF_CHARS)}
|
||||
\`\`\`
|
||||
|
||||
Use the Read tool to inspect the current state of the file, related tests,
|
||||
or callers if needed. Output findings as JSON per the schema in the system
|
||||
prompt.`;
|
||||
|
||||
let response = '';
|
||||
const result = query({
|
||||
prompt: userPrompt,
|
||||
options: {
|
||||
model: MODEL,
|
||||
systemPrompt,
|
||||
cwd: XRPLD_ROOT,
|
||||
allowedTools: ['Read', 'Glob', 'Grep', 'Bash'],
|
||||
permissionMode: 'acceptEdits',
|
||||
},
|
||||
});
|
||||
|
||||
for await (const message of result) {
|
||||
if (message.type === 'assistant') {
|
||||
const content = message.message?.content;
|
||||
if (Array.isArray(content)) {
|
||||
for (const block of content) {
|
||||
if (block.type === 'text') {
|
||||
response += block.text;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const parsed = extractJson(response);
|
||||
if (parsed === null) {
|
||||
console.warn(` No JSON output for ${file}, skipping`);
|
||||
return null;
|
||||
}
|
||||
|
||||
const issues: ReviewIssue[] = parsed.issues.map((issue) => ({
|
||||
file: issue.file ?? file,
|
||||
line: issue.line,
|
||||
severity: issue.severity,
|
||||
message: issue.message,
|
||||
...(issue.suggested_doc !== undefined && { suggestedDoc: issue.suggested_doc }),
|
||||
}));
|
||||
|
||||
return { file, summary: parsed.summary, issues };
|
||||
}
|
||||
|
||||
/** Build the markdown report posted to the PR. */
|
||||
function buildReport(fileCount: number, results: readonly FileReviewResult[]): string {
|
||||
const issues = results.flatMap((r) => r.issues);
|
||||
const warnings = issues.filter((i) => i.severity === 'warning').length;
|
||||
const suggestions = issues.length - warnings;
|
||||
|
||||
const lines: string[] = ['## Documentation Review Report', ''];
|
||||
lines.push(
|
||||
issues.length === 0
|
||||
? 'No documentation issues found.'
|
||||
: `Found **${issues.length}** issue(s) across **${fileCount}** changed file(s): ${warnings} warning(s), ${suggestions} suggestion(s).`,
|
||||
);
|
||||
lines.push('');
|
||||
|
||||
for (const result of results) {
|
||||
if (result.issues.length === 0) continue;
|
||||
lines.push(`### \`${result.file}\``, '', result.summary, '');
|
||||
for (const issue of result.issues) {
|
||||
const tag = issue.severity === 'warning' ? '**Warning:**' : '**Suggestion:**';
|
||||
lines.push(`- ${tag} Line ${issue.line}: ${issue.message}`);
|
||||
}
|
||||
lines.push('');
|
||||
}
|
||||
|
||||
lines.push('---', '*Automated review by doc-agent.*');
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
* Review the documentation drift introduced by a git range or PR.
|
||||
*
|
||||
* Writes two output files in the current working directory:
|
||||
* - doc-review-report.md (markdown summary for PR comment)
|
||||
* - doc-review-comments.json (inline review comments)
|
||||
*/
|
||||
export async function reviewDiff(args: readonly string[]): Promise<void> {
|
||||
const range = parseRangeArgs(args);
|
||||
console.log(`Reviewing range: ${range.base}...${range.head}`);
|
||||
|
||||
const files = getChangedCppFiles(range);
|
||||
if (files.length === 0) {
|
||||
console.log('No C++ files changed in this range.');
|
||||
await writeFile('doc-review-report.md', '## Documentation Review\n\nNo C++ files changed.\n');
|
||||
await writeFile('doc-review-comments.json', '[]');
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`Found ${files.length} changed C++ file(s).`);
|
||||
|
||||
const results: FileReviewResult[] = [];
|
||||
for (const file of files) {
|
||||
try {
|
||||
const result = await reviewFile(range, file);
|
||||
if (result !== null) results.push(result);
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
console.warn(` Review failed for ${file}: ${message}`);
|
||||
}
|
||||
}
|
||||
|
||||
const report = buildReport(files.length, results);
|
||||
const allIssues = results.flatMap((r) => r.issues);
|
||||
|
||||
await writeFile('doc-review-report.md', report);
|
||||
await writeFile('doc-review-comments.json', JSON.stringify(allIssues, null, 2));
|
||||
|
||||
console.log('\nReport: doc-review-report.md');
|
||||
console.log(`Inline comments: doc-review-comments.json (${allIssues.length} issues)`);
|
||||
}
|
||||
37
.github/scripts/doc-agent/src/types.ts
vendored
Normal file
37
.github/scripts/doc-agent/src/types.ts
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
/**
|
||||
* Shared type definitions for the doc-agent.
|
||||
*/
|
||||
|
||||
export type Severity = 'warning' | 'suggestion';
|
||||
|
||||
export interface ReviewIssue {
|
||||
file: string;
|
||||
line: number;
|
||||
severity: Severity;
|
||||
message: string;
|
||||
suggestedDoc?: string;
|
||||
}
|
||||
|
||||
export interface FileReviewResult {
|
||||
file: string;
|
||||
summary: string;
|
||||
issues: ReviewIssue[];
|
||||
}
|
||||
|
||||
export interface ReviewOutput {
|
||||
summary: string;
|
||||
issues: Array<{
|
||||
file?: string;
|
||||
line: number;
|
||||
severity: Severity;
|
||||
message: string;
|
||||
suggested_doc?: string;
|
||||
}>;
|
||||
}
|
||||
|
||||
export interface GitRange {
|
||||
base: string;
|
||||
head: string;
|
||||
}
|
||||
|
||||
export type AgentMode = 'document' | 'review';
|
||||
39
.github/scripts/doc-agent/tsconfig.json
vendored
Normal file
39
.github/scripts/doc-agent/tsconfig.json
vendored
Normal file
@@ -0,0 +1,39 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "NodeNext",
|
||||
"moduleResolution": "NodeNext",
|
||||
"lib": ["ES2023"],
|
||||
"outDir": "dist",
|
||||
"rootDir": "src",
|
||||
"strict": true,
|
||||
"noImplicitAny": true,
|
||||
"strictNullChecks": true,
|
||||
"strictFunctionTypes": true,
|
||||
"strictBindCallApply": true,
|
||||
"strictPropertyInitialization": true,
|
||||
"noImplicitThis": true,
|
||||
"alwaysStrict": true,
|
||||
"noUnusedLocals": true,
|
||||
"noUnusedParameters": true,
|
||||
"exactOptionalPropertyTypes": true,
|
||||
"noImplicitReturns": true,
|
||||
"noFallthroughCasesInSwitch": true,
|
||||
"noUncheckedIndexedAccess": true,
|
||||
"noImplicitOverride": true,
|
||||
"noPropertyAccessFromIndexSignature": true,
|
||||
"allowUnusedLabels": false,
|
||||
"allowUnreachableCode": false,
|
||||
"esModuleInterop": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"skipLibCheck": true,
|
||||
"declaration": true,
|
||||
"declarationMap": true,
|
||||
"sourceMap": true,
|
||||
"resolveJsonModule": true,
|
||||
"isolatedModules": true,
|
||||
"verbatimModuleSyntax": true
|
||||
},
|
||||
"include": ["src/**/*"],
|
||||
"exclude": ["node_modules", "dist"]
|
||||
}
|
||||
126
docs/skills/index.md
Normal file
126
docs/skills/index.md
Normal file
@@ -0,0 +1,126 @@
|
||||
# xrpld Codebase Skills Index
|
||||
|
||||
## Description
|
||||
This is the top-level guide for all best-practices skills in this repository. Use this to understand the codebase organization and find the right skill for any task.
|
||||
|
||||
## When to Use Skills
|
||||
Reference a skill whenever you are:
|
||||
- **Writing new code** in a module - check the skill first for established patterns
|
||||
- **Modifying existing code** - verify your changes follow module conventions
|
||||
- **Adding a new transaction type** - see `libxrpl/tx/transactors.md` for the full template
|
||||
- **Debugging** - skills list key files and common pitfalls per module
|
||||
- **Reviewing code** - skills document what "correct" looks like for each module
|
||||
|
||||
## Codebase Architecture
|
||||
|
||||
The codebase is split into two main areas:
|
||||
|
||||
### `src/libxrpl/` — The Library (skills in `.claude/skills/libxrpl/`)
|
||||
Reusable library code: data types, serialization, cryptography, ledger state, transaction processing, and storage. This is the **protocol layer**.
|
||||
|
||||
| Module | Responsibility |
|
||||
|--------|---------------|
|
||||
| `basics` | Foundational types: Buffer, Slice, base_uint, Number, logging, error contracts |
|
||||
| `beast` | Support layer: Journal logging, test framework, instrumentation, IP types |
|
||||
| `conditions` | Crypto-conditions (RFC): fulfillment validation, DER encoding |
|
||||
| `core` | Job queue, load monitoring, hash-based message dedup |
|
||||
| `crypto` | CSPRNG, secure erasure, RFC1751 encoding |
|
||||
| `json` | Json::Value, parsing, serialization, StaticString optimization |
|
||||
| `ledger` | ReadView/ApplyView, state tables, payment sandbox, credit ops |
|
||||
| `net` | HTTP/HTTPS client, SSL certs, async I/O |
|
||||
| `nodestore` | Persistent node storage: RocksDB, NuDB, Memory backends |
|
||||
| `protocol` | STObject hierarchy, SField, Serializer, TER codes, Features, Keylets |
|
||||
| `proto` | Protocol Buffer generated headers (gRPC API definitions) |
|
||||
| `rdb` | SOCI database wrapper, checkpointing |
|
||||
| `resource` | Rate limiting, endpoint tracking, abuse prevention |
|
||||
| `server` | Port config, SSL/TLS, WebSocket, admin networks |
|
||||
| `shamap` | SHA-256 Merkle radix tree (16-way branching, COW) |
|
||||
| `tx` | Transaction pipeline: Transactor base, preflight/preclaim/doApply |
|
||||
|
||||
### `src/xrpld/` — The Server Application (skills in `.claude/skills/xrpld/`)
|
||||
The running rippled server: application lifecycle, consensus, networking, RPC, and peer management. This is the **application layer**.
|
||||
|
||||
| Module | Responsibility |
|
||||
|--------|---------------|
|
||||
| `app` | Application singleton, ledger management, consensus adapters, services |
|
||||
| `app/main` | Application initialization and lifecycle |
|
||||
| `app/ledger` | Ledger storage, retrieval, immutable state management |
|
||||
| `app/consensus` | RCL consensus adapters (bridges generic algorithm to rippled) |
|
||||
| `app/misc` | Fee voting, amendments, SHAMapStore, TxQ, validators, NetworkOPs |
|
||||
| `app/paths` | Payment path finding algorithm, trust line caching |
|
||||
| `app/rdb` | Application-level database operations |
|
||||
| `app/tx` | Application-level transaction handling |
|
||||
| `consensus` | Generic consensus algorithm (CRTP-based, app-independent) |
|
||||
| `core` | Configuration (Config.h), time keeping, network ID |
|
||||
| `overlay` | P2P networking: peer connections, protocol buffers, clustering |
|
||||
| `peerfinder` | Network discovery: bootcache, livecache, slot management |
|
||||
| `perflog` | Performance logging and instrumentation |
|
||||
| `rpc` | RPC handler dispatch, coroutine suspension, 40+ command handlers |
|
||||
| `shamap` | Application-level SHAMap operations (NodeFamily) |
|
||||
|
||||
### `include/xrpl/` — Header Files
|
||||
Headers live in `include/xrpl/` and mirror the `src/libxrpl/` structure. Each skill already references its corresponding headers in the "Key Files" section.
|
||||
|
||||
## Cross-Cutting Conventions
|
||||
|
||||
### Error Handling
|
||||
- **Transaction errors**: Return `TER` enum (tesSUCCESS, tecFROZEN, temBAD_AMOUNT, etc.)
|
||||
- **Logic errors**: `Throw<std::runtime_error>()`, `LogicError()`, `XRPL_ASSERT()`
|
||||
- **I/O errors**: `Status` enum or `boost::system::error_code`
|
||||
- **RPC errors**: Inject via `context.params`
|
||||
|
||||
### Assertions
|
||||
```cpp
|
||||
XRPL_ASSERT(condition, "ClassName::method : description"); // Debug only
|
||||
XRPL_VERIFY(condition, "ClassName::method : description"); // Always enabled
|
||||
```
|
||||
|
||||
### Logging
|
||||
```cpp
|
||||
JLOG(j_.warn()) << "Message"; // Always wrap in JLOG macro
|
||||
```
|
||||
|
||||
### Memory Management
|
||||
- `IntrusiveRefCounts` + `SharedIntrusive` for shared ownership in libxrpl
|
||||
- `std::shared_ptr` for shared ownership in xrpld
|
||||
- `std::unique_ptr` for exclusive ownership
|
||||
- `CountedObject<T>` mixin for instance tracking
|
||||
|
||||
### Feature Gating
|
||||
```cpp
|
||||
if (ctx.rules.enabled(featureMyFeature)) { /* new behavior */ }
|
||||
```
|
||||
|
||||
### Code Organization
|
||||
- Headers in `include/xrpl/`, implementations in `src/libxrpl/` or `src/xrpld/`
|
||||
- `#pragma once` (never `#ifndef` guards)
|
||||
- `namespace xrpl { }` for all code
|
||||
- `detail/` namespace for internal helpers
|
||||
- Factory functions: `make_*()` returning `unique_ptr` or `shared_ptr`
|
||||
|
||||
## Subsystem Skills (soul/)
|
||||
|
||||
Concise invariants, bug patterns, and review checklists for each subsystem:
|
||||
|
||||
| Subsystem | File | Focus |
|
||||
|-----------|------|-------|
|
||||
| Consensus | `soul/consensus.md` | State machine, amendments, UNL, validations, TX ordering |
|
||||
| Cryptography | `soul/cryptography.md` | Key types, signing, hashing, handshake |
|
||||
| Ledger | `soul/ledger.md` | Immutability, acquisition, entry types |
|
||||
| NodeStore | `soul/nodestore.md` | Backends, caching, online deletion |
|
||||
| Peering | `soul/peering.md` | Overlay, connections, squelching |
|
||||
| Protocol | `soul/protocol.md` | Macros, serialization format, field ordering |
|
||||
| RPC | `soul/rpc.md` | Handler dispatch, roles, subscriptions |
|
||||
| SHAMap | `soul/shamap.md` | COW, node types, sync, proofs |
|
||||
| SQL | `soul/sql.md` | Schema, config, checkpointing |
|
||||
| Testing | `soul/test.md` | JTx framework, Env setup, TER expectations, amendment gating |
|
||||
| Transactors | `soul/transactors.md` | Pipeline, new TX types, signing, transactor template |
|
||||
| WebSockets | `soul/websockets.md` | Session lifecycle, flow control |
|
||||
|
||||
### Workflow & Processes
|
||||
| Skill | File |
|
||||
|-------|------|
|
||||
| Workflow Orchestration | `workflow.md` |
|
||||
| Task Management | `task-management.md` |
|
||||
| Amendment Creation | `amendment.md` |
|
||||
| Merge Conflicts | `merge-conflicts.md` |
|
||||
86
docs/skills/soul/consensus.md
Normal file
86
docs/skills/soul/consensus.md
Normal file
@@ -0,0 +1,86 @@
|
||||
# Consensus
|
||||
|
||||
Template-based state machine in `Consensus.h` parameterized by an Adaptor (`RCLConsensus`). Three phases: open -> establish -> accepted. Modes: proposing, observing, wrongLedger, switchedLedger.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- A ledger cannot close until the previous ledger reaches consensus AND (has transactions OR close time reached)
|
||||
- Proposals must have strictly increasing sequence numbers per peer; stale proposals are silently dropped
|
||||
- The Avalanche state machine progressively lowers consensus thresholds over time (init -> mid -> late -> stuck) to prevent livelock
|
||||
- `minCONSENSUS_PCT = 80` is the baseline; timing params: `ledgerMIN_CONSENSUS = 1950ms`, `ledgerMAX_CONSENSUS = 15s`
|
||||
- Dead nodes (`deadNodes_`) are permanently excluded for the round once they bow out
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- Proposals referencing a stale `prevLedgerID_` after a ledger switch cause split-brain; always check `newPeerProp.prevLedger() != prevLedgerID_` before processing
|
||||
- Resetting the consensus timer during `establish` phase causes re-convergence and potential split; timer must only reset on phase transitions
|
||||
- `DisputedTx::updateVote` changes local vote based on peer pressure; bugs here cause determinism failures across nodes
|
||||
- `createDisputes()` deduplicates via `compares` set; missing this check creates duplicate disputes that skew vote counts
|
||||
- The `peerUnchangedCounter_` is reset to 0 when any vote changes; bugs in this counter cause premature consensus declaration
|
||||
|
||||
## Amendments
|
||||
|
||||
- 80% validator support for 2 weeks to enable; tracked via `AmendmentTable` with `amendmentMap_`
|
||||
- New amendments: add to `features.macro` with `XRPL_FEATURE`/`XRPL_FIX`, increment `numFeatures` in `Feature.h`
|
||||
- Unsupported enabled amendment blocks the server (`setAmendmentBlocked`); no mechanism to disable/revoke
|
||||
- Voting happens each consensus round in `doVoting`; votes are persisted in `FeatureVotes` SQLite table
|
||||
- `fixAmendmentMajorityCalc` changed the threshold calculation; check which calculation applies
|
||||
|
||||
## UNL and Negative UNL
|
||||
|
||||
- Negative UNL temporarily disables unreliable validators (max 25% of UNL: `negativeUNLMaxListed = 0.25`)
|
||||
- Scoring uses `buildScoreTable` over recent ledger history; low watermark (50%) = disable candidate, high watermark (80%) = re-enable candidate
|
||||
- Candidate selection is deterministic via previous ledger hash as randomizing pad
|
||||
- `newValidatorDisableSkip = FLAG_LEDGER_INTERVAL * 2` prevents disabling newly joined validators prematurely
|
||||
|
||||
## Validations
|
||||
|
||||
- `ValidationParms` defines freshness windows: CURRENT_WALL=5min, CURRENT_LOCAL=3min, SET_EXPIRES=10min, FRESHNESS=20s
|
||||
- `SeqEnforcer` rejects validations with regressed or duplicate sequence numbers (`ValStatus::badSeq`)
|
||||
- Conflicting validations (same seq, different hash) are logged as byzantine behavior
|
||||
- `handleNewValidation` is the entry point: checks trust, adds to `Validations` set, triggers `checkAccept` if current+trusted
|
||||
|
||||
## Transaction Ordering
|
||||
|
||||
- `CanonicalTXSet` orders by: salted account key (XOR with random salt) -> sequence proxy -> transaction ID
|
||||
- Salt prevents manipulation of ordering by account selection
|
||||
- `TxQ` uses `OrderCandidates`: higher fee level first, then `txID XOR parentHash` as tiebreaker
|
||||
- Per-account limit: `maximumTxnPerAccount`; blocked transactions held until blocker resolves
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Proposal Validation (prevents split-brain)
|
||||
```cpp
|
||||
// REQUIRED: reject proposals referencing stale previous ledger
|
||||
if (newPeerProp.prevLedger() != prevLedgerID_)
|
||||
{
|
||||
JLOG(j_.debug()) << "Got proposal for " << newPeerProp.prevLedger()
|
||||
<< " but we are on " << prevLedgerID_;
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
### Complete Bow-Out Handling
|
||||
```cpp
|
||||
// REQUIRED: all three steps — unvote, erase position, mark dead
|
||||
if (newPeerProp.isBowOut())
|
||||
{
|
||||
if (result_)
|
||||
for (auto& it : result_->disputes)
|
||||
it.second.unVote(peerID);
|
||||
if (currPeerPositions_.find(peerID) != currPeerPositions_.end())
|
||||
currPeerPositions_.erase(peerID);
|
||||
deadNodes_.insert(peerID); // permanently excluded this round
|
||||
}
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/xrpld/consensus/Consensus.h` - state machine
|
||||
- `src/xrpld/consensus/ConsensusParms.h` - timing/threshold params
|
||||
- `src/xrpld/app/consensus/RCLConsensus.cpp` - XRPL adaptor
|
||||
- `src/xrpld/consensus/DisputedTx.h` - dispute tracking
|
||||
- `src/xrpld/app/misc/detail/AmendmentTable.cpp` - amendment logic
|
||||
- `src/xrpld/app/misc/NegativeUNLVote.cpp` - N-UNL voting
|
||||
- `src/xrpld/consensus/Validations.h` - validation tracking
|
||||
- `src/xrpld/app/misc/CanonicalTXSet.h` - TX ordering
|
||||
59
docs/skills/soul/cryptography.md
Normal file
59
docs/skills/soul/cryptography.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# Cryptography
|
||||
|
||||
XRPL supports secp256k1 (ECDSA) and ed25519 key types. All crypto uses OpenSSL + dedicated libs (libsecp256k1, ed25519-donna).
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- `SecretKey` destructor calls `secure_erase` on internal buffer; any code handling secret keys must follow this pattern
|
||||
- ed25519 public keys are prefixed with `0xED` (33 bytes total); secp256k1 keys are 33-byte compressed
|
||||
- `sha512Half` (first 32 bytes of SHA-512) is the standard hash used throughout XRPL for node hashing, signing, etc.
|
||||
- `RIPEMD-160(SHA-256(x))` is used for account ID derivation (`ripesha_hasher`)
|
||||
- Base58 encoding includes a type byte prefix and 4-byte checksum (double SHA-256)
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- Mixing up key types: secp256k1 signing hashes the message with sha512Half first, ed25519 signs the raw message
|
||||
- `signDigest` only works with secp256k1; calling it with ed25519 throws a logic error
|
||||
- Signature canonicality: ed25519 `verify` checks signature canonicality before calling `ed25519_sign_open`; non-canonical signatures are rejected
|
||||
- Overlay handshake uses `signDigest` to sign the session fingerprint (`sharedValue`); the signature binds the TLS session to the node identity
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- New crypto code must use `crypto_prng()` singleton for randomness, never raw `rand()`
|
||||
- Secret key buffers must be `secure_erase`d after use
|
||||
- Verify that key type dispatch handles both secp256k1 and ed25519 (or explicitly rejects one with a clear error)
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Secure Erasure
|
||||
```cpp
|
||||
// REQUIRED: destructor must erase secret material
|
||||
SecretKey::~SecretKey()
|
||||
{
|
||||
secure_erase(buf_, sizeof(buf_));
|
||||
}
|
||||
|
||||
// REQUIRED: erase intermediate buffers after use
|
||||
beast::rngfill(buf, sizeof(buf), crypto_prng());
|
||||
SecretKey sk(Slice{buf, sizeof(buf)});
|
||||
secure_erase(buf, sizeof(buf)); // MUST erase raw buffer
|
||||
```
|
||||
|
||||
### Key Type Dispatch
|
||||
```cpp
|
||||
// REQUIRED: handle both key types or explicitly reject
|
||||
if (type == KeyType::ed25519)
|
||||
{ /* ed25519 path */ }
|
||||
else if (type == KeyType::secp256k1)
|
||||
{ /* secp256k1 path */ }
|
||||
else
|
||||
LogicError("unknown key type"); // MUST NOT fall through silently
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `include/xrpl/protocol/SecretKey.h` / `PublicKey.h` - key types
|
||||
- `src/libxrpl/protocol/SecretKey.cpp` - signing, key generation
|
||||
- `src/libxrpl/protocol/PublicKey.cpp` - verification
|
||||
- `include/xrpl/protocol/digest.h` - hash functions
|
||||
- `src/xrpld/overlay/detail/Handshake.cpp` - overlay handshake crypto
|
||||
63
docs/skills/soul/ledger.md
Normal file
63
docs/skills/soul/ledger.md
Normal file
@@ -0,0 +1,63 @@
|
||||
# Ledger
|
||||
|
||||
Each ledger is an immutable snapshot: header (seq, hashes, close time) + state SHAMap + transaction SHAMap. `LedgerMaster` is the central coordinator.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- Once `setImmutable()` is called, the ledger and its SHAMaps cannot change; only immutable ledgers can be set in `LedgerHolder`
|
||||
- Every server always has an open ledger; the open ledger cannot close until previous consensus completes
|
||||
- Ledger header hashes to the ledger's identity hash; includes state root, tx root, parent hash, total coins, close time
|
||||
- `LedgerMaster` tracks: `mPubLedger` (last published), `mValidLedger` (last validated), `mLedgerHistory` (cache)
|
||||
- Validation requires minimum trusted validations (`minVal`); filtered by Negative UNL
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- Modifying a ledger after `setImmutable()` corrupts shared state; always check `mImmutable` before mutation
|
||||
- Gap detection: if ledgers 603 and 600 exist but 601-602 are missing, `LedgerMaster` requests 602 first, then backfills 601
|
||||
- `InboundLedger::gotData()` queues data for processing; calling `done()` before all data arrives creates incomplete ledgers
|
||||
- `checkAccept` won't accept a ledger that isn't ahead of the last validated ledger; stale validations are silently ignored
|
||||
|
||||
## Ledger Entry Types
|
||||
|
||||
- Defined in `ledger_entries.macro` using `LEDGER_ENTRY(type, code, class, name, fields)`
|
||||
- Each entry has an `SOTemplate` defining required/optional fields
|
||||
- Key computation: `Indexes.cpp` computes unique keys (keylets) for each ledger object type
|
||||
- `STLedgerEntry` wraps the serialized data with type-safe field access
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- New ledger entry types: add to `ledger_entries.macro`, implement keylet in `Indexes.cpp`
|
||||
- Verify `LedgerCleaner` can handle the new entry type for repair
|
||||
- Check that acquisition code handles the entry in both `InboundLedger` and `LedgerMaster`
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Immutability Guard
|
||||
```cpp
|
||||
// After this, no mutations allowed on the ledger or its SHAMaps
|
||||
inline void SHAMap::setImmutable()
|
||||
{
|
||||
XRPL_ASSERT(state_ != SHAMapState::Invalid, "...");
|
||||
state_ = SHAMapState::Immutable;
|
||||
}
|
||||
// VERIFY: code never calls peek()/insert()/erase() after setImmutable()
|
||||
```
|
||||
|
||||
### New Ledger Entry Keylet
|
||||
```cpp
|
||||
// REQUIRED: every new ledger entry type needs unique keylet computation
|
||||
Keylet keylet::myEntry(AccountID const& id)
|
||||
{
|
||||
return {ltMY_ENTRY,
|
||||
sha512Half(std::uint16_t(spaceMyEntry), id)};
|
||||
}
|
||||
// Also add to ledger_entries.macro and Indexes.cpp
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/xrpld/app/ledger/Ledger.h` - ledger class
|
||||
- `src/xrpld/app/ledger/detail/LedgerMaster.cpp` - central coordinator
|
||||
- `src/xrpld/app/ledger/detail/InboundLedger.cpp` - ledger acquisition
|
||||
- `include/xrpl/protocol/detail/ledger_entries.macro` - entry type definitions
|
||||
- `src/libxrpl/protocol/Indexes.cpp` - keylet computation
|
||||
52
docs/skills/soul/nodestore.md
Normal file
52
docs/skills/soul/nodestore.md
Normal file
@@ -0,0 +1,52 @@
|
||||
# NodeStore
|
||||
|
||||
Persistent key-value store for `NodeObject`s (ledger entries). All ledger state is stored here between launches. Keys are 256-bit hashes.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- `NodeObject` types: `hotLEDGER` (1), `hotACCOUNT_NODE` (3), `hotTRANSACTION_NODE` (4), `hotDUMMY` (512, cache marker for missing entries)
|
||||
- Preferred backends: NuDB (append-only) and RocksDB; LevelDB/HyperLevelDB are deprecated
|
||||
- `TaggedCache` evicts by both `cache_size` (max items) and `cache_age` (max minutes)
|
||||
- `DatabaseRotatingImp` uses two backends (writable + archive) for online deletion; rotation moves writable to archive, creates new writable, deletes old archive
|
||||
- Corrupt data triggers fatal logging; unknown/backend errors logged with appropriate severity
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- `fetchNodeObject` with `duplicate=true` copies from archive to writable backend; forgetting this in rotating mode means objects disappear after rotation
|
||||
- `hotDUMMY` objects in cache mark missing entries; code that checks cache hits must distinguish real objects from dummies
|
||||
- Batch write limit is 65536 objects; exceeding this silently truncates or fails depending on backend
|
||||
- `fdRequired()` must be called during resource planning; running out of file descriptors causes silent backend failures
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- Config changes: verify `[node_db]` section has valid `type`, `path`, and `compression` settings
|
||||
- Online deletion: ensure `SHAMapStoreImp` coordinates rotation with the application lifecycle
|
||||
- New backend types: implement the full `Backend` interface including `fdRequired()`
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Cache Lookup — Distinguish Real vs Dummy
|
||||
```cpp
|
||||
// REQUIRED: hotDUMMY marks "confirmed missing" — not a real object
|
||||
auto obj = cache_.fetch(hash);
|
||||
if (obj && obj->getType() == hotDUMMY)
|
||||
return nullptr; // not found, just cached as missing
|
||||
return obj;
|
||||
```
|
||||
|
||||
### Backend File Descriptor Reporting
|
||||
```cpp
|
||||
// REQUIRED: every backend must accurately report FD needs
|
||||
int fdRequired() const override
|
||||
{
|
||||
return fdLimit_; // inaccurate values cause silent failures
|
||||
}
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `include/xrpl/nodestore/NodeObject.h` - object types
|
||||
- `include/xrpl/nodestore/Backend.h` - backend interface
|
||||
- `include/xrpl/nodestore/detail/DatabaseNodeImp.h` - standard implementation
|
||||
- `src/libxrpl/nodestore/DatabaseRotatingImp.cpp` - rotating/online deletion
|
||||
- `src/xrpld/app/misc/SHAMapStoreImp.cpp` - lifecycle management
|
||||
60
docs/skills/soul/peering.md
Normal file
60
docs/skills/soul/peering.md
Normal file
@@ -0,0 +1,60 @@
|
||||
# Overlay Peering
|
||||
|
||||
P2P network using persistent TCP/IP connections. Messages serialized via Protocol Buffers. `OverlayImpl` manages connections; `PeerImp` handles per-peer logic.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- Connection preference order: Fixed Peers -> Livecache -> Bootcache
|
||||
- Cluster connections do NOT count toward connection limits (unlimited)
|
||||
- Protobuf message changes MUST maintain wire compatibility or risk network partitioning
|
||||
- Squelching: after enough peers relay a validator's messages, a subset is "Selected" and the rest are temporarily muted to reduce bandwidth
|
||||
- Handshake binds TLS session to node identity via `signDigest` of the session fingerprint
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- PeerFinder slot exhaustion: if `maxPeers` is reached, new outbound connections silently fail; check slot availability before connecting
|
||||
- `HashRouter::shouldRelay` prevents duplicate relay; bypassing it causes message storms
|
||||
- `ConnectAttempt::processResponse` on HTTP 503 parses "peer-ips" for alternatives; malformed responses here can crash with bad IP parsing
|
||||
- `PeerImp::close` must run on the strand; calling from wrong thread causes race conditions on socket and timer state
|
||||
- Destructor chain: `~PeerImp` -> `deletePeer` -> `onPeerDeactivate` -> `on_closed` -> `remove`; interrupting this chain leaks slots
|
||||
|
||||
## Connection Lifecycle
|
||||
|
||||
1. `OverlayImpl::connect` -> check resource limits -> allocate PeerFinder slot -> create `ConnectAttempt`
|
||||
2. Async TCP connect -> TLS handshake -> HTTP upgrade with identity headers
|
||||
3. `processResponse` -> verify handshake -> create `PeerImp` -> `add_active` -> `run()`
|
||||
4. `doProtocolStart` -> start async message receive loop -> exchange validator lists and manifests
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- Verify resource manager checks on both inbound and outbound connections
|
||||
- New protocol messages: update protobuf definitions AND verify wire compatibility
|
||||
- Squelch changes: test with high peer counts; incorrect squelch logic can silence validators
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Strand Execution
|
||||
```cpp
|
||||
// REQUIRED: socket operations must run on the strand
|
||||
if (!strand_.running_in_this_thread())
|
||||
return post(strand_, std::bind(
|
||||
&PeerImp::close, shared_from_this()));
|
||||
// Calling socket ops from wrong thread causes races on state
|
||||
```
|
||||
|
||||
### Duplicate Relay Prevention
|
||||
```cpp
|
||||
// REQUIRED: check HashRouter before relaying
|
||||
if (!hashRouter_.shouldRelay(hash))
|
||||
return; // already relayed — suppress duplicate
|
||||
overlay_.relay(message, hash);
|
||||
// Bypassing this causes message storms across the network
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/xrpld/overlay/detail/OverlayImpl.cpp` - main overlay manager
|
||||
- `src/xrpld/overlay/detail/PeerImp.cpp` - per-peer logic
|
||||
- `src/xrpld/overlay/detail/ConnectAttempt.cpp` - outbound connection
|
||||
- `src/xrpld/overlay/Slot.h` - squelch state machine
|
||||
- `src/xrpld/overlay/detail/Handshake.cpp` - handshake crypto
|
||||
64
docs/skills/soul/protocol.md
Normal file
64
docs/skills/soul/protocol.md
Normal file
@@ -0,0 +1,64 @@
|
||||
# Protocol and Serialization
|
||||
|
||||
Macro-driven system for defining features, transactions, ledger entries, and serialized fields. Canonical binary format is required for signatures and consensus.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- Fields are sorted by (type code, field code) for canonical serialization; sorting by Field ID bytes produces WRONG results
|
||||
- Field ID encoding: 1-3 bytes depending on type/field code values (both < 16 = 1 byte)
|
||||
- Signing hash prefix: `0x53545800` for single-signing, `0x534D5400` for multi-signing
|
||||
- `STObject[sfFoo]` returns value or default; `STObject[~sfFoo]` returns optional (nothing if absent)
|
||||
- All ST types inherit from `STBase`; `STVar` provides type-erased storage with stack/heap allocation
|
||||
|
||||
## Macro System
|
||||
|
||||
- `XRPL_FEATURE(name, supported, vote)` / `XRPL_FIX` / `XRPL_RETIRE` in `features.macro`
|
||||
- `TRANSACTION(tag, value, class, delegation, fields)` in `transactions.macro`
|
||||
- `LEDGER_ENTRY(type, code, class, name, fields)` in `ledger_entries.macro`
|
||||
- `TYPED_SFIELD(name, TYPE, code)` in `sfields.macro`
|
||||
- Adding any new definition requires updating the count in the corresponding header
|
||||
|
||||
## Serialization Format
|
||||
|
||||
- XRP Amount: 8 bytes, MSB=0, next bit=1 for positive, remaining 62 bits = value
|
||||
- Token Amount: 8 bytes mantissa/exponent + 20 bytes currency + 20 bytes issuer
|
||||
- AccountID: 20 bytes, length-prefixed when top-level
|
||||
- STArray: elements between start (`0xf0`) and end (`0xf1`) markers
|
||||
- STObject: fields in canonical order between start (`0xe0`) and end (`0xe1`) markers
|
||||
- Length prefixing: 1 byte (0-192), 2 bytes (193-12480), 3 bytes (12481-918744)
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- Adding a field to `transactions.macro` without adding it to `sfields.macro` causes silent serialization failures
|
||||
- Forgetting to increment `numFeatures` after adding to `features.macro` causes out-of-bounds access
|
||||
- Non-canonical field ordering in hand-built binary blobs causes signature verification failures
|
||||
- `soeMPTSupported` flag on amount fields enables MPT token support; omitting it silently rejects MPT payments
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Amendment Registration
|
||||
```cpp
|
||||
// In features.macro — REQUIRED format:
|
||||
XRPL_FEATURE(MyNewFeature, Supported::yes, VoteBehavior::DefaultNo)
|
||||
XRPL_FIX (MyBugFix, Supported::yes, VoteBehavior::DefaultNo)
|
||||
// MUST also increment numFeatures in Feature.h — omitting causes OOB access
|
||||
```
|
||||
|
||||
### STObject Field Access
|
||||
```cpp
|
||||
// Safe: optional access — returns std::optional, never throws
|
||||
if (auto const val = tx[~sfAmount])
|
||||
use(*val);
|
||||
|
||||
// Throws if field is absent — only safe when preflight guarantees presence
|
||||
auto const amount = tx[sfAmount];
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `include/xrpl/protocol/detail/features.macro` - amendment definitions
|
||||
- `include/xrpl/protocol/detail/transactions.macro` - transaction types
|
||||
- `include/xrpl/protocol/detail/ledger_entries.macro` - ledger objects
|
||||
- `include/xrpl/protocol/detail/sfields.macro` - field definitions
|
||||
- `include/xrpl/protocol/Feature.h` - `numFeatures` constant
|
||||
- `src/libxrpl/protocol/STObject.cpp` - core serialized object
|
||||
61
docs/skills/soul/rpc.md
Normal file
61
docs/skills/soul/rpc.md
Normal file
@@ -0,0 +1,61 @@
|
||||
# RPC
|
||||
|
||||
JSON-RPC over HTTP/WebSocket and gRPC. Central handler table dispatches by method name + API version. Roles: ADMIN, USER, IDENTIFIED, PROXY, FORBID.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- Handler table in `Handler.cpp`: each entry = `{name, function, role, condition, minApiVer, maxApiVer}`
|
||||
- `conditionMet` checks server state (e.g., `NEEDS_CURRENT_LEDGER`) before invoking handler
|
||||
- API v2.0+ errors: structured objects with `status`, `code`, `message`; earlier: flat fields in response
|
||||
- Sensitive fields (`passphrase`, `secret`, `seed`, `seed_hex`) are masked in error responses
|
||||
- Batch requests: `"method": "batch"` with `"params"` array; each sub-request processed independently
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- New handler without entry in `Handler.cpp` static array = handler silently unreachable
|
||||
- Wrong `role_` on handler: USER-level handler with admin-only data leaks; ADMIN handler accessible to users = security hole
|
||||
- `conditionMet` returning false causes a generic error; ensure new conditions are documented
|
||||
- Resource charging: each request gets a fee via `Resource::Consumer`; missing charge allows DoS
|
||||
- `maxRequestSize` (RPC::Tuning) rejection happens before JSON parsing; oversized requests get no error detail
|
||||
|
||||
## Adding New RPC Handler
|
||||
|
||||
1. Declare in `Handlers.h`: `Json::Value doMyCommand(RPC::JsonContext&);`
|
||||
2. Implement in new file under `src/xrpld/rpc/handlers/`
|
||||
3. Register in `Handler.cpp` static array with role, condition, version range
|
||||
4. For gRPC: define in `xrp_ledger.proto`, add `CallData` in `GRPCServerImpl::setupListeners()`
|
||||
|
||||
## Subscriptions
|
||||
|
||||
- WebSocket clients can subscribe to: `server`, `ledger`, `book_changes`, `transactions`, `validations`, `manifests`, `peer_status` (admin), `consensus`
|
||||
- `WSInfoSub` delivers events via weak pointer to `WSSession`; dead sessions are automatically cleaned up
|
||||
- `RPCSub` delivers to remote URL endpoints with auth and SSL support
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Handler Table Registration
|
||||
```cpp
|
||||
// In Handler.cpp handlerArray[] — REQUIRED for every new handler:
|
||||
{"my_command", byRef(&doMyCommand), Role::USER, NO_CONDITION},
|
||||
// role MUST match security requirements:
|
||||
// Role::ADMIN for internal-only, Role::USER for public API
|
||||
// condition: NEEDS_CURRENT_LEDGER, NEEDS_NETWORK_CONNECTION, or NO_CONDITION
|
||||
```
|
||||
|
||||
### Version-Ranged Handler
|
||||
```cpp
|
||||
// New-style handler with API version range
|
||||
template <> Handler handlerFrom<MyCommandHandler>()
|
||||
{ return {MyCommandHandler::name, &handle<Json::Value, MyCommandHandler>,
|
||||
MyCommandHandler::role, MyCommandHandler::condition,
|
||||
MyCommandHandler::minApiVer, MyCommandHandler::maxApiVer};
|
||||
}
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/xrpld/rpc/handlers/Handlers.h` - authoritative handler list
|
||||
- `src/xrpld/rpc/detail/Handler.cpp` - handler table and dispatch
|
||||
- `src/xrpld/rpc/detail/RPCHandler.cpp` - request processing pipeline
|
||||
- `src/xrpld/rpc/detail/ServerHandler.cpp` - HTTP/WS entry points
|
||||
- `include/xrpl/protocol/ErrorCodes.h` - error code definitions
|
||||
61
docs/skills/soul/shamap.md
Normal file
61
docs/skills/soul/shamap.md
Normal file
@@ -0,0 +1,61 @@
|
||||
# SHAMap
|
||||
|
||||
Merkle radix trie (radix 16) enabling O(1) subtree comparison via hash. Used for both state tree and transaction tree. Root is always a `SHAMapInnerNode`.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- Mutable SHAMaps have non-zero `cowid`; immutable have `cowid=0`. Once immutable, nodes persist for the map's lifetime with NO mechanism to remove them
|
||||
- Copy-on-write: `unshareNode` must be called before mutating any node in a mutable SHAMap; failing this corrupts shared snapshots
|
||||
- Inner nodes have up to 16 children; hash is computed from children's hashes. Leaf hash is computed from data + type-specific prefix
|
||||
- `canonicalize` ensures only one instance per hash in the cache; prevents races between threads
|
||||
- `SHAMapInnerNode` uses atomic operations + locking (`std::atomic<std::uint16_t> lock_`) for concurrent child access
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- Modifying a node without calling `unshareNode` first corrupts the snapshot that shares it; this is the #1 SHAMap bug class
|
||||
- `getMissingNodes` uses deferred async reads; processing completions out of order causes incorrect "full below" marking
|
||||
- Inner node serialization has two formats (compressed vs full) chosen by branch count; mismatched deserializer causes corruption
|
||||
- `addKnownNode` traverses toward target; if branch is empty or hash mismatches, returns "invalid" -- callers must handle this gracefully
|
||||
- Proof path verification walks leaf-to-root; incorrect key at any level causes false negative
|
||||
|
||||
## Serialization Formats
|
||||
|
||||
- **Compressed**: only non-empty branches serialized (saves space for sparse nodes)
|
||||
- **Full**: all 16 branches including empty ones (used for dense nodes)
|
||||
- Choice is automatic in `serializeForWire` based on branch count
|
||||
|
||||
## Leaf Node Types
|
||||
|
||||
- `SHAMapAccountStateLeafNode` - account state entries
|
||||
- `SHAMapTxLeafNode` - transactions
|
||||
- `SHAMapTxPlusMetaLeafNode` - transactions with metadata
|
||||
- Each uses a different hash prefix for domain separation
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### State Machine
|
||||
```cpp
|
||||
enum class SHAMapState {
|
||||
Modifying = 0, // can add/remove objects
|
||||
Immutable = 1, // FROZEN — no changes allowed
|
||||
Synching = 2, // hash fixed, missing nodes can be added
|
||||
Invalid = 3, // corrupt — do not use
|
||||
};
|
||||
// VERIFY: no peek()/insert()/erase() calls on Immutable maps
|
||||
```
|
||||
|
||||
### COW Discipline (#1 Bug Class)
|
||||
```cpp
|
||||
// REQUIRED before mutating any shared node:
|
||||
auto node = unshareNode(branch, key); // copies if shared
|
||||
node->setChild(index, child); // now safe to modify
|
||||
// BUG: skipping unshareNode corrupts snapshots sharing the node
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `include/xrpl/shamap/SHAMap.h` - main class
|
||||
- `include/xrpl/shamap/SHAMapInnerNode.h` - inner node (COW, threading)
|
||||
- `include/xrpl/shamap/SHAMapLeafNode.h` - leaf node base
|
||||
- `src/libxrpl/shamap/SHAMapSync.cpp` - sync, missing nodes, proofs
|
||||
- `src/libxrpl/shamap/SHAMapDelta.cpp` - walkMap, parallel traversal
|
||||
60
docs/skills/soul/sql.md
Normal file
60
docs/skills/soul/sql.md
Normal file
@@ -0,0 +1,60 @@
|
||||
# SQL Database
|
||||
|
||||
SQLite via SOCI for ledger/transaction history. Only SQLite is supported; Postgres has no implementation despite interface comments.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- Two main databases: `lgrdb_` (ledger) and `txdb_` (transactions, optional via `useTxTables` config)
|
||||
- Transaction tables are optional; disabling them means no transaction history or account_tx queries
|
||||
- WAL checkpointing triggers when WAL file grows beyond threshold; scheduled via job queue
|
||||
- Database init failure is fatal (throws exception, prevents construction)
|
||||
- Free disk space < 512MB triggers fatal error on write operations
|
||||
|
||||
## Schema
|
||||
|
||||
- `Ledgers` table: seq, hash, parent hash, total coins, close time, etc. Indexed by `LedgerSeq`
|
||||
- `Transactions` table: TransID, TransType, FromAcct, FromSeq, LedgerSeq, Status, RawTxn, TxnMeta. Indexed by `LedgerSeq`
|
||||
- `AccountTransactions` table: TransID, Account, LedgerSeq, TxnSeq. Triple-indexed for account_tx queries
|
||||
- Secondary DBs: Wallet (node identity, manifests), PeerFinder (bootstrap cache), State (deletion tracking)
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- No schema migration system; `CREATE TABLE IF NOT EXISTS` means old schemas silently persist with missing columns
|
||||
- PeerFinder DB is the exception -- it has schema versioning via `SchemaVersion` table
|
||||
- `safety_level` config affects journal_mode and synchronous; "low" can lose data on crash
|
||||
- `page_size` must be power of 2 between 512-65536; invalid values cause init failure
|
||||
- Online deletion coordinates between NodeStore rotation and SQL table pruning; race conditions here lose history
|
||||
|
||||
## Configuration
|
||||
|
||||
| Option | Section | Values | Default |
|
||||
|--------|---------|--------|---------|
|
||||
| `backend` | `[relational_db]` | `sqlite` only | sqlite |
|
||||
| `page_size` | `[sqlite]` | 512-65536, power of 2 | 4096 |
|
||||
| `safety_level` | `[sqlite]` | high, medium, low | high |
|
||||
| `journal_size_limit` | `[sqlite]` | integer >= 0 | 1582080 |
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Schema Evolution Caveat
|
||||
```cpp
|
||||
// WARNING: no migration system — old databases keep old schemas
|
||||
// CREATE TABLE IF NOT EXISTS silently skips if table exists with old columns
|
||||
// New columns on existing tables require manual ALTER TABLE or
|
||||
// documentation that the column is optional and may be absent
|
||||
```
|
||||
|
||||
### Disk Space Guard
|
||||
```cpp
|
||||
// REQUIRED on write paths: < 512MB triggers fatal to prevent corruption
|
||||
if (freeDiskSpace < minDiskFree)
|
||||
Throw<std::runtime_error>("Not enough disk space for database write");
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/xrpld/app/rdb/backend/detail/SQLiteDatabase.cpp` - main implementation
|
||||
- `src/xrpld/app/main/DBInit.h` - schema definitions
|
||||
- `src/xrpld/core/detail/DatabaseCon.cpp` - connection setup and pragmas
|
||||
- `src/xrpld/app/rdb/backend/detail/Node.cpp` - ledger/tx operations
|
||||
- `src/xrpld/app/rdb/detail/State.cpp` - deletion state tracking
|
||||
75
docs/skills/soul/test.md
Normal file
75
docs/skills/soul/test.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# Testing
|
||||
|
||||
JTx framework for in-memory ledger testing. Tests live in `src/test/`, derive from `beast::unit_test::suite`, and register with `BEAST_DEFINE_TESTSUITE`.
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Test File Structure
|
||||
```cpp
|
||||
class MyFeature_test : public beast::unit_test::suite {
|
||||
void testBasic() {
|
||||
testcase("basic");
|
||||
using namespace jtx;
|
||||
Env env{*this};
|
||||
// ... test logic ...
|
||||
}
|
||||
void run() override { testBasic(); }
|
||||
};
|
||||
BEAST_DEFINE_TESTSUITE(MyFeature, app, ripple);
|
||||
```
|
||||
|
||||
### Amendment Gating
|
||||
```cpp
|
||||
// REQUIRED: test with AND without the feature amendment
|
||||
Env env{*this}; // all amendments on
|
||||
Env env{*this, testable_amendments() - featureMyFeature}; // feature disabled
|
||||
// With custom config:
|
||||
Env env{*this, envconfig([](std::unique_ptr<Config> cfg) {
|
||||
/* modify config */ return cfg;
|
||||
})};
|
||||
```
|
||||
|
||||
### Transaction Submission
|
||||
```cpp
|
||||
auto const alice = Account{"alice"};
|
||||
auto const bob = Account{"bob"};
|
||||
env.fund(XRP(10000), alice, bob);
|
||||
env.close(); // REQUIRED between setup and test transactions
|
||||
|
||||
env(pay(alice, bob, XRP(100))); // expects tesSUCCESS
|
||||
env(pay(alice, bob, XRP(999999)), ter(tecUNFUNDED_PAYMENT)); // specific TER
|
||||
env(pay(alice, bob, XRP(100)), txflags(tfPartialPayment)); // with flags
|
||||
```
|
||||
|
||||
### State Verification
|
||||
```cpp
|
||||
env.require(balance(alice, XRP(9900)));
|
||||
env.require(balance(alice, USD(1000)));
|
||||
env.require(owners(alice, 2));
|
||||
|
||||
auto const sle = env.le(keylet::account(alice.id()));
|
||||
BEAST_EXPECT(sle);
|
||||
BEAST_EXPECT((*sle)[sfBalance] == XRP(9900));
|
||||
```
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- New transaction types MUST have tests with AND without the feature amendment
|
||||
- Every error path (tem*, tec*, tef*) must have a test exercising it
|
||||
- `env.close()` required between transactions that need ledger boundaries
|
||||
- Verify state after transaction, not just TER code
|
||||
- Test class naming: `FeatureName_test`, registered with `BEAST_DEFINE_TESTSUITE`
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- Forgetting `*this` as first Env arg disconnects assertion reporting
|
||||
- Comparing XRPAmount with raw integers instead of `XRP()` or `drops()`
|
||||
- Trust lines must be established before IOU payments
|
||||
- Each Env creates a full Application — keep test methods focused
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/test/jtx/Env.h` — test environment class
|
||||
- `src/test/jtx/jtx.h` — convenience header (includes all helpers)
|
||||
- `src/test/jtx/Account.h` — test accounts
|
||||
- `src/test/jtx/amount.h` — XRP(), drops(), IOU helpers
|
||||
141
docs/skills/soul/transactors.md
Normal file
141
docs/skills/soul/transactors.md
Normal file
@@ -0,0 +1,141 @@
|
||||
# Transactors
|
||||
|
||||
Transaction processing pipeline: preflight (static validation) -> preclaim (ledger state checks) -> doApply (state mutation). Base class `Transactor` in `src/libxrpl/tx/`.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- Pipeline is strict: preflight runs WITHOUT ledger state, preclaim runs WITH read-only view, doApply runs with mutable view
|
||||
- `preflight` validates all fields exist and are well-formed; this is the ONLY place to reject malformed transactions cheaply
|
||||
- Fee is always deducted even if the transaction fails (`tecCLAIM` pattern); `payFee` runs before `doApply`
|
||||
- Sequence/ticket consumption happens in `consumeSeqProxy`; must succeed before any state changes
|
||||
- Invariant checkers run after `doApply`; they can veto the transaction post-execution
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- New transaction type missing preflight validation for new fields = malformed transactions reach doApply and corrupt state
|
||||
- Forgetting to handle `tecCLAIM` in doApply: fee is deducted but no other state changes should occur
|
||||
- Batch transactions (`Batch` type) have their own signing path (`checkBatchSign`); changes to signing must cover both paths
|
||||
- `calculateBaseFee` override without updating `minimumFee` causes fee calculation divergence between nodes
|
||||
- Missing invariant checker update for new ledger entry types = silent constraint violations
|
||||
|
||||
## Transactor Template
|
||||
|
||||
### Header (`include/xrpl/tx/transactors/MyTx.h`)
|
||||
```cpp
|
||||
#pragma once
|
||||
#include <xrpl/tx/Transactor.h>
|
||||
|
||||
namespace xrpl {
|
||||
class MyTransaction : public Transactor {
|
||||
public:
|
||||
static constexpr ConsequencesFactoryType ConsequencesFactory{Normal};
|
||||
explicit MyTransaction(ApplyContext& ctx) : Transactor(ctx) {}
|
||||
|
||||
static bool checkExtraFeatures(PreflightContext const& ctx);
|
||||
static std::uint32_t getFlagsMask(PreflightContext const& ctx);
|
||||
static NotTEC preflight(PreflightContext const& ctx); // NO ledger
|
||||
static TER preclaim(PreclaimContext const& ctx); // read-only
|
||||
TER doApply() override; // read-write
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### Implementation (`src/libxrpl/tx/transactors/MyFeature/MyTx.cpp`)
|
||||
```cpp
|
||||
bool MyTransaction::checkExtraFeatures(PreflightContext const& ctx)
|
||||
{ // REQUIRED: gate on amendment
|
||||
return ctx.rules.enabled(featureMyFeature);
|
||||
}
|
||||
|
||||
NotTEC MyTransaction::preflight(PreflightContext const& ctx)
|
||||
{ // Static validation — NO ctx.view, NO ledger access
|
||||
if (ctx.tx[sfAmount] <= beast::zero)
|
||||
return temBAD_AMOUNT;
|
||||
return tesSUCCESS;
|
||||
}
|
||||
|
||||
TER MyTransaction::preclaim(PreclaimContext const& ctx)
|
||||
{ // Read-only — ctx.view.read() only, NO peek/insert/erase
|
||||
if (!ctx.view.exists(keylet::account(ctx.tx[sfAccount])))
|
||||
return terNO_ACCOUNT;
|
||||
return tesSUCCESS;
|
||||
}
|
||||
|
||||
TER MyTransaction::doApply()
|
||||
{ // Mutable — view().peek(), view().insert(), view().update(), view().erase()
|
||||
auto sle = view().peek(keylet::account(account_));
|
||||
sle->setFieldAmount(sfBalance, newBal);
|
||||
view().update(sle); // REQUIRED after mutation
|
||||
return tesSUCCESS;
|
||||
}
|
||||
```
|
||||
|
||||
### Registration Checklist
|
||||
```cpp
|
||||
// ALL of these are REQUIRED for a new transaction type:
|
||||
// 1. transactions.macro: TRANSACTION(ttMY_TYPE, N, MyTx, delegation, fields)
|
||||
// 2. applySteps.cpp: case ttMY_TYPE: return invoke<MyTransaction>(...);
|
||||
// 3. features.macro: XRPL_FEATURE(MyFeature, Supported::yes, DefaultNo)
|
||||
// 4. Feature.h: increment numFeatures
|
||||
// 5. InvariantCheck.cpp: update if new ledger objects created
|
||||
// 6. Batch.cpp: add to disabledTxTypes if not batch-compatible
|
||||
```
|
||||
|
||||
## Transaction Lifecycle
|
||||
|
||||
1. `preflight` (static checks, no ledger) -> `PreflightResult`
|
||||
2. `preclaim` (ledger state, read-only) -> TER
|
||||
3. `operator()` orchestrates: `checkSeqProxy` -> `checkPriorTxAndLastLedger` -> `checkFee` -> `checkSign` -> `apply`
|
||||
4. `Transactor::apply()` runs `consumeSeqProxy` -> `payFee` -> `doApply` and returns a TER
|
||||
5. `operator()` inspects the TER, decides whether to commit (`ctx_.apply`) or discard (`ctx_.discard`/`reset`)
|
||||
|
||||
## State Commitment & tec* Rollback (CRITICAL for review)
|
||||
|
||||
**`doApply` mutations are NOT committed until `ctx_.apply()` is called at the end of `operator()`.** All peek/insert/update/erase during `doApply` go into an `ApplyContext` view (`view_`) layered on top of `base_`. Whether that view gets flushed to `base_` depends entirely on the TER that `doApply` returns.
|
||||
|
||||
`ApplyContext::discard()` ([src/libxrpl/tx/ApplyContext.cpp](src/libxrpl/tx/ApplyContext.cpp)) replaces `view_` with a fresh view on `base_` — **every doApply mutation is thrown away**:
|
||||
```cpp
|
||||
void ApplyContext::discard() { view_.emplace(&base_, flags_); }
|
||||
```
|
||||
|
||||
### Return-code decision table (in `Transactor::operator()`, [src/libxrpl/tx/Transactor.cpp](src/libxrpl/tx/Transactor.cpp))
|
||||
|
||||
| doApply returns | What commits to the ledger |
|
||||
|---|---|
|
||||
| `tesSUCCESS` | All doApply mutations + fee + seq (via `ctx_.apply`) |
|
||||
| `tec*` (normal, `!tapRETRY`) | `reset(fee)` calls `discard()`, then re-applies fee + seq only. **All doApply mutations reverted.** |
|
||||
| `tec*` with `tapFAIL_HARD` | `discard()` called directly, nothing committed (not even fee) |
|
||||
| `tec*` with `tapRETRY` | `applied=false`, `ctx_.apply` never called, tx re-queued |
|
||||
| `tef*` / `tem*` / `ter*` | `applied=false`, `ctx_.apply` never called |
|
||||
| `tecINVARIANT_FAILED` after invariants | reset again, commit fee only |
|
||||
|
||||
`isTecClaimHardFail(ter, flags) = isTecClaim(ter) && !(flags & tapRETRY)` ([include/xrpl/tx/applySteps.h](include/xrpl/tx/applySteps.h)) — this predicate is what drives the reset path for normal consensus application.
|
||||
|
||||
### What this means for transactor authors and reviewers
|
||||
|
||||
- **A `tec*` return from doApply acts as a full-transaction rollback.** You do NOT need to order mutations defensively so that all checks come before any state changes. If a helper called late in doApply returns `tec*`, everything mutated earlier in the same doApply is discarded via `discard()`.
|
||||
- **Orphan-state bugs of the form "we mutated X then returned tec* so X is now in an inconsistent state" are not possible at the transactor boundary.** The ApplyContext isolates the whole doApply as an atomic unit.
|
||||
- **The real failure mode is within `doApply` itself**: if you call `view().update(sle)` on a stale SLE pointer, or mutate a variable you read by value instead of peek, those are real bugs — but they are in-memory bugs, not state-commit bugs.
|
||||
- **Sandboxes inside `doApply` add nesting, not safety.** `PaymentSandbox` / nested `ApplyView` are useful when you need to conditionally commit a subset of changes *within* a single doApply (e.g., apply offers but revert if the net outcome fails). They are not needed to protect against doApply's own `tec*` return — that rollback is automatic.
|
||||
- **Only `ctx_.apply(result)` publishes to `base_`**; a doApply that `return`s early, throws, or crashes never reaches that call, so base_ stays clean.
|
||||
|
||||
### Verifying a suspected orphan-state bug
|
||||
|
||||
Before claiming "directory removed but SLE not erased because tec\*":
|
||||
1. Read the caller of `doApply` — confirm the TER path (`operator()` in Transactor.cpp).
|
||||
2. Check whether `discard()` is reached via `reset()` or the `tapFAIL_HARD` branch.
|
||||
3. If both paths call `discard()`, the mutations cannot persist on tec\*.
|
||||
4. Look instead for: missing `view().update(sle)` after mutation, stale SLE pointers, or genuine non-atomic side effects (e.g., hash router flags, which are NOT in the ApplyContext view).
|
||||
|
||||
## Permission System
|
||||
|
||||
- `checkSign` dispatches to `checkSingleSign`, `checkMultiSign`, or `checkBatchSign`
|
||||
- `checkPermission` validates delegated authority for delegatable transaction types
|
||||
- Multi-sign requires M-of-N signers matching the signer list; weight threshold must be met
|
||||
|
||||
## Key Files
|
||||
|
||||
- `src/xrpld/app/tx/detail/Transactor.cpp` - base class and pipeline
|
||||
- `include/xrpl/protocol/detail/transactions.macro` - type definitions
|
||||
- `src/xrpld/app/tx/detail/` - per-type implementations (Payment.cpp, OfferCreate.cpp, etc.)
|
||||
- `src/xrpld/app/tx/detail/InvariantCheck.cpp` - post-execution invariant checks
|
||||
62
docs/skills/soul/websockets.md
Normal file
62
docs/skills/soul/websockets.md
Normal file
@@ -0,0 +1,62 @@
|
||||
# WebSockets
|
||||
|
||||
Async WebSocket support for client RPC and real-time subscriptions. Both plain and SSL. Built on Boost.Beast + Boost.Asio.
|
||||
|
||||
## Key Invariants
|
||||
|
||||
- All async operations run on a per-session Boost.Asio strand for thread safety
|
||||
- Outgoing message queue (`wq_`) has a per-session limit (`port().ws_queue_limit`); exceeding it closes the connection with policy error "client is too slow"
|
||||
- `complete()` must be called after processing each message to resume reading; forgetting it stalls the session
|
||||
- `WSInfoSub` holds a weak pointer to `WSSession`; dead sessions are automatically skipped during event delivery
|
||||
- Message size limit: `RPC::Tuning::maxRequestSize`; oversized messages get a `jsonInvalid` error response
|
||||
|
||||
## Connection Flow
|
||||
|
||||
1. `Door` accepts TCP -> `Detector` probes for SSL vs plain -> creates `SSLHTTPPeer` or `PlainHTTPPeer`
|
||||
2. HTTP request with WebSocket upgrade -> `ServerHandler::onHandoff` -> `session.websocketUpgrade()` -> creates `PlainWSPeer` or `SSLWSPeer`
|
||||
3. `BaseWSPeer::run()` -> set permessage-deflate options -> `async_accept` handshake -> `on_ws_handshake` -> `do_read` loop
|
||||
4. `on_read` -> `ServerHandler::onWSMessage` -> validate JSON -> post to job queue -> `processSession` -> send response -> `complete()`
|
||||
|
||||
## Common Bug Patterns
|
||||
|
||||
- `on_read` consumes the buffer AFTER calling `onWSMessage`; if the handler accesses the buffer asynchronously after `onWSMessage` returns, it reads garbage
|
||||
- `close()` with pending messages defers the actual close; calling `send()` after `close()` but before actual close queues more messages
|
||||
- Missing `complete()` call after sending response = session never reads again = appears hung
|
||||
- Job queue shutdown: if `postCoro` returns nullptr, session must close with `going_away`; dropping this silently leaks sessions
|
||||
|
||||
## Review Checklist
|
||||
|
||||
- Verify strand execution for all socket operations (read, write, close, timer)
|
||||
- New subscription streams: ensure `WSInfoSub::send` handles the new event type
|
||||
- Flow control: test with slow clients to verify queue limit enforcement
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### complete() Resumes Read Loop
|
||||
```cpp
|
||||
// REQUIRED after sending response — missing this = session hangs forever
|
||||
void BaseWSPeer::complete()
|
||||
{
|
||||
if (!strand_.running_in_this_thread())
|
||||
return post(strand_, std::bind(
|
||||
&BaseWSPeer::complete, impl().shared_from_this()));
|
||||
do_read(); // resume reading next message
|
||||
}
|
||||
```
|
||||
|
||||
### Queue Limit Enforcement
|
||||
```cpp
|
||||
// REQUIRED: close slow clients to prevent memory exhaustion
|
||||
if (wq_.size() >= port().ws_queue_limit)
|
||||
{
|
||||
close(boost::beast::websocket::close_code::policy_error);
|
||||
return; // do NOT queue unboundedly
|
||||
}
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
- `include/xrpl/server/detail/BaseWSPeer.h` - session lifecycle and message queue
|
||||
- `include/xrpl/server/detail/Door.h` - connection acceptance
|
||||
- `src/xrpld/rpc/detail/ServerHandler.cpp` - `onWSMessage` and `onHandoff`
|
||||
- `src/xrpld/rpc/detail/WSInfoSub.h` - subscription delivery
|
||||
Reference in New Issue
Block a user