From b900bdb43ffb89a2e1f35df1f8d9ef4918dcbff4 Mon Sep 17 00:00:00 2001 From: Denis Angell Date: Wed, 13 May 2026 18:54:45 +0200 Subject: [PATCH] add doc-agent --- .github/scripts/doc-agent/.gitignore | 7 + .github/scripts/doc-agent/README.md | 101 ++ .github/scripts/doc-agent/biome.json | 57 + .github/scripts/doc-agent/package-lock.json | 1123 +++++++++++++++++ .github/scripts/doc-agent/package.json | 34 + .../doc-agent/prompts/document-file.md | 63 + .../scripts/doc-agent/prompts/review-diff.md | 55 + .github/scripts/doc-agent/src/config.ts | 77 ++ .github/scripts/doc-agent/src/document.ts | 114 ++ .github/scripts/doc-agent/src/index.ts | 69 + .../scripts/doc-agent/src/prompt-loader.ts | 34 + .github/scripts/doc-agent/src/review.ts | 222 ++++ .github/scripts/doc-agent/src/types.ts | 37 + .github/scripts/doc-agent/tsconfig.json | 39 + docs/skills/index.md | 126 ++ docs/skills/soul/consensus.md | 86 ++ docs/skills/soul/cryptography.md | 59 + docs/skills/soul/ledger.md | 63 + docs/skills/soul/nodestore.md | 52 + docs/skills/soul/peering.md | 60 + docs/skills/soul/protocol.md | 64 + docs/skills/soul/rpc.md | 61 + docs/skills/soul/shamap.md | 61 + docs/skills/soul/sql.md | 60 + docs/skills/soul/test.md | 75 ++ docs/skills/soul/transactors.md | 141 +++ docs/skills/soul/websockets.md | 62 + 27 files changed, 3002 insertions(+) create mode 100644 .github/scripts/doc-agent/.gitignore create mode 100644 .github/scripts/doc-agent/README.md create mode 100644 .github/scripts/doc-agent/biome.json create mode 100644 .github/scripts/doc-agent/package-lock.json create mode 100644 .github/scripts/doc-agent/package.json create mode 100644 .github/scripts/doc-agent/prompts/document-file.md create mode 100644 .github/scripts/doc-agent/prompts/review-diff.md create mode 100644 .github/scripts/doc-agent/src/config.ts create mode 100644 .github/scripts/doc-agent/src/document.ts create mode 100644 .github/scripts/doc-agent/src/index.ts create mode 100644 .github/scripts/doc-agent/src/prompt-loader.ts create mode 100644 .github/scripts/doc-agent/src/review.ts create mode 100644 .github/scripts/doc-agent/src/types.ts create mode 100644 .github/scripts/doc-agent/tsconfig.json create mode 100644 docs/skills/index.md create mode 100644 docs/skills/soul/consensus.md create mode 100644 docs/skills/soul/cryptography.md create mode 100644 docs/skills/soul/ledger.md create mode 100644 docs/skills/soul/nodestore.md create mode 100644 docs/skills/soul/peering.md create mode 100644 docs/skills/soul/protocol.md create mode 100644 docs/skills/soul/rpc.md create mode 100644 docs/skills/soul/shamap.md create mode 100644 docs/skills/soul/sql.md create mode 100644 docs/skills/soul/test.md create mode 100644 docs/skills/soul/transactors.md create mode 100644 docs/skills/soul/websockets.md diff --git a/.github/scripts/doc-agent/.gitignore b/.github/scripts/doc-agent/.gitignore new file mode 100644 index 0000000000..7fef08162f --- /dev/null +++ b/.github/scripts/doc-agent/.gitignore @@ -0,0 +1,7 @@ +node_modules/ +dist/ +*.log +.env +.env.local +doc-review-report.md +doc-review-comments.json diff --git a/.github/scripts/doc-agent/README.md b/.github/scripts/doc-agent/README.md new file mode 100644 index 0000000000..a0d93d17ac --- /dev/null +++ b/.github/scripts/doc-agent/README.md @@ -0,0 +1,101 @@ +# doc-agent + +Automated documentation agent for the xrpld C++ codebase. Built on the +Claude Agent SDK. + +## What it does + +Two modes: + +- **document** — Add Doxygen `/** */` documentation to a C++ file or + directory. The agent reads the file, related tests, and module skill + context, then writes documentation comments per the project standards in + `docs/DOCUMENTATION_STANDARDS.md`. +- **review** — Given a git diff range, detect documentation drift. Used by + the `doc-review` GitHub Action and locally for testing. + +## Requirements + +- Node.js >= 20 +- `ANTHROPIC_API_KEY` environment variable +- Tools the agent uses: `git`, `gh` (for `--pr`) + +## Install + +```sh +cd .github/scripts/doc-agent +npm install +``` + +## Build and lint + +```sh +npm run typecheck # type check without emitting +npm run build # compile to dist/ +npm run lint # biome lint +npm run format # biome format --write +npm run check # lint + format check (read-only) +npm run check:fix # lint + format + fix +``` + +## Usage + +```sh +export ANTHROPIC_API_KEY=sk-ant-... + +# Document a single file +npm run document include/xrpl/basics/base_uint.h + +# Document an entire module +npm run document include/xrpl/basics/ + +# Review a git range +npm run review develop..HEAD + +# Review a PR +npm run review -- --pr 1234 +``` + +When invoked outside the xrpld repo, set `XRPLD_ROOT` to the path of the +checkout you want to operate on. + +## Outputs + +The `review` mode writes two files in the current directory: + +- `doc-review-report.md` — markdown summary, posted as the PR comment +- `doc-review-comments.json` — array of inline review comments, posted + individually on the PR diff + +## Layout + +``` +doc-agent/ +├── package.json +├── tsconfig.json +├── biome.json +├── prompts/ +│ ├── document-file.md # System prompt for documentation mode +│ └── review-diff.md # System prompt for review mode +└── src/ + ├── index.ts # CLI entry point + ├── config.ts # Paths, model, module-skill map + ├── prompt-loader.ts # Loads prompts + module skill context + ├── document.ts # Document mode + ├── review.ts # Review mode + └── types.ts # Shared types +``` + +## Module skills + +The agent injects per-module context from `docs/skills/soul/*.md` into its +system prompt based on the file path being processed. The mapping lives in +`src/config.ts` (`MODULE_SKILL_MAP`). + +## Notes + +- Prompts live in markdown files, not source, so they can be edited without + touching code. +- The `document` mode uses `permissionMode: 'acceptEdits'` so the agent + writes directly to the target files. Run against a clean git tree so you + can review and revert if needed. diff --git a/.github/scripts/doc-agent/biome.json b/.github/scripts/doc-agent/biome.json new file mode 100644 index 0000000000..c0a1ee95bc --- /dev/null +++ b/.github/scripts/doc-agent/biome.json @@ -0,0 +1,57 @@ +{ + "$schema": "https://biomejs.dev/schemas/1.9.4/schema.json", + "vcs": { + "enabled": true, + "clientKind": "git", + "useIgnoreFile": true + }, + "files": { + "ignoreUnknown": false, + "ignore": ["dist", "node_modules"] + }, + "formatter": { + "enabled": true, + "indentStyle": "space", + "indentWidth": 2, + "lineWidth": 100, + "lineEnding": "lf" + }, + "javascript": { + "formatter": { + "quoteStyle": "single", + "trailingCommas": "all", + "semicolons": "always", + "arrowParentheses": "always" + } + }, + "linter": { + "enabled": true, + "rules": { + "recommended": true, + "correctness": { + "noUnusedVariables": "error", + "noUnusedImports": "error", + "useExhaustiveDependencies": "error" + }, + "style": { + "useConst": "error", + "useTemplate": "error", + "useImportType": "error", + "useExportType": "error", + "noNonNullAssertion": "warn" + }, + "suspicious": { + "noExplicitAny": "error", + "noConsoleLog": "off" + }, + "complexity": { + "noUselessTypeConstraint": "error", + "useArrowFunction": "error", + "useLiteralKeys": "off" + } + } + }, + "organizeImports": { + "enabled": true + } +} diff --git a/.github/scripts/doc-agent/package-lock.json b/.github/scripts/doc-agent/package-lock.json new file mode 100644 index 0000000000..87abebe727 --- /dev/null +++ b/.github/scripts/doc-agent/package-lock.json @@ -0,0 +1,1123 @@ +{ + "name": "xrpld-doc-agent", + "version": "0.1.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "xrpld-doc-agent", + "version": "0.1.0", + "dependencies": { + "@anthropic-ai/claude-agent-sdk": "^0.1.10" + }, + "bin": { + "doc-agent": "dist/index.js" + }, + "devDependencies": { + "@biomejs/biome": "^1.9.4", + "@types/node": "^22.10.0", + "tsx": "^4.19.2", + "typescript": "^5.7.0" + }, + "engines": { + "node": ">=20" + } + }, + "node_modules/@anthropic-ai/claude-agent-sdk": { + "version": "0.1.77", + "resolved": "https://registry.npmjs.org/@anthropic-ai/claude-agent-sdk/-/claude-agent-sdk-0.1.77.tgz", + "integrity": "sha512-ZEjWQtkoB2MEY6K16DWMmF+8OhywAynH0m08V265cerbZ8xPD/2Ng2jPzbbO40mPeFSsMDJboShL+a3aObP0Jg==", + "license": "SEE LICENSE IN README.md", + "engines": { + "node": ">=18.0.0" + }, + "optionalDependencies": { + "@img/sharp-darwin-arm64": "^0.33.5", + "@img/sharp-darwin-x64": "^0.33.5", + "@img/sharp-linux-arm": "^0.33.5", + "@img/sharp-linux-arm64": "^0.33.5", + "@img/sharp-linux-x64": "^0.33.5", + "@img/sharp-linuxmusl-arm64": "^0.33.5", + "@img/sharp-linuxmusl-x64": "^0.33.5", + "@img/sharp-win32-x64": "^0.33.5" + }, + "peerDependencies": { + "zod": "^3.25.0 || ^4.0.0" + } + }, + "node_modules/@biomejs/biome": { + "version": "1.9.4", + "resolved": "https://registry.npmjs.org/@biomejs/biome/-/biome-1.9.4.tgz", + "integrity": "sha512-1rkd7G70+o9KkTn5KLmDYXihGoTaIGO9PIIN2ZB7UJxFrWw04CZHPYiMRjYsaDvVV7hP1dYNRLxSANLaBFGpog==", + "dev": true, + "hasInstallScript": true, + "license": "MIT OR Apache-2.0", + "bin": { + "biome": "bin/biome" + }, + "engines": { + "node": ">=14.21.3" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/biome" + }, + "optionalDependencies": { + "@biomejs/cli-darwin-arm64": "1.9.4", + "@biomejs/cli-darwin-x64": "1.9.4", + "@biomejs/cli-linux-arm64": "1.9.4", + "@biomejs/cli-linux-arm64-musl": "1.9.4", + "@biomejs/cli-linux-x64": "1.9.4", + "@biomejs/cli-linux-x64-musl": "1.9.4", + "@biomejs/cli-win32-arm64": "1.9.4", + "@biomejs/cli-win32-x64": "1.9.4" + } + }, + "node_modules/@biomejs/cli-darwin-arm64": { + "version": "1.9.4", + "resolved": "https://registry.npmjs.org/@biomejs/cli-darwin-arm64/-/cli-darwin-arm64-1.9.4.tgz", + "integrity": "sha512-bFBsPWrNvkdKrNCYeAp+xo2HecOGPAy9WyNyB/jKnnedgzl4W4Hb9ZMzYNbf8dMCGmUdSavlYHiR01QaYR58cw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-darwin-x64": { + "version": "1.9.4", + "resolved": "https://registry.npmjs.org/@biomejs/cli-darwin-x64/-/cli-darwin-x64-1.9.4.tgz", + "integrity": "sha512-ngYBh/+bEedqkSevPVhLP4QfVPCpb+4BBe2p7Xs32dBgs7rh9nY2AIYUL6BgLw1JVXV8GlpKmb/hNiuIxfPfZg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-linux-arm64": { + "version": "1.9.4", + "resolved": "https://registry.npmjs.org/@biomejs/cli-linux-arm64/-/cli-linux-arm64-1.9.4.tgz", + "integrity": "sha512-fJIW0+LYujdjUgJJuwesP4EjIBl/N/TcOX3IvIHJQNsAqvV2CHIogsmA94BPG6jZATS4Hi+xv4SkBBQSt1N4/g==", + "cpu": [ + "arm64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-linux-arm64-musl": { + "version": "1.9.4", + "resolved": "https://registry.npmjs.org/@biomejs/cli-linux-arm64-musl/-/cli-linux-arm64-musl-1.9.4.tgz", + "integrity": "sha512-v665Ct9WCRjGa8+kTr0CzApU0+XXtRgwmzIf1SeKSGAv+2scAlW6JR5PMFo6FzqqZ64Po79cKODKf3/AAmECqA==", + "cpu": [ + "arm64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-linux-x64": { + "version": "1.9.4", + "resolved": "https://registry.npmjs.org/@biomejs/cli-linux-x64/-/cli-linux-x64-1.9.4.tgz", + "integrity": "sha512-lRCJv/Vi3Vlwmbd6K+oQ0KhLHMAysN8lXoCI7XeHlxaajk06u7G+UsFSO01NAs5iYuWKmVZjmiOzJ0OJmGsMwg==", + "cpu": [ + "x64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-linux-x64-musl": { + "version": "1.9.4", + "resolved": "https://registry.npmjs.org/@biomejs/cli-linux-x64-musl/-/cli-linux-x64-musl-1.9.4.tgz", + "integrity": "sha512-gEhi/jSBhZ2m6wjV530Yy8+fNqG8PAinM3oV7CyO+6c3CEh16Eizm21uHVsyVBEB6RIM8JHIl6AGYCv6Q6Q9Tg==", + "cpu": [ + "x64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-win32-arm64": { + "version": "1.9.4", + "resolved": "https://registry.npmjs.org/@biomejs/cli-win32-arm64/-/cli-win32-arm64-1.9.4.tgz", + "integrity": "sha512-tlbhLk+WXZmgwoIKwHIHEBZUwxml7bRJgk0X2sPyNR3S93cdRq6XulAZRQJ17FYGGzWne0fgrXBKpl7l4M87Hg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-win32-x64": { + "version": "1.9.4", + "resolved": "https://registry.npmjs.org/@biomejs/cli-win32-x64/-/cli-win32-x64-1.9.4.tgz", + "integrity": "sha512-8Y5wMhVIPaWe6jw2H+KlEm4wP/f7EW3810ZLmDlrEEy5KvBsb9ECEfu/kMWD484ijfQ8+nIi0giMgu9g1UAuuA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@esbuild/aix-ppc64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.27.7.tgz", + "integrity": "sha512-EKX3Qwmhz1eMdEJokhALr0YiD0lhQNwDqkPYyPhiSwKrh7/4KRjQc04sZ8db+5DVVnZ1LmbNDI1uAMPEUBnQPg==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.27.7.tgz", + "integrity": "sha512-jbPXvB4Yj2yBV7HUfE2KHe4GJX51QplCN1pGbYjvsyCZbQmies29EoJbkEc+vYuU5o45AfQn37vZlyXy4YJ8RQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.27.7.tgz", + "integrity": "sha512-62dPZHpIXzvChfvfLJow3q5dDtiNMkwiRzPylSCfriLvZeq0a1bWChrGx/BbUbPwOrsWKMn8idSllklzBy+dgQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.27.7.tgz", + "integrity": "sha512-x5VpMODneVDb70PYV2VQOmIUUiBtY3D3mPBG8NxVk5CogneYhkR7MmM3yR/uMdITLrC1ml/NV1rj4bMJuy9MCg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.27.7.tgz", + "integrity": "sha512-5lckdqeuBPlKUwvoCXIgI2D9/ABmPq3Rdp7IfL70393YgaASt7tbju3Ac+ePVi3KDH6N2RqePfHnXkaDtY9fkw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.27.7.tgz", + "integrity": "sha512-rYnXrKcXuT7Z+WL5K980jVFdvVKhCHhUwid+dDYQpH+qu+TefcomiMAJpIiC2EM3Rjtq0sO3StMV/+3w3MyyqQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.27.7.tgz", + "integrity": "sha512-B48PqeCsEgOtzME2GbNM2roU29AMTuOIN91dsMO30t+Ydis3z/3Ngoj5hhnsOSSwNzS+6JppqWsuhTp6E82l2w==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.27.7.tgz", + "integrity": "sha512-jOBDK5XEjA4m5IJK3bpAQF9/Lelu/Z9ZcdhTRLf4cajlB+8VEhFFRjWgfy3M1O4rO2GQ/b2dLwCUGpiF/eATNQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.27.7.tgz", + "integrity": "sha512-RkT/YXYBTSULo3+af8Ib0ykH8u2MBh57o7q/DAs3lTJlyVQkgQvlrPTnjIzzRPQyavxtPtfg0EopvDyIt0j1rA==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.27.7.tgz", + "integrity": "sha512-RZPHBoxXuNnPQO9rvjh5jdkRmVizktkT7TCDkDmQ0W2SwHInKCAV95GRuvdSvA7w4VMwfCjUiPwDi0ZO6Nfe9A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ia32": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.27.7.tgz", + "integrity": "sha512-GA48aKNkyQDbd3KtkplYWT102C5sn/EZTY4XROkxONgruHPU72l+gW+FfF8tf2cFjeHaRbWpOYa/uRBz/Xq1Pg==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-loong64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.27.7.tgz", + "integrity": "sha512-a4POruNM2oWsD4WKvBSEKGIiWQF8fZOAsycHOt6JBpZ+JN2n2JH9WAv56SOyu9X5IqAjqSIPTaJkqN8F7XOQ5Q==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-mips64el": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.27.7.tgz", + "integrity": "sha512-KabT5I6StirGfIz0FMgl1I+R1H73Gp0ofL9A3nG3i/cYFJzKHhouBV5VWK1CSgKvVaG4q1RNpCTR2LuTVB3fIw==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ppc64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.27.7.tgz", + "integrity": "sha512-gRsL4x6wsGHGRqhtI+ifpN/vpOFTQtnbsupUF5R5YTAg+y/lKelYR1hXbnBdzDjGbMYjVJLJTd2OFmMewAgwlQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-riscv64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.27.7.tgz", + "integrity": "sha512-hL25LbxO1QOngGzu2U5xeXtxXcW+/GvMN3ejANqXkxZ/opySAZMrc+9LY/WyjAan41unrR3YrmtTsUpwT66InQ==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-s390x": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.27.7.tgz", + "integrity": "sha512-2k8go8Ycu1Kb46vEelhu1vqEP+UeRVj2zY1pSuPdgvbd5ykAw82Lrro28vXUrRmzEsUV0NzCf54yARIK8r0fdw==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.27.7.tgz", + "integrity": "sha512-hzznmADPt+OmsYzw1EE33ccA+HPdIqiCRq7cQeL1Jlq2gb1+OyWBkMCrYGBJ+sxVzve2ZJEVeePbLM2iEIZSxA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.27.7.tgz", + "integrity": "sha512-b6pqtrQdigZBwZxAn1UpazEisvwaIDvdbMbmrly7cDTMFnw/+3lVxxCTGOrkPVnsYIosJJXAsILG9XcQS+Yu6w==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.27.7.tgz", + "integrity": "sha512-OfatkLojr6U+WN5EDYuoQhtM+1xco+/6FSzJJnuWiUw5eVcicbyK3dq5EeV/QHT1uy6GoDhGbFpprUiHUYggrw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.27.7.tgz", + "integrity": "sha512-AFuojMQTxAz75Fo8idVcqoQWEHIXFRbOc1TrVcFSgCZtQfSdc1RXgB3tjOn/krRHENUB4j00bfGjyl2mJrU37A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.27.7.tgz", + "integrity": "sha512-+A1NJmfM8WNDv5CLVQYJ5PshuRm/4cI6WMZRg1by1GwPIQPCTs1GLEUHwiiQGT5zDdyLiRM/l1G0Pv54gvtKIg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openharmony-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.27.7.tgz", + "integrity": "sha512-+KrvYb/C8zA9CU/g0sR6w2RBw7IGc5J2BPnc3dYc5VJxHCSF1yNMxTV5LQ7GuKteQXZtspjFbiuW5/dOj7H4Yw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/sunos-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.27.7.tgz", + "integrity": "sha512-ikktIhFBzQNt/QDyOL580ti9+5mL/YZeUPKU2ivGtGjdTYoqz6jObj6nOMfhASpS4GU4Q/Clh1QtxWAvcYKamA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.27.7.tgz", + "integrity": "sha512-7yRhbHvPqSpRUV7Q20VuDwbjW5kIMwTHpptuUzV+AA46kiPze5Z7qgt6CLCK3pWFrHeNfDd1VKgyP4O+ng17CA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-ia32": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.27.7.tgz", + "integrity": "sha512-SmwKXe6VHIyZYbBLJrhOoCJRB/Z1tckzmgTLfFYOfpMAx63BJEaL9ExI8x7v0oAO3Zh6D/Oi1gVxEYr5oUCFhw==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.27.7.tgz", + "integrity": "sha512-56hiAJPhwQ1R4i+21FVF7V8kSD5zZTdHcVuRFMW0hn753vVfQN8xlx4uOPT4xoGH0Z/oVATuR82AiqSTDIpaHg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@img/sharp-darwin-arm64": { + "version": "0.33.5", + "resolved": "https://registry.npmjs.org/@img/sharp-darwin-arm64/-/sharp-darwin-arm64-0.33.5.tgz", + "integrity": "sha512-UT4p+iz/2H4twwAoLCqfA9UH5pI6DggwKEGuaPy7nCVQ8ZsiY5PIcrRvD1DzuY3qYL07NtIQcWnBSY/heikIFQ==", + "cpu": [ + "arm64" + ], + "license": "Apache-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^18.17.0 || ^20.3.0 || >=21.0.0" + }, + "funding": { + "url": "https://opencollective.com/libvips" + }, + "optionalDependencies": { + "@img/sharp-libvips-darwin-arm64": "1.0.4" + } + }, + "node_modules/@img/sharp-darwin-x64": { + "version": "0.33.5", + "resolved": "https://registry.npmjs.org/@img/sharp-darwin-x64/-/sharp-darwin-x64-0.33.5.tgz", + "integrity": "sha512-fyHac4jIc1ANYGRDxtiqelIbdWkIuQaI84Mv45KvGRRxSAa7o7d1ZKAOBaYbnepLC1WqxfpimdeWfvqqSGwR2Q==", + "cpu": [ + "x64" + ], + "license": "Apache-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^18.17.0 || ^20.3.0 || >=21.0.0" + }, + "funding": { + "url": "https://opencollective.com/libvips" + }, + "optionalDependencies": { + "@img/sharp-libvips-darwin-x64": "1.0.4" + } + }, + "node_modules/@img/sharp-libvips-darwin-arm64": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-darwin-arm64/-/sharp-libvips-darwin-arm64-1.0.4.tgz", + "integrity": "sha512-XblONe153h0O2zuFfTAbQYAX2JhYmDHeWikp1LM9Hul9gVPjFY427k6dFEcOL72O01QxQsWi761svJ/ev9xEDg==", + "cpu": [ + "arm64" + ], + "license": "LGPL-3.0-or-later", + "optional": true, + "os": [ + "darwin" + ], + "funding": { + "url": "https://opencollective.com/libvips" + } + }, + "node_modules/@img/sharp-libvips-darwin-x64": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-darwin-x64/-/sharp-libvips-darwin-x64-1.0.4.tgz", + "integrity": "sha512-xnGR8YuZYfJGmWPvmlunFaWJsb9T/AO2ykoP3Fz/0X5XV2aoYBPkX6xqCQvUTKKiLddarLaxpzNe+b1hjeWHAQ==", + "cpu": [ + "x64" + ], + "license": "LGPL-3.0-or-later", + "optional": true, + "os": [ + "darwin" + ], + "funding": { + "url": "https://opencollective.com/libvips" + } + }, + "node_modules/@img/sharp-libvips-linux-arm": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-arm/-/sharp-libvips-linux-arm-1.0.5.tgz", + "integrity": "sha512-gvcC4ACAOPRNATg/ov8/MnbxFDJqf/pDePbBnuBDcjsI8PssmjoKMAz4LtLaVi+OnSb5FK/yIOamqDwGmXW32g==", + "cpu": [ + "arm" + ], + "libc": [ + "glibc" + ], + "license": "LGPL-3.0-or-later", + "optional": true, + "os": [ + "linux" + ], + "funding": { + "url": "https://opencollective.com/libvips" + } + }, + "node_modules/@img/sharp-libvips-linux-arm64": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-arm64/-/sharp-libvips-linux-arm64-1.0.4.tgz", + "integrity": "sha512-9B+taZ8DlyyqzZQnoeIvDVR/2F4EbMepXMc/NdVbkzsJbzkUjhXv/70GQJ7tdLA4YJgNP25zukcxpX2/SueNrA==", + "cpu": [ + "arm64" + ], + "libc": [ + "glibc" + ], + "license": "LGPL-3.0-or-later", + "optional": true, + "os": [ + "linux" + ], + "funding": { + "url": "https://opencollective.com/libvips" + } + }, + "node_modules/@img/sharp-libvips-linux-x64": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-x64/-/sharp-libvips-linux-x64-1.0.4.tgz", + "integrity": "sha512-MmWmQ3iPFZr0Iev+BAgVMb3ZyC4KeFc3jFxnNbEPas60e1cIfevbtuyf9nDGIzOaW9PdnDciJm+wFFaTlj5xYw==", + "cpu": [ + "x64" + ], + "libc": [ + "glibc" + ], + "license": "LGPL-3.0-or-later", + "optional": true, + "os": [ + "linux" + ], + "funding": { + "url": "https://opencollective.com/libvips" + } + }, + "node_modules/@img/sharp-libvips-linuxmusl-arm64": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linuxmusl-arm64/-/sharp-libvips-linuxmusl-arm64-1.0.4.tgz", + "integrity": "sha512-9Ti+BbTYDcsbp4wfYib8Ctm1ilkugkA/uscUn6UXK1ldpC1JjiXbLfFZtRlBhjPZ5o1NCLiDbg8fhUPKStHoTA==", + "cpu": [ + "arm64" + ], + "libc": [ + "musl" + ], + "license": "LGPL-3.0-or-later", + "optional": true, + "os": [ + "linux" + ], + "funding": { + "url": "https://opencollective.com/libvips" + } + }, + "node_modules/@img/sharp-libvips-linuxmusl-x64": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/@img/sharp-libvips-linuxmusl-x64/-/sharp-libvips-linuxmusl-x64-1.0.4.tgz", + "integrity": "sha512-viYN1KX9m+/hGkJtvYYp+CCLgnJXwiQB39damAO7WMdKWlIhmYTfHjwSbQeUK/20vY154mwezd9HflVFM1wVSw==", + "cpu": [ + "x64" + ], + "libc": [ + "musl" + ], + "license": "LGPL-3.0-or-later", + "optional": true, + "os": [ + "linux" + ], + "funding": { + "url": "https://opencollective.com/libvips" + } + }, + "node_modules/@img/sharp-linux-arm": { + "version": "0.33.5", + "resolved": "https://registry.npmjs.org/@img/sharp-linux-arm/-/sharp-linux-arm-0.33.5.tgz", + "integrity": "sha512-JTS1eldqZbJxjvKaAkxhZmBqPRGmxgu+qFKSInv8moZ2AmT5Yib3EQ1c6gp493HvrvV8QgdOXdyaIBrhvFhBMQ==", + "cpu": [ + "arm" + ], + "libc": [ + "glibc" + ], + "license": "Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^18.17.0 || ^20.3.0 || >=21.0.0" + }, + "funding": { + "url": "https://opencollective.com/libvips" + }, + "optionalDependencies": { + "@img/sharp-libvips-linux-arm": "1.0.5" + } + }, + "node_modules/@img/sharp-linux-arm64": { + "version": "0.33.5", + "resolved": "https://registry.npmjs.org/@img/sharp-linux-arm64/-/sharp-linux-arm64-0.33.5.tgz", + "integrity": "sha512-JMVv+AMRyGOHtO1RFBiJy/MBsgz0x4AWrT6QoEVVTyh1E39TrCUpTRI7mx9VksGX4awWASxqCYLCV4wBZHAYxA==", + "cpu": [ + "arm64" + ], + "libc": [ + "glibc" + ], + "license": "Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^18.17.0 || ^20.3.0 || >=21.0.0" + }, + "funding": { + "url": "https://opencollective.com/libvips" + }, + "optionalDependencies": { + "@img/sharp-libvips-linux-arm64": "1.0.4" + } + }, + "node_modules/@img/sharp-linux-x64": { + "version": "0.33.5", + "resolved": "https://registry.npmjs.org/@img/sharp-linux-x64/-/sharp-linux-x64-0.33.5.tgz", + "integrity": "sha512-opC+Ok5pRNAzuvq1AG0ar+1owsu842/Ab+4qvU879ippJBHvyY5n2mxF1izXqkPYlGuP/M556uh53jRLJmzTWA==", + "cpu": [ + "x64" + ], + "libc": [ + "glibc" + ], + "license": "Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^18.17.0 || ^20.3.0 || >=21.0.0" + }, + "funding": { + "url": "https://opencollective.com/libvips" + }, + "optionalDependencies": { + "@img/sharp-libvips-linux-x64": "1.0.4" + } + }, + "node_modules/@img/sharp-linuxmusl-arm64": { + "version": "0.33.5", + "resolved": "https://registry.npmjs.org/@img/sharp-linuxmusl-arm64/-/sharp-linuxmusl-arm64-0.33.5.tgz", + "integrity": "sha512-XrHMZwGQGvJg2V/oRSUfSAfjfPxO+4DkiRh6p2AFjLQztWUuY/o8Mq0eMQVIY7HJ1CDQUJlxGGZRw1a5bqmd1g==", + "cpu": [ + "arm64" + ], + "libc": [ + "musl" + ], + "license": "Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^18.17.0 || ^20.3.0 || >=21.0.0" + }, + "funding": { + "url": "https://opencollective.com/libvips" + }, + "optionalDependencies": { + "@img/sharp-libvips-linuxmusl-arm64": "1.0.4" + } + }, + "node_modules/@img/sharp-linuxmusl-x64": { + "version": "0.33.5", + "resolved": "https://registry.npmjs.org/@img/sharp-linuxmusl-x64/-/sharp-linuxmusl-x64-0.33.5.tgz", + "integrity": "sha512-WT+d/cgqKkkKySYmqoZ8y3pxx7lx9vVejxW/W4DOFMYVSkErR+w7mf2u8m/y4+xHe7yY9DAXQMWQhpnMuFfScw==", + "cpu": [ + "x64" + ], + "libc": [ + "musl" + ], + "license": "Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^18.17.0 || ^20.3.0 || >=21.0.0" + }, + "funding": { + "url": "https://opencollective.com/libvips" + }, + "optionalDependencies": { + "@img/sharp-libvips-linuxmusl-x64": "1.0.4" + } + }, + "node_modules/@img/sharp-win32-x64": { + "version": "0.33.5", + "resolved": "https://registry.npmjs.org/@img/sharp-win32-x64/-/sharp-win32-x64-0.33.5.tgz", + "integrity": "sha512-MpY/o8/8kj+EcnxwvrP4aTJSWw/aZ7JIGR4aBeZkZw5B7/Jn+tY9/VNwtcoGmdT7GfggGIU4kygOMSbYnOrAbg==", + "cpu": [ + "x64" + ], + "license": "Apache-2.0 AND LGPL-3.0-or-later", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": "^18.17.0 || ^20.3.0 || >=21.0.0" + }, + "funding": { + "url": "https://opencollective.com/libvips" + } + }, + "node_modules/@types/node": { + "version": "22.19.19", + "resolved": "https://registry.npmjs.org/@types/node/-/node-22.19.19.tgz", + "integrity": "sha512-dyh/xO2Fh5bYrfWaaqGrRQQGkNdmYw6AmaAUvYeUMNTWQtvb796ikLdmTchRmOlOiIJ1TDXfWgVx1QkUlQ6Hew==", + "dev": true, + "license": "MIT", + "dependencies": { + "undici-types": "~6.21.0" + } + }, + "node_modules/esbuild": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.27.7.tgz", + "integrity": "sha512-IxpibTjyVnmrIQo5aqNpCgoACA/dTKLTlhMHihVHhdkxKyPO1uBBthumT0rdHmcsk9uMonIWS0m4FljWzILh3w==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.27.7", + "@esbuild/android-arm": "0.27.7", + "@esbuild/android-arm64": "0.27.7", + "@esbuild/android-x64": "0.27.7", + "@esbuild/darwin-arm64": "0.27.7", + "@esbuild/darwin-x64": "0.27.7", + "@esbuild/freebsd-arm64": "0.27.7", + "@esbuild/freebsd-x64": "0.27.7", + "@esbuild/linux-arm": "0.27.7", + "@esbuild/linux-arm64": "0.27.7", + "@esbuild/linux-ia32": "0.27.7", + "@esbuild/linux-loong64": "0.27.7", + "@esbuild/linux-mips64el": "0.27.7", + "@esbuild/linux-ppc64": "0.27.7", + "@esbuild/linux-riscv64": "0.27.7", + "@esbuild/linux-s390x": "0.27.7", + "@esbuild/linux-x64": "0.27.7", + "@esbuild/netbsd-arm64": "0.27.7", + "@esbuild/netbsd-x64": "0.27.7", + "@esbuild/openbsd-arm64": "0.27.7", + "@esbuild/openbsd-x64": "0.27.7", + "@esbuild/openharmony-arm64": "0.27.7", + "@esbuild/sunos-x64": "0.27.7", + "@esbuild/win32-arm64": "0.27.7", + "@esbuild/win32-ia32": "0.27.7", + "@esbuild/win32-x64": "0.27.7" + } + }, + "node_modules/fsevents": { + "version": "2.3.3", + "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", + "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + } + }, + "node_modules/get-tsconfig": { + "version": "4.14.0", + "resolved": "https://registry.npmjs.org/get-tsconfig/-/get-tsconfig-4.14.0.tgz", + "integrity": "sha512-yTb+8DXzDREzgvYmh6s9vHsSVCHeC0G3PI5bEXNBHtmshPnO+S5O7qgLEOn0I5QvMy6kpZN8K1NKGyilLb93wA==", + "dev": true, + "license": "MIT", + "dependencies": { + "resolve-pkg-maps": "^1.0.0" + }, + "funding": { + "url": "https://github.com/privatenumber/get-tsconfig?sponsor=1" + } + }, + "node_modules/resolve-pkg-maps": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/resolve-pkg-maps/-/resolve-pkg-maps-1.0.0.tgz", + "integrity": "sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/privatenumber/resolve-pkg-maps?sponsor=1" + } + }, + "node_modules/tsx": { + "version": "4.21.0", + "resolved": "https://registry.npmjs.org/tsx/-/tsx-4.21.0.tgz", + "integrity": "sha512-5C1sg4USs1lfG0GFb2RLXsdpXqBSEhAaA/0kPL01wxzpMqLILNxIxIOKiILz+cdg/pLnOUxFYOR5yhHU666wbw==", + "dev": true, + "license": "MIT", + "dependencies": { + "esbuild": "~0.27.0", + "get-tsconfig": "^4.7.5" + }, + "bin": { + "tsx": "dist/cli.mjs" + }, + "engines": { + "node": ">=18.0.0" + }, + "optionalDependencies": { + "fsevents": "~2.3.3" + } + }, + "node_modules/typescript": { + "version": "5.9.3", + "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", + "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", + "dev": true, + "license": "Apache-2.0", + "bin": { + "tsc": "bin/tsc", + "tsserver": "bin/tsserver" + }, + "engines": { + "node": ">=14.17" + } + }, + "node_modules/undici-types": { + "version": "6.21.0", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz", + "integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/zod": { + "version": "4.4.3", + "resolved": "https://registry.npmjs.org/zod/-/zod-4.4.3.tgz", + "integrity": "sha512-ytENFjIJFl2UwYglde2jchW2Hwm4GJFLDiSXWdTrJQBIN9Fcyp7n4DhxJEiWNAJMV1/BqWfW/kkg71UDcHJyTQ==", + "license": "MIT", + "peer": true, + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + } + } +} diff --git a/.github/scripts/doc-agent/package.json b/.github/scripts/doc-agent/package.json new file mode 100644 index 0000000000..a4a4f0d5b2 --- /dev/null +++ b/.github/scripts/doc-agent/package.json @@ -0,0 +1,34 @@ +{ + "name": "xrpld-doc-agent", + "version": "0.1.0", + "description": "Automated documentation agent for the xrpld C++ codebase. Uses the Claude Agent SDK to generate Doxygen documentation and detect doc drift on PRs.", + "type": "module", + "private": true, + "bin": { + "doc-agent": "./dist/index.js" + }, + "scripts": { + "build": "tsc", + "start": "node dist/index.js", + "dev": "tsx src/index.ts", + "document": "tsx src/index.ts document", + "review": "tsx src/index.ts review", + "typecheck": "tsc --noEmit", + "lint": "biome lint src", + "format": "biome format --write src", + "check": "biome check src", + "check:fix": "biome check --write src" + }, + "dependencies": { + "@anthropic-ai/claude-agent-sdk": "^0.1.10" + }, + "devDependencies": { + "@biomejs/biome": "^1.9.4", + "@types/node": "^22.10.0", + "tsx": "^4.19.2", + "typescript": "^5.7.0" + }, + "engines": { + "node": ">=20" + } +} diff --git a/.github/scripts/doc-agent/prompts/document-file.md b/.github/scripts/doc-agent/prompts/document-file.md new file mode 100644 index 0000000000..701e7cbfc1 --- /dev/null +++ b/.github/scripts/doc-agent/prompts/document-file.md @@ -0,0 +1,63 @@ +You are documenting C++ code in the xrpld (XRP Ledger daemon) codebase. + +Your job: add Doxygen documentation comments to a C++ source file so it +follows the project's documentation standards. + +## Documentation Standards + +Read `docs/DOCUMENTATION_STANDARDS.md` for the full specification. Key rules: + +- Use `/** ... */` Javadoc-style Doxygen comments (dominant pattern in the + codebase) +- For multi-line comments, prefix each line with ` * ` (space, asterisk, space) +- Document every public class, struct, function, and enum +- Document public methods with `@param`, `@return`, `@throw`/`@throws`, `@note` +- Continuation lines for `@param` descriptions indent 4 spaces from the `*` +- Document `.cpp` files only where the algorithm or invariant is non-obvious +- `JAVADOC_AUTOBRIEF = YES` — the first sentence is automatically the brief, + so `@brief` is optional + +## Quality Rules + +- **Never paraphrase the signature.** `/** Returns the account ID. */` on + `AccountID getAccountID()` is worse than no doc. +- **Document behavior, invariants, and the WHY.** What does this function do + in terms a developer can use? What can go wrong? What's the contract? +- **Read the implementation before writing the doc.** Don't guess what the + function does — read it. +- **Cross-reference test files** to find edge cases worth documenting in + `@note` tags. +- **Be terse.** Target 2-5 lines for classes, 1-3 for functions, plus tag + lines. If you need a multi-paragraph essay, the code probably needs help. +- **Wrong docs are worse than no docs.** If you're not sure what the code + does, say so — don't invent. + +## Module Context + +Before you start, read the relevant skill file in `docs/skills/soul/` for +the module you're working on. These capture per-module conventions, key +classes, and gotchas: + +- `basics`, `crypto`, `json`, `beast` — foundation utilities +- `protocol` — STObject, SField, Serializer, TER codes, Features, Keylets +- `ledger` — ReadView/ApplyView, state tables, payment sandbox +- `tx` / `transactors` — transaction pipeline +- `consensus`, `peering`, `nodestore`, `shamap`, `rpc` — see `docs/skills/soul/` + +## Process + +1. Read the target file completely +2. Read the corresponding skill file in `docs/skills/soul/` if one applies +3. Identify entities that need documentation (public classes, structs, + public methods, free functions in headers, enums) +4. For each entity: read the implementation (and tests if helpful), then + write a Doxygen comment that captures behavior and intent +5. Use the Edit tool to add the comments to the file +6. Do NOT modify code logic — only add documentation +7. Do NOT add documentation to entities that don't need it (private members + with obvious purpose, simple getters where the name is self-explanatory) + +When you finish, summarize: +- How many entities you documented +- Any entities you skipped and why +- Any code patterns you discovered that should be added to a skill file diff --git a/.github/scripts/doc-agent/prompts/review-diff.md b/.github/scripts/doc-agent/prompts/review-diff.md new file mode 100644 index 0000000000..6597ca2529 --- /dev/null +++ b/.github/scripts/doc-agent/prompts/review-diff.md @@ -0,0 +1,55 @@ +You are reviewing a pull request to the xrpld (XRP Ledger daemon) codebase +for documentation drift. + +Your job: given a git diff, determine whether the changes invalidate +existing Doxygen documentation comments, or introduce new public API +surface that lacks documentation. + +## Rules + +- Only flag REAL semantic drift: changed behavior, new parameters, removed + functionality, changed return values, new error conditions, changed + invariants. +- Do NOT flag cosmetic changes (whitespace, formatting, internal renames + that don't change semantics). +- Do NOT suggest docs for private implementation details unless the logic + is genuinely non-obvious. +- Do NOT paraphrase function signatures. Good docs explain WHY and what + BEHAVIOR — not what the code literally does. +- Be terse: 1-3 sentences per finding. + +## Process + +1. For each changed file, get the git diff and the current file content +2. Read existing doc comments on the modified entities +3. For each modified entity, ask: + - Did behavior change in a way the docs miss? + - Did parameters or return values change? + - Are there new error conditions? + - Did the contract / invariant change? + - Is this a NEW public API surface with no docs? +4. Read the module's skill file in `docs/skills/soul/` for context +5. Read related tests if it helps you understand the change +6. Output findings as structured JSON (see below) + +## Output Format + +```json +{ + "summary": "One-paragraph summary of doc state for this PR", + "issues": [ + { + "file": "include/xrpl/protocol/Payment.h", + "line": 42, + "severity": "warning" | "suggestion", + "message": "Brief description of the doc issue", + "suggested_doc": "Optional: suggested doc comment text" + } + ] +} +``` + +- `severity: warning` = doc is now incorrect / misleading +- `severity: suggestion` = new code lacks docs, would be nice to add + +If no issues found, return `{"summary": "Documentation is up to date.", "issues": []}`. diff --git a/.github/scripts/doc-agent/src/config.ts b/.github/scripts/doc-agent/src/config.ts new file mode 100644 index 0000000000..f622afa3bf --- /dev/null +++ b/.github/scripts/doc-agent/src/config.ts @@ -0,0 +1,77 @@ +/** + * Shared configuration for doc-agent. + * + * Paths are resolved relative to the doc-agent directory so the tool works + * regardless of where it's invoked from. + */ + +import { dirname, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const __filename = fileURLToPath(import.meta.url); +const __dirname = dirname(__filename); + +/** Absolute path to the doc-agent root (parent of src/). */ +export const AGENT_DIR: string = resolve(__dirname, '..'); + +/** Absolute path to the prompts directory. */ +export const PROMPTS_DIR: string = resolve(AGENT_DIR, 'prompts'); + +/** + * Absolute path to the xrpld repo root. + * + * Defaults to three levels up from doc-agent (which lives at + * .github/scripts/doc-agent/). Override with the XRPLD_ROOT env var when + * running against a different checkout. + */ +export const XRPLD_ROOT: string = process.env['XRPLD_ROOT'] ?? resolve(AGENT_DIR, '..', '..', '..'); + +/** Model used for documentation generation and review. */ +export const MODEL: string = process.env['DOC_AGENT_MODEL'] ?? 'claude-opus-4-7'; + +/** Absolute path to the skills directory inside the xrpld repo. */ +export const SKILLS_DIR: string = resolve(XRPLD_ROOT, 'docs', 'skills'); + +/** + * Map module path prefixes to their skill file name in docs/skills/soul/. + * + * Used to inject module-specific context into the agent's system prompt + * when documenting or reviewing code in that module. + */ +export const MODULE_SKILL_MAP: Readonly> = { + 'src/libxrpl/basics/': null, + 'src/libxrpl/crypto/': 'cryptography.md', + 'src/libxrpl/json/': null, + 'src/libxrpl/beast/': null, + 'src/libxrpl/protocol/': 'protocol.md', + 'src/libxrpl/ledger/': 'ledger.md', + 'src/libxrpl/tx/': 'transactors.md', + 'src/libxrpl/nodestore/': 'nodestore.md', + 'src/libxrpl/shamap/': 'shamap.md', + 'src/libxrpl/rdb/': 'sql.md', + 'src/xrpld/consensus/': 'consensus.md', + 'src/xrpld/overlay/': 'peering.md', + 'src/xrpld/peerfinder/': 'peering.md', + 'src/xrpld/rpc/': 'rpc.md', + 'include/xrpl/crypto/': 'cryptography.md', + 'include/xrpl/protocol/': 'protocol.md', + 'include/xrpl/ledger/': 'ledger.md', + 'include/xrpl/tx/': 'transactors.md', + 'include/xrpl/nodestore/': 'nodestore.md', + 'include/xrpl/shamap/': 'shamap.md', +}; + +/** + * Resolve which skill file applies to a given source path. + * + * @param sourcePath - Path relative to the xrpld repo root + * @returns The skill file name, or null if no skill applies + */ +export function skillForPath(sourcePath: string): string | null { + for (const [prefix, skillFile] of Object.entries(MODULE_SKILL_MAP)) { + if (sourcePath.startsWith(prefix) || sourcePath.includes(`/${prefix}`)) { + return skillFile; + } + } + return null; +} diff --git a/.github/scripts/doc-agent/src/document.ts b/.github/scripts/doc-agent/src/document.ts new file mode 100644 index 0000000000..60d8c54c92 --- /dev/null +++ b/.github/scripts/doc-agent/src/document.ts @@ -0,0 +1,114 @@ +/** + * Document mode: add Doxygen docs to a file or all files in a directory. + */ + +import { existsSync, readdirSync, statSync } from 'node:fs'; +import { join, relative, resolve } from 'node:path'; +import { query } from '@anthropic-ai/claude-agent-sdk'; +import { MODEL, XRPLD_ROOT } from './config.js'; +import { loadSystemPrompt } from './prompt-loader.js'; + +const CPP_EXTENSIONS: ReadonlySet = new Set(['.h', '.hpp', '.cpp']); + +/** + * Recursively find all C++ source files under a target path. + * + * @param target - File or directory path (relative to xrpld root or absolute) + * @returns Absolute paths of all matching files + */ +function findCppFiles(target: string): string[] { + const absTarget = resolve(XRPLD_ROOT, target); + if (!existsSync(absTarget)) { + throw new Error(`Target does not exist: ${absTarget}`); + } + + const stat = statSync(absTarget); + if (stat.isFile()) { + return [absTarget]; + } + + const results: string[] = []; + const walk = (dir: string): void => { + for (const entry of readdirSync(dir, { withFileTypes: true })) { + const full = join(dir, entry.name); + if (entry.isDirectory()) { + walk(full); + } else if (entry.isFile()) { + const dotIdx = entry.name.lastIndexOf('.'); + if (dotIdx === -1) continue; + const ext = entry.name.slice(dotIdx); + if (CPP_EXTENSIONS.has(ext)) { + results.push(full); + } + } + } + }; + walk(absTarget); + return results; +} + +/** + * Document a single file by running the documentation agent against it. + */ +async function documentFile(absPath: string): Promise { + const relPath = relative(XRPLD_ROOT, absPath); + console.log(`\n=== Documenting: ${relPath} ===`); + + const systemPrompt = await loadSystemPrompt('document-file', relPath); + const userPrompt = `Add Doxygen documentation to: ${relPath} + +The file is rooted at ${XRPLD_ROOT}. Use the Read tool to read it, the Edit +tool to add documentation, and Glob/Grep to find related tests or callers +when needed. + +Do not modify any code logic — only add documentation comments.`; + + const result = query({ + prompt: userPrompt, + options: { + model: MODEL, + systemPrompt, + cwd: XRPLD_ROOT, + allowedTools: ['Read', 'Edit', 'Glob', 'Grep', 'Bash'], + permissionMode: 'acceptEdits', + }, + }); + + for await (const message of result) { + if (message.type === 'assistant') { + const content = message.message?.content; + if (Array.isArray(content)) { + for (const block of content) { + if (block.type === 'text') { + process.stdout.write(block.text); + } + } + } + } + if (message.type === 'result') { + const cost = message.total_cost_usd?.toFixed(4) ?? '?'; + const inTok = message.usage?.['input_tokens'] ?? 0; + const outTok = message.usage?.['output_tokens'] ?? 0; + console.log(`\n[Cost: $${cost}, Tokens: ${inTok}/${outTok}]`); + } + } +} + +/** + * Document a file or every C++ file under a directory. + * + * @param target - File or directory path + */ +export async function documentTarget(target: string): Promise { + const files = findCppFiles(target); + console.log(`Found ${files.length} C++ file(s) to document.`); + + for (const file of files) { + try { + await documentFile(file); + } catch (err) { + const message = err instanceof Error ? err.message : String(err); + console.error(`Failed to document ${file}: ${message}`); + } + } +} diff --git a/.github/scripts/doc-agent/src/index.ts b/.github/scripts/doc-agent/src/index.ts new file mode 100644 index 0000000000..491dbd84c0 --- /dev/null +++ b/.github/scripts/doc-agent/src/index.ts @@ -0,0 +1,69 @@ +#!/usr/bin/env node +/** + * xrpld doc-agent CLI entry point. + * + * @example + * doc-agent document src/libxrpl/basics/base_uint.h + * doc-agent document include/xrpl/basics/ + * doc-agent review develop..HEAD + * doc-agent review --pr 1234 + */ + +import { documentTarget } from './document.js'; +import { reviewDiff } from './review.js'; + +const USAGE = ` +xrpld doc-agent + +Usage: + doc-agent document Add Doxygen documentation + doc-agent review .. Detect doc drift in range + doc-agent review --pr Detect doc drift for a PR + +Environment: + ANTHROPIC_API_KEY (required) Anthropic API key + XRPLD_ROOT (optional) Path to xrpld repo root (default: repo root) + DOC_AGENT_MODEL (optional) Model override (default: claude-opus-4-7) +`; + +function printUsageAndExit(code: number): never { + console.error(USAGE); + process.exit(code); +} + +const HELP_MODES: ReadonlySet = new Set(['help', '--help', '-h']); + +async function main(): Promise { + const [mode, ...args] = process.argv.slice(2); + + if (process.env['ANTHROPIC_API_KEY'] === undefined) { + console.error('ERROR: ANTHROPIC_API_KEY environment variable is required.'); + process.exit(1); + } + + if (mode === undefined || HELP_MODES.has(mode)) { + printUsageAndExit(0); + } + + if (mode === 'document') { + const target = args[0]; + if (target === undefined) printUsageAndExit(1); + await documentTarget(target); + return; + } + + if (mode === 'review') { + if (args.length === 0) printUsageAndExit(1); + await reviewDiff(args); + return; + } + + console.error(`Unknown mode: ${mode}`); + printUsageAndExit(1); +} + +main().catch((err: unknown) => { + const message = err instanceof Error ? (err.stack ?? err.message) : String(err); + console.error('FATAL:', message); + process.exit(1); +}); diff --git a/.github/scripts/doc-agent/src/prompt-loader.ts b/.github/scripts/doc-agent/src/prompt-loader.ts new file mode 100644 index 0000000000..05eb63b9f4 --- /dev/null +++ b/.github/scripts/doc-agent/src/prompt-loader.ts @@ -0,0 +1,34 @@ +/** + * Loads system prompts and injects module-specific skill context. + */ + +import { existsSync } from 'node:fs'; +import { readFile } from 'node:fs/promises'; +import { resolve } from 'node:path'; +import { PROMPTS_DIR, SKILLS_DIR, skillForPath } from './config.js'; + +/** + * Load a system prompt from prompts/ and append the relevant module skill + * if one applies to the given source path. + * + * @param promptName - Base name of the prompt file (without .md extension) + * @param sourcePath - Path relative to the xrpld repo root + * @returns The fully-assembled system prompt + */ +export async function loadSystemPrompt(promptName: string, sourcePath: string): Promise { + const basePromptPath = resolve(PROMPTS_DIR, `${promptName}.md`); + const basePrompt = await readFile(basePromptPath, 'utf8'); + + const skillFile = skillForPath(sourcePath); + if (skillFile === null) { + return basePrompt; + } + + const skillPath = resolve(SKILLS_DIR, 'soul', skillFile); + if (!existsSync(skillPath)) { + return basePrompt; + } + + const skill = await readFile(skillPath, 'utf8'); + return `${basePrompt}\n\n## Module Skill (${skillFile})\n\n${skill}`; +} diff --git a/.github/scripts/doc-agent/src/review.ts b/.github/scripts/doc-agent/src/review.ts new file mode 100644 index 0000000000..65cee7ecb6 --- /dev/null +++ b/.github/scripts/doc-agent/src/review.ts @@ -0,0 +1,222 @@ +/** + * Review mode: detect documentation drift in a git diff range. + * + * Used by the doc-review GitHub Action and locally for testing. + */ + +import { execSync } from 'node:child_process'; +import { writeFile } from 'node:fs/promises'; +import { query } from '@anthropic-ai/claude-agent-sdk'; +import { MODEL, XRPLD_ROOT } from './config.js'; +import { loadSystemPrompt } from './prompt-loader.js'; +import type { FileReviewResult, GitRange, ReviewIssue, ReviewOutput } from './types.js'; + +const MAX_DIFF_CHARS = 12_000; +const TRACKED_PATH_PATTERN = /^(include|src\/libxrpl|src\/xrpld)\//; +const CPP_FILE_PATTERN = /\.(h|hpp|cpp)$/; + +/** + * Parse the CLI arguments into a base..head git range. + * + * Accepts either: + * - `base..head` (e.g. `develop..HEAD`) + * - `--pr ` (resolves via `gh pr view`) + */ +function parseRangeArgs(args: readonly string[]): GitRange { + const first = args[0]; + if (first === undefined) { + throw new Error('Expected range as base..head or --pr '); + } + + if (first === '--pr') { + const pr = args[1]; + if (pr === undefined) { + throw new Error('--pr requires a PR number'); + } + const base = execSync(`gh pr view ${pr} --json baseRefOid -q .baseRefOid`, { + cwd: XRPLD_ROOT, + }) + .toString() + .trim(); + const head = execSync(`gh pr view ${pr} --json headRefOid -q .headRefOid`, { + cwd: XRPLD_ROOT, + }) + .toString() + .trim(); + return { base, head }; + } + + const match = first.match(/^([^.]+)\.\.([^.]+)$/); + if (match === null || match[1] === undefined || match[2] === undefined) { + throw new Error('Expected range as base..head or --pr '); + } + return { base: match[1], head: match[2] }; +} + +/** + * Get the list of C++ source files changed in the given git range, + * filtered to paths the doc-agent cares about. + */ +function getChangedCppFiles(range: GitRange): string[] { + const out = execSync(`git diff --name-only ${range.base}...${range.head}`, { + cwd: XRPLD_ROOT, + }).toString(); + + return out + .split('\n') + .filter((line) => line.length > 0) + .filter((file) => CPP_FILE_PATTERN.test(file)) + .filter((file) => TRACKED_PATH_PATTERN.test(file)); +} + +/** Get the unified diff for a single file in the given range. */ +function getFileDiff(range: GitRange, file: string): string { + return execSync(`git diff ${range.base}...${range.head} -- "${file}"`, { + cwd: XRPLD_ROOT, + maxBuffer: 10 * 1024 * 1024, + }).toString(); +} + +/** Extract a JSON object from a possibly-fenced model response. */ +function extractJson(response: string): ReviewOutput | null { + const fenced = response.match(/```json\s*([\s\S]*?)```/); + const raw = fenced?.[1] ?? response.match(/(\{[\s\S]*\})/)?.[1]; + if (raw === undefined) return null; + try { + return JSON.parse(raw) as ReviewOutput; + } catch { + return null; + } +} + +/** Send one file's diff to the agent and parse the response. */ +async function reviewFile(range: GitRange, file: string): Promise { + console.log(`\n=== Reviewing: ${file} ===`); + const diff = getFileDiff(range, file); + if (diff.trim().length === 0) return null; + + const systemPrompt = await loadSystemPrompt('review-diff', file); + const userPrompt = `Review this diff for documentation drift: + +## File: ${file} + +## Diff +\`\`\` +${diff.slice(0, MAX_DIFF_CHARS)} +\`\`\` + +Use the Read tool to inspect the current state of the file, related tests, +or callers if needed. Output findings as JSON per the schema in the system +prompt.`; + + let response = ''; + const result = query({ + prompt: userPrompt, + options: { + model: MODEL, + systemPrompt, + cwd: XRPLD_ROOT, + allowedTools: ['Read', 'Glob', 'Grep', 'Bash'], + permissionMode: 'acceptEdits', + }, + }); + + for await (const message of result) { + if (message.type === 'assistant') { + const content = message.message?.content; + if (Array.isArray(content)) { + for (const block of content) { + if (block.type === 'text') { + response += block.text; + } + } + } + } + } + + const parsed = extractJson(response); + if (parsed === null) { + console.warn(` No JSON output for ${file}, skipping`); + return null; + } + + const issues: ReviewIssue[] = parsed.issues.map((issue) => ({ + file: issue.file ?? file, + line: issue.line, + severity: issue.severity, + message: issue.message, + ...(issue.suggested_doc !== undefined && { suggestedDoc: issue.suggested_doc }), + })); + + return { file, summary: parsed.summary, issues }; +} + +/** Build the markdown report posted to the PR. */ +function buildReport(fileCount: number, results: readonly FileReviewResult[]): string { + const issues = results.flatMap((r) => r.issues); + const warnings = issues.filter((i) => i.severity === 'warning').length; + const suggestions = issues.length - warnings; + + const lines: string[] = ['## Documentation Review Report', '']; + lines.push( + issues.length === 0 + ? 'No documentation issues found.' + : `Found **${issues.length}** issue(s) across **${fileCount}** changed file(s): ${warnings} warning(s), ${suggestions} suggestion(s).`, + ); + lines.push(''); + + for (const result of results) { + if (result.issues.length === 0) continue; + lines.push(`### \`${result.file}\``, '', result.summary, ''); + for (const issue of result.issues) { + const tag = issue.severity === 'warning' ? '**Warning:**' : '**Suggestion:**'; + lines.push(`- ${tag} Line ${issue.line}: ${issue.message}`); + } + lines.push(''); + } + + lines.push('---', '*Automated review by doc-agent.*'); + return lines.join('\n'); +} + +/** + * Review the documentation drift introduced by a git range or PR. + * + * Writes two output files in the current working directory: + * - doc-review-report.md (markdown summary for PR comment) + * - doc-review-comments.json (inline review comments) + */ +export async function reviewDiff(args: readonly string[]): Promise { + const range = parseRangeArgs(args); + console.log(`Reviewing range: ${range.base}...${range.head}`); + + const files = getChangedCppFiles(range); + if (files.length === 0) { + console.log('No C++ files changed in this range.'); + await writeFile('doc-review-report.md', '## Documentation Review\n\nNo C++ files changed.\n'); + await writeFile('doc-review-comments.json', '[]'); + return; + } + + console.log(`Found ${files.length} changed C++ file(s).`); + + const results: FileReviewResult[] = []; + for (const file of files) { + try { + const result = await reviewFile(range, file); + if (result !== null) results.push(result); + } catch (err) { + const message = err instanceof Error ? err.message : String(err); + console.warn(` Review failed for ${file}: ${message}`); + } + } + + const report = buildReport(files.length, results); + const allIssues = results.flatMap((r) => r.issues); + + await writeFile('doc-review-report.md', report); + await writeFile('doc-review-comments.json', JSON.stringify(allIssues, null, 2)); + + console.log('\nReport: doc-review-report.md'); + console.log(`Inline comments: doc-review-comments.json (${allIssues.length} issues)`); +} diff --git a/.github/scripts/doc-agent/src/types.ts b/.github/scripts/doc-agent/src/types.ts new file mode 100644 index 0000000000..1022b80b72 --- /dev/null +++ b/.github/scripts/doc-agent/src/types.ts @@ -0,0 +1,37 @@ +/** + * Shared type definitions for the doc-agent. + */ + +export type Severity = 'warning' | 'suggestion'; + +export interface ReviewIssue { + file: string; + line: number; + severity: Severity; + message: string; + suggestedDoc?: string; +} + +export interface FileReviewResult { + file: string; + summary: string; + issues: ReviewIssue[]; +} + +export interface ReviewOutput { + summary: string; + issues: Array<{ + file?: string; + line: number; + severity: Severity; + message: string; + suggested_doc?: string; + }>; +} + +export interface GitRange { + base: string; + head: string; +} + +export type AgentMode = 'document' | 'review'; diff --git a/.github/scripts/doc-agent/tsconfig.json b/.github/scripts/doc-agent/tsconfig.json new file mode 100644 index 0000000000..211634f00d --- /dev/null +++ b/.github/scripts/doc-agent/tsconfig.json @@ -0,0 +1,39 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "NodeNext", + "moduleResolution": "NodeNext", + "lib": ["ES2023"], + "outDir": "dist", + "rootDir": "src", + "strict": true, + "noImplicitAny": true, + "strictNullChecks": true, + "strictFunctionTypes": true, + "strictBindCallApply": true, + "strictPropertyInitialization": true, + "noImplicitThis": true, + "alwaysStrict": true, + "noUnusedLocals": true, + "noUnusedParameters": true, + "exactOptionalPropertyTypes": true, + "noImplicitReturns": true, + "noFallthroughCasesInSwitch": true, + "noUncheckedIndexedAccess": true, + "noImplicitOverride": true, + "noPropertyAccessFromIndexSignature": true, + "allowUnusedLabels": false, + "allowUnreachableCode": false, + "esModuleInterop": true, + "forceConsistentCasingInFileNames": true, + "skipLibCheck": true, + "declaration": true, + "declarationMap": true, + "sourceMap": true, + "resolveJsonModule": true, + "isolatedModules": true, + "verbatimModuleSyntax": true + }, + "include": ["src/**/*"], + "exclude": ["node_modules", "dist"] +} diff --git a/docs/skills/index.md b/docs/skills/index.md new file mode 100644 index 0000000000..c68a81def4 --- /dev/null +++ b/docs/skills/index.md @@ -0,0 +1,126 @@ +# xrpld Codebase Skills Index + +## Description +This is the top-level guide for all best-practices skills in this repository. Use this to understand the codebase organization and find the right skill for any task. + +## When to Use Skills +Reference a skill whenever you are: +- **Writing new code** in a module - check the skill first for established patterns +- **Modifying existing code** - verify your changes follow module conventions +- **Adding a new transaction type** - see `libxrpl/tx/transactors.md` for the full template +- **Debugging** - skills list key files and common pitfalls per module +- **Reviewing code** - skills document what "correct" looks like for each module + +## Codebase Architecture + +The codebase is split into two main areas: + +### `src/libxrpl/` — The Library (skills in `.claude/skills/libxrpl/`) +Reusable library code: data types, serialization, cryptography, ledger state, transaction processing, and storage. This is the **protocol layer**. + +| Module | Responsibility | +|--------|---------------| +| `basics` | Foundational types: Buffer, Slice, base_uint, Number, logging, error contracts | +| `beast` | Support layer: Journal logging, test framework, instrumentation, IP types | +| `conditions` | Crypto-conditions (RFC): fulfillment validation, DER encoding | +| `core` | Job queue, load monitoring, hash-based message dedup | +| `crypto` | CSPRNG, secure erasure, RFC1751 encoding | +| `json` | Json::Value, parsing, serialization, StaticString optimization | +| `ledger` | ReadView/ApplyView, state tables, payment sandbox, credit ops | +| `net` | HTTP/HTTPS client, SSL certs, async I/O | +| `nodestore` | Persistent node storage: RocksDB, NuDB, Memory backends | +| `protocol` | STObject hierarchy, SField, Serializer, TER codes, Features, Keylets | +| `proto` | Protocol Buffer generated headers (gRPC API definitions) | +| `rdb` | SOCI database wrapper, checkpointing | +| `resource` | Rate limiting, endpoint tracking, abuse prevention | +| `server` | Port config, SSL/TLS, WebSocket, admin networks | +| `shamap` | SHA-256 Merkle radix tree (16-way branching, COW) | +| `tx` | Transaction pipeline: Transactor base, preflight/preclaim/doApply | + +### `src/xrpld/` — The Server Application (skills in `.claude/skills/xrpld/`) +The running rippled server: application lifecycle, consensus, networking, RPC, and peer management. This is the **application layer**. + +| Module | Responsibility | +|--------|---------------| +| `app` | Application singleton, ledger management, consensus adapters, services | +| `app/main` | Application initialization and lifecycle | +| `app/ledger` | Ledger storage, retrieval, immutable state management | +| `app/consensus` | RCL consensus adapters (bridges generic algorithm to rippled) | +| `app/misc` | Fee voting, amendments, SHAMapStore, TxQ, validators, NetworkOPs | +| `app/paths` | Payment path finding algorithm, trust line caching | +| `app/rdb` | Application-level database operations | +| `app/tx` | Application-level transaction handling | +| `consensus` | Generic consensus algorithm (CRTP-based, app-independent) | +| `core` | Configuration (Config.h), time keeping, network ID | +| `overlay` | P2P networking: peer connections, protocol buffers, clustering | +| `peerfinder` | Network discovery: bootcache, livecache, slot management | +| `perflog` | Performance logging and instrumentation | +| `rpc` | RPC handler dispatch, coroutine suspension, 40+ command handlers | +| `shamap` | Application-level SHAMap operations (NodeFamily) | + +### `include/xrpl/` — Header Files +Headers live in `include/xrpl/` and mirror the `src/libxrpl/` structure. Each skill already references its corresponding headers in the "Key Files" section. + +## Cross-Cutting Conventions + +### Error Handling +- **Transaction errors**: Return `TER` enum (tesSUCCESS, tecFROZEN, temBAD_AMOUNT, etc.) +- **Logic errors**: `Throw()`, `LogicError()`, `XRPL_ASSERT()` +- **I/O errors**: `Status` enum or `boost::system::error_code` +- **RPC errors**: Inject via `context.params` + +### Assertions +```cpp +XRPL_ASSERT(condition, "ClassName::method : description"); // Debug only +XRPL_VERIFY(condition, "ClassName::method : description"); // Always enabled +``` + +### Logging +```cpp +JLOG(j_.warn()) << "Message"; // Always wrap in JLOG macro +``` + +### Memory Management +- `IntrusiveRefCounts` + `SharedIntrusive` for shared ownership in libxrpl +- `std::shared_ptr` for shared ownership in xrpld +- `std::unique_ptr` for exclusive ownership +- `CountedObject` mixin for instance tracking + +### Feature Gating +```cpp +if (ctx.rules.enabled(featureMyFeature)) { /* new behavior */ } +``` + +### Code Organization +- Headers in `include/xrpl/`, implementations in `src/libxrpl/` or `src/xrpld/` +- `#pragma once` (never `#ifndef` guards) +- `namespace xrpl { }` for all code +- `detail/` namespace for internal helpers +- Factory functions: `make_*()` returning `unique_ptr` or `shared_ptr` + +## Subsystem Skills (soul/) + +Concise invariants, bug patterns, and review checklists for each subsystem: + +| Subsystem | File | Focus | +|-----------|------|-------| +| Consensus | `soul/consensus.md` | State machine, amendments, UNL, validations, TX ordering | +| Cryptography | `soul/cryptography.md` | Key types, signing, hashing, handshake | +| Ledger | `soul/ledger.md` | Immutability, acquisition, entry types | +| NodeStore | `soul/nodestore.md` | Backends, caching, online deletion | +| Peering | `soul/peering.md` | Overlay, connections, squelching | +| Protocol | `soul/protocol.md` | Macros, serialization format, field ordering | +| RPC | `soul/rpc.md` | Handler dispatch, roles, subscriptions | +| SHAMap | `soul/shamap.md` | COW, node types, sync, proofs | +| SQL | `soul/sql.md` | Schema, config, checkpointing | +| Testing | `soul/test.md` | JTx framework, Env setup, TER expectations, amendment gating | +| Transactors | `soul/transactors.md` | Pipeline, new TX types, signing, transactor template | +| WebSockets | `soul/websockets.md` | Session lifecycle, flow control | + +### Workflow & Processes +| Skill | File | +|-------|------| +| Workflow Orchestration | `workflow.md` | +| Task Management | `task-management.md` | +| Amendment Creation | `amendment.md` | +| Merge Conflicts | `merge-conflicts.md` | diff --git a/docs/skills/soul/consensus.md b/docs/skills/soul/consensus.md new file mode 100644 index 0000000000..54e426cf49 --- /dev/null +++ b/docs/skills/soul/consensus.md @@ -0,0 +1,86 @@ +# Consensus + +Template-based state machine in `Consensus.h` parameterized by an Adaptor (`RCLConsensus`). Three phases: open -> establish -> accepted. Modes: proposing, observing, wrongLedger, switchedLedger. + +## Key Invariants + +- A ledger cannot close until the previous ledger reaches consensus AND (has transactions OR close time reached) +- Proposals must have strictly increasing sequence numbers per peer; stale proposals are silently dropped +- The Avalanche state machine progressively lowers consensus thresholds over time (init -> mid -> late -> stuck) to prevent livelock +- `minCONSENSUS_PCT = 80` is the baseline; timing params: `ledgerMIN_CONSENSUS = 1950ms`, `ledgerMAX_CONSENSUS = 15s` +- Dead nodes (`deadNodes_`) are permanently excluded for the round once they bow out + +## Common Bug Patterns + +- Proposals referencing a stale `prevLedgerID_` after a ledger switch cause split-brain; always check `newPeerProp.prevLedger() != prevLedgerID_` before processing +- Resetting the consensus timer during `establish` phase causes re-convergence and potential split; timer must only reset on phase transitions +- `DisputedTx::updateVote` changes local vote based on peer pressure; bugs here cause determinism failures across nodes +- `createDisputes()` deduplicates via `compares` set; missing this check creates duplicate disputes that skew vote counts +- The `peerUnchangedCounter_` is reset to 0 when any vote changes; bugs in this counter cause premature consensus declaration + +## Amendments + +- 80% validator support for 2 weeks to enable; tracked via `AmendmentTable` with `amendmentMap_` +- New amendments: add to `features.macro` with `XRPL_FEATURE`/`XRPL_FIX`, increment `numFeatures` in `Feature.h` +- Unsupported enabled amendment blocks the server (`setAmendmentBlocked`); no mechanism to disable/revoke +- Voting happens each consensus round in `doVoting`; votes are persisted in `FeatureVotes` SQLite table +- `fixAmendmentMajorityCalc` changed the threshold calculation; check which calculation applies + +## UNL and Negative UNL + +- Negative UNL temporarily disables unreliable validators (max 25% of UNL: `negativeUNLMaxListed = 0.25`) +- Scoring uses `buildScoreTable` over recent ledger history; low watermark (50%) = disable candidate, high watermark (80%) = re-enable candidate +- Candidate selection is deterministic via previous ledger hash as randomizing pad +- `newValidatorDisableSkip = FLAG_LEDGER_INTERVAL * 2` prevents disabling newly joined validators prematurely + +## Validations + +- `ValidationParms` defines freshness windows: CURRENT_WALL=5min, CURRENT_LOCAL=3min, SET_EXPIRES=10min, FRESHNESS=20s +- `SeqEnforcer` rejects validations with regressed or duplicate sequence numbers (`ValStatus::badSeq`) +- Conflicting validations (same seq, different hash) are logged as byzantine behavior +- `handleNewValidation` is the entry point: checks trust, adds to `Validations` set, triggers `checkAccept` if current+trusted + +## Transaction Ordering + +- `CanonicalTXSet` orders by: salted account key (XOR with random salt) -> sequence proxy -> transaction ID +- Salt prevents manipulation of ordering by account selection +- `TxQ` uses `OrderCandidates`: higher fee level first, then `txID XOR parentHash` as tiebreaker +- Per-account limit: `maximumTxnPerAccount`; blocked transactions held until blocker resolves + +## Key Patterns + +### Proposal Validation (prevents split-brain) +```cpp +// REQUIRED: reject proposals referencing stale previous ledger +if (newPeerProp.prevLedger() != prevLedgerID_) +{ + JLOG(j_.debug()) << "Got proposal for " << newPeerProp.prevLedger() + << " but we are on " << prevLedgerID_; + return; +} +``` + +### Complete Bow-Out Handling +```cpp +// REQUIRED: all three steps — unvote, erase position, mark dead +if (newPeerProp.isBowOut()) +{ + if (result_) + for (auto& it : result_->disputes) + it.second.unVote(peerID); + if (currPeerPositions_.find(peerID) != currPeerPositions_.end()) + currPeerPositions_.erase(peerID); + deadNodes_.insert(peerID); // permanently excluded this round +} +``` + +## Key Files + +- `src/xrpld/consensus/Consensus.h` - state machine +- `src/xrpld/consensus/ConsensusParms.h` - timing/threshold params +- `src/xrpld/app/consensus/RCLConsensus.cpp` - XRPL adaptor +- `src/xrpld/consensus/DisputedTx.h` - dispute tracking +- `src/xrpld/app/misc/detail/AmendmentTable.cpp` - amendment logic +- `src/xrpld/app/misc/NegativeUNLVote.cpp` - N-UNL voting +- `src/xrpld/consensus/Validations.h` - validation tracking +- `src/xrpld/app/misc/CanonicalTXSet.h` - TX ordering diff --git a/docs/skills/soul/cryptography.md b/docs/skills/soul/cryptography.md new file mode 100644 index 0000000000..effda4a1b9 --- /dev/null +++ b/docs/skills/soul/cryptography.md @@ -0,0 +1,59 @@ +# Cryptography + +XRPL supports secp256k1 (ECDSA) and ed25519 key types. All crypto uses OpenSSL + dedicated libs (libsecp256k1, ed25519-donna). + +## Key Invariants + +- `SecretKey` destructor calls `secure_erase` on internal buffer; any code handling secret keys must follow this pattern +- ed25519 public keys are prefixed with `0xED` (33 bytes total); secp256k1 keys are 33-byte compressed +- `sha512Half` (first 32 bytes of SHA-512) is the standard hash used throughout XRPL for node hashing, signing, etc. +- `RIPEMD-160(SHA-256(x))` is used for account ID derivation (`ripesha_hasher`) +- Base58 encoding includes a type byte prefix and 4-byte checksum (double SHA-256) + +## Common Bug Patterns + +- Mixing up key types: secp256k1 signing hashes the message with sha512Half first, ed25519 signs the raw message +- `signDigest` only works with secp256k1; calling it with ed25519 throws a logic error +- Signature canonicality: ed25519 `verify` checks signature canonicality before calling `ed25519_sign_open`; non-canonical signatures are rejected +- Overlay handshake uses `signDigest` to sign the session fingerprint (`sharedValue`); the signature binds the TLS session to the node identity + +## Review Checklist + +- New crypto code must use `crypto_prng()` singleton for randomness, never raw `rand()` +- Secret key buffers must be `secure_erase`d after use +- Verify that key type dispatch handles both secp256k1 and ed25519 (or explicitly rejects one with a clear error) + +## Key Patterns + +### Secure Erasure +```cpp +// REQUIRED: destructor must erase secret material +SecretKey::~SecretKey() +{ + secure_erase(buf_, sizeof(buf_)); +} + +// REQUIRED: erase intermediate buffers after use +beast::rngfill(buf, sizeof(buf), crypto_prng()); +SecretKey sk(Slice{buf, sizeof(buf)}); +secure_erase(buf, sizeof(buf)); // MUST erase raw buffer +``` + +### Key Type Dispatch +```cpp +// REQUIRED: handle both key types or explicitly reject +if (type == KeyType::ed25519) +{ /* ed25519 path */ } +else if (type == KeyType::secp256k1) +{ /* secp256k1 path */ } +else + LogicError("unknown key type"); // MUST NOT fall through silently +``` + +## Key Files + +- `include/xrpl/protocol/SecretKey.h` / `PublicKey.h` - key types +- `src/libxrpl/protocol/SecretKey.cpp` - signing, key generation +- `src/libxrpl/protocol/PublicKey.cpp` - verification +- `include/xrpl/protocol/digest.h` - hash functions +- `src/xrpld/overlay/detail/Handshake.cpp` - overlay handshake crypto diff --git a/docs/skills/soul/ledger.md b/docs/skills/soul/ledger.md new file mode 100644 index 0000000000..ce561c57cd --- /dev/null +++ b/docs/skills/soul/ledger.md @@ -0,0 +1,63 @@ +# Ledger + +Each ledger is an immutable snapshot: header (seq, hashes, close time) + state SHAMap + transaction SHAMap. `LedgerMaster` is the central coordinator. + +## Key Invariants + +- Once `setImmutable()` is called, the ledger and its SHAMaps cannot change; only immutable ledgers can be set in `LedgerHolder` +- Every server always has an open ledger; the open ledger cannot close until previous consensus completes +- Ledger header hashes to the ledger's identity hash; includes state root, tx root, parent hash, total coins, close time +- `LedgerMaster` tracks: `mPubLedger` (last published), `mValidLedger` (last validated), `mLedgerHistory` (cache) +- Validation requires minimum trusted validations (`minVal`); filtered by Negative UNL + +## Common Bug Patterns + +- Modifying a ledger after `setImmutable()` corrupts shared state; always check `mImmutable` before mutation +- Gap detection: if ledgers 603 and 600 exist but 601-602 are missing, `LedgerMaster` requests 602 first, then backfills 601 +- `InboundLedger::gotData()` queues data for processing; calling `done()` before all data arrives creates incomplete ledgers +- `checkAccept` won't accept a ledger that isn't ahead of the last validated ledger; stale validations are silently ignored + +## Ledger Entry Types + +- Defined in `ledger_entries.macro` using `LEDGER_ENTRY(type, code, class, name, fields)` +- Each entry has an `SOTemplate` defining required/optional fields +- Key computation: `Indexes.cpp` computes unique keys (keylets) for each ledger object type +- `STLedgerEntry` wraps the serialized data with type-safe field access + +## Review Checklist + +- New ledger entry types: add to `ledger_entries.macro`, implement keylet in `Indexes.cpp` +- Verify `LedgerCleaner` can handle the new entry type for repair +- Check that acquisition code handles the entry in both `InboundLedger` and `LedgerMaster` + +## Key Patterns + +### Immutability Guard +```cpp +// After this, no mutations allowed on the ledger or its SHAMaps +inline void SHAMap::setImmutable() +{ + XRPL_ASSERT(state_ != SHAMapState::Invalid, "..."); + state_ = SHAMapState::Immutable; +} +// VERIFY: code never calls peek()/insert()/erase() after setImmutable() +``` + +### New Ledger Entry Keylet +```cpp +// REQUIRED: every new ledger entry type needs unique keylet computation +Keylet keylet::myEntry(AccountID const& id) +{ + return {ltMY_ENTRY, + sha512Half(std::uint16_t(spaceMyEntry), id)}; +} +// Also add to ledger_entries.macro and Indexes.cpp +``` + +## Key Files + +- `src/xrpld/app/ledger/Ledger.h` - ledger class +- `src/xrpld/app/ledger/detail/LedgerMaster.cpp` - central coordinator +- `src/xrpld/app/ledger/detail/InboundLedger.cpp` - ledger acquisition +- `include/xrpl/protocol/detail/ledger_entries.macro` - entry type definitions +- `src/libxrpl/protocol/Indexes.cpp` - keylet computation diff --git a/docs/skills/soul/nodestore.md b/docs/skills/soul/nodestore.md new file mode 100644 index 0000000000..2674719981 --- /dev/null +++ b/docs/skills/soul/nodestore.md @@ -0,0 +1,52 @@ +# NodeStore + +Persistent key-value store for `NodeObject`s (ledger entries). All ledger state is stored here between launches. Keys are 256-bit hashes. + +## Key Invariants + +- `NodeObject` types: `hotLEDGER` (1), `hotACCOUNT_NODE` (3), `hotTRANSACTION_NODE` (4), `hotDUMMY` (512, cache marker for missing entries) +- Preferred backends: NuDB (append-only) and RocksDB; LevelDB/HyperLevelDB are deprecated +- `TaggedCache` evicts by both `cache_size` (max items) and `cache_age` (max minutes) +- `DatabaseRotatingImp` uses two backends (writable + archive) for online deletion; rotation moves writable to archive, creates new writable, deletes old archive +- Corrupt data triggers fatal logging; unknown/backend errors logged with appropriate severity + +## Common Bug Patterns + +- `fetchNodeObject` with `duplicate=true` copies from archive to writable backend; forgetting this in rotating mode means objects disappear after rotation +- `hotDUMMY` objects in cache mark missing entries; code that checks cache hits must distinguish real objects from dummies +- Batch write limit is 65536 objects; exceeding this silently truncates or fails depending on backend +- `fdRequired()` must be called during resource planning; running out of file descriptors causes silent backend failures + +## Review Checklist + +- Config changes: verify `[node_db]` section has valid `type`, `path`, and `compression` settings +- Online deletion: ensure `SHAMapStoreImp` coordinates rotation with the application lifecycle +- New backend types: implement the full `Backend` interface including `fdRequired()` + +## Key Patterns + +### Cache Lookup — Distinguish Real vs Dummy +```cpp +// REQUIRED: hotDUMMY marks "confirmed missing" — not a real object +auto obj = cache_.fetch(hash); +if (obj && obj->getType() == hotDUMMY) + return nullptr; // not found, just cached as missing +return obj; +``` + +### Backend File Descriptor Reporting +```cpp +// REQUIRED: every backend must accurately report FD needs +int fdRequired() const override +{ + return fdLimit_; // inaccurate values cause silent failures +} +``` + +## Key Files + +- `include/xrpl/nodestore/NodeObject.h` - object types +- `include/xrpl/nodestore/Backend.h` - backend interface +- `include/xrpl/nodestore/detail/DatabaseNodeImp.h` - standard implementation +- `src/libxrpl/nodestore/DatabaseRotatingImp.cpp` - rotating/online deletion +- `src/xrpld/app/misc/SHAMapStoreImp.cpp` - lifecycle management diff --git a/docs/skills/soul/peering.md b/docs/skills/soul/peering.md new file mode 100644 index 0000000000..8f7effafcd --- /dev/null +++ b/docs/skills/soul/peering.md @@ -0,0 +1,60 @@ +# Overlay Peering + +P2P network using persistent TCP/IP connections. Messages serialized via Protocol Buffers. `OverlayImpl` manages connections; `PeerImp` handles per-peer logic. + +## Key Invariants + +- Connection preference order: Fixed Peers -> Livecache -> Bootcache +- Cluster connections do NOT count toward connection limits (unlimited) +- Protobuf message changes MUST maintain wire compatibility or risk network partitioning +- Squelching: after enough peers relay a validator's messages, a subset is "Selected" and the rest are temporarily muted to reduce bandwidth +- Handshake binds TLS session to node identity via `signDigest` of the session fingerprint + +## Common Bug Patterns + +- PeerFinder slot exhaustion: if `maxPeers` is reached, new outbound connections silently fail; check slot availability before connecting +- `HashRouter::shouldRelay` prevents duplicate relay; bypassing it causes message storms +- `ConnectAttempt::processResponse` on HTTP 503 parses "peer-ips" for alternatives; malformed responses here can crash with bad IP parsing +- `PeerImp::close` must run on the strand; calling from wrong thread causes race conditions on socket and timer state +- Destructor chain: `~PeerImp` -> `deletePeer` -> `onPeerDeactivate` -> `on_closed` -> `remove`; interrupting this chain leaks slots + +## Connection Lifecycle + +1. `OverlayImpl::connect` -> check resource limits -> allocate PeerFinder slot -> create `ConnectAttempt` +2. Async TCP connect -> TLS handshake -> HTTP upgrade with identity headers +3. `processResponse` -> verify handshake -> create `PeerImp` -> `add_active` -> `run()` +4. `doProtocolStart` -> start async message receive loop -> exchange validator lists and manifests + +## Review Checklist + +- Verify resource manager checks on both inbound and outbound connections +- New protocol messages: update protobuf definitions AND verify wire compatibility +- Squelch changes: test with high peer counts; incorrect squelch logic can silence validators + +## Key Patterns + +### Strand Execution +```cpp +// REQUIRED: socket operations must run on the strand +if (!strand_.running_in_this_thread()) + return post(strand_, std::bind( + &PeerImp::close, shared_from_this())); +// Calling socket ops from wrong thread causes races on state +``` + +### Duplicate Relay Prevention +```cpp +// REQUIRED: check HashRouter before relaying +if (!hashRouter_.shouldRelay(hash)) + return; // already relayed — suppress duplicate +overlay_.relay(message, hash); +// Bypassing this causes message storms across the network +``` + +## Key Files + +- `src/xrpld/overlay/detail/OverlayImpl.cpp` - main overlay manager +- `src/xrpld/overlay/detail/PeerImp.cpp` - per-peer logic +- `src/xrpld/overlay/detail/ConnectAttempt.cpp` - outbound connection +- `src/xrpld/overlay/Slot.h` - squelch state machine +- `src/xrpld/overlay/detail/Handshake.cpp` - handshake crypto diff --git a/docs/skills/soul/protocol.md b/docs/skills/soul/protocol.md new file mode 100644 index 0000000000..ff41015ffc --- /dev/null +++ b/docs/skills/soul/protocol.md @@ -0,0 +1,64 @@ +# Protocol and Serialization + +Macro-driven system for defining features, transactions, ledger entries, and serialized fields. Canonical binary format is required for signatures and consensus. + +## Key Invariants + +- Fields are sorted by (type code, field code) for canonical serialization; sorting by Field ID bytes produces WRONG results +- Field ID encoding: 1-3 bytes depending on type/field code values (both < 16 = 1 byte) +- Signing hash prefix: `0x53545800` for single-signing, `0x534D5400` for multi-signing +- `STObject[sfFoo]` returns value or default; `STObject[~sfFoo]` returns optional (nothing if absent) +- All ST types inherit from `STBase`; `STVar` provides type-erased storage with stack/heap allocation + +## Macro System + +- `XRPL_FEATURE(name, supported, vote)` / `XRPL_FIX` / `XRPL_RETIRE` in `features.macro` +- `TRANSACTION(tag, value, class, delegation, fields)` in `transactions.macro` +- `LEDGER_ENTRY(type, code, class, name, fields)` in `ledger_entries.macro` +- `TYPED_SFIELD(name, TYPE, code)` in `sfields.macro` +- Adding any new definition requires updating the count in the corresponding header + +## Serialization Format + +- XRP Amount: 8 bytes, MSB=0, next bit=1 for positive, remaining 62 bits = value +- Token Amount: 8 bytes mantissa/exponent + 20 bytes currency + 20 bytes issuer +- AccountID: 20 bytes, length-prefixed when top-level +- STArray: elements between start (`0xf0`) and end (`0xf1`) markers +- STObject: fields in canonical order between start (`0xe0`) and end (`0xe1`) markers +- Length prefixing: 1 byte (0-192), 2 bytes (193-12480), 3 bytes (12481-918744) + +## Common Bug Patterns + +- Adding a field to `transactions.macro` without adding it to `sfields.macro` causes silent serialization failures +- Forgetting to increment `numFeatures` after adding to `features.macro` causes out-of-bounds access +- Non-canonical field ordering in hand-built binary blobs causes signature verification failures +- `soeMPTSupported` flag on amount fields enables MPT token support; omitting it silently rejects MPT payments + +## Key Patterns + +### Amendment Registration +```cpp +// In features.macro — REQUIRED format: +XRPL_FEATURE(MyNewFeature, Supported::yes, VoteBehavior::DefaultNo) +XRPL_FIX (MyBugFix, Supported::yes, VoteBehavior::DefaultNo) +// MUST also increment numFeatures in Feature.h — omitting causes OOB access +``` + +### STObject Field Access +```cpp +// Safe: optional access — returns std::optional, never throws +if (auto const val = tx[~sfAmount]) + use(*val); + +// Throws if field is absent — only safe when preflight guarantees presence +auto const amount = tx[sfAmount]; +``` + +## Key Files + +- `include/xrpl/protocol/detail/features.macro` - amendment definitions +- `include/xrpl/protocol/detail/transactions.macro` - transaction types +- `include/xrpl/protocol/detail/ledger_entries.macro` - ledger objects +- `include/xrpl/protocol/detail/sfields.macro` - field definitions +- `include/xrpl/protocol/Feature.h` - `numFeatures` constant +- `src/libxrpl/protocol/STObject.cpp` - core serialized object diff --git a/docs/skills/soul/rpc.md b/docs/skills/soul/rpc.md new file mode 100644 index 0000000000..665863deb5 --- /dev/null +++ b/docs/skills/soul/rpc.md @@ -0,0 +1,61 @@ +# RPC + +JSON-RPC over HTTP/WebSocket and gRPC. Central handler table dispatches by method name + API version. Roles: ADMIN, USER, IDENTIFIED, PROXY, FORBID. + +## Key Invariants + +- Handler table in `Handler.cpp`: each entry = `{name, function, role, condition, minApiVer, maxApiVer}` +- `conditionMet` checks server state (e.g., `NEEDS_CURRENT_LEDGER`) before invoking handler +- API v2.0+ errors: structured objects with `status`, `code`, `message`; earlier: flat fields in response +- Sensitive fields (`passphrase`, `secret`, `seed`, `seed_hex`) are masked in error responses +- Batch requests: `"method": "batch"` with `"params"` array; each sub-request processed independently + +## Common Bug Patterns + +- New handler without entry in `Handler.cpp` static array = handler silently unreachable +- Wrong `role_` on handler: USER-level handler with admin-only data leaks; ADMIN handler accessible to users = security hole +- `conditionMet` returning false causes a generic error; ensure new conditions are documented +- Resource charging: each request gets a fee via `Resource::Consumer`; missing charge allows DoS +- `maxRequestSize` (RPC::Tuning) rejection happens before JSON parsing; oversized requests get no error detail + +## Adding New RPC Handler + +1. Declare in `Handlers.h`: `Json::Value doMyCommand(RPC::JsonContext&);` +2. Implement in new file under `src/xrpld/rpc/handlers/` +3. Register in `Handler.cpp` static array with role, condition, version range +4. For gRPC: define in `xrp_ledger.proto`, add `CallData` in `GRPCServerImpl::setupListeners()` + +## Subscriptions + +- WebSocket clients can subscribe to: `server`, `ledger`, `book_changes`, `transactions`, `validations`, `manifests`, `peer_status` (admin), `consensus` +- `WSInfoSub` delivers events via weak pointer to `WSSession`; dead sessions are automatically cleaned up +- `RPCSub` delivers to remote URL endpoints with auth and SSL support + +## Key Patterns + +### Handler Table Registration +```cpp +// In Handler.cpp handlerArray[] — REQUIRED for every new handler: +{"my_command", byRef(&doMyCommand), Role::USER, NO_CONDITION}, +// role MUST match security requirements: +// Role::ADMIN for internal-only, Role::USER for public API +// condition: NEEDS_CURRENT_LEDGER, NEEDS_NETWORK_CONNECTION, or NO_CONDITION +``` + +### Version-Ranged Handler +```cpp +// New-style handler with API version range +template <> Handler handlerFrom() +{ return {MyCommandHandler::name, &handle, + MyCommandHandler::role, MyCommandHandler::condition, + MyCommandHandler::minApiVer, MyCommandHandler::maxApiVer}; +} +``` + +## Key Files + +- `src/xrpld/rpc/handlers/Handlers.h` - authoritative handler list +- `src/xrpld/rpc/detail/Handler.cpp` - handler table and dispatch +- `src/xrpld/rpc/detail/RPCHandler.cpp` - request processing pipeline +- `src/xrpld/rpc/detail/ServerHandler.cpp` - HTTP/WS entry points +- `include/xrpl/protocol/ErrorCodes.h` - error code definitions diff --git a/docs/skills/soul/shamap.md b/docs/skills/soul/shamap.md new file mode 100644 index 0000000000..e8baa8060b --- /dev/null +++ b/docs/skills/soul/shamap.md @@ -0,0 +1,61 @@ +# SHAMap + +Merkle radix trie (radix 16) enabling O(1) subtree comparison via hash. Used for both state tree and transaction tree. Root is always a `SHAMapInnerNode`. + +## Key Invariants + +- Mutable SHAMaps have non-zero `cowid`; immutable have `cowid=0`. Once immutable, nodes persist for the map's lifetime with NO mechanism to remove them +- Copy-on-write: `unshareNode` must be called before mutating any node in a mutable SHAMap; failing this corrupts shared snapshots +- Inner nodes have up to 16 children; hash is computed from children's hashes. Leaf hash is computed from data + type-specific prefix +- `canonicalize` ensures only one instance per hash in the cache; prevents races between threads +- `SHAMapInnerNode` uses atomic operations + locking (`std::atomic lock_`) for concurrent child access + +## Common Bug Patterns + +- Modifying a node without calling `unshareNode` first corrupts the snapshot that shares it; this is the #1 SHAMap bug class +- `getMissingNodes` uses deferred async reads; processing completions out of order causes incorrect "full below" marking +- Inner node serialization has two formats (compressed vs full) chosen by branch count; mismatched deserializer causes corruption +- `addKnownNode` traverses toward target; if branch is empty or hash mismatches, returns "invalid" -- callers must handle this gracefully +- Proof path verification walks leaf-to-root; incorrect key at any level causes false negative + +## Serialization Formats + +- **Compressed**: only non-empty branches serialized (saves space for sparse nodes) +- **Full**: all 16 branches including empty ones (used for dense nodes) +- Choice is automatic in `serializeForWire` based on branch count + +## Leaf Node Types + +- `SHAMapAccountStateLeafNode` - account state entries +- `SHAMapTxLeafNode` - transactions +- `SHAMapTxPlusMetaLeafNode` - transactions with metadata +- Each uses a different hash prefix for domain separation + +## Key Patterns + +### State Machine +```cpp +enum class SHAMapState { + Modifying = 0, // can add/remove objects + Immutable = 1, // FROZEN — no changes allowed + Synching = 2, // hash fixed, missing nodes can be added + Invalid = 3, // corrupt — do not use +}; +// VERIFY: no peek()/insert()/erase() calls on Immutable maps +``` + +### COW Discipline (#1 Bug Class) +```cpp +// REQUIRED before mutating any shared node: +auto node = unshareNode(branch, key); // copies if shared +node->setChild(index, child); // now safe to modify +// BUG: skipping unshareNode corrupts snapshots sharing the node +``` + +## Key Files + +- `include/xrpl/shamap/SHAMap.h` - main class +- `include/xrpl/shamap/SHAMapInnerNode.h` - inner node (COW, threading) +- `include/xrpl/shamap/SHAMapLeafNode.h` - leaf node base +- `src/libxrpl/shamap/SHAMapSync.cpp` - sync, missing nodes, proofs +- `src/libxrpl/shamap/SHAMapDelta.cpp` - walkMap, parallel traversal diff --git a/docs/skills/soul/sql.md b/docs/skills/soul/sql.md new file mode 100644 index 0000000000..30bd2d713e --- /dev/null +++ b/docs/skills/soul/sql.md @@ -0,0 +1,60 @@ +# SQL Database + +SQLite via SOCI for ledger/transaction history. Only SQLite is supported; Postgres has no implementation despite interface comments. + +## Key Invariants + +- Two main databases: `lgrdb_` (ledger) and `txdb_` (transactions, optional via `useTxTables` config) +- Transaction tables are optional; disabling them means no transaction history or account_tx queries +- WAL checkpointing triggers when WAL file grows beyond threshold; scheduled via job queue +- Database init failure is fatal (throws exception, prevents construction) +- Free disk space < 512MB triggers fatal error on write operations + +## Schema + +- `Ledgers` table: seq, hash, parent hash, total coins, close time, etc. Indexed by `LedgerSeq` +- `Transactions` table: TransID, TransType, FromAcct, FromSeq, LedgerSeq, Status, RawTxn, TxnMeta. Indexed by `LedgerSeq` +- `AccountTransactions` table: TransID, Account, LedgerSeq, TxnSeq. Triple-indexed for account_tx queries +- Secondary DBs: Wallet (node identity, manifests), PeerFinder (bootstrap cache), State (deletion tracking) + +## Common Bug Patterns + +- No schema migration system; `CREATE TABLE IF NOT EXISTS` means old schemas silently persist with missing columns +- PeerFinder DB is the exception -- it has schema versioning via `SchemaVersion` table +- `safety_level` config affects journal_mode and synchronous; "low" can lose data on crash +- `page_size` must be power of 2 between 512-65536; invalid values cause init failure +- Online deletion coordinates between NodeStore rotation and SQL table pruning; race conditions here lose history + +## Configuration + +| Option | Section | Values | Default | +|--------|---------|--------|---------| +| `backend` | `[relational_db]` | `sqlite` only | sqlite | +| `page_size` | `[sqlite]` | 512-65536, power of 2 | 4096 | +| `safety_level` | `[sqlite]` | high, medium, low | high | +| `journal_size_limit` | `[sqlite]` | integer >= 0 | 1582080 | + +## Key Patterns + +### Schema Evolution Caveat +```cpp +// WARNING: no migration system — old databases keep old schemas +// CREATE TABLE IF NOT EXISTS silently skips if table exists with old columns +// New columns on existing tables require manual ALTER TABLE or +// documentation that the column is optional and may be absent +``` + +### Disk Space Guard +```cpp +// REQUIRED on write paths: < 512MB triggers fatal to prevent corruption +if (freeDiskSpace < minDiskFree) + Throw("Not enough disk space for database write"); +``` + +## Key Files + +- `src/xrpld/app/rdb/backend/detail/SQLiteDatabase.cpp` - main implementation +- `src/xrpld/app/main/DBInit.h` - schema definitions +- `src/xrpld/core/detail/DatabaseCon.cpp` - connection setup and pragmas +- `src/xrpld/app/rdb/backend/detail/Node.cpp` - ledger/tx operations +- `src/xrpld/app/rdb/detail/State.cpp` - deletion state tracking diff --git a/docs/skills/soul/test.md b/docs/skills/soul/test.md new file mode 100644 index 0000000000..5f6de01096 --- /dev/null +++ b/docs/skills/soul/test.md @@ -0,0 +1,75 @@ +# Testing + +JTx framework for in-memory ledger testing. Tests live in `src/test/`, derive from `beast::unit_test::suite`, and register with `BEAST_DEFINE_TESTSUITE`. + +## Key Patterns + +### Test File Structure +```cpp +class MyFeature_test : public beast::unit_test::suite { + void testBasic() { + testcase("basic"); + using namespace jtx; + Env env{*this}; + // ... test logic ... + } + void run() override { testBasic(); } +}; +BEAST_DEFINE_TESTSUITE(MyFeature, app, ripple); +``` + +### Amendment Gating +```cpp +// REQUIRED: test with AND without the feature amendment +Env env{*this}; // all amendments on +Env env{*this, testable_amendments() - featureMyFeature}; // feature disabled +// With custom config: +Env env{*this, envconfig([](std::unique_ptr cfg) { + /* modify config */ return cfg; +})}; +``` + +### Transaction Submission +```cpp +auto const alice = Account{"alice"}; +auto const bob = Account{"bob"}; +env.fund(XRP(10000), alice, bob); +env.close(); // REQUIRED between setup and test transactions + +env(pay(alice, bob, XRP(100))); // expects tesSUCCESS +env(pay(alice, bob, XRP(999999)), ter(tecUNFUNDED_PAYMENT)); // specific TER +env(pay(alice, bob, XRP(100)), txflags(tfPartialPayment)); // with flags +``` + +### State Verification +```cpp +env.require(balance(alice, XRP(9900))); +env.require(balance(alice, USD(1000))); +env.require(owners(alice, 2)); + +auto const sle = env.le(keylet::account(alice.id())); +BEAST_EXPECT(sle); +BEAST_EXPECT((*sle)[sfBalance] == XRP(9900)); +``` + +## Review Checklist + +- New transaction types MUST have tests with AND without the feature amendment +- Every error path (tem*, tec*, tef*) must have a test exercising it +- `env.close()` required between transactions that need ledger boundaries +- Verify state after transaction, not just TER code +- Test class naming: `FeatureName_test`, registered with `BEAST_DEFINE_TESTSUITE` + +## Common Pitfalls + +- Forgetting `*this` as first Env arg disconnects assertion reporting +- Comparing XRPAmount with raw integers instead of `XRP()` or `drops()` +- Trust lines must be established before IOU payments +- Each Env creates a full Application — keep test methods focused + +## Key Files + +- `src/test/jtx/Env.h` — test environment class +- `src/test/jtx/jtx.h` — convenience header (includes all helpers) +- `src/test/jtx/Account.h` — test accounts +- `src/test/jtx/amount.h` — XRP(), drops(), IOU helpers diff --git a/docs/skills/soul/transactors.md b/docs/skills/soul/transactors.md new file mode 100644 index 0000000000..b0be63aa32 --- /dev/null +++ b/docs/skills/soul/transactors.md @@ -0,0 +1,141 @@ +# Transactors + +Transaction processing pipeline: preflight (static validation) -> preclaim (ledger state checks) -> doApply (state mutation). Base class `Transactor` in `src/libxrpl/tx/`. + +## Key Invariants + +- Pipeline is strict: preflight runs WITHOUT ledger state, preclaim runs WITH read-only view, doApply runs with mutable view +- `preflight` validates all fields exist and are well-formed; this is the ONLY place to reject malformed transactions cheaply +- Fee is always deducted even if the transaction fails (`tecCLAIM` pattern); `payFee` runs before `doApply` +- Sequence/ticket consumption happens in `consumeSeqProxy`; must succeed before any state changes +- Invariant checkers run after `doApply`; they can veto the transaction post-execution + +## Common Bug Patterns + +- New transaction type missing preflight validation for new fields = malformed transactions reach doApply and corrupt state +- Forgetting to handle `tecCLAIM` in doApply: fee is deducted but no other state changes should occur +- Batch transactions (`Batch` type) have their own signing path (`checkBatchSign`); changes to signing must cover both paths +- `calculateBaseFee` override without updating `minimumFee` causes fee calculation divergence between nodes +- Missing invariant checker update for new ledger entry types = silent constraint violations + +## Transactor Template + +### Header (`include/xrpl/tx/transactors/MyTx.h`) +```cpp +#pragma once +#include + +namespace xrpl { +class MyTransaction : public Transactor { +public: + static constexpr ConsequencesFactoryType ConsequencesFactory{Normal}; + explicit MyTransaction(ApplyContext& ctx) : Transactor(ctx) {} + + static bool checkExtraFeatures(PreflightContext const& ctx); + static std::uint32_t getFlagsMask(PreflightContext const& ctx); + static NotTEC preflight(PreflightContext const& ctx); // NO ledger + static TER preclaim(PreclaimContext const& ctx); // read-only + TER doApply() override; // read-write +}; +} +``` + +### Implementation (`src/libxrpl/tx/transactors/MyFeature/MyTx.cpp`) +```cpp +bool MyTransaction::checkExtraFeatures(PreflightContext const& ctx) +{ // REQUIRED: gate on amendment + return ctx.rules.enabled(featureMyFeature); +} + +NotTEC MyTransaction::preflight(PreflightContext const& ctx) +{ // Static validation — NO ctx.view, NO ledger access + if (ctx.tx[sfAmount] <= beast::zero) + return temBAD_AMOUNT; + return tesSUCCESS; +} + +TER MyTransaction::preclaim(PreclaimContext const& ctx) +{ // Read-only — ctx.view.read() only, NO peek/insert/erase + if (!ctx.view.exists(keylet::account(ctx.tx[sfAccount]))) + return terNO_ACCOUNT; + return tesSUCCESS; +} + +TER MyTransaction::doApply() +{ // Mutable — view().peek(), view().insert(), view().update(), view().erase() + auto sle = view().peek(keylet::account(account_)); + sle->setFieldAmount(sfBalance, newBal); + view().update(sle); // REQUIRED after mutation + return tesSUCCESS; +} +``` + +### Registration Checklist +```cpp +// ALL of these are REQUIRED for a new transaction type: +// 1. transactions.macro: TRANSACTION(ttMY_TYPE, N, MyTx, delegation, fields) +// 2. applySteps.cpp: case ttMY_TYPE: return invoke(...); +// 3. features.macro: XRPL_FEATURE(MyFeature, Supported::yes, DefaultNo) +// 4. Feature.h: increment numFeatures +// 5. InvariantCheck.cpp: update if new ledger objects created +// 6. Batch.cpp: add to disabledTxTypes if not batch-compatible +``` + +## Transaction Lifecycle + +1. `preflight` (static checks, no ledger) -> `PreflightResult` +2. `preclaim` (ledger state, read-only) -> TER +3. `operator()` orchestrates: `checkSeqProxy` -> `checkPriorTxAndLastLedger` -> `checkFee` -> `checkSign` -> `apply` +4. `Transactor::apply()` runs `consumeSeqProxy` -> `payFee` -> `doApply` and returns a TER +5. `operator()` inspects the TER, decides whether to commit (`ctx_.apply`) or discard (`ctx_.discard`/`reset`) + +## State Commitment & tec* Rollback (CRITICAL for review) + +**`doApply` mutations are NOT committed until `ctx_.apply()` is called at the end of `operator()`.** All peek/insert/update/erase during `doApply` go into an `ApplyContext` view (`view_`) layered on top of `base_`. Whether that view gets flushed to `base_` depends entirely on the TER that `doApply` returns. + +`ApplyContext::discard()` ([src/libxrpl/tx/ApplyContext.cpp](src/libxrpl/tx/ApplyContext.cpp)) replaces `view_` with a fresh view on `base_` — **every doApply mutation is thrown away**: +```cpp +void ApplyContext::discard() { view_.emplace(&base_, flags_); } +``` + +### Return-code decision table (in `Transactor::operator()`, [src/libxrpl/tx/Transactor.cpp](src/libxrpl/tx/Transactor.cpp)) + +| doApply returns | What commits to the ledger | +|---|---| +| `tesSUCCESS` | All doApply mutations + fee + seq (via `ctx_.apply`) | +| `tec*` (normal, `!tapRETRY`) | `reset(fee)` calls `discard()`, then re-applies fee + seq only. **All doApply mutations reverted.** | +| `tec*` with `tapFAIL_HARD` | `discard()` called directly, nothing committed (not even fee) | +| `tec*` with `tapRETRY` | `applied=false`, `ctx_.apply` never called, tx re-queued | +| `tef*` / `tem*` / `ter*` | `applied=false`, `ctx_.apply` never called | +| `tecINVARIANT_FAILED` after invariants | reset again, commit fee only | + +`isTecClaimHardFail(ter, flags) = isTecClaim(ter) && !(flags & tapRETRY)` ([include/xrpl/tx/applySteps.h](include/xrpl/tx/applySteps.h)) — this predicate is what drives the reset path for normal consensus application. + +### What this means for transactor authors and reviewers + +- **A `tec*` return from doApply acts as a full-transaction rollback.** You do NOT need to order mutations defensively so that all checks come before any state changes. If a helper called late in doApply returns `tec*`, everything mutated earlier in the same doApply is discarded via `discard()`. +- **Orphan-state bugs of the form "we mutated X then returned tec* so X is now in an inconsistent state" are not possible at the transactor boundary.** The ApplyContext isolates the whole doApply as an atomic unit. +- **The real failure mode is within `doApply` itself**: if you call `view().update(sle)` on a stale SLE pointer, or mutate a variable you read by value instead of peek, those are real bugs — but they are in-memory bugs, not state-commit bugs. +- **Sandboxes inside `doApply` add nesting, not safety.** `PaymentSandbox` / nested `ApplyView` are useful when you need to conditionally commit a subset of changes *within* a single doApply (e.g., apply offers but revert if the net outcome fails). They are not needed to protect against doApply's own `tec*` return — that rollback is automatic. +- **Only `ctx_.apply(result)` publishes to `base_`**; a doApply that `return`s early, throws, or crashes never reaches that call, so base_ stays clean. + +### Verifying a suspected orphan-state bug + +Before claiming "directory removed but SLE not erased because tec\*": +1. Read the caller of `doApply` — confirm the TER path (`operator()` in Transactor.cpp). +2. Check whether `discard()` is reached via `reset()` or the `tapFAIL_HARD` branch. +3. If both paths call `discard()`, the mutations cannot persist on tec\*. +4. Look instead for: missing `view().update(sle)` after mutation, stale SLE pointers, or genuine non-atomic side effects (e.g., hash router flags, which are NOT in the ApplyContext view). + +## Permission System + +- `checkSign` dispatches to `checkSingleSign`, `checkMultiSign`, or `checkBatchSign` +- `checkPermission` validates delegated authority for delegatable transaction types +- Multi-sign requires M-of-N signers matching the signer list; weight threshold must be met + +## Key Files + +- `src/xrpld/app/tx/detail/Transactor.cpp` - base class and pipeline +- `include/xrpl/protocol/detail/transactions.macro` - type definitions +- `src/xrpld/app/tx/detail/` - per-type implementations (Payment.cpp, OfferCreate.cpp, etc.) +- `src/xrpld/app/tx/detail/InvariantCheck.cpp` - post-execution invariant checks diff --git a/docs/skills/soul/websockets.md b/docs/skills/soul/websockets.md new file mode 100644 index 0000000000..12023dd9ab --- /dev/null +++ b/docs/skills/soul/websockets.md @@ -0,0 +1,62 @@ +# WebSockets + +Async WebSocket support for client RPC and real-time subscriptions. Both plain and SSL. Built on Boost.Beast + Boost.Asio. + +## Key Invariants + +- All async operations run on a per-session Boost.Asio strand for thread safety +- Outgoing message queue (`wq_`) has a per-session limit (`port().ws_queue_limit`); exceeding it closes the connection with policy error "client is too slow" +- `complete()` must be called after processing each message to resume reading; forgetting it stalls the session +- `WSInfoSub` holds a weak pointer to `WSSession`; dead sessions are automatically skipped during event delivery +- Message size limit: `RPC::Tuning::maxRequestSize`; oversized messages get a `jsonInvalid` error response + +## Connection Flow + +1. `Door` accepts TCP -> `Detector` probes for SSL vs plain -> creates `SSLHTTPPeer` or `PlainHTTPPeer` +2. HTTP request with WebSocket upgrade -> `ServerHandler::onHandoff` -> `session.websocketUpgrade()` -> creates `PlainWSPeer` or `SSLWSPeer` +3. `BaseWSPeer::run()` -> set permessage-deflate options -> `async_accept` handshake -> `on_ws_handshake` -> `do_read` loop +4. `on_read` -> `ServerHandler::onWSMessage` -> validate JSON -> post to job queue -> `processSession` -> send response -> `complete()` + +## Common Bug Patterns + +- `on_read` consumes the buffer AFTER calling `onWSMessage`; if the handler accesses the buffer asynchronously after `onWSMessage` returns, it reads garbage +- `close()` with pending messages defers the actual close; calling `send()` after `close()` but before actual close queues more messages +- Missing `complete()` call after sending response = session never reads again = appears hung +- Job queue shutdown: if `postCoro` returns nullptr, session must close with `going_away`; dropping this silently leaks sessions + +## Review Checklist + +- Verify strand execution for all socket operations (read, write, close, timer) +- New subscription streams: ensure `WSInfoSub::send` handles the new event type +- Flow control: test with slow clients to verify queue limit enforcement + +## Key Patterns + +### complete() Resumes Read Loop +```cpp +// REQUIRED after sending response — missing this = session hangs forever +void BaseWSPeer::complete() +{ + if (!strand_.running_in_this_thread()) + return post(strand_, std::bind( + &BaseWSPeer::complete, impl().shared_from_this())); + do_read(); // resume reading next message +} +``` + +### Queue Limit Enforcement +```cpp +// REQUIRED: close slow clients to prevent memory exhaustion +if (wq_.size() >= port().ws_queue_limit) +{ + close(boost::beast::websocket::close_code::policy_error); + return; // do NOT queue unboundedly +} +``` + +## Key Files + +- `include/xrpl/server/detail/BaseWSPeer.h` - session lifecycle and message queue +- `include/xrpl/server/detail/Door.h` - connection acceptance +- `src/xrpld/rpc/detail/ServerHandler.cpp` - `onWSMessage` and `onHandoff` +- `src/xrpld/rpc/detail/WSInfoSub.h` - subscription delivery