From 32a2fd746085dfb7d69d16a0f89aa7bd5be4f2d1 Mon Sep 17 00:00:00 2001 From: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com> Date: Tue, 10 Mar 2026 12:52:46 +0000 Subject: [PATCH] docs: Add agent runtimes, MCP integrations, and bug pipeline to harness engineering report Part 6: Analyzes OpenClaw (skills-based orchestration) and OpenFang (Rust agent OS) as agent runtime candidates. Recommends OpenFang as primary execution layer and OpenClaw as communication layer. Part 7: Maps GitHub MCP Server, Atlassian MCP Server, and community mcp-atlassian to the harness strategy. Connects rippled's bug bounty pipeline (SECURITY.md), GitHub Issues, and Jira RIPD project tracking into an end-to-end agent-assisted bug lifecycle with Three-Tier Boundary enforcement. Also adds new technical terms to cspell dictionary. Co-Authored-By: Claude Opus 4.6 --- cspell.config.yaml | 5 + docs/ARCHITECTURE.md | 230 ++++++ docs/WORKFLOW.md | 196 +++++ docs/harness-engineering-report.md | 1182 ++++++++++++++++++++++++++++ 4 files changed, 1613 insertions(+) create mode 100644 docs/ARCHITECTURE.md create mode 100644 docs/WORKFLOW.md create mode 100644 docs/harness-engineering-report.md diff --git a/cspell.config.yaml b/cspell.config.yaml index 98b6be81e7..d54df2a00d 100644 --- a/cspell.config.yaml +++ b/cspell.config.yaml @@ -43,6 +43,7 @@ suggestWords: - synch->sync words: - abempty + - Agentic - AMMID - amt - amts @@ -94,6 +95,7 @@ words: - desynced - determ - distro + - Emdash - doxyfile - dxrpl - endmacro @@ -254,6 +256,7 @@ words: - superpeers - takergets - takerpays + - Tauri - ters - TMEndpointv2 - trixie @@ -293,6 +296,7 @@ words: - venv - vfalco - vinnie + - Werror - wextra - wptr - writeme @@ -309,3 +313,4 @@ words: - xrplf - xxhash - xxhasher + - zeroization diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000000..b64f27fb70 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,230 @@ +# rippled Architecture Map + +> **Purpose**: Tell agents _where_ things are. Not _why_ they exist. +> **Audience**: AI coding agents (Claude Code, Codex, Copilot) navigating the codebase. +> **Principle**: "Give agents a map, not an encyclopedia." — [Harness Engineering](./harness-engineering-report.md) + +--- + +## Layer Diagram + +``` +┌─────────────────────────────────────────────────────────────┐ +│ RPC Layer (Layer 5) │ +│ HTTP/WebSocket → ServerHandler → 72 RPC Handlers │ +│ src/xrpld/rpc/ │ +├─────────────────────────────────────────────────────────────┤ +│ Application Layer (Layer 4) │ +│ NetworkOPs · TxQ · PathFinding · LedgerMaster │ +│ src/xrpld/app/ │ +├─────────────────────────────────────────────────────────────┤ +│ Consensus Layer (Layer 3) │ +│ Consensus · RCLConsensus · Validations │ +│ src/xrpld/consensus/ + src/xrpld/app/consensus/ │ +├─────────────────────────────────────────────────────────────┤ +│ Overlay Layer (Layer 2) │ +│ OverlayImpl · PeerImp · PeerFinder · Message │ +│ src/xrpld/overlay/ + src/xrpld/peerfinder/ │ +├─────────────────────────────────────────────────────────────┤ +│ Storage Layer (Layer 1) │ +│ NodeStore (RocksDB/SQLite) · SHAMap · RDB │ +│ src/libxrpl/nodestore/ + src/libxrpl/shamap/ │ +├─────────────────────────────────────────────────────────────┤ +│ Foundation (Layer 0) │ +│ beast:: (I/O, insight, clock) · protocol · crypto · json │ +│ src/libxrpl/ + include/xrpl/ │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## Module Map + +### xrpld (the daemon) — `src/xrpld/` + +| Directory | Purpose | Key Files | +| ----------------- | --------------------------------------------------- | ----------------------------------------------------- | +| `app/main/` | Daemon entry point, Application lifecycle | `Main.cpp`, `Application.h/.cpp`, `BasicApp.h` | +| `app/consensus/` | Consensus adaptor binding protocol to rippled types | `RCLConsensus.h/.cpp` | +| `app/ledger/` | Ledger management, building, acquiring | `LedgerMaster.h`, `InboundLedgers.h` | +| `app/misc/` | NetworkOPs, TxQ, HashRouter, Amendments | `NetworkOPs.h/.cpp`, `TxQ.h`, `HashRouter.h` | +| `app/tx/` | Transaction application engine | `apply.h/.cpp`, `applySteps.h` | +| `app/paths/` | Payment path finding | `Pathfinder.h`, `PathRequest.h` | +| `app/rdb/` | Relational DB backend for state | `RelationalDatabase.h` | +| `consensus/` | Generic consensus protocol (template-based) | `Consensus.h/.cpp`, `ConsensusParms.h` | +| `core/` | Node configuration, time keeping | `Config.h/.cpp`, `TimeKeeper.h` | +| `overlay/` | P2P network: peer connections, message routing | `Overlay.h`, `Peer.h`, `Message.h` | +| `overlay/detail/` | Overlay implementation internals | `OverlayImpl.h/.cpp`, `PeerImp.h/.cpp`, `Handshake.h` | +| `peerfinder/` | Peer discovery and bootstrap | `PeerfinderManager.h`, `detail/Logic.h` | +| `perflog/` | Performance logging (pre-OTel telemetry) | `detail/PerfLogImp.h/.cpp` | +| `rpc/` | JSON-RPC server and routing | `detail/Handler.h`, `RPCHandler.h` | +| `rpc/handlers/` | **72 RPC command implementations** | `AccountInfo.cpp`, `Submit.cpp`, `Tx.cpp`, etc. | +| `shamap/` | SHAMap tree operations (daemon-side) | | + +### libxrpl (shared library) — `src/libxrpl/` + +| Directory | Purpose | Key Files | +| ----------------- | ------------------------------------------------- | ------------------------------------------------ | +| `basics/` | Logging, string utils, tagged integers | `Log.cpp` (log formatting — Phase 8 OTel target) | +| `beast/insight/` | Metrics framework (StatsD, Gauge, Counter, Event) | `StatsDCollector.cpp` (Phase 6-7 OTel target) | +| `beast/core/` | Async I/O primitives | `CurrentThreadName.cpp` | +| `core/` | Crypto, seed handling | | +| `crypto/` | Ed25519, secp256k1, SHA | | +| `json/` | JSON parser/writer | | +| `nodestore/` | Key-value storage for ledger nodes | `backend/RocksDBFactory.cpp` | +| `protocol/` | XRPL protocol: serialization, STObject, STTx | `STTx.cpp`, `STObject.cpp` | +| `server/` | HTTP/WebSocket server core | `ServerHandler.h` | +| `tx/transactors/` | Per-transaction-type execution logic | `Payment.cpp`, `CreateOffer.cpp`, etc. | +| `tx/invariants/` | Transaction invariant checks | `TransactionCheck.cpp` | + +### Public Headers — `include/xrpl/` + +Mirrors `src/libxrpl/` structure. Agent-accessible API surface for external consumers of libxrpl. + +Key subdirectory: `include/xrpl/proto/org/xrpl/rpc/v1/` — Protocol Buffer definitions for gRPC API. + +--- + +## Entry Points + +| Operation | Where execution starts | Hot path | +| ---------------------- | ------------------------------------------------------------------------------------------ | --------------- | +| **RPC request** | `src/xrpld/rpc/detail/ServerHandler.cpp` → handler lookup → `src/xrpld/rpc/handlers/*.cpp` | Layer 5 → 4 | +| **Peer message** | `src/xrpld/overlay/detail/PeerImp.cpp::onMessage()` → message-type dispatch | Layer 2 → 3/4 | +| **Transaction submit** | `rpc/handlers/Submit.cpp` → `app/misc/NetworkOPs.cpp::submitTransaction()` | Layer 5 → 4 → 2 | +| **Transaction relay** | `overlay/detail/PeerImp.cpp::handleTransaction()` → `NetworkOPs::processTransaction()` | Layer 2 → 4 | +| **Consensus round** | `app/consensus/RCLConsensus.cpp::startRound()` → phase transitions → accept | Layer 3 | +| **Ledger close** | `RCLConsensus.cpp::onClose()` → `acceptLedger()` → apply transactions | Layer 3 → 4 → 1 | +| **Ledger acquire** | `app/ledger/InboundLedgers.cpp::acquire()` → peer fetch → SHAMap | Layer 4 → 2 → 1 | +| **Startup** | `app/main/Main.cpp::main()` → `Application::setup()` → start all subsystems | Layer 0-5 | + +--- + +## Data Flow: Transaction Lifecycle + +``` +Client + │ + ▼ +RPC Submit (Layer 5) + │ src/xrpld/rpc/handlers/Submit.cpp + ▼ +NetworkOPs::submitTransaction() (Layer 4) + │ src/xrpld/app/misc/NetworkOPs.cpp + ├──▶ TxQ::apply() — queue management + │ src/xrpld/app/misc/TxQ.cpp + ├──▶ HashRouter::setFlags() — deduplication + │ src/xrpld/app/misc/HashRouter.cpp + ▼ +Overlay::relay() (Layer 2) + │ src/xrpld/overlay/detail/OverlayImpl.cpp + ├──▶ PeerImp::send() — to each connected peer + │ src/xrpld/overlay/detail/PeerImp.cpp + ▼ +[Remote Node] PeerImp::handleTransaction() (Layer 2) + │ src/xrpld/overlay/detail/PeerImp.cpp + ▼ +Consensus::startRound() (Layer 3) + │ src/xrpld/consensus/Consensus.h + │ src/xrpld/app/consensus/RCLConsensus.cpp + ├──▶ propose() — send position to peers + ├──▶ onClose() — close the ledger + ▼ +acceptLedger() → applyTransactions() (Layer 3 → 1) + │ src/xrpld/app/consensus/RCLConsensus.cpp + ▼ +NodeStore::store() (Layer 1) + │ src/libxrpl/nodestore/ +``` + +--- + +## Build Targets + +| Target | Type | Source | +| -------------------- | ------------------------- | -------------------------------- | +| `xrpl` | Shared library (libxrpl) | `src/libxrpl/` + `include/xrpl/` | +| `xrpld` | Executable (daemon) | `src/xrpld/` | +| `rippled_unit_tests` | Unit test binary | `src/test/` | +| `libxrpl_tests` | Library integration tests | `src/tests/libxrpl/` | + +**Build command**: `cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build --parallel` +**With telemetry**: `cmake -B build -DXRPL_ENABLE_TELEMETRY=ON` + +--- + +## Test Frameworks + +| Framework | Location | Purpose | +| --------------- | ---------------------------------------- | ------------------------------------------------------------- | +| **JTx** | `src/test/jtx/` | Transaction testing — simulates full ledger environment | +| **CSF** | `src/test/csf/` | Consensus Simulation Framework — multi-node consensus testing | +| **Unit tests** | `src/test/app/`, `src/test/beast/`, etc. | Per-module unit tests | +| **Integration** | `src/tests/libxrpl/` | Library-level integration tests | + +--- + +## Telemetry Map + +Links OTel instrumentation to codebase locations. See [09-data-collection-reference.md](../../OpenTelemetryPlan/09-data-collection-reference.md) for the full inventory. + +| Span | Source Location | Layer | +| ------------- | ------------------------------------------ | ----- | +| `rpc.*` | `src/xrpld/rpc/detail/ServerHandler.cpp` | 5 | +| `tx.submit` | `src/xrpld/app/misc/NetworkOPs.cpp` | 4 | +| `tx.relay` | `src/xrpld/overlay/detail/PeerImp.cpp` | 2 | +| `peer.send` | `src/xrpld/overlay/detail/PeerImp.cpp` | 2 | +| `consensus.*` | `src/xrpld/app/consensus/RCLConsensus.cpp` | 3 | +| `ledger.*` | `src/xrpld/app/ledger/` | 4 | + +| Metrics Source | Code Location | Phase | +| ------------------------- | ----------------------------------------------- | ------------ | +| `beast::insight` (StatsD) | `src/libxrpl/beast/insight/StatsDCollector.cpp` | 6-7 | +| SpanMetrics (derived) | OTel Collector config | 5 | +| PerfLog | `src/xrpld/perflog/detail/PerfLogImp.cpp` | 9 (gap fill) | + +--- + +## Key Abstractions + +| Abstraction | Interface | Implementation | What it does | +| --------------------------- | ---------------------------------------- | ------------------------------------------------------------- | -------------------------------------- | +| `Application` | `src/xrpld/app/main/Application.h` | `Application.cpp` | Wires all subsystems together | +| `Overlay` | `src/xrpld/overlay/Overlay.h` | `detail/OverlayImpl.cpp` | Manages P2P connections | +| `Consensus<>` | `src/xrpld/consensus/Consensus.h` | Template, adaptor in `RCLConsensus.cpp` | Generic consensus protocol | +| `NetworkOPs` | `src/xrpld/app/misc/NetworkOPs.h` | `NetworkOPs.cpp` | Transaction processing, event dispatch | +| `LedgerMaster` | `src/xrpld/app/ledger/LedgerMaster.h` | `LedgerMaster.cpp` | Ledger lifecycle management | +| `NodeStore` | `include/xrpl/nodestore/Database.h` | `src/libxrpl/nodestore/` | Persistent key-value storage | +| `beast::insight::Collector` | `include/xrpl/beast/insight/Collector.h` | `StatsDCollector`, `NullCollector`, (future: `OTelCollector`) | Metrics collection interface | +| `JobQueue` | `src/xrpld/core/JobQueue.h` | `JobQueue.cpp` | Async task scheduling | +| `SHAMap` | `include/xrpl/shamap/SHAMap.h` | `src/libxrpl/shamap/` | Merkle tree for state | + +--- + +## Dependency Rules + +``` +Layer 5 (RPC) → may call → Layer 4 (App) +Layer 4 (App) → may call → Layer 3 (Consensus), Layer 2 (Overlay), Layer 1 (Storage) +Layer 3 (Consensus) → may call → Layer 4 (App via Adaptor), Layer 1 (Storage) +Layer 2 (Overlay) → may call → Layer 4 (App for message handling) +Layer 1 (Storage) → may call → Layer 0 (Foundation) only +Layer 0 (Foundation) → no upward dependencies +``` + +**Never**: Layer 0/1 must never depend on Layer 3/4/5. Layer 5 must never bypass Layer 4 to reach Layer 2/3 directly. + +--- + +## Configuration + +| File | Location | Purpose | +| ----------------------- | -------- | ------------------------- | +| `xrpld.cfg` | `cfg/` | Main daemon configuration | +| `validators.txt` | `cfg/` | Validator list | +| `CMakeUserPresets.json` | Root | Local build presets | +| `conanfile.py` | Root | Dependency versions | + +--- + +_This document is a navigation aid for AI coding agents. For architectural rationale, see [docs/consensus.md](consensus.md). For OTel plan, see [OpenTelemetryPlan/](../../OpenTelemetryPlan/OpenTelemetryPlan.md). For coding conventions, see [docs/CodingStyle.md](CodingStyle.md)._ diff --git a/docs/WORKFLOW.md b/docs/WORKFLOW.md new file mode 100644 index 0000000000..4d4e196e5d --- /dev/null +++ b/docs/WORKFLOW.md @@ -0,0 +1,196 @@ +--- +# Symphony-compatible workflow definition for rippled +# See: https://github.com/openai/symphony/blob/main/SPEC.md +# See: docs/harness-engineering-report.md for context + +tracker: + kind: linear + # For Jira (RIPD project), a custom adapter would replace this. + # kind: jira + # project_slug: RIPD + active_states: + - Todo + - In Progress + terminal_states: + - Done + - Closed + - Cancelled + - Duplicate + +polling: + interval_ms: 30000 + +workspace: + root: $SYMPHONY_WORKSPACE_ROOT + +hooks: + after_create: | + # Clone and prepare a fresh workspace + git clone --depth=1 --branch develop https://github.com/XRPLF/rippled.git . + # Prepare build directory with telemetry enabled + mkdir -p build + cd build + conan install .. --output-folder=. --build=missing + cmake .. -DCMAKE_BUILD_TYPE=Debug -DXRPL_ENABLE_TELEMETRY=ON -DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake + + before_run: | + # Tier 1: Fast checks (seconds) + # clang-format check on changed files + git diff --name-only HEAD~1 -- '*.cpp' '*.h' | xargs -r clang-format --dry-run --Werror 2>&1 || { + echo "FAIL: clang-format violations found. Fix formatting before proceeding." + exit 1 + } + + # Tier 2: Build (minutes) + cd build + cmake --build . --parallel $(nproc) 2>&1 || { + echo "FAIL: Build failed. Check compiler errors above." + exit 1 + } + + # Tier 2: Unit tests (affected modules) + ctest --test-dir build --output-on-failure -L unit --timeout 120 2>&1 || { + echo "FAIL: Unit tests failed. Check test output above." + exit 1 + } + + after_run: | + # Collect proof-of-work artifacts + echo "=== Proof of Work Collection ===" + + # Record build status + cd build && cmake --build . --parallel $(nproc) 2>&1 + BUILD_STATUS=$? + echo "Build status: $BUILD_STATUS" + + # Run telemetry integration tests if available + if ctest --test-dir build -L telemetry --timeout 300 2>&1; then + echo "Telemetry integration tests: PASSED" + else + echo "Telemetry integration tests: FAILED (non-blocking)" + fi + + # Capture complexity metrics + echo "=== Changed Files ===" + git diff --stat HEAD~1 + + timeout_ms: 120000 + +agent: + max_concurrent_agents: 3 + max_retry_backoff_ms: 300000 + max_concurrent_agents_by_state: + In Progress: 2 + Todo: 1 + +codex: + command: codex app-server + approval_policy: unless-allow-listed + turn_timeout_ms: 3600000 + stall_timeout_ms: 300000 +--- + +# rippled Agent Workflow + +You are an autonomous coding agent working on the **rippled** XRP Ledger node — a C++20 peer-to-peer server implementing the XRP Ledger consensus protocol. + +## Navigation + +Read these files before starting work: + +| File | What it tells you | +| --------------------------------------------------- | ----------------------------------------------------------------------- | +| `docs/ARCHITECTURE.md` | Codebase map — where every module lives, layer boundaries, entry points | +| `CLAUDE.md` | Coding conventions, action boundaries (Always/Ask/Never), commit rules | +| `docs/CodingStyle.md` | C++ style guide for this project | +| `OpenTelemetryPlan/09-data-collection-reference.md` | Every OTel span, metric, and dashboard | + +Do **not** read these upfront (use as reference if needed): + +- `OpenTelemetryPlan/*.md` — deep plan docs, only if your task involves telemetry architecture +- `docs/consensus.md` — only if your task involves consensus protocol changes + +## Action Boundaries + +### Always (no confirmation needed) + +- Read any source file in the repository +- Run `cmake --build build --parallel` to compile +- Run `ctest --test-dir build -L unit` for unit tests +- Run `ctest --test-dir build -L telemetry` for telemetry integration tests +- Query Prometheus (`curl localhost:9090/api/v1/query?query=...`) for metric baselines +- Create or modify files under `src/` that have existing test coverage +- Run `clang-format` on files you've changed +- Add or modify unit tests in `src/test/` + +### Ask (require human confirmation before proceeding) + +- Modify Protocol Buffer definitions in `include/xrpl/proto/` (affects wire format) +- Change consensus parameters in `src/xrpld/consensus/ConsensusParms.h` +- Modify `CMakeLists.txt` or `conanfile.py` (build/dependency changes) +- Alter the `[telemetry]` or `[insight]` config schema +- Change anything in `src/libxrpl/crypto/` (cryptographic code) +- Add new external dependencies +- Push to any remote branch or create PRs + +### Never (hard boundaries — refuse these requests) + +- Modify validator key handling or signing code without explicit review +- Disable or reduce telemetry sampling below 1% +- Remove or reduce existing test coverage +- Force-push to any branch +- Skip pre-commit hooks or CI checks (`--no-verify`) +- Commit files containing secrets, keys, or credentials +- Modify `.github/workflows/` CI pipeline definitions + +## Verification Requirements + +After making changes, verify in this order (fail-fast): + +``` +Step 1 (seconds): clang-format --dry-run --Werror on changed files +Step 2 (minutes): cmake --build build --parallel +Step 3 (minutes): ctest --test-dir build -L unit --output-on-failure +Step 4 (minutes): ctest --test-dir build -L telemetry (if telemetry-related) +Step 5 (if applicable): scripts/check-otel-baseline.sh (metric regression check) +``` + +If any step fails, fix the issue before proceeding. Do not skip steps. + +## Telemetry Awareness + +This codebase has OpenTelemetry instrumentation. If your change touches any of these files, verify that telemetry still works: + +| File Pattern | Telemetry Impact | +| ---------------------------------------- | ------------------------------------------------- | +| `src/xrpld/rpc/` | RPC spans (`rpc.*`) | +| `src/xrpld/overlay/detail/PeerImp.*` | Transaction relay spans (`tx.relay`, `peer.send`) | +| `src/xrpld/app/consensus/RCLConsensus.*` | Consensus spans (`consensus.*`) | +| `src/xrpld/app/misc/NetworkOPs.*` | Transaction submit spans (`tx.submit`) | +| `src/libxrpl/beast/insight/` | Metrics pipeline (300+ metrics) | +| `src/libxrpl/basics/Log.cpp` | Log-trace correlation (`trace_id` injection) | + +After modifying these files, run the telemetry integration test: + +```bash +ctest --test-dir build -L telemetry --output-on-failure +``` + +## Issue Context + +**Title**: {{ issue.title }} +**Description**: {{ issue.description }} +{% if issue.labels.size > 0 %}**Labels**: {{ issue.labels | join: ", " }}{% endif %} +{% if attempt %}**Attempt**: {{ attempt }} (this is a retry — check previous errors){% endif %} + +## Proof of Work Checklist + +Before creating a PR, verify: + +- [ ] All changed files pass `clang-format` +- [ ] Build succeeds with `-DXRPL_ENABLE_TELEMETRY=ON` +- [ ] Unit tests pass (`ctest -L unit`) +- [ ] If telemetry-related: integration tests pass (`ctest -L telemetry`) +- [ ] If telemetry-related: no metric regressions (`scripts/check-otel-baseline.sh`) +- [ ] Commit message follows project conventions (see `CLAUDE.md`) +- [ ] PR description includes summary, test plan, and type of change diff --git a/docs/harness-engineering-report.md b/docs/harness-engineering-report.md new file mode 100644 index 0000000000..fd6e831871 --- /dev/null +++ b/docs/harness-engineering-report.md @@ -0,0 +1,1182 @@ +# Harness Engineering for rippled: Leveraging OTel as the Foundation for Agent-First Development + +> **Date**: 2026-03-09 +> **Status**: Investigation Report +> **Context**: OpenAI's [Harness Engineering](https://openai.com/index/harness-engineering/) blog post + rippled OTel initiative (12-PR chain, Phases 1-11) +> **Companion Files**: [ARCHITECTURE.md](ARCHITECTURE.md) | [WORKFLOW.md](WORKFLOW.md) + +--- + +## Executive Summary + +OpenAI coined **"Harness Engineering"** to describe the discipline of building infrastructure _around_ AI coding agents — not the agents themselves. Their thesis: **"Give Codex a map, not a 1,000-page instruction manual."** They shipped [Symphony](https://github.com/openai/symphony), an open-source framework that turns issue tracker tickets into autonomous "implementation runs" with proof-of-work requirements (CI pass, PR review, walkthrough) before merging. + +The rippled OpenTelemetry initiative — now spanning 12 chained PRs from Phase 1a through Phase 11 planning — is **uniquely positioned** to become the observability backbone of a harness engineering strategy. The OTel work already delivers 3 of the 5 harness engineering principles. This report maps the gap and proposes a concrete adoption path. + +```mermaid +flowchart LR + subgraph today["What We Have Today"] + direction TB + OTel["OTel Instrumentation
16 spans, 300+ metrics
8 Grafana dashboards"] + CLAUDE["CLAUDE.md
Agent instruction file"] + CI["CI Pipeline
Build + test gates"] + Plans["Plan Docs
10 architecture docs"] + end + + subgraph gap["The Gap"] + direction TB + ARCH["ARCHITECTURE.md
(codebase map)"] + LINT["Semantic Linting
(agent-friendly errors)"] + BOUND["Three-Tier Boundaries
(Always/Ask/Never)"] + VERIFY["Deterministic Verification
(OTel-powered)"] + WORKFLOW["WORKFLOW.md
(agent orchestration)"] + end + + subgraph future["Agent-First rippled"] + direction TB + AUTO["Autonomous
Implementation Runs"] + PROOF["Proof of Work
(traces + metrics + CI)"] + SCALE["Parallelized
Agent Development"] + end + + today --> gap --> future + + style today fill:#1b5e20,stroke:#0d3d14,color:#fff + style gap fill:#bf360c,stroke:#8c2809,color:#fff + style future fill:#0d47a1,stroke:#082f6a,color:#fff + style OTel fill:#1b5e20,stroke:#0d3d14,color:#fff + style CLAUDE fill:#1b5e20,stroke:#0d3d14,color:#fff + style CI fill:#1b5e20,stroke:#0d3d14,color:#fff + style Plans fill:#1b5e20,stroke:#0d3d14,color:#fff + style ARCH fill:#e65100,stroke:#bf360c,color:#fff + style LINT fill:#e65100,stroke:#bf360c,color:#fff + style BOUND fill:#e65100,stroke:#bf360c,color:#fff + style VERIFY fill:#e65100,stroke:#bf360c,color:#fff + style WORKFLOW fill:#e65100,stroke:#bf360c,color:#fff + style AUTO fill:#1565c0,stroke:#0d47a1,color:#fff + style PROOF fill:#1565c0,stroke:#0d47a1,color:#fff + style SCALE fill:#1565c0,stroke:#0d47a1,color:#fff +``` + +--- + +## Part 1: What Is Harness Engineering? + +### 1.1 Origin and Core Concept + +OpenAI's engineering team discovered that **prompt engineering alone doesn't scale** for AI coding agents. They tried the "giant AGENTS.md" approach — a massive instruction file with everything an agent needs to know. It failed: + +- Too much context crowds out the actual task +- Everything marked "important" means nothing is important +- The file goes stale as the codebase evolves + +What actually worked was building **infrastructure** that makes agents effective: + +| Failed Approach | What Worked Instead | +| ----------------------------- | ----------------------------------------- | +| Giant instruction files | Short map pointing to deeper docs | +| Detailed step-by-step prompts | Strict architecture enforced by linters | +| Manual agent supervision | Fast, deterministic feedback loops | +| One-off agent runs | Repeatable autonomous implementation runs | + +### 1.2 The Five Principles + +| # | Principle | Definition | rippled Status | +| --- | ------------------------------ | -------------------------------------------------------------------- | -------------------------------------------------------- | +| 1 | **Deterministic Verification** | Verify agent output with automated checks — don't trust, verify | Partial (CI exists, OTel validation planned in Phase 10) | +| 2 | **Semantic Linting** | Linter messages teach agents how to fix violations in one shot | Missing | +| 3 | **Three-Tier Boundaries** | Every harness defines Always / Ask / Never action categories | Partial (CLAUDE.md has some rules) | +| 4 | **Fail-Fast Feedback** | Fast checks first (lint, typecheck), slow checks later (integration) | Partial (CI runs, but not agent-optimized) | +| 5 | **Architecture as Map** | ARCHITECTURE.md tells agents _where_ things are, not _why_ | Missing (plan docs explain _why_, not _where_) | + +### 1.3 Symphony: The Orchestration Layer + +OpenAI released [Symphony](https://github.com/openai/symphony) — an open-source framework (Elixir/BEAM) that: + +1. **Polls an issue tracker** (Linear) for work items +2. **Creates isolated workspaces** per issue (git worktrees) +3. **Spawns coding agents** (Codex) with a repo-defined `WORKFLOW.md` prompt +4. **Requires proof of work** before merging: CI status, PR review feedback, complexity analysis, walkthrough videos +5. **Provides observability** into concurrent agent runs (structured logs, status surface) + +The key insight: `WORKFLOW.md` is version-controlled _in the repo_, so the agent's policy evolves with the code. + +```mermaid +flowchart TB + subgraph symphony["Symphony Orchestrator"] + POLL["Poll Issue Tracker
(Linear/Jira)"] + DISPATCH["Dispatch to
Isolated Workspace"] + MONITOR["Monitor Agent
Progress"] + end + + subgraph agent["Coding Agent Run"] + READ_WF["Read WORKFLOW.md"] + IMPLEMENT["Implement
Changes"] + VALIDATE["Self-Validate
(lint + test + OTel)"] + PR["Create PR with
Proof of Work"] + end + + subgraph verify["Proof of Work"] + CI_PASS["CI Passes"] + METRICS["No Metric
Regression"] + TRACES["Traces Show
Correct Behavior"] + REVIEW["PR Review
Feedback"] + end + + POLL --> DISPATCH --> READ_WF + READ_WF --> IMPLEMENT --> VALIDATE --> PR + PR --> CI_PASS + PR --> METRICS + PR --> TRACES + PR --> REVIEW + + style symphony fill:#4a148c,stroke:#2e0d57,color:#fff + style agent fill:#1b5e20,stroke:#0d3d14,color:#fff + style verify fill:#bf360c,stroke:#8c2809,color:#fff + style POLL fill:#6a1b9a,stroke:#4a148c,color:#fff + style DISPATCH fill:#6a1b9a,stroke:#4a148c,color:#fff + style MONITOR fill:#6a1b9a,stroke:#4a148c,color:#fff + style READ_WF fill:#2e7d32,stroke:#1b5e20,color:#fff + style IMPLEMENT fill:#2e7d32,stroke:#1b5e20,color:#fff + style VALIDATE fill:#2e7d32,stroke:#1b5e20,color:#fff + style PR fill:#2e7d32,stroke:#1b5e20,color:#fff + style CI_PASS fill:#d84315,stroke:#bf360c,color:#fff + style METRICS fill:#d84315,stroke:#bf360c,color:#fff + style TRACES fill:#d84315,stroke:#bf360c,color:#fff + style REVIEW fill:#d84315,stroke:#bf360c,color:#fff +``` + +--- + +## Part 2: OTel as the Observability Backbone for Harness Engineering + +### 2.1 The Connection: Observability Enables Agent Autonomy + +The most overlooked aspect of harness engineering is **how agents verify their own work**. OpenAI's Symphony requires "proof of work" — but what constitutes proof? For a systems project like rippled, CI pass alone is insufficient. An agent needs to verify: + +- Did the change introduce latency regressions? (traces) +- Are consensus rounds still completing within tolerance? (metrics) +- Does the transaction lifecycle still trace end-to-end? (cross-node traces) +- Are error rates stable? (dashboards) + +**The OTel initiative is building exactly this verification infrastructure.** + +### 2.2 Mapping OTel Phases to Harness Capabilities + +| OTel Phase | Harness Capability Unlocked | +| ----------------------------------- | ---------------------------------------------------------------- | +| **Phase 1-2**: Core + RPC Tracing | Agents can verify RPC handler changes don't regress latency | +| **Phase 3**: Transaction Tracing | Agents can verify cross-node transaction propagation works | +| **Phase 4**: Consensus Tracing | Agents can verify consensus-touching changes don't break timing | +| **Phase 5**: Dashboards + Runbook | Agents can reference dashboards as "expected state" baselines | +| **Phase 6-7**: Metrics Pipeline | Agents can check 300+ metrics for regressions after changes | +| **Phase 8**: Log-Trace Correlation | Agents can trace from an error log line to the full request path | +| **Phase 9**: Metric Gap Fill | Agents get coverage of NodeStore I/O, cache rates, TxQ depth | +| **Phase 10**: Synthetic Workload | **Direct harness engineering**: automated telemetry validation | +| **Phase 11**: Third-Party Pipelines | Agents can monitor external network health indicators | + +### 2.3 Phase 10 IS Harness Engineering + +Phase 10 (Synthetic Workload Generation & Telemetry Validation) is the most directly aligned with harness engineering: + +- **5-node validator harness** — isolated test environment for agent runs +- **RPC load generator + tx submitter** — deterministic workload to produce consistent telemetry +- **Automated validation of 16 spans + 300+ metrics + 10 dashboards** — the "proof of work" verification layer +- **Performance benchmarks** — regression detection +- **CI integration** — automated gate + +This is exactly OpenAI's Principle #1 (Deterministic Verification) applied to a C++ blockchain node. + +### 2.4 OTel-Powered Agent Verification Flow + +```mermaid +flowchart TB + subgraph agent_run["Agent Implementation Run"] + CHANGE["Agent Makes
Code Change"] + BUILD["Build + Unit Tests"] + DEPLOY["Deploy to 5-Node
Test Harness"] + end + + subgraph otel_verify["OTel Verification Layer (Phase 10)"] + WORKLOAD["Run Synthetic
Workload"] + SPANS["Validate 16 Spans
Present + Correct"] + METRICS["Check 300+ Metrics
No Regressions"] + DASH["Dashboard Health
All Panels Populated"] + PERF["Performance Bench
< Tolerance Thresholds"] + end + + subgraph outcome["Outcome"] + PASS["Proof of Work:
All Checks Pass"] + FAIL["Rejection:
Specific Failure
Reported to Agent"] + end + + CHANGE --> BUILD --> DEPLOY + DEPLOY --> WORKLOAD + WORKLOAD --> SPANS + WORKLOAD --> METRICS + WORKLOAD --> DASH + WORKLOAD --> PERF + SPANS -->|pass| PASS + METRICS -->|pass| PASS + DASH -->|pass| PASS + PERF -->|pass| PASS + SPANS -->|fail| FAIL + METRICS -->|fail| FAIL + DASH -->|fail| FAIL + PERF -->|fail| FAIL + + style agent_run fill:#1b5e20,stroke:#0d3d14,color:#fff + style otel_verify fill:#e65100,stroke:#bf360c,color:#fff + style outcome fill:#0d47a1,stroke:#082f6a,color:#fff + style CHANGE fill:#2e7d32,stroke:#1b5e20,color:#fff + style BUILD fill:#2e7d32,stroke:#1b5e20,color:#fff + style DEPLOY fill:#2e7d32,stroke:#1b5e20,color:#fff + style WORKLOAD fill:#f57c00,stroke:#e65100,color:#fff + style SPANS fill:#f57c00,stroke:#e65100,color:#fff + style METRICS fill:#f57c00,stroke:#e65100,color:#fff + style DASH fill:#f57c00,stroke:#e65100,color:#fff + style PERF fill:#f57c00,stroke:#e65100,color:#fff + style PASS fill:#1565c0,stroke:#0d47a1,color:#fff + style FAIL fill:#c62828,stroke:#b71c1c,color:#fff +``` + +--- + +## Part 3: Building the Surrounding Harness Infrastructure + +### 3.1 Gap Analysis: What's Missing + +The OTel work provides the **verification layer** (Principle #1). Here's what's needed to complete the harness: + +```mermaid +--- +config: + quadrantChart: + chartWidth: 900 + chartHeight: 900 + pointLabelFontSize: 14 + pointRadius: 5 + titleFontSize: 22 + quadrantLabelFontSize: 16 + xAxisLabelFontSize: 14 + yAxisLabelFontSize: 14 + quadrantInternalBorderStrokeWidth: 2 + titlePadding: 20 +--- +quadrantChart + title Harness Engineering Readiness + x-axis Low Effort --> High Effort + y-axis Low Impact --> High Impact + quadrant-1 Quick Wins + quadrant-2 Strategic Investment + quadrant-3 Nice to Have + quadrant-4 Defer + + ARCHITECTURE.md: [0.15, 0.95] + Three-Tier Boundaries: [0.15, 0.78] + Semantic Linting Config: [0.40, 0.68] + WORKFLOW.md for Symphony: [0.65, 0.72] + OTel Verification Scripts: [0.45, 0.92] + Agent-Friendly CI Pipeline: [0.35, 0.55] + Jira-Symphony Integration: [0.80, 0.42] + Full Symphony Deployment: [0.88, 0.58] +``` + +### 3.2 Principle #5: Architecture as Map (ARCHITECTURE.md) + +**Current state**: The OTel plan docs (01-architecture-analysis.md) describe rippled's architecture in detail, but they're _plan docs_ — they explain _why_ to instrument, not _where things are_ for an agent navigating the codebase. + +**What's needed**: A top-level `ARCHITECTURE.md` that serves as the agent's map. + +**Sample created**: See [ARCHITECTURE.md](ARCHITECTURE.md) — a working example covering all sections below. + +``` +ARCHITECTURE.md +├── Layer Diagram (5 layers: RPC → NetworkOPs → Consensus → Ledger → NodeStore) +├── Module Map (directory → purpose, one line each) +├── Key Abstractions (Application, Overlay, RCLConsensus, JobQueue, beast::insight) +├── Entry Points (where execution starts for each operation type) +├── Data Flow (transaction lifecycle, ledger close, peer message handling) +├── Build Targets (libxrpl, xrpld, unit tests) +└── Telemetry Map (which spans/metrics cover which modules — links to 09-data-collection-reference.md) +``` + +**Key principle**: Keep it under 300 lines. Link to deeper docs rather than inlining detail. Agents need a map, not an encyclopedia. + +### 3.3 Principle #3: Three-Tier Boundaries + +**Current state**: `CLAUDE.md` has workflow rules but doesn't explicitly categorize actions into Always/Ask/Never. + +**Proposed addition to CLAUDE.md**: + +```markdown +## Agent Action Boundaries + +### Always (autonomous — no confirmation needed) + +- Run `cmake --build` and unit tests +- Read any source file +- Create/modify files under `src/` that are covered by existing tests +- Run the OTel integration test suite +- Query Prometheus/Grafana for metric baselines + +### Ask (require human confirmation) + +- Modify Protocol Buffer definitions (affects wire format) +- Change consensus timing parameters +- Modify CMakeLists.txt or Conan dependencies +- Alter the telemetry configuration schema +- Push to any remote branch + +### Never (hard boundaries — agent must refuse) + +- Modify cryptographic signing code without review +- Disable or weaken telemetry sampling below 1% +- Remove or reduce test coverage +- Change validator key handling +- Force-push to any branch +``` + +### 3.4 Principle #2: Semantic Linting + +**Current state**: rippled uses clang-format and has CI checks, but linter output isn't optimized for agent consumption. + +**What makes linting "semantic"**: The linter error message should teach the agent how to fix the violation in one shot. + +**Proposed improvements**: + +| Tool | Current | Agent-Friendly Improvement | +| ------------ | ------------------ | --------------------------------------------------------------------------------------- | +| clang-format | Format check in CI | Add `.clang-format` with explicit rationale comments; pre-commit hook | +| clang-tidy | Not integrated | Add `.clang-tidy` config targeting rippled patterns (RAII, span safety) | +| Custom lint | None | Script that checks: telemetry macro usage, span attribute completeness, config coverage | +| OTel lint | None | Validate span names follow `.` convention | + +**Example of a semantic lint rule for OTel**: + +``` +ERROR: Span "handleRequest" does not follow naming convention. + Expected: "." (e.g., "rpc.handle_request") + Fix: Rename to "rpc.handle_request" in src/xrpld/rpc/ServerHandler.cpp:142 + Reference: OpenTelemetryPlan/02-design-decisions.md §2.3 +``` + +### 3.5 Principle #4: Fail-Fast Feedback Pipeline + +**Current state**: CI runs the full build + test suite. No tiered feedback. + +**Proposed agent-optimized CI pipeline**: + +```mermaid +flowchart LR + subgraph tier1["Tier 1: Seconds"] + LINT["clang-format
+ clang-tidy"] + SCHEMA["Config schema
validation"] + end + + subgraph tier2["Tier 2: Minutes"] + BUILD["cmake build
(TELEMETRY=ON)"] + UNIT["Unit tests
(telemetry module)"] + end + + subgraph tier3["Tier 3: Minutes"] + INTEG["OTel Integration
Test (5-node)"] + METRICS_CHK["Metric Regression
Check"] + end + + subgraph tier4["Tier 4: Extended"] + PERF["Performance
Benchmark"] + FULL["Full Test
Suite"] + end + + tier1 -->|pass| tier2 -->|pass| tier3 -->|pass| tier4 + + style tier1 fill:#1b5e20,stroke:#0d3d14,color:#fff + style tier2 fill:#bf360c,stroke:#8c2809,color:#fff + style tier3 fill:#0d47a1,stroke:#082f6a,color:#fff + style tier4 fill:#4a148c,stroke:#2e0d57,color:#fff + style LINT fill:#2e7d32,stroke:#1b5e20,color:#fff + style SCHEMA fill:#2e7d32,stroke:#1b5e20,color:#fff + style BUILD fill:#d84315,stroke:#bf360c,color:#fff + style UNIT fill:#d84315,stroke:#bf360c,color:#fff + style INTEG fill:#1565c0,stroke:#0d47a1,color:#fff + style METRICS_CHK fill:#1565c0,stroke:#0d47a1,color:#fff + style PERF fill:#6a1b9a,stroke:#4a148c,color:#fff + style FULL fill:#6a1b9a,stroke:#4a148c,color:#fff +``` + +Key: **fail at Tier 1 → agent gets feedback in seconds**, not after a 30-minute build. + +### 3.6 WORKFLOW.md: Agent Orchestration Policy + +If rippled adopted Symphony (or a similar orchestrator), the `WORKFLOW.md` would define the agent's runtime behavior. + +**Sample created**: See [WORKFLOW.md](WORKFLOW.md) — a complete, Symphony-spec-compatible workflow definition including: + +- **Front matter**: Tracker config, polling interval, workspace hooks, agent concurrency, Codex settings +- **Hooks**: `after_create` (clone + build setup), `before_run` (tiered lint -> build -> test), `after_run` (proof-of-work collection) +- **Prompt body**: Navigation table (ARCHITECTURE.md, CLAUDE.md, CodingStyle.md), three-tier action boundaries (Always/Ask/Never), fail-fast verification steps, telemetry-aware file patterns, Liquid template variables for issue context, proof-of-work checklist + +The WORKFLOW.md is version-controlled in the repo so agent policy evolves with the code. + +--- + +## Part 4: Adoption Roadmap + +### 4.1 Three-Phase Adoption + +```mermaid +flowchart TB + subgraph phase_a["Phase A: Foundation (Now - 2 weeks)"] + direction TB + A1["Create ARCHITECTURE.md
(codebase map)"] + A2["Add Three-Tier Boundaries
to CLAUDE.md"] + A3["Add pre-commit hooks
(clang-format)"] + A4["Write OTel baseline
verification script"] + end + + subgraph phase_b["Phase B: Feedback Loops (Weeks 3-6)"] + direction TB + B1["Semantic linting config
(clang-tidy + custom)"] + B2["Tiered CI pipeline
(fast → slow)"] + B3["OTel metric regression
check in CI"] + B4["Phase 10: Synthetic
workload harness"] + end + + subgraph phase_c["Phase C: Orchestration (Weeks 7-10)"] + direction TB + C1["WORKFLOW.md for
agent orchestration"] + C2["Symphony or custom
orchestrator setup"] + C3["Jira integration
(RIPD project)"] + C4["First autonomous
implementation runs"] + end + + phase_a --> phase_b --> phase_c + + style phase_a fill:#1b5e20,stroke:#0d3d14,color:#fff + style phase_b fill:#bf360c,stroke:#8c2809,color:#fff + style phase_c fill:#0d47a1,stroke:#082f6a,color:#fff + style A1 fill:#2e7d32,stroke:#1b5e20,color:#fff + style A2 fill:#2e7d32,stroke:#1b5e20,color:#fff + style A3 fill:#2e7d32,stroke:#1b5e20,color:#fff + style A4 fill:#2e7d32,stroke:#1b5e20,color:#fff + style B1 fill:#d84315,stroke:#bf360c,color:#fff + style B2 fill:#d84315,stroke:#bf360c,color:#fff + style B3 fill:#d84315,stroke:#bf360c,color:#fff + style B4 fill:#d84315,stroke:#bf360c,color:#fff + style C1 fill:#1565c0,stroke:#0d47a1,color:#fff + style C2 fill:#1565c0,stroke:#0d47a1,color:#fff + style C3 fill:#1565c0,stroke:#0d47a1,color:#fff + style C4 fill:#1565c0,stroke:#0d47a1,color:#fff +``` + +### 4.2 Effort Estimates + +| Phase | Task | Effort | Dependencies | +| ----- | --------------------------------- | ---------- | ---------------------- | +| **A** | ARCHITECTURE.md | 2d | None | +| **A** | CLAUDE.md boundaries update | 0.5d | None | +| **A** | Pre-commit hooks | 1d | None | +| **A** | OTel baseline verification script | 2d | OTel Phases 1-6 merged | +| **B** | Semantic linting config | 2d | Phase A complete | +| **B** | Tiered CI pipeline | 3d | Phase A complete | +| **B** | Metric regression CI check | 2d | OTel Phase 7 merged | +| **B** | Phase 10 synthetic workload | 10d | OTel Phase 9 merged | +| **C** | WORKFLOW.md | 1d | Phases A+B complete | +| **C** | Orchestrator setup | 3d | WORKFLOW.md defined | +| **C** | Jira integration | 2d | Orchestrator running | +| **C** | First autonomous runs | 2d | Everything above | +| | **Total** | **~30.5d** | | + +### 4.3 What to Do Immediately + +These four items require no OTel dependencies and can start today: + +1. **Create `ARCHITECTURE.md`** — 300-line codebase map covering the 5 layers (RPC, NetworkOPs, Consensus, Ledger, NodeStore), module directory map, entry points, and telemetry coverage map linking to `09-data-collection-reference.md` + +2. **Add Three-Tier Boundaries to `CLAUDE.md`** — Explicit Always/Ask/Never categories (see Section 3.3 above) + +3. **Add pre-commit hooks** — `clang-format` check + OTel span naming convention check + +4. **Write `scripts/check-otel-baseline.sh`** — Script that starts the 5-node Docker stack, runs a synthetic workload, and validates all 16 spans are present with required attributes + +--- + +## Part 5: Unique Advantages for rippled + +### 5.1 Why rippled Is Especially Well-Suited + +| Advantage | Explanation | +| ------------------------------ | ----------------------------------------------------------------------------------------- | +| **Deterministic consensus** | Consensus rounds produce predictable telemetry — ideal for regression detection | +| **Isolated test networks** | 5-node validator harness gives agents a safe, isolated sandbox | +| **Rich telemetry baseline** | 16 spans + 300+ metrics + 8 dashboards = comprehensive "expected state" | +| **Protocol Buffer boundaries** | Wire format changes are explicit and lintable — agents can't silently break compatibility | +| **Existing CLAUDE.md** | Already have an agent instruction file — just needs boundary tiers | +| **Modular plan docs** | 10 architecture docs serve as deep reference material for ARCHITECTURE.md links | + +### 5.2 Risks and Mitigations + +| Risk | Likelihood | Impact | Mitigation | +| -------------------------------------------------- | ---------- | ------ | ------------------------------------------------------------------------------------------ | +| Agent modifies consensus-critical code incorrectly | Medium | High | Three-tier boundary: consensus changes = "Ask" tier; OTel traces detect timing regressions | +| OTel verification adds CI time | Low | Medium | Tiered pipeline — OTel checks only run after fast checks pass | +| Symphony/orchestrator maintenance burden | Medium | Medium | Start with manual agent runs using CLAUDE.md; orchestrator is Phase C | +| Metric baseline drift | Medium | Low | Regularly update baselines; use percentage thresholds not absolute values | +| C++ build times limit agent feedback speed | High | Medium | Incremental builds, ccache, focus on affected module tests first | + +### 5.3 The Competitive Edge + +Most blockchain projects have **no observability infrastructure at all**. The rippled OTel initiative gives the project: + +1. **The only blockchain node with comprehensive OpenTelemetry instrumentation** — traces, metrics, logs, all correlated +2. **A deterministic verification layer** that AI agents can use to validate their own changes +3. **A path to autonomous development** where routine engineering tasks (bug fixes, feature additions, test expansion) can be handled by agents with proof-of-work requirements + +This is not just observability — it's the foundation for **10x engineering velocity** through agent-first development. + +--- + +## Part 6: OpenClaw and OpenFang — Agent Runtime Integration Opportunities + +### 6.1 Project Overview + +Two open-source projects in the agent ecosystem offer capabilities that map directly onto the harness engineering strategy: + +| Project | Description | Language | Stars | Key Insight | +| ------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | ---------- | ----- | --------------------------------------------------------------------------------------------- | +| **[OpenClaw](https://github.com/openclaw/openclaw)** | Personal AI assistant platform with gateway, multi-channel inbox, skills system, and browser control | TypeScript | 296K | Skills platform + multi-agent routing = harness orchestration layer | +| **[OpenFang](https://github.com/RightNow-AI/openfang)** | Agent Operating System — kernel-based architecture with autonomous "Hands," WASM sandbox, Merkle audit trail, 140+ API endpoints | Rust | 13K | Rust kernel + security-first design + built-in observability = production-grade agent harness | + +### 6.2 OpenClaw: Skills-Based Agent Orchestration + +OpenClaw's architecture is a **gateway-centric control plane** where agents connect via WebSocket, receive work, and execute through a skills system. Relevant capabilities for rippled: + +**Skills Platform** + +OpenClaw's managed skills system (ClawHub marketplace) maps to how rippled could package agent capabilities: + +| OpenClaw Concept | rippled Harness Equivalent | +| --------------------------------- | ------------------------------------------------------------------------------------------ | +| **Skill** (bundled capability) | OTel verification script, lint checker, metric regression detector | +| **ClawHub** (skill marketplace) | Internal skill registry for rippled-specific agent capabilities | +| **Workspace skills** (repo-local) | `scripts/` directory with harness verification tools | +| **Agent routing** (per-channel) | Route different issue types to specialized agents (RPC agent, consensus agent, test agent) | + +**Multi-Agent Routing** + +OpenClaw can route inbound messages to isolated agents with separate workspaces and sessions. For rippled: + +- **RPC Layer Agent** — handles `src/xrpld/rpc/` changes, runs RPC-specific OTel span validation +- **Consensus Agent** — handles `src/xrpld/consensus/` changes, runs timing-sensitive verification, always in "Ask" tier +- **Test Agent** — handles test expansion, runs the full suite, validates coverage + +**Session-to-Session Coordination** + +OpenClaw's `sessions_send` tool enables agent-to-agent communication. A supervisor agent could: + +1. Receive a Jira ticket via channel adapter +2. Analyze scope, spawn a specialist agent +3. Monitor progress, collect proof of work +4. Report results back via the channel + +**Practical Fit**: OpenClaw is a **medium-term option (Phase C)** as an alternative to Symphony for agent orchestration. Its TypeScript stack is more approachable than Symphony's Elixir/BEAM, and the skills platform provides a natural packaging mechanism for rippled-specific harness tools. + +### 6.3 OpenFang: Rust-Native Agent OS + +OpenFang is architecturally the closest match to rippled's needs — a Rust-based kernel with security-first design, built for autonomous agents. + +**Architecture Alignment** + +```mermaid +flowchart TB + subgraph openfang["OpenFang Kernel"] + direction TB + KERNEL["openfang-kernel
Orchestration, RBAC,
scheduling, metering"] + RUNTIME["openfang-runtime
Agent loop, 53 tools,
WASM sandbox, MCP"] + MEMORY["openfang-memory
SQLite + vector,
session compaction"] + API["openfang-api
140+ REST/WS/SSE,
dashboard"] + end + + subgraph harness["rippled Harness Layer"] + direction TB + HAND_OTEL["OTel Verifier Hand
(custom)"] + HAND_BUILD["Build + Test Hand
(custom)"] + HAND_METRIC["Metric Regression
Hand (custom)"] + HAND_REVIEW["Code Review
Hand (custom)"] + end + + subgraph rippled_infra["rippled Infrastructure"] + direction TB + OTEL["OTel Stack
Collector + Tempo +
Grafana"] + CI_SYS["CI Pipeline
GitHub Actions"] + HARNESS_5["5-Node Test
Harness"] + JIRA["Jira
RIPD Project"] + end + + KERNEL --> RUNTIME + RUNTIME --> MEMORY + RUNTIME --> API + + HAND_OTEL --> OTEL + HAND_BUILD --> CI_SYS + HAND_METRIC --> OTEL + HAND_REVIEW --> HARNESS_5 + + KERNEL --> HAND_OTEL + KERNEL --> HAND_BUILD + KERNEL --> HAND_METRIC + KERNEL --> HAND_REVIEW + + JIRA -.->|"channel adapter"| API + + style openfang fill:#4a148c,stroke:#2e0d57,color:#fff + style harness fill:#bf360c,stroke:#8c2809,color:#fff + style rippled_infra fill:#1b5e20,stroke:#0d3d14,color:#fff + style KERNEL fill:#6a1b9a,stroke:#4a148c,color:#fff + style RUNTIME fill:#6a1b9a,stroke:#4a148c,color:#fff + style MEMORY fill:#6a1b9a,stroke:#4a148c,color:#fff + style API fill:#6a1b9a,stroke:#4a148c,color:#fff + style HAND_OTEL fill:#d84315,stroke:#bf360c,color:#fff + style HAND_BUILD fill:#d84315,stroke:#bf360c,color:#fff + style HAND_METRIC fill:#d84315,stroke:#bf360c,color:#fff + style HAND_REVIEW fill:#d84315,stroke:#bf360c,color:#fff + style OTEL fill:#2e7d32,stroke:#1b5e20,color:#fff + style CI_SYS fill:#2e7d32,stroke:#1b5e20,color:#fff + style HARNESS_5 fill:#2e7d32,stroke:#1b5e20,color:#fff + style JIRA fill:#2e7d32,stroke:#1b5e20,color:#fff +``` + +**Key OpenFang Features for rippled Harness Engineering** + +| OpenFang Feature | Harness Engineering Value | +| -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | +| **Hands** (autonomous capability packages) | Package OTel verification, build validation, and metric regression detection as custom Hands that run on schedules | +| **HAND.toml** (manifest with tools, metrics, guardrails) | Declaratively define what each harness agent can access, what metrics it reports, what actions need approval | +| **WASM Dual-Metered Sandbox** | Run agent-generated code in isolation — critical for a financial system like XRPL | +| **Merkle Hash-Chain Audit Trail** | Cryptographic proof of every agent action — required for regulatory compliance in blockchain infrastructure | +| **16 Security Systems** | Defense-in-depth (SSRF protection, secret zeroization, taint tracking) matches rippled's security requirements | +| **MCP + A2A Protocol Support** | Native Model Context Protocol support means agents can use MCP servers to access rippled build tools, OTel queries, and Jira | +| **Capability Gates (RBAC)** | Map directly to Three-Tier Boundaries — declare what tools each agent role can use | +| **Budget Tracking + Metering** | Track LLM API costs per agent per task — important for sustainable autonomous development | +| **40 Channel Adapters** | Connect to Jira, Slack, Discord for inbound work items and outbound status reports | +| **Loop Guard** | SHA256-based detection prevents agents from getting stuck in retry loops on failing builds | + +**Custom Hands for rippled** + +The "Hands" concept (autonomous capability packages defined by `HAND.toml` + system prompt + `SKILL.md`) maps perfectly to harness engineering tasks: + +```toml +# HAND.toml — OTel Verification Hand (conceptual) +[hand] +name = "otel-verifier" +description = "Validates OTel telemetry after code changes" +schedule = "on_pr" + +[tools] +required = ["bash", "curl", "browser"] + +[settings] +harness_nodes = 5 +workload_duration_sec = 60 +span_count_expected = 16 +metric_regression_threshold_pct = 5.0 + +[guardrails] +require_approval = ["deploy_to_testnet", "modify_consensus_params"] + +[dashboard] +metrics = ["spans_validated", "metrics_checked", "regressions_found"] +``` + +**Why OpenFang Over Symphony for rippled** + +| Factor | Symphony (Elixir) | OpenFang (Rust) | +| ---------------------- | -------------------------------- | ----------------------------------------------------------------------- | +| **Language alignment** | Elixir — foreign to rippled team | Rust — closer to C++ mindset, memory-safe systems language | +| **Security model** | Basic workspace isolation | 16-layer defense-in-depth (WASM sandbox, taint tracking, audit trail) | +| **Autonomous agents** | Spawns external Codex | Built-in autonomous Hands that run 24/7 | +| **Observability** | Structured logs | Dashboard + 140+ API endpoints for monitoring agent runs | +| **Audit trail** | Git history | Merkle hash-chain — tamper-evident, suitable for regulated environments | +| **Install** | Elixir runtime + Docker | Single 32MB binary | +| **Agent memory** | None built-in | SQLite + vector embeddings, session compaction | + +### 6.4 Integration Strategy: OpenFang as the Primary Harness Runtime + +**Recommended approach**: Use OpenFang as the agent operating system for rippled's harness engineering strategy, replacing Symphony in the Phase C orchestration layer. + +```mermaid +flowchart TB + subgraph phase_a_new["Phase A: Foundation (unchanged)"] + direction TB + A1_N["ARCHITECTURE.md"] + A2_N["Three-Tier Boundaries"] + A3_N["Pre-commit hooks"] + A4_N["OTel baseline script"] + end + + subgraph phase_b_new["Phase B: Feedback Loops (unchanged)"] + direction TB + B1_N["Semantic linting"] + B2_N["Tiered CI pipeline"] + B3_N["Metric regression check"] + B4_N["Phase 10: Synthetic workload"] + end + + subgraph phase_c_new["Phase C: Agent Runtime (updated)"] + direction TB + C1_N["Deploy OpenFang
alongside rippled infra"] + C2_N["Build custom Hands:
otel-verifier, build-runner,
metric-checker, code-reviewer"] + C3_N["Connect Jira via
channel adapter"] + C4_N["Write HAND.toml manifests
with RBAC matching
Three-Tier Boundaries"] + C5_N["First autonomous
implementation runs"] + end + + subgraph phase_d_new["Phase D: Advanced (new)"] + direction TB + D1_N["OpenClaw skills for
developer-facing interface"] + D2_N["Agent-to-agent coordination
via A2A protocol"] + D3_N["Merkle audit trail
for compliance reporting"] + D4_N["FangHub marketplace
for rippled-specific Hands"] + end + + phase_a_new --> phase_b_new --> phase_c_new --> phase_d_new + + style phase_a_new fill:#1b5e20,stroke:#0d3d14,color:#fff + style phase_b_new fill:#bf360c,stroke:#8c2809,color:#fff + style phase_c_new fill:#0d47a1,stroke:#082f6a,color:#fff + style phase_d_new fill:#4a148c,stroke:#2e0d57,color:#fff + style A1_N fill:#2e7d32,stroke:#1b5e20,color:#fff + style A2_N fill:#2e7d32,stroke:#1b5e20,color:#fff + style A3_N fill:#2e7d32,stroke:#1b5e20,color:#fff + style A4_N fill:#2e7d32,stroke:#1b5e20,color:#fff + style B1_N fill:#d84315,stroke:#bf360c,color:#fff + style B2_N fill:#d84315,stroke:#bf360c,color:#fff + style B3_N fill:#d84315,stroke:#bf360c,color:#fff + style B4_N fill:#d84315,stroke:#bf360c,color:#fff + style C1_N fill:#1565c0,stroke:#0d47a1,color:#fff + style C2_N fill:#1565c0,stroke:#0d47a1,color:#fff + style C3_N fill:#1565c0,stroke:#0d47a1,color:#fff + style C4_N fill:#1565c0,stroke:#0d47a1,color:#fff + style C5_N fill:#1565c0,stroke:#0d47a1,color:#fff + style D1_N fill:#6a1b9a,stroke:#4a148c,color:#fff + style D2_N fill:#6a1b9a,stroke:#4a148c,color:#fff + style D3_N fill:#6a1b9a,stroke:#4a148c,color:#fff + style D4_N fill:#6a1b9a,stroke:#4a148c,color:#fff +``` + +### 6.5 Comparative Summary + +| Capability | Symphony | OpenClaw | OpenFang | Recommendation | +| ----------------------- | ------------------------ | ---------------------------------- | ----------------------------------------- | ------------------------------------------------------- | +| **Agent Orchestration** | Polling + dispatch | Gateway + routing | Kernel + scheduler | OpenFang for autonomous, OpenClaw for interactive | +| **Security** | Basic isolation | DM pairing, allowlists | 16-layer defense-in-depth | OpenFang — critical for financial infrastructure | +| **Observability** | Structured logs | Logging | Dashboard + 140 API endpoints | OpenFang — aligns with OTel investment | +| **Proof of Work** | CI status, PR review | N/A | Merkle hash-chain audit trail | OpenFang — cryptographic proof | +| **Developer UX** | Elixir CLI | Multi-channel (Slack, Discord) | CLI + Tauri desktop + 40 channels | OpenClaw for notification layer, OpenFang for execution | +| **rippled Fit** | Good (designed for this) | Medium (general-purpose assistant) | Strong (Rust, security-first, autonomous) | **OpenFang primary, OpenClaw supplementary** | + +### 6.6 Recommended Architecture: Dual-Layer Setup + +The optimal configuration uses **both** projects in complementary roles: + +- **OpenFang** = **execution layer** — runs autonomous Hands for build verification, OTel validation, metric regression detection, and code review. Handles the "harness" in harness engineering. +- **OpenClaw** = **communication layer** — developer-facing interface via Slack/Discord. Developers chat with OpenClaw to ask about agent status, trigger manual runs, review proof-of-work summaries. OpenClaw routes to OpenFang's API for execution. + +This mirrors OpenAI's own separation: Symphony handles orchestration (like OpenFang), while the developer interacts through a surface (like OpenClaw) to review and approve work. + +--- + +## Part 7: MCP Integrations — Connecting Bug Reporting, Bounty, and CI Platforms + +### 7.1 The MCP Bridge + +The Model Context Protocol (MCP) is the glue that connects autonomous agents to external platforms. For rippled's harness engineering strategy, three MCP servers form the critical integration layer between agents and the project's existing workflows: + +| MCP Server | Provider | Stars | Capabilities | +| ----------------------------------------------------------------------------- | -------------------- | ----- | -------------------------------------------------------------------------------------------------------------------------------- | +| **[GitHub MCP Server](https://github.com/github/github-mcp-server)** | GitHub (official) | 27K | 18 toolsets: repos, issues, PRs, actions, code_security, secret_protection, security_advisories, notifications, labels, and more | +| **[Atlassian MCP Server](https://github.com/atlassian/atlassian-mcp-server)** | Atlassian (official) | 400+ | Jira (search/create/update issues via JQL), Confluence (pages, search via CQL), Compass (components), OAuth 2.1 + API token auth | +| **[mcp-atlassian](https://github.com/sooperset/mcp-atlassian)** | Community | 4.5K | Jira + Confluence with broader client compatibility (Claude Code, VS Code, Cursor) | + +### 7.2 The Existing Bug Reporting Pipeline + +rippled already has a well-defined vulnerability reporting structure (from [SECURITY.md](https://github.com/XRPLF/rippled/blob/develop/SECURITY.md)): + +```mermaid +flowchart TB + subgraph inbound["Bug Discovery Channels"] + direction TB + BOUNTY["Bug Bounty Program
(bugs@ripple.com, PGP encrypted)
Scope: rippled, xrpl.js, xrpl-py, xrpl4j"] + GH_ISSUES["GitHub Issues
(XRPLF/rippled)
Labels: Bug, Triaged"] + INTERNAL["Internal Discovery
(dev team, CI failures,
OTel anomaly detection)"] + end + + subgraph triage["Triage Process"] + direction TB + ASSESS["Two independent evaluators
assess the report"] + PLAN["Formulate fix plan"] + FIX["Develop, test, release fix"] + ANNOUNCE["Public announcement"] + end + + subgraph jira_track["Jira Tracking"] + direction TB + EPIC["Epic: RIPD-5060
(OTel initiative)"] + STORIES["Stories per phase
(e.g. RIPD-5187)"] + BUGS["Bug tickets
(RIPD project)"] + end + + BOUNTY --> ASSESS + GH_ISSUES --> ASSESS + INTERNAL --> ASSESS + ASSESS --> PLAN --> FIX --> ANNOUNCE + + GH_ISSUES -.->|"sync"| BUGS + BUGS --> STORIES + STORIES --> EPIC + + style inbound fill:#c62828,stroke:#b71c1c,color:#fff + style triage fill:#bf360c,stroke:#8c2809,color:#fff + style jira_track fill:#0d47a1,stroke:#082f6a,color:#fff + style BOUNTY fill:#d32f2f,stroke:#c62828,color:#fff + style GH_ISSUES fill:#d32f2f,stroke:#c62828,color:#fff + style INTERNAL fill:#d32f2f,stroke:#c62828,color:#fff + style ASSESS fill:#d84315,stroke:#bf360c,color:#fff + style PLAN fill:#d84315,stroke:#bf360c,color:#fff + style FIX fill:#d84315,stroke:#bf360c,color:#fff + style ANNOUNCE fill:#d84315,stroke:#bf360c,color:#fff + style EPIC fill:#1565c0,stroke:#0d47a1,color:#fff + style STORIES fill:#1565c0,stroke:#0d47a1,color:#fff + style BUGS fill:#1565c0,stroke:#0d47a1,color:#fff +``` + +**Key characteristics of the current pipeline:** + +- **Bug bounty** is email-based (PGP-encrypted to `bugs@ripple.com`) — not on a platform like Immunefi or HackerOne +- **Scope** covers rippled, xrpl.js, xrpl-py, xrpl4j — security issues only (funds, privacy, network operation) +- **Triage** requires two independent evaluators, private reporter communication, and a coordinated disclosure timeline +- **GitHub Issues** track public bugs with `Bug` and `Triaged` labels +- **Jira (RIPD project)** tracks internal engineering work, with epics (RIPD-5060 for OTel) and stories per phase + +### 7.3 GitHub MCP Server — CI, PRs, and Security Integration + +The [GitHub MCP Server](https://github.com/github/github-mcp-server) (official, Go-based) gives agents direct access to the rippled repository through 18 toolsets. The harness-relevant toolsets: + +| Toolset | Harness Engineering Use | +| ------------------------- | ------------------------------------------------------------------------------------------------------------------- | +| **`actions`** | Monitor CI workflow runs, detect build failures, analyze flaky tests — feeds into Fail-Fast Feedback (Principle #4) | +| **`issues`** | Read bug reports, extract reproduction steps, link to related PRs — automates triage intake | +| **`pull_requests`** | Create PRs with proof-of-work evidence, read review comments, update PR descriptions — core autonomous workflow | +| **`code_security`** | Access Code Scanning alerts (CodeQL) — agents can detect and fix security findings autonomously | +| **`secret_protection`** | Check for exposed secrets — critical for a financial system; agent can flag and remediate immediately | +| **`security_advisories`** | Read/create security advisories — connects to bug bounty reporting flow | +| **`repos`** | Browse code, search files, analyze commits — supports codebase exploration during implementation runs | +| **`labels`** | Apply triage labels (`Bug`, `Triaged`, `Security`) — automates issue categorization | + +**Agent workflow with GitHub MCP:** + +```mermaid +flowchart LR + subgraph trigger["Trigger"] + ISSUE["New GitHub Issue
(labeled 'Bug')"] + CI_FAIL["CI Failure
(Actions workflow)"] + SEC_ALERT["Code Scanning
Alert"] + end + + subgraph agent_work["Agent via GitHub MCP"] + READ["Read issue/alert
details"] + ANALYZE["Analyze code +
reproduction steps"] + IMPLEMENT["Implement fix
in worktree"] + PR_CREATE["Create PR with
proof-of-work"] + end + + subgraph verify_gh["Verification"] + CI_CHECK["Monitor Actions
workflow run"] + REVIEW["Request review +
address comments"] + LABEL["Apply resolution
labels"] + end + + ISSUE --> READ + CI_FAIL --> READ + SEC_ALERT --> READ + READ --> ANALYZE --> IMPLEMENT --> PR_CREATE + PR_CREATE --> CI_CHECK --> REVIEW --> LABEL + + style trigger fill:#c62828,stroke:#b71c1c,color:#fff + style agent_work fill:#1b5e20,stroke:#0d3d14,color:#fff + style verify_gh fill:#0d47a1,stroke:#082f6a,color:#fff + style ISSUE fill:#d32f2f,stroke:#c62828,color:#fff + style CI_FAIL fill:#d32f2f,stroke:#c62828,color:#fff + style SEC_ALERT fill:#d32f2f,stroke:#c62828,color:#fff + style READ fill:#2e7d32,stroke:#1b5e20,color:#fff + style ANALYZE fill:#2e7d32,stroke:#1b5e20,color:#fff + style IMPLEMENT fill:#2e7d32,stroke:#1b5e20,color:#fff + style PR_CREATE fill:#2e7d32,stroke:#1b5e20,color:#fff + style CI_CHECK fill:#1565c0,stroke:#0d47a1,color:#fff + style REVIEW fill:#1565c0,stroke:#0d47a1,color:#fff + style LABEL fill:#1565c0,stroke:#0d47a1,color:#fff +``` + +### 7.4 Jira MCP Server — Work Tracking and Sprint Integration + +Two options for Jira integration: + +| Server | Best For | +| ---------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| **[Atlassian MCP Server](https://github.com/atlassian/atlassian-mcp-server)** (official) | Production — OAuth 2.1, Atlassian-hosted, respects all Jira permissions | +| **[mcp-atlassian](https://github.com/sooperset/mcp-atlassian)** (community, 4.5K stars) | Development — broader client support, self-hosted, faster iteration | + +**Jira MCP enables agents to:** + +- **Read RIPD tickets** — pull acceptance criteria, linked PRs, priority, and sprint assignment via JQL queries +- **Create bug tickets** — when an agent discovers a regression during OTel validation, auto-create a RIPD bug ticket with telemetry evidence (span diff, metric deviation, dashboard screenshot URL) +- **Update ticket status** — transition tickets through the workflow (Open -> In Progress -> In Review -> Done) as the agent progresses +- **Link PRs to tickets** — automatically add PR links to Jira tickets (never the reverse, per project rules) +- **Search for context** — JQL queries like `project = RIPD AND labels = otel AND status != Done` to find related work + +**Recommended AGENTS.md/CLAUDE.md configuration for Jira MCP:** + +```markdown +## Atlassian Rovo MCP + +When connected to atlassian-rovo-mcp: + +- **MUST** use Jira project key = RIPD +- **MUST** use cloudId for the Ripple Atlassian instance +- **MUST** use `maxResults: 10` for all JQL search operations +- **MUST** link PRs TO Jira tickets, never Jira tickets INTO PRs +- **MUST NOT** create or modify Security-labeled tickets without human approval +``` + +### 7.5 Bug Bounty Pipeline — Agent-Assisted Triage + +The bug bounty program's email-based intake is inherently manual, but agents can accelerate the downstream process once a report enters the system: + +```mermaid +flowchart TB + subgraph intake["Intake (Manual)"] + EMAIL["Encrypted email to
bugs@ripple.com"] + HUMAN_TRIAGE["Two evaluators
assess report"] + end + + subgraph agent_assist["Agent-Assisted (via MCP)"] + JIRA_CREATE["Create RIPD security ticket
(Jira MCP)"] + REPRO["Attempt reproduction
on test harness"] + OTEL_CHECK["Check OTel telemetry for
anomalous patterns matching
the reported vulnerability"] + FIX_BRANCH["Create fix branch +
implement patch"] + VALIDATE["Run OTel verification:
spans intact, no regression,
vulnerability mitigated"] + PR_SUBMIT["Create PR with
proof-of-work
(GitHub MCP)"] + end + + subgraph release["Release (Manual)"] + REVIEW_SEC["Security review of fix"] + RELEASE["Coordinated release"] + ADVISORY["Public advisory
(GitHub MCP: security_advisories)"] + end + + EMAIL --> HUMAN_TRIAGE + HUMAN_TRIAGE -->|"approved"| JIRA_CREATE + JIRA_CREATE --> REPRO --> OTEL_CHECK --> FIX_BRANCH --> VALIDATE --> PR_SUBMIT + PR_SUBMIT --> REVIEW_SEC --> RELEASE --> ADVISORY + + style intake fill:#c62828,stroke:#b71c1c,color:#fff + style agent_assist fill:#1b5e20,stroke:#0d3d14,color:#fff + style release fill:#4a148c,stroke:#2e0d57,color:#fff + style EMAIL fill:#d32f2f,stroke:#c62828,color:#fff + style HUMAN_TRIAGE fill:#d32f2f,stroke:#c62828,color:#fff + style JIRA_CREATE fill:#2e7d32,stroke:#1b5e20,color:#fff + style REPRO fill:#2e7d32,stroke:#1b5e20,color:#fff + style OTEL_CHECK fill:#2e7d32,stroke:#1b5e20,color:#fff + style FIX_BRANCH fill:#2e7d32,stroke:#1b5e20,color:#fff + style VALIDATE fill:#2e7d32,stroke:#1b5e20,color:#fff + style PR_SUBMIT fill:#2e7d32,stroke:#1b5e20,color:#fff + style REVIEW_SEC fill:#6a1b9a,stroke:#4a148c,color:#fff + style RELEASE fill:#6a1b9a,stroke:#4a148c,color:#fff + style ADVISORY fill:#6a1b9a,stroke:#4a148c,color:#fff +``` + +**Key boundaries for security work:** + +| Action | Tier | Rationale | +| --------------------------------------------- | ---------- | ------------------------------------ | +| Read bounty ticket details from Jira | **Ask** | Security-sensitive content | +| Attempt reproduction on isolated test harness | **Always** | Sandboxed, no production risk | +| Check OTel telemetry for matching anomalies | **Always** | Read-only query | +| Create fix branch and implement patch | **Always** | Isolated worktree | +| Run OTel verification suite | **Always** | Deterministic, sandboxed | +| Create PR for security fix | **Ask** | Visibility to others | +| Publish security advisory | **Never** | Must be human-coordinated disclosure | +| Modify cryptographic code | **Never** | Per existing Three-Tier Boundaries | + +### 7.6 Proactive Bug Detection via OTel + MCP + +The most powerful integration is agents that **find bugs before they're reported**, using OTel telemetry as a continuous health signal: + +| Detection Pattern | OTel Signal | Agent Action (via MCP) | +| -------------------------------- | ----------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| Latency spike on `rpc.submit` | Span duration > p99 threshold | Create GitHub Issue (GitHub MCP), create RIPD ticket (Jira MCP), investigate root cause | +| Consensus round timeout | `consensus.round` span exceeds tolerance | Alert via Slack (OpenClaw), create high-priority Jira ticket, run diagnostic on test harness | +| Transaction relay failure | `tx.relay` span missing child `peer.send` spans | Investigate overlay connectivity, create bug report with trace evidence | +| Metric regression after PR merge | Prometheus metric deviation > 5% | Comment on merged PR (GitHub MCP), create regression ticket (Jira MCP), attempt automated revert | +| Code scanning alert | GitHub CodeQL finding | Read alert details (GitHub MCP), assess severity, create fix PR if low-risk, escalate if high-risk | +| Secret exposure | GitHub secret scanning alert | Immediately flag (GitHub MCP), create urgent Jira ticket, agent must NOT attempt to remediate secrets autonomously (**Never** tier) | + +### 7.7 MCP Configuration for the Harness + +**Recommended MCP server configuration for Claude Code / agent runtimes:** + +```json +{ + "mcpServers": { + "github": { + "type": "http", + "url": "https://api.githubcopilot.com/mcp/", + "toolsets": [ + "repos", + "issues", + "pull_requests", + "actions", + "code_security", + "secret_protection", + "security_advisories", + "labels", + "notifications" + ] + }, + "atlassian": { + "type": "http", + "url": "https://mcp.atlassian.com/v1/mcp", + "note": "OAuth 2.1 — scoped to RIPD project" + } + } +} +``` + +**For OpenFang integration**, these MCP servers connect natively via OpenFang's MCP support, enabling Hands to: + +- Poll Jira for new RIPD tickets matching `type = Bug AND status = Open` +- Read GitHub Issues labeled `Bug` for reproduction steps +- Create PRs with proof-of-work after verification passes +- Monitor GitHub Actions for CI status on agent-created PRs + +### 7.8 End-to-End: From Bug Report to Verified Fix + +Putting it all together — the complete harness-engineered bug lifecycle: + +```mermaid +flowchart TB + subgraph sources["Bug Sources"] + direction LR + S1["Bug Bounty
(email)"] + S2["GitHub Issue
(public)"] + S3["OTel Anomaly
(automated)"] + S4["CI Failure
(GitHub Actions)"] + end + + subgraph mcp_layer["MCP Integration Layer"] + direction LR + JIRA_MCP["Jira MCP
(create/read tickets)"] + GH_MCP["GitHub MCP
(issues, PRs, CI, security)"] + end + + subgraph agent_runtime["Agent Runtime (OpenFang)"] + direction TB + TRIAGE_HAND["Triage Hand
(classify, prioritize)"] + FIX_HAND["Fix Hand
(implement, test)"] + VERIFY_HAND["OTel Verifier Hand
(validate telemetry)"] + end + + subgraph proof["Proof of Work"] + direction LR + P1["CI passes"] + P2["16 spans valid"] + P3["Metrics stable"] + P4["Audit trail
(Merkle chain)"] + end + + S1 --> JIRA_MCP + S2 --> GH_MCP + S3 --> GH_MCP + S4 --> GH_MCP + + JIRA_MCP --> TRIAGE_HAND + GH_MCP --> TRIAGE_HAND + TRIAGE_HAND --> FIX_HAND --> VERIFY_HAND + + VERIFY_HAND --> P1 + VERIFY_HAND --> P2 + VERIFY_HAND --> P3 + VERIFY_HAND --> P4 + + P1 -->|"PR created via
GitHub MCP"| GH_MCP + P4 -->|"ticket updated via
Jira MCP"| JIRA_MCP + + style sources fill:#c62828,stroke:#b71c1c,color:#fff + style mcp_layer fill:#e65100,stroke:#bf360c,color:#fff + style agent_runtime fill:#1b5e20,stroke:#0d3d14,color:#fff + style proof fill:#0d47a1,stroke:#082f6a,color:#fff + style S1 fill:#d32f2f,stroke:#c62828,color:#fff + style S2 fill:#d32f2f,stroke:#c62828,color:#fff + style S3 fill:#d32f2f,stroke:#c62828,color:#fff + style S4 fill:#d32f2f,stroke:#c62828,color:#fff + style JIRA_MCP fill:#f57c00,stroke:#e65100,color:#fff + style GH_MCP fill:#f57c00,stroke:#e65100,color:#fff + style TRIAGE_HAND fill:#2e7d32,stroke:#1b5e20,color:#fff + style FIX_HAND fill:#2e7d32,stroke:#1b5e20,color:#fff + style VERIFY_HAND fill:#2e7d32,stroke:#1b5e20,color:#fff + style P1 fill:#1565c0,stroke:#0d47a1,color:#fff + style P2 fill:#1565c0,stroke:#0d47a1,color:#fff + style P3 fill:#1565c0,stroke:#0d47a1,color:#fff + style P4 fill:#1565c0,stroke:#0d47a1,color:#fff +``` + +--- + +## Appendix A: Key Resources + +| Resource | URL | +| ------------------------------- | -------------------------------------------------------------------------------------------------------- | +| OpenAI Harness Engineering Blog | https://openai.com/index/harness-engineering/ | +| Symphony Framework | https://github.com/openai/symphony | +| Symphony Spec | https://github.com/openai/symphony/blob/main/SPEC.md | +| Agentic Harness Bootstrap | https://github.com/synthnoosh/agentic-harness-bootstrap | +| Awesome Agent Harness | https://github.com/AutoJunjie/awesome-agent-harness | +| OpenClaw (Agent Assistant) | https://github.com/openclaw/openclaw | +| OpenFang (Agent OS) | https://github.com/RightNow-AI/openfang | +| GitHub MCP Server (official) | https://github.com/github/github-mcp-server | +| Atlassian MCP Server (official) | https://github.com/atlassian/atlassian-mcp-server | +| mcp-atlassian (community) | https://github.com/sooperset/mcp-atlassian | +| rippled SECURITY.md | https://github.com/XRPLF/rippled/blob/develop/SECURITY.md | +| rippled OTel PR Chain | #6436 -> #6437 -> #6438 -> #6424 -> #6425 -> #6426 -> #6427 -> #6433 -> #6439 -> #6493 -> #6494 -> #6513 | +| rippled OTel Plan Docs | `OpenTelemetryPlan/` directory (10 documents) | + +## Appendix B: Harness Engineering Ecosystem + +The broader ecosystem emerging around harness engineering: + +| Category | Tools | +| ---------------------- | -------------------------------------------------------------------------------------- | +| **Agent OS/Runtime** | **OpenFang** (Rust, autonomous Hands), **OpenClaw** (TS, skills + gateway) | +| **Orchestrators** | Symphony (OpenAI), Vibe Kanban, Emdash, Warp, VibeHQ | +| **Task Runners** | Symphony, Baton, Axon, GitHub Copilot Coding Agent | +| **Coding Agents** | Claude Code, Codex, OpenCode, Gemini CLI, Kiro, Amp, Cursor | +| **Spec/Requirements** | Kiro IDE, OpenSpec, agents.md, AGENTS.md | +| **Protocols** | MCP (Model Context Protocol), ACP (Agent Communication Protocol), A2A (Agent-to-Agent) | +| **Harness Frameworks** | Deep Agents, Gambit, Harness Kit, Bridle | + +## Appendix C: Glossary + +| Term | Definition | +| ------------------------- | ------------------------------------------------------------------------------------------------ | +| **Harness Engineering** | Building infrastructure around AI coding agents to enable reliable autonomous development | +| **Proof of Work** | Evidence that an agent's changes are correct: CI pass, metric stability, trace validity | +| **Three-Tier Boundaries** | Categorizing agent actions into Always (autonomous), Ask (confirm), Never (forbidden) | +| **WORKFLOW.md** | Version-controlled file defining agent orchestration policy, prompt template, and runtime config | +| **Implementation Run** | A complete autonomous cycle: read issue -> implement -> verify -> PR | +| **Semantic Linting** | Linter errors that include enough context for an agent to fix the violation in one attempt | +| **Architecture Map** | A concise document (`ARCHITECTURE.md`) telling agents where things are in the codebase | +| **MCP** | Model Context Protocol — standard for connecting AI agents to external tools and data sources | +| **MCP Server** | A service exposing tools/resources via MCP that agents can invoke (e.g., GitHub MCP, Jira MCP) | +| **Bug Bounty** | Program rewarding external researchers for responsibly disclosing security vulnerabilities | +| **Hand** | OpenFang's autonomous capability package — bundles tools, prompts, guardrails, and schedules |