Commit Graph

32 Commits

Author SHA1 Message Date
Nicholas Dudfield
a956abb2d1 docs(consensus): clarify export sig quorum gates 2026-04-28 08:56:53 +07:00
Nicholas Dudfield
aa36a80ab7 docs(consensus): document sidecar acquire rationale 2026-04-28 08:35:45 +07:00
Nicholas Dudfield
15662eb1b1 fix(consensus): cap export proposal signatures
Limit outbound TMProposeSet export signature attachments to ExportLimits::maxPendingExports so honest proposals stay within the same bound enforced by inbound proposal validation. Extra exports remain unsigned for that proposal and rely on the existing retry/expiry path.
2026-04-27 21:03:00 +07:00
Nicholas Dudfield
492fe90643 fix(consensus): expire stale export signatures
Stamp export signatures learned from proposals, sidecar sets, and candidate tx-set upgrades with a ledger sequence so cleanupStale can age them out. Remove invalid unverified signatures after tx-local verification fails, with a buffer match check to avoid deleting newer replacements.
2026-04-27 20:56:54 +07:00
Nicholas Dudfield
625419eab7 fix(consensus): verify export sigs against tx set 2026-04-27 18:07:09 +07:00
Nicholas Dudfield
2218bdd7f3 fix(consensus): require export sigset quorum alignment 2026-04-27 17:36:06 +07:00
Nicholas Dudfield
f4e78c9a24 fix(consensus): apply negative unl to sidecar validator view 2026-04-27 12:50:43 +07:00
Nicholas Dudfield
7b5865c69c fix(consensus): sign export proposal attachments 2026-04-27 11:57:29 +07:00
Nicholas Dudfield
9f1ad521e1 fix(consensus): use active validator snapshots for sidecars 2026-04-27 10:59:33 +07:00
Nicholas Dudfield
26bbef8efd fix(consensus): harden sidecar quorum inputs 2026-04-27 10:14:12 +07:00
Nicholas Dudfield
6e71f84867 refactor: add typed sidecar SHAMap sync 2026-04-27 09:58:34 +07:00
Nicholas Dudfield
92ec07a1be chore: regenerate hook/sfcodes.h + format fix
Regenerate sfcodes.h to include the new sfSidecarType field
(UINT8, code 20).  Also apply clang-format to ConsensusExtensions.cpp.
2026-04-10 10:36:50 +07:00
Nicholas Dudfield
664db62588 fix: sidecar kind lost on cache hit + harden export sig parse
1. Record SidecarKind in pendingRngFetches_ before calling
   onAcquiredSidecarSet on local-cache-hit path. Without this,
   cached reveal/exportSig sets silently fell back to commit kind
   and were rejected by the sfSidecarType check.

2. Wrap export sig visitLeaves callback in try/catch (matching the
   RNG path) and enforce sfSidecarType == sidecarExportSig before
   processing — closes the shape-only acceptance gap.
2026-04-10 10:22:58 +07:00
Nicholas Dudfield
03a436d918 refactor: convert sidecar SHAMap entries from STTx to STObject
Replace STTx-based sidecar entries with plain STObject(sfGeneric)
using sfSidecarType (UINT8) discriminator. Eliminates unnecessary
transaction envelope overhead (sfSequence, sfFee, sfFlags) and
content-sniffing heuristics from the parse path.

Build: STObject with sidecarRngCommit/sidecarRngReveal/sidecarExportSig
Parse: sfSidecarType dispatch + typed field accessors
2026-04-10 10:14:06 +07:00
Nicholas Dudfield
7474048295 refactor: typed sidecar dispatch — eliminate content-sniffing heuristic
Replace the content-sniffing heuristic in onAcquiredSidecarSet with
typed dispatch based on SidecarKind.

The type is already known at fetch time:
- commitSetHash → SidecarKind::commit
- entropySetHash → SidecarKind::reveal
- exportSigSetHash → SidecarKind::exportSig

pendingRngFetches_ changes from hash_set<uint256> to
hash_map<uint256, SidecarKind>.  When the set arrives,
look up the kind by hash and dispatch — no leaf inspection.

This is the set-classification fix (Option E from the design doc):
no new SField, no STTx changes, no protocol additions, no RNG
proof-chain churn.
2026-04-10 09:18:43 +07:00
Nicholas Dudfield
f27cd2c567 refactor: consolidate env vars into RuntimeConfig
Move XAHAU_RNG_POLL_MS and XAHAUD_NO_EXPORT_SIG into RuntimeConfig
as rngPollMs and noExportSig fields.  Both are now configurable via
the XAHAU_RUNTIME_CONFIG JSON blob or individual env vars, and
controllable at runtime via the runtime_config RPC.

rngPollMs is clamped to minimum 50ms (prevents tight-loop polling).
Default remains 250ms.

This removes the last loose std::getenv calls from production code
outside of RuntimeConfig.  All env-var-based configuration now flows
through a single system.
2026-04-10 08:24:20 +07:00
Nicholas Dudfield
db302a0f78 fix(rng): add selfSeedReveal to fix CSF reveal counting
The CSF never self-seeded its own reveal into pendingReveals_ because
harvestRngData only processes peer proposals, not self.  The real code
handles this in decorateMessage, but the CSF has no equivalent.

Add selfSeedReveal() called from the tick at reveal transition.
Both the real ConsensusExtensions and the CSF Extensions implement it.
The real code now has belt-and-suspenders: tick + decorateMessage.

This fixes CSF peers having N-1 reveals instead of N, which caused
every peer to compute entropy from a different subset.
2026-04-09 17:23:53 +07:00
Nicholas Dudfield
2a9b1c9c22 fix(export): guard against empty verified sigs + add invariant asserts
- Skip addVerifiedSignature in decorateMessage when sigBuf is empty
  (sign() threw — don't mark a failed sign as "verified")
- Add XRPL_ASSERT in addVerifiedSignature and addUnverifiedSignature
  requiring non-empty signature buffers
- Add XRPL_ASSERT in checkQuorumAndSnapshot verifying that every
  entry in the verified set exists in the signatures map with a
  non-empty buffer
2026-04-09 16:02:35 +07:00
Nicholas Dudfield
462db6004c fix(rng): replace nonexistent leafCount() with std::distance
SHAMap has no leafCount() method — it was a local variable in
SHAMap.cpp, not a public API.  Use std::distance(begin(), end())
on the SHAMap's ForwardRange iterators instead.  Cost is O(n) but
the set is bounded by UNL size (~20-35 entries).
2026-04-09 15:42:04 +07:00
Nicholas Dudfield
cfca708aae fix(rng): remove pendingReveals fallback from entropy output path
shouldZeroEntropy() and sfEntropyCount no longer fall back to
pendingReveals_.  If entropySetMap_ is null, entropy failed — the
pipeline didn't complete, and the map is the only canonical source.

pendingReveals_ is now strictly an internal staging area for the
commit/reveal pipeline.  All final entropy decisions flow through
entropySetMap_, which is the consensus-agreed set.
2026-04-09 15:40:22 +07:00
Nicholas Dudfield
5f70e5259c fix(rng): use entropySetMap for shouldZeroEntropy and sfEntropyCount
The H2 entropy fix switched the digest computation to entropySetMap_
but shouldZeroEntropy() and sfEntropyCount still used pendingReveals_.
Since pendingReveals_ can diverge from the published entropySetMap_
(late reveals mutate it after the map hash is published), two nodes
agreeing on the same entropySetHash could still build different
ttCONSENSUS_ENTROPY pseudo-transactions.

Now shouldZeroEntropy() checks entropySetMap_ leaf count when the map
is available, and sfEntropyCount uses the map's leaf count.  Both
fall back to pendingReveals_ only during pipeline stages before the
map is built.
2026-04-09 15:35:00 +07:00
Nicholas Dudfield
8697c5d821 refactor(export): explicit verified/unverified sig API in collector
Replace the ambiguous addSignature/hasSignature API with clearly
named methods that make verification state explicit:

  addVerifiedSignature()   — sig passed buildMultiSigningData + verify()
  addUnverifiedSignature() — trusted source but no multisign check yet
  addStandaloneSignature() — pubkey-only for standalone/test mode
  hasVerifiedSignature()   — only returns true for verified sigs

Unverified sigs (relay ordering fallback) are no longer treated as
verified by the cache.  When the same sig is encountered again via a
path that CAN verify (e.g. SHAMap merge after the tx arrives), the
verification runs and upgrades it to verified.

addUnverifiedSignature() won't overwrite a verified sig, preventing
downgrade.  SigEntry tracks verified validators in a separate set.
2026-04-09 15:34:13 +07:00
Nicholas Dudfield
9436e5868e fix(export): soften hard reject to best-effort verify for relay ordering
Revert the hard reject when ttEXPORT is not in the open ledger.
Under relay ordering, a node can receive a proposal with export sigs
before the ttEXPORT tx itself arrives.  Dropping these sigs loses a
valid validator contribution for the entire round with no recovery
path until terRETRY_EXPORT on the next round.

Post C1+C2, the proposal-level authentication is sufficient trust:
checkSign() verified the sender holds the private key, and sender
binding verified the embedded pubkey matches.  Store the sig and
let the multisign content be verified on the destination chain.
The collector's stale cleanup (256 ledgers) bounds retention.

When the tx IS in the open ledger (common case), the multisign sig
is still fully verified via buildMultiSigningData + verify().
2026-04-09 15:22:20 +07:00
Nicholas Dudfield
c6fa973cf6 fix(rng): compute entropy from entropySetMap instead of pendingReveals
H2: Compute final entropy from the agreed-upon entropySetMap_ SHAMap
rather than from the local pendingReveals_ in-memory map.

Previously, two nodes with different reveal subsets at timeout would
compute different entropy from their local pendingReveals_ maps,
despite both passing haveConsensus() (which only checks txSetHash).
This could cause a ledger fork.

Now the entropy computation reads directly from the entropySetMap_
whose hash was published in proposals and converged via SHAMap
fetch/merge.  Nodes that agree on entropySetHash deterministically
produce the same entropy regardless of local pendingReveals_ state.

If entropySetMap_ is null (bootstrap skip, pipeline failure), the
existing shouldZeroEntropy() fallback handles it.
2026-04-09 15:18:45 +07:00
Nicholas Dudfield
939e03714c fix(export): cap exportSignatures count per proposal
Reject proposals with more than ExportLimits::maxPendingExports (8)
export sig entries.  Honest validators attach at most one sig per
pending export, bounded by the same limit.  Prevents DoS via
proposals with millions of entries triggering lock contention on
the validator list and collector mutexes.
2026-04-09 15:13:15 +07:00
Nicholas Dudfield
969f98f57e perf(export): skip redundant sig verification via collector lookup
Add hasSignature() to ExportSigCollector — checks if a verified sig
already exists for a given (txHash, validator) pair.  Both the
proposal ingestion path and the SHAMap merge path now check this
before calling verify(), avoiding redundant ed25519 verification
when the same sig arrives via multiple paths.

No external sig cache exists in rippled, so the collector itself
serves as the verification cache: once a sig is stored (always
post-verify), subsequent encounters skip the crypto work.
2026-04-09 15:03:57 +07:00
Nicholas Dudfield
435deb0e78 fix(export): close remaining sig verification gaps
Three fixes from codex review:

1. Remove unsafe fallback in proposal ingestion path: reject export
   sigs when the ttEXPORT tx is not in the open ledger instead of
   storing them unverified.  The tx must be in the open ledger for
   validators to have signed it, so this is not a legitimate case.

2. Add full sig verification to the SHAMap merge path
   (onAcquiredSidecarSet): verify each export sig entry against
   buildMultiSigningData + verify() before storing in the collector.
   Previously this path only checked trusted() on the pubkey,
   allowing a malicious UNL validator to publish a sidecar set with
   forged sigs for other validators.

3. Close cluster mode bypass: always call checkSign() and gate export
   sig harvesting on sigValid, even when cluster() is true.  Cluster
   trust is for relay/resource charging, not for accepting on-chain
   cryptographic artifacts.
2026-04-09 14:59:20 +07:00
Nicholas Dudfield
b80352e512 fix(export): verify multisign signatures at ingestion time
C3: Cryptographically verify each export signature blob against the
inner transaction's signing data before storing in the collector.
Looks up the ttEXPORT tx from the open ledger, reconstructs the
signing data via buildMultiSigningData, and calls verify().

If the tx isn't in our open ledger yet (timing/relay), the sig is
stored unverified as a fallback — it can be verified later at the
SHAMap merge path or will be rejected at Export::doApply if invalid.

This runs on the jtPROPOSAL_t job queue thread (not the IO strand
or transactor), so the verify() cost has no impact on consensus
critical path performance.
2026-04-09 14:43:30 +07:00
Nicholas Dudfield
57c46c61fc fix(export): two-pass sender validation and atomic quorum+snapshot
C2 hardening: validate all export sig blobs before committing any,
preventing partial state if a later blob fails the sender binding
check. Also moves the trusted() check before the loop since senderPK
is constant.

H1: Add checkQuorumAndSnapshot() to ExportSigCollector that performs
the quorum threshold check and signature snapshot under a single lock
acquisition. Export::doApply now uses this instead of separate
signatureCount() + snapshotWithSigs() calls, eliminating the TOCTOU
window where concurrent overlay threads could mutate the collector
between the two operations.
2026-04-09 14:39:35 +07:00
Nicholas Dudfield
37ff13df50 fix(export): move sig harvesting after checkSign and bind pubkey to sender
C1: Move onTrustedPeerMessage() from the synchronous onMessage(TMProposeSet)
handler into checkPropose(), after checkSign() verifies the proposal's
cryptographic signature. Previously, export sigs were ingested before
signature verification, allowing any peer to inject forged sigs by
spoofing nodepubkey to a trusted validator's key.

C2: Add sender binding in onTrustedPeerMessage() — each export sig
blob's embedded validator pubkey must match the proposal sender's
nodepubkey. Reject the entire proposal's export sigs on any mismatch,
preventing a compromised validator from impersonating other validators
to single-handedly forge quorum.
2026-04-09 14:32:38 +07:00
Nicholas Dudfield
724633ceb5 refactor(consensus): decouple CSF tests from xrpld.app via PeerTick.h
Move ConsensusExtensionsTick.h from xrpld/app/consensus/ to
xrpld/consensus/ — it's a pure template with no app-layer deps.
Extract Peer::Extensions::onTick() definition into test/csf/PeerTick.h
so Peer.h no longer includes from xrpld/app/.

Eliminates the test.csf > xrpld.app levelization edge.

Add --explain flag to levelization.py for tracing dependency edges.
2026-03-23 09:36:59 +07:00
Nicholas Dudfield
152d82e798 refactor(consensus): extract RNG/Export into ConsensusExtensions
Extract all Xahau consensus extension logic (RNG commit-reveal entropy
and Export validator multisign collection) from Consensus.h and
RCLCxAdaptor into a dedicated ConsensusExtensions class owned by
Application.

Implements all 10 lifecycle hooks from the design doc:
  onRoundStart, onTrustedPeerMessage, onTrustedPeerProposal,
  decoratePosition, decorateMessage, onTick, onPreBuild,
  onAcceptComplete, isSidecarSet, onAcquiredSidecarSet

Key design decisions:
- ConsensusTick<> template in ConsensusTypes.h keeps dependency
  direction clean (generic consensus defines contract, extensions
  implement it)
- extensionsTick<> shared template in ConsensusExtensionsTick.h
  ensures CSF test framework runs the same state machine as production
- ExportSigCollector ownership moved from global singleton to CE
- Sidecar acquisition routed through RCLConsensus mutex for thread
  safety (isExtensionSet + gotExtensionSet)
- RCLCxAdaptor reduced to thin ce() accessor + generic consensus
  interface methods

Files:
  new: ConsensusExtensions.h/.cpp, ConsensusExtensionsTick.h
  reduced: Consensus.h (-1060 lines), RCLConsensus.cpp (-1400 lines)
  updated: ConsensusTypes.h, Application.h/.cpp, NetworkOPs.cpp,
           PeerImp.cpp, Export.cpp, ExportSigCollector.h, Peer.h

7/7 testnet scenarios + 1463 consensus + 260 Export unit tests pass.
2026-03-19 20:23:19 +07:00