Port the Hook API surface from the tt-rng branch, adapted to use our
commit-reveal consensus entropy (ltCONSENSUS_ENTROPY / sfDigest).
Hook APIs:
- dice(sides): returns random int [0, sides) from consensus entropy
- random(write_ptr, write_len): fills buffer with 1-512 random bytes
Internal fairRng() derives per-execution entropy by hashing: ledger
seq + tx ID + hook hash + account + chain position + execution phase
+ consensus entropy + incrementing call counter. This ensures each
call within a single hook execution returns different values.
Quality gate: fairRng returns empty (TOO_LITTLE_ENTROPY) if fewer
than 5 validators contributed, preventing weak entropy from being
consumed by hooks.
Also adds sfEntropyCount and sfLedgerSequence to the consensus
entropy SLE and pseudo-tx, enabling the freshness and quality
checks needed by the Hook API.
setExpectedProposers() now filters incoming proposers against the
on-chain UNL Report, preventing non-UNL nodes from inflating the
expected set and causing unnecessary timeouts.
quorumThreshold() uses expectedProposers_.size() (recent proposers ∩
UNL) when available, falling back to full UNL Report count on cold
boot. This adapts to actual network conditions rather than relying
on a potentially stale UNL Report that over-counts offline validators.
Renamed activeUNLNodeIds_/cacheActiveUNL/isActiveUNLMember to
unlReportNodeIds_/cacheUNLReport/isUNLReportMember to make the
on-chain data source explicit.
When expected proposers don't all arrive before rngPIPELINE_TIMEOUT,
check if we still have quorum (80% of UNL). If so, build the commitSet
with available commits and continue to reveals. Only fall back to zero
entropy when truly below quorum.
Previously any missing expected proposer caused a full timeout with zero
entropy for that round. Now: kill 3 of 20 nodes → one 3s timeout round
per kill but entropy preserved (17/16 quorum met).
Wait for commits from last round's proposers (falling back to activeUNL
on cold boot) instead of 80% of UNL. This ensures all nodes build the
commitSet at the same moment with the same entries.
Split proof storage: commitProofs_ (seq=0 only, deterministic) and
proposalProofs_ (latest with reveal, for entropySet). Previously the
proof blob contained whichever proposeSeq was last seen, causing
identical commits to produce different SHAMap hashes across nodes.
20-node testnet: all nodes now produce identical commitSet hashes.
Previously rngPIPELINE_TIMEOUT (3s) was measured from round start,
meaning txSet convergence could eat into the reveal budget. Now reveals
get their own rngREVEAL_TIMEOUT (1.5s) measured from the moment we
enter ConvergingReveal, ensuring consistent time for reveal collection
regardless of how long txSet convergence took.
- Change hasMinimumReveals() to wait for reveals from ALL committers
(pendingCommits_.size()) instead of 80% quorum. The commit set is
deterministic, so we know exactly which reveals to expect.
rngPIPELINE_TIMEOUT remains the safety valve for crash/partition.
Fixes reveal-set non-determinism causing entropy divergence on
15-node testnets.
- Resource manager: preserve port for loopback addresses so local
testnet nodes each get their own resource bucket instead of all
sharing one on 127.0.0.1 (causing rate-limit disconnections).
- Make RNG fast-poll interval configurable via XAHAU_RNG_POLL_MS
env var (default 250ms) for testnet tuning.
When fewer participants are present than the quorum threshold, skip the
RNG commit wait immediately instead of waiting the full pipeline timeout.
Also base the quorum on activeUNLNodeIds_ (UNL Report with fallback)
instead of the full trusted key set, so the denominator reflects who is
actually active on the network.
Add rngPIPELINE_TIMEOUT (3s) to replace ledgerMAX_CONSENSUS (10s) in
the commit/reveal quorum gates. Late-joining nodes enter as
proposing=false and cannot contribute commitments until promoted, so
waiting beyond a few seconds just delays the ZERO-entropy fallback and
penalizes recovery. Add inline comments documenting the late-joiner
constraint and SHAMap sync's role as a dropped-proposal safety net.
Prevent grinding attacks by verifying sha512Half(reveal, pubKey, seq)
matches the stored commitment before accepting a reveal. Also move
cacheActiveUNL() into startRound so non-proposing nodes (exchanges,
block explorers) correctly accept RNG data instead of diverging with
zero entropy.
During ConvergingTx all RNG data arrives via proposal leaves, so
fetching a peer's commitSet before we've built our own just generates
unnecessary traffic. Only fetch commitSetHash once in ConvergingCommit+,
and entropySetHash once in ConvergingReveal.
Prevents spoofed SHAMap entries by embedding verifiable proof blobs
(proposal signature + metadata) in each commit/reveal entry via sfBlob.
- Store ProposalProof in harvestRngData (peers) and propose() (self)
- serializeProof: pack proposeSeq/closeTime/prevLedger/position/sig
- verifyProof: reconstruct signingHash, verify against public key
- Embed proofs in buildCommitSet/buildEntropySet via sfBlob field
- Verify proofs in handleAcquiredRngSet (both diff and visitLeaves paths)
- Add stall fix: gate ConvergingTx on timeout when commits unavailable
- Clear proposalProofs_ in clearRngState
Cache active UNL NodeIDs per round from UNL Report (in-ledger),
falling back to getTrustedMasterKeys() on fresh testnets.
Reject non-UNL validators at all entry points: harvestRngData,
buildCommitSet, buildEntropySet, and handleAcquiredRngSet.
Build real ephemeral (unbacked) SHAMaps for commitSet and entropySet using
ttCONSENSUS_ENTROPY entries with tfEntropyCommit/tfEntropyReveal flags.
Reuse InboundTransactions pipeline for peer fetch/diff/merge with no new
classes. Encode NodeID in sfAccount to avoid master-vs-signing key mismatch.
Add isPseudoTx guard in ConsensusTransSetSF to prevent pseudo-tx submission.
Route acquired RNG sets via isRngSet/gotRngSet in NetworkOPs mapComplete.
During ConvergingCommit and ConvergingReveal sub-states, poll at 250ms
instead of the default 1s ledgerGRANULARITY. This reduces total RNG
pipeline overhead from ~3s to ~500ms while keeping the normal heartbeat
interval unchanged for all other consensus phases.
Three critical fixes that unblock the RNG commit-reveal pipeline:
- Remove entropy secret regeneration in ConvergingTx->ConvergingCommit
transition that was overwriting the onClose() secret, breaking reveal
verification against the original commitment
- Change ExtendedPosition operator== to compare txSetHash only, preventing
deadlock where nodes transitioning sub-states at different times would
break haveConsensus() for all peers
- Self-seed own commitment and reveal into pending collections so the
node counts toward its own quorum checks
Also adds ExtendedPosition_test with signing, suppression, serialization
round-trip and equality tests, iterator safety fix in BuildLedger, wire
compatibility early-return, and RNG debug logging throughout the pipeline.
- Implement injectEntropyPseudoTx() to combine reveals into final
entropy hash and inject as pseudo-tx into CanonicalTXSet in doAccept()
- Modify BuildLedger applyTransactions() to apply entropy tx FIRST
before all other transactions to prevent front-running
- Remove redundant explicit threading in applyConsensusEntropy() as
sfPreviousTxnID/sfPreviousTxnLgrSeq are set automatically by
ApplyStateTable::threadItem()
- Register ttCONSENSUS_ENTROPY in applySteps.cpp dispatch tables
(preflight, preclaim, calculateBaseFee, apply)
- Add ltCONSENSUS_ENTROPY to InvariantCheck.cpp valid type whitelist
Add protocol definitions for consensus-derived entropy pseudo-transaction:
- ttCONSENSUS_ENTROPY = 105 transaction type
- ltCONSENSUS_ENTROPY = 0x0058 ledger entry type
- keylet::consensusEntropy() singleton keylet (namespace 'X')
- applyConsensusEntropy() handler in Change.cpp
- Added to isPseudoTx() in STTx.cpp
The entropy value is stored in sfDigest field of the singleton ledger object.
This provides the protocol foundation for same-ledger entropy injection.
- Add clearRngState() call in startRoundInternal
- Reset estState_ in closeLedger when entering establish phase
- Implement three-phase RNG checkpoint gating:
- ConvergingTx: wait for quorum commits, build commitSet
- ConvergingCommit: reveal entropy, transition immediately
- ConvergingReveal: wait for reveals or timeout, build entropySet
- Use if constexpr for test framework compatibility
- Serialize full ExtendedPosition in share() and propose()
- Deserialize ExtendedPosition in PeerImp using fromSerialIter()
- Add harvestRngData() to collect commits/reveals from peer proposals
- Conditionally call harvest via if constexpr for test compatibility
Introduce data structures for consensus-derived randomness using
commit-reveal scheme:
- Add ExtendedPosition struct with consensus targets (txSetHash,
commitSetHash, entropySetHash) and pipelined leaves (myCommitment,
myReveal)
- operator== excludes leaves to allow convergence with unique leaves
- add() includes ALL fields to prevent signature stripping attacks
- Add EstablishState enum for sub-phases: ConvergingTx, ConvergingCommit,
ConvergingReveal
- Update Consensus template to use Adaptor::Position_t
- Add Position_t typedef to RCLConsensus::Adaptor and test CSF Peer
This is the foundational data structure work for the RNG implementation.
The gating logic and entropy computation will follow.
