Use the agreed exportSigSetHash sidecar map, not the live ExportSigCollector, as the network-mode ttEXPORT signer snapshot. Preserve export convergence state through RNG onPreBuild cleanup so buildLCL apply can see the gate result, and require a UNLReport-backed validator view before non-standalone Export finalization. Add regression coverage for agreed-vs-live collector mutation and onPreBuild export-state preservation.
A drift review (10-section adversarial pass + skeptical verification) found the design doc had fallen behind two recent code changes: the UNLReport-only gate (8368e12ab) and the F1 active-view alignment filter (fb9e2710c).
- Fallback Semantics: add the second fallback trigger -- a non-UNLReport-backed view mints consensus_fallback regardless of alignment.
- Validator Set And Quorum: note the config-fallback view is !fromUNLReport and forces consensus_fallback entropy.
- Entropy Alignment Rules: state the UNLReport precondition for non-fallback tiers; correct the alignment-count formula to the active-view-filtered count (non-active proposers and a non-active local +1 excluded -- the F1 fix); separate the two distinct counts (the peer-alignment GATE count vs the agreed entropySetMap_ leaf count that drives the tier LABEL); flag the explicit-final unfiltered exception.
- Worked examples: note they assume a UNLReport-anchored view (else all fall back).
- Export Principles: document that export success also requires full observation (peersSeen == txConverged), not just quorum alignment.
- Review Checklist: add items for the UNLReport-anchored-tier and alignment-count-universe (F1) invariants.
Doc-only. Raw drift findings kept in .ai-docs (gitignored).
The explicit-final proposal path counts participants/alignment over the unfiltered trusted-proposer set (ctx.peerPositions) plus an unconditional local +1, not the active validator view -- the same gap the F1 fix closed for the main entropy gate (inspectTxConvergedSidecarPeers). It is default-off/experimental with no robust timing model found (per the existing TBD, may never ship); if it is ever enabled it must apply the same active-view membership filter. Comment-only.
Follow-up to F1 (fb9e2710c).
testInvalidEntropyRequirements rejects four invalid dice/random requirements then returned accept(0,0,0) on success. But a valid dice(6,..) returns 0..5, so a regression that let the min_tier=0 requirement through (returning a value that happened to be 0) would still pass ~1/6 of the time -- the weakest spot is exactly the min_tier lower bound it most needs to prove (review finding). Return sentinel 42 (distinct from any dice/random result and from INVALID_ARGUMENT) and assert ==42, so any leaked requirement returns its own non-42 code and fails. WASM block recompiled; ConsensusEntropy 138 tests, 0 failures.
Follow-up to F1 (fb9e2710c).
The entropy/export bless-vs-fallback gate counted alignment over ALL tx-converged trusted proposers (currPeerPositions_), while the thresholds and the entropy leaf set are computed over the active validator view. A node that locally trusts proposers outside the on-ledger UNLReport active set could pad alignedParticipants(), inflating the counting universe N above originalViewSize and eroding the Tier-2 intersection margin (2t - N) below the Byzantine floor f -- so two equivocation cohorts padded by non-active aligners could each clear the gate (review finding F1). Backstopped by the 80% validation quorum, but the proof's universe and the code's universe must match.
inspectTxConvergedSidecarPeers now takes an active-view membership predicate and counts only member peers toward alignment; the local +1 is gated on localIsActiveValidator(). Both the RNG entropy gate and the export-sig gate pass ext.isUNLReportMember / ext.localIsActiveValidator, mirroring buildEntropySet / hasQuorumOfCommits' containsNode filter. New method on ConsensusExtensions + CSF Peer + the FakeExtensions stub.
Regression test (Sidecar peer alignment helper): a trusted-but-non-active aligned proposer pushes unfiltered aligned to 2 but filtered aligned stays 1, and a non-active local node's +1 is suppressed -- padding cannot satisfy the gate. ConsensusExtensions 900, ConsensusRng 386, Consensus 1399, all green.
Note: the explicit-final proposal path (ConsensusExtensionsTick.h ~:934) counts over prevProposers and is NOT yet filtered (separate, experimental path).
The tier-2 anchor pin (20d52d8b6) used an 8/6 (original/effective) nUNL config whose rationale was wrong: it computed cohort overlap against the original view (2*4-8=0), but aligned cohorts form in the EFFECTIVE view, so two 4-of-6 cohorts overlap by 2*4-6=2 > floor(8/5)=1 -- 8/6 is NOT actually forkable, the original-view anchor is merely conservative there. The test still caught the originalViewSize->size() regression, but the stated reason was misleading (review catch).
Switch to 10 active / 2 disabled (original 10, effective 8), a genuinely forkable case: a 5-of-8 cohort (what the effective-size threshold would admit) overlaps only 2*5-8=2, which does NOT exceed the f=floor(10/5)=2 faulty the original UNL still tolerates -> an equivocator could mint two distinct tier-2 digests. The correct original anchor requires 7 (overlap 6 > 2) and keeps the band closed (tier2==quorum==7); a regression to size() drops the floor to 5 and re-opens the forkable [5,7) band. Assertions and rationale updated to match; suite green (897, 0 failures).
tier2Threshold() anchors to originalViewSize (pre-nUNL), not the effective post-nUNL size(): nUNL shrinks the effective view while leaving faulty nodes in it, so a sub-quorum fraction of the effective view can exceed the Byzantine bound. A one-line regression to size() would relabel a forkable cohort as participant_aligned -- and no existing test caught it (the only nUNL test never calls tier2Threshold()/selectEntropy, and the only tier-2 selection test has no nUNL, so original == effective there).
Add testTier2ThresholdAnchorsToOriginalView: 8 active validators, 2 disabled via NegativeUNL (original 8, effective 6), asserting tier2Threshold()==5 (from the original 8) not 4 (from the effective 6), with quorum/gate cross-checks. Mutation-verified: flipping the production read to size() turns exactly this test red (tier2Threshold + entropyGateThreshold) while the existing tier-2 selection test stays green.
Also: pointer comment at tier2Threshold() linking the invariant to its guard test, and a design-doc fix -- the worked example used n=5, where the band is empty (quorum == participant_aligned == 4) so it illustrated an unreachable tier; now n=6.
The 5/6 phase asserted validator_quorum on whichever single ledger the validated tip happened to sit on. But the ledger right at the node-5 drop can be a transient consensus_fallback (EntropyTier=1, count=0) — deterministic and by design, while the commit/reveal pipeline re-primes — so when the tip landed there the assert failed (tier=1, observed ~1 run in 4). Same class of bug as the old post-recovery flake: depending on exactly where the tip lands.
Now settle 4 ledgers past the drop, then scan the post-drop validated ledgers for a clean validator_quorum (tier 3, count >= quorum). The window is entirely 5-node cohorts, so a tier-3 there has count == 5 (faithful to '5/6 still tier 3'); the transition fallback is tolerated, not asserted on. Verified 3/3: each run found tier=3 count=5.
Also harden _closed_entropy() to raise on != 1 ConsensusEntropy pseudo-tx (mirroring get_entropy_tx) instead of silently skipping — a duplicate/missing injection now fails with a clear error rather than resurfacing as a generic 'no tier-2 ledger'.
Codex review minors: assert_validator_quorum at the 5/6 boundary (EntropyTier=3 AND count >= quorum AND non-zero digest, not just tier==3 — catches a bad tier/count pairing); added explicit assert_validator_quorum / assert_consensus_fallback helpers and an entropy_fields() warning that its is_fallback (tier != 3) lumps participant_aligned in with fallback (safe only where no tier-2 band exists).
Robustness: tier 2 is below the validation quorum, so the validated tip stalls and which provisional ledgers it later reaches is timing-dependent (the post-recovery inspection was fragile). Replaced it with direct inspection of the surviving cohort's CLOSED ledger via ledger('closed') DURING the window; recovery is now a pure liveness check.
Verified on a live testnet: PASS — EntropyTier=2 count=4 confirmed on 4 distinct provisional ledgers (seq 7-10).
A 6-node smoke (the smallest NON-degenerate tier-2 size: tier2 floor 4, quorum 5) driving the 4/6 band. n=5 has no band (tier2 == quorum), which is why the existing degradation smoke only ever sees tier 3 / fallback.
Tier 2 is below the 80% validation quorum, so the 4/6 cohort's ledgers are provisional: the scenario confirms tier-2 injection from the cohort's logs during the window, then verifies the on-ledger EntropyTier=2 count=4 POST-RECOVERY once those ledgers become canonical and validate (the mechanism the degradation smoke also relies on). Adds the assert_participant_aligned helper and the node_count:6 suite entry.
Verified on a live testnet: PASS — 12 tier-2 injections in the 4/6 window, seq 7 & 8 validated as participant_aligned (count 4).
Codex final-review minors, both comment-only (no behavior change): EntropyTier participant_aligned no longer says "reserved for a future" tier (it is implemented in this stack); and the commit-timeout path comment said "fixed UNL quorum" but the code now uses entropyGateThreshold() (min(quorum, tier2)).
Codex final-review minor (optional hardening): finalizeRoundEntropy now requires the fetched sidecar entry to be reveal-type before counting it as entropy. lastEntropySetHash_ only ever names a reveal set (hashRngSet's per-type salt rules out a cross-type hash collision), so this is purely defensive — behavior unchanged, all CSF sims green.
Codex review caught a CSF/production fidelity gap: production injects entropy from the AGREED entropySetMap_ (selectEntropy), which is frozen at advertise time — late-fetched or conflicting reveals merge into pendingReveals_ but are NOT injected unless a rebuild republishes the hash. The CSF peer fetched into pendingReveals_ AND finalized from pendingReveals_, so a conflict/fetch sim could count reveals production would never inject from.
Track the last advertised entropy-set hash (buildEntropySet) and finalize from the sidecar-store snapshot under that hash — the analog of the frozen entropySetMap_. Clean/no-conflict sims are unchanged (the snapshot equals pendingReveals_ there); the model is now faithful for conflict/fetch cases too. Production unaffected (test-harness only). Addresses finding 1 of codex-tier2-final-review.
6 validators is the smallest non-degenerate tier-2 size (f=1, one-wide band {4}: tier2=4, quorum=5). Isolate 2 so the surviving 4-cohort is below the 80% quorum but at the tier-2 floor; it mints participant_aligned entropy (count 4), and all four agree on the same non-zero digest with branches==1 — no hang, no fork. Distributed confirmation of the selector ladder the unit tests already cover.
The CSF peer now mirrors production's tier-2 behavior so the RNG simulations
exercise participant_aligned, not just validator_quorum/fallback:
- entropyGateThreshold() = min(quorumThreshold(), tier2Threshold()), plus a
tier2Threshold() helper (calculateParticipantThreshold over the sim's UNL).
- finalizeRoundEntropy() labels by aligned count via the 3-tier ladder
(>= quorum -> validator_quorum, >= tier2 -> participant_aligned, else
fallback) instead of hardcoding tier 3.
The n%5==0 band-collapse keeps the bulk of the suite unaffected: at n=5
tier2 == quorum == 4, so the gate is unchanged for every n=5 and n<=2 network.
Only the two n=3 sub-quorum sims cross a live band (n=3: tier2=2 < quorum=3) and
now mint tier 2 instead of falling back -- the feature working, with each test's
intent preserved:
- "impossible quorum fallback" -> "quorum-impossible cohort falls to
participant_aligned": 2 of 3 is below 80% but at the tier-2 floor, so it makes
progress as tier 2 (no hang, no fork) rather than falling all the way back.
- "persistent loss does not shrink quorum": the 2 survivors now mint the
labeled-weaker tier 2, NOT tier 3 -- the tier-3 quorum still did not shrink
(a min_tier=3 hook rejects it). Intent preserved, outcome relabeled.
Other CSF suites (Consensus, ByzantineFailureSim) are unaffected (they do not
enable RNG). Next: dedicated tier-2 mint + conflict sims at n=6 (smallest size
with a non-degenerate band, f=1).
Two small follow-ups from the Codex review of the tier-2 implementation, both
on the consensus-extension internals:
- Rename shouldZeroEntropy() -> belowValidatorQuorum(). The selector replaced its
callers, leaving it production-dead and MISNAMED: post-tier-2, "below the 80%
validator quorum" is no longer "zero entropy" (a participant_aligned set is
sub-quorum but non-zero). The new name + a doc comment make it a tier-3
eligibility predicate only and warn against gating injection on it
(selectEntropy() owns the tiering). Behavior unchanged.
- buildExplicitFinalProposalTxSet now dedups the entropy pseudo-tx by VALUE, not
type. Its comment claimed it "mirrors onPreBuild", but onPreBuild went
value-based: it verifies a present pseudo-tx is the EXACT txID it would have
produced and logs a determinism-violation on mismatch. Explicit-final now does
the same (skip-duplicate-verified on match, error log on mismatch), returning
the base unchanged either way. Default-off experimental path, so low blast
radius, but the two paths now agree.
calculateParticipantThreshold returned ceil(0.6*n). At every n divisible by 5
(n=5,10,15,20,...) that leaves two aligned cohorts overlapping in exactly
floor(0.2*n) = f validators -- NOT strictly greater than f. So up to f Byzantine
nodes can occupy the entire overlap, leaving no honest validator shared between
the two cohorts, and a single equivocator backed by f-1 colluders can split the
round into two distinct aligned digests -> fork. The spec caught n=5 ("never 5")
but the same failure recurs at every multiple of 5.
Derive the floor from the safety invariant instead: the smallest t with
2t - n > floor(n/5), i.e. floor((n + floor(n/5)) / 2) + 1. This equals
ceil(0.6*n) everywhere except multiples of 5, where it is one higher
(n=10 -> 7, not 6). n=5 now collapses the band (tier2 == quorum), so the
"never 5" operational caveat is enforced by the math rather than a footnote.
Tests: the arithmetic test now asserts the defining invariant
2t - n > floor(n/5) AND t <= quorum for every n in 1..256 (this fails at n=10
under the old formula), plus the bumped boundaries. The onPreBuild tier-2 test
moves off n=5 (no band) onto n=6 (tier2=4, quorum=5; 5/4/3 revealers ->
validator_quorum / participant_aligned / fallback).
Found by Codex adversarial review of the tier-2 implementation.
A 60-79% aligned cohort now mints entropy labelled participant_aligned (tier 2)
instead of falling back. Healthy >=80% rounds are unchanged (validator_quorum),
and a true minority (<60%) still falls back.
Mechanics:
- entropyGateThreshold() = min(quorumThreshold(), tier2Threshold()): the bar at
which the pipeline engages and the entropy conflict gate resolves. In the
normal band this is the 60% floor (of the ORIGINAL, pre-nUNL view); under
heavy nUNL the band collapses to the 80% quorum and tier 2 vanishes.
- selectEntropy() labels the AGREED entropySetMap_ by participant count:
>= quorum -> validator_quorum, >= tier2 -> participant_aligned, else fallback.
- Tick.h gates (bootstrap-skip, impossible-quorum, commit-timeout, entropy
conflict gate) key off entropyGateThreshold() so sub-quorum rounds reach
injection instead of short-circuiting.
Determinism: the tier LABEL is a function of the agreed set's leaf count
(identical on every node holding that hash); the local entropyGateThreshold
alignment only decides proceed-vs-fall-back, so divergent local views fall back
rather than fork. The 60% floor is over the ORIGINAL view -- the
quorum-intersection bound that stops a single equivocator minting two distinct
aligned digests under the ~20% Byzantine bound.
hasQuorumOfCommits() is deliberately LEFT at 80%: healthy networks keep their
exact fast-path and only step down to tier 2 via the commit-timeout path, so
this adds zero behavior change above quorum (the degraded band pays one pipeline
timeout; a fast-path is a separable follow-up). Folds into featureConsensusEntropy
(not yet active), so no new amendment.
Tests: new onPreBuild tier-2 case (5-validator view; 4/3/2 revealers ->
validator_quorum / participant_aligned / fallback) plus threshold assertions.
CSF Peer and the tick test stub mirror entropyGateThreshold() to quorumThreshold()
for now -- the end-to-end tier-2 sims (which lower it) land next. Extracted
makeUNLReportLedger / harvestCommitReveal test helpers, now shared across the
view and harvest tests.
Collapse the duplicated tier-selection logic in onPreBuild and
buildExplicitFinalProposalTxSet into a single selectEntropy() over the AGREED
entropySetMap_. No behavior change on the production (implicit) path — the
onPreBuild fallback/entropy-set/standalone/mismatch tests pass byte-identically.
Fixes the pre-existing divergence flagged in review: buildExplicitFinalProposalTxSet
derived entropy from local pendingReveals_ while onPreBuild used the agreed
entropySetMap_, so the two could mint different digests for the same round.
Both now share the selector, so the implicit and (experimental, default-off)
explicit-final paths — and any two nodes — derive identical entropy from
identical agreed inputs.
selectEntropy() returns {digest, tier, count} and is a pure function of agreed
round state, so it is directly unit-testable. Injection becomes unconditional:
the selector always yields a fallback digest when there is no validator entropy,
so every RNG-enabled ledger still carries exactly one ConsensusEntropy tx.
Sets up the tier-2 (participant_aligned) ladder, which lands next.
Foundation for Tier 2 (participant_aligned) sub-quorum entropy. No behavior
change — nothing consumes these yet; the tier ladder, gates, and selector
arrive in later commits.
- ActiveValidatorView::originalViewSize: master-key count BEFORE the nUNL
subtraction. size() stays the effective (post-nUNL) count used by the 80%
validator-quorum gate; originalViewSize is the original-UNL denominator that
the 60% Tier 2 floor anchors to, since nUNL can shrink the effective view
while leaving faulty nodes in it.
- calculateParticipantThreshold(): ceil(0.6 * count), the quorum-intersection
floor (two such cohorts always share an honest validator under the ~20%
Byzantine bound).
- ConsensusExtensions::tier2Threshold(): ceil(0.6 * originalView), anchored to
originalViewSize.
Tests: originalViewSize asserted on the UNLReport, fallback, and real-ledger
nUNL paths; new arithmetic testcase locks ceil(0.6) and the sizing-note
boundaries (5->3 banded/unsafe, 6->4 smallest-safe, 8->5 one-nUNL-off).
The 'Tier N' preference-order shorthand collides with EntropyTier's
strength-ordered values (consensus_fallback=1). Use the enum names in
the degradation-smoke comments/descr to remove the ambiguity.
Follow-up to d6481a386, addressing review of that diff:
- The mismatch-log branch in onPreBuild read present pseudo-tx fields
(sfDigest/sfEntropyTier/sfEntropyCount) unconditionally — the same
throw hazard fairRng was hardened against, here inside onPreBuild
during build. Read them defensively (isFieldPresent ? value :
'<missing>'). (Note: STTx deserialization enforces all soeREQUIRED
fields, so a tier-less ttCONSENSUS_ENTROPY cannot actually reach the
set — this is belt-and-suspenders, not a reachable bug.)
- Clarify the comment + add action=keep-agreed-and-flag: this is
detect-and-log, NOT rejection. The present pseudo-tx is KEPT and still
applied at BuildLedger; a hard-fail policy on mismatch is a deliberate
future decision (determinism-violation vs halt-risk-under-skew).
- Add testOnPreBuildEntropyMismatchKeepsAgreed: a present-but-different
entropy pseudo-tx is kept (not replaced), set stays at one entry.
- Fix stale 'type-based dedup' comment in the standalone test.
Address review findings on the tier 3 commits:
- onPreBuild dedup was type-based (skip if any ttCONSENSUS_ENTROPY
present), which silently trusts a pre-present pseudo-tx. Injection is
deterministic, so every honest node derives the identical pseudo-tx
(identical txID) for the same agreed inputs. Switch to value-based
dedup: skip only when the present pseudo-tx EQUALS the one we would
produce; a present-but-different entropy pseudo-tx is a determinism
violation (version skew / divergent peer) and is now logged at error
rather than accepted blindly. (The earlier 'cannot reconstruct the
txID locally' justification was wrong — determinism guarantees it can.)
- fairRng: read sfEntropyTier defensively (missing => 0 => fail closed).
The field is soeREQUIRED so any entry this code wrote carries it; this
only guards a pre-tier-3 persisted entry on a long-lived testnet.
- quorum_degradation_smoke: assert EntropyTier==consensus_fallback(1) and
EntropyCount==0 and non-zero digest explicitly, not just is_fallback.
The entropy-quality requirement becomes an explicit, required argument
at every call site — there is deliberately no default and no network
constant:
- dice(sides) -> dice(sides, min_tier, min_count)
- random(ptr, len) -> random(ptr, len, min_tier, min_count)
- fairRng gates on freshness && tier >= min_tier && count >= min_count;
the hard-coded 'EntropyCount >= 5' network constant is deleted —
the hook author states what their application needs and gets
TOO_LITTLE_ENTROPY when this ledger cannot meet it
- WASM imports have no default parameters, so the old no-argument shape
was really a hidden constant invisible at the call site; requiring the
arguments makes weak-entropy acceptance (min_tier=1) a deliberate,
reviewable opt-in rather than an accident
- hook/extern.h and hook/sfcodes.h regenerated (CI-verified)
- test hooks updated and recompiled; new test pins the requirement
gate: dice(6, 3, 21) against count=20 returns TOO_LITTLE_ENTROPY
Replace the zero-entropy fallback with a deterministic consensus-bound
digest so every RNG-enabled ledger carries usable entropy:
- sha512Half(HashPrefix::entropyFallback, prevLedgerHash, baseTxSetHash,
seq) — all inputs are already consensus-agreed at injection time, so
no new agreement machinery is needed and the digest is identical on
every node building the same ledger
- new sfEntropyTier (UINT8) on the ttCONSENSUS_ENTROPY pseudo-tx and
ConsensusEntropy ledger entry: EntropyCount says how many validators
contributed, EntropyTier says which gate the result passed
(validator_quorum vs consensus_fallback; participant_aligned reserved)
- the fallback digest derives from the BASE (pre-injection) tx set hash
to avoid circularity; entropy pseudo-tx dedup is now type-based since
an explicit-final synthetic set can carry a pseudo-tx whose txID
implicit nodes cannot re-derive
- unparseable-entropy-set residual now falls back instead of skipping
injection, so a fresh ConsensusEntropy entry exists every ledger
- CSF Peer mirrors the fallback analog; sims assert deterministic
non-zero fallback digests across same-LCL peers
- testnet scenarios updated: degraded windows expect labeled fallback
entropy, never validator-tier
The fallback tier is user-influenceable via tx submission (quiet-ledger
grinding) and is labeled accordingly — hook-facing gating lands with the
min_tier/min_count API change.
