Commit Graph

14404 Commits

Author SHA1 Message Date
Nicholas Dudfield
92bdd2ed9f fix(consensus): harvest replayed export signatures 2026-04-28 10:16:52 +07:00
Nicholas Dudfield
d87cfdc604 fix(consensus): clear export sigs when export disabled 2026-04-28 09:17:17 +07:00
Nicholas Dudfield
a956abb2d1 docs(consensus): clarify export sig quorum gates 2026-04-28 08:56:53 +07:00
Nicholas Dudfield
aa36a80ab7 docs(consensus): document sidecar acquire rationale 2026-04-28 08:35:45 +07:00
Nicholas Dudfield
e729aa11eb fix(hooks): preserve finalization semantics
Keep hook result/state finalization non-fatal while enforcing the hook-export backlog cap through the transaction-level ApplyContext guard. This avoids resetting non-success tec metadata and preserves hook_again weak execution behavior.
2026-04-28 07:47:20 +07:00
Nicholas Dudfield
c58da3da58 fix(export): cap hook export backlog
Enforce the pending export cap for hook-emitted ttEXPORT work before commit. Replace the non-present sfEmittedTxn template field when building ltEMITTED_TXN entries so in-flight ledger checks see the emitted wrapper.

Overflowing xport emission now returns tecDIR_FULL and leaves the emitted backlog capped at ExportLimits::maxPendingExports.
2026-04-27 22:55:23 +07:00
Nicholas Dudfield
0c2c59d258 fix(export): enforce pending export limits
Cap pending ttEXPORT work in open/apply ledgers, including hook-emitted exports when TxQ drains the emitted directory into the open ledger. Enforce the same bound for per-account shadow tickets so durable pending imports cannot grow unbounded.
2026-04-27 21:30:24 +07:00
Nicholas Dudfield
15662eb1b1 fix(consensus): cap export proposal signatures
Limit outbound TMProposeSet export signature attachments to ExportLimits::maxPendingExports so honest proposals stay within the same bound enforced by inbound proposal validation. Extra exports remain unsigned for that proposal and rely on the existing retry/expiry path.
2026-04-27 21:03:00 +07:00
Nicholas Dudfield
492fe90643 fix(consensus): expire stale export signatures
Stamp export signatures learned from proposals, sidecar sets, and candidate tx-set upgrades with a ledger sequence so cleanupStale can age them out. Remove invalid unverified signatures after tx-local verification fails, with a buffer match check to avoid deleting newer replacements.
2026-04-27 20:56:54 +07:00
Nicholas Dudfield
ea413873b2 fix(consensus): preserve export state without rng 2026-04-27 20:34:58 +07:00
Nicholas Dudfield
625419eab7 fix(consensus): verify export sigs against tx set 2026-04-27 18:07:09 +07:00
Nicholas Dudfield
2218bdd7f3 fix(consensus): require export sigset quorum alignment 2026-04-27 17:36:06 +07:00
Nicholas Dudfield
f13233b00a docs(consensus): clarify validation sidecar signing rule
Remove the stale TMValidation exportSignatures field from the draft proto path now that export signatures ride signed proposal sidecars. Document that any future validation-carried ConsensusExtensions data must be covered by the signed validation payload and duplicate/replay identity, not an unsigned wrapper field.
2026-04-27 15:45:27 +07:00
Nicholas Dudfield
a61f334ca2 docs(consensus): capture extension design principles
Document the consensus-extension invariants for RNG, sidecars, export sig convergence, validator quorum, zero-entropy fallback, and proposal signing. Link the note from the RCL consensus README so future changes have a durable checklist.
2026-04-27 15:33:15 +07:00
Nicholas Dudfield
53a119ce30 fix(consensus): require rng entropy quorum alignment
Count the local proposer when deciding whether the previous round had enough participants for RNG, since prevProposers only tracks peers. This avoids a 4/5 honest quorum being treated as below quorum after one validator diverges.

Allow an already quorum-aligned entropySetHash to proceed despite below-quorum conflicting hashes, while retaining zero-entropy fallback when no entropy hash reaches quorum alignment. Add CSF coverage for a persistent single bogus entropy hash and for conflicting bogus hashes without quorum.
2026-04-27 15:29:36 +07:00
Nicholas Dudfield
63d1197345 fix(consensus): zero rng on unresolved entropy hash conflict 2026-04-27 15:10:39 +07:00
Nicholas Dudfield
aafd5b940b test(consensus): avoid brittle rng lcl quorum check 2026-04-27 14:47:18 +07:00
Nicholas Dudfield
efc497cf23 chore(levelization): refresh app overlay loop summary
This does not introduce a new levelization cycle; the existing xrpld.app <-> xrpld.overlay loop now has equal aggregate include counts after the consensus-extension work. Treat this as essentially the same architectural situation, not a meaningful worsening by itself.

TODO: if we want to fix the boundary properly, extract a small shared consensus-extension wire/interface layer below both app and overlay instead of shaving includes to change the generated ratio.
2026-04-27 14:01:54 +07:00
Nicholas Dudfield
f4e78c9a24 fix(consensus): apply negative unl to sidecar validator view 2026-04-27 12:50:43 +07:00
Nicholas Dudfield
7b5865c69c fix(consensus): sign export proposal attachments 2026-04-27 11:57:29 +07:00
Nicholas Dudfield
9f1ad521e1 fix(consensus): use active validator snapshots for sidecars 2026-04-27 10:59:33 +07:00
Nicholas Dudfield
26bbef8efd fix(consensus): harden sidecar quorum inputs 2026-04-27 10:14:12 +07:00
Nicholas Dudfield
6e71f84867 refactor: add typed sidecar SHAMap sync 2026-04-27 09:58:34 +07:00
Nicholas Dudfield
ab9b48f67a Merge remote-tracking branch 'origin/dev' into feature-export-rng
# Conflicts:
#	.github/workflows/levelization.yml
#	Builds/levelization/README.md
#	Builds/levelization/levelization.py
#	Builds/levelization/levelization.sh
#	cmake/RippledCore.cmake
2026-04-27 09:14:59 +07:00
Alloy Networks
cd00ed72d8 change build instructions url 2026-04-24 11:12:28 +10:00
tequ
05a3e04f2d Fix BEAST_ENHANCED_LOGGING not working and restore original behavior 2026-04-24 11:11:40 +10:00
tequ
66f7294120 Test: hint build_test_hooks.sh when hook wasm is empty in hso() 2026-04-24 11:10:46 +10:00
Nicholas Dudfield
7f6ac75617 Revert "chore: use improved levelization script with threading and argparse"
This reverts commit 5c1d7d9ae9.
2026-04-24 11:09:19 +10:00
Nicholas Dudfield
4150f0383c chore: use improved levelization script with threading and argparse 2026-04-24 11:09:19 +10:00
Nicholas Dudfield
25123b370a chore: replace levelization shell script with python
Backport of XRPLF/rippled#6325. The python version runs ~80x faster.
2026-04-24 11:09:19 +10:00
tequ
f90ed41802 enable ccache direct_mode 2026-04-24 11:06:51 +10:00
tequ
8c4c158d3a output ccache configuration in release-builder 2026-04-24 11:06:51 +10:00
tequ
2d2951875d fix: typo SignersListSet 2026-04-24 11:05:20 +10:00
tequ
9bfca63574 Update util_keylet fee test 2026-04-24 11:00:31 +10:00
tequ
1ba444ae7f Updated tests to align with the changes merged into the dev branch. 2026-04-24 11:00:31 +10:00
tequ
f96d9b6e51 Add tests for Hooks fee 2026-04-24 11:00:31 +10:00
Nicholas Dudfield
04077c1a55 test(testnet): assert zero entropy in degraded ledgers 2026-04-10 12:04:46 +07:00
Nicholas Dudfield
d94079d762 test(rng): relax PartialReveals sync assertion 2026-04-10 11:18:52 +07:00
Nicholas Dudfield
92ec07a1be chore: regenerate hook/sfcodes.h + format fix
Regenerate sfcodes.h to include the new sfSidecarType field
(UINT8, code 20).  Also apply clang-format to ConsensusExtensions.cpp.
2026-04-10 10:36:50 +07:00
Nicholas Dudfield
664db62588 fix: sidecar kind lost on cache hit + harden export sig parse
1. Record SidecarKind in pendingRngFetches_ before calling
   onAcquiredSidecarSet on local-cache-hit path. Without this,
   cached reveal/exportSig sets silently fell back to commit kind
   and were rejected by the sfSidecarType check.

2. Wrap export sig visitLeaves callback in try/catch (matching the
   RNG path) and enforce sfSidecarType == sidecarExportSig before
   processing — closes the shape-only acceptance gap.
2026-04-10 10:22:58 +07:00
Nicholas Dudfield
03a436d918 refactor: convert sidecar SHAMap entries from STTx to STObject
Replace STTx-based sidecar entries with plain STObject(sfGeneric)
using sfSidecarType (UINT8) discriminator. Eliminates unnecessary
transaction envelope overhead (sfSequence, sfFee, sfFlags) and
content-sniffing heuristics from the parse path.

Build: STObject with sidecarRngCommit/sidecarRngReveal/sidecarExportSig
Parse: sfSidecarType dispatch + typed field accessors
2026-04-10 10:14:06 +07:00
Nicholas Dudfield
7474048295 refactor: typed sidecar dispatch — eliminate content-sniffing heuristic
Replace the content-sniffing heuristic in onAcquiredSidecarSet with
typed dispatch based on SidecarKind.

The type is already known at fetch time:
- commitSetHash → SidecarKind::commit
- entropySetHash → SidecarKind::reveal
- exportSigSetHash → SidecarKind::exportSig

pendingRngFetches_ changes from hash_set<uint256> to
hash_map<uint256, SidecarKind>.  When the set arrives,
look up the kind by hash and dispatch — no leaf inspection.

This is the set-classification fix (Option E from the design doc):
no new SField, no STTx changes, no protocol additions, no RNG
proof-chain churn.
2026-04-10 09:18:43 +07:00
Nicholas Dudfield
1ee660529e fix: RPC handler sync, unused local, idiomatic Buffer comparison
- Add rng_poll_ms, no_export_sig, bootstrap_fast_start to the
  runtime_config RPC handler (SET and GET paths) so all ConfigVals
  fields are configurable live via admin RPC.
- Remove unused `added` counter in CSF fetchRngSetIfNeeded (was
  causing compiler warnings after debug logging removal).
- Use Buffer::operator== instead of std::memcmp in upgradeSignature,
  drop <cstring> include.
2026-04-10 08:56:16 +07:00
Nicholas Dudfield
311dfa1c23 chore: add TODO for RuntimeConfig activation gating
Both runtime_config and disconnect RPC handlers are already
Role::ADMIN.  Add a TODO to consider gating the entire
RuntimeConfig system on a config flag or compile-time define
for production nodes.
2026-04-10 08:31:54 +07:00
Nicholas Dudfield
f27cd2c567 refactor: consolidate env vars into RuntimeConfig
Move XAHAU_RNG_POLL_MS and XAHAUD_NO_EXPORT_SIG into RuntimeConfig
as rngPollMs and noExportSig fields.  Both are now configurable via
the XAHAU_RUNTIME_CONFIG JSON blob or individual env vars, and
controllable at runtime via the runtime_config RPC.

rngPollMs is clamped to minimum 50ms (prevents tight-loop polling).
Default remains 250ms.

This removes the last loose std::getenv calls from production code
outside of RuntimeConfig.  All env-var-based configuration now flows
through a single system.
2026-04-10 08:24:20 +07:00
Nicholas Dudfield
f34fdc297c fix(export): close upgradeSignature TOCTOU with buffer comparison
upgradeSignature now takes the verified buffer and compares it against
the currently stored buffer before promoting to verified.  This guards
against concurrent overlay threads overwriting the buffer between the
caller's unverifiedSignatures() snapshot and the upgrade call.

If the stored buffer was overwritten (different size or content), the
upgrade is silently skipped — the new buffer will be verified on its
next encounter.
2026-04-10 08:19:45 +07:00
Nicholas Dudfield
65fa63883d chore: remove CSF debug logging that floods CI output
Strip JLOG(j_.debug()) calls from buildEntropySet, fetchRngSetIfNeeded,
and finalizeRoundEntropy in CSF Peer.h.  These were added for local
debugging and caused CI failures due to output size limits.
2026-04-09 20:21:37 +07:00
Nicholas Dudfield
d8c683fb4c test(rng): fix AlignmentRequired test to run 1 round not 3
Running 3 rounds caused peer 0 to desync on round 2, dropping
prevProposers for the majority on round 3, triggering bootstrap
skip → zero entropy on the last round.  The gate works correctly
(logs show aligned=3, peersSeen=3) but the test was checking the
LAST round's entropy, not the round where the gate was exercised.

Run 1 round after warmup — sufficient to exercise the gate.
2026-04-09 18:09:17 +07:00
Nicholas Dudfield
fd53af304b fix(rng): measure entropy deadline from publish time, not reveal start
The entropy convergence deadline was measured from revealPhaseStart_,
which is set when entering ConvergingReveal.  By the time the entropy
set is published (after reveal timeout + observation tick), most of
the deadline budget was already spent — leaving insufficient time
for peer alignment.

Add entropyPublishStart_ timestamp set when the entropy set is first
published.  All convergence gate deadlines now measure from this
point, giving the full 2x rngREVEAL_TIMEOUT window for peer
proposals to propagate and alignment to be observed.
2026-04-09 18:06:18 +07:00
Nicholas Dudfield
2a3f0ec923 fix(rng): bounded wait for alignment instead of immediate fallback
When peers have published entropySetHash but none match ours yet
(e.g. a subset peer is the only one seen so far), wait for the
bounded deadline instead of immediately falling back to zero.
Other aligned peers may not have published yet — give them time.

Only fall back to zero if no alignment is observed within the
deadline (2x rngREVEAL_TIMEOUT).
2026-04-09 17:58:41 +07:00