Compare commits

..

491 Commits

Author SHA1 Message Date
Nicholas Dudfield
3eaf7b9e3d feat(consensus): record entropy contributors 2026-07-03 19:21:10 +07:00
Nicholas Dudfield
02313caa00 fix(hooks): remove consensus entropy keylet 2026-07-03 12:27:52 +07:00
Nicholas Dudfield
7c5ddb06d8 feat(hooks): expose consensus entropy keylet 2026-07-03 07:23:07 +07:00
Nicholas Dudfield
e8d40472e9 feat(consensus): add validator full entropy tier 2026-07-02 21:08:37 +07:00
Nicholas Dudfield
ac659faec7 fix(consensus): refresh extension gates before round cleanup 2026-07-02 19:26:32 +07:00
Nicholas Dudfield
986646ec90 fix(consensus): refresh extension gates on round start 2026-07-02 18:35:29 +07:00
Nicholas Dudfield
e3e1d847a4 test(consensus): assert export quorum cohort sync 2026-07-02 18:34:44 +07:00
Nicholas Dudfield
fa876927a9 feat: record consensus entropy denominator 2026-07-02 17:38:40 +07:00
Nicholas Dudfield
9a99c2d400 chore: update levelization results 2026-07-02 16:56:02 +07:00
Nicholas Dudfield
e001aca8ae fix(consensus): trust accepted export signature roots 2026-07-02 16:09:29 +07:00
Nicholas Dudfield
777743792e docs(consensus): qualify entropy fallback boundary as lab-only 2026-07-02 14:53:21 +07:00
Nicholas Dudfield
3af3b33c07 docs(consensus): clarify entropy boundary after tx-set agreement 2026-07-02 14:39:40 +07:00
Nicholas Dudfield
030206d632 docs(consensus): document entropy fallback boundary 2026-07-02 14:33:43 +07:00
Nicholas Dudfield
9324bd59e5 fix(consensus): keep entropy selection rooted in accepted hash
Revert the local entropyFailed selector override introduced during sidecar reconciliation excision. ConsensusEntropy injection is ledger-defining, so accepted entropy roots remain authoritative; nodes without an accepted root still fall back through the normal missing-root path.

Update CSF parity, the focused onPreBuild regression, and the design intent note to capture that local timeout/diagnostic state must not override an accepted root.
2026-07-02 13:50:42 +07:00
Nicholas Dudfield
3c01901119 feat(consensus): excise sidecar reconciliation
Remove peer fetch/acquire/merge support for CE/export sidecar SHAMaps and keep them as local same-process snapshots only.

Harden generic transaction-set paths so sidecar maps are not advertised, served, acquired, or wrapped as RCLTxSet, and delete the stale fetch-positive .testnet suite.

Make entropy failure terminal at selection time and mirror that invariant in the CSF harness; keep Export witness materialization rooted in accepted exportSigSetHash rather than local timeout state.
2026-07-02 12:52:43 +07:00
Nicholas Dudfield
e8fa4e1154 fix(consensus): stop serving sidecars when reconciliation is off 2026-07-01 11:30:42 +07:00
Nicholas Dudfield
d4650441fb feat(consensus): default sidecar reconciliation off
Compile out sidecar acquisition and merge plumbing unless xahaud_sidecar_reconciliation is enabled, and move fetch-positive .testnet scenarios into an opt-in suite.

When reconciliation is enabled, gate eager fetches on current trusted root support so first-seen transient roots do not trigger network acquisition. This keeps the recovery path available for measurement without making it the default path.

Verified default OFF with x-format-changed, git diff --check, x-quick-check, cmake configure/build, and ./build/rippled --unittest=ConsensusExtensions. ON path syntax-checked; Claude is assigned the parallel ON build/test review in a separate worktree.
2026-06-30 16:10:50 +07:00
Nicholas Dudfield
67c2c44263 feat(consensus): gate sidecar reconciliation 2026-06-30 14:03:56 +07:00
Nicholas Dudfield
ee6c4ff31f test(consensus): measure entropy sidecar fetch yield 2026-06-30 12:07:33 +07:00
Nicholas Dudfield
63f99ad9d8 feat(consensus): salt tx ordering with entropy 2026-06-29 16:32:16 +07:00
Nicholas Dudfield
be649d5112 fix(export): bound retry window 2026-06-29 14:28:20 +07:00
Nicholas Dudfield
8c48e500e7 fix(consensus): domain-separate participant diagnostics 2026-06-29 11:49:26 +07:00
Nicholas Dudfield
5195fc64b3 fix(hooks): block callback from canceling current export latch 2026-06-29 10:54:31 +07:00
Nicholas Dudfield
f849b70a96 docs(export): clarify target network id guard 2026-06-29 10:18:53 +07:00
Nicholas Dudfield
8ad82f328d fix(export): prevent callback hook from pre-canceling import latch 2026-06-29 10:03:04 +07:00
Nicholas Dudfield
618314afb7 test(consensus): isolate export sidecar rejection cases 2026-06-29 07:55:06 +07:00
Nicholas Dudfield
0668caacf2 test(consensus): fix round-pinned sidecar fixture 2026-06-29 07:26:19 +07:00
Nicholas Dudfield
583c4e2461 test(export): prove sidecar fetch under proposal loss 2026-06-28 20:36:53 +07:00
Nicholas Dudfield
2fe984297a fix(consensus): filter agreed export signers by active view 2026-06-28 20:10:28 +07:00
Nicholas Dudfield
d6de1d90b5 fix(consensus): pin RNG round sequence at start 2026-06-28 20:01:08 +07:00
Nicholas Dudfield
3eda8d664e fix(consensus): share entropy sidecar leaf admission 2026-06-28 19:53:42 +07:00
Nicholas Dudfield
a0866d1559 fix(import): use ledger time for VL freshness 2026-06-28 19:45:36 +07:00
Nicholas Dudfield
4a12f4c231 fix(hook): harden xport and xpop host paths 2026-06-28 19:41:20 +07:00
Nicholas Dudfield
a93b8cfdc4 refactor(export): share inner export transaction parsing 2026-06-28 19:18:40 +07:00
Nicholas Dudfield
c2bc4fabe8 fix(consensus): filter export signature candidates 2026-06-28 19:02:32 +07:00
Nicholas Dudfield
94ffc86bd9 fix(consensus): cap export sidecar candidates 2026-06-28 18:18:45 +07:00
Nicholas Dudfield
1d158a910b fix(consensus): cap fetched sidecar cardinality 2026-06-28 17:18:14 +07:00
Nicholas Dudfield
62dc85df86 test(export): cover replay witness pre-scan 2026-06-28 15:49:06 +07:00
Nicholas Dudfield
d7a8467705 refactor(consensus): share active export signer filter 2026-06-28 15:44:18 +07:00
Nicholas Dudfield
8cc4574e0f refactor(export): split closed-ledger result assembly 2026-06-28 15:39:37 +07:00
Nicholas Dudfield
cf3a9ef96b fix(consensus): freeze rng commits before reveal 2026-06-28 15:15:17 +07:00
Nicholas Dudfield
806aa56aec fix(consensus): centralize rng admission 2026-06-28 14:52:00 +07:00
Nicholas Dudfield
cb1f932236 refactor(consensus): share sidecar build helpers 2026-06-28 14:20:55 +07:00
Nicholas Dudfield
e01c0cba70 fix(export): require replay parent for historical apply 2026-06-28 13:53:19 +07:00
Nicholas Dudfield
5e27c2fea9 refactor(consensus): centralize sidecar admission checks 2026-06-28 13:43:16 +07:00
Nicholas Dudfield
19ec9ece35 fix(replay): enforce manual replay ledger hash 2026-06-28 13:33:53 +07:00
Nicholas Dudfield
a714eaa621 fix(consensus): bind entropy reveals to proofed commits 2026-06-28 13:32:26 +07:00
Nicholas Dudfield
f0f998d897 test(export): cover stale ImportVL callback rejection 2026-06-28 12:51:32 +07:00
Nicholas Dudfield
1dbda8f6b3 style(export): satisfy clang-format 2026-06-28 12:34:50 +07:00
Nicholas Dudfield
5149f5ae9b refactor(import): make VL ratchet explicit for callbacks 2026-06-28 12:32:58 +07:00
Nicholas Dudfield
46632be87b refactor(rng): reuse named commit quorum threshold 2026-06-28 12:13:39 +07:00
Nicholas Dudfield
8bd79c4759 test(export): align testnet export metadata assertions 2026-06-28 12:06:33 +07:00
Nicholas Dudfield
6661e7a40a test(export): move reserve setup to failing test 2026-06-28 11:55:29 +07:00
Nicholas Dudfield
46207a3fca test(export): initialize account root before reserve edit 2026-06-28 11:38:48 +07:00
Nicholas Dudfield
a16e8886c2 test(testnet): resolve export signature witnesses 2026-06-28 11:24:14 +07:00
Nicholas Dudfield
268db0b6d2 fix(export): enforce shadow ticket reserve 2026-06-28 11:24:06 +07:00
Nicholas Dudfield
73f3b7a167 fix(consensus): reject miskeyed rng sidecar leaves 2026-06-28 11:24:00 +07:00
Nicholas Dudfield
1b4535cee5 docs(consensus): clarify rng proposal key checks 2026-06-28 10:30:17 +07:00
Nicholas Dudfield
bb6242213b fix(hooks): enforce xport emit budget 2026-06-28 10:16:53 +07:00
Nicholas Dudfield
0aa8e3ca67 fix(consensus): reject non-secp rng proposal proofs 2026-06-28 10:16:29 +07:00
Nicholas Dudfield
516edb8147 fix(hooks): return export wrapper hash from xport 2026-06-28 10:09:00 +07:00
Nicholas Dudfield
1186a3a7c4 style: apply clang-format 2026-06-27 17:38:11 +07:00
Nicholas Dudfield
9d9329ece4 refactor: reuse proof parser and named entropy tiers 2026-06-27 17:32:37 +07:00
Nicholas Dudfield
465131bb3e refactor: name legacy NetworkID transaction boundary 2026-06-27 16:59:20 +07:00
Nicholas Dudfield
1b0c633eda refactor(consensus): trim small duplicated extension constants
Reuse SidecarPeerAlignment::quorumAligned for export sidecar gates, name the shared sidecar convergence timeout, and centralize the RNG commitment tuple.

Also make ExportLimits::maxExportsPerHook the value enforced by xport_reserve, with a static assertion against the hook ABI constant.
2026-06-27 15:24:06 +07:00
Nicholas Dudfield
ceabb893bf docs(consensus): tighten proposal proof wording
Clarify that fetched RNG commit proofs carry the signed proposal fields plus signature, not only the serialized position bytes.
2026-06-27 15:16:35 +07:00
Nicholas Dudfield
d5ca581cea docs(consensus): explain sidecar reconciliation model
Document the common proposal-relay path, why RNG/Export use fixed-denominator quorum over proposal-borne material, and how sidecar SHAMap fetch/proofs act as secondary distribution rather than authority.

This captures the reconciliation cost/benefit in the committed design doc, separate from the reviewer-facing PR description.
2026-06-27 15:13:22 +07:00
Nicholas Dudfield
8a81588b71 fix(consensus): let quorum-aligned entropy conflicts proceed
Use the fixed-denominator entropy threshold as the conflict boundary: once our entropy hash is quorum-aligned, a below-threshold conflicting minority or silent peer cannot force fallback by withholding full observation.

Update the consensus entropy invariant spine and design doc so the no-veto conflict rule is explicit, and flip the regression test to cover the liveness-preserving path.
2026-06-27 15:09:30 +07:00
Nicholas Dudfield
0a9023c334 fix(export): avoid live manifest filtering of accepted witnesses
Materialize ttEXPORT_SIGNATURES from the accepted export sidecar root during live consensus builds, then let apply trust that tx-stream witness membership while still verifying each signature and enforcing quorum.

Direct apply paths remain conservative, and historical replay uses the same witness-membership rule after manifests may have rotated. Update the intent docs and tests for live, direct, and replay modes.
2026-06-27 13:39:58 +07:00
Nicholas Dudfield
c38567792e fix(hooks): separate callback rng streams 2026-06-27 13:06:28 +07:00
Nicholas Dudfield
3cfc79ea59 fix(consensus): bind rng sidecar proofs to round parent 2026-06-27 13:04:08 +07:00
Nicholas Dudfield
eb3a3b888f fix(consensus): allow shared diagnostics in export-only rounds 2026-06-27 12:56:19 +07:00
Nicholas Dudfield
c56cb34d6d fix(consensus): total-order entropy selector inputs 2026-06-27 12:45:21 +07:00
Nicholas Dudfield
306566ccb6 fix(hooks): bound export reservations by emit budget 2026-06-27 12:40:16 +07:00
Nicholas Dudfield
8ddbe0891a docs(consensus): capture sidecar validation boundary 2026-06-27 12:32:10 +07:00
Nicholas Dudfield
deb1b2af0b fix(consensus): validate extension sidecar inputs
Reject malformed RNG sidecar signing keys before PublicKey construction so fetched untrusted leaves cannot trigger an uncatchable LogicError abort. Also re-check proposal signatures at the extension harvest boundary so cluster-relayed invalid proposals cannot populate RNG or export sidecar state.
2026-06-27 12:25:52 +07:00
Nicholas Dudfield
4dcb3f3557 fix(consensus): remove local entropy fallback shortcuts
Encode fallback-vs-nonfallback as a ledger-defining decision, remove the prevProposers/bootstrap and visible-peer impossible-quorum shortcuts, and use proofed commit material for the timeout transition. Update unit/CSF/testnet coverage and document the remaining validation-backstopped timing residual.

Tests: x-run-tests --stop-on-fail ConsensusExtensions; x-run-tests --stop-on-fail ConsensusRng; x-format-changed; git diff --check
2026-06-27 12:08:35 +07:00
Nicholas Dudfield
efcd2a0ab7 fix(export): close replay witness review gaps
Thread the replay parent through ApplyOptions so export replay can rebuild the historical validator view without relying on LedgerMaster visibility for freshly replayed parents.

Harden ttEXPORT_SIGNATURES canonicality before witness indexing, validate witness signer accounts against their pubkeys, reject ambiguous absent NetworkID on low-ID source chains, and lock the consensus phase accessor.

Add focused regression coverage and clarify that destination SignerList/quorum compatibility is an operator/client contract.
2026-06-27 10:53:34 +07:00
Nicholas Dudfield
8ddb0a1134 docs(export): document witness assembly contract
State that clients expanding sfExportSignatureHash must match ExportResultBuilder's canonical signer ordering, empty SigningPubKey, and target-chain signer cap before hashing or submitting.
2026-06-27 10:14:43 +07:00
Nicholas Dudfield
891077daa7 fix(export): harden witness edge checks
Validate ttEXPORT_SIGNATURES witness bindings in preflight, guard standalone witness synthesis against malformed inner transactions, and align ticket-path XPOP quorum with Export's ceiling threshold.

Regenerate the genesis hook sfcodes header and add it to generated-header verification so CI catches future drift.
2026-06-27 10:10:45 +07:00
Nicholas Dudfield
f0224cac1e fix(export): replay witnesses without current manifests
Historical LedgerReplay now treats ttEXPORT_SIGNATURES witnesses as validated-history membership input, while still verifying signatures and threshold. Live build and validation keep the current manifest-backed active-signer check.

Harden witness handling by indexing only canonical same-ledger pseudos, dropping duplicate witnesses as ambiguous, and replacing pre-existing mismatched witnesses with the accepted sidecar witness during onPreBuild.

Document the lean witness-vs-XPOP proof distinction and cover the manifest-rotation replay case with an Export regression test.
2026-06-27 09:38:07 +07:00
Nicholas Dudfield
845798b072 fix(export): replay signatures through witness pseudo 2026-06-27 08:13:23 +07:00
Nicholas Dudfield
9f04dcf582 docs(export): capture replay witness invariant 2026-06-27 06:00:16 +07:00
Nicholas Dudfield
8a8a0b434d test(entropy): assert sub-quorum closed-ledger fallback 2026-06-27 05:34:01 +07:00
Nicholas Dudfield
2fbf2e11bc fix(consensus): drop late sidecars after accept 2026-06-27 05:25:31 +07:00
Nicholas Dudfield
bac359fd96 docs(consensus): add entropy intent spine 2026-06-27 05:14:00 +07:00
Nicholas Dudfield
fcedd01761 chore(hooks): regenerate sfcodes header 2026-06-26 22:22:46 +07:00
Nicholas Dudfield
9f24cd0b75 docs(consensus): describe entropy no-veto gate 2026-06-26 22:11:43 +07:00
Nicholas Dudfield
d6e7cf428b fix(consensus): undo b12cee5d4 entropy observation veto 2026-06-26 21:56:40 +07:00
Nicholas Dudfield
72f73ccbad test(hooks): pin export reservation budget 2026-06-26 21:31:34 +07:00
Nicholas Dudfield
a78f09a0fe fix(import): allow ticket xpop network id 2026-06-26 20:48:26 +07:00
Nicholas Dudfield
f3526952b6 fix(import): skip b2m fee checks for ticket imports 2026-06-26 20:29:16 +07:00
Nicholas Dudfield
da31668a2a fix(export): require retry expiry 2026-06-26 20:26:23 +07:00
Nicholas Dudfield
5dc214ab48 fix(hooks): report exports separately from emits 2026-06-26 20:20:09 +07:00
Nicholas Dudfield
4e217fcc8f fix(hooks): remove dice modulo bias 2026-06-26 20:13:26 +07:00
Nicholas Dudfield
d5608c3110 style(export): match CI clang-format wrapping 2026-06-26 20:06:57 +07:00
Nicholas Dudfield
7dddc8256a refactor(proof): remove unused tx leaf helper 2026-06-26 19:57:44 +07:00
Nicholas Dudfield
45a69538cb fix(consensus): ignore stale acquired sidecars 2026-06-26 19:51:20 +07:00
Nicholas Dudfield
e8cbf0f968 fix(consensus): drop own-key proposals without aborting 2026-06-26 19:38:40 +07:00
Nicholas Dudfield
1cdf850e4a fix(consensus): guard keyless proposal authoring 2026-06-26 19:35:07 +07:00
Nicholas Dudfield
cddcb83047 refactor(consensus): remove unused reveal proof cache
Reveal sidecars intentionally omit proof blobs to keep entropySet hashes deterministic. Remove the write-only proposalProofs_ cache and update the surrounding comments to match the current sidecar format.
2026-06-26 19:22:33 +07:00
Nicholas Dudfield
ed3c0f07a7 fix(export): return duplicate for live shadow tickets
A second pending export for the same shadow-ticket key is a ledger-object conflict, not an internal failure. Return tecDUPLICATE and cover the live-latch case in Export_test.
2026-06-26 19:14:10 +07:00
Nicholas Dudfield
c74295e51d docs(consensus): clarify validator view fallback scope
Document that makeActiveValidatorView's null-ledger fallback is for startup, standalone, and diagnostics paths, while closed-ledger apply callers must fail closed before using it.
2026-06-26 19:06:18 +07:00
Nicholas Dudfield
370e02ee1f docs(export): note shadow-ticket replay caveat
Document that shadow tickets are re-creatable latches rather than permanent tombstones, so import-callback hooks that move value must deduplicate same-XPOP replays themselves.
2026-06-26 18:00:10 +07:00
Nicholas Dudfield
38672fba9f fix(consensus): reject noncanonical extended positions
Require legacy consensus positions to use the exact 32-byte form. Extended encodings now need a nonzero known flag byte, preventing alternate wire encodings of the same tx-set hash.
2026-06-26 17:57:55 +07:00
Nicholas Dudfield
932ccbe0e5 fix(export): bind trusted wire signatures to proposal digest
Require wire-carried export signatures to match the signed ExtendedPosition digest before harvesting, and keep the per-blob size guard local to the helper.
2026-06-26 17:48:19 +07:00
Nicholas Dudfield
927ec66a9e fix(consensus): gate observed participant diagnostics
Treat observedParticipantsHash as entropy extension material during proposal precheck, so disabled featureConsensusEntropy rejects diagnostic-only extension payloads too.
2026-06-26 17:44:26 +07:00
Nicholas Dudfield
3c108d3a87 fix(protocol): preserve local TER code values
Keep telENV_RPC_FAILED in its original local-error slot and append telSHADOW_TICKET_REQUIRED after it, with a TER regression test pinning both values.
2026-06-26 17:40:46 +07:00
Nicholas Dudfield
07030e2b87 fix(export): retry without parent validator view
Network-mode Export apply must build its signer universe from the closed ledger's parent. If that parent ledger is unavailable, retry instead of falling back through makeActiveValidatorView to the node-local latest validated ledger.
2026-06-26 17:23:19 +07:00
Nicholas Dudfield
8373621445 docs(consensus): clarify liveness exit scope
Document that stalled, moved-on, and expired consensus exits are generic consensus-engine liveness behavior coordinated by software release, not amendment-gated ledger-rule behavior.
2026-06-26 17:18:56 +07:00
Nicholas Dudfield
b4208459cf fix(consensus): harden export signatures and weak RNG domain
Bound proposal-carried export signature blobs before hashing them during proposal precheck, and apply the same size cap at export signature harvest as defense in depth.

Cap the unverified export-signature cache by distinct tx hash while leaving verified signatures and existing entries ungated, so real exports can still reach quorum.

Domain-separate Hook RNG using the actual strong/weak execution role at draw time, and document the residual commit/reveal withholding bias across all entropy tiers.
2026-06-26 17:08:32 +07:00
Nicholas Dudfield
d091ebfb7e Pin extension apply to accepted sidecar roots
Require entropy injection and export apply to consume only sidecar roots accepted by their convergence gates. Local timeout state and later collector growth no longer participate in closed-ledger material once a sidecar root is accepted.

Also guard RCLConsensus::extensionsBusy() under the consensus mutex, cap exported multisig signer arrays to the protocol maximum after canonical signer sorting, and document the remaining accept-vs-timeout liveness tradeoff.

Add regressions for unaccepted export sidecar roots, late export collector mutation, entropy accepted-root handling, exported signer caps, and CSF accepted-sidecar behavior.
2026-06-26 16:50:19 +07:00
Nicholas Dudfield
229eb5b573 refactor(consensus): clarify extension sidecar seams 2026-06-26 13:08:46 +07:00
Nicholas Dudfield
938d3055ce fix(consensus): harden threshold helpers 2026-06-26 11:47:16 +07:00
Nicholas Dudfield
aee3a638ee fix(consensus): avoid NegativeUNL cap overflow 2026-06-26 10:26:47 +07:00
Nicholas Dudfield
4fb91ea9f5 refactor(consensus): expose threshold policy seams 2026-06-24 18:58:24 +07:00
Nicholas Dudfield
acb492a2c9 docs(consensus): polish guided review excerpts 2026-06-24 14:50:43 +07:00
Nicholas Dudfield
92fe444323 docs(consensus): tighten guided review anchors 2026-06-24 14:15:23 +07:00
Nicholas Dudfield
9549901014 docs(consensus): clarify extended position identity 2026-06-24 13:35:25 +07:00
Nicholas Dudfield
9f6e7dd315 refactor(consensus): make extended position identity explicit 2026-06-24 13:10:57 +07:00
Nicholas Dudfield
f6d986bdbc chore: remove stale branch TODO comments 2026-06-24 12:48:52 +07:00
Nicholas Dudfield
e55bf43986 test: document consensus entropy fixture opt-in 2026-06-24 12:38:09 +07:00
Nicholas Dudfield
b4beb92c34 chore: match levelization output 2026-06-23 16:49:01 +07:00
Nicholas Dudfield
40edbfc7f2 chore: update levelization results 2026-06-23 16:46:25 +07:00
Nicholas Dudfield
ff205b1b81 docs(hooks): generate entropy API comments 2026-06-23 16:43:45 +07:00
Nicholas Dudfield
ddfb1dbeb6 docs: clarify entropy and runtime test APIs 2026-06-23 16:42:05 +07:00
Nicholas Dudfield
8bf1ece0a0 test(export): pin expiry and no-veto boundaries 2026-06-23 16:42:02 +07:00
Nicholas Dudfield
639e153f3b fix(consensus): gate proposal extensions by parent ledger 2026-06-23 16:41:58 +07:00
Nicholas Dudfield
8e542c32e0 test(testnet): clarify directed runtime latency syntax 2026-06-23 16:19:10 +07:00
Nicholas Dudfield
38f4d53ebf test(testnet): add runtime latency probe suite 2026-06-23 16:04:58 +07:00
Nicholas Dudfield
fe66a11c69 build(test): allow external hook Env tests 2026-06-23 13:38:43 +07:00
Nicholas Dudfield
77e48d553c test(export): cover sidecar rejection preflights 2026-06-23 11:53:51 +07:00
Nicholas Dudfield
439031dc92 Merge remote-tracking branch 'origin/dev' into feature-export-rng 2026-06-23 10:55:43 +07:00
Nicholas Dudfield
1a5b934881 test(csf): model sidecar split-brain equivocation 2026-06-23 10:52:38 +07:00
Nicholas Dudfield
7f0d2959e8 test(export): cover no-veto sidecar withholding 2026-06-23 10:34:01 +07:00
Nicholas Dudfield
1440d1495f test(consensus): cover split-brain sidecar threshold 2026-06-23 10:20:34 +07:00
Nicholas Dudfield
57f5a9d6cc test(testnet): cover export without UNLReport 2026-06-23 10:14:09 +07:00
Nicholas Dudfield
cacd1f71fe test(consensus): mint tier 2 with active nUNL 2026-06-23 10:11:40 +07:00
Nicholas Dudfield
93a6f0fbec docs(testnet): add export degradation projection marker 2026-06-23 10:07:02 +07:00
Nicholas Dudfield
c5b1cb222d docs(consensus): add test projection markers 2026-06-23 10:03:04 +07:00
Nicholas Dudfield
e447f9f021 docs(consensus): add export apply projection markers 2026-06-23 09:55:01 +07:00
Nicholas Dudfield
0757094ed2 docs(consensus): add formal proof projection markers 2026-06-23 09:41:55 +07:00
Nicholas Dudfield
4b8107d57c docs(consensus): clarify nUNL entropy gate thresholds 2026-06-23 08:42:22 +07:00
Nicholas Dudfield
d4b9e2f22c chore(formal): ignore local Lean proof workspace 2026-06-23 08:14:03 +07:00
Nicholas Dudfield
3fc199017e docs(cmake): note runtime config conan opt-in 2026-06-22 16:57:30 +07:00
Nicholas Dudfield
4d55443976 refactor(runtime-config): scope fault injection controls 2026-06-22 16:51:23 +07:00
Nicholas Dudfield
0d1d649867 refactor(consensus): remove orphaned txset cache callback 2026-06-22 15:33:29 +07:00
Nicholas Dudfield
60469dbd86 refactor(rng): remove explicit-final proposal path 2026-06-22 15:07:18 +07:00
Nicholas Dudfield
3970912735 chore: update levelization results 2026-06-22 13:31:56 +07:00
Nicholas Dudfield
456f4144ba docs(rng): mark explicit-final for removal 2026-06-22 12:50:19 +07:00
Nicholas Dudfield
570cad4c44 test(export): pin network apply regressions 2026-06-22 12:14:42 +07:00
Nicholas Dudfield
b13868b71e fix(export): apply agreed sidecar signatures
Use the agreed exportSigSetHash sidecar map, not the live ExportSigCollector, as the network-mode ttEXPORT signer snapshot. Preserve export convergence state through RNG onPreBuild cleanup so buildLCL apply can see the gate result, and require a UNLReport-backed validator view before non-standalone Export finalization. Add regression coverage for agreed-vs-live collector mutation and onPreBuild export-state preservation.
2026-06-22 09:43:03 +07:00
Nicholas Dudfield
288b9e6d25 docs(rng): clarify commit substate pipelining 2026-06-22 09:06:12 +07:00
Nicholas Dudfield
e55c2c6dc8 fix(export): allow quorum-aligned sidecar despite missing observation 2026-06-22 09:00:46 +07:00
Nicholas Dudfield
35e981e509 docs(rng): align ConsensusExtensionsDesign with the UNLReport-only + F1 changes
A drift review (10-section adversarial pass + skeptical verification) found the design doc had fallen behind two recent code changes: the UNLReport-only gate (8368e12ab) and the F1 active-view alignment filter (fb9e2710c).

- Fallback Semantics: add the second fallback trigger -- a non-UNLReport-backed view mints consensus_fallback regardless of alignment.
- Validator Set And Quorum: note the config-fallback view is !fromUNLReport and forces consensus_fallback entropy.
- Entropy Alignment Rules: state the UNLReport precondition for non-fallback tiers; correct the alignment-count formula to the active-view-filtered count (non-active proposers and a non-active local +1 excluded -- the F1 fix); separate the two distinct counts (the peer-alignment GATE count vs the agreed entropySetMap_ leaf count that drives the tier LABEL); flag the explicit-final unfiltered exception.
- Worked examples: note they assume a UNLReport-anchored view (else all fall back).
- Export Principles: document that export success also requires full observation (peersSeen == txConverged), not just quorum alignment.
- Review Checklist: add items for the UNLReport-anchored-tier and alignment-count-universe (F1) invariants.

Doc-only. Raw drift findings kept in .ai-docs (gitignored).
2026-06-22 08:41:11 +07:00
Richard Holland
bb244ef772 put release builds into a candidate folder to prevent auto-update scripts running before smoke tests (#761) 2026-06-21 12:12:43 +10:00
Nicholas Dudfield
b9a733c831 docs(rng): flag the F1 gap on the explicit-final alignment path
The explicit-final proposal path counts participants/alignment over the unfiltered trusted-proposer set (ctx.peerPositions) plus an unconditional local +1, not the active validator view -- the same gap the F1 fix closed for the main entropy gate (inspectTxConvergedSidecarPeers). It is default-off/experimental with no robust timing model found (per the existing TBD, may never ship); if it is ever enabled it must apply the same active-view membership filter. Comment-only.

Follow-up to F1 (fb9e2710c).
2026-06-18 13:32:38 +07:00
Nicholas Dudfield
301e546aa9 chore(rng): assert active view source at round start 2026-06-18 10:42:28 +07:00
Nicholas Dudfield
e2da1db6d2 chore(rng): harden entropy fallback diagnostics 2026-06-18 10:32:07 +07:00
Nicholas Dudfield
16cd02156e test(rng): use a nonzero sentinel in the hook entropy-requirement test
testInvalidEntropyRequirements rejects four invalid dice/random requirements then returned accept(0,0,0) on success. But a valid dice(6,..) returns 0..5, so a regression that let the min_tier=0 requirement through (returning a value that happened to be 0) would still pass ~1/6 of the time -- the weakest spot is exactly the min_tier lower bound it most needs to prove (review finding). Return sentinel 42 (distinct from any dice/random result and from INVALID_ARGUMENT) and assert ==42, so any leaked requirement returns its own non-42 code and fails. WASM block recompiled; ConsensusEntropy 138 tests, 0 failures.

Follow-up to F1 (fb9e2710c).
2026-06-18 09:54:30 +07:00
Nicholas Dudfield
fb9e2710cc fix(rng): scope sidecar alignment count to the active validator view (F1)
The entropy/export bless-vs-fallback gate counted alignment over ALL tx-converged trusted proposers (currPeerPositions_), while the thresholds and the entropy leaf set are computed over the active validator view. A node that locally trusts proposers outside the on-ledger UNLReport active set could pad alignedParticipants(), inflating the counting universe N above originalViewSize and eroding the Tier-2 intersection margin (2t - N) below the Byzantine floor f -- so two equivocation cohorts padded by non-active aligners could each clear the gate (review finding F1). Backstopped by the 80% validation quorum, but the proof's universe and the code's universe must match.

inspectTxConvergedSidecarPeers now takes an active-view membership predicate and counts only member peers toward alignment; the local +1 is gated on localIsActiveValidator(). Both the RNG entropy gate and the export-sig gate pass ext.isUNLReportMember / ext.localIsActiveValidator, mirroring buildEntropySet / hasQuorumOfCommits' containsNode filter. New method on ConsensusExtensions + CSF Peer + the FakeExtensions stub.

Regression test (Sidecar peer alignment helper): a trusted-but-non-active aligned proposer pushes unfiltered aligned to 2 but filtered aligned stays 1, and a non-active local node's +1 is suppressed -- padding cannot satisfy the gate. ConsensusExtensions 900, ConsensusRng 386, Consensus 1399, all green.

Note: the explicit-final proposal path (ConsensusExtensionsTick.h ~:934) counts over prevProposers and is NOT yet filtered (separate, experimental path).
2026-06-18 09:44:56 +07:00
Nicholas Dudfield
c8fad50d66 Merge origin/dev into feature-export-rng
# Conflicts:
#	src/xrpld/app/hook/applyHook.h
2026-06-17 11:45:42 +07:00
Nicholas Dudfield
d9b5fc26fc style(rng): apply clang-format 2026-06-17 11:35:05 +07:00
Nicholas Dudfield
72d03620f9 test(rng): require UNLReport in testnet entropy suites 2026-06-17 11:33:04 +07:00
Nicholas Dudfield
8368e12ab3 fix(rng): require UNLReport view for non-fallback entropy 2026-06-17 11:32:51 +07:00
Nicholas Dudfield
23f745bd01 fix(hook): validate entropy requirements 2026-06-17 11:32:38 +07:00
Nicholas Dudfield
27ae39b91a fix(rng): use a genuinely forkable nUNL config in the anchor test
The tier-2 anchor pin (20d52d8b6) used an 8/6 (original/effective) nUNL config whose rationale was wrong: it computed cohort overlap against the original view (2*4-8=0), but aligned cohorts form in the EFFECTIVE view, so two 4-of-6 cohorts overlap by 2*4-6=2 > floor(8/5)=1 -- 8/6 is NOT actually forkable, the original-view anchor is merely conservative there. The test still caught the originalViewSize->size() regression, but the stated reason was misleading (review catch).

Switch to 10 active / 2 disabled (original 10, effective 8), a genuinely forkable case: a 5-of-8 cohort (what the effective-size threshold would admit) overlaps only 2*5-8=2, which does NOT exceed the f=floor(10/5)=2 faulty the original UNL still tolerates -> an equivocator could mint two distinct tier-2 digests. The correct original anchor requires 7 (overlap 6 > 2) and keeps the band closed (tier2==quorum==7); a regression to size() drops the floor to 5 and re-opens the forkable [5,7) band. Assertions and rationale updated to match; suite green (897, 0 failures).
2026-06-16 17:33:57 +07:00
Nicholas Dudfield
20d52d8b66 test(rng): pin tier-2 threshold to pre-nUNL original view
tier2Threshold() anchors to originalViewSize (pre-nUNL), not the effective post-nUNL size(): nUNL shrinks the effective view while leaving faulty nodes in it, so a sub-quorum fraction of the effective view can exceed the Byzantine bound. A one-line regression to size() would relabel a forkable cohort as participant_aligned -- and no existing test caught it (the only nUNL test never calls tier2Threshold()/selectEntropy, and the only tier-2 selection test has no nUNL, so original == effective there).

Add testTier2ThresholdAnchorsToOriginalView: 8 active validators, 2 disabled via NegativeUNL (original 8, effective 6), asserting tier2Threshold()==5 (from the original 8) not 4 (from the effective 6), with quorum/gate cross-checks. Mutation-verified: flipping the production read to size() turns exactly this test red (tier2Threshold + entropyGateThreshold) while the existing tier-2 selection test stays green.

Also: pointer comment at tier2Threshold() linking the invariant to its guard test, and a design-doc fix -- the worked example used n=5, where the band is empty (quorum == participant_aligned == 4) so it illustrated an unreachable tier; now n=6.
2026-06-16 17:02:00 +07:00
Richard Holland
639ea34377 Fixhookmap (#756) 2026-06-16 17:06:25 +10:00
tequ
089c0dc3fe Fix ammLPHolds logic to include escrowed cases (#757) 2026-06-16 15:26:29 +10:00
Nicholas Dudfield
b070785dee fix(rng): make tier-2 scenario's 5/6 check fallback-tolerant
The 5/6 phase asserted validator_quorum on whichever single ledger the validated tip happened to sit on. But the ledger right at the node-5 drop can be a transient consensus_fallback (EntropyTier=1, count=0) — deterministic and by design, while the commit/reveal pipeline re-primes — so when the tip landed there the assert failed (tier=1, observed ~1 run in 4). Same class of bug as the old post-recovery flake: depending on exactly where the tip lands.

Now settle 4 ledgers past the drop, then scan the post-drop validated ledgers for a clean validator_quorum (tier 3, count >= quorum). The window is entirely 5-node cohorts, so a tier-3 there has count == 5 (faithful to '5/6 still tier 3'); the transition fallback is tolerated, not asserted on. Verified 3/3: each run found tier=3 count=5.

Also harden _closed_entropy() to raise on != 1 ConsensusEntropy pseudo-tx (mirroring get_entropy_tx) instead of silently skipping — a duplicate/missing injection now fails with a clear error rather than resurfacing as a generic 'no tier-2 ledger'.
2026-06-16 11:15:55 +07:00
tequ
78167e09c0 change sfHookOn of ltHookDefinition to soeOptional for HookOnV2 (#755) 2026-06-16 13:58:53 +10:00
tequ
607a7fdf98 Change AMM to Supported::no (#758) 2026-06-16 13:56:58 +10:00
Nicholas Dudfield
def617e3f9 test(rng): harden tier-2 testnet scenario per review + robust on-ledger check
Codex review minors: assert_validator_quorum at the 5/6 boundary (EntropyTier=3 AND count >= quorum AND non-zero digest, not just tier==3 — catches a bad tier/count pairing); added explicit assert_validator_quorum / assert_consensus_fallback helpers and an entropy_fields() warning that its is_fallback (tier != 3) lumps participant_aligned in with fallback (safe only where no tier-2 band exists).

Robustness: tier 2 is below the validation quorum, so the validated tip stalls and which provisional ledgers it later reaches is timing-dependent (the post-recovery inspection was fragile). Replaced it with direct inspection of the surviving cohort's CLOSED ledger via ledger('closed') DURING the window; recovery is now a pure liveness check.

Verified on a live testnet: PASS — EntropyTier=2 count=4 confirmed on 4 distinct provisional ledgers (seq 7-10).
2026-06-16 10:51:33 +07:00
Nicholas Dudfield
c322b59961 test(rng): tier-2 (participant_aligned) testnet scenario at n=6
A 6-node smoke (the smallest NON-degenerate tier-2 size: tier2 floor 4, quorum 5) driving the 4/6 band. n=5 has no band (tier2 == quorum), which is why the existing degradation smoke only ever sees tier 3 / fallback.

Tier 2 is below the 80% validation quorum, so the 4/6 cohort's ledgers are provisional: the scenario confirms tier-2 injection from the cohort's logs during the window, then verifies the on-ledger EntropyTier=2 count=4 POST-RECOVERY once those ledgers become canonical and validate (the mechanism the degradation smoke also relies on). Adds the assert_participant_aligned helper and the node_count:6 suite entry.

Verified on a live testnet: PASS — 12 tier-2 injections in the 4/6 window, seq 7 & 8 validated as participant_aligned (count 4).
2026-06-16 10:28:57 +07:00
Nicholas Dudfield
c3cc3513c9 docs(rng): refresh tier 2 terminology 2026-06-16 10:05:57 +07:00
Nicholas Dudfield
a0be935227 docs(rng): correct stale tier-2 comments
Codex final-review minors, both comment-only (no behavior change): EntropyTier participant_aligned no longer says "reserved for a future" tier (it is implemented in this stack); and the commit-timeout path comment said "fixed UNL quorum" but the code now uses entropyGateThreshold() (min(quorum, tier2)).
2026-06-16 09:45:08 +07:00
Nicholas Dudfield
6b1b18dd38 refactor(rng): guard CSF entropy finalize to reveal-type sidecar
Codex final-review minor (optional hardening): finalizeRoundEntropy now requires the fetched sidecar entry to be reveal-type before counting it as entropy. lastEntropySetHash_ only ever names a reveal set (hashRngSet's per-type salt rules out a cross-type hash collision), so this is purely defensive — behavior unchanged, all CSF sims green.
2026-06-16 09:30:42 +07:00
Nicholas Dudfield
946f25249b fix(rng): CSF finalizes from advertised set, not live pendingReveals_
Codex review caught a CSF/production fidelity gap: production injects entropy from the AGREED entropySetMap_ (selectEntropy), which is frozen at advertise time — late-fetched or conflicting reveals merge into pendingReveals_ but are NOT injected unless a rebuild republishes the hash. The CSF peer fetched into pendingReveals_ AND finalized from pendingReveals_, so a conflict/fetch sim could count reveals production would never inject from.

Track the last advertised entropy-set hash (buildEntropySet) and finalize from the sidecar-store snapshot under that hash — the analog of the frozen entropySetMap_. Clean/no-conflict sims are unchanged (the snapshot equals pendingReveals_ there); the model is now faithful for conflict/fetch cases too. Production unaffected (test-harness only). Addresses finding 1 of codex-tier2-final-review.
2026-06-15 18:44:05 +07:00
Nicholas Dudfield
a95306142e test(rng): tier-2 mint sim — in-band cohort at n=6
6 validators is the smallest non-degenerate tier-2 size (f=1, one-wide band {4}: tier2=4, quorum=5). Isolate 2 so the surviving 4-cohort is below the 80% quorum but at the tier-2 floor; it mints participant_aligned entropy (count 4), and all four agree on the same non-zero digest with branches==1 — no hang, no fork. Distributed confirmation of the selector ladder the unit tests already cover.
2026-06-15 17:48:53 +07:00
Nicholas Dudfield
8d43154061 feat(rng): model tier 2 in CSF sims + reconcile sub-quorum sims
The CSF peer now mirrors production's tier-2 behavior so the RNG simulations
exercise participant_aligned, not just validator_quorum/fallback:
- entropyGateThreshold() = min(quorumThreshold(), tier2Threshold()), plus a
  tier2Threshold() helper (calculateParticipantThreshold over the sim's UNL).
- finalizeRoundEntropy() labels by aligned count via the 3-tier ladder
  (>= quorum -> validator_quorum, >= tier2 -> participant_aligned, else
  fallback) instead of hardcoding tier 3.

The n%5==0 band-collapse keeps the bulk of the suite unaffected: at n=5
tier2 == quorum == 4, so the gate is unchanged for every n=5 and n<=2 network.
Only the two n=3 sub-quorum sims cross a live band (n=3: tier2=2 < quorum=3) and
now mint tier 2 instead of falling back -- the feature working, with each test's
intent preserved:
- "impossible quorum fallback" -> "quorum-impossible cohort falls to
  participant_aligned": 2 of 3 is below 80% but at the tier-2 floor, so it makes
  progress as tier 2 (no hang, no fork) rather than falling all the way back.
- "persistent loss does not shrink quorum": the 2 survivors now mint the
  labeled-weaker tier 2, NOT tier 3 -- the tier-3 quorum still did not shrink
  (a min_tier=3 hook rejects it). Intent preserved, outcome relabeled.

Other CSF suites (Consensus, ByzantineFailureSim) are unaffected (they do not
enable RNG). Next: dedicated tier-2 mint + conflict sims at n=6 (smallest size
with a non-degenerate band, f=1).
2026-06-15 17:41:47 +07:00
Nicholas Dudfield
156a8cbb85 refactor(rng): address review minors (dead-API rename + explicit-final dedup)
Two small follow-ups from the Codex review of the tier-2 implementation, both
on the consensus-extension internals:

- Rename shouldZeroEntropy() -> belowValidatorQuorum(). The selector replaced its
  callers, leaving it production-dead and MISNAMED: post-tier-2, "below the 80%
  validator quorum" is no longer "zero entropy" (a participant_aligned set is
  sub-quorum but non-zero). The new name + a doc comment make it a tier-3
  eligibility predicate only and warn against gating injection on it
  (selectEntropy() owns the tiering). Behavior unchanged.

- buildExplicitFinalProposalTxSet now dedups the entropy pseudo-tx by VALUE, not
  type. Its comment claimed it "mirrors onPreBuild", but onPreBuild went
  value-based: it verifies a present pseudo-tx is the EXACT txID it would have
  produced and logs a determinism-violation on mismatch. Explicit-final now does
  the same (skip-duplicate-verified on match, error log on mismatch), returning
  the base unchanged either way. Default-off experimental path, so low blast
  radius, but the two paths now agree.
2026-06-15 17:14:30 +07:00
Nicholas Dudfield
14ebe74a56 fix(rng): derive tier2 threshold from intersection bound (safe at n%5==0)
calculateParticipantThreshold returned ceil(0.6*n). At every n divisible by 5
(n=5,10,15,20,...) that leaves two aligned cohorts overlapping in exactly
floor(0.2*n) = f validators -- NOT strictly greater than f. So up to f Byzantine
nodes can occupy the entire overlap, leaving no honest validator shared between
the two cohorts, and a single equivocator backed by f-1 colluders can split the
round into two distinct aligned digests -> fork. The spec caught n=5 ("never 5")
but the same failure recurs at every multiple of 5.

Derive the floor from the safety invariant instead: the smallest t with
2t - n > floor(n/5), i.e. floor((n + floor(n/5)) / 2) + 1. This equals
ceil(0.6*n) everywhere except multiples of 5, where it is one higher
(n=10 -> 7, not 6). n=5 now collapses the band (tier2 == quorum), so the
"never 5" operational caveat is enforced by the math rather than a footnote.

Tests: the arithmetic test now asserts the defining invariant
2t - n > floor(n/5) AND t <= quorum for every n in 1..256 (this fails at n=10
under the old formula), plus the bumped boundaries. The onPreBuild tier-2 test
moves off n=5 (no band) onto n=6 (tier2=4, quorum=5; 5/4/3 revealers ->
validator_quorum / participant_aligned / fallback).

Found by Codex adversarial review of the tier-2 implementation.
2026-06-15 16:59:55 +07:00
Nicholas Dudfield
580f07ce35 feat(rng): enable tier 2 (participant_aligned) sub-quorum entropy
A 60-79% aligned cohort now mints entropy labelled participant_aligned (tier 2)
instead of falling back. Healthy >=80% rounds are unchanged (validator_quorum),
and a true minority (<60%) still falls back.

Mechanics:
- entropyGateThreshold() = min(quorumThreshold(), tier2Threshold()): the bar at
  which the pipeline engages and the entropy conflict gate resolves. In the
  normal band this is the 60% floor (of the ORIGINAL, pre-nUNL view); under
  heavy nUNL the band collapses to the 80% quorum and tier 2 vanishes.
- selectEntropy() labels the AGREED entropySetMap_ by participant count:
  >= quorum -> validator_quorum, >= tier2 -> participant_aligned, else fallback.
- Tick.h gates (bootstrap-skip, impossible-quorum, commit-timeout, entropy
  conflict gate) key off entropyGateThreshold() so sub-quorum rounds reach
  injection instead of short-circuiting.

Determinism: the tier LABEL is a function of the agreed set's leaf count
(identical on every node holding that hash); the local entropyGateThreshold
alignment only decides proceed-vs-fall-back, so divergent local views fall back
rather than fork. The 60% floor is over the ORIGINAL view -- the
quorum-intersection bound that stops a single equivocator minting two distinct
aligned digests under the ~20% Byzantine bound.

hasQuorumOfCommits() is deliberately LEFT at 80%: healthy networks keep their
exact fast-path and only step down to tier 2 via the commit-timeout path, so
this adds zero behavior change above quorum (the degraded band pays one pipeline
timeout; a fast-path is a separable follow-up). Folds into featureConsensusEntropy
(not yet active), so no new amendment.

Tests: new onPreBuild tier-2 case (5-validator view; 4/3/2 revealers ->
validator_quorum / participant_aligned / fallback) plus threshold assertions.
CSF Peer and the tick test stub mirror entropyGateThreshold() to quorumThreshold()
for now -- the end-to-end tier-2 sims (which lower it) land next. Extracted
makeUNLReportLedger / harvestCommitReveal test helpers, now shared across the
view and harvest tests.
2026-06-15 15:55:43 +07:00
Nicholas Dudfield
d4c0ba3769 refactor(rng): unify entropy selection into one deterministic selector
Collapse the duplicated tier-selection logic in onPreBuild and
buildExplicitFinalProposalTxSet into a single selectEntropy() over the AGREED
entropySetMap_. No behavior change on the production (implicit) path — the
onPreBuild fallback/entropy-set/standalone/mismatch tests pass byte-identically.

Fixes the pre-existing divergence flagged in review: buildExplicitFinalProposalTxSet
derived entropy from local pendingReveals_ while onPreBuild used the agreed
entropySetMap_, so the two could mint different digests for the same round.
Both now share the selector, so the implicit and (experimental, default-off)
explicit-final paths — and any two nodes — derive identical entropy from
identical agreed inputs.

selectEntropy() returns {digest, tier, count} and is a pure function of agreed
round state, so it is directly unit-testable. Injection becomes unconditional:
the selector always yields a fallback digest when there is no validator entropy,
so every RNG-enabled ledger still carries exactly one ConsensusEntropy tx.

Sets up the tier-2 (participant_aligned) ladder, which lands next.
2026-06-15 15:22:14 +07:00
Nicholas Dudfield
378d6b78c8 feat(rng): add pre-nUNL originalViewSize + tier2 participant threshold
Foundation for Tier 2 (participant_aligned) sub-quorum entropy. No behavior
change — nothing consumes these yet; the tier ladder, gates, and selector
arrive in later commits.

- ActiveValidatorView::originalViewSize: master-key count BEFORE the nUNL
  subtraction. size() stays the effective (post-nUNL) count used by the 80%
  validator-quorum gate; originalViewSize is the original-UNL denominator that
  the 60% Tier 2 floor anchors to, since nUNL can shrink the effective view
  while leaving faulty nodes in it.
- calculateParticipantThreshold(): ceil(0.6 * count), the quorum-intersection
  floor (two such cohorts always share an honest validator under the ~20%
  Byzantine bound).
- ConsensusExtensions::tier2Threshold(): ceil(0.6 * originalView), anchored to
  originalViewSize.

Tests: originalViewSize asserted on the UNLReport, fallback, and real-ledger
nUNL paths; new arithmetic testcase locks ceil(0.6) and the sizing-note
boundaries (5->3 banded/unsafe, 6->4 smallest-safe, 8->5 one-nUNL-off).
2026-06-15 15:11:41 +07:00
Nicholas Dudfield
4b219cdef8 test(testnet): use strength-ordered enum names, drop 'Tier 3' shorthand
The 'Tier N' preference-order shorthand collides with EntropyTier's
strength-ordered values (consensus_fallback=1). Use the enum names in
the degradation-smoke comments/descr to remove the ambiguity.
2026-06-12 17:24:23 +07:00
Nicholas Dudfield
9d4b97c824 fix(rng): defensive mismatch-log reads + mismatch-path test
Follow-up to d6481a386, addressing review of that diff:

- The mismatch-log branch in onPreBuild read present pseudo-tx fields
  (sfDigest/sfEntropyTier/sfEntropyCount) unconditionally — the same
  throw hazard fairRng was hardened against, here inside onPreBuild
  during build. Read them defensively (isFieldPresent ? value :
  '<missing>'). (Note: STTx deserialization enforces all soeREQUIRED
  fields, so a tier-less ttCONSENSUS_ENTROPY cannot actually reach the
  set — this is belt-and-suspenders, not a reachable bug.)
- Clarify the comment + add action=keep-agreed-and-flag: this is
  detect-and-log, NOT rejection. The present pseudo-tx is KEPT and still
  applied at BuildLedger; a hard-fail policy on mismatch is a deliberate
  future decision (determinism-violation vs halt-risk-under-skew).
- Add testOnPreBuildEntropyMismatchKeepsAgreed: a present-but-different
  entropy pseudo-tx is kept (not replaced), set stays at one entry.
- Fix stale 'type-based dedup' comment in the standalone test.
2026-06-12 17:09:53 +07:00
Nicholas Dudfield
d6481a3869 fix(rng): value-based entropy pseudo-tx dedup + tier-read hardening
Address review findings on the tier 3 commits:

- onPreBuild dedup was type-based (skip if any ttCONSENSUS_ENTROPY
  present), which silently trusts a pre-present pseudo-tx. Injection is
  deterministic, so every honest node derives the identical pseudo-tx
  (identical txID) for the same agreed inputs. Switch to value-based
  dedup: skip only when the present pseudo-tx EQUALS the one we would
  produce; a present-but-different entropy pseudo-tx is a determinism
  violation (version skew / divergent peer) and is now logged at error
  rather than accepted blindly. (The earlier 'cannot reconstruct the
  txID locally' justification was wrong — determinism guarantees it can.)
- fairRng: read sfEntropyTier defensively (missing => 0 => fail closed).
  The field is soeREQUIRED so any entry this code wrote carries it; this
  only guards a pre-tier-3 persisted entry on a long-lived testnet.
- quorum_degradation_smoke: assert EntropyTier==consensus_fallback(1) and
  EntropyCount==0 and non-zero digest explicitly, not just is_fallback.
2026-06-12 16:39:39 +07:00
Nicholas Dudfield
0cf6f73441 test(testnet): align degradation smoke comments with tier 3 semantics 2026-06-11 13:57:53 +07:00
Nicholas Dudfield
08a6f3cd57 docs(consensus): describe tier 3 fallback entropy semantics 2026-06-10 16:58:43 +07:00
Nicholas Dudfield
77d78236e8 feat(hooks): dice/random take required min_tier + min_count args
The entropy-quality requirement becomes an explicit, required argument
at every call site — there is deliberately no default and no network
constant:

- dice(sides) -> dice(sides, min_tier, min_count)
- random(ptr, len) -> random(ptr, len, min_tier, min_count)
- fairRng gates on freshness && tier >= min_tier && count >= min_count;
  the hard-coded 'EntropyCount >= 5' network constant is deleted —
  the hook author states what their application needs and gets
  TOO_LITTLE_ENTROPY when this ledger cannot meet it
- WASM imports have no default parameters, so the old no-argument shape
  was really a hidden constant invisible at the call site; requiring the
  arguments makes weak-entropy acceptance (min_tier=1) a deliberate,
  reviewable opt-in rather than an accident
- hook/extern.h and hook/sfcodes.h regenerated (CI-verified)
- test hooks updated and recompiled; new test pins the requirement
  gate: dice(6, 3, 21) against count=20 returns TOO_LITTLE_ENTROPY
2026-06-10 16:56:05 +07:00
Nicholas Dudfield
c92c0656ec feat(rng): tier 3 consensus-bound fallback entropy
Replace the zero-entropy fallback with a deterministic consensus-bound
digest so every RNG-enabled ledger carries usable entropy:

- sha512Half(HashPrefix::entropyFallback, prevLedgerHash, baseTxSetHash,
  seq) — all inputs are already consensus-agreed at injection time, so
  no new agreement machinery is needed and the digest is identical on
  every node building the same ledger
- new sfEntropyTier (UINT8) on the ttCONSENSUS_ENTROPY pseudo-tx and
  ConsensusEntropy ledger entry: EntropyCount says how many validators
  contributed, EntropyTier says which gate the result passed
  (validator_quorum vs consensus_fallback; participant_aligned reserved)
- the fallback digest derives from the BASE (pre-injection) tx set hash
  to avoid circularity; entropy pseudo-tx dedup is now type-based since
  an explicit-final synthetic set can carry a pseudo-tx whose txID
  implicit nodes cannot re-derive
- unparseable-entropy-set residual now falls back instead of skipping
  injection, so a fresh ConsensusEntropy entry exists every ledger
- CSF Peer mirrors the fallback analog; sims assert deterministic
  non-zero fallback digests across same-LCL peers
- testnet scenarios updated: degraded windows expect labeled fallback
  entropy, never validator-tier

The fallback tier is user-influenceable via tx submission (quiet-ledger
grinding) and is labeled accordingly — hook-facing gating lands with the
min_tier/min_count API change.
2026-06-10 16:32:49 +07:00
Nicholas Dudfield
fbdec3be66 chore: update levelization results 2026-06-10 12:46:09 +07:00
Nicholas Dudfield
bb619cc100 fix: defer proposal feature checks 2026-06-10 11:09:12 +07:00
Nicholas Dudfield
10f22c84f2 fix: preserve malformed proposal rejection order 2026-06-10 09:08:33 +07:00
Nicholas Dudfield
dd21024c0e test: improve export rng coverage 2026-06-09 19:56:46 +07:00
Nicholas Dudfield
16a72172b4 test(coverage): cover xpop empty proof output 2026-06-09 14:29:32 +07:00
Nicholas Dudfield
265012e16a test(coverage): expand runtime config option coverage 2026-06-09 14:26:04 +07:00
Nicholas Dudfield
d072527bc5 test(coverage): cover consensus extension refresh paths 2026-06-09 14:23:20 +07:00
Nicholas Dudfield
5f5ce12fa6 test(coverage): cover export signature helper edges 2026-06-09 14:18:12 +07:00
Nicholas Dudfield
0804a01b9b test(coverage): cover consensus extension tick states 2026-06-09 14:12:54 +07:00
Nicholas Dudfield
80cd1bed34 test(coverage): cover export metadata roundtrip 2026-06-09 13:50:44 +07:00
Nicholas Dudfield
b4e98ac1d7 test(coverage): cover peer proposal wrapper 2026-06-09 13:40:04 +07:00
Nicholas Dudfield
4f7e751fbd test(coverage): avoid xpop levelization dependency 2026-06-09 13:36:25 +07:00
Nicholas Dudfield
748fef6267 test(coverage): cover xpop proof edge cases 2026-06-09 13:32:26 +07:00
Nicholas Dudfield
4d48c9f949 test(coverage): cover export rng helper seams 2026-06-09 13:21:08 +07:00
Nicholas Dudfield
03c1216661 merge: absorb origin/dev 2026-06-09 09:44:39 +07:00
Nicholas Dudfield
537474cb5f style(consensus): apply ci clang-format 2026-06-09 09:20:27 +07:00
Nicholas Dudfield
ec086d6765 test(export): cover signature upgrade race guard 2026-06-08 15:50:32 +07:00
Nicholas Dudfield
7362c1dac1 fix(hook): preserve xport nonce failure result 2026-06-08 15:50:17 +07:00
Nicholas Dudfield
f0550ca625 refactor(export): extract signature upgrade policy 2026-06-08 14:49:16 +07:00
Nicholas Dudfield
526b60bf3d refactor(hook): extract xport wrapper builder 2026-06-08 14:37:58 +07:00
Nicholas Dudfield
9347b47639 refactor(export): extract result assembly 2026-06-08 14:21:44 +07:00
Nicholas Dudfield
9988568a08 refactor(consensus): extract export signature harvester 2026-06-08 14:10:43 +07:00
Nicholas Dudfield
13260b9ef7 test(runtime-config): cover effective config paths 2026-06-08 13:58:39 +07:00
Nicholas Dudfield
65dab780de refactor(consensus): share sidecar alignment scan 2026-06-05 15:07:44 +07:00
Nicholas Dudfield
ee70e4cdbb refactor(consensus): extract active validator view builder 2026-06-05 15:02:51 +07:00
Nicholas Dudfield
7fb1509673 test(testnet): harden scenario feature checks 2026-06-04 15:50:31 +07:00
Nicholas Dudfield
64620e2825 test(testnet): assert export degradation logs 2026-06-02 12:30:07 +08:00
Nicholas Dudfield
a87a7896ca test(testnet): align degradation log checks 2026-06-02 12:25:12 +08:00
Nicholas Dudfield
443aca8611 chore(logging): polish RNG final diagnostics 2026-06-01 14:08:12 +08:00
Nicholas Dudfield
804b76b4ab chore(logging): normalize consensus extension diagnostics 2026-06-01 13:57:46 +08:00
tequ
c55420bcd8 Fix duplicate and incorrect fields in server_definitions (#753) 2026-05-27 07:58:13 +10:00
tequ
cb91b4e88e Restore pre-reserved tem codes (#754) 2026-05-27 07:57:55 +10:00
tequ
90333b6fd0 Fix HookAPI Expected and Refactor Enum classes (#729) 2026-05-26 11:02:17 +10:00
tequ
7f9a9364b0 fix: Ensures canonical order for PriceDataSeries upon PriceOracle creation (#5485) (#744) 2026-05-25 11:34:24 +10:00
tequ
706d31f01d Update .git-blame-ignore-revs (#746) 2026-05-25 11:28:56 +10:00
tequ
083e9e4315 Generate hook extern declarations with C linkage (#750) 2026-05-25 11:27:21 +10:00
tequ
b9e0c56def Guard depth 32 (#653) 2026-05-25 11:10:49 +10:00
Nicholas Dudfield
ab6571a20f fix(runtime-config): avoid overlay dependency 2026-05-22 14:58:22 +08:00
Nicholas Dudfield
331e1606a3 feat(runtime-config): support startup message filters 2026-05-22 14:50:59 +08:00
Nicholas Dudfield
24d6dea1a2 test(testnet): enable consensus extension diagnostics
Add ConsensusExtensions debug logging to the entropy and export scenario suite defaults so participant bitmap diagnostics are captured without live log-level changes.
2026-05-21 10:21:56 +08:00
Nicholas Dudfield
03e0bb5fc3 feat(consensus): add active participant diagnostics
Carry a signed observedParticipantsHash in ExtendedPosition and log a local canonical bitmap over the active validator view for degraded-round debugging. The field is diagnostic only and does not affect proposal equality or quorum thresholds.
2026-05-21 09:56:51 +08:00
Nicholas Dudfield
0a77dbf68e Merge remote-tracking branch 'origin/dev' into feature-export-rng 2026-05-19 12:11:18 +08:00
tequ
663ed4edb8 Named Hook (#718) 2026-05-19 12:00:25 +10:00
tequ
8422758d4d chore: Improve codecov coverage reporting (#4977) (#745) 2026-05-18 12:21:47 +10:00
Nicholas Dudfield
60a9a2c9fb Merge remote-tracking branch 'origin/dev' into feature-export-rng
# Conflicts:
#	include/xrpl/protocol/Feature.h
#	include/xrpl/protocol/detail/features.macro
2026-05-18 09:15:17 +07:00
tequ
586c78e812 fix: Replace badCurrency() checks with isBadCurrency() for improved clarity (#742) 2026-05-18 11:57:25 +10:00
Niq Dudfield
9b50b68d39 perf(ledger): optimize catalogue loading memory usage and performance (#548) 2026-05-06 17:29:44 +10:00
tequ
5e8d26f67a refactor: Calculate numFeatures automatically (#5324) (#739)
Co-authored-by: Ed Hennis <ed@ripple.com>
2026-04-30 18:17:50 +10:00
Nicholas Dudfield
445d0070d8 Merge remote-tracking branch 'origin/dev' into feature-export-rng
# Conflicts:
#	hook/sfcodes.h
#	include/xrpl/protocol/Feature.h
#	include/xrpl/protocol/detail/sfields.macro
2026-04-30 14:22:06 +07:00
tequ
a6186d7855 IOURewardClaim (#500) 2026-04-30 15:27:51 +10:00
Nicholas Dudfield
61a8d8bba7 chore(hook): update generated tx flags 2026-04-30 11:03:58 +07:00
Nicholas Dudfield
fbedb8a73a Merge remote-tracking branch 'origin/dev' into feature-export-rng 2026-04-30 10:57:07 +07:00
Nicholas Dudfield
8ae541fcc1 Merge remote-tracking branch 'origin/dev' into feature-export-rng
# Conflicts:
#	src/test/app/SetHookTSH_test.cpp
#	src/xrpld/app/tx/detail/InvariantCheck.cpp
#	src/xrpld/app/tx/detail/InvariantCheck.h
2026-04-30 08:14:51 +07:00
Nicholas Dudfield
c8f3f6f05f docs(consensus): clarify extension fallback semantics 2026-04-29 15:27:46 +07:00
Nicholas Dudfield
b12cee5d47 fix(consensus): wait for sidecar observation
Require tx-converged peers to advertise sidecar hashes before accepting RNG entropy or export signature success from local quorum alignment.

The RNG reveal fast path now publishes the entropy set and waits for peer observation instead of accepting in the same tick. On timeout, RNG clears the advertised entropy hash and falls back to deterministic zero.

Add unit and CSF regression coverage for asymmetric peer observation.
2026-04-28 12:20:42 +07:00
Nicholas Dudfield
a3b1e45f4d fix(consensus): bound export sidecar observation
When a candidate set contains ttEXPORT but a node has no local verified export sig material yet, give tx-converged peers one bounded opportunity to advertise an exportSigSetHash before closed-ledger apply.

This is a safety coordination window, not a wait-for-Export-success mechanism. If no advertised sidecar arrives or fetched material cannot be merged by the deadline, Export convergence is marked failed and the transaction retries or expires through normal rules.

Add CSF coverage for a peer that can only succeed by fetching peer-advertised export sidecars, plus a direct ConsensusExtensionsTick test for the pre-advertisement observation window. Document the consensus-extension priority order: safe, fast, works.
2026-04-28 11:15:03 +07:00
Nicholas Dudfield
3938ba7af4 docs(consensus): record export sidecar material flow
Clarify that export sidecar publication is local verified material only, and fetched sidecar leaves must be active-view checked, candidate-tx verified, and promoted into ExportSigCollector before closed-ledger apply can use them.
2026-04-28 10:59:00 +07:00
Nicholas Dudfield
96b1104646 fix(consensus): use quorum for export-only sidecars
Export-only originally used unanimity as a conservative substitute for the CE/RNG sidecar machinery. That made sense before Export had its own signed ExtendedPosition field and exportSigSetHash convergence gate.

Now Export sidecars are signed and converged independently of RNG, so a quorum-aligned exportSigSetHash plus verified active-view signature quorum is deterministic enough for Export-only mode. Keeping unanimity would let one active validator veto an otherwise converged export round.

Update CSF and testnet coverage to treat Export-only the same way: one missing/conflicting signer in a 5-validator network succeeds at 4/5, while below-quorum still retries or expires.
2026-04-28 10:54:04 +07:00
Nicholas Dudfield
92bdd2ed9f fix(consensus): harvest replayed export signatures 2026-04-28 10:16:52 +07:00
Nicholas Dudfield
d87cfdc604 fix(consensus): clear export sigs when export disabled 2026-04-28 09:17:17 +07:00
Nicholas Dudfield
a956abb2d1 docs(consensus): clarify export sig quorum gates 2026-04-28 08:56:53 +07:00
Nicholas Dudfield
aa36a80ab7 docs(consensus): document sidecar acquire rationale 2026-04-28 08:35:45 +07:00
Nicholas Dudfield
e729aa11eb fix(hooks): preserve finalization semantics
Keep hook result/state finalization non-fatal while enforcing the hook-export backlog cap through the transaction-level ApplyContext guard. This avoids resetting non-success tec metadata and preserves hook_again weak execution behavior.
2026-04-28 07:47:20 +07:00
Nicholas Dudfield
c58da3da58 fix(export): cap hook export backlog
Enforce the pending export cap for hook-emitted ttEXPORT work before commit. Replace the non-present sfEmittedTxn template field when building ltEMITTED_TXN entries so in-flight ledger checks see the emitted wrapper.

Overflowing xport emission now returns tecDIR_FULL and leaves the emitted backlog capped at ExportLimits::maxPendingExports.
2026-04-27 22:55:23 +07:00
Nicholas Dudfield
0c2c59d258 fix(export): enforce pending export limits
Cap pending ttEXPORT work in open/apply ledgers, including hook-emitted exports when TxQ drains the emitted directory into the open ledger. Enforce the same bound for per-account shadow tickets so durable pending imports cannot grow unbounded.
2026-04-27 21:30:24 +07:00
Nicholas Dudfield
15662eb1b1 fix(consensus): cap export proposal signatures
Limit outbound TMProposeSet export signature attachments to ExportLimits::maxPendingExports so honest proposals stay within the same bound enforced by inbound proposal validation. Extra exports remain unsigned for that proposal and rely on the existing retry/expiry path.
2026-04-27 21:03:00 +07:00
Nicholas Dudfield
492fe90643 fix(consensus): expire stale export signatures
Stamp export signatures learned from proposals, sidecar sets, and candidate tx-set upgrades with a ledger sequence so cleanupStale can age them out. Remove invalid unverified signatures after tx-local verification fails, with a buffer match check to avoid deleting newer replacements.
2026-04-27 20:56:54 +07:00
Nicholas Dudfield
ea413873b2 fix(consensus): preserve export state without rng 2026-04-27 20:34:58 +07:00
Nicholas Dudfield
625419eab7 fix(consensus): verify export sigs against tx set 2026-04-27 18:07:09 +07:00
Nicholas Dudfield
2218bdd7f3 fix(consensus): require export sigset quorum alignment 2026-04-27 17:36:06 +07:00
Nicholas Dudfield
f13233b00a docs(consensus): clarify validation sidecar signing rule
Remove the stale TMValidation exportSignatures field from the draft proto path now that export signatures ride signed proposal sidecars. Document that any future validation-carried ConsensusExtensions data must be covered by the signed validation payload and duplicate/replay identity, not an unsigned wrapper field.
2026-04-27 15:45:27 +07:00
Nicholas Dudfield
a61f334ca2 docs(consensus): capture extension design principles
Document the consensus-extension invariants for RNG, sidecars, export sig convergence, validator quorum, zero-entropy fallback, and proposal signing. Link the note from the RCL consensus README so future changes have a durable checklist.
2026-04-27 15:33:15 +07:00
Nicholas Dudfield
53a119ce30 fix(consensus): require rng entropy quorum alignment
Count the local proposer when deciding whether the previous round had enough participants for RNG, since prevProposers only tracks peers. This avoids a 4/5 honest quorum being treated as below quorum after one validator diverges.

Allow an already quorum-aligned entropySetHash to proceed despite below-quorum conflicting hashes, while retaining zero-entropy fallback when no entropy hash reaches quorum alignment. Add CSF coverage for a persistent single bogus entropy hash and for conflicting bogus hashes without quorum.
2026-04-27 15:29:36 +07:00
Nicholas Dudfield
63d1197345 fix(consensus): zero rng on unresolved entropy hash conflict 2026-04-27 15:10:39 +07:00
Nicholas Dudfield
aafd5b940b test(consensus): avoid brittle rng lcl quorum check 2026-04-27 14:47:18 +07:00
Nicholas Dudfield
efc497cf23 chore(levelization): refresh app overlay loop summary
This does not introduce a new levelization cycle; the existing xrpld.app <-> xrpld.overlay loop now has equal aggregate include counts after the consensus-extension work. Treat this as essentially the same architectural situation, not a meaningful worsening by itself.

TODO: if we want to fix the boundary properly, extract a small shared consensus-extension wire/interface layer below both app and overlay instead of shaving includes to change the generated ratio.
2026-04-27 14:01:54 +07:00
Nicholas Dudfield
f4e78c9a24 fix(consensus): apply negative unl to sidecar validator view 2026-04-27 12:50:43 +07:00
Nicholas Dudfield
7b5865c69c fix(consensus): sign export proposal attachments 2026-04-27 11:57:29 +07:00
Nicholas Dudfield
9f1ad521e1 fix(consensus): use active validator snapshots for sidecars 2026-04-27 10:59:33 +07:00
Nicholas Dudfield
26bbef8efd fix(consensus): harden sidecar quorum inputs 2026-04-27 10:14:12 +07:00
Nicholas Dudfield
6e71f84867 refactor: add typed sidecar SHAMap sync 2026-04-27 09:58:34 +07:00
Nicholas Dudfield
ab9b48f67a Merge remote-tracking branch 'origin/dev' into feature-export-rng
# Conflicts:
#	.github/workflows/levelization.yml
#	Builds/levelization/README.md
#	Builds/levelization/levelization.py
#	Builds/levelization/levelization.sh
#	cmake/RippledCore.cmake
2026-04-27 09:14:59 +07:00
Alloy Networks
cd00ed72d8 change build instructions url 2026-04-24 11:12:28 +10:00
tequ
05a3e04f2d Fix BEAST_ENHANCED_LOGGING not working and restore original behavior 2026-04-24 11:11:40 +10:00
tequ
66f7294120 Test: hint build_test_hooks.sh when hook wasm is empty in hso() 2026-04-24 11:10:46 +10:00
Nicholas Dudfield
7f6ac75617 Revert "chore: use improved levelization script with threading and argparse"
This reverts commit 5c1d7d9ae9.
2026-04-24 11:09:19 +10:00
Nicholas Dudfield
4150f0383c chore: use improved levelization script with threading and argparse 2026-04-24 11:09:19 +10:00
Nicholas Dudfield
25123b370a chore: replace levelization shell script with python
Backport of XRPLF/rippled#6325. The python version runs ~80x faster.
2026-04-24 11:09:19 +10:00
tequ
f90ed41802 enable ccache direct_mode 2026-04-24 11:06:51 +10:00
tequ
8c4c158d3a output ccache configuration in release-builder 2026-04-24 11:06:51 +10:00
tequ
2d2951875d fix: typo SignersListSet 2026-04-24 11:05:20 +10:00
tequ
9bfca63574 Update util_keylet fee test 2026-04-24 11:00:31 +10:00
tequ
1ba444ae7f Updated tests to align with the changes merged into the dev branch. 2026-04-24 11:00:31 +10:00
tequ
f96d9b6e51 Add tests for Hooks fee 2026-04-24 11:00:31 +10:00
Nicholas Dudfield
04077c1a55 test(testnet): assert zero entropy in degraded ledgers 2026-04-10 12:04:46 +07:00
Nicholas Dudfield
d94079d762 test(rng): relax PartialReveals sync assertion 2026-04-10 11:18:52 +07:00
Nicholas Dudfield
92ec07a1be chore: regenerate hook/sfcodes.h + format fix
Regenerate sfcodes.h to include the new sfSidecarType field
(UINT8, code 20).  Also apply clang-format to ConsensusExtensions.cpp.
2026-04-10 10:36:50 +07:00
Nicholas Dudfield
664db62588 fix: sidecar kind lost on cache hit + harden export sig parse
1. Record SidecarKind in pendingRngFetches_ before calling
   onAcquiredSidecarSet on local-cache-hit path. Without this,
   cached reveal/exportSig sets silently fell back to commit kind
   and were rejected by the sfSidecarType check.

2. Wrap export sig visitLeaves callback in try/catch (matching the
   RNG path) and enforce sfSidecarType == sidecarExportSig before
   processing — closes the shape-only acceptance gap.
2026-04-10 10:22:58 +07:00
Nicholas Dudfield
03a436d918 refactor: convert sidecar SHAMap entries from STTx to STObject
Replace STTx-based sidecar entries with plain STObject(sfGeneric)
using sfSidecarType (UINT8) discriminator. Eliminates unnecessary
transaction envelope overhead (sfSequence, sfFee, sfFlags) and
content-sniffing heuristics from the parse path.

Build: STObject with sidecarRngCommit/sidecarRngReveal/sidecarExportSig
Parse: sfSidecarType dispatch + typed field accessors
2026-04-10 10:14:06 +07:00
Nicholas Dudfield
7474048295 refactor: typed sidecar dispatch — eliminate content-sniffing heuristic
Replace the content-sniffing heuristic in onAcquiredSidecarSet with
typed dispatch based on SidecarKind.

The type is already known at fetch time:
- commitSetHash → SidecarKind::commit
- entropySetHash → SidecarKind::reveal
- exportSigSetHash → SidecarKind::exportSig

pendingRngFetches_ changes from hash_set<uint256> to
hash_map<uint256, SidecarKind>.  When the set arrives,
look up the kind by hash and dispatch — no leaf inspection.

This is the set-classification fix (Option E from the design doc):
no new SField, no STTx changes, no protocol additions, no RNG
proof-chain churn.
2026-04-10 09:18:43 +07:00
Nicholas Dudfield
1ee660529e fix: RPC handler sync, unused local, idiomatic Buffer comparison
- Add rng_poll_ms, no_export_sig, bootstrap_fast_start to the
  runtime_config RPC handler (SET and GET paths) so all ConfigVals
  fields are configurable live via admin RPC.
- Remove unused `added` counter in CSF fetchRngSetIfNeeded (was
  causing compiler warnings after debug logging removal).
- Use Buffer::operator== instead of std::memcmp in upgradeSignature,
  drop <cstring> include.
2026-04-10 08:56:16 +07:00
Nicholas Dudfield
311dfa1c23 chore: add TODO for RuntimeConfig activation gating
Both runtime_config and disconnect RPC handlers are already
Role::ADMIN.  Add a TODO to consider gating the entire
RuntimeConfig system on a config flag or compile-time define
for production nodes.
2026-04-10 08:31:54 +07:00
Nicholas Dudfield
f27cd2c567 refactor: consolidate env vars into RuntimeConfig
Move XAHAU_RNG_POLL_MS and XAHAUD_NO_EXPORT_SIG into RuntimeConfig
as rngPollMs and noExportSig fields.  Both are now configurable via
the XAHAU_RUNTIME_CONFIG JSON blob or individual env vars, and
controllable at runtime via the runtime_config RPC.

rngPollMs is clamped to minimum 50ms (prevents tight-loop polling).
Default remains 250ms.

This removes the last loose std::getenv calls from production code
outside of RuntimeConfig.  All env-var-based configuration now flows
through a single system.
2026-04-10 08:24:20 +07:00
Nicholas Dudfield
f34fdc297c fix(export): close upgradeSignature TOCTOU with buffer comparison
upgradeSignature now takes the verified buffer and compares it against
the currently stored buffer before promoting to verified.  This guards
against concurrent overlay threads overwriting the buffer between the
caller's unverifiedSignatures() snapshot and the upgrade call.

If the stored buffer was overwritten (different size or content), the
upgrade is silently skipped — the new buffer will be verified on its
next encounter.
2026-04-10 08:19:45 +07:00
Nicholas Dudfield
65fa63883d chore: remove CSF debug logging that floods CI output
Strip JLOG(j_.debug()) calls from buildEntropySet, fetchRngSetIfNeeded,
and finalizeRoundEntropy in CSF Peer.h.  These were added for local
debugging and caused CI failures due to output size limits.
2026-04-09 20:21:37 +07:00
Nicholas Dudfield
d8c683fb4c test(rng): fix AlignmentRequired test to run 1 round not 3
Running 3 rounds caused peer 0 to desync on round 2, dropping
prevProposers for the majority on round 3, triggering bootstrap
skip → zero entropy on the last round.  The gate works correctly
(logs show aligned=3, peersSeen=3) but the test was checking the
LAST round's entropy, not the round where the gate was exercised.

Run 1 round after warmup — sufficient to exercise the gate.
2026-04-09 18:09:17 +07:00
Nicholas Dudfield
fd53af304b fix(rng): measure entropy deadline from publish time, not reveal start
The entropy convergence deadline was measured from revealPhaseStart_,
which is set when entering ConvergingReveal.  By the time the entropy
set is published (after reveal timeout + observation tick), most of
the deadline budget was already spent — leaving insufficient time
for peer alignment.

Add entropyPublishStart_ timestamp set when the entropy set is first
published.  All convergence gate deadlines now measure from this
point, giving the full 2x rngREVEAL_TIMEOUT window for peer
proposals to propagate and alignment to be observed.
2026-04-09 18:06:18 +07:00
Nicholas Dudfield
2a3f0ec923 fix(rng): bounded wait for alignment instead of immediate fallback
When peers have published entropySetHash but none match ours yet
(e.g. a subset peer is the only one seen so far), wait for the
bounded deadline instead of immediately falling back to zero.
Other aligned peers may not have published yet — give them time.

Only fall back to zero if no alignment is observed within the
deadline (2x rngREVEAL_TIMEOUT).
2026-04-09 17:58:41 +07:00
Nicholas Dudfield
00f1f7ba30 fix(rng): subset-aware conflict detection in entropy convergence gate
After fetch/merge, if our entropy set hash didn't change, the
conflicting peer had a subset of our data — not a real threat.
Clear the conflict flag so we don't fall back to zero when a peer
simply has fewer reveals than us.

If the hash DID change (merge added data), re-count alignment
with the updated hash before treating it as a real conflict.

This prevents the majority from falling back to zero just because
one peer (e.g. isolated) has a smaller reveal set.
2026-04-09 17:53:58 +07:00
Nicholas Dudfield
49f05e4e47 fix(rng): require positive peer alignment for non-zero entropy
The observation tick alone was insufficient — a node could pass the
gate without any peer confirming its entropySetHash.  Now the gate
requires at least one tx-converged peer with a matching hash before
accepting non-zero entropy.

Three cases after the observation tick:
1. aligned > 0: peers confirm our hash → proceed with entropy
2. conflict: fetch/merge/rebuild → bounded wait → zero fallback
3. aligned=0, peersSeen=0: no peers published yet → bounded wait →
   zero fallback if still no peers at deadline
4. aligned=0, peersSeen>0: peers published but none match → zero

Also:
- CSF finalizeRoundEntropy now uses shouldZeroEntropy() (quorum check)
- Two new TDD tests:
  - testRngNoEntropyWithoutPeerAlignment: healthy network must agree
  - testRngAlignmentRequiredForNonZeroEntropy: isolated peer must not
    produce non-zero entropy that differs from majority
2026-04-09 17:51:51 +07:00
Nicholas Dudfield
1f51b9c594 fix(csf): quorum threshold in shouldZeroEntropy + test adjustments
CSF shouldZeroEntropy() now checks reveals < quorumThreshold (80% of
UNL), matching production.  MajorRevealLoss test adjusted to verify
majority group agreement rather than requiring full synchronization
(peer 0 may desync when it misses most reveals).

All 15 ConsensusRng tests now pass.
2026-04-09 17:40:05 +07:00
Nicholas Dudfield
88a548a8ef fix(rng): observation tick + CSF quorum threshold in shouldZeroEntropy
Two fixes addressing the asymmetric-view problem:

1. Convergence gate now forces one observation tick after first
   publishing the entropySet before accepting.  Previously a node
   could publish + accept in the same tick, never seeing a peer's
   different hash.  The entropySetPublished_ flag ensures at least
   one round-trip for proposal propagation.

2. CSF shouldZeroEntropy() now checks quorum threshold (80% of UNL),
   matching production behavior.  Previously it only checked empty().

Result: PartialReveals test now passes — all 6 peers converge on
the same entropy (count=6) via union merge after the observation tick.
14/15 ConsensusRng tests pass.
2026-04-09 17:31:36 +07:00
Nicholas Dudfield
db302a0f78 fix(rng): add selfSeedReveal to fix CSF reveal counting
The CSF never self-seeded its own reveal into pendingReveals_ because
harvestRngData only processes peer proposals, not self.  The real code
handles this in decorateMessage, but the CSF has no equivalent.

Add selfSeedReveal() called from the tick at reveal transition.
Both the real ConsensusExtensions and the CSF Extensions implement it.
The real code now has belt-and-suspenders: tick + decorateMessage.

This fixes CSF peers having N-1 reveals instead of N, which caused
every peer to compute entropy from a different subset.
2026-04-09 17:23:53 +07:00
Nicholas Dudfield
383d9ec2e7 feat(csf): add SidecarStore for sidecar set fetch/merge simulation
Add a content-addressed SidecarStore to the CSF, simulating the
InboundTransactions SHAMap fetch pipeline.  Tagged entries (commit
or reveal) are published by hash during buildCommitSet/buildEntropySet
and fetched by hash during fetchRngSetIfNeeded, with type-aware
union merge into the correct local pending set.

Also adds debug logging to CSF Extensions for entropy pipeline
troubleshooting.
2026-04-09 17:17:53 +07:00
Nicholas Dudfield
52671bfc99 test(rng): add XAHAU_RNG_TEST env var filter for focused test runs
Set XAHAU_RNG_TEST=<substring> to run only matching test methods.
e.g. XAHAU_RNG_TEST=SingleByzantine runs only that test.
2026-04-09 16:51:26 +07:00
Nicholas Dudfield
8307fca3b9 fix(rng): add entropySetHash convergence gate before accept
Add a bounded pre-accept convergence check for entropySetHash,
closing the gap where two honest validators could accept with
different reveal subsets and compute different entropy (ledger fork).

After publishing the entropy set, the gate:
1. Inspects tx-converged peer positions for conflicting entropySetHash
2. Fetches differing sets via fetchRngSetIfNeeded (union merge)
3. Rebuilds and re-publishes the local entropy set after merge
4. Waits within a bounded window (2x rngREVEAL_TIMEOUT)
5. Falls back to zero entropy if conflict persists past deadline

This follows the same pattern as the existing commitSetHash conflict
handling and exportSigSetHash convergence gate.  Union merge ensures
monotonic set growth — honest timing skew resolves quickly, and
hostile hash spam hits the hard deadline and falls back safely.

The "one bad actor shouldn't deny entropy" optimization (supermajority
vote) is deferred to a follow-up patch per codex recommendation.
2026-04-09 16:30:02 +07:00
Nicholas Dudfield
6526621c16 test(rng): add TDD tests for entropySetHash convergence gate
Three new CSF tests that document expected behavior for the
entropySetHash convergence gate (not yet implemented):

1. testRngEntropyConvergesWithPartialReveals: two groups each drop
   one peer's reveal, creating different quorate subsets.  Must not
   fork — either converge via SHAMap merge or both fall back to zero.

2. testRngEntropyFallbackOnMajorRevealLoss: one peer drops most
   reveals (below quorum locally).  Network must still agree.

3. testRngSingleByzantineCannotDenyEntropy: one Byzantine peer
   (future: forced garbage entropySetHash) should not prevent the
   other 80% from producing valid entropy.

Also adds dropRevealFrom_ test knob to CSF Peer::Extensions for
simulating asymmetric reveal delivery.
2026-04-09 16:26:30 +07:00
Nicholas Dudfield
2a9b1c9c22 fix(export): guard against empty verified sigs + add invariant asserts
- Skip addVerifiedSignature in decorateMessage when sigBuf is empty
  (sign() threw — don't mark a failed sign as "verified")
- Add XRPL_ASSERT in addVerifiedSignature and addUnverifiedSignature
  requiring non-empty signature buffers
- Add XRPL_ASSERT in checkQuorumAndSnapshot verifying that every
  entry in the verified set exists in the signatures map with a
  non-empty buffer
2026-04-09 16:02:35 +07:00
Nicholas Dudfield
54ca21b604 fix(export): verified-only quorum, SHAMap, and transactor upgrade pass
Enforce the contract: source chain finalizes an export only when it
has a quorum of cryptographically verified multisignatures.

ExportSigCollector changes:
- signatureCount() now counts verified entries only
- checkQuorumAndSnapshot() returns verified-only snapshot
- snapshot() and snapshotWithSigs() return verified-only data
- buildExportSigSet (via snapshot) publishes verified-only entries
- unverifiedSignatures() returns sigs needing verification
- upgradeSignature() promotes unverified to verified
- addStandaloneSignature() marks as verified (no consensus to check)
- All add methods now set firstSeenSeq (fixes stale cleanup bug)

Export::doApply changes:
- Upgrade pass before quorum check: deserializes the inner tx (which
  is always available as ctx_.tx), verifies any unverified sigs via
  buildMultiSigningData + verify(), upgrades them in the collector
- Then checks quorum on verified-only count
- Assembles blob from verified-only snapshot

This means:
- Unverified sigs (relay ordering) are local cache only
- They don't count toward quorum until upgraded
- SHAMap convergence operates on verified sigs only
- Destination chain verification remains defense-in-depth
2026-04-09 15:54:41 +07:00
Nicholas Dudfield
462db6004c fix(rng): replace nonexistent leafCount() with std::distance
SHAMap has no leafCount() method — it was a local variable in
SHAMap.cpp, not a public API.  Use std::distance(begin(), end())
on the SHAMap's ForwardRange iterators instead.  Cost is O(n) but
the set is bounded by UNL size (~20-35 entries).
2026-04-09 15:42:04 +07:00
Nicholas Dudfield
cfca708aae fix(rng): remove pendingReveals fallback from entropy output path
shouldZeroEntropy() and sfEntropyCount no longer fall back to
pendingReveals_.  If entropySetMap_ is null, entropy failed — the
pipeline didn't complete, and the map is the only canonical source.

pendingReveals_ is now strictly an internal staging area for the
commit/reveal pipeline.  All final entropy decisions flow through
entropySetMap_, which is the consensus-agreed set.
2026-04-09 15:40:22 +07:00
Nicholas Dudfield
5f70e5259c fix(rng): use entropySetMap for shouldZeroEntropy and sfEntropyCount
The H2 entropy fix switched the digest computation to entropySetMap_
but shouldZeroEntropy() and sfEntropyCount still used pendingReveals_.
Since pendingReveals_ can diverge from the published entropySetMap_
(late reveals mutate it after the map hash is published), two nodes
agreeing on the same entropySetHash could still build different
ttCONSENSUS_ENTROPY pseudo-transactions.

Now shouldZeroEntropy() checks entropySetMap_ leaf count when the map
is available, and sfEntropyCount uses the map's leaf count.  Both
fall back to pendingReveals_ only during pipeline stages before the
map is built.
2026-04-09 15:35:00 +07:00
Nicholas Dudfield
8697c5d821 refactor(export): explicit verified/unverified sig API in collector
Replace the ambiguous addSignature/hasSignature API with clearly
named methods that make verification state explicit:

  addVerifiedSignature()   — sig passed buildMultiSigningData + verify()
  addUnverifiedSignature() — trusted source but no multisign check yet
  addStandaloneSignature() — pubkey-only for standalone/test mode
  hasVerifiedSignature()   — only returns true for verified sigs

Unverified sigs (relay ordering fallback) are no longer treated as
verified by the cache.  When the same sig is encountered again via a
path that CAN verify (e.g. SHAMap merge after the tx arrives), the
verification runs and upgrades it to verified.

addUnverifiedSignature() won't overwrite a verified sig, preventing
downgrade.  SigEntry tracks verified validators in a separate set.
2026-04-09 15:34:13 +07:00
Nicholas Dudfield
9436e5868e fix(export): soften hard reject to best-effort verify for relay ordering
Revert the hard reject when ttEXPORT is not in the open ledger.
Under relay ordering, a node can receive a proposal with export sigs
before the ttEXPORT tx itself arrives.  Dropping these sigs loses a
valid validator contribution for the entire round with no recovery
path until terRETRY_EXPORT on the next round.

Post C1+C2, the proposal-level authentication is sufficient trust:
checkSign() verified the sender holds the private key, and sender
binding verified the embedded pubkey matches.  Store the sig and
let the multisign content be verified on the destination chain.
The collector's stale cleanup (256 ledgers) bounds retention.

When the tx IS in the open ledger (common case), the multisign sig
is still fully verified via buildMultiSigningData + verify().
2026-04-09 15:22:20 +07:00
Nicholas Dudfield
c6fa973cf6 fix(rng): compute entropy from entropySetMap instead of pendingReveals
H2: Compute final entropy from the agreed-upon entropySetMap_ SHAMap
rather than from the local pendingReveals_ in-memory map.

Previously, two nodes with different reveal subsets at timeout would
compute different entropy from their local pendingReveals_ maps,
despite both passing haveConsensus() (which only checks txSetHash).
This could cause a ledger fork.

Now the entropy computation reads directly from the entropySetMap_
whose hash was published in proposals and converged via SHAMap
fetch/merge.  Nodes that agree on entropySetHash deterministically
produce the same entropy regardless of local pendingReveals_ state.

If entropySetMap_ is null (bootstrap skip, pipeline failure), the
existing shouldZeroEntropy() fallback handles it.
2026-04-09 15:18:45 +07:00
Nicholas Dudfield
939e03714c fix(export): cap exportSignatures count per proposal
Reject proposals with more than ExportLimits::maxPendingExports (8)
export sig entries.  Honest validators attach at most one sig per
pending export, bounded by the same limit.  Prevents DoS via
proposals with millions of entries triggering lock contention on
the validator list and collector mutexes.
2026-04-09 15:13:15 +07:00
Nicholas Dudfield
969f98f57e perf(export): skip redundant sig verification via collector lookup
Add hasSignature() to ExportSigCollector — checks if a verified sig
already exists for a given (txHash, validator) pair.  Both the
proposal ingestion path and the SHAMap merge path now check this
before calling verify(), avoiding redundant ed25519 verification
when the same sig arrives via multiple paths.

No external sig cache exists in rippled, so the collector itself
serves as the verification cache: once a sig is stored (always
post-verify), subsequent encounters skip the crypto work.
2026-04-09 15:03:57 +07:00
Nicholas Dudfield
435deb0e78 fix(export): close remaining sig verification gaps
Three fixes from codex review:

1. Remove unsafe fallback in proposal ingestion path: reject export
   sigs when the ttEXPORT tx is not in the open ledger instead of
   storing them unverified.  The tx must be in the open ledger for
   validators to have signed it, so this is not a legitimate case.

2. Add full sig verification to the SHAMap merge path
   (onAcquiredSidecarSet): verify each export sig entry against
   buildMultiSigningData + verify() before storing in the collector.
   Previously this path only checked trusted() on the pubkey,
   allowing a malicious UNL validator to publish a sidecar set with
   forged sigs for other validators.

3. Close cluster mode bypass: always call checkSign() and gate export
   sig harvesting on sigValid, even when cluster() is true.  Cluster
   trust is for relay/resource charging, not for accepting on-chain
   cryptographic artifacts.
2026-04-09 14:59:20 +07:00
Nicholas Dudfield
b80352e512 fix(export): verify multisign signatures at ingestion time
C3: Cryptographically verify each export signature blob against the
inner transaction's signing data before storing in the collector.
Looks up the ttEXPORT tx from the open ledger, reconstructs the
signing data via buildMultiSigningData, and calls verify().

If the tx isn't in our open ledger yet (timing/relay), the sig is
stored unverified as a fallback — it can be verified later at the
SHAMap merge path or will be rejected at Export::doApply if invalid.

This runs on the jtPROPOSAL_t job queue thread (not the IO strand
or transactor), so the verify() cost has no impact on consensus
critical path performance.
2026-04-09 14:43:30 +07:00
Nicholas Dudfield
57c46c61fc fix(export): two-pass sender validation and atomic quorum+snapshot
C2 hardening: validate all export sig blobs before committing any,
preventing partial state if a later blob fails the sender binding
check. Also moves the trusted() check before the loop since senderPK
is constant.

H1: Add checkQuorumAndSnapshot() to ExportSigCollector that performs
the quorum threshold check and signature snapshot under a single lock
acquisition. Export::doApply now uses this instead of separate
signatureCount() + snapshotWithSigs() calls, eliminating the TOCTOU
window where concurrent overlay threads could mutate the collector
between the two operations.
2026-04-09 14:39:35 +07:00
Nicholas Dudfield
37ff13df50 fix(export): move sig harvesting after checkSign and bind pubkey to sender
C1: Move onTrustedPeerMessage() from the synchronous onMessage(TMProposeSet)
handler into checkPropose(), after checkSign() verifies the proposal's
cryptographic signature. Previously, export sigs were ingested before
signature verification, allowing any peer to inject forged sigs by
spoofing nodepubkey to a trusted validator's key.

C2: Add sender binding in onTrustedPeerMessage() — each export sig
blob's embedded validator pubkey must match the proposal sender's
nodepubkey. Reject the entire proposal's export sigs on any mismatch,
preventing a compromised validator from impersonating other validators
to single-handedly forge quorum.
2026-04-09 14:32:38 +07:00
Nicholas Dudfield
1b363b7eac fix: correct stale function name in ConsensusExtensionsTick comment 2026-04-09 14:03:07 +07:00
Nicholas Dudfield
9562b457cf chore: remove stale Peer-level RNG forwarders
All callsites now go through ce() consistently.
2026-03-23 09:57:53 +07:00
Nicholas Dudfield
724633ceb5 refactor(consensus): decouple CSF tests from xrpld.app via PeerTick.h
Move ConsensusExtensionsTick.h from xrpld/app/consensus/ to
xrpld/consensus/ — it's a pure template with no app-layer deps.
Extract Peer::Extensions::onTick() definition into test/csf/PeerTick.h
so Peer.h no longer includes from xrpld/app/.

Eliminates the test.csf > xrpld.app levelization edge.

Add --explain flag to levelization.py for tracing dependency edges.
2026-03-23 09:36:59 +07:00
Nicholas Dudfield
152d82e798 refactor(consensus): extract RNG/Export into ConsensusExtensions
Extract all Xahau consensus extension logic (RNG commit-reveal entropy
and Export validator multisign collection) from Consensus.h and
RCLCxAdaptor into a dedicated ConsensusExtensions class owned by
Application.

Implements all 10 lifecycle hooks from the design doc:
  onRoundStart, onTrustedPeerMessage, onTrustedPeerProposal,
  decoratePosition, decorateMessage, onTick, onPreBuild,
  onAcceptComplete, isSidecarSet, onAcquiredSidecarSet

Key design decisions:
- ConsensusTick<> template in ConsensusTypes.h keeps dependency
  direction clean (generic consensus defines contract, extensions
  implement it)
- extensionsTick<> shared template in ConsensusExtensionsTick.h
  ensures CSF test framework runs the same state machine as production
- ExportSigCollector ownership moved from global singleton to CE
- Sidecar acquisition routed through RCLConsensus mutex for thread
  safety (isExtensionSet + gotExtensionSet)
- RCLCxAdaptor reduced to thin ce() accessor + generic consensus
  interface methods

Files:
  new: ConsensusExtensions.h/.cpp, ConsensusExtensionsTick.h
  reduced: Consensus.h (-1060 lines), RCLConsensus.cpp (-1400 lines)
  updated: ConsensusTypes.h, Application.h/.cpp, NetworkOPs.cpp,
           PeerImp.cpp, Export.cpp, ExportSigCollector.h, Peer.h

7/7 testnet scenarios + 1463 consensus + 260 Export unit tests pass.
2026-03-19 20:23:19 +07:00
Nicholas Dudfield
0bb31ce7ce chore: add projected-source markers for consensus extension docs
Non-functional comment markers (//@@start, //@@end) for projected-source
documentation extraction.
2026-03-19 12:14:11 +07:00
Nicholas Dudfield
4cb3de0497 refactor(export): build xport wrapper via STObject then serialise to STTx
Replace the duplicated throwaway-STTx + real-STTx pattern with a
single STObject: set all fields including fee=0, serialise to compute
the fee, patch the fee, then serialise once into the final STTx.

20 lines shorter, no duplication.
2026-03-18 17:36:30 +07:00
Nicholas Dudfield
c6b315412d test(export): harden scenario tests with proper assertions
Replace warnings and logs with hard assertions across all export
scenario tests:

- export_helpers: add assert_hook_accepted(), assert_export_result(),
  assert_shadow_ticket() shared assertion helpers
- steady_state_export: assert hook ACCEPT + emitCount, ExportResult
  contents (signers, inner tx fields), shadow ticket exists
- retriable_export: assert ExportResult well-formed, shadow ticket
  created, payment not blocked
- export_degradation: assert export FAILS (not just log), no shadow
  ticket, payment still works
- export_unanimity: assert ExportResult + shadow ticket on success,
  absence on failure
2026-03-18 17:18:12 +07:00
Nicholas Dudfield
72395bec75 chore: clang-format 2026-03-18 17:12:41 +07:00
Nicholas Dudfield
8ed4d86f0f test(export): verify emitted ttEXPORT lifecycle end-to-end
testXportPayment now asserts the full emitted tx lifecycle:
- hook fires with ACCEPT, emitCount=1, returnCode=0
- sfHookEmissions present with 1 entry
- ltEMITTED_TXN in AffectedNodes
- emitted dir is not empty
- after close, emitted ttEXPORT appears in closed ledger

Also add FOCUSED_TEST env var gate for fast iteration during
development (set FOCUSED_TEST=1 to run only focused_test()).
2026-03-18 17:11:19 +07:00
Nicholas Dudfield
419fd16b9a chore: move export scenarios to export-suite.yml, add SuiteLogsWithOverrides
Move export scenario tests from suite.yml into their own
export-suite.yml file. The defaults already set CE+Export features
so individual test entries no longer need to repeat them.

Add SuiteLogsWithOverrides test utility: a Logs subclass that routes
specified journal partitions to stderr (always visible) while keeping
others on suite_.log (only on failure). Useful for debugging specific
subsystems during test development.
2026-03-18 17:09:47 +07:00
Nicholas Dudfield
a8097cd9a6 fix(export): compute emit fee before STTx construction
Mutating the fee via const_cast after STTx construction left a stale
cached getTransactionID(). When the emitted ttEXPORT was serialised
into the emitted directory and later deserialised, the round-tripped
txid differed from the original, causing tefNONDIR_EMIT in
Transactor::preclaim (the emitted dir entry was keyed with the stale
hash).

Build a throwaway STTx with fee=0 to calculate the fee size, then
construct the real STTx with the correct fee from the start.
2026-03-18 17:04:46 +07:00
Nicholas Dudfield
02a0552325 docs(export): clarify LLS semantics for retriable exports
Add comment explaining the three-state outcome table for exports
relative to LastLedgerSequence:
  ledger < LLS:  tesSUCCESS or terRETRY_EXPORT
  ledger == LLS: tesSUCCESS or tecEXPORT_EXPIRED
  ledger > LLS:  tefMAX_LEDGER (never reaches doApply)
2026-03-18 14:43:50 +07:00
Nicholas Dudfield
3698193b0a chore: clang-format 2026-03-18 14:21:00 +07:00
Nicholas Dudfield
de43ca2385 refactor(export): store multisigned tx as sfExportedTxn object in metadata
Use sfExportedTxn (OBJECT) instead of sfBlob (VL) for the multisigned
transaction in sfExportResult metadata. This renders as readable JSON
with all fields visible (Account, Signers, etc.) instead of an opaque
hex blob.

Also compute the tx hash directly via getHash(HashPrefix::transactionID)
on the STObject instead of serializing/deserializing through STTx.
2026-03-18 14:16:25 +07:00
Nicholas Dudfield
8c747a1916 feat(export): produce multisigned blob in export metadata
Export::doApply now builds the fully multisigned inner tx and stores
it as sfBlob in sfExportResult metadata. In standalone mode, the node
signs directly with its own validator keys (no consensus needed).

Key changes:
- ExportSigCollector stores actual multisign signatures, not just pubkeys
- RCLConsensus proposal attachment computes real multisign sigs over inner tx
- PeerImp harvests variable-length sig entries from proposals
- Export::doApply assembles Signers array, builds multisigned blob first,
  then uses its hash for the shadow ticket (getTransactionID includes all
  fields including Signers)
- Import skips OperationLimit and signing key checks for export callback
  path (sfTicketSequence present) — shadow ticket proves the relationship
- Full Export→XRPL→Import round-trip test: export on Xahau, submit
  multisigned blob to XRPL (with matching SignerList), build XPOP,
  import back, verify shadow ticket consumed
2026-03-18 13:54:29 +07:00
Nicholas Dudfield
cea110f29a feat: add XPOP test helper and XPOP_test suite
- src/test/jtx/xpop.h: test utilities for building XPOPs from Env ledgers
  (TestValidator, TestVLPublisher, TestXPOPContext, buildTestXPOP)
- src/test/app/XPOP_test.cpp: 4 tests (173 assertions)
  - LedgerProof construction from payment tx
  - XPOP v1 JSON structure verification
  - Merkle proof verification for multi-tx ledgers
  - Full Import round-trip: source Env payment → XPOP → dest Env Import → tesSUCCESS
2026-03-18 11:59:34 +07:00
Nicholas Dudfield
3ca056a94b feat: add XPOP test helper and XPOP_test suite
- src/test/jtx/xpop.h: test utilities for building XPOPs from Env ledgers
  (TestValidator, TestVLPublisher, buildTestXPOP)
- src/test/app/XPOP_test.cpp: 3 tests (133 assertions)
  - LedgerProof construction from payment tx
  - XPOP v1 JSON structure verification
  - Merkle proof verification for multi-tx ledgers
2026-03-18 11:41:16 +07:00
Nicholas Dudfield
705d8400db feat: add proof module for XPOP construction
New module at src/xrpld/app/proof/ with layered design:

- ProofBuilder: SHAMap merkle proof extraction (extractProofV1)
  Binary trie proof with 16-way branching, root hash verification.
- LedgerProof: proof-of-ledger (header fields + tx blob/meta + merkle proof)
  buildLedgerProof() extracts everything from a closed Ledger.
- XPOPv1: JSON format builder matching Import.cpp expectations
  buildXPOPv1() creates complete XPOP with validation signatures.

Designed for versioning: v1 JSON (current Import compat), future v2
binary proofs and account state proofs layer on the same core.
2026-03-18 11:29:29 +07:00
Nicholas Dudfield
655b751698 chore: regenerate hook/sfcodes.h for new sfields
Adds sfCancelTicketSequence (UINT32 101) and sfExportResult (OBJECT 98).
2026-03-18 11:06:04 +07:00
Nicholas Dudfield
f324081277 fix(import): verify XPOP tx hash matches shadow ticket
Shadow tickets store the exported transaction hash. Import now verifies
the XPOP's inner tx hash matches, preventing use of a different XPOP
with the same TicketSequence.
2026-03-18 10:52:19 +07:00
Nicholas Dudfield
24a284180a fix(export): address review findings from code audit
- Validate incoming export sig pubkeys against trusted UNL (PeerImp)
- Fix handleAcquiredRngSet misclassifying export sets when local map is null
- Add stale-entry cleanup (cleanupStale with 256-ledger timeout)
- Gate sig attachment on featureExport amendment (RCLConsensus)
- Fail with tefINTERNAL if ExportResult metadata can't be written
- Make export and cancel mutually exclusive (reject both in preflight)
- Remove dead ExportLimits.h include
2026-03-18 10:30:41 +07:00
Nicholas Dudfield
6f003cc983 feat(export): add SHAMap-based export sig convergence
Add deterministic export sig set convergence using CE infrastructure:
- exportSigSetHash in ExtendedPosition (flag 0x10)
- buildExportSigSet() builds SHAMap from collected sigs
- hasPendingExportSigs() gates convergence in phaseEstablish
- Parallel convergence gate alongside RNG sub-states
- handleAcquiredRngSet extended to merge export sig sets
- Tiered quorum: 80% with CE, 100% unanimity without CE

Scenario tests for all 3 feature combos:
- CE+Export, 1 node down: 4/5=80% → tesSUCCESS
- Export only, all up: 5/5=100% → tesSUCCESS
- Export only, 1 node down: 4/5≠100% → tecEXPORT_EXPIRED
2026-03-18 10:17:28 +07:00
Nicholas Dudfield
3a58020388 fix(export): tiered quorum threshold based on CE availability
Without CE: require unanimity (100% UNL) to avoid non-deterministic
quorum disagreement. With CE: use standard 80% quorum via
calculateQuorumThreshold (SHAMap convergence will ensure agreement).
Standalone/unit tests: require 1 sig.
2026-03-18 09:46:36 +07:00
Nicholas Dudfield
829441b52e fix(export): deduplicate export sigs across proposals within a round 2026-03-18 09:32:27 +07:00
Nicholas Dudfield
3a055663cc chore: add export-sig-attachment marker for projected-source 2026-03-18 09:26:33 +07:00
Nicholas Dudfield
985a194bdc feat(export): migrate to retriable ttEXPORT with proposal-based sigs
Replace the old ltEXPORTED_TXN + ttEXPORT_FINALIZE (validation-based
sigs, TxQ injection) approach with a retriable ttEXPORT that collects
validator signatures via TMProposeSet during consensus.

Added:
- terRETRY_EXPORT: keeps tx in retry set across ledger boundaries
- tecEXPORT_EXPIRED (200): LLS expiry frees sequence cleanly
- sfExportResult (OBJECT 98): signed export result in tx metadata
- ExportSigCollector: minimal thread-safe sig tracker
- Proposal sig attachment (RCLConsensus) + harvesting (PeerImp)
- exportSignatures field in TMProposeSet (ripple.proto)
- Metadata plumbing (TxMeta, ApplyViewImpl, ApplyStateTable)
- Hook xport() now emits ttEXPORT via normal emitted txn path

Removed:
- ttEXPORT_FINALIZE (type 90) pseudo-tx and Change::applyExportFinalize
- ltEXPORTED_TXN ledger entry and exportedDir/exportedTxn keylets
- ExportSignatureCollector (replaced by ExportSigCollector)
- TxQ export injection (quorum check + rawTxInsert)
- Validation-based export signing in RCLConsensus
- Application::getExportSignatureCollector

Verified on 5-node testnet: golden path (same-ledger finalization with
ExportResult in metadata), degraded path (tecEXPORT_EXPIRED on sub-quorum),
and hook xport() path (emitted ttEXPORT with shadow ticket creation).
2026-03-18 09:23:52 +07:00
Nicholas Dudfield
869f366d8a feat(export): add sfCancelTicketSequence for shadow ticket cancellation
Add sfCancelTicketSequence (UINT32 field 101) to ttEXPORT, allowing
users to cancel shadow tickets via a transaction. Both sfExportedTxn
and sfCancelTicketSequence are optional — at least one must be present.
This allows export, cancel, or both in a single transaction.

Test: create shadow ticket via export, cancel via sfCancelTicketSequence,
verify ticket is gone and owner reserve is freed.
2026-03-17 14:13:57 +07:00
Nicholas Dudfield
03936aa928 fix(export): require TicketSequence on exported transactions
Exported transactions must use TicketSequence (with Sequence=0)
because a bounced tx on the destination chain would jam sequential
sequence numbers. This is enforced in both the hook xport() API
and the Export transactor via ExportLedgerOps::validateTicketSequence().

Adds test: ttEXPORT rejects export without TicketSequence.
Updates existing test hooks to include TicketSequence in exported txns.
2026-03-17 12:30:55 +07:00
Nicholas Dudfield
6d180307ad feat(export): split Import into B2M and export callback paths
When the inner XPOP transaction has sfTicketSequence, Import now
takes the export callback path: consume the shadow ticket via
ExportLedgerOps::cancelShadowTicket() and return. No B2M balance
crediting, no account creation. Hooks fire normally and can inspect
the result via xpop_slot().

The B2M path is unchanged for non-ticket imports.

Also migrates the shadow ticket check in preclaim from the old
hookState namespace approach to keylet::shadowTicket().

Removes the unused shadowTicketNamespace constant.
2026-03-17 12:23:27 +07:00
Nicholas Dudfield
f2ca499c97 feat(export): add ltSHADOW_TICKET and xport_cancel hook API
Introduce shadow tickets for export replay protection:

- ltSHADOW_TICKET ledger entry: account-owned, keyed by
  account + ticket sequence. Fields: sfAccount, sfTicketSequence,
  sfTransactionHash, sfLedgerSequence, sfOwnerNode.

- ExportLedgerOps::createShadowTicket(): creates shadow ticket
  when exported tx has sfTicketSequence. Charges owner reserve.
  Called from both hook xport() path and Export transactor.

- ExportLedgerOps::cancelShadowTicket(): deletes shadow ticket,
  frees reserve. Used by xport_cancel hook API.

- xport_cancel(ticket_seq) hook API: allows hooks to cancel
  shadow tickets for exports that will never get a callback.

- InvariantCheck: add ltSHADOW_TICKET to valid entry types.

- Test: verify shadow ticket creation with correct fields and
  owner count bump via ttEXPORT with TicketSequence.
2026-03-17 12:13:41 +07:00
Nicholas Dudfield
bd68364f25 feat(export): add ttEXPORT user transaction and extract ExportLedgerOps
Rename the existing ttEXPORT pseudo-tx to ttEXPORT_FINALIZE (type 90)
to make room for a user-submittable ttEXPORT (type 91).

ttEXPORT allows non-hook users to submit export transactions directly,
creating the same ltEXPORTED_TXN entries that the hook xport() API
creates inline.

Extract shared logic into ExportLedgerOps.h:
- createExportedTxn(): creates ltEXPORTED_TXN, enforces directory cap
- validateNetworkID(): self-target and unconfigured guards
- validateExportAccount(): account ownership check

Both the hook API (HookAPI.cpp) and the Export transactor now call
into ExportLedgerOps, eliminating duplicated validation and ledger
mutation code.
2026-03-17 11:43:45 +07:00
Nicholas Dudfield
42a6407815 fix(export): reject exports when NETWORK_ID is unconfigured
If the node's NETWORK_ID is 0 (default/unconfigured) and the exported
transaction has no sfNetworkID field, we can't distinguish self-targeting
from legitimate cross-chain export. Reject to be safe.

Also adds exportTestConfig() helper and test for the unconfigured case.
2026-03-17 07:41:36 +07:00
Nicholas Dudfield
a387c853ab test(export): add NetworkID self-target guard test
Verify that xport() rejects exported transactions whose sfNetworkID
matches the local network. The hook builds a Payment with
NetworkID=21337 (matching the test env), and the guard correctly
returns EXPORT_FAILURE causing tecHOOK_REJECTED.

Also fix log level for the guard rejection to warn (not trace).
2026-03-17 07:31:35 +07:00
Nicholas Dudfield
9311e567d3 fix(export): reject exports targeting the local network
Explicitly forbid exported transactions whose sfNetworkID matches the
local network's ID. An exported txn re-executing on its origin chain
could cause exploits or logic issues.

The check is intentionally non-mandatory: XRPL mainnet (the primary
export target) doesn't use NetworkID, so absent = allowed.
2026-03-16 17:30:14 +07:00
Nicholas Dudfield
c26582bdf9 fix(export): move ExportLimits.h to xrpl/protocol
Both xrpld.overlay and xrpl.hook depend on xrpl.protocol, so placing
the header there avoids introducing a new xrpld.overlay > xrpl.hook
levelization dependency.
2026-03-16 15:58:58 +07:00
Nicholas Dudfield
417b999c7f chore(levelization): add xrpld.overlay > xrpl.hook dependency
New include of ExportLimits.h in PeerImp.cpp introduces this
module dependency (from feat(export) commit 89274b538).
2026-03-16 15:47:45 +07:00
Nicholas Dudfield
0205be4500 chore: add testnet scenario scripts
Entropy and export scenario scripts for local testnet validation.
2026-03-16 15:17:32 +07:00
Nicholas Dudfield
89274b5387 feat(export): wip export system limits
- max_export per hook: 4 → 2
- maxPendingExports: cap exported directory at 8 entries (tecDIR_FULL)
- clamp inbound signature processing in PeerImp to directory cap

The directory cap is the root DoS constraint: each pending export
requires every validator to sign and broadcast every round. Inbound
processing and signing throughput are transitively bounded by it.
2026-03-16 13:59:06 +07:00
Nicholas Dudfield
b65d9faf12 docs(consensus): add MERGE NOTE comments for upstream 86ef16dbeb resolution
Extends merge guidance to cover the empty-disputes bugfix (not yet in
sync-2.5.0): !disputes.empty() guard, stalled() j/clog params,
"should be rare" doc wording, debug→warn promotion, and auto-merged
testDisputes duplicate warning.
2026-03-11 10:45:04 +07:00
Nicholas Dudfield
aa1a7e5320 docs(consensus): add MERGE NOTE comments for sync-2.5.0 resolution
Inline comments at all 6 conflict points guiding the maintainer
through the expected merge conflicts when sync-2.5.0 lands:
ledgerMAX_CONSENSUS const, bootstrap params, calculateQuorumThreshold,
effectiveParms+stalled in haveConsensus, DisputedTx::stalled() j/clog
params, and testDisputes placement.
2026-03-11 10:09:02 +07:00
Nicholas Dudfield
6f0f17aad9 fix(consensus): cherry-pick upstream 86ef16dbeb empty-disputes stall fix
Cherry-pick of ripple/rippled@86ef16dbeb ("Fix: Don't flag consensus
as stalled prematurely (#5627)"). Not yet in any xahau sync branch.

Fixes false stall detection when there are no disputed transactions:
std::ranges::all_of on an empty set is vacuously true, so consensus
was incorrectly flagged as stalled. Adds !result_->disputes.empty()
guard.

Also adds diagnostic logging to DisputedTx::stalled() and the
stall detection path in haveConsensus(), and promotes the
"Need validated ledger" log from debug to warn.
2026-03-11 09:47:11 +07:00
Nicholas Dudfield
407bfa1467 feat(consensus): cherry-pick dd085e5d8 (upstream d22a5057b9) anti-stall mechanisms
Cherry-pick of ripple/rippled@d22a5057b9 / xahau dd085e5d8 ("Prevent
consensus from getting stuck in the establish phase (#5277)"), resolved
against our RNG pipeline and bootstrap fast-start changes.

Upstream adds three layered anti-stall mechanisms:
- Stateful per-dispute avalanche state machine (init→mid→late→stuck)
- Stall detection: declares consensus when all disputes individually settled
- Hard expiration: clamp(10× prev round, 15s, 120s) wall-clock safety net

Conflict resolution:
- ConsensusParms.h: kept both avalanche state machine (const members,
  avMIN_ROUNDS, avSTALLED_ROUNDS, getNeededWeight) and our bootstrap
  params (bootstrapRoundTimeSeed, bootstrapStableRoundsRequired).
  ledgerMAX_CONSENSUS left non-const for bootstrap override.
- Consensus.h: pass both stalled flag and effectiveParms to checkConsensus.
  Stall check uses original parms, bootstrap override only affects max
  consensus timeout.
- Consensus_test.cpp: kept all 12 RNG tests and new testDisputes test.
2026-03-11 09:36:38 +07:00
Nicholas Dudfield
f0dfcf6b81 fix(consensus): cap bootstrap ledgerMAX_CONSENSUS at 5s
Use an explicit 5s cap instead of dividing the default 15s.
5s is the sweet spot: long enough for peers to exchange proposals
and converge naturally, short enough to avoid wasted time.
Shorter values (e.g. 3.75s) cause nodes to hit reachedMax before
peers converge, cascading into slower subsequent rounds.
2026-03-10 14:30:20 +07:00
Nicholas Dudfield
503d2ebf98 feat(consensus): add XAHAUD_BOOTSTRAP_FAST_START for faster cold-start
Seed prevRoundTime_ to 3s instead of 15s on first round, override
idle interval to bypass closeTimeResolution (10-30s on early ledgers),
and halve ledgerMAX_CONSENSUS during bootstrap. Auto-disables after 3
consecutive rounds with UNL quorum participation.

Cuts 5-node testnet cold-start from ~28s to ~13s.

Also adds projected-source markers to TxQ, NetworkOPs, and Submit for
the transaction-submission documentation template.
2026-03-10 12:52:56 +07:00
Nicholas Dudfield
e52bc51384 refactor(consensus): extract shouldZeroEntropy() for quorum-gated entropy
Consolidate the repeated entropy fallback condition
(entropyFailed || no reveals || sub-quorum reveals) into a single
method. Fixes EntropyCount field reporting non-zero when the digest
was correctly zeroed due to sub-quorum reveals.
2026-03-10 08:42:10 +07:00
Nicholas Dudfield
91860db578 fix(consensus): require quorum-many reveals for non-zero entropy
Sub-quorum reveals (e.g. 3/4 threshold) were producing real entropy,
allowing a minority of validators to disproportionately influence the
output. Both injectEntropyPseudoTx and buildExplicitFinalProposalTxSet
now fall back to zero entropy when reveals < quorumThreshold().
2026-03-09 17:13:02 +07:00
Nicholas Dudfield
0b317a8e7a fix(consensus): skip rng pipeline during bootstrap convergence
When prevProposers < quorumThreshold, the network is still converging
and RNG can only produce zero entropy. Skip the commit/reveal pipeline
to avoid PIPELINE_TIMEOUT and conflict-wait delays that compound across
staggered startup rounds.
2026-03-09 16:27:36 +07:00
Nicholas Dudfield
dbd230b695 feat(rpc): add rng state to consensus_info response 2026-03-09 16:05:42 +07:00
Nicholas Dudfield
30cefcba85 chore: clang-format alignment fixes 2026-03-06 18:39:37 +07:00
Nicholas Dudfield
94edb5759d fix(export): gate pre-quorum on verified signature count
hasQuorum() and getExportsWithQuorum() were using raw signerMap.size()
which includes unverified signatures. TxQ could inject a ttEXPORT
pseudo-tx that then fails the stricter verified-signature check in
Change::applyExport(). Use verifiedSignatureCount() instead so TxQ
only injects when cryptographically verified quorum is actually met.

Also add cmake plumbing for enhanced logging: link date::date-tz when
available and enable BEAST_ENHANCED_LOGGING for Debug builds.
2026-03-06 18:38:54 +07:00
Nicholas Dudfield
ce57b6a3a0 fix(consensus): fix rng quorum to active UNL and demote rng log noise
Quorum fix:
- Rename expectedProposers_ → likelyParticipants_ to clarify role
- Fix commit quorum to 80% of active UNL snapshot (not shrinkable by
  recent proposer count, which was allowing 2/3 to pass as quorum)
- hasQuorumOfCommits() now uses simple threshold check only
- Add CSF test: persistent loss does not shrink quorum

Log level cleanup:
- Demote ~30 RNG/STALLDIAG per-peer/per-tick lines from info/debug to
  debug/trace across Consensus.h and RCLConsensus.cpp
- Principle: per-peer/per-tick → trace; state transitions → debug;
  milestones → info
- Reduces testnet log volume by ~93%
2026-03-06 18:36:43 +07:00
Nicholas Dudfield
fca5cad470 fix(log): catch tzdb exception in local-time formatter
date::current_zone() can throw if the timezone database is unavailable
or misconfigured (e.g. minimal container images). Fall back to UTC
formatting so enhanced logging does not make startup fatal.
2026-03-06 18:36:22 +07:00
Nicholas Dudfield
bb77c2090b consensus: gate RNG substates by amendment state 2026-03-06 14:09:06 +07:00
Nicholas Dudfield
90a94294e4 protocol: split export and consensus entropy amendments 2026-03-06 14:08:15 +07:00
Nicholas Dudfield
c2209b4472 docs(consensus): explain why seq=3 may mirror seq=2
Clarify inline that seq=3 publish can carry unchanged txSetHash while still providing extra entropySetHash delivery/fetch opportunity under packet loss or reordering.
2026-03-03 17:41:55 +07:00
Nicholas Dudfield
8fcb2ed336 docs(consensus): annotate implicit entropy injection rationale
Document why synthetic entropy pseudo-tx is canonically injected at onAccept/buildLCL and why explicit-final proposal remains experimental/default-off.
2026-03-03 17:31:04 +07:00
Nicholas Dudfield
fd1567d1ba consensus: document explicit-final tradeoffs and tighten rng diagnostics
Keep explicit final proposal as an opt-in experimental path with implicit mode as default.

Add inline rationale/TBD notes, extend stall diagnostics, and cover runtime-config + CSF txn-path behavior with tests.
2026-03-03 17:08:38 +07:00
Nicholas Dudfield
d32f34d3bf build(levelization): add fast python generator with CI parity check
Add Builds/levelization/levelization.py for fast local iteration and semantic comparison against canonical shell output via --compare-to.

Keep Builds/levelization/levelization.sh as canonical path, and update levelization workflow to fail if python output diverges from shell-generated results.

Also harden interactive-shell detection in levelization.sh for portability and document local usage in README.
2026-03-03 10:17:46 +07:00
Nicholas Dudfield
c491c5c82f refactor(consensus): reduce header fanout for faster iteration
Decouple RCLConsensus.h from Consensus.h by forward-declaring Consensus and storing Consensus<Adaptor> behind std::unique_ptr, moving thin wrappers out-of-line into RCLConsensus.cpp.

Also remove direct RCLConsensus.h include from NetworkOPs.h (forward declare), and add explicit includes in DatagramMonitor.h and ServerDefinitions.cpp to replace transitive dependencies.

Keep RNG fast-path behavior unchanged in Consensus.h; build and ripple.consensus.Consensus remain green.
2026-03-03 09:49:59 +07:00
Nicholas Dudfield
74817765ae consensus: restore full entropySet broadcast and document fanout tradeoffs 2026-03-03 08:32:09 +07:00
Nicholas Dudfield
fc23fa8535 consensus: reduce entropy-set proposal fanout
Keep entropy-set recovery path but elect a deterministic single broadcaster (lowest NodeID among tx-converged participants) instead of every proposer broadcasting entropySetHash.

This lowers steady-state proposal chatter while preserving liveness for lagging peers that need entropy-set fetch/merge.
2026-03-03 07:42:27 +07:00
Nicholas Dudfield
34c0f17b6b runtimeconfig: add rng_claim_drop_pct testing control
Expose rng_claim_drop_pct in runtime config (RPC + env) as a clamped 0-100 percentage used by RNG claim-drop testing.

Include RuntimeConfig RPC tests for round-trip and clamping behavior.
2026-03-03 07:20:32 +07:00
Nicholas Dudfield
765ad6a278 consensus: harden RNG set convergence under dropped claims
Track active RNG round sequence for fetched set validation so lagging observers can merge current-round commit sets instead of rejecting them as closed+1 out-of-round.

Refresh/re-publish commitSetHash after fetch-merge conflicts and publish entropySetHash in ConvergingReveal so peers can recover reveal sets.

Add inline tradeoff notes: extra proposal traffic is accepted to preserve consensus safety/liveness under packet loss or drop injection.
2026-03-03 07:14:46 +07:00
Nicholas Dudfield
f623ca89b9 chore(levelization): update loops result after format/merge 2026-03-02 17:01:47 +07:00
Nicholas Dudfield
e4865f09f9 Merge remote-tracking branch 'origin/dev' into feature-export-rng 2026-03-02 16:59:57 +07:00
Nicholas Dudfield
4c182e4738 consensus: guard commit-set conflicts and extend RNG CSF coverage 2026-03-02 16:59:41 +07:00
Nicholas Dudfield
d0c869c8a6 fix(consensus): tighten RNG acquired-set validation and observer quorum
Harden acquired RNG merge paths with strict entry typing, trusted key/node binding, round-sequence gating, reveal-to-commit linkage checks, and stale reveal/proof invalidation on commitment changes.

Adjust proposer expectation logic so non-proposing observers are not counted as expected committers, and add a CSF regression test covering observer self-commit exclusion.
2026-03-02 16:36:03 +07:00
Nicholas Dudfield
cac5efcd3c fix(consensus): harden acquired RNG set ingestion
Reject mixed commit/reveal maps, enforce per-entry type checks, bind node identity to trusted validator keys, and gate acquired entries to the active round.

Also verify acquired reveals against stored commitments and clear stale reveal/proof state when commitments change.
2026-03-02 16:18:55 +07:00
Nicholas Dudfield
514e60b71c fix(export): age and validate stashed tx data for signature checks 2026-03-02 15:54:53 +07:00
Nicholas Dudfield
2a34e32e05 fix(export): harden addSignature validation and verification 2026-03-02 15:46:07 +07:00
Nicholas Dudfield
b969024a25 fix(export): update duplicates and prevent phantom pending entries 2026-03-02 15:39:43 +07:00
Nicholas Dudfield
f30b9a4c3a fix(export): avoid stale-age poisoning from rejected signatures 2026-03-02 15:35:36 +07:00
Nicholas Dudfield
0e019fec4e fix(export): prune invalid early signatures when stashing tx data 2026-03-02 15:29:42 +07:00
Nicholas Dudfield
7e0c72fd22 fix(export): run stale signature cleanup during TxQ processing 2026-03-02 15:27:30 +07:00
Nicholas Dudfield
07d741cdd7 fix(export): harden collector duplicate and identity handling 2026-03-02 15:25:19 +07:00
Nicholas Dudfield
b99c38c09d test(consensus): add asymmetric delay reveal-timeout scenario 2026-03-02 15:11:01 +07:00
Nicholas Dudfield
64e50209ff fix(consensus): invalidate stale reveals when commitment changes
Add RNG regression tests for non-UNL data, reveal-without-commit, invalid reveal, and commitment-change stale-reveal handling in CSF consensus tests.
2026-03-02 15:04:35 +07:00
Nicholas Dudfield
b1ce2103ad test(csf): add RNG consensus hooks and edge-case tests 2026-03-02 14:28:34 +07:00
Nicholas Dudfield
50c4cf1df3 refactor: move xport_reserve and xport logic into HookAPI class
Move core xport_reserve and xport implementations from applyHook.cpp
DEFINE_HOOK_FUNCTION wrappers into the decoupled HookAPI class, following
the same pattern used for etxn_reserve and emit.
2026-03-02 14:10:03 +07:00
Nicholas Dudfield
6fc14f398d feat(rpc): add disconnect by ip:port [TESTNET] 2026-03-02 12:06:00 +07:00
Nicholas Dudfield
592a8600c7 fix: add missing <mutex> include for GCC compatibility 2026-02-27 16:42:10 +07:00
Nicholas Dudfield
e71768700a chore: update levelization after RuntimeConfig overlay dependency 2026-02-27 16:40:00 +07:00
Nicholas Dudfield
e598e405bd fix: harden RuntimeConfig validation and add startup diagnostics
- Error on unknown message_types instead of silently widening scope
- Make messageCategories optional so per-peer can override global filter
  to "all categories" (nullopt=inherit, empty set=explicitly all)
- Clamp send_drop_pct to 0-100% range
- Add STARTDIAG: logging for consensus startup diagnostics
- Add 3 test cases (11 total, 58 assertions)
2026-02-27 13:38:26 +07:00
Nicholas Dudfield
8af3ce2f5b fix: allow extended proposals in PeerImp and add message type filtering
- Fix convergence regression caused by 2.4.0 merge: replace
  stringIsUint256Sized(currenttxhash) with size() < uint256::size()
  to accept extended proposals (>32 bytes) containing RNG fields
- Add message_types filter to RuntimeConfig for targeting specific
  protocol message categories (proposal, validation, transaction, etc.)
- Add appliesTo() method and messageCategories set to ConfigVals
- Add category name mapping helpers in RPC handler
- Add 2 test cases for message type filtering (8 total)
2026-02-27 13:10:49 +07:00
Nicholas Dudfield
b67cb78b97 feat: add RuntimeConfig service with overlay artificial delays
Add a generic RuntimeConfig service for runtime-configurable parameters,
initially supporting artificial send delays and packet drops for testing
consensus behavior on local testnets.

- RuntimeConfig class with atomic fast-path gate (zero cost when inactive)
- Per-peer targeting via "*" (global) and "ip:port" keys with inheritance
- Pre-merged caching at write time for single-lookup read path
- Admin RPC handler `runtime_config` (set/clear/clear_all/get)
- Env var support: XAHAU_RUNTIME_CONFIG (JSON) or XAHAU_SEND_* vars
- PeerImp::send() integration with delay timer and probabilistic drops
- RPC handler test covering all operations and merge behavior
2026-02-27 09:46:19 +07:00
Nicholas Dudfield
0b1b82282e fix: reject single-signed exports and fix test hook SigningPubKey
Add single-sign rejection check in Change::applyExport() matching
rippled's multi-sign validation: SigningPubKey must be present but
empty, TxnSignature must not be present.

Fix Export_test.cpp hook to encode an empty VL blob for SigningPubKey
instead of 33 zero bytes (AI slop from export-uvtxn branch).
2026-02-25 14:55:55 +07:00
Nicholas Dudfield
d4c5a7e8ab fix: update copyright headers to 2026 XRPL Labs for new files 2026-02-25 14:38:40 +07:00
Nicholas Dudfield
82837864fa fix: extract calculateQuorumThreshold() and revert Import.cpp quorum change
Extract duplicated (n * 80 + 99) / 100 ceiling quorum formula into shared
calculateQuorumThreshold() in ConsensusParms.h, matching the standard
ValidatorList::calculateQuorum(). Used by ExportSignatureCollector,
Change.cpp, and RCLConsensus.cpp.

Revert Import.cpp quorum from ceiling back to original truncating formula
(totalValidatorCount * 0.8) since Import handles XPOP imports, not the
new Export feature. Added TODO for future upgrade.
2026-02-25 14:22:43 +07:00
Nicholas Dudfield
e1caee6459 fix: regenerate hook/sfcodes.h after sfHookExportCount field code change 2026-02-25 13:40:25 +07:00
Nicholas Dudfield
3206b4a4e1 fix: address @tequdev review comments (cbak, render, Change.cpp, markers)
- Remove unnecessary cbak() stubs from ConsensusEntropy test hooks and
  recompile WASM (cbak is optional per Guard.h validator)
- Restore RCLCxPeerPos::render() lost during merge (delegates to
  ConsensusProposal::render())
- Fix Change.cpp applyAmendment() fixInnerObjTemplate2 reversion:
  use STObject::makeInnerObject() and bracket assignment (fbcff932)
- Restore txq-export-quorum-check documentation marker in TxQ.cpp
2026-02-25 13:25:41 +07:00
Nicholas Dudfield
0c2e09050e fix: move sfHookExportCount to Xahau-reserved field code range
sfHookExportCount was at field code 23, colliding with the mainline
rippled UINT16 range. Move to 98 in the Xahau-reserved range.

Also reorder sfExportedTxn (90) before sfAmountEntry (91) for
consistency.
2026-02-25 12:05:28 +07:00
Nicholas Dudfield
83922d5c20 fix: restore XRPL_ASSERT and UNREACHABLE macros reverted during merge
The merge with origin/dev accidentally reverted 19 XRPL_ASSERT() calls
back to plain assert() and 1 UNREACHABLE() back to assert(0). These
macros provide descriptive diagnostic messages on failure and are the
project convention since the rippled 2.4.0 migration.

Files fixed:
- Consensus.h: 9 XRPL_ASSERT reversions
- RCLConsensus.cpp: 5 XRPL_ASSERT reversions
- BuildLedger.cpp: 3 XRPL_ASSERT reversions
- Change.cpp: 1 UNREACHABLE + 1 XRPL_ASSERT reversion
2026-02-25 11:55:07 +07:00
Nicholas Dudfield
6bae42ff01 fix: restore CLOG consensus logging removed during merge
The merge with origin/dev accidentally stripped all CLOG diagnostic
statements from the consensus code path. This restores the clog
parameter to internal Consensus.h functions (checkLedger, phaseOpen,
closeLedger, updateOurPositions, handleWrongLedger, leaveConsensus,
createDisputes) and re-adds all 46 CLOG statements that provide
per-round diagnostic detail for phase transitions, convergence
progress, dispute tracking, and pause decisions.

Also restores the origin/dev structure of Consensus.cpp by removing
the anonymous-namespace wrapper and forwarding overloads that were
merge artifacts.
2026-02-25 11:53:27 +07:00
Nicholas Dudfield
35e86d926e fix(consensus-entropy): align pseudo tx/sle formats and hook handling
Add missing ttEXPORT/ttCONSENSUS_ENTROPY pseudo transaction fields required by runtime logic and ensure corresponding ledger entries carry threading/sequence fields.

Handle ttEXPORT and ttCONSENSUS_ENTROPY in hook stakeholder routing to avoid Unknown transaction type assertion during ledger close.
2026-02-24 18:44:00 +07:00
Nicholas Dudfield
9c4ee9315d chore: update levelization results after merge 2026-02-24 15:56:36 +07:00
Nicholas Dudfield
0f17cf02aa chore: clang-format 2026-02-24 15:51:55 +07:00
Nicholas Dudfield
7753dc3cbe fix(invariants): exempt export and entropy pseudo-ledger entries
Handle ltEXPORTED_TXN and ltCONSENSUS_ENTROPY in LedgerEntryTypesMatch so creating/destroying these pseudo-ledger entries does not trigger XRP balance invariant violations.
2026-02-24 15:48:32 +07:00
Nicholas Dudfield
cc7f3c59ae merge: port export-rng onto post-2.4.0 tree restructure
Resolve the origin/dev post-2.4.0 sync conflicts across the xrpld path migration and macro-based protocol registration changes.

Re-apply export/RNG integration on top of the new structure, including consensus/build plumbing, tx/apply paths, peer ingest, and tests.

Regenerate hook headers and restore a green build via x-run-tests (Export_test build path).
2026-02-24 15:32:45 +07:00
Nicholas Dudfield
e8c1b25ab4 fix: harden export signature trust model and quorum verification
- unify validator trust checks into isExportValidatorTrusted() preferring
  UNLReport with local trust fallback
- add last-line-of-defense sig verification in Change::applyExport()
  requiring 80% (ceil) verified trusted UNL signatures
- filter untrusted export signatures at ingestion in PeerImp
- fix Import quorum from floor(n*0.8) to ceil(n*80%) matching export side
2026-02-24 12:52:23 +07:00
Nicholas Dudfield
b9dd854595 refactor: unify featureExport + featureConsensusEntropy into featureExportRNG
Single amendment flag for both features. numFeatures 94 → 93.
Exclude featureExportRNG from default test set to prevent
ConsensusEntropy pseudo-tx injection from breaking existing tests.
2026-02-21 17:46:46 +07:00
Nicholas Dudfield
3bead8dcb6 merge: integrate origin/export-uvtxn into consensus-phase-entropy
Resolve 14 conflicts keeping both sides. Renumber TOO_LITTLE_ENTROPY
from -46 to -48 to avoid collision with export error codes.
Fix sfHookExportCount to soeOPTIONAL in InnerObjectFormats (only set
when featureExportRNG is enabled).
2026-02-21 17:41:37 +07:00
Nicholas Dudfield
908a78a1d9 fix: regenerate hook/extern.h to match hook_api.macro ordering 2026-02-20 10:11:37 +07:00
Nicholas Dudfield
a9e3dc41d4 fix: add featureExport stub for standalone guard_checker build 2026-02-20 10:05:58 +07:00
Nicholas Dudfield
02990eb4ee Merge remote-tracking branch 'origin/dev' into consensus-phase-entropy
# Conflicts:
#	hook/extern.h
#	src/ripple/app/hook/hook_api.macro
#	src/ripple/protocol/Feature.h
#	src/ripple/protocol/impl/Feature.cpp
2026-02-19 10:57:40 +07:00
Nicholas Dudfield
ce76632322 Merge remote-tracking branch 'origin/dev' into export-uvtxn
# Conflicts:
#	Builds/CMake/RippledCore.cmake
#	hook/extern.h
#	src/ripple/app/hook/Guard.h
#	src/ripple/app/hook/applyHook.h
#	src/ripple/app/hook/guard_checker.cpp
#	src/ripple/app/tx/impl/Change.cpp
#	src/ripple/app/tx/impl/SetHook.cpp
#	src/ripple/protocol/Feature.h
#	src/ripple/protocol/impl/Feature.cpp
#	src/ripple/protocol/jss.h
2026-02-19 10:12:55 +07:00
Nicholas Dudfield
9eac54d690 Merge remote-tracking branch 'origin/dev' into consensus-phase-entropy
# Conflicts:
#	src/ripple/app/hook/Guard.h
#	src/ripple/app/hook/applyHook.h
#	src/ripple/app/tx/impl/SetHook.cpp
2026-02-17 10:12:46 +07:00
Nicholas Dudfield
24e4ac16ad docs(consensus): add extraction markers for remaining RNG sections
Add @@start/@@end comment markers to pseudo-tx submission filtering,
fast-polling, local testnet resource bucketing, and test environment
gating. No logic changes.
2026-02-13 12:52:14 +07:00
Nicholas Dudfield
94ce15d233 docs(consensus): add extraction markers for guided code review
Add @@start/@@end comment markers to key RNG pipeline sections for
automated documentation extraction. No logic changes.
2026-02-13 12:47:22 +07:00
Nicholas Dudfield
8f331a538e fix(consensus): harden proposal parser and guard dice(0) UB
Address findings from code review:

- dice(0): add early return with INVALID_ARGUMENT before modulo
  operation to prevent undefined behavior
- fromSerialIter: return std::optional to safely reject malformed
  payloads (truncated, unknown flag bits, trailing bytes) instead
  of throwing
- Update all callers (PeerImp, RCLConsensus, tests) for optional
- Add unit tests for dice(0) error code and 7 malformed wire cases
2026-02-12 16:18:30 +07:00
Nicholas Dudfield
7425ab0a39 fix(consensus): avoid structured bindings in lambda captures
clang-14 (CI) does not implement P2036R3 — structured bindings cannot
be captured by lambdas. Use explicit .first/.second instead.
2026-02-10 19:02:42 +07:00
Nicholas Dudfield
c5292bfe0d fix(test): use large dice range to avoid deterministic collision
Standalone synthetic entropy produces identical dice(6) results for
consecutive calls due to hash collision mod 6. Switch to dice(1000000)
and add diagnostic output for return code debugging.
2026-02-10 18:53:24 +07:00
Nicholas Dudfield
79b2f9f410 feat(hooks): add consensus entropy definitions to hook headers
Add dice/random externs, TOO_LITTLE_ENTROPY error code, sfEntropyCount
field code, and ttCONSENSUS_ENTROPY transaction type to hook SDK headers.
2026-02-10 18:53:16 +07:00
Nicholas Dudfield
e8358a82b1 feat(hooks): register dice/random with WasmEdge and add hook API tests
- ADD_HOOK_FUNCTION for dice/random (was defined+declared but not registered)
- Relax fairRng() seq check to allow previous ledger entropy (open ledger)
- Add hook tests: dice range, random fill, consecutive calls differ
- TODO: open-ledger entropy semantics need further thought
2026-02-10 17:58:52 +07:00
Nicholas Dudfield
d850e740e1 feat(consensus): standalone synthetic entropy and ConsensusEntropy test
Generate deterministic entropy in standalone mode so Hook APIs (dice/random)
work for testing. Add test suite verifying SLE creation on ledger close.
2026-02-10 17:27:57 +07:00
Nicholas Dudfield
61a166bcb0 feat(hooks): add dice() and random() hook APIs for consensus entropy
Port the Hook API surface from the tt-rng branch, adapted to use our
commit-reveal consensus entropy (ltCONSENSUS_ENTROPY / sfDigest).

Hook APIs:
- dice(sides): returns random int [0, sides) from consensus entropy
- random(write_ptr, write_len): fills buffer with 1-512 random bytes

Internal fairRng() derives per-execution entropy by hashing: ledger
seq + tx ID + hook hash + account + chain position + execution phase
+ consensus entropy + incrementing call counter. This ensures each
call within a single hook execution returns different values.

Quality gate: fairRng returns empty (TOO_LITTLE_ENTROPY) if fewer
than 5 validators contributed, preventing weak entropy from being
consumed by hooks.

Also adds sfEntropyCount and sfLedgerSequence to the consensus
entropy SLE and pseudo-tx, enabling the freshness and quality
checks needed by the Hook API.
2026-02-10 17:12:27 +07:00
Nicholas Dudfield
41a41ec625 feat(consensus): intersect expected proposers with UNL Report and adaptive quorum
setExpectedProposers() now filters incoming proposers against the
on-chain UNL Report, preventing non-UNL nodes from inflating the
expected set and causing unnecessary timeouts.

quorumThreshold() uses expectedProposers_.size() (recent proposers ∩
UNL) when available, falling back to full UNL Report count on cold
boot. This adapts to actual network conditions rather than relying
on a potentially stale UNL Report that over-counts offline validators.

Renamed activeUNLNodeIds_/cacheActiveUNL/isActiveUNLMember to
unlReportNodeIds_/cacheUNLReport/isUNLReportMember to make the
on-chain data source explicit.
2026-02-10 16:14:47 +07:00
Nicholas Dudfield
bc98c589b7 docs(consensus): fix stale quorum comment in phaseEstablish
Update inline comment to reflect that hasQuorumOfCommits() checks
expected proposers first, with 80% of active UNL as fallback.
2026-02-06 16:56:01 +07:00
Nicholas Dudfield
4f009e4698 fix(consensus): proceed with partial commitSet on timeout instead of zero entropy
When expected proposers don't all arrive before rngPIPELINE_TIMEOUT,
check if we still have quorum (80% of UNL). If so, build the commitSet
with available commits and continue to reveals. Only fall back to zero
entropy when truly below quorum.

Previously any missing expected proposer caused a full timeout with zero
entropy for that round. Now: kill 3 of 20 nodes → one 3s timeout round
per kill but entropy preserved (17/16 quorum met).
2026-02-06 16:40:33 +07:00
Nicholas Dudfield
b6811a6f59 feat(consensus): deterministic commitSets via expected proposers and seq=0 proofs
Wait for commits from last round's proposers (falling back to activeUNL
on cold boot) instead of 80% of UNL. This ensures all nodes build the
commitSet at the same moment with the same entries.

Split proof storage: commitProofs_ (seq=0 only, deterministic) and
proposalProofs_ (latest with reveal, for entropySet). Previously the
proof blob contained whichever proposeSeq was last seen, causing
identical commits to produce different SHAMap hashes across nodes.

20-node testnet: all nodes now produce identical commitSet hashes.
2026-02-06 16:27:10 +07:00
Nicholas Dudfield
ae88fd3d24 feat(consensus): add dedicated reveal-phase timeout measured from phase entry
Previously rngPIPELINE_TIMEOUT (3s) was measured from round start,
meaning txSet convergence could eat into the reveal budget. Now reveals
get their own rngREVEAL_TIMEOUT (1.5s) measured from the moment we
enter ConvergingReveal, ensuring consistent time for reveal collection
regardless of how long txSet convergence took.
2026-02-06 16:00:40 +07:00
Nicholas Dudfield
db3ed0c2eb fix(consensus): wait for all committers' reveals and fix local testnet resource charging
- Change hasMinimumReveals() to wait for reveals from ALL committers
  (pendingCommits_.size()) instead of 80% quorum. The commit set is
  deterministic, so we know exactly which reveals to expect.
  rngPIPELINE_TIMEOUT remains the safety valve for crash/partition.
  Fixes reveal-set non-determinism causing entropy divergence on
  15-node testnets.

- Resource manager: preserve port for loopback addresses so local
  testnet nodes each get their own resource bucket instead of all
  sharing one on 127.0.0.1 (causing rate-limit disconnections).

- Make RNG fast-poll interval configurable via XAHAU_RNG_POLL_MS
  env var (default 250ms) for testnet tuning.
2026-02-06 15:22:42 +07:00
Nicholas Dudfield
960808b172 fix(consensus): skip RNG wait when quorum is impossible and base threshold on active UNL
When fewer participants are present than the quorum threshold, skip the
RNG commit wait immediately instead of waiting the full pipeline timeout.
Also base the quorum on activeUNLNodeIds_ (UNL Report with fallback)
instead of the full trusted key set, so the denominator reflects who is
actually active on the network.
2026-02-06 14:34:28 +07:00
Nicholas Dudfield
a9dffd38ff fix(consensus): shorten RNG pipeline timeout to 3s for faster recovery
Add rngPIPELINE_TIMEOUT (3s) to replace ledgerMAX_CONSENSUS (10s) in
the commit/reveal quorum gates. Late-joining nodes enter as
proposing=false and cannot contribute commitments until promoted, so
waiting beyond a few seconds just delays the ZERO-entropy fallback and
penalizes recovery. Add inline comments documenting the late-joiner
constraint and SHAMap sync's role as a dropped-proposal safety net.
2026-02-06 14:04:53 +07:00
Nicholas Dudfield
382e6fa673 fix(consensus): verify reveals match commitments and cache UNL for observers
Prevent grinding attacks by verifying sha512Half(reveal, pubKey, seq)
matches the stored commitment before accepting a reveal. Also move
cacheActiveUNL() into startRound so non-proposing nodes (exchanges,
block explorers) correctly accept RNG data instead of diverging with
zero entropy.
2026-02-06 13:28:55 +07:00
Nicholas Dudfield
2905b0509c perf(consensus): gate RNG SHAMap fetches on sub-state
During ConvergingTx all RNG data arrives via proposal leaves, so
fetching a peer's commitSet before we've built our own just generates
unnecessary traffic. Only fetch commitSetHash once in ConvergingCommit+,
and entropySetHash once in ConvergingReveal.
2026-02-06 13:18:53 +07:00
Nicholas Dudfield
4911c1bf52 feat(consensus): embed proposal signature proofs in RNG SHAMap entries
Prevents spoofed SHAMap entries by embedding verifiable proof blobs
(proposal signature + metadata) in each commit/reveal entry via sfBlob.

- Store ProposalProof in harvestRngData (peers) and propose() (self)
- serializeProof: pack proposeSeq/closeTime/prevLedger/position/sig
- verifyProof: reconstruct signingHash, verify against public key
- Embed proofs in buildCommitSet/buildEntropySet via sfBlob field
- Verify proofs in handleAcquiredRngSet (both diff and visitLeaves paths)
- Add stall fix: gate ConvergingTx on timeout when commits unavailable
- Clear proposalProofs_ in clearRngState
2026-02-06 11:47:42 +07:00
Nicholas Dudfield
1744d21410 docs(consensus): explain union convergence model for RNG sets 2026-02-06 11:17:52 +07:00
Nicholas Dudfield
34ff53f65d feat(consensus): add UNL enforcement for RNG commit-reveal pipeline
Cache active UNL NodeIDs per round from UNL Report (in-ledger),
falling back to getTrustedMasterKeys() on fresh testnets.
Reject non-UNL validators at all entry points: harvestRngData,
buildCommitSet, buildEntropySet, and handleAcquiredRngSet.
2026-02-06 11:12:03 +07:00
Nicholas Dudfield
893f8d5a10 feat(consensus): replace fake hashes with real SHAMap-backed commitSet/entropySet
Build real ephemeral (unbacked) SHAMaps for commitSet and entropySet using
ttCONSENSUS_ENTROPY entries with tfEntropyCommit/tfEntropyReveal flags.
Reuse InboundTransactions pipeline for peer fetch/diff/merge with no new
classes. Encode NodeID in sfAccount to avoid master-vs-signing key mismatch.
Add isPseudoTx guard in ConsensusTransSetSF to prevent pseudo-tx submission.
Route acquired RNG sets via isRngSet/gotRngSet in NetworkOPs mapComplete.
2026-02-06 10:38:06 +07:00
Nicholas Dudfield
3e5389d652 feat(consensus): add 250ms fast-poll for RNG sub-state transitions
During ConvergingCommit and ConvergingReveal sub-states, poll at 250ms
instead of the default 1s ledgerGRANULARITY. This reduces total RNG
pipeline overhead from ~3s to ~500ms while keeping the normal heartbeat
interval unchanged for all other consensus phases.
2026-02-06 09:21:42 +07:00
Nicholas Dudfield
c44dea3acf fix(consensus): resolve commit-reveal pipeline bugs enabling non-zero entropy
Three critical fixes that unblock the RNG commit-reveal pipeline:

- Remove entropy secret regeneration in ConvergingTx->ConvergingCommit
  transition that was overwriting the onClose() secret, breaking reveal
  verification against the original commitment
- Change ExtendedPosition operator== to compare txSetHash only, preventing
  deadlock where nodes transitioning sub-states at different times would
  break haveConsensus() for all peers
- Self-seed own commitment and reveal into pending collections so the
  node counts toward its own quorum checks

Also adds ExtendedPosition_test with signing, suppression, serialization
round-trip and equality tests, iterator safety fix in BuildLedger, wire
compatibility early-return, and RNG debug logging throughout the pipeline.
2026-02-06 09:03:26 +07:00
Nicholas Dudfield
a6dd54fa48 feat(consensus): add featureConsensusEntropy amendment gating
- Register ConsensusEntropy amendment (Supported::yes, DefaultNo)
- Gate entropy pseudo-tx injection behind amendment in doAccept()
- Gate preflight with temDISABLED when amendment not enabled
- Bump numFeatures 90 -> 91
- Exclude featureConsensusEntropy from default test environment to
  avoid breaking existing test transaction count assumptions
2026-02-06 07:29:48 +07:00
Nicholas Dudfield
28bd0a22d3 feat(consensus): add entropy injection, tx ordering, and dispatch registration
- Implement injectEntropyPseudoTx() to combine reveals into final
  entropy hash and inject as pseudo-tx into CanonicalTXSet in doAccept()
- Modify BuildLedger applyTransactions() to apply entropy tx FIRST
  before all other transactions to prevent front-running
- Remove redundant explicit threading in applyConsensusEntropy() as
  sfPreviousTxnID/sfPreviousTxnLgrSeq are set automatically by
  ApplyStateTable::threadItem()
- Register ttCONSENSUS_ENTROPY in applySteps.cpp dispatch tables
  (preflight, preclaim, calculateBaseFee, apply)
- Add ltCONSENSUS_ENTROPY to InvariantCheck.cpp valid type whitelist
2026-02-06 05:43:19 +07:00
Nicholas Dudfield
960fffcf82 feat(consensus): add ttCONSENSUS_ENTROPY pseudo-transaction protocol layer
Add protocol definitions for consensus-derived entropy pseudo-transaction:
- ttCONSENSUS_ENTROPY = 105 transaction type
- ltCONSENSUS_ENTROPY = 0x0058 ledger entry type
- keylet::consensusEntropy() singleton keylet (namespace 'X')
- applyConsensusEntropy() handler in Change.cpp
- Added to isPseudoTx() in STTx.cpp

The entropy value is stored in sfDigest field of the singleton ledger object.
This provides the protocol foundation for same-ledger entropy injection.
2026-02-05 17:26:31 +07:00
Nicholas Dudfield
e7867c07a1 feat(consensus): add RNG sub-state gating logic in phaseEstablish
- Add clearRngState() call in startRoundInternal
- Reset estState_ in closeLedger when entering establish phase
- Implement three-phase RNG checkpoint gating:
  - ConvergingTx: wait for quorum commits, build commitSet
  - ConvergingCommit: reveal entropy, transition immediately
  - ConvergingReveal: wait for reveals or timeout, build entropySet
- Use if constexpr for test framework compatibility
2026-02-05 16:53:00 +07:00
Nicholas Dudfield
a828e8a44d feat(consensus): add RNG wire protocol and harvest logic
- Serialize full ExtendedPosition in share() and propose()
- Deserialize ExtendedPosition in PeerImp using fromSerialIter()
- Add harvestRngData() to collect commits/reveals from peer proposals
- Conditionally call harvest via if constexpr for test compatibility
2026-02-05 16:41:13 +07:00
Nicholas Dudfield
bb33e7cf64 feat(consensus): add ExtendedPosition for RNG entropy support
Introduce data structures for consensus-derived randomness using
commit-reveal scheme:

- Add ExtendedPosition struct with consensus targets (txSetHash,
  commitSetHash, entropySetHash) and pipelined leaves (myCommitment,
  myReveal)
- operator== excludes leaves to allow convergence with unique leaves
- add() includes ALL fields to prevent signature stripping attacks
- Add EstablishState enum for sub-phases: ConvergingTx, ConvergingCommit,
  ConvergingReveal
- Update Consensus template to use Adaptor::Position_t
- Add Position_t typedef to RCLConsensus::Adaptor and test CSF Peer

This is the foundational data structure work for the RNG implementation.
The gating logic and entropy computation will follow.
2026-02-05 16:20:54 +07:00
Nicholas Dudfield
7e8e0654cd chore: add documentation markers for pr-description-outline 2026-01-28 15:00:14 +07:00
Nicholas Dudfield
38af0626e0 chore: add documentation markers for pr-description 2026-01-28 14:50:56 +07:00
Nicholas Dudfield
8500e86f57 chore: remove projected-source documentation markers 2026-01-28 11:30:47 +07:00
Nicholas Dudfield
1fc4fd9bfd chore: regenerate hook headers for export feature 2026-01-28 11:27:13 +07:00
Nicholas Dudfield
e4875e5398 refactor: remove ttEXPORT_SIGN and UVTx infrastructure
- Delete ExportSign.cpp/h transactor (ttEXPORT_SIGN no longer used)
- Remove isUVTx() function and all UVTx checks from Transactor/TxQ
- Remove ttEXPORT_SIGN from TxFormats enum and format definition
- Remove jss::ExportSign
- Move signPendingExports() to ExportSignatureCollector

Export signatures are now collected ephemerally via TMValidation
messages, not via ttEXPORT_SIGN transactions.
2026-01-28 10:59:45 +07:00
Nicholas Dudfield
5b1b142be0 chore: remove stray iostream includes 2026-01-28 10:34:11 +07:00
Nicholas Dudfield
5ba832204a test: remove unused scaffolding from Export_test
- Remove accept_wasm and emit_wasm hooks (not export-related)
- Remove testBasicSetup, testEmitPayment, testXportPayment
- Keep only testXportPaymentWithValidator which tests the export flow
2026-01-28 10:31:49 +07:00
Nicholas Dudfield
1257b3a65c Merge remote-tracking branch 'origin/dev' into export-uvtxn 2026-01-28 10:21:37 +07:00
Nicholas Dudfield
6013ed2cb6 refactor: remove vestigial on-ledger export signature code
- Remove makeExportSignTxns() function (signatures now via TMValidation)
- Simplify ExportSign::doApply() to no-op (ttEXPORT_SIGN kept for protocol)
- Remove sfSigners from ltEXPORTED_TXN format (collected in memory now)
- Remove unused OpenView include and forward declaration
- Remove vestigial comment in TxQ about makeExportSignTxns
2026-01-28 10:19:14 +07:00
Nicholas Dudfield
034010716e feat: add signature verification cache for export signatures
Add cryptographic verification of export signatures as they arrive:
- stashTxnData() caches serialized txn for verification
- verifyAndAddSignature() verifies against cached data, rejects invalid
- isSignatureVerified() / verifySignature() for Transactor fallback
- Cleanup methods updated to clear verification cache

Also removes leftover debug std::cerr from OpenView, STObject, and tests.
2026-01-27 18:03:55 +07:00
Nicholas Dudfield
b28793b0fa chore: clean up export debug logging
- remove DBG_EXPORT macros and all usages
- remove [EXPORT-TRACE] and [EXPORT-TIMING] debug prefixes
- adjust log levels (verbose logs to trace, summaries to debug)
- upgrade "quorum reached" to info level (important event)
- standardize log prefixes to use "Export:"
- re-enable relay loop in OpenLedger.cpp
- remove reentrant call detection debug code
2026-01-27 16:21:02 +07:00
Nicholas Dudfield
4bce392c31 feat: continuous signature broadcasting for export robustness
Validators now sign ALL pending ltEXPORTED_TXN entries every ledger
(not just those from the current ledger). Signatures are cached in
ExportSignatureCollector and re-broadcast until the export is finalized.

Changes:
- Add hasSignatureFrom() and getSignatureFrom() to collector for
  checking/retrieving cached signatures
- signPendingExports() now iterates ALL pending exports, uses cached
  signature if available, otherwise signs fresh
- Signatures keep broadcasting until ltEXPORTED_TXN is deleted

This ensures:
- Late validators can contribute (sign when they come online)
- Network partitions self-heal (signatures propagate on reconnect)
- Node restarts recover (re-sign from ledger state)

The ltEXPORTED_TXN acts as a "ticket" - signatures only valid while it
exists. No explicit expiry check needed; ledger state is the gatekeeper.
2026-01-26 18:25:24 +07:00
Nicholas Dudfield
244a28b981 feat: implement ephemeral export signature collection
Replace on-ledger ttEXPORT_SIGN transactions with ephemeral signature
collection via TMValidation messages. This eliminates O(n²) metadata
bloat from accumulating signatures on-ledger.

Changes:
- Add ExportSignatureCollector for in-memory signature storage with
  quorum tracking (80% UNL threshold)
- Extend TMValidation protobuf with exportSignatures field
- Sign pending exports during validate() and broadcast via validation
- Extract signatures from received TMValidation in PeerImp
- TxQ checks quorum from memory instead of ledger
- Inject ttEXPORT when quorum reached (can be ledger N+1 or N+2)
- Clean up collector after ttEXPORT processed

Includes [EXPORT-TIMING] debug logging for timing analysis.
2026-01-26 17:54:17 +07:00
Nicholas Dudfield
f2838351c9 chore: add [EXPORT-TRACE] debug logging for export flow tracing
adds step-by-step trace logging with [EXPORT-TRACE] prefix to track
the complete export transaction lifecycle:
- STEP-1: xport() creates ltEXPORTED_TXN
- STEP-2a: rawTxInsert ttEXPORT_SIGN in callback
- STEP-2b: doApply ttEXPORT_SIGN
- STEP-3a: rawTxInsert ttEXPORT
- STEP-4: doApply ttEXPORT (cleanup)

filter with: grep '\[EXPORT-TRACE\]'
2026-01-23 08:10:33 +07:00
Nicholas Dudfield
dae082d6a5 chore: format files with clang-format 2026-01-22 16:42:05 +07:00
Nicholas Dudfield
619a4a68f7 fix: resolve export feature bugs and add comprehensive tests
- fix Guard.h: add import_whitelist_2 to signature lookup chain
  (was causing "Function type is inconsistent" errors for xport APIs)
- fix InvariantCheck.cpp: add ltEXPORTED_TXN to valid ledger entry types
  (was causing "invalid ledger entry type added" invariant failures)
- add SetHook.cpp: TODO comment documenting API version confusion

- add Export_test.cpp: comprehensive test suite for export feature
  - testBasicSetup: verify hook installation works
  - testEmitPayment: verify emit() flow works
  - testXportPayment: verify xport() creates ltEXPORTED_TXN
  - includes DebugLogs helper for per-partition log levels
  - parameterized runXportTest helper for future validator tests

Note: validator signing flow (ttEXPORT_SIGN) still needs debugging -
causes internal error on env.close() when validator config enabled.
2026-01-22 09:51:50 +07:00
Nicholas Dudfield
4a6db8bb05 Merge remote-tracking branch 'origin/dev' into export-uvtxn 2026-01-22 08:07:58 +07:00
Nicholas Dudfield
c86479bc58 fix: correct xport api signature and sfExportedTxn type usage
- Fix xport hook API whitelist to declare 4 args (I32, I32, I32, I32)
  instead of 2, matching the actual implementation signature
- Fix TxQ.cpp to use emplace_back with STObject for sfExportedTxn
  instead of setFieldVL, since sfExportedTxn is OBJECT type not VL.
  The previous code would throw "Wrong field type" at runtime.
2026-01-22 07:41:12 +07:00
Nicholas Dudfield
dc6a2dc6ff refactor: separate ExportSign transactor from Change
Move ttEXPORT_SIGN handling to dedicated ExportSign transactor class,
following the same pattern as ttENTROPY/Entropy from the RNG feature.
UVTxns (signed validator transactions) should not be mixed with
pseudo-transactions in the Change transactor.

- Create ExportSign.h/cpp with preflight, preclaim, doApply
- Route ttEXPORT_SIGN through ExportSign in applySteps.cpp
- Remove UVTx branches from Change transactor
- Add documentation markers to View.h for inUNLReport functions
2026-01-22 07:41:12 +07:00
Nicholas Dudfield
c01b9a657b feat: implement uvtxn pattern for ttEXPORT_SIGN
Port the UNL Validator Transaction (UVTxn) pattern from the RNG feature
to allow validators to submit signed ttEXPORT_SIGN transactions without
requiring a funded account.

Changes:
- Add isUVTx() to identify UVTxn transaction types
- Add inUNLReport() templates to check validator UNLReport membership
- Add getValidationSecretKey() to Application for signing
- Modify Transactor for UVTxn bypasses (fee, seq, signature checks)
- Add makeExportSignTxns() to generate validator signatures
- Hook into RCLConsensus to submit ttEXPORT_SIGN during accept
- Update applySteps.cpp routing for ttEXPORT_SIGN
- Remove direct ttEXPORT_SIGN injection from TxQ::accept

Note: Currently uses Change transactor with UVTx branches.
May refactor to dedicated ExportSign transactor class.
2026-01-20 13:44:38 +07:00
Nicholas Dudfield
652b181b5d chore: clang format 2026-01-20 12:44:14 +07:00
RichardAH
8329d78f32 Update src/ripple/app/tx/impl/Import.cpp
Co-authored-by: tequ <git@tequ.dev>
2025-12-21 13:42:46 +10:00
RichardAH
bf4579c1d1 Update src/ripple/app/tx/impl/Change.cpp
Co-authored-by: tequ <git@tequ.dev>
2025-12-21 13:42:37 +10:00
RichardAH
73e099eb23 Update src/ripple/app/hook/impl/applyHook.cpp
Co-authored-by: tequ <git@tequ.dev>
2025-12-21 13:42:29 +10:00
RichardAH
2e311b4259 Update src/ripple/app/hook/applyHook.h
Co-authored-by: tequ <git@tequ.dev>
2025-12-21 13:42:20 +10:00
RichardAH
7c8e940091 Merge branch 'dev' into export 2025-12-19 13:27:02 +10:00
Richard Holland
9b90c50789 featureExport compiling, untested 2025-12-19 14:19:17 +11:00
Richard Holland
a18e2cb2c6 remainder of the export feature... untested uncompiled 2025-12-14 19:04:37 +11:00
Richard Holland
be5f425122 change symbol name to xport 2025-12-14 13:27:44 +11:00
Richard Holland
fc6f4762da export hook apis, untested 2025-12-13 15:46:08 +11:00
355 changed files with 39317 additions and 131431 deletions

View File

@@ -1,6 +1,37 @@
codecov:
require_ci_to_pass: true
comment:
behavior: default
layout: reach,diff,flags,tree,reach
show_carryforward_flags: false
coverage:
range: "60..80"
precision: 1
round: nearest
status:
project:
default:
target: 60%
threshold: 2%
patch:
default:
target: auto
threshold: 2%
changes: false
github_checks:
annotations: true
parsers:
cobertura:
partials_as_hits: true
handle_missing_conditions : true
slack_app: false
ignore:
- "src/test/"
- "include/xrpl/beast/test/"
- "include/xrpl/beast/unit_test/"

View File

@@ -1,8 +1,25 @@
# This feature requires Git >= 2.24
# To use it by default in git blame:
# git config blame.ignoreRevsFile .git-blame-ignore-revs
# Format first-party source according to .clang-format
50760c693510894ca368e90369b0cc2dabfd07f3
e2384885f5f630c8f0ffe4bf21a169b433a16858
241b9ddde9e11beb7480600fd5ed90e1ef109b21
760f16f56835663d9286bd29294d074de26a7ba6
0eebe6a5f4246fced516d52b83ec4e7f47373edd
# Reintroduce Clang-Format & Levelization
da1d20d6d5d862716125d60899b80fab5302954a
# Consolidate external libraries
da1d20d6d5d862716125d60899b80fab5302954a
# Rename .hpp to .h
0345a2645d0f5ad900f4fbbcaff96040d3a887fc
# Format formerly .hpp files
5a227dc719016e10045e17c9396ad401118044f1
# Rewrite includes
e61880699997398f5a746e6c4034edc7632661f5
# Move CMake directory (#4997)
e47b1c1b3b97c3f6d11858ee02f463596e29e7f0
# Rearrange sources (#4997)
bfafa2bb39e562901736d656806bd700c3699a2f
# Rewrite includes (#4997)
e61880699997398f5a746e6c4034edc7632661f5
# Recompute loops (#4997)
d25b5dcd568bb96c18e347d55fac10fe901a1bfb
# Reformat code with clang-format-18
02749feea88ce61c1f7eeb2d61a57d8ecf07ab11

3
.gitattributes vendored
View File

@@ -11,6 +11,3 @@ LICENSE binary
*.vcxproj text eol=crlf
*.props text eol=crlf
*.filters text eol=crlf
# Mark it as generated and folded out by default
src/test/app/SetJSHook_wasm.h linguist-generated=true

View File

@@ -40,7 +40,7 @@ jobs:
run: |
# Download install.sh
curl -o /tmp/wasienv-install.sh https://raw.githubusercontent.com/wasienv/wasienv/master/install.sh
# Replace /bin to /local/bin
sed -i 's|/bin|/local/bin|g' /tmp/wasienv-install.sh

View File

@@ -20,7 +20,7 @@ jobs:
sudo apt-get update
sudo apt-get install clang-format-${CLANG_VERSION}
- name: Format first-party sources
run: find include src -type f \( -name '*.cpp' -o -name '*.hpp' -o -name '*.h' -o -name '*.ipp' \) -not -path "src/quickjs/*" -exec clang-format-${CLANG_VERSION} -i {} +
run: find include src -type f \( -name '*.cpp' -o -name '*.hpp' -o -name '*.h' -o -name '*.ipp' \) -exec clang-format-${CLANG_VERSION} -i {} +
- name: Check for differences
id: assert
run: |

View File

@@ -16,6 +16,8 @@ jobs:
generator: ./hook/generate_extern.sh
- target: hook/sfcodes.h
generator: bash ./hook/generate_sfcodes.sh
- target: hook/genesis/headers/sfcodes.h
generator: bash ./hook/generate_sfcodes.sh
- target: hook/tts.h
generator: ./hook/generate_tts.sh
- target: hook/ls_flags.h

9
.gitignore vendored
View File

@@ -124,10 +124,15 @@ pkg
CMakeUserPresets.json
bld.rippled/
# binary
qjsc
generated
.vscode
# AI docs (local working documents)
.ai-docs/
# Local formal-methods workspace; kept as a separate repository and optionally
# symlinked here for navigation.
formal/lean/xahau_consensus
# Suggested in-tree build directory
/.build/

4
.testnet/.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
output/
__pycache__/
scenarios/odd-cases/
scenarios/suite-experiments.yml

View File

@@ -0,0 +1,29 @@
"""Scenario: ConsensusEntropy amendment crashes non-supporting node.
Votes ConsensusEntropy accept on all nodes except n4, then waits for n4
to crash as the amendment activates without its support.
x-testnet run --scenario-script consensus_entropy_crash.py
"""
from helpers import CONSENSUS_ENTROPY_FEATURE
async def scenario(ctx, log):
await ctx.wait_for_ledger_close()
ctx.feature(CONSENSUS_ENTROPY_FEATURE, vetoed=False, exclude_nodes=[4])
log("Waiting for ConsensusEntropy to be voted for...")
await ctx.wait_for_feature(
CONSENSUS_ENTROPY_FEATURE,
check=lambda s: not s.get("vetoed"),
exclude_nodes=[4],
timeout=60,
)
log("Waiting for n4 to crash...")
op = await ctx.wait_for_nodes_down(nodes=[4], timeout=600)
ctx.assert_log("unsupported amendments activated", since=op.started, nodes=[4])
ctx.assert_exit_status(0, nodes=[4])
log("PASS: n4 shut down due to unsupported amendment")

View File

@@ -0,0 +1,52 @@
""":descr: entropy stays valid under transaction load"""
from __future__ import annotations
from helpers import require_entropy, get_entropy_tx, assert_valid_entropy
variants = [
{"label": "light", "min_txns": 5, "max_txns": 10},
{"label": "heavy", "min_txns": 50, "max_txns": 60},
{"label": "super_heavy", "min_txns": 90, "max_txns": 120},
]
async def scenario(ctx, log, *, min_txns=5, max_txns=10, **_):
await require_entropy(ctx, log)
gen = ctx.txn_generator(min_txns=min_txns, max_txns=max_txns)
await gen.start()
await gen.wait_until_ready()
log(f"Transaction generator ready ({min_txns}-{max_txns} txns/ledger)")
# Wait for pipeline warmup + a few txn-bearing ledgers.
await ctx.wait_for_ledgers(3, node_id=0, timeout=60)
start_seq = ctx.validated_ledger_index(0)
await ctx.wait_for_ledgers(10, node_id=0, timeout=120)
end_seq = ctx.validated_ledger_index(0)
log(f"Inspecting ledgers {start_seq + 1}{end_seq}")
digests = set()
total_user_txns = 0
for seq in range(start_seq + 1, end_seq + 1):
ce, user_txns = get_entropy_tx(ctx, seq)
digest, count = assert_valid_entropy(ce, seq, seen_digests=digests)
total_user_txns += len(user_txns)
log(
f" Ledger {seq}: EntropyCount={count} "
f"user_txns={len(user_txns)} Digest={digest[:16]}..."
)
await gen.stop()
log(
f"Verified {end_seq - start_seq} ledgers: {total_user_txns} user txns, "
f"all entropy valid and unique"
)
if total_user_txns == 0:
raise AssertionError("No user transactions were included in any ledger")
log("PASS")

View File

@@ -0,0 +1,28 @@
""":descr: healthy non-standalone testnet without UNLReport mints Tier 1 fallback"""
from __future__ import annotations
from helpers import require_entropy, get_entropy_tx, assert_consensus_fallback
async def scenario(ctx, log):
await require_entropy(ctx, log)
# Non-standalone nodes require a ledger-anchored UNLReport before assigning
# validator_quorum / participant_aligned labels. Without it, the RNG pipeline
# may still collect commits/reveals, but injection must remain Tier 1.
await ctx.wait_for_ledgers(3, node_id=0, timeout=60)
log("Pipeline warmed up without UNLReport")
start_seq = ctx.validated_ledger_index(0)
await ctx.wait_for_ledgers(5, node_id=0, timeout=90)
end_seq = ctx.validated_ledger_index(0)
log(f"Inspecting ledgers {start_seq + 1} -> {end_seq}")
for seq in range(start_seq + 1, end_seq + 1):
ce, _ = get_entropy_tx(ctx, seq)
digest, count = assert_consensus_fallback(ce, seq)
log(f" Ledger {seq}: EntropyCount={count} Digest={digest[:16]}...")
log(f"Verified {end_seq - start_seq} ledgers: all consensus_fallback")
log("PASS")

View File

@@ -0,0 +1,162 @@
""":descr: 5/6 validator_quorum, 4/6 participant_aligned (tier 2), recovery
Requires node_count: 6 (see suite.yml) — the smallest NON-degenerate Tier 2
size. At n=6: tier2 floor = 4, validator quorum = 5, validation quorum = 5. So
6/6, 5/6 present -> validator_quorum (EntropyTier=3)
4/6 present -> participant_aligned (EntropyTier=2, count 4) <-- the band
3/6 present -> consensus_fallback (EntropyTier=1)
n=5 has NO tier-2 band (tier2 == quorum == 4), which is why the existing
degradation smoke at 5 nodes only ever sees tier 3 / fallback.
KEY: the 4/6 window is BELOW the 80% validation quorum (5). The 4 survivors
keep CLOSING ledgers that carry tier-2 entropy, but those ledgers do NOT
validate until the network recovers — exactly the transition window Tier 2
serves. So validated_ledger_index() stalls; we instead inspect a surviving
node's CLOSED ledger (its LCL) directly, and cross-check the injection from the
cohort's logs.
"""
from __future__ import annotations
from helpers import (
require_entropy,
get_entropy_tx,
assert_participant_aligned,
assert_validator_quorum,
)
def _closed_entropy(result):
"""(seq, ConsensusEntropy tx) from a ctx.ledger('closed', transactions=True)
result, or (None, None) if the fetch returned no usable ledger.
Enforces the per-ledger invariant that an entropy-enabled closed ledger
carries EXACTLY ONE ConsensusEntropy pseudo-tx (mirroring get_entropy_tx):
a duplicate or missing injection raises here with a clear error instead of
being silently skipped and resurfacing later as a generic 'no tier-2 ledger'.
"""
if not result or not isinstance(result.get("ledger"), dict):
return None, None
led = result["ledger"]
try:
seq = int(led.get("ledger_index"))
except (TypeError, ValueError):
return None, None
ce = [
t
for t in led.get("transactions", [])
if isinstance(t, dict) and t.get("TransactionType") == "ConsensusEntropy"
]
if len(ce) != 1:
raise AssertionError(
f"Closed ledger {seq}: expected 1 ConsensusEntropy txn, got {len(ce)}"
)
return seq, ce[0]
async def scenario(ctx, log):
await require_entropy(ctx, log)
# Baseline: healthy 6/6 produces validator_quorum entropy.
await ctx.wait_for_ledgers(1, node_id=0, timeout=30)
# --- 5/6: settles back to validator_quorum (5 present >= quorum 5) ---
val_before_drop = ctx.validated_ledger_index(0)
ctx.stop_node(5)
await ctx.wait_for_nodes_down(nodes=[5], timeout=30)
# Settle a few ledgers past the membership change. The ledger right at a
# validator drop can carry a transient consensus_fallback (tier 1, count 0,
# deterministic and by design) before the commit/reveal pipeline re-primes,
# so we do NOT assume any single post-drop ledger is already tier 3.
await ctx.wait_for_ledgers(4, node_id=0, timeout=90)
# 5/6 is at/above the 80% quorum (5), so steady state is validator_quorum.
# Scan the post-drop validated ledgers (all carry the 5-node cohort, so a
# tier-3 here has count == 5) and require at least one clean validator_quorum
# — EntropyTier=3, count >= quorum, non-zero digest — tolerating the
# transition fallback instead of depending on where the tip happened to land.
val_5of6 = ctx.validated_ledger_index(0)
t3_seq = None
for seq in range(val_5of6, val_before_drop, -1):
ce, _ = get_entropy_tx(ctx, seq)
tier = ce.get("EntropyTier")
log(f" 5/6 ledger {seq}: tier={tier} count={ce.get('EntropyCount')}")
if tier == 3:
assert_validator_quorum(ce, seq, min_count=5)
t3_seq = seq
break
if t3_seq is None:
raise AssertionError(
f"5/6: no validator_quorum (tier 3) entropy in post-drop validated "
f"ledgers {val_before_drop + 1}..{val_5of6}"
)
log(f"5/6: validator_quorum at validated seq {t3_seq}")
#@@start test-participant-aligned-window
# --- 4/6: participant_aligned (Tier 2) degraded window ---
ctx.stop_node(4)
await ctx.wait_for_nodes_down(nodes=[4], timeout=30)
# ~12s window: confirm tier-2 INJECTION from the cohort's logs, and that the
# round is NOT the below-quorum fallback path (which is what distinguishes
# the tier-2 band from the tier-1 fallback regime).
op = await ctx.sleep(12, name="tier2_window")
selected_t2 = ctx.search_logs(
r"RNG: entropy selected seq=\d+ tier=2 count=4",
within=op.window,
nodes=[0, 1, 2, 3],
)
log(f"4/6: 'entropy selected tier=2 count=4' logs: {selected_t2.count}")
if selected_t2.count == 0:
raise AssertionError(
"4/6 window injected no participant_aligned (tier 2) entropy: no "
"'RNG: entropy selected ... tier=2 count=4' on the surviving cohort"
)
ctx.assert_not_log(
r"STALLDIAG: rng-commit-timeout-below-quorum",
within=op.window,
nodes=[0, 1, 2, 3],
)
# Verify the on-ledger EntropyTier=2 DIRECTLY: validation is stalled (4 < 5),
# so sample the surviving cohort's CLOSED ledger (its LCL — built but not yet
# validated). At least one must be participant_aligned with EntropyCount=4.
tier2_on_ledger = 0
last_seq = None
for _ in range(5):
seq, ce = _closed_entropy(
ctx.ledger("closed", transactions=True, node_id=0)
)
if ce is not None and seq is not None and seq != last_seq:
last_seq = seq
tier = ce.get("EntropyTier")
count = ce.get("EntropyCount", -1)
log(f" closed ledger {seq}: tier={tier} count={count}")
if tier == 2:
assert_participant_aligned(ce, seq, expected_count=4)
tier2_on_ledger += 1
await ctx.sleep(3)
if tier2_on_ledger == 0:
raise AssertionError(
"no closed participant_aligned (tier 2) ledger observed during the "
"4/6 window (tier 2 was injected per logs, but not seen on a closed "
"ledger)"
)
log(f"4/6: {tier2_on_ledger} participant_aligned closed ledger(s) verified")
#@@end test-participant-aligned-window
# --- Recovery: liveness — validation resumes once quorum is restored ---
ctx.start_node(4)
ctx.start_node(5)
await ctx.wait_for_ledgers(1, node_id=0, timeout=120)
val_recovered = ctx.validated_ledger_index(0)
if not val_recovered or val_recovered <= val_5of6:
raise AssertionError(
f"Validated ledger did not advance after recovery "
f"({val_5of6} -> {val_recovered})"
)
log(f"Recovered: validated seq {val_5of6} -> {val_recovered}")
log("PASS")

View File

@@ -0,0 +1,164 @@
""":descr: 4/5 liveness, 3/5 fail-closed sub-quorum window, recovery"""
from __future__ import annotations
from helpers import (
require_entropy,
get_entropy_tx,
entropy_fields,
assert_consensus_fallback,
)
def _closed_entropy(result):
"""Return (seq, ConsensusEntropy tx) from a closed-ledger RPC result.
The 3/5 window is below validation quorum, so validated-ledger history is
expected to stall. Sampling a surviving node's closed ledger catches any
local LCL that advanced despite the sub-quorum condition.
"""
if not result or not isinstance(result.get("ledger"), dict):
return None, None
ledger = result["ledger"]
try:
seq = int(ledger.get("ledger_index"))
except (TypeError, ValueError):
return None, None
ce = [
tx
for tx in ledger.get("transactions", [])
if isinstance(tx, dict) and tx.get("TransactionType") == "ConsensusEntropy"
]
if len(ce) != 1:
raise AssertionError(
f"Closed ledger {seq}: expected 1 ConsensusEntropy txn, got {len(ce)}"
)
return seq, ce[0]
async def scenario(ctx, log):
await require_entropy(ctx, log)
# Baseline: wait 1 ledger to confirm network is healthy.
await ctx.wait_for_ledgers(1, node_id=0, timeout=30)
# --- 4/5 liveness ---
ctx.stop_node(4)
await ctx.wait_for_nodes_down(nodes=[4], timeout=30)
await ctx.wait_for_ledgers(1, node_id=0, timeout=30)
log("4/5: liveness OK")
# Snapshot validated seq before dropping to 3/5.
val_before = ctx.validated_ledger_index(0)
# --- 3/5 degraded window ---
ctx.stop_node(3)
await ctx.wait_for_nodes_down(nodes=[3], timeout=30)
# 10s ≈ 3 rounds at 3s cadence.
await ctx.sleep(10)
val_after = ctx.validated_ledger_index(0)
log(f"3/5: validated ledger {val_before}{val_after}")
if val_after and val_before and val_after > val_before:
raise AssertionError(
f"3/5 sub-quorum window unexpectedly validated ledgers "
f"({val_before} -> {val_after})"
)
# If the surviving cohort exposes an advanced closed ledger despite being
# below validation quorum, it must fail closed to consensus_fallback. This
# keeps the entropy assertion live without pretending validated history
# should advance at 3/5.
degraded_fallback = 0
last_closed = None
for _ in range(5):
seq, ce = _closed_entropy(ctx.ledger("closed", transactions=True, node_id=0))
if seq and val_before and seq > val_before and seq != last_closed:
last_closed = seq
digest, count = assert_consensus_fallback(ce, seq)
degraded_fallback += 1
log(
f" 3/5 closed ledger {seq}: EntropyCount={count} "
f"Digest={digest[:16]}... FALLBACK"
)
await ctx.sleep(2)
log(f"3/5 closed-ledger fallback samples: {degraded_fallback}")
# Log checks tied to current transition mechanics:
# - commit-set SHAMap publication is the observable output of entering the
# commit sidecar phase
# - ConvergingCommit transition is the gateway out of seq=0-only behavior
# - rng-commit-timeout-below-quorum is the degraded-window fallback path
ctx.log_level("LedgerConsensus", "trace")
ctx.log_level("ConsensusExtensions", "trace")
op = await ctx.sleep(6, name="stall_window")
ctx.assert_not_log(
r"RNG: transitioned to ConvergingCommit", within=op.window, nodes=[0, 1, 2]
)
ctx.assert_not_log(
r"RNG: built commitSet SHAMap", within=op.window, nodes=[0, 1, 2]
)
gate_blocked = ctx.search_logs(
r"STALLDIAG: establish gate blocked reason=(pause|no-tx-consensus)",
within=op.window,
nodes=[0, 1, 2],
)
log(f"3/5: establish gate-blocked logs in 6s: {gate_blocked.count}")
below_quorum = ctx.search_logs(
r"STALLDIAG: rng-commit-timeout-below-quorum",
within=op.window,
nodes=[0, 1, 2],
)
log(f"3/5: RNG commit timeout below quorum logs in 6s: {below_quorum.count}")
# --- Recovery: restart nodes, verify ledger advancement ---
ctx.start_node(3)
ctx.start_node(4)
await ctx.wait_for_ledgers(1, node_id=0, timeout=120)
val_recovered = ctx.validated_ledger_index(0)
pre_recovery = max(v for v in [val_before, val_after] if v is not None)
log(f"Recovered: validated seq {pre_recovery}{val_recovered}")
if not val_recovered or val_recovered <= pre_recovery:
raise AssertionError(
f"Validated ledger did not advance after recovery "
f"({pre_recovery}{val_recovered})"
)
# Inspect post-recovery ledgers separately from the degraded window above.
# Once the network is back at quorum, validator-tier entropy is expected
# again (transitional fallback ledgers are fine) and must be quorum-met.
fallback_count = 0
validator_count = 0
for seq in range(pre_recovery + 1, val_recovered + 1):
ce, _ = get_entropy_tx(ctx, seq)
digest, entropy_count, is_fallback = entropy_fields(ce)
if is_fallback:
fallback_count += 1
else:
validator_count += 1
if entropy_count < 4:
raise AssertionError(
f"Ledger {seq}: validator entropy with sub-quorum "
f"EntropyCount={entropy_count} (need >= 4)"
)
log(
f" Ledger {seq}: EntropyCount={entropy_count} "
f"{'FALLBACK' if is_fallback else 'VALIDATOR'}"
)
log(
f"Entropy summary: {fallback_count} fallback, "
f"{validator_count} validator"
)
log("PASS")

View File

@@ -0,0 +1,44 @@
""":descr: drop 2 nodes (3/5 stall), restart both, verify recovery"""
from __future__ import annotations
from helpers import require_entropy
async def scenario(ctx, log):
await require_entropy(ctx, log)
await ctx.wait_for_ledgers(1, node_id=0, timeout=60)
log("Baseline OK")
# Drop 2 nodes → validation stall.
ctx.stop_node(3)
ctx.stop_node(4)
await ctx.wait_for_nodes_down(nodes=[3, 4], timeout=30)
info = ctx.rpc.server_info(node_id=0)
val_before = info.get("info", {}).get("validated_ledger", {}).get("seq", 0)
log(f"Stalled at validated seq {val_before}")
# Let it sit for a few rounds in degraded state.
await ctx.sleep(6)
# Bring both nodes back.
ctx.start_node(3)
ctx.start_node(4)
log("Restarted n3 and n4, waiting for recovery...")
# Recovery: wait for ANY validated ledger advance on n0.
await ctx.wait_for_ledger_close(node_id=0, timeout=60)
info = ctx.rpc.server_info(node_id=0)
val_after = info.get("info", {}).get("validated_ledger", {}).get("seq", 0)
log(f"Recovered: validated seq {val_before}{val_after}")
if val_after <= val_before:
raise AssertionError(
f"Validated ledger did not advance after recovery "
f"({val_before}{val_after})"
)
log("PASS")

View File

@@ -0,0 +1,27 @@
""":descr: all 5 nodes healthy, every ledger has valid unique quorum-met entropy"""
from __future__ import annotations
from helpers import require_entropy, get_entropy_tx, assert_valid_entropy
async def scenario(ctx, log):
await require_entropy(ctx, log)
# Wait for the RNG pipeline to warm up past initial proposal/sidecar gossip.
await ctx.wait_for_ledgers(3, node_id=0, timeout=60)
log("Pipeline warmed up")
start_seq = ctx.validated_ledger_index(0)
await ctx.wait_for_ledgers(10, node_id=0, timeout=120)
end_seq = ctx.validated_ledger_index(0)
log(f"Inspecting ledgers {start_seq + 1}{end_seq}")
digests = set()
for seq in range(start_seq + 1, end_seq + 1):
ce, _ = get_entropy_tx(ctx, seq)
digest, count = assert_valid_entropy(ce, seq, seen_digests=digests)
log(f" Ledger {seq}: EntropyCount={count} Digest={digest[:16]}...")
log(f"Verified {end_seq - start_seq} ledgers: all quorum entropy, all unique")
log("PASS")

View File

@@ -0,0 +1,104 @@
defaults:
network:
node_count: 5
launcher: tmux
find_ports: true
slave_delay: 0.2
features:
- ConsensusEntropy
- Export
track_features:
- ConsensusEntropy
- Export
unl_report: true
log_levels:
TxQ: info
Protocol: debug
Peer: debug
LedgerConsensus: debug
ConsensusExtensions: debug
NetworkOPs: info
env:
XAHAU_RESOURCE_PER_PORT: "1"
rc:
- rng_poll_ms=333
tests:
# --- CE + Export (80% quorum, SHAMap convergence) ---
- name: steady_state_export_ce
script: .testnet/scenarios/export/steady_state_export.py
- name: retriable_export_ce
script: .testnet/scenarios/export/retriable_export.py
- name: export_degradation_ce
script: .testnet/scenarios/export/export_degradation.py
network:
rc:
- rng_poll_ms=333
- n3:no_export_sig=true
- n4:no_export_sig=true
- name: export_without_unl_report
script: .testnet/scenarios/export/export_without_unl_report.py
network:
features:
- Export
track_features:
- Export
unl_report: false
- name: export_no_veto_missing_observation
script: .testnet/scenarios/export/export_no_veto_missing_observation.py
network:
rc:
- rng_poll_ms=333
- n4:no_export_sig_hash=true
# CE + Export: 1 node suppressed, 4/5 = 80% quorum, should succeed
- name: export_ce_one_node_down
script: .testnet/scenarios/export/export_quorum.py
params:
expect_success: true
network:
rc:
- rng_poll_ms=333
- n4:no_export_sig=true
# --- Export only, no CE (80% active-view quorum) ---
- name: export_only_all_up
script: .testnet/scenarios/export/export_quorum.py
params:
expect_success: true
network:
features:
- Export
track_features:
- Export
- name: export_only_one_node_down
script: .testnet/scenarios/export/export_quorum.py
params:
expect_success: true
network:
features:
- Export
track_features:
- Export
rc:
- rng_poll_ms=333
- n4:no_export_sig=true
- name: export_only_two_nodes_down
script: .testnet/scenarios/export/export_quorum.py
params:
expect_success: false
network:
features:
- Export
track_features:
- Export
rc:
- rng_poll_ms=333
- n3:no_export_sig=true
- n4:no_export_sig=true

View File

@@ -0,0 +1,127 @@
""":descr: Submit ttEXPORT with 2 nodes suppressing export sigs, verify it
retries via terRETRY_EXPORT until LLS expiry (insufficient signatures).
Nodes 3 and 4 have runtime_config no_export_sig=true, so only 3/5 nodes
provide export signatures. With 80% quorum = ceil(5*0.8) = 4 required,
the export cannot reach quorum and should expire via tecEXPORT_EXPIRED.
Flow:
1. Fund alice and bob
2. alice submits ttEXPORT with tight LLS
3. Export retries (only 3/5 sigs available, need 4)
4. Verify export expires with tecEXPORT_EXPIRED
5. Verify subsequent payment still works (sequence not permanently blocked)
"""
from __future__ import annotations
from export_helpers import (
EXPORT_RETRY_LEDGER_WINDOW,
require_export,
assert_shadow_ticket,
)
async def scenario(ctx, log):
await require_export(ctx, log)
# --- Setup ---
await ctx.fund_accounts({"alice": 10000, "bob": 1000})
log("Accounts funded")
alice = ctx.account("alice")
bob = ctx.account("bob")
current_seq = ctx.validated_ledger_index(0)
log(f"Current ledger: {current_seq}")
log("Nodes 3,4 have runtime_config no_export_sig=true (3/5 sigs, need 4)")
#@@start test-export-below-quorum-expiry
# --- Submit ttEXPORT (should retry then expire -- only 3/5 sigs) ---
export_start = ctx.mark("export-degradation-submit-start")
result = await ctx.submit_and_wait(
{
"TransactionType": "Export",
"LastLedgerSequence": current_seq + EXPORT_RETRY_LEDGER_WINDOW,
"Fee": "1000000",
"ExportedTxn": {
"TransactionType": "Payment",
"Account": alice.address,
"Destination": bob.address,
"Amount": "1000000",
"Fee": "10",
"Sequence": 0,
"TicketSequence": 1,
"FirstLedgerSequence": current_seq + 1,
"LastLedgerSequence": current_seq + EXPORT_RETRY_LEDGER_WINDOW,
"Flags": 2147483648,
"SigningPubKey": "",
},
},
alice.wallet,
timeout=60,
)
export_end = ctx.mark("export-degradation-submit-end")
final_seq = ctx.validated_ledger_index(0)
engine_result = result.get("engine_result", "")
log(f"Export completed at ledger {final_seq}, result: {engine_result}")
# With only 3/5 sigs and 80% quorum (4 required), export MUST fail
if engine_result == "tesSUCCESS":
raise AssertionError(
"Export should NOT have succeeded with only 3/5 sigs "
"(need 4 for 80% quorum) -- check runtime_config no_export_sig"
)
# Should be tecEXPORT_EXPIRED (LLS reached without quorum). Be exact here:
# any other non-success means the retry/expiry boundary regressed.
if engine_result != "tecEXPORT_EXPIRED":
raise AssertionError(
f"Expected tecEXPORT_EXPIRED below quorum, got {engine_result}"
)
log(f"Export failed as expected ({engine_result})")
retry_logs = ctx.assert_log(
r"Export: insufficient signatures .*result=terRETRY_EXPORT",
since=export_start,
until=export_end,
)
log(f"Export insufficient-signature retries: {retry_logs.count}")
expired_logs = ctx.assert_log(
r"Export: last ledger expired .*result=tecEXPORT_EXPIRED",
since=export_start,
until=export_end,
)
log(f"Export LLS expiry logs: {expired_logs.count}")
# No shadow ticket should exist (export never reached quorum)
assert_shadow_ticket(ctx, alice.address, log, expect_exists=False)
#@@end test-export-below-quorum-expiry
# --- Verify subsequent payment works regardless ---
log("Submitting payment from alice to bob...")
pay_result = await ctx.submit_and_wait(
{
"TransactionType": "Payment",
"Destination": bob.address,
"Amount": "1000000",
"Fee": "12",
},
alice.wallet,
timeout=30,
)
pay_engine = pay_result.get("engine_result", "")
log(f"Payment result: {pay_engine}")
if pay_engine != "tesSUCCESS":
raise AssertionError(
f"Payment failed after expired export: {pay_engine} "
f"-- sequence may be blocked"
)
log("Payment succeeded -- account not permanently blocked")
log("PASS")

View File

@@ -0,0 +1,246 @@
"""Shared helpers for Export scenario tests."""
from __future__ import annotations
from xahaud_scripts.testnet.config import _unl_report_index, feature_name_to_hash
EXPORT_RETRY_LEDGER_WINDOW = 5
async def require_export(
ctx, log, *, require_unl_report=True, require_runtime_config=True
):
"""Wait for first ledger and assert Export is enabled.
Network-mode Export success requires a parent-ledger UNLReport-backed
active validator view. Most export scenarios seed that report in genesis;
assert it here so a success-path test cannot accidentally pass setup
without the condition Export::doApply requires. The no-UNLReport retry
scenario opts out deliberately.
The tracked export suite also uses XAHAUD_RUNTIME_TEST_CONFIG for polling
and fault-injection knobs. Default binaries reject the runtime_config RPC,
so check it up front rather than silently running without those knobs.
"""
await ctx.wait_for_ledger_close(timeout=120)
if require_runtime_config:
result = ctx.rpc.runtime_config(0)
if not result or result.get("error"):
raise AssertionError(
"Export suite requires a binary built with "
"xahaud_runtime_test_config=ON; runtime_config RPC returned "
f"{result}"
)
log("RuntimeConfig RPC active")
feature = ctx.feature_check(feature_name_to_hash("Export"), node_id=0)
if not feature or not feature.get("enabled", False):
raise AssertionError(f"Export not enabled: {feature}")
log("Export enabled")
if require_unl_report:
result = ctx.rpc.ledger_entry(0, _unl_report_index())
node = (result or {}).get("node", {})
active = node.get("ActiveValidators", [])
if node.get("LedgerEntryType") != "UNLReport" or not active:
raise AssertionError(
"Export success scenario requires a ledger UNLReport with "
f"ActiveValidators, got: {result}"
)
log(f"UNLReport active validators: {len(active)}")
def find_export_txns(ctx, seq):
"""Find Export transactions in a ledger.
Returns list of Export transaction dicts.
"""
result = ctx.ledger(seq, transactions=True)
if not result:
return []
txns = result.get("ledger", {}).get("transactions", [])
return [tx for tx in txns if tx.get("TransactionType") == "Export"]
def find_export_signature_witness(ctx, seq, witness_hash):
"""Find the same-ledger ExportSignatures witness by transaction hash."""
result = ctx.ledger(seq, transactions=True)
txns = (result or {}).get("ledger", {}).get("transactions", [])
for tx in txns:
if not isinstance(tx, dict):
continue
if tx.get("hash") != witness_hash:
continue
if tx.get("TransactionType") != "ExportSignatures":
raise AssertionError(
f"ExportSignatureHash {witness_hash} resolved to "
f"{tx.get('TransactionType')}, not ExportSignatures"
)
return tx
raise AssertionError(
f"ExportSignatures witness {witness_hash} not found in ledger {seq}"
)
def dst_param(address):
"""Encode an address as a HookParameter entry for the DST param."""
from xrpl.core.addresscodec import decode_classic_address
dst_hex = decode_classic_address(address).hex().upper()
return {
"HookParameter": {
"HookParameterName": "445354", # "DST"
"HookParameterValue": dst_hex,
}
}
def assert_hook_accepted(
meta, log, *, expected_emits=1, expected_exports=None
):
"""Assert hook executed with ACCEPT and expected emission counts.
Checks sfHookExecutions in transaction metadata.
Returns the hook execution entry for further inspection.
"""
hook_execs = meta.get("HookExecutions", [])
if not hook_execs:
raise AssertionError("No HookExecutions in metadata")
exec_entry = hook_execs[0].get("HookExecution", {})
hook_result = exec_entry.get("HookResult", -1)
emit_count = exec_entry.get("HookEmitCount", -1)
export_count = exec_entry.get("HookExportCount")
return_code = exec_entry.get("HookReturnCode", "")
log(
f" HookResult={hook_result} EmitCount={emit_count} "
f"ExportCount={export_count} ReturnCode={return_code}"
)
# HookResult 3 = ExitType::ACCEPT
if hook_result != 3:
raise AssertionError(
f"Hook did not ACCEPT: HookResult={hook_result} "
f"ReturnCode={return_code}"
)
if emit_count != expected_emits:
raise AssertionError(
f"Expected {expected_emits} emits, got {emit_count}"
)
if expected_exports is not None and export_count != expected_exports:
raise AssertionError(
f"Expected {expected_exports} exports, got {export_count}"
)
# ReturnCode 0 = success; non-zero = ASSERT line number in hook
if return_code and str(return_code) != "0":
raise AssertionError(
f"Hook returned error code {return_code} "
f"(likely ASSERT failure at that line)"
)
return exec_entry
def _signer_entries(witness):
entries = []
for entry in witness.get("Signers", []):
signer = entry.get("Signer", entry)
entries.append(signer)
return entries
def _account_sort_key(address):
from xrpl.core.addresscodec import decode_classic_address
return decode_classic_address(address)
def assert_export_result(meta, log, *, ctx=None, require_signers=True):
"""Assert ExportResult is present and well-formed in metadata.
Returns the ExportResult dict. When signers are required, the result is
annotated with _Witness and _WitnessSigners from the same-ledger
ttEXPORT_SIGNATURES pseudo transaction.
"""
export_result = meta.get("ExportResult", {})
if not export_result:
raise AssertionError("ExportResult not found in metadata")
# Must have LedgerSequence and TransactionHash
if "LedgerSequence" not in export_result:
raise AssertionError("ExportResult missing LedgerSequence")
if "TransactionHash" not in export_result:
raise AssertionError("ExportResult missing TransactionHash")
if "ExportSignatureHash" not in export_result:
raise AssertionError("ExportResult missing ExportSignatureHash")
log(f" ExportResult: seq={export_result['LedgerSequence']} "
f"hash={export_result['TransactionHash'][:16]}... "
f"witness={export_result['ExportSignatureHash'][:16]}...")
if "ExportedTxn" in export_result:
raise AssertionError(
"ExportResult should reference ExportSignatureHash, not embed "
"ExportedTxn"
)
if require_signers:
if ctx is None:
raise AssertionError(
"assert_export_result(require_signers=True) needs ctx to "
"dereference ExportSignatureHash"
)
witness = find_export_signature_witness(
ctx,
export_result["LedgerSequence"],
export_result["ExportSignatureHash"],
)
if witness.get("TransactionHash") != export_result["TransactionHash"]:
raise AssertionError(
"ExportSignatures witness TransactionHash does not match "
"ExportResult.TransactionHash"
)
if witness.get("LedgerSequence") != export_result["LedgerSequence"]:
raise AssertionError(
"ExportSignatures witness LedgerSequence does not match "
"ExportResult.LedgerSequence"
)
signers = _signer_entries(witness)
if not signers:
raise AssertionError("ExportSignatures witness has no Signers")
accounts = [s.get("Account") for s in signers]
if accounts != sorted(accounts, key=_account_sort_key):
raise AssertionError("ExportSignatures Signers are not Account-sorted")
log(f" Witness signers: {len(signers)} validator(s)")
export_result["_Witness"] = witness
export_result["_WitnessSigners"] = signers
return export_result
def assert_shadow_ticket(ctx, account_address, log, *, expect_exists=True):
"""Assert shadow ticket exists (or doesn't) for the account."""
obj_result = ctx.rpc.request(
0, "account_objects", {"account": account_address}
)
all_objects = (obj_result or {}).get("account_objects", [])
shadow_tickets = [
obj for obj in all_objects
if obj.get("LedgerEntryType") == "ShadowTicket"
]
log(f" Shadow tickets: {len(shadow_tickets)}")
if expect_exists and not shadow_tickets:
raise AssertionError("Expected shadow ticket but none found")
if not expect_exists and shadow_tickets:
raise AssertionError(
f"Expected no shadow tickets but found {len(shadow_tickets)}"
)
return shadow_tickets

View File

@@ -0,0 +1,88 @@
""":descr: Export succeeds when quorum sidecar material exists but one active
validator withholds exportSigSetHash observation.
Node 4 has runtime_config no_export_sig_hash=true. It still attaches export
signatures, but it does not publish its exportSigSetHash in proposals. The
remaining 4/5 active validators can still align on the same export sidecar
hash, so the round must not retry/expire just because fullObservation is false.
"""
from __future__ import annotations
from export_helpers import (
EXPORT_RETRY_LEDGER_WINDOW,
require_export,
assert_export_result,
assert_shadow_ticket,
)
async def scenario(ctx, log):
await require_export(ctx, log)
await ctx.fund_accounts({"alice": 10000, "bob": 1000})
log("Accounts funded")
alice = ctx.account("alice")
bob = ctx.account("bob")
current_seq = ctx.validated_ledger_index(0)
log(f"Current ledger: {current_seq}")
log("Node 4 withholds exportSigSetHash but still attaches export signatures")
export_start = ctx.mark("export-no-veto-submit-start")
result = await ctx.submit_and_wait(
{
"TransactionType": "Export",
"LastLedgerSequence": current_seq + EXPORT_RETRY_LEDGER_WINDOW,
"Fee": "1000000",
"ExportedTxn": {
"TransactionType": "Payment",
"Account": alice.address,
"Destination": bob.address,
"Amount": "1000000",
"Fee": "10",
"Sequence": 0,
"TicketSequence": 1,
"FirstLedgerSequence": current_seq + 1,
"LastLedgerSequence": current_seq + EXPORT_RETRY_LEDGER_WINDOW,
"Flags": 2147483648,
"SigningPubKey": "",
},
},
alice.wallet,
timeout=60,
)
export_end = ctx.mark("export-no-veto-submit-end")
final_seq = ctx.validated_ledger_index(0)
engine_result = result.get("engine_result", "")
meta = result.get("meta", {})
log(f"Export completed at ledger {final_seq}, result: {engine_result}")
if engine_result != "tesSUCCESS":
raise AssertionError(f"Expected tesSUCCESS, got {engine_result}")
export_result = assert_export_result(meta, log, ctx=ctx, require_signers=True)
signers = export_result.get("_WitnessSigners", [])
if len(signers) < 4:
raise AssertionError(f"Expected at least 4 signers, got {len(signers)}")
log(f"Export signer count: {len(signers)}")
no_veto_logs = ctx.assert_log(
r"Export: missing exportSigSetHash observation ignored",
since=export_start,
until=export_end,
)
log(f"Export no-veto missing-observation logs: {no_veto_logs.count}")
withhold_logs = ctx.assert_log(
r"Export: withholding exportSigSetHash",
since=export_start,
until=export_end,
)
log(f"Export sidecar hash withholding logs: {withhold_logs.count}")
assert_shadow_ticket(ctx, alice.address, log, expect_exists=True)
log("PASS")

View File

@@ -0,0 +1,118 @@
""":descr: Test Export quorum behavior. When enough active validators sign,
the export should succeed whether or not CE is enabled. When fewer than the
active-view quorum sign, the export should expire.
Parameterized via `expect_success` kwarg from suite.yml.
Flow:
1. Fund alice and bob
2. alice submits ttEXPORT
3. Verify result matches expectation (tesSUCCESS or tecEXPORT_EXPIRED)
4. Verify ExportResult + shadow ticket on success, absence on failure
5. Verify subsequent payment works regardless
"""
from __future__ import annotations
from export_helpers import (
EXPORT_RETRY_LEDGER_WINDOW,
require_export,
assert_export_result,
assert_shadow_ticket,
)
async def scenario(ctx, log, expect_success=True):
await require_export(ctx, log)
# --- Setup ---
await ctx.fund_accounts({"alice": 10000, "bob": 1000})
log("Accounts funded")
alice = ctx.account("alice")
bob = ctx.account("bob")
current_seq = ctx.validated_ledger_index(0)
log(f"Current ledger: {current_seq}")
outcome = "success" if expect_success else "failure (below quorum)"
log(f"Expecting export {outcome}")
# --- Submit ttEXPORT ---
result = await ctx.submit_and_wait(
{
"TransactionType": "Export",
"LastLedgerSequence": current_seq + EXPORT_RETRY_LEDGER_WINDOW,
"Fee": "1000000",
"ExportedTxn": {
"TransactionType": "Payment",
"Account": alice.address,
"Destination": bob.address,
"Amount": "1000000",
"Fee": "10",
"Sequence": 0,
"TicketSequence": 1,
"FirstLedgerSequence": current_seq + 1,
"LastLedgerSequence": current_seq + EXPORT_RETRY_LEDGER_WINDOW,
"Flags": 2147483648,
"SigningPubKey": "",
},
},
alice.wallet,
timeout=60,
)
final_seq = ctx.validated_ledger_index(0)
engine_result = result.get("engine_result", "")
meta = result.get("meta", {})
log(f"Export at ledger {final_seq}, result: {engine_result}")
if expect_success:
if engine_result != "tesSUCCESS":
raise AssertionError(
f"Expected tesSUCCESS, got {engine_result}"
)
# Assert ExportResult is well-formed with signers
assert_export_result(meta, log, ctx=ctx, require_signers=True)
# Assert shadow ticket was created
assert_shadow_ticket(ctx, alice.address, log, expect_exists=True)
log("Export succeeded as expected (active-view quorum reached)")
else:
if engine_result == "tesSUCCESS":
raise AssertionError(
"Export should NOT have succeeded below active-view quorum"
)
if engine_result != "tecEXPORT_EXPIRED":
raise AssertionError(
"Expected tecEXPORT_EXPIRED below active-view quorum, "
f"got {engine_result}"
)
log(f"Export failed as expected ({engine_result})")
# No shadow ticket should exist
assert_shadow_ticket(ctx, alice.address, log, expect_exists=False)
# --- Verify subsequent payment works ---
log("Submitting payment from alice to bob...")
pay_result = await ctx.submit_and_wait(
{
"TransactionType": "Payment",
"Destination": bob.address,
"Amount": "1000000",
"Fee": "12",
},
alice.wallet,
timeout=30,
)
pay_engine = pay_result.get("engine_result", "")
log(f"Payment result: {pay_engine}")
if pay_engine != "tesSUCCESS":
raise AssertionError(f"Payment failed: {pay_engine}")
log("Payment succeeded -- account not blocked")
log("PASS")

View File

@@ -0,0 +1,96 @@
""":descr: Export retries/expires without a ledger-anchored UNLReport view.
All validators may sign, but network-mode Export must not assemble quorum
material from a node-local trusted-config view. Without UNLReport, the export
should retry until LastLedgerSequence and expire without creating a shadow
ticket.
"""
from __future__ import annotations
from export_helpers import (
EXPORT_RETRY_LEDGER_WINDOW,
require_export,
assert_shadow_ticket,
)
async def scenario(ctx, log):
await require_export(ctx, log, require_unl_report=False)
await ctx.fund_accounts({"alice": 10000, "bob": 1000})
log("Accounts funded")
alice = ctx.account("alice")
bob = ctx.account("bob")
current_seq = ctx.validated_ledger_index(0)
log(f"Current ledger: {current_seq}")
log("UNLReport intentionally absent; export must not use local config view")
export_start = ctx.mark("export-without-unlreport-submit-start")
result = await ctx.submit_and_wait(
{
"TransactionType": "Export",
"LastLedgerSequence": current_seq + EXPORT_RETRY_LEDGER_WINDOW,
"Fee": "1000000",
"ExportedTxn": {
"TransactionType": "Payment",
"Account": alice.address,
"Destination": bob.address,
"Amount": "1000000",
"Fee": "10",
"Sequence": 0,
"TicketSequence": 1,
"FirstLedgerSequence": current_seq + 1,
"LastLedgerSequence": current_seq + EXPORT_RETRY_LEDGER_WINDOW,
"Flags": 2147483648,
"SigningPubKey": "",
},
},
alice.wallet,
timeout=60,
)
export_end = ctx.mark("export-without-unlreport-submit-end")
final_seq = ctx.validated_ledger_index(0)
engine_result = result.get("engine_result", "")
log(f"Export completed at ledger {final_seq}, result: {engine_result}")
if engine_result == "tesSUCCESS":
raise AssertionError(
"Export should not succeed without a ledger-anchored UNLReport view"
)
# Be exact: without a UNLReport view the export should retry until LLS and
# expire, not fail by some unrelated terminal code.
if engine_result != "tecEXPORT_EXPIRED":
raise AssertionError(
"Expected tecEXPORT_EXPIRED without UNLReport view, "
f"got {engine_result}"
)
warning_logs = ctx.assert_log(
r"Export: retrying without ledger-anchored validator view",
since=export_start,
until=export_end,
)
log(f"Export no-UNLReport retry warnings: {warning_logs.count}")
retry_logs = ctx.assert_log(
r"Export: insufficient signatures .*result=terRETRY_EXPORT",
since=export_start,
until=export_end,
)
log(f"Export retry logs: {retry_logs.count}")
expired_logs = ctx.assert_log(
r"Export: last ledger expired .*result=tecEXPORT_EXPIRED",
since=export_start,
until=export_end,
)
log(f"Export expiry logs: {expired_logs.count}")
assert_shadow_ticket(ctx, alice.address, log, expect_exists=False)
log("PASS")

View File

@@ -0,0 +1,99 @@
""":descr: Submit ttEXPORT directly (no hook), verify it succeeds with
ExportResult in metadata. Then submit a payment from the same account
to verify sequence handling doesn't block subsequent transactions.
Flow:
1. Fund alice and bob
2. alice submits ttEXPORT with inner payment -> tesSUCCESS (provisional)
3. Validators attach sigs via proposals -> quorum -> ExportResult in metadata
4. alice submits a Payment to bob -> should succeed (sequence not blocked)
"""
from __future__ import annotations
from export_helpers import (
EXPORT_RETRY_LEDGER_WINDOW,
require_export,
assert_export_result,
assert_shadow_ticket,
)
async def scenario(ctx, log):
await require_export(ctx, log)
# --- Setup ---
await ctx.fund_accounts({"alice": 10000, "bob": 1000})
log("Accounts funded")
alice = ctx.account("alice")
bob = ctx.account("bob")
current_seq = ctx.validated_ledger_index(0)
log(f"Current ledger: {current_seq}")
# --- 1. Submit ttEXPORT ---
result = await ctx.submit_and_wait(
{
"TransactionType": "Export",
"LastLedgerSequence": current_seq + EXPORT_RETRY_LEDGER_WINDOW,
"Fee": "1000000",
"ExportedTxn": {
"TransactionType": "Payment",
"Account": alice.address,
"Destination": bob.address,
"Amount": "1000000",
"Fee": "10",
"Sequence": 0,
"TicketSequence": 1,
"FirstLedgerSequence": current_seq + 1,
"LastLedgerSequence": current_seq + EXPORT_RETRY_LEDGER_WINDOW,
"Flags": 2147483648,
"SigningPubKey": "",
},
},
alice.wallet,
timeout=60,
)
export_seq = ctx.validated_ledger_index(0)
engine_result = result.get("engine_result", "")
log(f"Export completed at ledger {export_seq}, result: {engine_result}")
if engine_result != "tesSUCCESS":
raise AssertionError(
f"Expected tesSUCCESS for export, got {engine_result}"
)
# Assert ExportResult is well-formed with signers
meta = result.get("meta", {})
assert_export_result(meta, log, ctx=ctx, require_signers=True)
# Assert shadow ticket was created
assert_shadow_ticket(ctx, alice.address, log, expect_exists=True)
# --- 2. Submit Payment from same account ---
log("Submitting payment from alice to bob...")
pay_result = await ctx.submit_and_wait(
{
"TransactionType": "Payment",
"Destination": bob.address,
"Amount": "1000000",
"Fee": "12",
},
alice.wallet,
timeout=30,
)
pay_engine = pay_result.get("engine_result", "")
log(f"Payment result: {pay_engine}")
if pay_engine != "tesSUCCESS":
raise AssertionError(f"Payment failed: {pay_engine}")
log(
f"Both transactions succeeded: "
f"Export at ledger {export_seq}, Payment at ledger {ctx.validated_ledger_index(0)}"
)
log("Sequence handling OK - export didn't block subsequent txns")
log("PASS")

View File

@@ -0,0 +1,212 @@
""":descr: install xport hook, trigger export, verify emitted ttEXPORT lifecycle
1. Fund alice (hook holder), bob (trigger), carol (export destination)
2. Install xport hook on alice
3. bob pays alice with DST=carol → hook calls xport() → emits ttEXPORT
4. Emitted ttEXPORT enters open ledger, validators attach sigs via proposals
5. Verify Export transaction appears in a subsequent ledger
"""
from __future__ import annotations
from export_helpers import (
require_export,
find_export_txns,
dst_param,
assert_hook_accepted,
assert_export_result,
assert_shadow_ticket,
)
# C source for the xport hook — verbatim from src/test/app/Export_test_hooks.h
# On Payment to the hook account, exports a 1 XAH payment to the DST param.
XPORT_HOOK_C = r"""
#include <stdint.h>
extern int32_t _g(uint32_t id, uint32_t maxiter);
extern int64_t accept(uint32_t read_ptr, uint32_t read_len, int64_t error_code);
extern int64_t rollback(uint32_t read_ptr, uint32_t read_len, int64_t error_code);
extern int64_t xport(uint32_t write_ptr, uint32_t write_len, uint32_t read_ptr, uint32_t read_len);
extern int64_t xport_reserve(uint32_t count);
extern int64_t hook_account(uint32_t write_ptr, uint32_t write_len);
extern int64_t otxn_param(uint32_t write_ptr, uint32_t write_len, uint32_t name_ptr, uint32_t name_len);
extern int64_t otxn_type(void);
extern int64_t ledger_seq(void);
#define SBUF(x) (uint32_t)(x), sizeof(x)
#define ASSERT(x) if (!(x)) rollback((uint32_t)#x, sizeof(#x), __LINE__)
#define ttPAYMENT 0
#define tfCANONICAL 0x80000000UL
#define amAMOUNT 1
#define amFEE 8
#define atACCOUNT 1
#define atDESTINATION 3
#define ENCODE_TT(buf_out, tt) \
buf_out[0] = 0x12U; buf_out[1] = (tt >> 8) & 0xFFU; buf_out[2] = tt & 0xFFU; buf_out += 3;
#define ENCODE_FLAGS(buf_out, flags) \
buf_out[0] = 0x22U; buf_out[1] = (flags >> 24) & 0xFFU; buf_out[2] = (flags >> 16) & 0xFFU; \
buf_out[3] = (flags >> 8) & 0xFFU; buf_out[4] = flags & 0xFFU; buf_out += 5;
#define ENCODE_SEQUENCE(buf_out, seq) \
buf_out[0] = 0x24U; buf_out[1] = (seq >> 24) & 0xFFU; buf_out[2] = (seq >> 16) & 0xFFU; \
buf_out[3] = (seq >> 8) & 0xFFU; buf_out[4] = seq & 0xFFU; buf_out += 5;
#define ENCODE_FLS(buf_out, fls) \
buf_out[0] = 0x20U; buf_out[1] = 0x1AU; buf_out[2] = (fls >> 24) & 0xFFU; \
buf_out[3] = (fls >> 16) & 0xFFU; buf_out[4] = (fls >> 8) & 0xFFU; \
buf_out[5] = fls & 0xFFU; buf_out += 6;
#define ENCODE_LLS(buf_out, lls) \
buf_out[0] = 0x20U; buf_out[1] = 0x1BU; buf_out[2] = (lls >> 24) & 0xFFU; \
buf_out[3] = (lls >> 16) & 0xFFU; buf_out[4] = (lls >> 8) & 0xFFU; \
buf_out[5] = lls & 0xFFU; buf_out += 6;
#define ENCODE_DROPS(buf_out, drops, amt_type) \
buf_out[0] = 0x60U + amt_type; buf_out[1] = 0x40U + ((drops >> 56) & 0x3FU); \
buf_out[2] = (drops >> 48) & 0xFFU; buf_out[3] = (drops >> 40) & 0xFFU; \
buf_out[4] = (drops >> 32) & 0xFFU; buf_out[5] = (drops >> 24) & 0xFFU; \
buf_out[6] = (drops >> 16) & 0xFFU; buf_out[7] = (drops >> 8) & 0xFFU; \
buf_out[8] = drops & 0xFFU; buf_out += 9;
#define ENCODE_SIGNING_PUBKEY_EMPTY(buf_out) \
buf_out[0] = 0x73U; buf_out[1] = 0x00U; buf_out += 2;
#define ENCODE_ACCOUNT(buf_out, acc, acc_type) \
buf_out[0] = 0x80U + acc_type; buf_out[1] = 0x14U; \
for (int i = 0; i < 20; ++i) buf_out[2+i] = acc[i]; buf_out += 22;
#define PREPARE_PAYMENT_SIMPLE_SIZE 270U
int64_t hook(uint32_t reserved) {
_g(1, 1);
if (otxn_type() != ttPAYMENT)
return accept(0, 0, 0);
ASSERT(xport_reserve(1) == 1);
uint8_t dst[20];
int64_t dst_len = otxn_param(SBUF(dst), "DST", 3);
ASSERT(dst_len == 20);
uint8_t acc[20];
ASSERT(hook_account(SBUF(acc)) == 20);
uint32_t cls = (uint32_t)ledger_seq();
uint8_t tx[PREPARE_PAYMENT_SIMPLE_SIZE];
uint8_t* buf = tx;
ENCODE_TT(buf, ttPAYMENT);
ENCODE_FLAGS(buf, tfCANONICAL);
ENCODE_SEQUENCE(buf, 0);
ENCODE_FLS(buf, cls + 1);
ENCODE_LLS(buf, cls + 5);
// sfTicketSequence = UINT32 field 41 = 0x20 0x29
buf[0] = 0x20U; buf[1] = 0x29U;
buf[2] = 0; buf[3] = 0; buf[4] = 0; buf[5] = 1;
buf += 6;
uint64_t drops = 1000000;
ENCODE_DROPS(buf, drops, amAMOUNT);
ENCODE_DROPS(buf, 10, amFEE);
ENCODE_SIGNING_PUBKEY_EMPTY(buf);
ENCODE_ACCOUNT(buf, acc, atACCOUNT);
ENCODE_ACCOUNT(buf, dst, atDESTINATION);
uint8_t hash[32];
int64_t xport_result = xport(SBUF(hash), (uint32_t)tx, buf - tx);
ASSERT(xport_result == 32);
return accept(0, 0, 0);
}
"""
async def scenario(ctx, log):
# Wait for network to start and amendments to activate
await require_export(ctx, log)
# --- Setup ---
await ctx.fund_accounts({"alice": 10000, "bob": 10000, "carol": 1000})
log("Accounts funded")
alice = ctx.account("alice")
carol = ctx.account("carol")
# Compile and install xport hook on alice
wasm = ctx.compile_hook(XPORT_HOOK_C, label="xport")
await ctx.submit_and_wait(
{
"TransactionType": "SetHook",
"Hooks": [
{
"Hook": {
"CreateCode": wasm.hex().upper(),
"HookOn": "0" * 64,
"HookNamespace": "0" * 64,
"HookApiVersion": 0,
"Flags": 1, # hsfOVERRIDE
}
}
],
"Fee": "100000000",
},
alice.wallet,
)
log(
f"Hook installed on alice ({alice.address[:12]}...) "
f"ledger {ctx.validated_ledger_index(0)}"
)
# --- Trigger ---
# bob pays alice → hook calls xport() → emits ttEXPORT
trigger_result = await ctx.submit_and_wait(
{
"TransactionType": "Payment",
"Destination": alice.address,
"Amount": "100000000",
"Fee": "1000000",
"HookParameters": [dst_param(carol.address)],
},
ctx.account("bob").wallet,
)
trigger_seq = ctx.validated_ledger_index(0)
log(f"Export triggered at ledger {trigger_seq}")
# xport() schedules a ttEXPORT through the emitted directory, but hook
# metadata reports it separately from ordinary HookEmissions.
trigger_meta = trigger_result.get("meta", {})
assert_hook_accepted(trigger_meta, log, expected_emits=0, expected_exports=1)
# --- Verify: check each ledger close for the Export transaction ---
max_ledgers = 10
for i in range(max_ledgers):
await ctx.wait_for_ledgers(1, node_id=0, timeout=30)
seq = ctx.validated_ledger_index(0)
exports = find_export_txns(ctx, seq)
if exports:
export_tx = exports[0]
meta = export_tx.get("meta", export_tx.get("metaData", {}))
result = meta.get("TransactionResult", "")
log(f"Ledger {seq}: Export txn found, result={result}")
if result != "tesSUCCESS":
raise AssertionError(f"Export did not succeed: {result}")
# Assert ExportResult is well-formed with signers and inner tx
assert_export_result(meta, log, ctx=ctx, require_signers=True)
# Assert shadow ticket was created
assert_shadow_ticket(ctx, alice.address, log, expect_exists=True)
log("PASS")
return
log(f"Ledger {seq}: no Export txn yet")
raise AssertionError(
f"No Export transaction found after {max_ledgers} ledger closes"
)

View File

@@ -0,0 +1,180 @@
"""Shared helpers for ConsensusEntropy scenario tests."""
from __future__ import annotations
from xahaud_scripts.testnet.config import feature_name_to_hash
ZERO_DIGEST = "0" * 64
CONSENSUS_ENTROPY_FEATURE = feature_name_to_hash("ConsensusEntropy")
def feature_hash(name: str) -> str:
"""Return the amendment hash accepted by feature RPC."""
return feature_name_to_hash(name)
def feature_status(ctx, name: str, node_id=0):
"""Query a feature by amendment hash; feature RPC names are ambiguous."""
return ctx.feature_check(feature_hash(name), node_id=node_id)
def consensus_entropy_feature(ctx, node_id=0):
"""Query ConsensusEntropy by amendment hash."""
return feature_status(ctx, "ConsensusEntropy", node_id=node_id)
async def require_entropy(ctx, log):
"""Wait for first ledger and assert ConsensusEntropy is enabled."""
await ctx.wait_for_ledger_close(timeout=120)
feature = consensus_entropy_feature(ctx, node_id=0)
if not feature or not feature.get("enabled", False):
raise AssertionError(f"ConsensusEntropy not enabled: {feature}")
log("ConsensusEntropy enabled")
def get_entropy_tx(ctx, seq):
"""Fetch ledger and return (ce_tx, user_txns) or raise."""
result = ctx.ledger(seq, transactions=True)
if not result:
raise AssertionError(f"Ledger {seq}: fetch failed")
ledger = result.get("ledger")
if not isinstance(ledger, dict):
raise AssertionError(f"Ledger {seq}: fetch returned no ledger: {result}")
txns = ledger.get("transactions", [])
ce = [tx for tx in txns if tx.get("TransactionType") == "ConsensusEntropy"]
user = [tx for tx in txns if tx.get("TransactionType") != "ConsensusEntropy"]
if len(ce) != 1:
raise AssertionError(
f"Ledger {seq}: expected 1 ConsensusEntropy txn, got {len(ce)}"
)
return ce[0], user
def entropy_fields(ce_tx):
"""Return (digest, entropy_count, is_fallback) from a ConsensusEntropy tx.
consensus_fallback rounds carry a deterministic non-zero consensus-bound
digest with EntropyCount=0 and EntropyTier=1 (consensus_fallback).
Validator entropy has EntropyTier=3 (validator_quorum).
WARNING: is_fallback is ``tier != 3``, so it lumps participant_aligned
(Tier 2) in with fallback. It is only safe where no Tier 2 band exists
(e.g. 5-node networks, where tier2 == quorum). For band-aware scenarios use
the explicit assert_consensus_fallback / assert_participant_aligned /
assert_validator_quorum helpers, which check EntropyTier directly.
"""
digest = ce_tx.get("Digest", "")
entropy_count = ce_tx.get("EntropyCount", -1)
tier = ce_tx.get("EntropyTier", None)
if tier is not None:
is_fallback = tier != 3
else:
is_fallback = entropy_count == 0
return digest, entropy_count, is_fallback
def assert_participant_aligned(ce_tx, seq, expected_count=None):
"""Assert participant_aligned (Tier 2) entropy on a ConsensusEntropy tx.
Tier 2 is the sub-quorum band: the agreed reveal cohort is >= the
participant floor but < the 80% validator quorum, so it carries
EntropyTier=2 with a deterministic non-zero digest. NOTE entropy_fields()'s
is_fallback lumps tier 2 in with fallback (is_fallback = tier != 3), so the
tier must be checked EXPLICITLY here.
"""
digest = ce_tx.get("Digest", "")
count = ce_tx.get("EntropyCount", -1)
tier = ce_tx.get("EntropyTier", None)
if tier != 2:
raise AssertionError(
f"Ledger {seq}: expected EntropyTier==2 (participant_aligned), "
f"got {tier} (EntropyCount={count})"
)
if not digest or digest == ZERO_DIGEST:
raise AssertionError(
f"Ledger {seq}: participant_aligned digest must be non-zero, got "
f"{digest[:16]}..."
)
if expected_count is not None and count != expected_count:
raise AssertionError(
f"Ledger {seq}: participant_aligned EntropyCount must be "
f"{expected_count} (the surviving cohort), got {count}"
)
return digest, count
def assert_validator_quorum(ce_tx, seq, min_count=None):
"""Assert validator_quorum (Tier 3) entropy on a ConsensusEntropy tx:
EntropyTier=3, a deterministic non-zero digest, and (optionally)
EntropyCount >= min_count (the active quorum). The count can EXCEED the
quorum (e.g. a still-full 6/6 ledger caught at a 6->5 transition), so check
>=, not ==.
"""
digest = ce_tx.get("Digest", "")
count = ce_tx.get("EntropyCount", -1)
tier = ce_tx.get("EntropyTier", None)
if tier != 3:
raise AssertionError(
f"Ledger {seq}: expected EntropyTier==3 (validator_quorum), got "
f"{tier} (EntropyCount={count})"
)
if not digest or digest == ZERO_DIGEST:
raise AssertionError(
f"Ledger {seq}: validator_quorum digest must be non-zero, got "
f"{digest[:16]}..."
)
if min_count is not None and count < min_count:
raise AssertionError(
f"Ledger {seq}: validator_quorum EntropyCount={count} < quorum "
f"{min_count}"
)
return digest, count
def assert_consensus_fallback(ce_tx, seq):
"""Assert consensus_fallback (Tier 1) entropy on a ConsensusEntropy tx:
EntropyTier=1, EntropyCount=0, and a deterministic NON-zero digest.
"""
digest = ce_tx.get("Digest", "")
count = ce_tx.get("EntropyCount", -1)
tier = ce_tx.get("EntropyTier", None)
if tier != 1:
raise AssertionError(
f"Ledger {seq}: expected EntropyTier==1 (consensus_fallback), got "
f"{tier} (EntropyCount={count})"
)
if count != 0:
raise AssertionError(
f"Ledger {seq}: consensus_fallback EntropyCount must be 0, got "
f"{count}"
)
if not digest or digest == ZERO_DIGEST:
raise AssertionError(
f"Ledger {seq}: consensus_fallback digest must be non-zero, got "
f"{digest[:16]}..."
)
return digest, count
def assert_valid_entropy(ce_tx, seq, seen_digests=None):
"""Assert quorum-met validator entropy. Optionally check uniqueness."""
digest, entropy_count, is_fallback = entropy_fields(ce_tx)
if is_fallback or not digest or digest == ZERO_DIGEST:
raise AssertionError(f"Ledger {seq}: fallback/empty Digest")
if entropy_count < 4:
raise AssertionError(
f"Ledger {seq}: EntropyCount={entropy_count} < 4 (sub-quorum)"
)
if seen_digests is not None:
if digest in seen_digests:
raise AssertionError(f"Ledger {seq}: duplicate Digest {digest[:16]}...")
seen_digests.add(digest)
return digest, entropy_count

View File

@@ -0,0 +1,92 @@
defaults:
network:
node_count: 5
launcher: tmux
find_ports: true
slave_delay: 0.2
features:
- ConsensusEntropy
- Export
track_features:
- ConsensusEntropy
- Export
unl_report: true
log_levels:
TxQ: info
Protocol: debug
Peer: debug
LedgerConsensus: debug
ConsensusExtensions: debug
NetworkOPs: info
env:
XAHAU_RESOURCE_PER_PORT: "1"
rc:
- rng_poll_ms=250
tests:
- name: latency_baseline_ce
script: .testnet/scenarios/perf/ce_export_latency_probe.py
params:
warmup_ledgers: 3
ledgers: 8
submit_export: false
- name: latency_baseline_export
script: .testnet/scenarios/perf/ce_export_latency_probe.py
params:
warmup_ledgers: 3
ledgers: 8
submit_export: true
- name: latency_proposal_delay_export
script: .testnet/scenarios/perf/ce_export_latency_probe.py
params:
warmup_ledgers: 3
ledgers: 8
submit_export: true
network:
rc:
- rng_poll_ms=250
- delay=100,jitter=25,msg=proposal
- name: latency_directed_pair_delay_export
script: .testnet/scenarios/perf/ce_export_latency_probe.py
params:
warmup_ledgers: 3
ledgers: 8
submit_export: true
network:
rc:
- rng_poll_ms=250
- n0->n2:delay=750,jitter=100,msg=proposal
- n2->n0:delay=750,jitter=100,msg=proposal
- name: latency_slow_minority_export
script: .testnet/scenarios/perf/ce_export_latency_probe.py
params:
warmup_ledgers: 3
ledgers: 8
submit_export: true
export_timeout: 120
network:
rc:
- rng_poll_ms=250
- n3->n0:delay=500,jitter=100,msg=proposal
- n3->n1:delay=500,jitter=100,msg=proposal
- n3->n2:delay=500,jitter=100,msg=proposal
- n4->n0:delay=500,jitter=100,msg=proposal
- n4->n1:delay=500,jitter=100,msg=proposal
- n4->n2:delay=500,jitter=100,msg=proposal
- n0->n3:delay=500,jitter=100,msg=proposal
- n1->n3:delay=500,jitter=100,msg=proposal
- n2->n3:delay=500,jitter=100,msg=proposal
- n0->n4:delay=500,jitter=100,msg=proposal
- n1->n4:delay=500,jitter=100,msg=proposal
- n2->n4:delay=500,jitter=100,msg=proposal
- name: latency_export_no_veto_with_delay
script: .testnet/scenarios/export/export_no_veto_missing_observation.py
network:
rc:
- rng_poll_ms=250
- delay=300,jitter=100,msg=proposal
- n4:no_export_sig_hash=true

View File

@@ -0,0 +1,196 @@
""":descr: measure CE/export behavior while RuntimeConfig injects latency/drop.
The suite supplies runtime fault injection through network.rc. This scenario
does not mutate RuntimeConfig itself; it observes what the launched network does
under that condition and logs enough counters to compare variants.
"""
from __future__ import annotations
from collections import Counter
import json
from export.export_helpers import assert_export_result, require_export
from helpers import consensus_entropy_feature, get_entropy_tx
async def _require_runtime_config(ctx, log):
result = ctx.rpc.runtime_config(0)
if not result or result.get("error"):
raise AssertionError(
"Latency probe requires a binary built with "
"xahaud_runtime_test_config=ON; runtime_config RPC returned "
f"{result}"
)
log("RuntimeConfig RPC active")
async def _require_consensus_entropy(ctx, log):
feature = consensus_entropy_feature(ctx, node_id=0)
if not feature or not feature.get("enabled", False):
raise AssertionError(f"ConsensusEntropy not enabled: {feature}")
log("ConsensusEntropy enabled")
def _log_runtime_config(ctx, log):
for node_id in range(ctx.node_count):
cfg = ctx.rpc.runtime_config(node_id)
if cfg is None:
raise AssertionError(f"runtime_config RPC failed on node {node_id}")
log(
f"runtime_config n{node_id}: "
f"{json.dumps(cfg, sort_keys=True, separators=(',', ':'))}"
)
async def _submit_direct_export(ctx, log, *, timeout):
await ctx.fund_accounts({"alice": 10000, "bob": 1000})
alice = ctx.account("alice")
bob = ctx.account("bob")
current_seq = ctx.validated_ledger_index(0)
if current_seq is None:
raise AssertionError("validated ledger is not available before Export")
log(f"Submitting direct Export at validated ledger {current_seq}")
started = ctx.mark("latency-export-submit-start")
result = await ctx.submit_and_wait(
{
"TransactionType": "Export",
"LastLedgerSequence": current_seq + 12,
"Fee": "1000000",
"ExportedTxn": {
"TransactionType": "Payment",
"Account": alice.address,
"Destination": bob.address,
"Amount": "1000000",
"Fee": "10",
"Sequence": 0,
"TicketSequence": 1,
"FirstLedgerSequence": current_seq + 1,
"LastLedgerSequence": current_seq + 10,
"Flags": 2147483648,
"SigningPubKey": "",
},
},
alice.wallet,
timeout=timeout,
)
ended = ctx.mark("latency-export-submit-end")
elapsed = (ended.monotonic_ns - started.monotonic_ns) / 1_000_000_000
engine_result = result.get("engine_result", "")
log(f"Export result={engine_result} elapsed={elapsed:.3f}s")
if engine_result != "tesSUCCESS":
raise AssertionError(f"Expected Export tesSUCCESS, got {engine_result}")
export_result = assert_export_result(result.get("meta", {}), log)
signers = export_result.get("ExportedTxn", {}).get("Signers", [])
log(f"Export signer count={len(signers)}")
return started, ended
def _summarize_logs(ctx, log, *, label, started, ended):
patterns = {
"rng_selected": r"RNG: entropy selected",
"rng_fallback": r"tier=1",
"rng_participant_aligned": r"tier=2",
"rng_validator_quorum": r"tier=3",
"export_retry": r"terRETRY_EXPORT",
"export_quorum_timeout": r"Export: exportSigSet quorum alignment timeout",
"export_missing_observation_ignored": (
r"Export: missing exportSigSetHash observation ignored"
),
}
for name, pattern in patterns.items():
result = ctx.search_logs(pattern, since=started, until=ended, limit=500)
log(f"log_count {label}.{name}={result.count}")
async def scenario(
ctx,
log,
*,
warmup_ledgers=3,
ledgers=8,
submit_export=False,
export_timeout=90,
):
await ctx.wait_for_ledger_close(timeout=120)
await _require_runtime_config(ctx, log)
_log_runtime_config(ctx, log)
await _require_consensus_entropy(ctx, log)
if submit_export:
# require_export also asserts the UNLReport precondition for successful
# network-mode Export. Keep that explicit in perf runs so a missing
# report does not masquerade as a latency failure.
await require_export(ctx, log, require_runtime_config=False)
await ctx.wait_for_ledgers(warmup_ledgers, node_id=0, timeout=120)
warm_seq = ctx.validated_ledger_index(0)
log(f"Warmup complete at validated ledger {warm_seq}")
export_window = None
if submit_export:
export_window = await _submit_direct_export(
ctx, log, timeout=export_timeout
)
started = ctx.mark("latency-probe-start")
start_seq = ctx.validated_ledger_index(0)
await ctx.wait_for_ledgers(ledgers, node_id=0, timeout=max(120, ledgers * 30))
ended = ctx.mark("latency-probe-end")
end_seq = ctx.validated_ledger_index(0)
if start_seq is None or end_seq is None:
raise AssertionError("validated ledger index unavailable during probe")
elapsed = (ended.monotonic_ns - started.monotonic_ns) / 1_000_000_000
closed = max(0, end_seq - start_seq)
cadence = elapsed / closed if closed else 0.0
log(
f"Observed validated ledgers {start_seq + 1}..{end_seq} "
f"closed={closed} elapsed={elapsed:.3f}s cadence={cadence:.3f}s/ledger"
)
tiers: Counter[int] = Counter()
counts: Counter[int] = Counter()
missing_entropy = 0
for seq in range(start_seq + 1, end_seq + 1):
try:
ce, user_txns = get_entropy_tx(ctx, seq)
except AssertionError as exc:
missing_entropy += 1
log(f" Ledger {seq}: no ConsensusEntropy tx ({exc})")
continue
tier = ce.get("EntropyTier", -1)
count = ce.get("EntropyCount", -1)
tiers[tier] += 1
counts[count] += 1
log(
f" Ledger {seq}: tier={tier} count={count} "
f"user_txns={len(user_txns)} digest={ce.get('Digest', '')[:16]}..."
)
log(
"SUMMARY "
f"closed={closed} elapsed_s={elapsed:.3f} cadence_s={cadence:.3f} "
f"tiers={dict(sorted(tiers.items()))} "
f"counts={dict(sorted(counts.items()))} "
f"missing_entropy={missing_entropy}"
)
_summarize_logs(ctx, log, label="probe", started=started, ended=ended)
if export_window is not None:
_summarize_logs(
ctx,
log,
label="export",
started=export_window[0],
ended=export_window[1],
)
log("PASS")

View File

@@ -0,0 +1,62 @@
defaults:
network:
node_count: 5
launcher: tmux
find_ports: true
slave_delay: 0.2
features:
- ConsensusEntropy
track_features:
- ConsensusEntropy
unl_report: true
log_levels:
TxQ: info
Protocol: debug
Peer: debug
LedgerConsensus: debug
ConsensusExtensions: debug
NetworkOPs: info
env:
XAHAU_RESOURCE_PER_PORT: "1"
rc:
- rng_poll_ms=333
tests:
- name: steady_state_entropy
script: .testnet/scenarios/entropy/steady_state_entropy.py
- name: fallback_without_unl_report
script: .testnet/scenarios/entropy/fallback_without_unl_report.py
network:
unl_report: false
- name: steady_state_entropy_fast_start
script: .testnet/scenarios/entropy/steady_state_entropy.py
network:
env:
XAHAUD_RUNTIME_TEST_CONFIG: '{"set":{"global":{"rng_poll_ms":333,"bootstrap_fast_start":true}}}'
- name: entropy_with_transactions
script: .testnet/scenarios/entropy/entropy_with_transactions.py
- name: quorum_recovery_smoke
script: .testnet/scenarios/entropy/quorum_recovery_smoke.py
- name: quorum_degradation_smoke
script: .testnet/scenarios/entropy/quorum_degradation_smoke.py
network:
log_levels:
LedgerConsensus: trace
ConsensusExtensions: trace
# Tier 2 (participant_aligned) needs 6 nodes: n=5 has no band (tier2 ==
# quorum). At 6, the 4/6 window is the participant_aligned band.
- name: participant_aligned_smoke
script: .testnet/scenarios/entropy/participant_aligned_smoke.py
network:
node_count: 6
log_levels:
LedgerConsensus: trace
ConsensusExtensions: trace
# Export scenarios: see export-suite.yml

View File

@@ -26,7 +26,7 @@ Loop: xrpld.app xrpld.nodestore
xrpld.app > xrpld.nodestore
Loop: xrpld.app xrpld.overlay
xrpld.overlay ~= xrpld.app
xrpld.overlay == xrpld.app
Loop: xrpld.app xrpld.peerfinder
xrpld.app > xrpld.peerfinder

View File

@@ -12,6 +12,7 @@ libxrpl.server > xrpl.basics
libxrpl.server > xrpl.json
libxrpl.server > xrpl.protocol
libxrpl.server > xrpl.server
test.app > test.shamap
test.app > test.toplevel
test.app > test.unit_test
test.app > xrpl.basics
@@ -21,6 +22,7 @@ test.app > xrpld.ledger
test.app > xrpld.nodestore
test.app > xrpld.overlay
test.app > xrpld.rpc
test.app > xrpld.shamap
test.app > xrpl.hook
test.app > xrpl.json
test.app > xrpl.protocol
@@ -43,6 +45,9 @@ test.consensus > xrpld.app
test.consensus > xrpld.consensus
test.consensus > xrpld.core
test.consensus > xrpld.ledger
test.consensus > xrpld.overlay
test.consensus > xrpld.shamap
test.consensus > xrpl.json
test.consensus > xrpl.protocol
test.core > test.jtx
test.core > test.toplevel
@@ -84,6 +89,7 @@ test.nodestore > xrpl.basics
test.nodestore > xrpld.core
test.nodestore > xrpld.nodestore
test.nodestore > xrpld.unity
test.nodestore > xrpl.protocol
test.overlay > test.jtx
test.overlay > test.toplevel
test.overlay > test.unit_test
@@ -118,6 +124,7 @@ test.rpc > xrpld.core
test.rpc > xrpld.net
test.rpc > xrpld.overlay
test.rpc > xrpld.rpc
test.rpc > xrpld.shamap
test.rpc > xrpl.hook
test.rpc > xrpl.json
test.rpc > xrpl.protocol
@@ -150,7 +157,6 @@ xrpl.resource > xrpl.protocol
xrpl.server > xrpl.basics
xrpl.server > xrpl.json
xrpl.server > xrpl.protocol
xrpld.app > quickjs.toplevel
xrpld.app > test.unit_test
xrpld.app > xrpl.basics
xrpld.app > xrpld.conditions

View File

@@ -145,8 +145,6 @@ target_link_libraries(ripple_libs INTERFACE
SQLite::SQLite3
)
include(deps/quickjs)
if(coverage)
include(RippledCov)
endif()

View File

@@ -2,7 +2,7 @@
**Note:** Throughout this README, references to "we" or "our" pertain to the community and contributors involved in the Xahau network. It does not imply a legal entity or a specific collection of individuals.
[Xahau](https://xahau.network/) is a decentralized cryptographic ledger that builds upon the robust foundation of the XRP Ledger. It inherits the XRP Ledger's Byzantine Fault Tolerant consensus algorithm and enhances it with additional features and functionalities. Developers and users familiar with the XRP Ledger will find that most documentation and tutorials available on [xrpl.org](https://xrpl.org) are relevant and applicable to Xahau, including those related to running validators and managing validator keys. For Xahau specific documentation you can visit our [documentation](https://xahau.network/)
[Xahau](https://xahau.network/) is a decentralized cryptographic ledger that builds upon the robust foundation of the XRP Ledger. It inherits the XRP Ledger's Byzantine Fault Tolerant consensus algorithm under the normal XRPL assumptions about configured validator-list overlap, timing, and fault bounds, and enhances it with additional features and functionalities. Developers and users familiar with the XRP Ledger will find that most documentation and tutorials available on [xrpl.org](https://xrpl.org) are relevant and applicable to Xahau, including those related to running validators and managing validator keys. For Xahau specific documentation you can visit our [documentation](https://xahau.network/)
## XAH
XAH is the public, counterparty-free asset native to Xahau and functions primarily as network gas. Transactions submitted to the Xahau network must supply an appropriate amount of XAH, to be burnt by the network as a fee, in order to be successfully included in a validated ledger. In addition, XAH also acts as a bridge currency within the Xahau DEX. XAH is traded on the open-market and is available for anyone to access. Xahau was created in 2023 with a supply of 600 million units of XAH.

View File

@@ -95,8 +95,16 @@ if [[ "$4" == "" ]]; then
echo "Non GH, local building, no Action runner magic"
else
# GH Action, runner
cp /io/release-build/xahaud /data/builds/$(date +%Y).$(date +%-m).$(date +%-d)-$(git rev-parse --abbrev-ref HEAD)+$4
cp /io/release-build/release.info /data/builds/$(date +%Y).$(date +%-m).$(date +%-d)-$(git rev-parse --abbrev-ref HEAD)+$4.releaseinfo
if [[ "$(git rev-parse --abbrev-ref HEAD)" == "release" ]]; then
echo "building on the release branch... placing it in builds/candidate"
mkdir /data/builds/candidate
cp /io/release-build/xahaud /data/builds/candidate/$(date +%Y).$(date +%-m).$(date +%-d)-$(git rev-parse --abbrev-ref HEAD)+$4
cp /io/release-build/release.info /data/builds/candidate/$(date +%Y).$(date +%-m).$(date +%-d)-$(git rev-parse --abbrev-ref HEAD)+$4.releaseinfo
else
echo "building non-release branch, placing it in builds root"
cp /io/release-build/xahaud /data/builds/$(date +%Y).$(date +%-m).$(date +%-d)-$(git rev-parse --abbrev-ref HEAD)+$4
cp /io/release-build/release.info /data/builds/$(date +%Y).$(date +%-m).$(date +%-d)-$(git rev-parse --abbrev-ref HEAD)+$4.releaseinfo
fi
echo "Published build to: http://build.xahau.tech/"
echo $(date +%Y).$(date +%-m).$(date +%-d)-$(git rev-parse --abbrev-ref HEAD)+$4
fi

View File

@@ -12,17 +12,16 @@ echo "-- GITHUB_REPOSITORY: $1"
echo "-- GITHUB_SHA: $2"
echo "-- GITHUB_RUN_NUMBER: $4"
umask 0000;
umask 0000
####
cd /io;
mkdir -p src/certs;
curl --silent -k https://raw.githubusercontent.com/RichardAH/rippled-release-builder/main/ca-bundle/certbundle.h -o src/certs/certbundle.h;
if [ "`grep certbundle.h src/xrpld/net/detail/RegisterSSLCerts.cpp | wc -l`" -eq "0" ]
then
cp src/xrpld/net/detail/RegisterSSLCerts.cpp src/xrpld/net/detail/RegisterSSLCerts.cpp.old
perl -i -pe "s/^{/{
cd /io
mkdir -p src/certs
curl --silent -k https://raw.githubusercontent.com/RichardAH/rippled-release-builder/main/ca-bundle/certbundle.h -o src/certs/certbundle.h
if [ "$(grep certbundle.h src/xrpld/net/detail/RegisterSSLCerts.cpp | wc -l)" -eq "0" ]; then
cp src/xrpld/net/detail/RegisterSSLCerts.cpp src/xrpld/net/detail/RegisterSSLCerts.cpp.old
perl -i -pe "s/^{/{
#ifdef EMBEDDED_CA_BUNDLE
BIO *cbio = BIO_new_mem_buf(ca_bundle.data(), ca_bundle.size());
X509_STORE *cts = SSL_CTX_get_cert_store(ctx.native_handle());
@@ -68,15 +67,14 @@ fi
source /opt/rh/gcc-toolset-11/enable
export PATH=/usr/local/bin:$PATH
export CC='/usr/lib64/ccache/gcc' &&
export CXX='/usr/lib64/ccache/g++' &&
echo "-- Build Rippled --" &&
pwd &&
export CXX='/usr/lib64/ccache/g++' &&
echo "-- Build Rippled --" &&
pwd &&
echo "MOVING TO [ build-core.sh ]"
echo "MOVING TO [ build-core.sh ]";
printenv > .env.temp;
cat .env.temp | grep '=' | sed s/\\\(^[^=]\\+=\\\)/\\1\\\"/g|sed s/\$/\\\"/g > .env;
rm .env.temp;
printenv >.env.temp
cat .env.temp | grep '=' | sed s/\\\(^[^=]\\+=\\\)/\\1\\\"/g | sed s/\$/\\\"/g >.env
rm .env.temp
echo "Persisting ENV:"
cat .env

View File

@@ -62,8 +62,7 @@
"3C43D9A973AA4443EF3FC38E42DD306160FBFFDAB901CD8BAA15D09F2597EB87",
"0285B7E5E08E1A8E4C15636F0591D87F73CB6A7B6452A932AD72BBC8E5D1CBE3",
"6E739F4F8B07BED29FC9FF440DA3C301CD14A180DF45819F658FEC2F7DE31427",
"36799EA497B1369B170805C078AEFE6188345F9B3E324C21E9CA3FF574E3C3D6",
"DD4F86291F142A20761B32B4D0CE4291F86CA33F0B46F0D04171482FBA52E536"
"36799EA497B1369B170805C078AEFE6188345F9B3E324C21E9CA3FF574E3C3D6"
],
"Flags": 0,
"LedgerEntryType": "Amendments",

View File

@@ -1,4 +1,4 @@
# standalone: ./xahaud -a --ledgerfile cfg/genesis.json --conf cfg/xahaud-standalone.cfg
# standalone: ./xahaud -a --ledgerfile config/genesis.json --conf config/xahaud-standalone.cfg
[server]
port_rpc_admin_local
port_ws_public
@@ -154,4 +154,3 @@ D686F2538F410C9D0D856788E98E3579595DAF7B38D38887F81ECAC934B06040 HooksUpdate1
EDB4EE4C524E16BDD91D9A529332DED08DCAAA51CC6DC897ACFA1A0ED131C5B6 fix240819
8063140E9260799D6716756B891CEC3E7006C4E4F277AB84670663A88F94B9C4 fixPageCap
88693F108C3CD8A967F3F4253A32DEF5E35F9406ACD2A11B88B11D90865763A9 fix240911
DD4F86291F142A20761B32B4D0CE4291F86CA33F0B46F0D04171482FBA52E536 JsHooks

View File

@@ -95,6 +95,9 @@
# - replace both functions setup_target_for_coverage_gcovr_* with a single setup_target_for_coverage_gcovr
# - add support for all gcovr output formats
#
# 2024-04-03, Bronek Kozicki
# - add support for output formats: jacoco, clover, lcov
#
# USAGE:
#
# 1. Copy this file into your cmake modules path.
@@ -256,10 +259,10 @@ endif()
# BASE_DIRECTORY "../" # Base directory for report
# # (defaults to PROJECT_SOURCE_DIR)
# FORMAT "cobertura" # Output format, one of:
# # xml cobertura sonarqube json-summary
# # json-details coveralls csv txt
# # html-single html-nested html-details
# # (xml is an alias to cobertura;
# # xml cobertura sonarqube jacoco clover
# # json-summary json-details coveralls csv
# # txt html-single html-nested html-details
# # lcov (xml is an alias to cobertura;
# # if no format is set, defaults to xml)
# EXCLUDE "src/dir1/*" "src/dir2/*" # Patterns to exclude (can be relative
# # to BASE_DIRECTORY, with CMake 3.4+)
@@ -308,6 +311,8 @@ function(setup_target_for_coverage_gcovr)
set(GCOVR_OUTPUT_FILE ${Coverage_NAME}.txt)
elseif(Coverage_FORMAT STREQUAL "csv")
set(GCOVR_OUTPUT_FILE ${Coverage_NAME}.csv)
elseif(Coverage_FORMAT STREQUAL "lcov")
set(GCOVR_OUTPUT_FILE ${Coverage_NAME}.lcov)
else()
set(GCOVR_OUTPUT_FILE ${Coverage_NAME}.xml)
endif()
@@ -320,6 +325,14 @@ function(setup_target_for_coverage_gcovr)
set(Coverage_FORMAT cobertura) # overwrite xml
elseif(Coverage_FORMAT STREQUAL "sonarqube")
list(APPEND GCOVR_ADDITIONAL_ARGS --sonarqube "${GCOVR_OUTPUT_FILE}" )
elseif(Coverage_FORMAT STREQUAL "jacoco")
list(APPEND GCOVR_ADDITIONAL_ARGS --jacoco "${GCOVR_OUTPUT_FILE}" )
list(APPEND GCOVR_ADDITIONAL_ARGS --jacoco-pretty )
elseif(Coverage_FORMAT STREQUAL "clover")
list(APPEND GCOVR_ADDITIONAL_ARGS --clover "${GCOVR_OUTPUT_FILE}" )
list(APPEND GCOVR_ADDITIONAL_ARGS --clover-pretty )
elseif(Coverage_FORMAT STREQUAL "lcov")
list(APPEND GCOVR_ADDITIONAL_ARGS --lcov "${GCOVR_OUTPUT_FILE}" )
elseif(Coverage_FORMAT STREQUAL "json-summary")
list(APPEND GCOVR_ADDITIONAL_ARGS --json-summary "${GCOVR_OUTPUT_FILE}" )
list(APPEND GCOVR_ADDITIONAL_ARGS --json-summary-pretty)

View File

@@ -58,7 +58,6 @@ target_link_libraries(xrpl.imports.main
OpenSSL::Crypto
Ripple::boost
wasmedge::wasmedge
NIH::quickjs
Ripple::opts
Ripple::syslibs
absl::random_random
@@ -166,6 +165,9 @@ if(xrpld)
if(tests)
target_compile_definitions(rippled PUBLIC ENABLE_TESTS)
endif()
if(xahaud_runtime_test_config)
target_compile_definitions(rippled PUBLIC XAHAUD_ENABLE_RUNTIME_TEST_CONFIG=1)
endif()
target_include_directories(rippled
PRIVATE
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/src>
@@ -181,6 +183,21 @@ if(xrpld)
"${CMAKE_CURRENT_SOURCE_DIR}/src/test/*.cpp"
)
target_sources(rippled PRIVATE ${sources})
set(HOOKS_TEST_DIR "" CACHE PATH "External hook Env-test directory")
if(NOT HOOKS_TEST_DIR AND DEFINED ENV{HOOKS_TEST_DIR})
set(HOOKS_TEST_DIR "$ENV{HOOKS_TEST_DIR}")
endif()
if(HOOKS_TEST_DIR)
file(GLOB_RECURSE hook_test_sources CONFIGURE_DEPENDS
"${HOOKS_TEST_DIR}/*_test.cpp"
)
if(hook_test_sources)
message(STATUS "Including external hook Env tests from ${HOOKS_TEST_DIR}")
target_sources(rippled PRIVATE ${hook_test_sources})
target_include_directories(rippled PRIVATE "${HOOKS_TEST_DIR}")
endif()
endif()
endif()
target_link_libraries(rippled

View File

@@ -6,7 +6,6 @@ include(create_symbolic_link)
install (
TARGETS
quickjs
common
opts
ripple_syslibs

View File

@@ -22,6 +22,9 @@ target_compile_definitions (opts
$<$<BOOL:${beast_no_unit_test_inline}>:BEAST_NO_UNIT_TEST_INLINE=1>
$<$<BOOL:${beast_disable_autolink}>:BEAST_DONT_AUTOLINK_TO_WIN32_LIBRARIES=1>
$<$<BOOL:${single_io_service_thread}>:RIPPLE_SINGLE_IO_SERVICE_THREAD=1>
# Enhanced logging is enabled for Debug builds, or explicitly via
# -DBEAST_ENHANCED_LOGGING=ON for other build types.
$<$<OR:$<CONFIG:Debug>,$<BOOL:${BEAST_ENHANCED_LOGGING}>>:BEAST_ENHANCED_LOGGING=1>
$<$<BOOL:${voidstar}>:ENABLE_VOIDSTAR>)
target_compile_options (opts
INTERFACE

View File

@@ -12,6 +12,13 @@ option(xrpld "Build xrpld" ON)
option(tests "Build tests" ON)
option(xahaud_runtime_test_config
"Enable XAHAUD_RUNTIME_TEST_CONFIG env and runtime_config RPC fault-injection controls"
OFF)
# Conan 2 local opt-in:
# [conf]
# tools.cmake.cmaketoolchain:extra_variables={"xahaud_runtime_test_config":"ON"}
option(unity "Creates a build using UNITY support in cmake. This is the default" ON)
if(unity)
if(NOT is_ci)

View File

@@ -1,52 +0,0 @@
# QuickJS source files
set(QUICKJS_SOURCES
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/libutf.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/qsort_r.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/tutf8e.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/vector.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/char-utils.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/buffer-utils.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/utils.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/ringbuffer.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/libregexp.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/quickjs.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/libregexp.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/libunicode.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/cutils.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/quickjs-libc.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/libbf.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/quickjs-textcode.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/quickjs-internal.c
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/debug.c
)
# QuickJS include directories
set(QUICKJS_INCLUDE_DIRS ${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs)
# RHTODO: Fix this to be dynamic
set(QUICKJS_VERSION "2023-01-01")
# QuickJS compile definitions
set(QUICKJS_COMPILE_DEFINITIONS
_GNU_SOURCE
CONFIG_VERSION="${QUICKJS_VERSION}"
CONFIG_BIGNUM
)
# Create static library
add_library(quickjs STATIC ${QUICKJS_SOURCES})
target_include_directories(quickjs
PUBLIC
$<BUILD_INTERFACE:${QUICKJS_INCLUDE_DIRS}>
$<INSTALL_INTERFACE:include/quickjs>
)
target_compile_definitions(quickjs PUBLIC ${QUICKJS_COMPILE_DEFINITIONS})
add_library(NIH::quickjs ALIAS quickjs)
target_link_libraries(ripple_libs INTERFACE NIH::quickjs)
install(
FILES
${CMAKE_CURRENT_SOURCE_DIR}/src/quickjs/*.h
DESTINATION include/quickjs
)

View File

@@ -47,5 +47,8 @@
#define MEM_OVERLAP -43
#define TOO_MANY_STATE_MODIFICATIONS -44
#define TOO_MANY_NAMESPACES -45
#define EXPORT_FAILURE -46
#define TOO_MANY_EXPORTED_TXN -47
#define TOO_LITTLE_ENTROPY -48
#define HOOK_ERROR_CODES
#endif //HOOK_ERROR_CODES

View File

@@ -2,6 +2,9 @@
// Generated using generate_extern.sh
#include <stdint.h>
#ifndef HOOK_EXTERN
#ifdef __cplusplus
extern "C" {
#endif
extern int32_t __attribute__((noduplicate))
_g(uint32_t guard_id, uint32_t maxiter);
@@ -336,5 +339,43 @@ prepare(
uint32_t read_ptr,
uint32_t read_len);
extern int64_t
xport_reserve(uint32_t count);
extern int64_t
xport(
uint32_t write_ptr,
uint32_t write_len,
uint32_t read_ptr,
uint32_t read_len);
extern int64_t
xport_cancel(uint32_t ticket_seq);
/*
Consensus entropy APIs.
min_tier is a fail-closed floor:
1 = consensus_fallback, 2 = participant_aligned, 3 = validator_quorum.
min_count is the minimum validator/reveal count the caller accepts.
If the most recent finalized entropy object does not satisfy both floors,
these APIs return TOO_LITTLE_ENTROPY. Open-ledger and simulate execution
are provisional previews over the entropy currently visible to the node;
final ordered ledger execution may see a different entropy object.
*/
extern int64_t
dice(uint32_t sides, uint32_t min_tier, uint32_t min_count);
extern int64_t
random(
uint32_t write_ptr,
uint32_t write_len,
uint32_t min_tier,
uint32_t min_count);
#ifdef __cplusplus
}
#endif
#define HOOK_EXTERN
#endif // HOOK_EXTERN

View File

@@ -9,7 +9,7 @@ ENUM_FILE="$SCRIPT_DIR/../include/xrpl/hook/Enum.h"
echo '// For documentation please see: https://xrpl-hooks.readme.io/reference/'
echo '// Generated using generate_error.sh'
echo '#ifndef HOOK_ERROR_CODES'
sed -n '/enum hook_return_code/,/};/p' "$ENUM_FILE" |
sed -n '/enum class hook_return_code/,/};/p' "$ENUM_FILE" |
awk '
function ltrim(s) { sub(/^[[:space:]]+/, "", s); return s }
function rtrim(s) { sub(/[[:space:]]+$/, "", s); return s }
@@ -31,7 +31,7 @@ sed -n '/enum hook_return_code/,/};/p' "$ENUM_FILE" |
{
line = $0
if (line ~ /enum[[:space:]]+hook_return_code/)
if (line ~ /enum[[:space:]]+class[[:space:]]+hook_return_code/)
next
if (line ~ /^[[:space:]]*\{/)
next

View File

@@ -11,6 +11,9 @@ APPLY_HOOK="$SCRIPT_DIR/../include/xrpl/hook/hook_api.macro"
echo '// Generated using generate_extern.sh'
echo '#include <stdint.h>'
echo '#ifndef HOOK_EXTERN'
echo '#ifdef __cplusplus'
echo 'extern "C" {'
echo '#endif'
echo
awk '
function trim(s) {
@@ -27,8 +30,8 @@ APPLY_HOOK="$SCRIPT_DIR/../include/xrpl/hook/hook_api.macro"
next;
}
# Look for comment lines that start with // int64_t or // int32_t
if (line ~ /^[[:space:]]*\/\/[[:space:]]*(int64_t|int32_t)[[:space:]]+[a-zA-Z_][a-zA-Z0-9_]*[[:space:]]*\(/) {
# Look for comment lines that start with // and contain function signature
if (line ~ /^[[:space:]]*\/\/[[:space:]]*[a-zA-Z_][a-zA-Z0-9_]*[[:space:]]+[a-zA-Z_][a-zA-Z0-9_]*[[:space:]]*\(/) {
# Remove leading // and trim
sub(/^[[:space:]]*\/\/[[:space:]]*/, "", line);
line = trim(line);
@@ -38,6 +41,21 @@ APPLY_HOOK="$SCRIPT_DIR/../include/xrpl/hook/hook_api.macro"
# Insert __attribute__((noduplicate)) before _g
sub(/[[:space:]]+_g/, " __attribute__((noduplicate)) _g", line);
}
if (line ~ /[[:space:]]+dice[[:space:]]*\(/) {
print "/*";
print " Consensus entropy APIs.";
print "";
print " min_tier is a fail-closed floor:";
print " 1 = consensus_fallback, 2 = participant_aligned, 3 = validator_quorum.";
print " min_count is the minimum validator/reveal count the caller accepts.";
print "";
print " If the most recent finalized entropy object does not satisfy both floors,";
print " these APIs return TOO_LITTLE_ENTROPY. Open-ledger and simulate execution";
print " are provisional previews over the entropy currently visible to the node;";
print " final ordered ledger execution may see a different entropy object.";
print "*/";
}
# printf("\n");
@@ -46,6 +64,9 @@ APPLY_HOOK="$SCRIPT_DIR/../include/xrpl/hook/hook_api.macro"
}
' "$APPLY_HOOK"
echo '#ifdef __cplusplus'
echo '}'
echo '#endif'
echo '#define HOOK_EXTERN'
echo '#endif // HOOK_EXTERN'
} | (

View File

@@ -47,4 +47,4 @@
#include "macro.h"
#include "types.h"
#endif
#endif

View File

@@ -607,31 +607,37 @@ int out_len = 0;\
#define PREPARE_PAYMENT_SIMPLE_SIZE 248U
#endif
#define PREPARE_PAYMENT_SIMPLE(buf_out_master, drops_amount_raw, to_address, dest_tag_raw, src_tag_raw)\
{\
uint8_t* buf_out = buf_out_master;\
uint8_t acc[20];\
uint64_t drops_amount = (drops_amount_raw);\
uint32_t dest_tag = (dest_tag_raw);\
uint32_t src_tag = (src_tag_raw);\
uint32_t cls = (uint32_t)ledger_seq();\
hook_account(SBUF(acc));\
_01_02_ENCODE_TT (buf_out, ttPAYMENT ); /* uint16 | size 3 */ \
_02_02_ENCODE_FLAGS (buf_out, tfCANONICAL ); /* uint32 | size 5 */ \
_02_03_ENCODE_TAG_SRC (buf_out, src_tag ); /* uint32 | size 5 */ \
_02_04_ENCODE_SEQUENCE (buf_out, 0 ); /* uint32 | size 5 */ \
_02_14_ENCODE_TAG_DST (buf_out, dest_tag ); /* uint32 | size 5 */ \
_02_26_ENCODE_FLS (buf_out, cls + 1 ); /* uint32 | size 6 */ \
_02_27_ENCODE_LLS (buf_out, cls + 5 ); /* uint32 | size 6 */ \
_06_01_ENCODE_DROPS_AMOUNT (buf_out, drops_amount ); /* amount | size 9 */ \
uint8_t* fee_ptr = buf_out;\
_06_08_ENCODE_DROPS_FEE (buf_out, 0 ); /* amount | size 9 */ \
_07_03_ENCODE_SIGNING_PUBKEY_NULL (buf_out ); /* pk | size 35 */ \
_08_01_ENCODE_ACCOUNT_SRC (buf_out, acc ); /* account | size 22 */ \
_08_03_ENCODE_ACCOUNT_DST (buf_out, to_address ); /* account | size 22 */ \
int64_t edlen = etxn_details((uint32_t)buf_out, PREPARE_PAYMENT_SIMPLE_SIZE); /* emitdet | size 1?? */ \
int64_t fee = etxn_fee_base(buf_out_master, PREPARE_PAYMENT_SIMPLE_SIZE); \
_06_08_ENCODE_DROPS_FEE (fee_ptr, fee ); \
#define PREPARE_PAYMENT_SIMPLE( \
buf_out_master, drops_amount_raw, to_address, dest_tag_raw, src_tag_raw) \
{ \
uint8_t* buf_out = buf_out_master; \
uint8_t acc[20]; \
uint64_t drops_amount = (drops_amount_raw); \
uint32_t dest_tag = (dest_tag_raw); \
uint32_t src_tag = (src_tag_raw); \
uint32_t cls = (uint32_t)ledger_seq(); \
hook_account(SBUF(acc)); \
_01_02_ENCODE_TT(buf_out, ttPAYMENT); /* uint16 | size 3 */ \
_02_02_ENCODE_FLAGS(buf_out, tfCANONICAL); /* uint32 | size 5 */ \
_02_03_ENCODE_TAG_SRC(buf_out, src_tag); /* uint32 | size 5 */ \
_02_04_ENCODE_SEQUENCE(buf_out, 0); /* uint32 | size 5 */ \
_02_14_ENCODE_TAG_DST(buf_out, dest_tag); /* uint32 | size 5 */ \
_02_26_ENCODE_FLS(buf_out, cls + 1); /* uint32 | size 6 */ \
_02_27_ENCODE_LLS(buf_out, cls + 5); /* uint32 | size 6 */ \
_06_01_ENCODE_DROPS_AMOUNT( \
buf_out, drops_amount); /* amount | size 9 */ \
uint8_t* fee_ptr = buf_out; \
_06_08_ENCODE_DROPS_FEE(buf_out, 0); /* amount | size 9 */ \
_07_03_ENCODE_SIGNING_PUBKEY_NULL(buf_out); /* pk | size 35 */ \
_08_01_ENCODE_ACCOUNT_SRC(buf_out, acc); /* account | size 22 */ \
_08_03_ENCODE_ACCOUNT_DST( \
buf_out, to_address); /* account | size 22 */ \
int64_t edlen = etxn_details( \
(uint32_t)buf_out, \
PREPARE_PAYMENT_SIMPLE_SIZE); /* emitdet | size 1?? */ \
int64_t fee = \
etxn_fee_base(buf_out_master, PREPARE_PAYMENT_SIMPLE_SIZE); \
_06_08_ENCODE_DROPS_FEE(fee_ptr, fee); \
}
#ifdef HAS_CALLBACK
@@ -639,33 +645,35 @@ int out_len = 0;\
#else
#define PREPARE_PAYMENT_SIMPLE_TRUSTLINE_SIZE 287
#endif
#define PREPARE_PAYMENT_SIMPLE_TRUSTLINE(buf_out_master, tlamt, to_address, dest_tag_raw, src_tag_raw)\
{\
uint8_t* buf_out = buf_out_master;\
uint8_t acc[20];\
uint32_t dest_tag = (dest_tag_raw);\
uint32_t src_tag = (src_tag_raw);\
uint32_t cls = (uint32_t)ledger_seq();\
hook_account(SBUF(acc));\
_01_02_ENCODE_TT (buf_out, ttPAYMENT ); /* uint16 | size 3 */ \
_02_02_ENCODE_FLAGS (buf_out, tfCANONICAL ); /* uint32 | size 5 */ \
_02_03_ENCODE_TAG_SRC (buf_out, src_tag ); /* uint32 | size 5 */ \
_02_04_ENCODE_SEQUENCE (buf_out, 0 ); /* uint32 | size 5 */ \
_02_14_ENCODE_TAG_DST (buf_out, dest_tag ); /* uint32 | size 5 */ \
_02_26_ENCODE_FLS (buf_out, cls + 1 ); /* uint32 | size 6 */ \
_02_27_ENCODE_LLS (buf_out, cls + 5 ); /* uint32 | size 6 */ \
_06_01_ENCODE_TL_AMOUNT (buf_out, tlamt ); /* amount | size 48 */ \
uint8_t* fee_ptr = buf_out;\
_06_08_ENCODE_DROPS_FEE (buf_out, 0 ); /* amount | size 9 */ \
_07_03_ENCODE_SIGNING_PUBKEY_NULL (buf_out ); /* pk | size 35 */ \
_08_01_ENCODE_ACCOUNT_SRC (buf_out, acc ); /* account | size 22 */ \
_08_03_ENCODE_ACCOUNT_DST (buf_out, to_address ); /* account | size 22 */ \
etxn_details((uint32_t)buf_out, PREPARE_PAYMENT_SIMPLE_TRUSTLINE_SIZE); /* emitdet | size 1?? */ \
int64_t fee = etxn_fee_base(buf_out_master, PREPARE_PAYMENT_SIMPLE_TRUSTLINE_SIZE); \
_06_08_ENCODE_DROPS_FEE (fee_ptr, fee ); \
#define PREPARE_PAYMENT_SIMPLE_TRUSTLINE( \
buf_out_master, tlamt, to_address, dest_tag_raw, src_tag_raw) \
{ \
uint8_t* buf_out = buf_out_master; \
uint8_t acc[20]; \
uint32_t dest_tag = (dest_tag_raw); \
uint32_t src_tag = (src_tag_raw); \
uint32_t cls = (uint32_t)ledger_seq(); \
hook_account(SBUF(acc)); \
_01_02_ENCODE_TT(buf_out, ttPAYMENT); /* uint16 | size 3 */ \
_02_02_ENCODE_FLAGS(buf_out, tfCANONICAL); /* uint32 | size 5 */ \
_02_03_ENCODE_TAG_SRC(buf_out, src_tag); /* uint32 | size 5 */ \
_02_04_ENCODE_SEQUENCE(buf_out, 0); /* uint32 | size 5 */ \
_02_14_ENCODE_TAG_DST(buf_out, dest_tag); /* uint32 | size 5 */ \
_02_26_ENCODE_FLS(buf_out, cls + 1); /* uint32 | size 6 */ \
_02_27_ENCODE_LLS(buf_out, cls + 5); /* uint32 | size 6 */ \
_06_01_ENCODE_TL_AMOUNT(buf_out, tlamt); /* amount | size 48 */ \
uint8_t* fee_ptr = buf_out; \
_06_08_ENCODE_DROPS_FEE(buf_out, 0); /* amount | size 9 */ \
_07_03_ENCODE_SIGNING_PUBKEY_NULL(buf_out); /* pk | size 35 */ \
_08_01_ENCODE_ACCOUNT_SRC(buf_out, acc); /* account | size 22 */ \
_08_03_ENCODE_ACCOUNT_DST( \
buf_out, to_address); /* account | size 22 */ \
etxn_details( \
(uint32_t)buf_out, \
PREPARE_PAYMENT_SIMPLE_TRUSTLINE_SIZE); /* emitdet | size 1?? */ \
int64_t fee = etxn_fee_base( \
buf_out_master, PREPARE_PAYMENT_SIMPLE_TRUSTLINE_SIZE); \
_06_08_ENCODE_DROPS_FEE(fee_ptr, fee); \
}
#endif

View File

@@ -3,18 +3,30 @@
#define sfCloseResolution ((16U << 16U) + 1U)
#define sfMethod ((16U << 16U) + 2U)
#define sfTransactionResult ((16U << 16U) + 3U)
#define sfScale ((16U << 16U) + 4U)
#define sfAssetScale ((16U << 16U) + 5U)
#define sfTickSize ((16U << 16U) + 16U)
#define sfUNLModifyDisabling ((16U << 16U) + 17U)
#define sfHookResult ((16U << 16U) + 18U)
#define sfWasLockingChainSend ((16U << 16U) + 19U)
#define sfSidecarType ((16U << 16U) + 20U)
#define sfEntropyTier ((16U << 16U) + 21U)
#define sfLedgerEntryType ((1U << 16U) + 1U)
#define sfTransactionType ((1U << 16U) + 2U)
#define sfSignerWeight ((1U << 16U) + 3U)
#define sfTransferFee ((1U << 16U) + 4U)
#define sfTradingFee ((1U << 16U) + 5U)
#define sfDiscountedFee ((1U << 16U) + 6U)
#define sfVersion ((1U << 16U) + 16U)
#define sfHookStateChangeCount ((1U << 16U) + 17U)
#define sfHookEmitCount ((1U << 16U) + 18U)
#define sfHookExecutionIndex ((1U << 16U) + 19U)
#define sfHookApiVersion ((1U << 16U) + 20U)
#define sfHookStateScale ((1U << 16U) + 21U)
#define sfLedgerFixType ((1U << 16U) + 22U)
#define sfHookExportCount ((1U << 16U) + 98U)
#define sfEntropyCount ((1U << 16U) + 99U)
#define sfEntropyDenominator ((1U << 16U) + 100U)
#define sfNetworkID ((2U << 16U) + 1U)
#define sfFlags ((2U << 16U) + 2U)
#define sfSourceTag ((2U << 16U) + 3U)
@@ -29,6 +41,7 @@
#define sfWalletSize ((2U << 16U) + 12U)
#define sfOwnerCount ((2U << 16U) + 13U)
#define sfDestinationTag ((2U << 16U) + 14U)
#define sfLastUpdateTime ((2U << 16U) + 15U)
#define sfHighQualityIn ((2U << 16U) + 16U)
#define sfHighQualityOut ((2U << 16U) + 17U)
#define sfLowQualityIn ((2U << 16U) + 18U)
@@ -60,10 +73,19 @@
#define sfBurnedNFTokens ((2U << 16U) + 44U)
#define sfHookStateCount ((2U << 16U) + 45U)
#define sfEmitGeneration ((2U << 16U) + 46U)
#define sfLockCount ((2U << 16U) + 47U)
#define sfVoteWeight ((2U << 16U) + 48U)
#define sfLockCount ((2U << 16U) + 49U)
#define sfFirstNFTokenSequence ((2U << 16U) + 50U)
#define sfOracleDocumentID ((2U << 16U) + 51U)
#define sfStartTime ((2U << 16U) + 93U)
#define sfRepeatCount ((2U << 16U) + 94U)
#define sfDelaySeconds ((2U << 16U) + 95U)
#define sfXahauActivationLgrSeq ((2U << 16U) + 96U)
#define sfImportSequence ((2U << 16U) + 97U)
#define sfRewardTime ((2U << 16U) + 98U)
#define sfRewardLgrFirst ((2U << 16U) + 99U)
#define sfRewardLgrLast ((2U << 16U) + 100U)
#define sfCancelTicketSequence ((2U << 16U) + 101U)
#define sfIndexNext ((3U << 16U) + 1U)
#define sfIndexPrevious ((3U << 16U) + 2U)
#define sfBookNode ((3U << 16U) + 3U)
@@ -80,12 +102,25 @@
#define sfHookInstructionCount ((3U << 16U) + 17U)
#define sfHookReturnCode ((3U << 16U) + 18U)
#define sfReferenceCount ((3U << 16U) + 19U)
#define sfXChainClaimID ((3U << 16U) + 20U)
#define sfXChainAccountCreateCount ((3U << 16U) + 21U)
#define sfXChainAccountClaimCount ((3U << 16U) + 22U)
#define sfAssetPrice ((3U << 16U) + 23U)
#define sfMaximumAmount ((3U << 16U) + 24U)
#define sfOutstandingAmount ((3U << 16U) + 25U)
#define sfMPTAmount ((3U << 16U) + 26U)
#define sfIssuerNode ((3U << 16U) + 27U)
#define sfSubjectNode ((3U << 16U) + 28U)
#define sfTouchCount ((3U << 16U) + 97U)
#define sfAccountIndex ((3U << 16U) + 98U)
#define sfAccountCount ((3U << 16U) + 99U)
#define sfRewardAccumulator ((3U << 16U) + 100U)
#define sfEmailHash ((4U << 16U) + 1U)
#define sfTakerPaysCurrency ((10U << 16U) + 1U)
#define sfTakerPaysIssuer ((10U << 16U) + 2U)
#define sfTakerGetsCurrency ((10U << 16U) + 3U)
#define sfTakerGetsIssuer ((10U << 16U) + 4U)
#define sfTakerPaysCurrency ((17U << 16U) + 1U)
#define sfTakerPaysIssuer ((17U << 16U) + 2U)
#define sfTakerGetsCurrency ((17U << 16U) + 3U)
#define sfTakerGetsIssuer ((17U << 16U) + 4U)
#define sfMPTokenIssuanceID ((21U << 16U) + 1U)
#define sfLedgerHash ((5U << 16U) + 1U)
#define sfParentHash ((5U << 16U) + 2U)
#define sfTransactionHash ((5U << 16U) + 3U)
@@ -99,6 +134,8 @@
#define sfEmitParentTxnID ((5U << 16U) + 11U)
#define sfEmitNonce ((5U << 16U) + 12U)
#define sfEmitHookHash ((5U << 16U) + 13U)
#define sfObjectID ((5U << 16U) + 14U)
#define sfAMMID ((5U << 16U) + 15U)
#define sfBookDirectory ((5U << 16U) + 16U)
#define sfInvoiceID ((5U << 16U) + 17U)
#define sfNickname ((5U << 16U) + 18U)
@@ -120,6 +157,16 @@
#define sfOfferID ((5U << 16U) + 34U)
#define sfEscrowID ((5U << 16U) + 35U)
#define sfURITokenID ((5U << 16U) + 36U)
#define sfDomainID ((5U << 16U) + 37U)
#define sfExportSignatureHash ((5U << 16U) + 38U)
#define sfHookOnOutgoing ((5U << 16U) + 93U)
#define sfHookOnIncoming ((5U << 16U) + 94U)
#define sfCron ((5U << 16U) + 95U)
#define sfHookCanEmit ((5U << 16U) + 96U)
#define sfEmittedTxnID ((5U << 16U) + 97U)
#define sfGovernanceMarks ((5U << 16U) + 98U)
#define sfGovernanceFlags ((5U << 16U) + 99U)
#define sfNumber ((9U << 16U) + 1U)
#define sfAmount ((6U << 16U) + 1U)
#define sfBalance ((6U << 16U) + 2U)
#define sfLimitAmount ((6U << 16U) + 3U)
@@ -130,12 +177,26 @@
#define sfFee ((6U << 16U) + 8U)
#define sfSendMax ((6U << 16U) + 9U)
#define sfDeliverMin ((6U << 16U) + 10U)
#define sfAmount2 ((6U << 16U) + 11U)
#define sfBidMin ((6U << 16U) + 12U)
#define sfBidMax ((6U << 16U) + 13U)
#define sfMinimumOffer ((6U << 16U) + 16U)
#define sfRippleEscrow ((6U << 16U) + 17U)
#define sfDeliveredAmount ((6U << 16U) + 18U)
#define sfNFTokenBrokerFee ((6U << 16U) + 19U)
#define sfHookCallbackFee ((6U << 16U) + 20U)
#define sfLockedBalance ((6U << 16U) + 21U)
#define sfBaseFeeDrops ((6U << 16U) + 22U)
#define sfReserveBaseDrops ((6U << 16U) + 23U)
#define sfReserveIncrementDrops ((6U << 16U) + 24U)
#define sfLPTokenOut ((6U << 16U) + 25U)
#define sfLPTokenIn ((6U << 16U) + 26U)
#define sfEPrice ((6U << 16U) + 27U)
#define sfPrice ((6U << 16U) + 28U)
#define sfSignatureReward ((6U << 16U) + 29U)
#define sfMinAccountCreateAmount ((6U << 16U) + 30U)
#define sfLPTokenBalance ((6U << 16U) + 31U)
#define sfTrustLineRewardAccumulator ((6U << 16U) + 99U)
#define sfPublicKey ((7U << 16U) + 1U)
#define sfMessageKey ((7U << 16U) + 2U)
#define sfSigningPubKey ((7U << 16U) + 3U)
@@ -161,6 +222,16 @@
#define sfHookParameterName ((7U << 16U) + 24U)
#define sfHookParameterValue ((7U << 16U) + 25U)
#define sfBlob ((7U << 16U) + 26U)
#define sfDIDDocument ((7U << 16U) + 27U)
#define sfData ((7U << 16U) + 28U)
#define sfAssetClass ((7U << 16U) + 29U)
#define sfProvider ((7U << 16U) + 30U)
#define sfMPTokenMetadata ((7U << 16U) + 31U)
#define sfCredentialType ((7U << 16U) + 32U)
#define sfEntropyContributors ((7U << 16U) + 33U)
#define sfHookName ((7U << 16U) + 97U)
#define sfRemarkValue ((7U << 16U) + 98U)
#define sfRemarkName ((7U << 16U) + 99U)
#define sfAccount ((8U << 16U) + 1U)
#define sfOwner ((8U << 16U) + 2U)
#define sfDestination ((8U << 16U) + 3U)
@@ -170,13 +241,32 @@
#define sfRegularKey ((8U << 16U) + 8U)
#define sfNFTokenMinter ((8U << 16U) + 9U)
#define sfEmitCallback ((8U << 16U) + 10U)
#define sfHolder ((8U << 16U) + 11U)
#define sfHookAccount ((8U << 16U) + 16U)
#define sfOtherChainSource ((8U << 16U) + 18U)
#define sfOtherChainDestination ((8U << 16U) + 19U)
#define sfAttestationSignerAccount ((8U << 16U) + 20U)
#define sfAttestationRewardAccount ((8U << 16U) + 21U)
#define sfLockingChainDoor ((8U << 16U) + 22U)
#define sfIssuingChainDoor ((8U << 16U) + 23U)
#define sfSubject ((8U << 16U) + 24U)
#define sfInform ((8U << 16U) + 99U)
#define sfIndexes ((19U << 16U) + 1U)
#define sfHashes ((19U << 16U) + 2U)
#define sfAmendments ((19U << 16U) + 3U)
#define sfNFTokenOffers ((19U << 16U) + 4U)
#define sfHookNamespaces ((19U << 16U) + 5U)
#define sfCredentialIDs ((19U << 16U) + 6U)
#define sfURITokenIDs ((19U << 16U) + 99U)
#define sfPaths ((18U << 16U) + 1U)
#define sfBaseAsset ((26U << 16U) + 1U)
#define sfQuoteAsset ((26U << 16U) + 2U)
#define sfLockingChainIssue ((24U << 16U) + 1U)
#define sfIssuingChainIssue ((24U << 16U) + 2U)
#define sfAsset ((24U << 16U) + 3U)
#define sfAsset2 ((24U << 16U) + 4U)
#define sfClaimCurrency ((24U << 16U) + 5U)
#define sfXChainBridge ((25U << 16U) + 1U)
#define sfTransactionMetaData ((14U << 16U) + 2U)
#define sfCreatedNode ((14U << 16U) + 3U)
#define sfDeletedNode ((14U << 16U) + 4U)
@@ -195,9 +285,28 @@
#define sfDisabledValidator ((14U << 16U) + 19U)
#define sfEmittedTxn ((14U << 16U) + 20U)
#define sfHookExecution ((14U << 16U) + 21U)
#define sfHookDefinition ((14U << 16U) + 22U)
#define sfHookParameter ((14U << 16U) + 23U)
#define sfHookGrant ((14U << 16U) + 24U)
#define sfVoteEntry ((14U << 16U) + 25U)
#define sfAuctionSlot ((14U << 16U) + 26U)
#define sfAuthAccount ((14U << 16U) + 27U)
#define sfXChainClaimProofSig ((14U << 16U) + 28U)
#define sfXChainCreateAccountProofSig ((14U << 16U) + 29U)
#define sfXChainClaimAttestationCollectionElement ((14U << 16U) + 30U)
#define sfXChainCreateAccountAttestationCollectionElement ((14U << 16U) + 31U)
#define sfPriceData ((14U << 16U) + 32U)
#define sfCredential ((14U << 16U) + 33U)
#define sfExportedTxn ((14U << 16U) + 90U)
#define sfAmountEntry ((14U << 16U) + 91U)
#define sfMintURIToken ((14U << 16U) + 92U)
#define sfHookEmission ((14U << 16U) + 93U)
#define sfImportVLKey ((14U << 16U) + 94U)
#define sfActiveValidator ((14U << 16U) + 95U)
#define sfGenesisMint ((14U << 16U) + 96U)
#define sfRemark ((14U << 16U) + 97U)
#define sfHighReward ((14U << 16U) + 98U)
#define sfLowReward ((14U << 16U) + 99U)
#define sfExportResult ((14U << 16U) + 100U)
#define sfSigners ((15U << 16U) + 3U)
#define sfSignerEntries ((15U << 16U) + 4U)
#define sfTemplate ((15U << 16U) + 5U)
@@ -207,9 +316,22 @@
#define sfMemos ((15U << 16U) + 9U)
#define sfNFTokens ((15U << 16U) + 10U)
#define sfHooks ((15U << 16U) + 11U)
#define sfVoteSlots ((15U << 16U) + 12U)
#define sfMajorities ((15U << 16U) + 16U)
#define sfDisabledValidators ((15U << 16U) + 17U)
#define sfHookExecutions ((15U << 16U) + 18U)
#define sfHookParameters ((15U << 16U) + 19U)
#define sfHookGrants ((15U << 16U) + 20U)
#define sfActiveValidators ((15U << 16U) + 95U)
#define sfXChainClaimAttestations ((15U << 16U) + 21U)
#define sfXChainCreateAccountAttestations ((15U << 16U) + 22U)
#define sfPriceDataSeries ((15U << 16U) + 24U)
#define sfAuthAccounts ((15U << 16U) + 25U)
#define sfAuthorizeCredentials ((15U << 16U) + 26U)
#define sfUnauthorizeCredentials ((15U << 16U) + 27U)
#define sfAcceptedCredentials ((15U << 16U) + 28U)
#define sfAmounts ((15U << 16U) + 92U)
#define sfHookEmissions ((15U << 16U) + 93U)
#define sfImportVLKeys ((15U << 16U) + 94U)
#define sfActiveValidators ((15U << 16U) + 95U)
#define sfGenesisMints ((15U << 16U) + 96U)
#define sfRemarks ((15U << 16U) + 97U)

View File

@@ -9,6 +9,8 @@
#define sfUNLModifyDisabling ((16U << 16U) + 17U)
#define sfHookResult ((16U << 16U) + 18U)
#define sfWasLockingChainSend ((16U << 16U) + 19U)
#define sfSidecarType ((16U << 16U) + 20U)
#define sfEntropyTier ((16U << 16U) + 21U)
#define sfLedgerEntryType ((1U << 16U) + 1U)
#define sfTransactionType ((1U << 16U) + 2U)
#define sfSignerWeight ((1U << 16U) + 3U)
@@ -22,6 +24,9 @@
#define sfHookApiVersion ((1U << 16U) + 20U)
#define sfHookStateScale ((1U << 16U) + 21U)
#define sfLedgerFixType ((1U << 16U) + 22U)
#define sfHookExportCount ((1U << 16U) + 98U)
#define sfEntropyCount ((1U << 16U) + 99U)
#define sfEntropyDenominator ((1U << 16U) + 100U)
#define sfNetworkID ((2U << 16U) + 1U)
#define sfFlags ((2U << 16U) + 2U)
#define sfSourceTag ((2U << 16U) + 3U)
@@ -80,6 +85,7 @@
#define sfRewardTime ((2U << 16U) + 98U)
#define sfRewardLgrFirst ((2U << 16U) + 99U)
#define sfRewardLgrLast ((2U << 16U) + 100U)
#define sfCancelTicketSequence ((2U << 16U) + 101U)
#define sfIndexNext ((3U << 16U) + 1U)
#define sfIndexPrevious ((3U << 16U) + 2U)
#define sfBookNode ((3U << 16U) + 3U)
@@ -152,6 +158,7 @@
#define sfEscrowID ((5U << 16U) + 35U)
#define sfURITokenID ((5U << 16U) + 36U)
#define sfDomainID ((5U << 16U) + 37U)
#define sfExportSignatureHash ((5U << 16U) + 38U)
#define sfHookOnOutgoing ((5U << 16U) + 93U)
#define sfHookOnIncoming ((5U << 16U) + 94U)
#define sfCron ((5U << 16U) + 95U)
@@ -189,6 +196,7 @@
#define sfSignatureReward ((6U << 16U) + 29U)
#define sfMinAccountCreateAmount ((6U << 16U) + 30U)
#define sfLPTokenBalance ((6U << 16U) + 31U)
#define sfTrustLineRewardAccumulator ((6U << 16U) + 99U)
#define sfPublicKey ((7U << 16U) + 1U)
#define sfMessageKey ((7U << 16U) + 2U)
#define sfSigningPubKey ((7U << 16U) + 3U)
@@ -220,6 +228,8 @@
#define sfProvider ((7U << 16U) + 30U)
#define sfMPTokenMetadata ((7U << 16U) + 31U)
#define sfCredentialType ((7U << 16U) + 32U)
#define sfEntropyContributors ((7U << 16U) + 33U)
#define sfHookName ((7U << 16U) + 97U)
#define sfRemarkValue ((7U << 16U) + 98U)
#define sfRemarkName ((7U << 16U) + 99U)
#define sfAccount ((8U << 16U) + 1U)
@@ -255,6 +265,7 @@
#define sfIssuingChainIssue ((24U << 16U) + 2U)
#define sfAsset ((24U << 16U) + 3U)
#define sfAsset2 ((24U << 16U) + 4U)
#define sfClaimCurrency ((24U << 16U) + 5U)
#define sfXChainBridge ((25U << 16U) + 1U)
#define sfTransactionMetaData ((14U << 16U) + 2U)
#define sfCreatedNode ((14U << 16U) + 3U)
@@ -285,6 +296,7 @@
#define sfXChainCreateAccountAttestationCollectionElement ((14U << 16U) + 31U)
#define sfPriceData ((14U << 16U) + 32U)
#define sfCredential ((14U << 16U) + 33U)
#define sfExportedTxn ((14U << 16U) + 90U)
#define sfAmountEntry ((14U << 16U) + 91U)
#define sfMintURIToken ((14U << 16U) + 92U)
#define sfHookEmission ((14U << 16U) + 93U)
@@ -292,6 +304,9 @@
#define sfActiveValidator ((14U << 16U) + 95U)
#define sfGenesisMint ((14U << 16U) + 96U)
#define sfRemark ((14U << 16U) + 97U)
#define sfHighReward ((14U << 16U) + 98U)
#define sfLowReward ((14U << 16U) + 99U)
#define sfExportResult ((14U << 16U) + 100U)
#define sfSigners ((15U << 16U) + 3U)
#define sfSignerEntries ((15U << 16U) + 4U)
#define sfTemplate ((15U << 16U) + 5U)

View File

@@ -61,6 +61,7 @@
#define ttNFTOKEN_MODIFY 70
#define ttPERMISSIONED_DOMAIN_SET 71
#define ttPERMISSIONED_DOMAIN_DELETE 72
#define ttEXPORT 91
#define ttCRON 92
#define ttCRON_SET 93
#define ttREMARKS_SET 94
@@ -74,3 +75,5 @@
#define ttUNL_MODIFY 102
#define ttEMIT_FAILURE 103
#define ttUNL_REPORT 104
#define ttCONSENSUS_ENTROPY 105
#define ttEXPORT_SIGNATURES 106

View File

@@ -115,3 +115,8 @@ enum AMMClawbackFlags : uint32_t {
enum BridgeModifyFlags : uint32_t {
tfClearAccountCreateAmount = 0x00010000,
};
enum ConsensusEntropyFlags : uint32_t {
tfEntropyCommit = 0x00000001, // entry is a commitment in commitSet
tfEntropyReveal = 0x00000002, // entry is a reveal in entropySet
};

View File

@@ -15,7 +15,10 @@
#define uint256 std::string
#define featureHooksUpdate1 "1"
#define featureHooksUpdate2 "1"
#define featureExport "1"
#define featureConsensusEntropy "1"
#define fix20250131 "1"
#define fixGuardDepth32 "1"
namespace hook_api {
struct Rules
{
@@ -264,11 +267,8 @@ enum hook_log_code : uint16_t {
SECTIONS_OUT_OF_SEQUENCE =
85, // the wasm contained sections out of sequence
CUSTOM_SECTION_DISALLOWED =
86, // the wasm contained a custom section (id=0)
INTERNAL_ERROR = 87, // an internal error described by the log text
JS_TEST_FAILURE = 88, // smoke test of js bytecode failed
JS_FEE_MISSING = 89,
JS_FEE_OUT_OF_RANGE = 90,
86, // the wasm contained a custom section (id=0)
INTERNAL_ERROR = 87, // an internal error described by the log text
// RH NOTE: only HookSet msgs got log codes, possibly all Hook log lines
// should get a code?
};
@@ -322,7 +322,7 @@ namespace compare_mode {
enum compare_mode : uint32_t { EQUAL = 1, LESS = 2, GREATER = 4 };
}
enum hook_return_code : int64_t {
enum class hook_return_code : int64_t {
SUCCESS =
0, // return codes > 0 are reserved for hook apis to return "success"
OUT_OF_BOUNDS =
@@ -386,29 +386,24 @@ enum hook_return_code : int64_t {
MEM_OVERLAP = -43, // one or more specified buffers are the same memory
TOO_MANY_STATE_MODIFICATIONS = -44, // more than 5000 modified state
// entires in the combined hook chains
TOO_MANY_NAMESPACES = -45
TOO_MANY_NAMESPACES = -45,
EXPORT_FAILURE = -46,
TOO_MANY_EXPORTED_TXN = -47,
TOO_LITTLE_ENTROPY = -48,
};
enum ExitType : uint8_t {
enum class ExitType : uint8_t {
UNSET = 0,
WASM_ERROR = 1,
ROLLBACK = 2,
ACCEPT = 3,
JSVM_ERROR = 4,
LEDGER_ERROR =
5, // if the ledger contained for example a nonsense hookapi number
INSTRUCTION_LIMIT_REACHED = 6,
};
enum CodeType : uint8_t {
WASM = 0,
JS = 1,
};
const uint16_t max_state_modifications = 256;
const uint8_t max_slots = 255;
const uint8_t max_nonce = 255;
const uint8_t max_emit = 255;
const uint8_t max_export = 2;
const uint8_t max_params = 16;
const double fee_base_multiplier = 1.1f;
@@ -436,10 +431,6 @@ getImportWhitelist(Rules const& rules)
if (AMENDMENT == uint256{} || rules.enabled(AMENDMENT)) \
whitelist[#FUNCTION_NAME] = { \
RETURN_TYPE, HOOK_WRAP_PARAMS PARAMS_TUPLE};
#define JSHOOK_API_DEFINITION( \
RETURN_TYPE, FUNCTION_NAME, PARAMS_TUPLE, AMENDMENT) \
{ \
}
#include "hook_api.macro"
@@ -453,12 +444,9 @@ getImportWhitelist(Rules const& rules)
return whitelist;
}
#undef HOOK_API_DEFINITION
#undef I32
#undef I64
enum GuardRulesVersion : uint64_t {
GuardRuleFix20250131 = 0x00000001,
GuardRuleDepth32 = 0x00000002,
};
inline uint64_t
@@ -467,6 +455,8 @@ getGuardRulesVersion(Rules const& rules)
uint64_t version = 0;
if (rules.enabled(fix20250131))
version |= GuardRuleFix20250131;
if (rules.enabled(fixGuardDepth32))
version |= GuardRuleDepth32;
return version;
}

View File

@@ -204,9 +204,13 @@ struct WasmBlkInf
}
// compute worst case execution time
inline uint64_t
compute_wce(const WasmBlkInf* blk, int level, bool* recursion_limit_reached)
compute_wce(
const WasmBlkInf* blk,
int level,
int max_level,
bool* recursion_limit_reached)
{
if (level > 16)
if (level > max_level)
{
*recursion_limit_reached = true;
return 0;
@@ -233,8 +237,8 @@ compute_wce(const WasmBlkInf* blk, int level, bool* recursion_limit_reached)
if (blk->children.size() > 0)
for (auto const& child : blk->children)
worst_case_execution +=
compute_wce(child, level + 1, recursion_limit_reached);
worst_case_execution += compute_wce(
child, level + 1, max_level, recursion_limit_reached);
if (parent == 0 ||
parent->iteration_bound ==
@@ -788,12 +792,17 @@ check_guard(
}
bool recursion_limit_reached = false;
uint64_t wce = compute_wce(&(*root), 0, &recursion_limit_reached);
int max_level = 16;
if (rulesVersion & hook_api::GuardRuleDepth32)
max_level = 32;
uint64_t wce =
compute_wce(&(*root), 0, max_level, &recursion_limit_reached);
if (recursion_limit_reached)
{
GUARDLOG(hook::log::NESTING_LIMIT)
<< "GuardCheck "
<< "Maximum allowable depth of blocks reached (16 levels). Flatten "
<< "Maximum allowable depth of blocks reached (" << max_level
<< " levels). Flatten "
"your loops and conditions!.\n";
return {};
}

View File

@@ -67,7 +67,6 @@
#define SEP_int32_t LPAREN int32_t COMMA
#define SEP_uint64_t LPAREN uint64_t COMMA
#define SEP_int64_t LPAREN int64_t COMMA
#define SEP_JSValue LPAREN JSValue COMMA
#define VAL_uint32_t WasmEdge_ValueGetI32(in[_stack++])
#define VAL_int32_t WasmEdge_ValueGetI32(in[_stack++])
@@ -90,174 +89,69 @@
#define WASM_VAL_TYPE(T, b) CAT2(TYP_, T)
#define HALF_COUNT(...) \
HALF_COUNT_IMPL( \
__VA_ARGS__, \
16, \
16, \
15, \
15, \
14, \
14, \
13, \
13, \
12, \
12, \
11, \
11, \
10, \
10, \
9, \
9, \
8, \
8, \
7, \
7, \
6, \
6, \
5, \
5, \
4, \
4, \
3, \
3, \
2, \
2, \
1, \
1, \
0, \
0)
#define UNSIGNED_TYPE(T) std::make_unsigned_t<T>
#define HALF_COUNT_IMPL( \
_1, \
_2, \
_3, \
_4, \
_5, \
_6, \
_7, \
_8, \
_9, \
_10, \
_11, \
_12, \
_13, \
_14, \
_15, \
_16, \
_17, \
_18, \
_19, \
_20, \
_21, \
_22, \
_23, \
_24, \
_25, \
_26, \
_27, \
_28, \
_29, \
_30, \
_31, \
_32, \
N, \
...) \
N
#define DECLARE_WASM_FUNCTION(R, F, ...) \
R F(hook::HookContext& hookCtx, \
WasmEdge_CallingFrameContext const& frameCtx __VA_OPT__( \
COMMA __VA_ARGS__)); \
extern WasmEdge_Result WasmFunction##F( \
void* data_ptr, \
const WasmEdge_CallingFrameContext* frameCtx, \
const WasmEdge_Value* in, \
WasmEdge_Value* out); \
extern WasmEdge_ValType WasmFunctionParams##F[]; \
extern WasmEdge_ValType WasmFunctionResult##F[]; \
extern WasmEdge_FunctionTypeContext* WasmFunctionType##F; \
#define DECLARE_HOOK_FUNCTION(R, F, ...) \
std::variant<UNSIGNED_TYPE(R), hook_api::hook_return_code> F( \
hook::HookContext& hookCtx, \
WasmEdge_CallingFrameContext const& frameCtx __VA_OPT__( \
COMMA __VA_ARGS__)); \
extern WasmEdge_Result WasmFunction##F( \
void* data_ptr, \
const WasmEdge_CallingFrameContext* frameCtx, \
const WasmEdge_Value* in, \
WasmEdge_Value* out); \
extern WasmEdge_ValType WasmFunctionParams##F[]; \
extern WasmEdge_ValType WasmFunctionResult##F[]; \
extern WasmEdge_FunctionTypeContext* WasmFunctionType##F; \
extern WasmEdge_String WasmFunctionName##F;
#define DECLARE_JS_FUNCTION(R, F, ...) \
extern JSValue JSFunction##F( \
JSContext* ctx, JSValueConst this_val, int argc, JSValueConst* argv); \
const int JSFunctionParamCount##F = __VA_OPT__(HALF_COUNT(__VA_ARGS__) +) 0;
#define DEFINE_WASM_FUNCTION(R, F, ...) \
WasmEdge_Result hook_api::WasmFunction##F( \
void* data_ptr, \
const WasmEdge_CallingFrameContext* frameCtx, \
const WasmEdge_Value* in, \
WasmEdge_Value* out) \
{ \
__VA_OPT__(int _stack = 0;) \
__VA_OPT__(FOR_VARS(VAR_ASSIGN, 2, __VA_ARGS__);) \
hook::HookContext* hookCtx = \
reinterpret_cast<hook::HookContext*>(data_ptr); \
R return_code = hook_api::F( \
*hookCtx, \
*const_cast<WasmEdge_CallingFrameContext*>(frameCtx) \
__VA_OPT__(COMMA STRIP_TYPES(__VA_ARGS__))); \
if (return_code == RC_ROLLBACK || return_code == RC_ACCEPT) \
return WasmEdge_Result_Terminate; \
out[0] = RET_ASSIGN(R, return_code); \
return WasmEdge_Result_Success; \
}; \
WasmEdge_ValType hook_api::WasmFunctionParams##F[] = { \
__VA_OPT__(FOR_VARS(WASM_VAL_TYPE, 0, __VA_ARGS__))}; \
WasmEdge_ValType hook_api::WasmFunctionResult##F[1] = { \
WASM_VAL_TYPE(R, dummy)}; \
WasmEdge_FunctionTypeContext* hook_api::WasmFunctionType##F = \
WasmEdge_FunctionTypeCreate( \
WasmFunctionParams##F, \
VA_NARGS(NULL __VA_OPT__(, __VA_ARGS__)), \
WasmFunctionResult##F, \
1); \
WasmEdge_String hook_api::WasmFunctionName##F = \
WasmEdge_StringCreateByCString(#F); \
R hook_api::F( \
hook::HookContext& hookCtx, \
WasmEdge_CallingFrameContext const& frameCtx __VA_OPT__( \
#define DEFINE_HOOK_FUNCTION(R, F, ...) \
WasmEdge_Result hook_api::WasmFunction##F( \
void* data_ptr, \
const WasmEdge_CallingFrameContext* frameCtx, \
const WasmEdge_Value* in, \
WasmEdge_Value* out) \
{ \
__VA_OPT__(int _stack = 0;) \
__VA_OPT__(FOR_VARS(VAR_ASSIGN, 2, __VA_ARGS__);) \
hook::HookContext* hookCtx = \
reinterpret_cast<hook::HookContext*>(data_ptr); \
auto const& return_code = hook_api::F( \
*hookCtx, \
*const_cast<WasmEdge_CallingFrameContext*>(frameCtx) \
__VA_OPT__(COMMA STRIP_TYPES(__VA_ARGS__))); \
if (std::holds_alternative<hook_api::hook_return_code>(return_code) && \
(std::get<hook_api::hook_return_code>(return_code) == \
RC_ROLLBACK || \
std::get<hook_api::hook_return_code>(return_code) == RC_ACCEPT)) \
return WasmEdge_Result_Terminate; \
out[0] = RET_ASSIGN( \
R, \
std::holds_alternative<UNSIGNED_TYPE(R)>(return_code) \
? std::get<UNSIGNED_TYPE(R)>(return_code) \
: R(std::get<hook_api::hook_return_code>(return_code))); \
return WasmEdge_Result_Success; \
}; \
WasmEdge_ValType hook_api::WasmFunctionParams##F[] = { \
__VA_OPT__(FOR_VARS(WASM_VAL_TYPE, 0, __VA_ARGS__))}; \
WasmEdge_ValType hook_api::WasmFunctionResult##F[1] = { \
WASM_VAL_TYPE(R, dummy)}; \
WasmEdge_FunctionTypeContext* hook_api::WasmFunctionType##F = \
WasmEdge_FunctionTypeCreate( \
WasmFunctionParams##F, \
VA_NARGS(NULL __VA_OPT__(, __VA_ARGS__)), \
WasmFunctionResult##F, \
1); \
WasmEdge_String hook_api::WasmFunctionName##F = \
WasmEdge_StringCreateByCString(#F); \
std::variant<UNSIGNED_TYPE(R), hook_api::hook_return_code> hook_api::F( \
hook::HookContext& hookCtx, \
WasmEdge_CallingFrameContext const& frameCtx __VA_OPT__( \
COMMA __VA_ARGS__))
#define VAR_JSASSIGN(T, V) \
if (_stack >= argc) \
returnJS(INVALID_ARGUMENT); \
T& V = argv[_stack++]
#define FORWARD_JS_FUNCTION_CALL(F, ac, av) \
hook_api::JSFunction##F(ctx, this_val, ac, av)
#define DEFINE_JS_FUNCTION(R, F, ...) \
JSValue hook_api::JSFunction##F( \
JSContext* ctx, JSValueConst this_val, int argc, JSValueConst* argv) \
{ \
__VA_OPT__(int _stack = 0;) \
__VA_OPT__(FOR_VARS(VAR_JSASSIGN, 2, __VA_ARGS__);)
#define JS_HOOK_SETUP() \
JSRuntime* rt = JS_GetRuntime(ctx); \
hook::HookContext& hookCtx = \
*reinterpret_cast<hook::HookContext*>(JS_GetRuntimeOpaque(rt)); \
[[maybe_unused]] ApplyContext& applyCtx = hookCtx.applyCtx; \
[[maybe_unused]] auto& view = applyCtx.view(); \
[[maybe_unused]] auto j = applyCtx.app.journal("View"); \
[[maybe_unused]] auto& api = hookCtx.api(); \
try \
{
#define JS_HOOK_TEARDOWN() \
} \
catch (const std::exception& e) \
{ \
JLOG(hookCtx.applyCtx.app.journal("View").error()) \
<< "HookError[" << HC_ACC() << "]: (JS) " << __func__ \
<< " threw uncaught exception, what=" << e.what(); \
return JS_NewInt64(ctx, INTERNAL_ERROR); \
} \
}
#define WASM_HOOK_SETUP() \
#define HOOK_SETUP() \
using enum hook_api::hook_return_code; \
try \
{ \
[[maybe_unused]] ApplyContext& applyCtx = hookCtx.applyCtx; \
@@ -274,14 +168,14 @@
if (!memoryCtx || !memory || !memory_length) \
return INTERNAL_ERROR;
#define WASM_HOOK_TEARDOWN() \
} \
catch (const std::exception& e) \
{ \
JLOG(hookCtx.applyCtx.app.journal("View").error()) \
<< "HookError[" << HC_ACC() << "]: (WASM) " << __func__ \
<< " threw uncaught exception, what=" << e.what(); \
return INTERNAL_ERROR; \
#define HOOK_TEARDOWN() \
} \
catch (const std::exception& e) \
{ \
JLOG(hookCtx.applyCtx.app.journal("View").error()) \
<< "HookError[" << HC_ACC() << "]: " << __func__ \
<< " threw uncaught exception, what=" << e.what(); \
return INTERNAL_ERROR; \
}
#define WRITE_WASM_MEMORY( \
@@ -320,7 +214,7 @@
host_memory_ptr, \
guest_memory_length) \
{ \
int64_t bytes_written = 0; \
uint64_t bytes_written = 0; \
WRITE_WASM_MEMORY( \
bytes_written, \
guest_dst_ptr, \
@@ -375,28 +269,6 @@
: RC_ROLLBACK); \
}
#define HOOK_EXIT_JS(error_msg, error_code, exit_type) \
{ \
int64_t val = 0; \
if (JS_IsNumber(error_code)) \
JS_ToInt64(ctx, &val, error_code); \
hookCtx.result.exitCode = val; \
hookCtx.result.exitType = exit_type; \
if (JS_IsString(error_msg)) \
{ \
size_t len; \
const char* cstr = JS_ToCStringLen(ctx, &len, error_msg); \
if (len > 256) \
len = 256; \
hookCtx.result.exitReason = std::string(cstr, len); \
JS_FreeCString(ctx, cstr); \
} \
return JS_Exit( \
ctx, \
exit_type == hook_api::ExitType::ACCEPT ? "HookExit Accept" \
: "HookExit Rollback"); \
}
#define WRITE_WASM_MEMORY_OR_RETURN_AS_INT64( \
write_ptr_in, write_len_in, data_ptr_in, data_len_in, is_account_in) \
{ \
@@ -411,7 +283,7 @@
data_ptr < (data_ptr_in)) \
return INTERNAL_ERROR; \
if (data_len == 0) \
return 0; \
return 0ULL; \
if ((write_ptr_in) == 0) \
return data_as_int64(data_ptr, data_len); \
if (data_len > (write_len_in)) \

View File

@@ -7,559 +7,311 @@ HOOK_API_DEFINITION(
HOOK_API_DEFINITION(
int64_t, accept, (uint32_t, uint32_t, int64_t),
uint256{})
// function accept(error_msg: string, error_code: number): number
JSHOOK_API_DEFINITION(
JSValue, accept, (JSValue, JSValue),
uint256{})
// int64_t rollback(uint32_t read_ptr, uint32_t read_len, int64_t error_code);
HOOK_API_DEFINITION(
int64_t, rollback, (uint32_t, uint32_t, int64_t),
uint256{})
// function rollback(error_msg: string, error_code: number): number
JSHOOK_API_DEFINITION(
JSValue, rollback, (JSValue, JSValue),
uint256{})
// int64_t util_raddr(uint32_t write_ptr, uint32_t write_len, uint32_t read_ptr, uint32_t read_len);
HOOK_API_DEFINITION(
int64_t, util_raddr, (uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function util_raddr(acc_id:string): string
JSHOOK_API_DEFINITION(
JSValue, util_raddr, (JSValue),
uint256{})
// int64_t util_accid(uint32_t write_ptr, uint32_t write_len, uint32_t read_ptr, uint32_t read_len);
HOOK_API_DEFINITION(
int64_t, util_accid, (uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function util_accid(acc_id:string): string
JSHOOK_API_DEFINITION(
JSValue, util_accid, (JSValue),
uint256{})
// int64_t util_verify(uint32_t dread_ptr, uint32_t dread_len, uint32_t sread_ptr, uint32_t sread_len, uint32_t kread_ptr, uint32_t kread_len);
HOOK_API_DEFINITION(
int64_t, util_verify, (uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function util_verify(data:string, sig: string, pubkey: string): number
JSHOOK_API_DEFINITION(
JSValue, util_verify, (JSValue, JSValue, JSValue),
uint256{})
// int64_t util_sha512h(uint32_t write_ptr, uint32_t write_len, uint32_t read_ptr, uint32_t read_len);
HOOK_API_DEFINITION(
int64_t, util_sha512h, (uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function util_sha512h(data: string): number
JSHOOK_API_DEFINITION(
JSValue, util_sha512h, (JSValue),
uint256{})
// int64_t util_keylet(uint32_t write_ptr, uint32_t write_len, uint32_t keylet_type, uint32_t a, uint32_t b, uint32_t c, uint32_t d, uint32_t e, uint32_t f);
HOOK_API_DEFINITION(
int64_t, util_keylet, (uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function util_keylet(keylet_type: number, keylet_data: string): number
JSHOOK_API_DEFINITION(
JSValue, util_keylet, (JSValue, JSValue),
uint256{})
// int64_t sto_validate(uint32_t tread_ptr, uint32_t tread_len);
HOOK_API_DEFINITION(
int64_t, sto_validate, (uint32_t, uint32_t),
uint256{})
// function sto_validate(sto: string): number
JSHOOK_API_DEFINITION(
JSValue, sto_validate, (JSValue),
uint256{})
// int64_t sto_subfield(uint32_t read_ptr, uint32_t read_len, uint32_t field_id);
HOOK_API_DEFINITION(
int64_t, sto_subfield, (uint32_t, uint32_t, uint32_t),
uint256{})
// function sto_subfield(sto: string, field_id: number): number
JSHOOK_API_DEFINITION(
JSValue, sto_subfield, (JSValue, JSValue),
uint256{})
// int64_t sto_subarray(uint32_t read_ptr, uint32_t read_len, uint32_t array_id);
HOOK_API_DEFINITION(
int64_t, sto_subarray, (uint32_t, uint32_t, uint32_t),
uint256{})
// function sto_subarray(sto: string, array_id: number): number
JSHOOK_API_DEFINITION(
JSValue, sto_subarray, (JSValue, JSValue),
uint256{})
// int64_t sto_emplace(uint32_t write_ptr, uint32_t write_len, uint32_t sread_ptr, uint32_t sread_len, uint32_t fread_ptr, uint32_t fread_len, uint32_t field_id);
HOOK_API_DEFINITION(
int64_t, sto_emplace, (uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function sto_emplace(sto: string, field_bytes: number, field_id: number): number
JSHOOK_API_DEFINITION(
JSValue, sto_emplace, (JSValue, JSValue, JSValue),
uint256{})
// int64_t sto_erase(uint32_t write_ptr, uint32_t write_len, uint32_t read_ptr, uint32_t read_len, uint32_t field_id);
HOOK_API_DEFINITION(
int64_t, sto_erase, (uint32_t, uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function sto_erase(sto: string, field_id: number): number
JSHOOK_API_DEFINITION(
JSValue, sto_erase, (JSValue, JSValue),
uint256{})
// int64_t etxn_burden();
HOOK_API_DEFINITION(
int64_t, etxn_burden, (),
uint256{})
// function etxn_burden(): number
JSHOOK_API_DEFINITION(
JSValue, etxn_burden, (),
uint256{})
// int64_t etxn_details(uint32_t write_ptr, uint32_t write_len);
HOOK_API_DEFINITION(
int64_t, etxn_details, (uint32_t, uint32_t),
uint256{})
// function etxn_details(): number
JSHOOK_API_DEFINITION(
JSValue, etxn_details, (),
uint256{})
// int64_t etxn_fee_base(uint32_t read_ptr, uint32_t read_len);
HOOK_API_DEFINITION(
int64_t, etxn_fee_base, (uint32_t, uint32_t),
uint256{})
// function etxn_fee_base(txblob: string): number
JSHOOK_API_DEFINITION(
JSValue, etxn_fee_base, (JSValue),
uint256{})
// int64_t etxn_reserve(uint32_t count);
HOOK_API_DEFINITION(
int64_t, etxn_reserve, (uint32_t),
uint256{})
// function etxn_reserve(count: number): number
JSHOOK_API_DEFINITION(
JSValue, etxn_reserve, (JSValue),
uint256{})
// int64_t etxn_generation();
HOOK_API_DEFINITION(
int64_t, etxn_generation, (),
uint256{})
// function etxn_generation(): number
JSHOOK_API_DEFINITION(
JSValue, etxn_generation, (),
uint256{})
// int64_t etxn_nonce(uint32_t write_ptr, uint32_t write_len);
HOOK_API_DEFINITION(
int64_t, etxn_nonce, (uint32_t, uint32_t),
uint256{})
// function etxn_nonce(): number
JSHOOK_API_DEFINITION(
JSValue, etxn_nonce, (),
uint256{})
// int64_t emit(uint32_t write_ptr, uint32_t write_len, uint32_t read_ptr, uint32_t read_len);
HOOK_API_DEFINITION(
int64_t, emit, (uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function emit(txn: string): number
JSHOOK_API_DEFINITION(
JSValue, emit, (JSValue),
uint256{})
// int64_t float_set(int32_t exponent, int64_t mantissa);
HOOK_API_DEFINITION(
int64_t, float_set, (int32_t, int64_t),
uint256{})
// function float_set(e: number, m: number): number
JSHOOK_API_DEFINITION(
JSValue, float_set, (JSValue, JSValue),
uint256{})
// int64_t float_multiply(int64_t float1, int64_t float2);
HOOK_API_DEFINITION(
int64_t, float_multiply, (int64_t, int64_t),
uint256{})
// function float_multiply(f1: number, f2: number): number
JSHOOK_API_DEFINITION(
JSValue, float_multiply, (JSValue, JSValue),
uint256{})
// int64_t float_mulratio(int64_t float1, uint32_t round_up, uint32_t numerator, uint32_t denominator);
HOOK_API_DEFINITION(
int64_t, float_mulratio, (int64_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function float_mulratio(f1: number, round_up: number, numerator: number, denominator: number): number
JSHOOK_API_DEFINITION(
JSValue, float_mulratio, (JSValue, JSValue, JSValue, JSValue),
uint256{})
// int64_t float_negate(int64_t float1);
HOOK_API_DEFINITION(
int64_t, float_negate, (int64_t),
uint256{})
// function float_negate(f1: number): number
JSHOOK_API_DEFINITION(
JSValue, float_negate, (JSValue),
uint256{})
// int64_t float_compare(int64_t float1, int64_t float2, uint32_t mode);
HOOK_API_DEFINITION(
int64_t, float_compare, (int64_t, int64_t, uint32_t),
uint256{})
// function float_compare(f1: number, f2: number, mode: number): number
JSHOOK_API_DEFINITION(
JSValue, float_compare, (JSValue, JSValue, JSValue),
uint256{})
// int64_t float_sum(int64_t float1, int64_t float2);
HOOK_API_DEFINITION(
int64_t, float_sum, (int64_t, int64_t),
uint256{})
// function float_sum(f1: number, f2: number): number
JSHOOK_API_DEFINITION(
JSValue, float_sum, (JSValue, JSValue),
uint256{})
// int64_t float_sto(uint32_t write_ptr, uint32_t write_len, uint32_t cread_ptr, uint32_t cread_len, uint32_t iread_ptr, uint32_t iread_len, int64_t float1, uint32_t field_code);
HOOK_API_DEFINITION(
int64_t, float_sto, (uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, int64_t, uint32_t),
uint256{})
// function float_sto(cur: string, isu: string, float1: number, field_code: number): number
JSHOOK_API_DEFINITION(
JSValue, float_sto, (JSValue, JSValue, JSValue, JSValue),
uint256{})
// int64_t float_sto_set(uint32_t read_ptr, uint32_t read_len);
HOOK_API_DEFINITION(
int64_t, float_sto_set, (uint32_t, uint32_t),
uint256{})
// function float_sto_set(buf: string): number
JSHOOK_API_DEFINITION(
JSValue, float_sto_set, (JSValue),
uint256{})
// int64_t float_invert(int64_t float1);
HOOK_API_DEFINITION(
int64_t, float_invert, (int64_t),
uint256{})
// function float_invert(f1: number): number
JSHOOK_API_DEFINITION(
JSValue, float_invert, (JSValue),
uint256{})
// int64_t float_divide(int64_t float1, int64_t float2);
HOOK_API_DEFINITION(
int64_t, float_divide, (int64_t, int64_t),
uint256{})
// function float_divide(f1: number, f2: number): number
JSHOOK_API_DEFINITION(
JSValue, float_divide, (JSValue, JSValue),
uint256{})
// int64_t float_one();
HOOK_API_DEFINITION(
int64_t, float_one, (),
uint256{})
// function float_one(): number
JSHOOK_API_DEFINITION(
JSValue, float_one, (),
uint256{})
// int64_t float_mantissa(int64_t float1);
HOOK_API_DEFINITION(
int64_t, float_mantissa, (int64_t),
uint256{})
// function float_mantissa(f1: number): number
JSHOOK_API_DEFINITION(
JSValue, float_mantissa, (JSValue),
uint256{})
// int64_t float_sign(int64_t float1);
HOOK_API_DEFINITION(
int64_t, float_sign, (int64_t),
uint256{})
// function float_sign(f1: number): number
JSHOOK_API_DEFINITION(
JSValue, float_sign, (JSValue),
uint256{})
// int64_t float_int(int64_t float1, uint32_t decimal_places, uint32_t abs);
HOOK_API_DEFINITION(
int64_t, float_int, (int64_t, uint32_t, uint32_t),
uint256{})
// function float_int(f1: number, decimal_places: number, abs: number): number
JSHOOK_API_DEFINITION(
JSValue, float_int, (JSValue, JSValue, JSValue),
uint256{})
// int64_t float_log(int64_t float1);
HOOK_API_DEFINITION(
int64_t, float_log, (int64_t),
uint256{})
// function float_log(f1: number): number
JSHOOK_API_DEFINITION(
JSValue, float_log, (JSValue),
uint256{})
// int64_t float_root(int64_t float1, uint32_t n);
HOOK_API_DEFINITION(
int64_t, float_root, (int64_t, uint32_t),
uint256{})
// function float_root(f1: number, n: number): number
JSHOOK_API_DEFINITION(
JSValue, float_root, (JSValue, JSValue),
uint256{})
// int64_t fee_base();
HOOK_API_DEFINITION(
int64_t, fee_base, (),
uint256{})
// function fee_base(): number
JSHOOK_API_DEFINITION(
JSValue, fee_base, (),
uint256{})
// int64_t ledger_seq();
HOOK_API_DEFINITION(
int64_t, ledger_seq, (),
uint256{})
// function ledger_seq(): number
JSHOOK_API_DEFINITION(
JSValue, ledger_seq, (),
uint256{})
// int64_t ledger_last_time();
HOOK_API_DEFINITION(
int64_t, ledger_last_time, (),
uint256{})
// function ledger_last_time(): number
JSHOOK_API_DEFINITION(
JSValue, ledger_last_time, (),
uint256{})
// int64_t ledger_last_hash(uint32_t write_ptr, uint32_t write_len);
HOOK_API_DEFINITION(
int64_t, ledger_last_hash, (uint32_t, uint32_t),
uint256{})
// function ledger_last_hash(): number
JSHOOK_API_DEFINITION(
JSValue, ledger_last_hash, (),
uint256{})
// int64_t ledger_nonce(uint32_t write_ptr, uint32_t write_len);
HOOK_API_DEFINITION(
int64_t, ledger_nonce, (uint32_t, uint32_t),
uint256{})
// function ledger_nonce(): number
JSHOOK_API_DEFINITION(
JSValue, ledger_nonce, (),
uint256{})
// int64_t ledger_keylet(uint32_t write_ptr, uint32_t write_len, uint32_t lread_ptr, uint32_t lread_len, uint32_t hread_ptr, uint32_t hread_len);
HOOK_API_DEFINITION(
int64_t, ledger_keylet, (uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function ledger_keylet(low: string, high: string): number
JSHOOK_API_DEFINITION(
JSValue, ledger_keylet, (JSValue, JSValue),
uint256{})
// int64_t hook_account(uint32_t write_ptr, uint32_t write_len);
HOOK_API_DEFINITION(
int64_t, hook_account, (uint32_t, uint32_t),
uint256{})
// function hook_account(): number
JSHOOK_API_DEFINITION(
JSValue, hook_account, (),
uint256{})
// int64_t hook_hash(uint32_t write_ptr, uint32_t write_len, int32_t hook_no);
HOOK_API_DEFINITION(
int64_t, hook_hash, (uint32_t, uint32_t, int32_t),
uint256{})
// function hook_hash(hook_no: number): number
JSHOOK_API_DEFINITION(
JSValue, hook_hash, (JSValue),
uint256{})
// int64_t hook_param_set(uint32_t read_ptr, uint32_t read_len, uint32_t kread_ptr, uint32_t kread_len, uint32_t hread_ptr, uint32_t hread_len);
HOOK_API_DEFINITION(
int64_t, hook_param_set, (uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function hook_param_set(value: string, key: string, hash: string): number
JSHOOK_API_DEFINITION(
JSValue, hook_param_set, (JSValue, JSValue, JSValue),
uint256{})
// int64_t hook_param(uint32_t write_ptr, uint32_t write_len, uint32_t read_ptr, uint32_t read_len);
HOOK_API_DEFINITION(
int64_t, hook_param, (uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function hook_param(key: string): number
JSHOOK_API_DEFINITION(
JSValue, hook_param, (JSValue),
uint256{})
// int64_t hook_again();
HOOK_API_DEFINITION(
int64_t, hook_again, (),
uint256{})
// function hook_again(): number
JSHOOK_API_DEFINITION(
JSValue, hook_again, (),
uint256{})
// int64_t hook_skip(uint32_t read_ptr, uint32_t read_len, uint32_t flags);
HOOK_API_DEFINITION(
int64_t, hook_skip, (uint32_t, uint32_t, uint32_t),
uint256{})
// function hook_skip(hhash: string, flags: number): number
JSHOOK_API_DEFINITION(
JSValue, hook_skip, (JSValue, JSValue),
uint256{})
// int64_t hook_pos();
HOOK_API_DEFINITION(
int64_t, hook_pos, (),
uint256{})
// function hook_pos(): number
JSHOOK_API_DEFINITION(
JSValue, hook_pos, (),
uint256{})
// int64_t slot(uint32_t write_ptr, uint32_t write_len, uint32_t slot);
HOOK_API_DEFINITION(
int64_t, slot, (uint32_t, uint32_t, uint32_t),
uint256{})
// function slot(slot_no: number): number
JSHOOK_API_DEFINITION(
JSValue, slot, (JSValue),
uint256{})
// int64_t slot_clear(uint32_t slot);
HOOK_API_DEFINITION(
int64_t, slot_clear, (uint32_t),
uint256{})
// function slot_clear(slot_no: number): number
JSHOOK_API_DEFINITION(
JSValue, slot_clear, (JSValue),
uint256{})
// int64_t slot_count(uint32_t slot);
HOOK_API_DEFINITION(
int64_t, slot_count, (uint32_t),
uint256{})
// function slot_count(slot_no: number): number
JSHOOK_API_DEFINITION(
JSValue, slot_count, (JSValue),
uint256{})
// int64_t slot_set(uint32_t read_ptr, uint32_t read_len, uint32_t slot);
HOOK_API_DEFINITION(
int64_t, slot_set, (uint32_t, uint32_t, uint32_t),
uint256{})
// function slot_set(kl: string, slot_no: number): number
JSHOOK_API_DEFINITION(
JSValue, slot_set, (JSValue, JSValue),
uint256{})
// int64_t slot_size(uint32_t slot);
HOOK_API_DEFINITION(
int64_t, slot_size, (uint32_t),
uint256{})
// function slot_size(slot_no: number): number
JSHOOK_API_DEFINITION(
JSValue, slot_size, (JSValue),
uint256{})
// int64_t slot_subarray(uint32_t parent_slot, uint32_t array_id, uint32_t new_slot);
HOOK_API_DEFINITION(
int64_t, slot_subarray, (uint32_t, uint32_t, uint32_t),
uint256{})
// function slot_subarray(parent_slot_no: number, array_id: number, new_slot: number): number
JSHOOK_API_DEFINITION(
JSValue, slot_subarray, (JSValue, JSValue, JSValue),
uint256{})
// int64_t slot_subfield(uint32_t parent_slot, uint32_t field_id, uint32_t new_slot);
HOOK_API_DEFINITION(
int64_t, slot_subfield, (uint32_t, uint32_t, uint32_t),
uint256{})
// function slot_subfield(parent_slot_no: number, field_id: number, new_slot: number): number
JSHOOK_API_DEFINITION(
JSValue, slot_subfield, (JSValue, JSValue, JSValue),
uint256{})
// int64_t slot_type(uint32_t slot_no, uint32_t flags);
HOOK_API_DEFINITION(
int64_t, slot_type, (uint32_t, uint32_t),
uint256{})
// function slot_type(slot_no: number, flags: number): number
JSHOOK_API_DEFINITION(
JSValue, slot_type, (JSValue, JSValue),
uint256{})
// int64_t slot_float(uint32_t slot_no);
HOOK_API_DEFINITION(
int64_t, slot_float, (uint32_t),
uint256{})
// function slot_float(slot_no: number): number
JSHOOK_API_DEFINITION(
JSValue, slot_float, (JSValue),
uint256{})
// int64_t state_set(uint32_t read_ptr, uint32_t read_len, uint32_t kread_ptr, uint32_t kread_len);
HOOK_API_DEFINITION(
int64_t, state_set, (uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function state_set(val: string, key: string): number
JSHOOK_API_DEFINITION(
JSValue, state_set, (JSValue, JSValue),
uint256{})
// int64_t state_foreign_set(uint32_t read_ptr, uint32_t read_len, uint32_t kread_ptr, uint32_t kread_len, uint32_t nread_ptr, uint32_t nread_len, uint32_t aread_ptr, uint32_t aread_len);
HOOK_API_DEFINITION(
int64_t, state_foreign_set, (uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function state_foreign_set(val: string, key: string, ns: string, accid: string): number
JSHOOK_API_DEFINITION(
JSValue, state_foreign_set, (JSValue, JSValue, JSValue, JSValue),
uint256{})
// int64_t state(uint32_t write_ptr, uint32_t write_len, uint32_t kread_ptr, uint32_t kread_len);
HOOK_API_DEFINITION(
int64_t, state, (uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function state(key: string): number
JSHOOK_API_DEFINITION(
JSValue, state, (JSValue),
uint256{})
// int64_t state_foreign(uint32_t write_ptr, uint32_t write_len, uint32_t kread_ptr, uint32_t kread_len, uint32_t nread_ptr, uint32_t nread_len, uint32_t aread_ptr, uint32_t aread_len);
HOOK_API_DEFINITION(
int64_t, state_foreign, (uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function state_foreign(key: string, ns: string, accid: string): number
JSHOOK_API_DEFINITION(
JSValue, state_foreign, (JSValue, JSValue, JSValue),
uint256{})
// int64_t trace(uint32_t mread_ptr, uint32_t mread_len, uint32_t dread_ptr, uint32_t dread_len, uint32_t as_hex);
HOOK_API_DEFINITION(
int64_t, trace, (uint32_t, uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function trace(msg: string, data: string, as_hex: number): number
JSHOOK_API_DEFINITION(
JSValue, trace, (JSValue, JSValue, JSValue),
uint256{})
// int64_t trace_num(uint32_t read_ptr, uint32_t read_len, int64_t number);
HOOK_API_DEFINITION(
@@ -575,108 +327,73 @@ HOOK_API_DEFINITION(
HOOK_API_DEFINITION(
int64_t, otxn_burden, (),
uint256{})
// function otxn_burden(): number
JSHOOK_API_DEFINITION(
JSValue, otxn_burden, (),
uint256{})
// int64_t otxn_field(uint32_t write_ptr, uint32_t write_len, uint32_t field_id);
HOOK_API_DEFINITION(
int64_t, otxn_field, (uint32_t, uint32_t, uint32_t),
uint256{})
// function otxn_field(field_id: number): number
JSHOOK_API_DEFINITION(
JSValue, otxn_field, (JSValue),
uint256{})
// int64_t otxn_generation();
HOOK_API_DEFINITION(
int64_t, otxn_generation, (),
uint256{})
// function otxn_generation(): number
JSHOOK_API_DEFINITION(
JSValue, otxn_generation, (),
uint256{})
// int64_t otxn_id(uint32_t write_ptr, uint32_t write_len, uint32_t flags);
HOOK_API_DEFINITION(
int64_t, otxn_id, (uint32_t, uint32_t, uint32_t),
uint256{})
// function otxn_id(flags: number): number
JSHOOK_API_DEFINITION(
JSValue, otxn_id, (JSValue),
uint256{})
// int64_t otxn_type();
HOOK_API_DEFINITION(
int64_t, otxn_type, (),
uint256{})
// function otxn_type(): number
JSHOOK_API_DEFINITION(
JSValue, otxn_type, (),
uint256{})
// int64_t otxn_slot(uint32_t slot_no);
HOOK_API_DEFINITION(
int64_t, otxn_slot, (uint32_t),
uint256{})
// function otxn_slot(slot_no: number): number
JSHOOK_API_DEFINITION(
JSValue, otxn_slot, (JSValue),
uint256{})
// int64_t otxn_param(uint32_t write_ptr, uint32_t write_len, uint32_t read_ptr, uint32_t read_len);
HOOK_API_DEFINITION(
int64_t, otxn_param, (uint32_t, uint32_t, uint32_t, uint32_t),
uint256{})
// function otxn_param(param_key: string): number
JSHOOK_API_DEFINITION(
JSValue, otxn_param, (JSValue),
uint256{})
// int64_t meta_slot(uint32_t slot_no);
HOOK_API_DEFINITION(
int64_t, meta_slot, (uint32_t),
uint256{})
// function meta_slot(slot_no: number): number
JSHOOK_API_DEFINITION(
JSValue, meta_slot, (JSValue),
uint256{})
// int64_t xpop_slot(uint32_t slot_no_tx, uint32_t slot_no_meta);
HOOK_API_DEFINITION(
int64_t, xpop_slot, (uint32_t, uint32_t),
featureHooksUpdate1)
// function xpop_slot(slot_no_tx: number, slot_no_meta: number): number
JSHOOK_API_DEFINITION(
JSValue, xpop_slot, (JSValue, JSValue),
uint256{})
// int64_t prepare(uint32_t write_ptr, uint32_t write_len, uint32_t read_ptr, uint32_t read_len);
HOOK_API_DEFINITION(
int64_t, prepare, (uint32_t, uint32_t, uint32_t, uint32_t),
featureHooksUpdate2)
// function prepare(tmpl: string): number
JSHOOK_API_DEFINITION(
JSValue, prepare, (JSValue),
uint256{})
// function otxn_json(): number
JSHOOK_API_DEFINITION(
JSValue, otxn_json, (),
uint256{})
// int64_t xport_reserve(uint32_t count);
HOOK_API_DEFINITION(
int64_t, xport_reserve, (uint32_t),
featureExport)
// function slot_json(slot_no: number): number
JSHOOK_API_DEFINITION(
JSValue, slot_json, (JSValue),
uint256{})
// int64_t xport(uint32_t write_ptr, uint32_t write_len, uint32_t read_ptr, uint32_t read_len);
HOOK_API_DEFINITION(
int64_t, xport, (uint32_t, uint32_t, uint32_t, uint32_t),
featureExport)
// function sto_to_json(sto_in: string): number
JSHOOK_API_DEFINITION(
JSValue, sto_to_json, (JSValue),
uint256{})
// int64_t xport_cancel(uint32_t ticket_seq);
HOOK_API_DEFINITION(
int64_t, xport_cancel, (uint32_t),
featureExport)
// function sto_from_json(json_in: string): number
JSHOOK_API_DEFINITION(
JSValue, sto_from_json, (JSValue),
uint256{})
// int64_t dice(uint32_t sides, uint32_t min_tier, uint32_t min_count);
HOOK_API_DEFINITION(
int64_t, dice, (uint32_t, uint32_t, uint32_t),
featureConsensusEntropy)
// int64_t random(uint32_t write_ptr, uint32_t write_len, uint32_t min_tier, uint32_t min_count);
HOOK_API_DEFINITION(
int64_t, random, (uint32_t, uint32_t, uint32_t, uint32_t),
featureConsensusEntropy)

View File

@@ -0,0 +1,2 @@
---
DisableFormat: true

View File

@@ -153,7 +153,11 @@ message TMStatusChange
message TMProposeSet
{
required uint32 proposeSeq = 1;
required bytes currentTxHash = 2; // the hash of the ledger we are proposing
// Proposed transaction-set identity. Legacy/plain proposals carry the
// tx-set hash directly; ConsensusExtensions proposals carry a serialized
// ExtendedPosition whose first field is that tx-set hash, followed by
// signed RNG/Export sidecar fields.
required bytes currentTxHash = 2;
required bytes nodePubKey = 3;
required uint32 closeTime = 4;
required bytes signature = 5; // signature of above fields
@@ -166,6 +170,14 @@ message TMProposeSet
// Number of hops traveled
optional uint32 hops = 12 [deprecated=true];
// Export signatures for pending exports seen in the proposal set. The
// proposal's ExtendedPosition includes a digest of this repeated field, so
// these side-channel blobs are covered by the proposal signature.
// Each entry is: txnHash (32 bytes) + validator pubkey (33 bytes)
// + multisign signature (variable length). Validators attach these
// so export quorum can be reached within the same consensus round.
repeated bytes exportSignatures = 13;
}
enum TxSetStatus
@@ -384,4 +396,3 @@ message TMHaveTransactions
{
repeated bytes hashes = 1;
}

View File

@@ -0,0 +1,62 @@
#ifndef RIPPLE_PROTOCOL_ENTROPY_TIER_H_INCLUDED
#define RIPPLE_PROTOCOL_ENTROPY_TIER_H_INCLUDED
#include <cstdint>
namespace ripple {
/// Which gate the ledger's entropy passed. Stored in sfEntropyTier (UINT8)
/// on the ttCONSENSUS_ENTROPY pseudo-transaction and the ConsensusEntropy
/// ledger entry.
///
/// EntropyCount says how many validators contributed; EntropyDenominator says
/// how many active validators were in the ledger-anchored view for that
/// non-fallback result; EntropyTier says which gate the result passed. Fallback
/// entropy carries count=0/denominator=0 because no validator-derived
/// denominator was accepted. Tier values are strength-ordered so consumers can
/// gate with a numeric comparison (tier >= required).
///
/// RESIDUAL BIAS — applies to fallback, participant_aligned, and
/// validator_quorum. This is a commit/reveal scheme: a validator can withhold
/// its reveal until after observing peers' reveals, choosing between two
/// outcomes (its contribution in vs. out) — up to one bit of influence per
/// withholder, and colluding withholders near a threshold can instead force a
/// downgrade to a lower tier. These tiers bound and *label* manipulation (it is
/// observable and limited); they are NOT bias-resistant against a colluding
/// validator minority. A hook that requires validator_full fails closed if any
/// active validator withholds, trading availability for no selective-
/// withholding slack.
enum EntropyTier : std::uint8_t {
/// No usable entropy (reserved; a fresh ConsensusEntropy entry should
/// always carry one of the tiers below).
entropyTierNone = 0,
/// Consensus-bound deterministic fallback: derived from already-agreed
/// round inputs (parent ledger hash, base tx set hash, sequence) under
/// HashPrefix::entropyFallback when no agreed reveal set reaches either
/// participant_aligned or validator_quorum. Unpredictable in practice but
/// user-influenceable via transaction submission — never suitable for
/// value-bearing outcomes.
entropyTierConsensusFallback = 1,
/// Participant-aligned sub-quorum entropy: the agreed reveal set aligned at
/// the tier-2 participant threshold — below the 80% validator quorum but at
/// or above the equivocation-intersection floor over the original
/// (pre-nUNL)
/// view. Weaker than validator_quorum; opt-in for hooks via min_tier.
entropyTierParticipantAligned = 2,
/// Validator commit/reveal entropy whose sidecar set passed the
/// active-validator-view quorum alignment gate.
entropyTierValidatorQuorum = 3,
/// Validator commit/reveal entropy with reveals from every validator in
/// the ledger-anchored active view. Any missing active validator downgrades
/// the tier, so hooks can require this to fail closed on selective
/// withholding.
entropyTierValidatorFull = 4,
};
} // namespace ripple
#endif

View File

@@ -0,0 +1,53 @@
#ifndef RIPPLE_PROTOCOL_EXPORT_LIMITS_H_INCLUDED
#define RIPPLE_PROTOCOL_EXPORT_LIMITS_H_INCLUDED
#include <cstddef>
#include <cstdint>
namespace ripple {
// Export system caps.
//
// These limits bound the DoS surface of the export signature system:
// - Each pending export requires every validator to sign it every round
// (sign-once, attach once via TMProposeSet)
// - Inbound signature processing involves crypto verification per sig
// - The open-ledger cap (maxPendingExports) is the root constraint;
// signing throughput and inbound processing are transitively bounded by it
struct ExportLimits
{
// Maximum exports a single hook execution may produce. Hook API ABI
// constant hook_api::max_export must stay equal.
static constexpr std::uint8_t maxExportsPerHook = 2;
// Maximum pending export transactions in an open/apply ledger.
// Hook-emitted export backlog drains into the open ledger at this cap.
// This transitively caps:
// - signatures per TMProposeSet message (1 per pending export)
// - inbound proposal signature processing (clamped to this)
// - validator signing work per round
static constexpr std::uint8_t maxPendingExports = 8;
// Maximum number of ledgers a pending export may retry before its
// mandatory LastLedgerSequence expires. This bounds validator signing work
// for both hook-emitted and user-submitted exports.
static constexpr std::uint32_t maxRetryLedgers = 5;
// Maximum byte length of a single export-signature wire blob:
// txHash(32) + validator pubkey(33) + multisign signature(<= 72).
// A fully-canonical secp256k1 signature is at most 72 bytes (ed25519 is
// 64), so 137 is the true upper bound for a well-formed entry. The proposal
// ingress path hashes these blobs BEFORE the proposal signature is
// verified, so bounding the per-blob size caps pre-auth hashing/copy work
// (DoS).
static constexpr std::size_t maxExportSignatureBytes = 32 + 33 + 72;
// Export-signature sidecar leaves wrap one export-signature blob in an
// STObject envelope. Keep this comfortably above the canonical encoding
// while bounding fetched, peer-supplied leaf bytes before parse/hash work.
static constexpr std::size_t maxExportSignatureSidecarBytes = 256;
};
} // namespace ripple
#endif

View File

@@ -33,35 +33,39 @@
*
* Steps required to add new features to the code:
*
* 1) In this file, increment `numFeatures` and add a uint256 declaration
* for the feature at the bottom
* 2) Add a uint256 definition for the feature to the corresponding source
* file (Feature.cpp). Use `registerFeature` to create the feature with
* the feature's name, `Supported::no`, and `VoteBehavior::DefaultNo`. This
* should be the only place the feature's name appears in code as a string.
* 3) Use the uint256 as the parameter to `view.rules.enabled()` to
* control flow into new code that this feature limits.
* 4) If the feature development is COMPLETE, and the feature is ready to be
* SUPPORTED, change the `registerFeature` parameter to Supported::yes.
* 5) When the feature is ready to be ENABLED, change the `registerFeature`
* parameter to `VoteBehavior::DefaultYes`.
* In general, any newly supported amendments (`Supported::yes`) should have
* a `VoteBehavior::DefaultNo` for at least one full release cycle. High
* priority bug fixes can be an exception to this rule of thumb.
* 1) Add the appropriate XRPL_FEATURE or XRPL_FIX macro definition for the
* feature to features.macro with the feature's name, `Supported::no`, and
* `VoteBehavior::DefaultNo`.
*
* 2) Use the generated variable name as the parameter to `view.rules.enabled()`
* to control flow into new code that this feature limits. (featureName or
* fixName)
*
* 3) If the feature development is COMPLETE, and the feature is ready to be
* SUPPORTED, change the macro parameter in features.macro to Supported::yes.
*
* 4) In general, any newly supported amendments (`Supported::yes`) should have
* a `VoteBehavior::DefaultNo` indefinitely so that external governance can
* make the decision on when to activate it. High priority bug fixes can be
* an exception to this rule. In such cases, ensure the fix has been
* clearly communicated to the community using appropriate channels,
* then change the macro parameter in features.macro to
* `VoteBehavior::DefaultYes`. The communication process is beyond
* the scope of these instructions.
*
*
* When a feature has been enabled for several years, the conditional code
* may be removed, and the feature "retired". To retire a feature:
* 1) Remove the uint256 declaration from this file.
* 2) MOVE the uint256 definition in Feature.cpp to the "retired features"
* section at the end of the file.
* 3) CHANGE the name of the variable to start with "retired".
* 4) CHANGE the parameters of the `registerFeature` call to `Supported::yes`
* and `VoteBehavior::DefaultNo`.
*
* 1) MOVE the macro definition in features.macro to the "retired features"
* section at the end of the file, and change the macro to XRPL_RETIRE.
*
* The feature must remain registered and supported indefinitely because it
* still exists in the ledger, but there is no need to vote for it because
* there's nothing to vote for. If it is removed completely from the code, any
* instances running that code will get amendment blocked. Removing the
* feature from the ledger is beyond the scope of these instructions.
* may exist in the Amendments object on ledger. There is no need to vote
* for it because there's nothing to vote for. If the feature definition is
* removed completely from the code, any instances running that code will get
* amendment blocked. Removing the feature from the ledger is beyond the scope
* of these instructions.
*
*/
@@ -76,11 +80,32 @@ allAmendments();
namespace detail {
#pragma push_macro("XRPL_FEATURE")
#undef XRPL_FEATURE
#pragma push_macro("XRPL_FIX")
#undef XRPL_FIX
#pragma push_macro("XRPL_RETIRE")
#undef XRPL_RETIRE
#define XRPL_FEATURE(name, supported, vote) +1
#define XRPL_FIX(name, supported, vote) +1
#define XRPL_RETIRE(name) +1
// This value SHOULD be equal to the number of amendments registered in
// Feature.cpp. Because it's only used to reserve storage, and determine how
// large to make the FeatureBitset, it MAY be larger. It MUST NOT be less than
// the actual number of amendments. A LogicError on startup will verify this.
static constexpr std::size_t numFeatures = 116;
static constexpr std::size_t numFeatures =
(0 +
#include <xrpl/protocol/detail/features.macro>
);
#undef XRPL_RETIRE
#pragma pop_macro("XRPL_RETIRE")
#undef XRPL_FIX
#pragma pop_macro("XRPL_FIX")
#undef XRPL_FEATURE
#pragma pop_macro("XRPL_FEATURE")
/** Amendments that this server supports and the default voting behavior.
Whether they are enabled depends on the Rules defined in the validated
@@ -320,12 +345,17 @@ foreachFeature(FeatureBitset bs, F&& f)
#undef XRPL_FEATURE
#pragma push_macro("XRPL_FIX")
#undef XRPL_FIX
#pragma push_macro("XRPL_RETIRE")
#undef XRPL_RETIRE
#define XRPL_FEATURE(name, supported, vote) extern uint256 const feature##name;
#define XRPL_FIX(name, supported, vote) extern uint256 const fix##name;
#define XRPL_RETIRE(name)
#include <xrpl/protocol/detail/features.macro>
#undef XRPL_RETIRE
#pragma pop_macro("XRPL_RETIRE")
#undef XRPL_FIX
#pragma pop_macro("XRPL_FIX")
#undef XRPL_FEATURE

View File

@@ -96,6 +96,21 @@ enum class HashPrefix : std::uint32_t {
/** Credentials signature */
credential = detail::make_hash_prefix('C', 'R', 'D'),
/** consensus extension sidecar object */
sidecar = detail::make_hash_prefix('S', 'C', 'R'),
/** consensus diagnostic observed participant set */
observedParticipants = detail::make_hash_prefix('O', 'B', 'P'),
/** consensus-bound fallback entropy digest (Tier 1: derived from
already-agreed round inputs when no agreed reveal set reaches an
accepted validator-participant tier; never to be confused with
validator entropy) */
entropyFallback = detail::make_hash_prefix('E', 'F', 'B'),
/** consensus entropy transaction-ordering salt */
entropyTxnOrder = detail::make_hash_prefix('E', 'T', 'O'),
};
template <class Hasher>

View File

@@ -454,6 +454,15 @@ getVLInfo(Json::Value const& xpop, beast::Journal const& j)
<< "Import: unl blob was not valid json (after base64 decoding)";
return {};
}
auto const isNonNegativeUInt = [](Json::Value const& value) {
return value.isUInt() || (value.isInt() && value.asInt() >= 0);
};
if (!list.isMember(jss::sequence) ||
!isNonNegativeUInt(list[jss::sequence]))
{
JLOG(j.warn()) << "Import: unl blob sequence was missing or negative";
return {};
}
auto const sequence = list[jss::sequence].asUInt();
auto const m = deserializeManifest(base64_decode(
xpop[jss::validation][jss::unl][jss::manifest].asString()));

View File

@@ -62,6 +62,9 @@ emittedDir() noexcept;
Keylet
emittedTxn(uint256 const& id) noexcept;
Keylet
shadowTicket(AccountID const& account, std::uint32_t ticketSeq) noexcept;
Keylet
hookDefinition(uint256 const& hash) noexcept;
@@ -118,6 +121,10 @@ negativeUNL() noexcept;
Keylet const&
UNLReport() noexcept;
/** The (fixed) index of the object containing consensus-derived entropy. */
Keylet const&
consensusEntropy() noexcept;
/** The beginning of an order book */
struct book_t
{

View File

@@ -43,6 +43,19 @@ std::size_t constexpr txMinSizeBytes = 10;
/** Largest legal byte size of a transaction. */
std::size_t constexpr txMaxSizeBytes = megabytes(1);
/** Largest network ID whose transactions retain the legacy canonical form.
Transactions on these networks must omit sfNetworkID. Transactions on
networks with larger IDs must include sfNetworkID matching the network.
*/
std::uint32_t constexpr maxNetworkIDWithoutTxField = 1024;
constexpr bool
requiresTxNetworkID(std::uint32_t networkID)
{
return networkID > maxNetworkIDWithoutTxField;
}
/** The maximum number of unfunded offers to delete at once */
std::size_t constexpr unfundedOfferRemoveLimit = 1000;

View File

@@ -0,0 +1,21 @@
#ifndef RIPPLE_PROTOCOL_SIDECAR_TYPE_H_INCLUDED
#define RIPPLE_PROTOCOL_SIDECAR_TYPE_H_INCLUDED
#include <cstdint>
namespace ripple {
/// Discriminator for sidecar set entries (SHAMap leaves used for
/// consensus extension data: RNG commit/reveal, export signatures).
///
/// Stored in sfSidecarType (UINT8) on each STObject entry.
/// Makes sidecar sets self-describing — no content-sniffing needed.
enum SidecarType : std::uint8_t {
sidecarRngCommit = 1,
sidecarRngReveal = 2,
sidecarExportSig = 3,
};
} // namespace ripple
#endif

View File

@@ -68,7 +68,10 @@ enum TELcodes : TERUnderlyingType {
telNON_LOCAL_EMITTED_TXN,
telIMPORT_VL_KEY_NOT_RECOGNISED,
telCAN_NOT_QUEUE_IMPORT,
// Keep test-only sentinels before appended branch-local TEL codes so their
// numeric values do not move.
telENV_RPC_FAILED,
telSHADOW_TICKET_REQUIRED,
};
//------------------------------------------------------------------------------
@@ -137,6 +140,7 @@ enum TEMcodes : TERUnderlyingType {
temXCHAIN_BRIDGE_NONDOOR_OWNER,
temXCHAIN_BRIDGE_BAD_MIN_ACCOUNT_CREATE_AMOUNT,
temXCHAIN_BRIDGE_BAD_REWARD_AMOUNT,
temXCHAIN_TOO_MANY_ATTESTATIONS, // RESERVED - not used
temHOOK_DATA_TOO_LARGE,
temEMPTY_DID,
@@ -233,8 +237,10 @@ enum TERcodes : TERUnderlyingType {
terQUEUED, // Transaction is being held in TxQ until fee drops
terPRE_TICKET, // Ticket is not yet in ledger but might be on its way
terNO_AMM, // AMM doesn't exist for the asset pair
terNO_HOOK // Transaction requires a non-existent hook definition
terNO_HOOK, // Transaction requires a non-existent hook definition
// (referenced by sfHookHash)
terRETRY_EXPORT // Export does not yet have enough validator signatures.
// Retained in retriable set for next ledger.
};
//------------------------------------------------------------------------------
@@ -362,7 +368,7 @@ enum TECcodes : TERUnderlyingType {
tecARRAY_TOO_LARGE = 197,
tecLOCKED = 198,
tecBAD_CREDENTIALS = 199,
tecINVALID_HOOK_API_VERSION = 200,
tecEXPORT_EXPIRED = 200,
tecLAST_POSSIBLE_ENTRY = 255,
};

View File

@@ -274,6 +274,13 @@ enum BridgeModifyFlags : uint32_t {
tfClearAccountCreateAmount = 0x00010000,
};
constexpr std::uint32_t tfBridgeModifyMask = ~(tfUniversal | tfClearAccountCreateAmount);
// ConsensusEntropy flags (used on ttCONSENSUS_ENTROPY SHAMap entries):
enum ConsensusEntropyFlags : uint32_t {
tfEntropyCommit = 0x00000001, // entry is a commitment in commitSet
tfEntropyReveal = 0x00000002, // entry is a reveal in entropySet
};
// flag=0 (no tfEntropyCommit/tfEntropyReveal) = final injected pseudo-tx
// clang-format on
} // namespace ripple

View File

@@ -140,6 +140,12 @@ public:
mHookEmissions = hookEmissions;
}
void
setExportResult(STObject const& exportResult)
{
mExportResult = exportResult;
}
bool
hasHookExecutions() const
{
@@ -152,6 +158,12 @@ public:
return static_cast<bool>(mHookEmissions);
}
bool
hasExportResult() const
{
return static_cast<bool>(mExportResult);
}
STAmount
getDeliveredAmount() const
{
@@ -176,6 +188,7 @@ private:
std::optional<STAmount> mDelivered;
std::optional<STArray> mHookExecutions;
std::optional<STArray> mHookEmissions;
std::optional<STObject> mExportResult;
STArray mNodes;
};

View File

@@ -23,6 +23,9 @@
#if !defined(XRPL_FIX)
#error "undefined macro: XRPL_FIX"
#endif
#if !defined(XRPL_RETIRE)
#error "undefined macro: XRPL_RETIRE"
#endif
// clang-format off
@@ -31,7 +34,10 @@
// If you add an amendment here, then do not forget to increment `numFeatures`
// in include/xrpl/protocol/Feature.h.
XRPL_FEATURE(JSHooks, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FIX (HookMap, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FIX (GuardDepth32, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(NamedHooks, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(IOURewardClaim, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FIX (IOULockedBalanceInvariant, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FIX (ImportIssuer, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FEATURE(HookAPISerializedType240, Supported::yes, VoteBehavior::DefaultNo)
@@ -55,18 +61,20 @@ XRPL_FIX (FillOrKill, Supported::yes, VoteBehavior::DefaultYe
XRPL_FEATURE(DID, Supported::no, VoteBehavior::DefaultNo)
XRPL_FIX (DisallowIncomingV1, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FEATURE(XChainBridge, Supported::no, VoteBehavior::DefaultNo)
XRPL_FEATURE(AMM, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(AMM, Supported::no, VoteBehavior::DefaultNo)
XRPL_FIX (ReducedOffersV1, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FEATURE(HooksUpdate2, Supported::yes, VoteBehavior::DefaultNo);
XRPL_FEATURE(HookOnV2, Supported::yes, VoteBehavior::DefaultNo);
XRPL_FIX (HookAPI20251128, Supported::yes, VoteBehavior::DefaultYes);
XRPL_FIX (CronStacking, Supported::yes, VoteBehavior::DefaultYes);
XRPL_FEATURE(ExtendedHookState, Supported::yes, VoteBehavior::DefaultNo);
XRPL_FIX (InvalidTxFlags, Supported::yes, VoteBehavior::DefaultYes);
XRPL_FEATURE(Cron, Supported::yes, VoteBehavior::DefaultNo);
XRPL_FEATURE(IOUIssuerWeakTSH, Supported::yes, VoteBehavior::DefaultNo);
XRPL_FEATURE(DeepFreeze, Supported::yes, VoteBehavior::DefaultNo);
XRPL_FIX (ProvisionalDoubleThreading, Supported::yes, VoteBehavior::DefaultYes);
XRPL_FEATURE(HooksUpdate2, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(HookOnV2, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(Export, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(ConsensusEntropy, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FIX (HookAPI20251128, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FIX (CronStacking, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FEATURE(ExtendedHookState, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FIX (InvalidTxFlags, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FEATURE(Cron, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(IOUIssuerWeakTSH, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FEATURE(DeepFreeze, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FIX (ProvisionalDoubleThreading, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FEATURE(Clawback, Supported::yes, VoteBehavior::DefaultNo)
XRPL_FIX (RewardClaimFlags, Supported::yes, VoteBehavior::DefaultYes)
XRPL_FEATURE(HookCanEmit, Supported::yes, VoteBehavior::DefaultNo)
@@ -150,4 +158,24 @@ XRPL_FIX (NFTokenDirV1, Supported::yes, VoteBehavior::Obsolete)
XRPL_FEATURE(NonFungibleTokensV1, Supported::yes, VoteBehavior::Obsolete)
XRPL_FEATURE(CryptoConditionsSuite, Supported::yes, VoteBehavior::Obsolete)
// The following amendments have been active for at least two years. Their
// pre-amendment code has been removed and the identifiers are deprecated.
// All known amendments and amendments that may appear in a validated
// ledger must be registered either here or above with the "active" amendments
XRPL_RETIRE(MultiSign)
XRPL_RETIRE(TrustSetAuth)
XRPL_RETIRE(FeeEscalation)
XRPL_RETIRE(PayChan)
XRPL_RETIRE(CryptoConditions)
XRPL_RETIRE(TickSize)
XRPL_RETIRE(fix1368)
XRPL_RETIRE(Escrow)
XRPL_RETIRE(fix1373)
XRPL_RETIRE(EnforceInvariants)
XRPL_RETIRE(SortedDirectories)
XRPL_RETIRE(fix1201)
XRPL_RETIRE(fix1512)
XRPL_RETIRE(fix1523)
XRPL_RETIRE(fix1528)
// clang-format on

View File

@@ -93,7 +93,7 @@ LEDGER_ENTRY(ltCHECK, 0x0043, Check, check, ({
*/
LEDGER_ENTRY(ltHOOK_DEFINITION, 'D', HookDefinition, hook_definition, ({
{sfHookHash, soeREQUIRED},
{sfHookOn, soeREQUIRED},
{sfHookOn, soeOPTIONAL},
{sfHookOnIncoming, soeOPTIONAL},
{sfHookOnOutgoing, soeOPTIONAL},
{sfHookCanEmit, soeOPTIONAL},
@@ -223,6 +223,23 @@ LEDGER_ENTRY(ltURI_TOKEN, 0x0055, URIToken, uri_token, ({
{sfPreviousTxnLgrSeq, soeREQUIRED},
}))
/** The ledger object which stores consensus-derived entropy.
\note This is a singleton: only one such object exists in the ledger.
\sa keylet::consensusEntropy
*/
LEDGER_ENTRY_DUPLICATE(ltCONSENSUS_ENTROPY, 0x0058, ConsensusEntropy, consensus_entropy, ({
{sfDigest, soeREQUIRED},
{sfEntropyCount, soeREQUIRED},
{sfEntropyDenominator, soeREQUIRED},
{sfEntropyContributors, soeREQUIRED},
{sfEntropyTier, soeREQUIRED},
{sfLedgerSequence, soeREQUIRED},
{sfPreviousTxnID, soeREQUIRED},
{sfPreviousTxnLgrSeq, soeREQUIRED},
}))
/** A ledger object which describes an account.
\sa keylet::account
@@ -396,6 +413,8 @@ LEDGER_ENTRY(ltRIPPLE_STATE, 0x0072, RippleState, state, ({
{sfHighQualityOut, soeOPTIONAL},
{sfLockedBalance, soeOPTIONAL},
{sfLockCount, soeOPTIONAL},
{sfHighReward, soeOPTIONAL},
{sfLowReward, soeOPTIONAL},
}))
/** The ledger object which lists the network's fee settings.
@@ -590,6 +609,24 @@ LEDGER_ENTRY(ltDID, 0x008D, DID, did, ({
{sfPreviousTxnLgrSeq, soeREQUIRED},
}))
//@@start shadow-ticket-ledger-entry
/** A live callback latch for an exported ticket transaction.
Created when a transaction is exported. Consumed when
proof-of-execution is imported back. Account-owned (pays reserve).
This is not a permanent replay tombstone; hook callbacks that move
value must still deduplicate their business action.
\sa keylet::shadowTicket
*/
LEDGER_ENTRY(ltSHADOW_TICKET, 0x5374, ShadowTicket, shadow_ticket, ({
{sfAccount, soeREQUIRED},
{sfTicketSequence, soeREQUIRED},
{sfTransactionHash, soeREQUIRED},
{sfLedgerSequence, soeREQUIRED},
{sfOwnerNode, soeREQUIRED},
}))
//@@end shadow-ticket-ledger-entry
#undef EXPAND
#undef LEDGER_ENTRY_DUPLICATE

View File

@@ -42,6 +42,8 @@ TYPED_SFIELD(sfTickSize, UINT8, 16)
TYPED_SFIELD(sfUNLModifyDisabling, UINT8, 17)
TYPED_SFIELD(sfHookResult, UINT8, 18)
TYPED_SFIELD(sfWasLockingChainSend, UINT8, 19)
TYPED_SFIELD(sfSidecarType, UINT8, 20)
TYPED_SFIELD(sfEntropyTier, UINT8, 21)
// 16-bit integers (common)
TYPED_SFIELD(sfLedgerEntryType, UINT16, 1, SField::sMD_Never)
@@ -59,6 +61,9 @@ TYPED_SFIELD(sfHookExecutionIndex, UINT16, 19)
TYPED_SFIELD(sfHookApiVersion, UINT16, 20)
TYPED_SFIELD(sfHookStateScale, UINT16, 21)
TYPED_SFIELD(sfLedgerFixType, UINT16, 22)
TYPED_SFIELD(sfHookExportCount, UINT16, 98)
TYPED_SFIELD(sfEntropyCount, UINT16, 99)
TYPED_SFIELD(sfEntropyDenominator, UINT16, 100)
// 32-bit integers (common)
TYPED_SFIELD(sfNetworkID, UINT32, 1)
@@ -123,6 +128,7 @@ TYPED_SFIELD(sfImportSequence, UINT32, 97)
TYPED_SFIELD(sfRewardTime, UINT32, 98)
TYPED_SFIELD(sfRewardLgrFirst, UINT32, 99)
TYPED_SFIELD(sfRewardLgrLast, UINT32, 100)
TYPED_SFIELD(sfCancelTicketSequence, UINT32, 101)
// 64-bit integers (common)
TYPED_SFIELD(sfIndexNext, UINT64, 1)
@@ -210,6 +216,7 @@ TYPED_SFIELD(sfOfferID, UINT256, 34)
TYPED_SFIELD(sfEscrowID, UINT256, 35)
TYPED_SFIELD(sfURITokenID, UINT256, 36)
TYPED_SFIELD(sfDomainID, UINT256, 37)
TYPED_SFIELD(sfExportSignatureHash, UINT256, 38)
TYPED_SFIELD(sfHookOnOutgoing, UINT256, 93)
TYPED_SFIELD(sfHookOnIncoming, UINT256, 94)
TYPED_SFIELD(sfCron, UINT256, 95)
@@ -257,6 +264,7 @@ TYPED_SFIELD(sfPrice, AMOUNT, 28)
TYPED_SFIELD(sfSignatureReward, AMOUNT, 29)
TYPED_SFIELD(sfMinAccountCreateAmount, AMOUNT, 30)
TYPED_SFIELD(sfLPTokenBalance, AMOUNT, 31)
TYPED_SFIELD(sfTrustLineRewardAccumulator,AMOUNT, 99)
// variable length (common)
TYPED_SFIELD(sfPublicKey, VL, 1)
@@ -292,6 +300,8 @@ TYPED_SFIELD(sfAssetClass, VL, 29)
TYPED_SFIELD(sfProvider, VL, 30)
TYPED_SFIELD(sfMPTokenMetadata, VL, 31)
TYPED_SFIELD(sfCredentialType, VL, 32)
TYPED_SFIELD(sfEntropyContributors, VL, 33)
TYPED_SFIELD(sfHookName, VL, 97)
TYPED_SFIELD(sfRemarkValue, VL, 98)
TYPED_SFIELD(sfRemarkName, VL, 99)
@@ -340,6 +350,7 @@ TYPED_SFIELD(sfLockingChainIssue, ISSUE, 1)
TYPED_SFIELD(sfIssuingChainIssue, ISSUE, 2)
TYPED_SFIELD(sfAsset, ISSUE, 3)
TYPED_SFIELD(sfAsset2, ISSUE, 4)
TYPED_SFIELD(sfClaimCurrency, ISSUE, 5)
// bridge
TYPED_SFIELD(sfXChainBridge, XCHAIN_BRIDGE, 1)
@@ -379,6 +390,7 @@ UNTYPED_SFIELD(sfXChainClaimAttestationCollectionElement, OBJECT, 30)
UNTYPED_SFIELD(sfXChainCreateAccountAttestationCollectionElement, OBJECT, 31)
UNTYPED_SFIELD(sfPriceData, OBJECT, 32)
UNTYPED_SFIELD(sfCredential, OBJECT, 33)
UNTYPED_SFIELD(sfExportedTxn, OBJECT, 90)
UNTYPED_SFIELD(sfAmountEntry, OBJECT, 91)
UNTYPED_SFIELD(sfMintURIToken, OBJECT, 92)
UNTYPED_SFIELD(sfHookEmission, OBJECT, 93)
@@ -386,6 +398,9 @@ UNTYPED_SFIELD(sfImportVLKey, OBJECT, 94)
UNTYPED_SFIELD(sfActiveValidator, OBJECT, 95)
UNTYPED_SFIELD(sfGenesisMint, OBJECT, 96)
UNTYPED_SFIELD(sfRemark, OBJECT, 97)
UNTYPED_SFIELD(sfHighReward, OBJECT, 98)
UNTYPED_SFIELD(sfLowReward, OBJECT, 99)
UNTYPED_SFIELD(sfExportResult, OBJECT, 100)
// array of objects (common)
// ARRAY/1 is reserved for end of array

View File

@@ -500,6 +500,17 @@ TRANSACTION(ttPERMISSIONED_DOMAIN_DELETE, 72, PermissionedDomainDelete, ({
{sfDomainID, soeREQUIRED},
}))
//@@start export-transaction-types
/* User-submittable export: creates a cross-chain transaction for
validator signing. Retries via terRETRY_EXPORT until quorum.
Also supports shadow ticket cancellation via sfCancelTicketSequence.
At least one of sfExportedTxn or sfCancelTicketSequence must be present. */
TRANSACTION(ttEXPORT, 91, Export, ({
{sfExportedTxn, soeOPTIONAL},
{sfCancelTicketSequence, soeOPTIONAL},
}))
//@@end export-transaction-types
/* A pseudo-txn alarm signal for invoking a hook, emitted by validators after alarm set conditions are met */
TRANSACTION(ttCRON, 92, Cron, ({
{sfOwner, soeREQUIRED},
@@ -550,6 +561,7 @@ TRANSACTION(ttIMPORT, 97, Import, ({
* from a specified hook */
TRANSACTION(ttCLAIM_REWARD, 98, ClaimReward, ({
{sfIssuer, soeOPTIONAL},
{sfClaimCurrency, soeOPTIONAL},
}))
/** This transaction invokes a hook, providing arbitrary data. Essentially as a 0 drop payment. **/
@@ -605,3 +617,17 @@ TRANSACTION(ttUNL_REPORT, 104, UNLReport, ({
{sfActiveValidator, soeOPTIONAL},
{sfImportVLKey, soeOPTIONAL},
}))
TRANSACTION(ttCONSENSUS_ENTROPY, 105, ConsensusEntropy, ({
{sfLedgerSequence, soeREQUIRED},
{sfDigest, soeREQUIRED},
{sfEntropyCount, soeREQUIRED},
{sfEntropyDenominator, soeREQUIRED},
{sfEntropyContributors, soeREQUIRED},
{sfEntropyTier, soeREQUIRED},
}))
TRANSACTION(ttEXPORT_SIGNATURES, 106, ExportSignatures, ({
{sfLedgerSequence, soeREQUIRED},
{sfTransactionHash, soeREQUIRED},
}))

View File

@@ -76,6 +76,7 @@ JSS(Holder); // field.
JSS(HookApiVersion); // field
JSS(HookCanEmit); // field
JSS(HookHash); // field
JSS(HookName); // field
JSS(HookNamespace); // field
JSS(HookOn); // field
JSS(HookOnIncoming); // field

View File

@@ -109,14 +109,22 @@ public:
Consumer
newInboundEndpoint(beast::IP::Endpoint const& address)
{
//@@start rng-local-testnet-resource-bucket
// Inbound connections from the same IP normally share one
// resource bucket (port stripped) for DoS protection. For
// loopback addresses, preserve the port so local testnet nodes
// each get their own bucket instead of all sharing one.
auto const key = is_loopback(address) ? address : address.at_port(0);
//@@end rng-local-testnet-resource-bucket
Entry* entry(nullptr);
{
std::lock_guard _(lock_);
auto [resultIt, resultInserted] = table_.emplace(
std::piecewise_construct,
std::make_tuple(kindInbound, address.at_port(0)), // Key
std::make_tuple(m_clock.now())); // Entry
std::make_tuple(kindInbound, key),
std::make_tuple(m_clock.now()));
entry = &resultIt->second;
entry->key = &resultIt->first;

View File

@@ -11,11 +11,11 @@ echo "START BUILDING (HOST)"
echo "Cleaning previously built binary"
rm -f release-build/xahaud
BUILD_CORES=$(echo "scale=0 ; `nproc` / 1.337" | bc)
BUILD_CORES=$(echo "scale=0 ; $(nproc) / 1.337" | bc)
if [[ "$GITHUB_REPOSITORY" == "" ]]; then
#Default
BUILD_CORES=${BUILD_CORES:-8}
BUILD_CORES=${BUILD_CORES:-8}
fi
# Ensure still works outside of GH Actions by setting these to /dev/null
@@ -31,21 +31,19 @@ echo "-- GITHUB_SHA: $GITHUB_SHA"
echo "-- GITHUB_RUN_NUMBER: $GITHUB_RUN_NUMBER"
echo "-- CONTAINER_NAME: $CONTAINER_NAME"
which docker 2> /dev/null 2> /dev/null
if [ "$?" -eq "1" ]
then
which docker 2>/dev/null 2>/dev/null
if [ "$?" -eq "1" ]; then
echo 'Docker not found. Install it first.'
exit 1
fi
stat .git 2> /dev/null 2> /dev/null
if [ "$?" -eq "1" ]
then
stat .git 2>/dev/null 2>/dev/null
if [ "$?" -eq "1" ]; then
echo 'Run this inside the source directory. (.git dir not found).'
exit 1
fi
STATIC_CONTAINER=$(docker ps -a | grep $CONTAINER_NAME |wc -l)
STATIC_CONTAINER=$(docker ps -a | grep $CONTAINER_NAME | wc -l)
CACHE_VOLUME_NAME="xahau-release-builder-cache"
@@ -57,13 +55,14 @@ if false; then
docker stop $CONTAINER_NAME
else
echo "No static container, build on temp container"
rm -rf release-build;
mkdir -p release-build;
rm -rf release-build
mkdir -p release-build
docker volume create $CACHE_VOLUME_NAME
# Create inline Dockerfile with environment setup for build-full.sh
DOCKERFILE_CONTENT=$(cat <<'DOCKERFILE_EOF'
DOCKERFILE_CONTENT=$(
cat <<'DOCKERFILE_EOF'
FROM ghcr.io/phusion/holy-build-box:4.0.1-amd64
ARG BUILD_CORES=8
@@ -218,7 +217,7 @@ RUN /hbb_exe/activate-exec bash -c "ccache -M 100G && \
ln -s ../../bin/ccache /usr/lib64/ccache/c++"
DOCKERFILE_EOF
)
)
# Build custom Docker image
IMAGE_NAME="xahaud-builder:latest"
@@ -228,14 +227,14 @@ DOCKERFILE_EOF
if [[ "$GITHUB_REPOSITORY" == "" ]]; then
# Non GH, local building
echo "Non-GH runner, local building, temp container"
docker run -i --user 0:$(id -g) --rm -v /data/builds:/data/builds -v `pwd`:/io -v "$CACHE_VOLUME_NAME":/cache --network host "$IMAGE_NAME" /hbb_exe/activate-exec bash -c "source /opt/rh/gcc-toolset-11/enable && bash -x /io/build-full.sh '$GITHUB_REPOSITORY' '$GITHUB_SHA' '$BUILD_CORES' '$GITHUB_RUN_NUMBER'"
docker run -i --user 0:$(id -g) --rm -v /data/builds:/data/builds -v $(pwd):/io -v "$CACHE_VOLUME_NAME":/cache --network host "$IMAGE_NAME" /hbb_exe/activate-exec bash -c "source /opt/rh/gcc-toolset-11/enable && bash -x /io/build-full.sh '$GITHUB_REPOSITORY' '$GITHUB_SHA' '$BUILD_CORES' '$GITHUB_RUN_NUMBER'"
else
# GH Action, runner
echo "GH Action, runner, clean & re-create create persistent container"
docker rm -f $CONTAINER_NAME
echo "echo 'Stopping container: $CONTAINER_NAME'" >> "$JOB_CLEANUP_SCRIPT"
echo "docker stop --time=15 \"$CONTAINER_NAME\" || echo 'Failed to stop container or container not running'" >> "$JOB_CLEANUP_SCRIPT"
docker run -di --user 0:$(id -g) --name $CONTAINER_NAME -v /data/builds:/data/builds -v `pwd`:/io -v "$CACHE_VOLUME_NAME":/cache --network host "$IMAGE_NAME" /hbb_exe/activate-exec bash
echo "echo 'Stopping container: $CONTAINER_NAME'" >>"$JOB_CLEANUP_SCRIPT"
echo "docker stop --time=15 \"$CONTAINER_NAME\" || echo 'Failed to stop container or container not running'" >>"$JOB_CLEANUP_SCRIPT"
docker run -di --user 0:$(id -g) --name $CONTAINER_NAME -v /data/builds:/data/builds -v $(pwd):/io -v "$CACHE_VOLUME_NAME":/cache --network host "$IMAGE_NAME" /hbb_exe/activate-exec bash
docker exec -i $CONTAINER_NAME /hbb_exe/activate-exec bash -c "source /opt/rh/gcc-toolset-11/enable && bash -x /io/build-full.sh '$GITHUB_REPOSITORY' '$GITHUB_SHA' '$BUILD_CORES' '$GITHUB_RUN_NUMBER'"
docker stop $CONTAINER_NAME
fi

View File

@@ -31,6 +31,7 @@
#include <cassert>
#include <cstring>
#include <ctime>
#include <exception>
#include <fstream>
#include <functional>
#include <iostream>
@@ -351,9 +352,18 @@ Logs::format(
if (useLocalTime)
{
auto now = std::chrono::system_clock::now();
auto local = date::make_zoned(date::current_zone(), now);
output = date::format(fmt, local);
try
{
auto now = std::chrono::system_clock::now();
auto local = date::make_zoned(date::current_zone(), now);
output = date::format(fmt, local);
}
catch (std::exception const&)
{
// Enhanced logging should not make startup fatal if tzdb lookup is
// unavailable or misconfigured. Fall back to UTC formatting.
output = date::format(fmt, std::chrono::system_clock::now());
}
}
else
{

View File

@@ -66,7 +66,7 @@ invalidAMMAsset(
Issue const& issue,
std::optional<std::pair<Issue, Issue>> const& pair)
{
if (badCurrency() == issue.currency)
if (isBadCurrency(issue.currency))
return temBAD_CURRENCY;
if (isXRP(issue) && issue.account.isNonZero())
return temBAD_ISSUER;

View File

@@ -250,12 +250,9 @@ FeatureCollections::registerFeature(
Feature const* i = getByName(name);
if (!i)
{
// If this check fails, and you just added a feature, increase the
// numFeatures value in Feature.h
check(
features.size() < detail::numFeatures,
"More features defined than allocated. Adjust numFeatures in "
"Feature.h.");
"More features defined than allocated.");
auto const f = sha512Half(Slice(name.data(), name.size()));
@@ -424,45 +421,26 @@ featureToName(uint256 const& f)
#undef XRPL_FEATURE
#pragma push_macro("XRPL_FIX")
#undef XRPL_FIX
#pragma push_macro("XRPL_RETIRE")
#undef XRPL_RETIRE
#define XRPL_FEATURE(name, supported, vote) \
uint256 const feature##name = registerFeature(#name, supported, vote);
#define XRPL_FIX(name, supported, vote) \
uint256 const fix##name = registerFeature("fix" #name, supported, vote);
#define XRPL_RETIRE(name) \
[[deprecated("The referenced amendment has been retired"), maybe_unused]] \
uint256 const retired##name = retireFeature(#name);
#include <xrpl/protocol/detail/features.macro>
#undef XRPL_RETIRE
#pragma pop_macro("XRPL_RETIRE")
#undef XRPL_FIX
#pragma pop_macro("XRPL_FIX")
#undef XRPL_FEATURE
#pragma pop_macro("XRPL_FEATURE")
// clang-format off
// The following amendments have been active for at least two years. Their
// pre-amendment code has been removed and the identifiers are deprecated.
// All known amendments and amendments that may appear in a validated
// ledger must be registered either here or above with the "active" amendments
[[deprecated("The referenced amendment has been retired"), maybe_unused]]
uint256 const
retiredMultiSign = retireFeature("MultiSign"),
retiredTrustSetAuth = retireFeature("TrustSetAuth"),
retiredFeeEscalation = retireFeature("FeeEscalation"),
retiredPayChan = retireFeature("PayChan"),
retiredCryptoConditions = retireFeature("CryptoConditions"),
retiredTickSize = retireFeature("TickSize"),
retiredFix1368 = retireFeature("fix1368"),
retiredEscrow = retireFeature("Escrow"),
retiredFix1373 = retireFeature("fix1373"),
retiredEnforceInvariants = retireFeature("EnforceInvariants"),
retiredSortedDirectories = retireFeature("SortedDirectories"),
retiredFix1201 = retireFeature("fix1201"),
retiredFix1512 = retireFeature("fix1512"),
retiredFix1523 = retireFeature("fix1523"),
retiredFix1528 = retireFeature("fix1528");
// clang-format on
// All of the features should now be registered, since variables in a cpp file
// are initialized from top to bottom.
//

View File

@@ -74,6 +74,7 @@ enum class LedgerNameSpace : std::uint16_t {
HOOK_DEFINITION = 'D',
EMITTED_TXN = 'E',
EMITTED_DIR = 'F',
SHADOW_TICKET = 0x5374, // St
NFTOKEN_OFFER = 'q',
NFTOKEN_BUY_OFFERS = 'h',
NFTOKEN_SELL_OFFERS = 'i',
@@ -81,6 +82,7 @@ enum class LedgerNameSpace : std::uint16_t {
IMPORT_VLSEQ = 'I',
UNL_REPORT = 'R',
CRON = 'L',
CONSENSUS_ENTROPY = 'X',
AMM = 'A',
BRIDGE = LEDGER_NAMESPACE2(0x01, 'H'),
XCHAIN_CLAIM_ID = 'Q',
@@ -188,6 +190,15 @@ emittedTxn(uint256 const& id) noexcept
return {ltEMITTED_TXN, indexHash(LedgerNameSpace::EMITTED_TXN, id)};
}
Keylet
shadowTicket(AccountID const& account, std::uint32_t ticketSeq) noexcept
{
return {
ltSHADOW_TICKET,
indexHash(
LedgerNameSpace::SHADOW_TICKET, account, std::uint32_t(ticketSeq))};
}
Keylet
hook(AccountID const& id) noexcept
{
@@ -546,6 +557,14 @@ cron(uint32_t timestamp, std::optional<AccountID> const& id)
return {ltCRON, uint256::fromVoid(h)};
}
Keylet const&
consensusEntropy() noexcept
{
static Keylet const ret{
ltCONSENSUS_ENTROPY, indexHash(LedgerNameSpace::CONSENSUS_ENTROPY)};
return ret;
}
Keylet
amm(Asset const& issue1, Asset const& issue2) noexcept
{

View File

@@ -78,6 +78,7 @@ InnerObjectFormats::InnerObjectFormats()
{sfHookExecutionIndex, soeREQUIRED},
{sfHookStateChangeCount, soeREQUIRED},
{sfHookEmitCount, soeREQUIRED},
{sfHookExportCount, soeOPTIONAL},
{sfFlags, soeOPTIONAL}});
add(sfHookEmission.jsonName,
@@ -99,7 +100,7 @@ InnerObjectFormats::InnerObjectFormats()
{sfHookOnOutgoing, soeOPTIONAL},
{sfHookCanEmit, soeOPTIONAL},
{sfHookApiVersion, soeOPTIONAL},
{sfFee, soeOPTIONAL},
{sfHookName, soeOPTIONAL},
{sfFlags, soeOPTIONAL}});
add(sfHookGrant.jsonName,
@@ -254,6 +255,24 @@ InnerObjectFormats::InnerObjectFormats()
{sfIssuer, soeREQUIRED},
{sfCredentialType, soeREQUIRED},
});
add(sfHighReward.jsonName,
sfHighReward.getCode(),
{
{sfRewardLgrFirst, soeREQUIRED},
{sfRewardLgrLast, soeREQUIRED},
{sfRewardTime, soeREQUIRED},
{sfTrustLineRewardAccumulator, soeREQUIRED},
});
add(sfLowReward.jsonName,
sfLowReward.getCode(),
{
{sfRewardLgrFirst, soeREQUIRED},
{sfRewardLgrLast, soeREQUIRED},
{sfRewardTime, soeREQUIRED},
{sfTrustLineRewardAccumulator, soeREQUIRED},
});
}
InnerObjectFormats const&

View File

@@ -111,7 +111,7 @@ issueFromJson(Json::Value const& v)
}
auto const currency = to_currency(curStr.asString());
if (currency == badCurrency() || currency == noCurrency())
if (isBadCurrency(currency) || currency == noCurrency())
{
Throw<Json::error>("issueFromJson currency must be a valid currency");
}

View File

@@ -103,7 +103,7 @@ currencyFromJson(SField const& name, Json::Value const& v)
}
auto const currency = to_currency(v.asString());
if (currency == badCurrency() || currency == noCurrency())
if (isBadCurrency(currency) || currency == noCurrency())
{
Throw<std::runtime_error>(
"currencyFromJson currency must be a valid currency");

View File

@@ -684,7 +684,8 @@ isPseudoTx(STObject const& tx)
auto tt = safe_cast<TxType>(*t);
return tt == ttAMENDMENT || tt == ttFEE || tt == ttUNL_MODIFY ||
tt == ttEMIT_FAILURE || tt == ttUNL_REPORT || tt == ttCRON;
tt == ttEMIT_FAILURE || tt == ttUNL_REPORT || tt == ttCRON ||
tt == ttCONSENSUS_ENTROPY || tt == ttEXPORT_SIGNATURES;
}
} // namespace ripple

View File

@@ -124,7 +124,7 @@ transResults()
MAKE_ERROR(tecARRAY_TOO_LARGE, "Array is too large."),
MAKE_ERROR(tecLOCKED, "Fund is locked."),
MAKE_ERROR(tecBAD_CREDENTIALS, "Bad credentials."),
MAKE_ERROR(tecINVALID_HOOK_API_VERSION, "Invalid field for current hook API version."),
MAKE_ERROR(tecEXPORT_EXPIRED, "Export expired without reaching signature quorum."),
MAKE_ERROR(tefALREADY, "The exact transaction was already in this ledger."),
MAKE_ERROR(tefBAD_ADD_AUTH, "Not authorized to add account."),
@@ -173,6 +173,7 @@ transResults()
MAKE_ERROR(telIMPORT_VL_KEY_NOT_RECOGNISED, "Import vl key was not recognized."),
MAKE_ERROR(telCAN_NOT_QUEUE_IMPORT, "Import transaction was not able to be directly applied and cannot be queued."),
MAKE_ERROR(telENV_RPC_FAILED, "Unit test RPC failure."),
MAKE_ERROR(telSHADOW_TICKET_REQUIRED, "The imported transaction uses a TicketSequence but no shadow ticket exists."),
MAKE_ERROR(temMALFORMED, "Malformed transaction."),
MAKE_ERROR(temBAD_AMM_TOKENS, "Malformed: Invalid LPTokens."),
@@ -239,6 +240,7 @@ transResults()
MAKE_ERROR(terPRE_TICKET, "Ticket is not yet in ledger."),
MAKE_ERROR(terNO_HOOK, "No hook with that hash exists on the ledger."),
MAKE_ERROR(terNO_AMM, "AMM doesn't exist for the asset pair."),
MAKE_ERROR(terRETRY_EXPORT, "Export awaiting validator signatures."),
MAKE_ERROR(tesSUCCESS, "The transaction was applied. Only final in a validated ledger."),
MAKE_ERROR(tesPARTIAL, "The transaction was applied but should be submitted again until returning tesSUCCESS."),

View File

@@ -48,6 +48,7 @@ TxFormats::TxFormats()
{sfFirstLedgerSequence, soeOPTIONAL},
{sfNetworkID, soeOPTIONAL},
{sfHookParameters, soeOPTIONAL},
{sfHookName, soeOPTIONAL},
};
#pragma push_macro("UNWRAP")

View File

@@ -49,6 +49,11 @@ TxMeta::TxMeta(
if (obj.isFieldPresent(sfHookEmissions))
setHookEmissions(obj.getFieldArray(sfHookEmissions));
if (obj.isFieldPresent(sfExportResult))
setExportResult(const_cast<STObject&>(obj)
.getField(sfExportResult)
.downcast<STObject>());
}
TxMeta::TxMeta(uint256 const& txid, std::uint32_t ledger, STObject const& obj)
@@ -75,6 +80,11 @@ TxMeta::TxMeta(uint256 const& txid, std::uint32_t ledger, STObject const& obj)
if (obj.isFieldPresent(sfHookEmissions))
setHookEmissions(obj.getFieldArray(sfHookEmissions));
if (obj.isFieldPresent(sfExportResult))
setExportResult(const_cast<STObject&>(obj)
.getField(sfExportResult)
.downcast<STObject>());
}
TxMeta::TxMeta(uint256 const& txid, std::uint32_t ledger, Blob const& vec)
@@ -245,6 +255,14 @@ TxMeta::getAsObject() const
if (hasHookEmissions())
metaData.setFieldArray(sfHookEmissions, getHookEmissions());
if (hasExportResult())
{
Serializer s;
mExportResult->add(s);
SerialIter sit(s.slice());
metaData.emplace_back(STObject(sit, sfExportResult));
}
return metaData;
}

View File

@@ -1,820 +0,0 @@
#include "defines.h"
#include "char-utils.h"
#include "buffer-utils.h"
#include "utils.h"
#ifdef _WIN32
#include <windows.h>
#elif defined(HAVE_TERMIOS_H)
#include <termios.h>
#include <sys/ioctl.h>
#include <unistd.h>
#endif
#include "debug.h"
/**
* \addtogroup buffer-utils
* @{
*/
size_t
ansi_length(const char* str, size_t len) {
size_t i, n = 0, p;
for(i = 0; i < len;) {
if(str[i] == 0x1b && (p = ansi_skip(&str[i], len - i)) > 0) {
i += p;
continue;
}
n++;
i++;
}
return n;
}
size_t
ansi_skip(const char* str, size_t len) {
size_t pos = 0;
if(str[pos] == 0x1b) {
if(++pos < len && str[pos] == '[') {
while(++pos < len)
if(is_alphanumeric_char(str[pos]))
break;
if(++pos < len && str[pos] == '~')
++pos;
return pos;
}
}
return 0;
}
size_t
ansi_truncate(const char* str, size_t len, size_t limit) {
size_t i, n = 0, p;
for(i = 0; i < len;) {
if((p = ansi_skip(&str[i], len - i)) > 0) {
i += p;
continue;
}
n += is_escape_char(str[i]) ? 2 : 1;
i++;
if(n > limit)
break;
}
return i;
}
int64_t
array_search(void* a, size_t m, size_t elsz, void* needle) {
char* ptr = a;
int64_t n, ret;
n = m / elsz;
for(ret = 0; ret < n; ret++) {
if(!memcmp(ptr, needle, elsz))
return ret;
ptr += elsz;
}
return -1;
}
char*
str_escape(const char* s) {
DynBuf dbuf;
dbuf_init2(&dbuf, 0, 0);
dbuf_put_escaped(&dbuf, s, strlen(s));
dbuf_0(&dbuf);
return (char*)dbuf.buf;
}
char*
byte_escape(const void* s, size_t n) {
DynBuf dbuf;
dbuf_init2(&dbuf, 0, 0);
dbuf_put_escaped(&dbuf, s, n);
dbuf_0(&dbuf);
return (char*)dbuf.buf;
}
size_t
byte_findb(const void* haystack, size_t hlen, const void* what, size_t wlen) {
size_t i, last;
const char* s = (const char*)haystack;
if(hlen < wlen)
return hlen;
last = hlen - wlen;
for(i = 0; i <= last; i++) {
if(byte_equal(s, wlen, what))
return i;
s++;
}
return hlen;
}
size_t
byte_finds(const void* haystack, size_t hlen, const char* what) {
return byte_findb(haystack, hlen, what, strlen(what));
}
size_t
byte_equal(const void* s, size_t n, const void* t) {
return memcmp(s, t, n) == 0;
}
void
byte_copy(void* out, size_t len, const void* in) {
char* s = (char*)out;
const char* t = (const char*)in;
size_t i;
for(i = 0; i < len; ++i)
s[i] = t[i];
}
void
byte_copyr(void* out, size_t len, const void* in) {
char* s = (char*)out + len;
const char* t = (const char*)in;
const char* u = t + len;
for(;;) {
if(t >= u)
break;
--u;
--s;
*s = *u;
}
}
size_t
byte_rchrs(const char* in, size_t len, const char needles[], size_t nn) {
const char *s = in, *end = in + len, *found = 0;
size_t i;
for(; s < end; s++) {
for(i = 0; i < nn; ++i) {
if(*s == needles[i])
found = s;
}
}
return (size_t)((found ? found : s) - in);
}
char*
dbuf_at_n(const DynBuf* db, size_t i, size_t* n, char sep) {
size_t p, l = 0;
for(p = 0; p < db->size; ++p) {
if(l == i) {
*n = byte_chr((const char*)&db->buf[p], db->size - p, sep);
return (char*)&db->buf[p];
}
if(db->buf[p] == sep)
++l;
}
*n = 0;
return 0;
}
const char*
dbuf_last_line(DynBuf* db, size_t* len) {
size_t i;
if((i = byte_rchr(db->buf, db->size, '\n')) < db->size)
i++;
else
i = 0;
if(len)
*len = db->size - i;
return (const char*)&db->buf[i];
}
int
dbuf_prepend(DynBuf* s, const uint8_t* data, size_t len) {
int ret;
if(!(ret = dbuf_reserve_start(s, len)))
memcpy(s->buf, data, len);
return 0;
}
void
dbuf_put_colorstr(DynBuf* db, const char* str, const char* color, int with_color) {
if(with_color)
dbuf_putstr(db, color);
dbuf_putstr(db, str);
if(with_color)
dbuf_putstr(db, COLOR_NONE);
}
void
dbuf_put_escaped_pred(DynBuf* db, const char* str, size_t len, int (*pred)(int)) {
size_t i = 0, j;
char c;
while(i < len) {
if((j = predicate_find(&str[i], len - i, pred))) {
dbuf_append(db, (const uint8_t*)&str[i], j);
i += j;
}
if(i == len)
break;
dbuf_putc(db, '\\');
if(str[i] == 0x1b) {
dbuf_append(db, (const uint8_t*)"x1b", 3);
} else {
int r = pred(str[i]);
dbuf_putc(db, (r > 1 && r <= 127) ? r : (c = escape_char_letter(str[i])) ? c : str[i]);
if(r == 'u' || r == 'x')
dbuf_printf(db, r == 'u' ? "%04x" : "%02x", str[i]);
}
i++;
}
}
const uint8_t escape_url_tab[256] = {
'%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%',
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0x5c, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '%',
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
'%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%', '%',
};
const uint8_t escape_noquote_tab[256] = {
'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 0x62, 0x74, 0x6e, 0x76, 0x66, 0x72, 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x',
'x', 'x', 'x', 'x', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0x5c, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 'x', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 'u', 'u', 'u', 'u', 'u',
'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u',
};
const uint8_t escape_singlequote_tab[256] = {
'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 0x62, 0x74, 0x6e, 0x76, 0x66, 0x72, 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x',
'x', 'x', 'x', 'x', 0, 0, 0, 0, 0, 0, 0, 0x27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0x5c, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 'x', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 'u', 'u', 'u', 'u', 'u',
'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u',
};
const uint8_t escape_doublequote_tab[256] = {
'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 0x62, 0x74, 0x6e, 0x76, 0x66, 0x72, 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x',
'x', 'x', 'x', 'x', 0, 0, 0x22, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0x5c, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 'x', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 'u', 'u', 'u', 'u', 'u',
'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u',
};
const uint8_t escape_backquote_tab[256] = {
'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 0x62, 0x74, 0, 0x76, 0x66, 0, 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x',
'x', 'x', 'x', 'x', 0, 0, 0, 0, 0x24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0x5c, 0, 0, 0, 0x60, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 'x', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 'u', 'u', 'u', 'u', 'u',
'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u',
};
void
dbuf_put_escaped_table(DynBuf* db, const char* str, size_t len, const uint8_t table[256]) {
size_t i = 0, clen;
int32_t c;
const uint8_t *pos, *end, *next;
for(pos = (const uint8_t*)str, end = pos + len; pos < end; pos = next) {
uint8_t r, ch;
if((c = unicode_from_utf8(pos, end - pos, &next)) < 0)
break;
clen = next - pos;
ch = c;
r = (clen >= 2 || c > 0xff) ? 'u' : table[c];
if(r == 'u' && clen > 1 && (c & 0xffffff00) == 0) {
r = 'x';
// ch = c >> 8;
}
if(r == '%') {
static const char hexdigits[] = "0123456789ABCDEF";
dbuf_putc(db, '%');
dbuf_putc(db, hexdigits[c >> 4]);
dbuf_putc(db, hexdigits[c & 0xf]);
} else if(c == 0x1b) {
dbuf_putstr(db, "\\x1b");
} else if(r == 'u') {
dbuf_printf(db, c > 0xffff ? "\\u{%X}" : "\\u%04x", c);
} else if(r == 'x') {
dbuf_printf(db, "\\x%02x", ch);
} else if(r) {
dbuf_putc(db, '\\');
dbuf_putc(db, (r > 1 && r <= 127) ? r : (c = escape_char_letter(ch)) ? c : ch);
} else {
dbuf_put(db, pos, next - pos);
}
i++;
}
}
void
dbuf_put_unescaped_pred(DynBuf* db, const char* str, size_t len, int (*pred)(const char*, size_t*)) {
size_t i = 0, j;
// char c;
while(i < len) {
int r = 0;
if((j = byte_chr(&str[i], len - i, '\\'))) {
dbuf_append(db, (const uint8_t*)&str[i], j);
i += j;
}
if(i == len)
break;
size_t n = 1;
if(pred) {
r = pred(&str[i + 1], &n);
if(!r && n == 1)
dbuf_putc(db, '\\');
}
if(r >= 0)
dbuf_putc(db, /*n > 1 ||*/ r ? /*(r > 1 && r < 256) ?*/ r : str[i]);
i += n;
}
}
static inline int
hexdigit(char c) {
if(c >= '0' && c <= '9')
return c - '0';
if(c >= 'a' && c <= 'f')
c -= 32;
if(c >= 'A' && c <= 'F')
return c - 'A' + 10;
return -1;
}
void
dbuf_put_unescaped_table(DynBuf* db, const char* str, size_t len, const uint8_t table[256]) {
size_t i = 0, j;
char escape_char = table == escape_url_tab ? '%' : '\\';
while(i < len) {
if((j = byte_chr(&str[i], len - i, escape_char))) {
dbuf_append(db, (const uint8_t*)&str[i], j);
i += j;
}
if(i == len)
break;
if(escape_char == '%') {
int hi = hexdigit(str[i + 1]), lo = hexdigit(str[i + 2]);
uint8_t c = (hi << 4) | (lo & 0xf);
dbuf_putc(db, c);
i += 2;
} else {
++i;
uint8_t c;
switch(str[i]) {
case 'b': c = '\b'; break;
case 't': c = '\t'; break;
case 'n': c = '\n'; break;
case 'v': c = '\v'; break;
case 'f': c = '\f'; break;
case 'r': c = '\r'; break;
default: c = str[i]; break;
}
uint8_t r = table[c];
if(!(r && r != 'x' && r != 'u')) {
dbuf_putc(db, '\\');
dbuf_putc(db, c);
} else {
dbuf_putc(db, str[i] == r ? c : r);
}
}
++i;
}
}
void
dbuf_put_escaped(DynBuf* db, const char* str, size_t len) {
return dbuf_put_escaped_table(db, str, len, escape_noquote_tab);
}
void
dbuf_put_value(DynBuf* db, JSContext* ctx, JSValueConst value) {
const char* str;
size_t len;
str = JS_ToCStringLen(ctx, &len, value);
dbuf_append(db, str, len);
JS_FreeCString(ctx, str);
}
void
dbuf_put_uint32(DynBuf* db, uint32_t num) {
char buf[FMT_ULONG];
dbuf_put(db, (const uint8_t*)buf, fmt_ulong(buf, num));
}
void
dbuf_put_atom(DynBuf* db, JSContext* ctx, JSAtom atom) {
const char* str;
str = JS_AtomToCString(ctx, atom);
dbuf_putstr(db, str);
JS_FreeCString(ctx, str);
}
int
dbuf_reserve_start(DynBuf* s, size_t len) {
if(unlikely((s->size + len) > s->allocated_size)) {
if(dbuf_realloc(s, s->size + len))
return -1;
}
if(s->size > 0)
memcpy(s->buf + len, s->buf, s->size);
s->size += len;
return 0;
}
uint8_t*
dbuf_reserve(DynBuf* s, size_t len) {
if(unlikely((s->size + len) > s->allocated_size))
if(dbuf_realloc(s, s->size + len))
return 0;
return &s->buf[s->size];
}
size_t
dbuf_token_pop(DynBuf* db, char delim) {
size_t n, p, len;
len = db->size;
for(n = db->size; n > 0;) {
if((p = byte_rchr(db->buf, n, delim)) == n) {
db->size = 0;
break;
}
if(p > 0 && db->buf[p - 1] == '\\') {
n = p - 1;
continue;
}
db->size = p;
break;
}
return len - db->size;
}
size_t
dbuf_token_push(DynBuf* db, const char* str, size_t len, char delim) {
size_t pos;
if(db->size)
dbuf_putc(db, delim);
pos = db->size;
dbuf_put_escaped_pred(db, str, len, is_dot_char);
return db->size - pos;
}
JSValue
dbuf_tostring_free(DynBuf* s, JSContext* ctx) {
JSValue r;
r = JS_NewStringLen(ctx, s->buf ? (const char*)s->buf : "", s->buf ? s->size : 0);
dbuf_free(s);
return r;
}
ssize_t
dbuf_load(DynBuf* s, const char* filename) {
FILE* fp;
size_t nbytes = 0;
if((fp = fopen(filename, "rb"))) {
char buf[4096];
size_t r;
while(!feof(fp)) {
if((r = fread(buf, 1, sizeof(buf), fp)) == 0) {
fclose(fp);
return -1;
}
dbuf_put(s, (uint8_t const*)buf, r);
nbytes += r;
}
fclose(fp);
}
return nbytes;
}
int
dbuf_vprintf(DynBuf* s, const char* fmt, va_list ap) {
s->size += vsnprintf((char*)(s->buf + s->size), s->allocated_size - s->size, fmt, ap);
return 0;
}
InputBuffer
js_input_buffer(JSContext* ctx, JSValueConst value) {
InputBuffer ret = {{{0, 0}}, 0, &input_buffer_free_default, JS_UNDEFINED, {0, INT64_MAX}};
if(js_is_typedarray(ctx, value)) {
ret.value = offset_typedarray(&ret.range, value, ctx);
} else if(js_is_arraybuffer(ctx, value) || js_is_sharedarraybuffer(ctx, value)) {
ret.value = JS_DupValue(ctx, value);
}
if(js_is_arraybuffer(ctx, ret.value) || js_is_sharedarraybuffer(ctx, ret.value)) {
block_arraybuffer(&ret.block, ret.value, ctx);
} else {
JS_ThrowTypeError(ctx, "Invalid type (%s) for input buffer", js_value_typestr(ctx, ret.value));
JS_FreeValue(ctx, ret.value);
ret.value = JS_EXCEPTION;
}
return ret;
}
#undef free
InputBuffer
js_input_chars(JSContext* ctx, JSValueConst value) {
InputBuffer ret = {{{0, 0}}, 0, &input_buffer_free_default, JS_UNDEFINED, OFFSET_INIT()};
if(JS_IsString(value)) {
ret.data = (uint8_t*)JS_ToCStringLen(ctx, &ret.size, value);
ret.value = JS_DupValue(ctx, value);
ret.free = &input_buffer_free_default;
} else {
ret = js_input_buffer(ctx, value);
}
return ret;
}
InputBuffer
js_input_args(JSContext* ctx, int argc, JSValueConst argv[]) {
InputBuffer input = js_input_chars(ctx, argv[0]);
if(argc > 1)
js_offset_length(ctx, input.size, argc - 1, argv + 1, &input.range);
return input;
}
InputBuffer
js_output_args(JSContext* ctx, int argc, JSValueConst argv[]) {
InputBuffer output = js_input_buffer(ctx, argv[0]);
if(argc > 1)
js_offset_length(ctx, output.size, argc - 1, argv + 1, &output.range);
return output;
}
BOOL
input_buffer_valid(const InputBuffer* in) {
return !JS_IsException(in->value);
}
InputBuffer
input_buffer_clone(const InputBuffer* in, JSContext* ctx) {
InputBuffer ret = js_input_buffer(ctx, in->value);
ret.pos = in->pos;
ret.size = in->size;
ret.free = in->free;
return ret;
}
void
input_buffer_dump(const InputBuffer* in, DynBuf* db) {
dbuf_printf(db, "(InputBuffer){ .data = %p, .size = %lu, .pos = %lu, .free = %p }", in->data, (unsigned long)in->size, (unsigned long)in->pos, in->free);
}
void
input_buffer_free(InputBuffer* in, JSContext* ctx) {
if(in->data) {
in->free(ctx, (const char*)in->data, in->value);
in->data = 0;
in->size = 0;
in->pos = 0;
in->value = JS_UNDEFINED;
}
}
const uint8_t*
input_buffer_peek(InputBuffer* in, size_t* lenp) {
input_buffer_peekc(in, lenp);
return input_buffer_data(in) + in->pos;
}
const uint8_t*
input_buffer_get(InputBuffer* in, size_t* lenp) {
const uint8_t* ret = input_buffer_peek(in, lenp);
in->pos += *lenp;
return ret;
}
const char*
input_buffer_currentline(InputBuffer* in, size_t* len) {
size_t i;
if((i = byte_rchr(input_buffer_data(in), in->pos, '\n')) < in->pos)
i++;
if(len)
*len = in->pos - i;
return (const char*)&input_buffer_data(in)[i];
}
size_t
input_buffer_column(InputBuffer* in, size_t* len) {
size_t i;
if((i = byte_rchr(input_buffer_data(in), in->pos, '\n')) < in->pos)
i++;
return in->pos - i;
}
int
js_offset_length(JSContext* ctx, int64_t size, int argc, JSValueConst argv[], OffsetLength* off_len_p) {
int ret = 0;
int64_t off = 0, len = size;
if(argc >= 1 && JS_IsNumber(argv[0])) {
if(!JS_ToInt64(ctx, &off, argv[0]))
ret = 1;
if(argc >= 2 && JS_IsNumber(argv[1]))
if(!JS_ToInt64(ctx, &len, argv[1]))
ret = 2;
if(size && off != size)
off = ((off % size) + size) % size;
if(len >= 0)
len = MIN_NUM(len, size - off);
else
len = size - off;
if(off_len_p) {
off_len_p->offset = off;
off_len_p->length = len;
}
}
return ret;
}
int
js_index_range(JSContext* ctx, int64_t size, int argc, JSValueConst argv[], IndexRange* idx_rng_p) {
int ret = 0;
int64_t start = 0, end = size;
if(argc >= 1 && JS_IsNumber(argv[0])) {
if(!JS_ToInt64(ctx, &start, argv[0]))
ret = 1;
if(argc >= 2 && JS_IsNumber(argv[1]))
if(!JS_ToInt64(ctx, &end, argv[1]))
ret = 2;
if(size > 0) {
start = ((start % size) + size) % size;
end = ((end % size) + size) % size;
}
if(end > size)
end = size;
if(idx_rng_p) {
idx_rng_p->start = start;
idx_rng_p->end = end;
}
}
return ret;
}
int
screen_size(int size[2]) {
#ifdef _WIN32
CONSOLE_SCREEN_BUFFER_INFO csbi;
GetConsoleScreenBufferInfo(GetStdHandle(STD_OUTPUT_HANDLE), &csbi);
size[0] = csbi.srWindow.Right - csbi.srWindow.Left + 1;
size[1] = csbi.srWindow.Bottom - csbi.srWindow.Top + 1;
return 0;
#elif defined(HAVE_TERMIOS_H)
{
struct winsize w = {.ws_col = -1, .ws_row = -1};
if(isatty(STDIN_FILENO))
ioctl(STDIN_FILENO, TIOCGWINSZ, &w);
else if(isatty(STDOUT_FILENO))
ioctl(STDOUT_FILENO, TIOCGWINSZ, &w);
else if(isatty(STDERR_FILENO))
ioctl(STDERR_FILENO, TIOCGWINSZ, &w);
size[0] = w.ws_col;
size[1] = w.ws_row;
return 0;
}
#else
size[0] = 80;
size[1] = 25;
return 0;
#endif
return -1;
}
#undef js_realloc_rt
void
js_dbuf_allocator(JSContext* ctx, DynBuf* s) {
dbuf_init2(s, JS_GetRuntime(ctx), (DynBufReallocFunc*)js_realloc_rt);
}
inline int
input_buffer_peekc(InputBuffer* in, size_t* lenp) {
const uint8_t *pos, *end, *next;
int cp;
pos = input_buffer_data(in) + in->pos;
end = input_buffer_data(in) + input_buffer_length(in);
cp = unicode_from_utf8(pos, end - pos, &next);
*lenp = next - pos;
return cp;
}
inline int
input_buffer_putc(InputBuffer* in, unsigned int c, JSContext* ctx) {
int len;
if(in->pos + UTF8_CHAR_LEN_MAX > in->size)
if(block_realloc(&in->block, in->pos + UTF8_CHAR_LEN_MAX, ctx))
return -1;
len = unicode_to_utf8(&in->data[in->pos], c);
in->pos += len;
return len;
}
size_t
dbuf_bitflags(DynBuf* db, uint32_t bits, const char* const names[]) {
size_t i, n = 0;
for(i = 0; i < sizeof(bits) * 8; i++) {
if(bits & (1 << i)) {
size_t len = strlen(names[i]);
if(n) {
n++;
dbuf_putstr(db, "|");
}
dbuf_append(db, names[i], len);
n += len;
}
}
return n;
}
/**
* @}
*/

View File

@@ -1,348 +0,0 @@
#ifndef BUFFER_UTILS_H
#define BUFFER_UTILS_H
#include <quickjs.h>
#include <cutils.h>
#include <stdarg.h>
#include "char-utils.h"
/**
* \defgroup buffer-utils buffer-utils: Buffer Utilities
* @{
*/
int64_t array_search(void* a, size_t m, size_t elsz, void* needle);
#define array_contains(a, m, elsz, needle) (array_search((a), (m), (elsz), (needle)) != -1)
size_t ansi_length(const char*, size_t);
size_t ansi_skip(const char*, size_t);
size_t ansi_truncate(const char*, size_t, size_t limit);
int64_t array_search(void*, size_t, size_t elsz, void* needle);
char* str_escape(const char*);
char* byte_escape(const void*, size_t);
size_t byte_findb(const void*, size_t, const void* what, size_t wlen);
size_t byte_finds(const void*, size_t, const char* what);
size_t byte_equal(const void* s, size_t n, const void* t);
void byte_copy(void* out, size_t len, const void* in);
void byte_copyr(void* out, size_t len, const void* in);
size_t byte_rchrs(const char* in, size_t len, const char needles[], size_t nn);
#define DBUF_INIT_0() \
(DynBuf) { 0, 0, 0, 0, 0, 0 }
#define DBUF_INIT_CTX(ctx) \
(DynBuf) { 0, 0, 0, 0, (DynBufReallocFunc*)js_realloc_rt, JS_GetRuntime(ctx) }
extern const uint8_t escape_url_tab[256], escape_noquote_tab[256], escape_singlequote_tab[256], escape_doublequote_tab[256], escape_backquote_tab[256];
char* dbuf_at_n(const DynBuf*, size_t, size_t* n, char sep);
const char* dbuf_last_line(DynBuf*, size_t*);
int dbuf_prepend(DynBuf*, const uint8_t*, size_t len);
void dbuf_put_colorstr(DynBuf*, const char*, const char* color, int with_color);
void dbuf_put_escaped_pred(DynBuf*, const char*, size_t len, int (*pred)(int));
void dbuf_put_escaped_table(DynBuf*, const char*, size_t len, const uint8_t table[256]);
void dbuf_put_unescaped_table(DynBuf* db, const char* str, size_t len, const uint8_t table[256]);
void dbuf_put_unescaped_pred(DynBuf*, const char*, size_t len, int (*pred)());
void dbuf_put_escaped(DynBuf*, const char*, size_t len);
void dbuf_put_value(DynBuf*, JSContext*, JSValue value);
void dbuf_put_uint32(DynBuf* db, uint32_t num);
void dbuf_put_atom(DynBuf* db, JSContext* ctx, JSAtom atom);
int dbuf_reserve_start(DynBuf*, size_t);
uint8_t* dbuf_reserve(DynBuf*, size_t);
size_t dbuf_token_pop(DynBuf*, char);
size_t dbuf_token_push(DynBuf*, const char*, size_t len, char delim);
JSValue dbuf_tostring_free(DynBuf*, JSContext*);
ssize_t dbuf_load(DynBuf*, const char*);
int dbuf_vprintf(DynBuf*, const char*, va_list);
int screen_size(int size[2]);
static inline int
dbuf_putm(DynBuf* db, ...) {
int r = 0;
va_list a;
const char* s;
va_start(a, db);
while((s = va_arg(a, char*)))
if(dbuf_putstr(db, s))
return -1;
va_end(a);
return r;
}
#define dbuf_append(d, x, n) dbuf_put((d), (const uint8_t*)(x), (n))
static inline size_t
dbuf_count(DynBuf* db, int ch) {
return byte_count(db->buf, db->size, ch);
}
static inline void
dbuf_0(DynBuf* db) {
dbuf_putc(db, '\0');
db->size--;
}
static inline void
dbuf_zero(DynBuf* db) {
dbuf_realloc(db, 0);
db->size = 0;
}
size_t dbuf_bitflags(DynBuf* db, uint32_t bits, const char* const names[]);
#define js_dbuf_init(ctx, buf) dbuf_init2((buf), (ctx), (realloc_func*)&utils_js_realloc)
#define js_dbuf_init_rt(rt, buf) dbuf_init2((buf), (rt), (realloc_func*)&utils_js_realloc_rt)
void js_dbuf_allocator(JSContext* ctx, DynBuf* s);
typedef struct {
uint8_t* base;
size_t size;
} MemoryBlock;
static inline void
block_init(MemoryBlock* mb) {
mb->base = 0;
mb->size = 0;
}
/* clang-format off */
static inline void* block_data(const MemoryBlock* mb) { return mb->base; }
static inline size_t block_length(const MemoryBlock* mb) { return mb->size; }
static inline void* block_begin(const MemoryBlock* mb) { return mb->base; }
static inline void* block_end(const MemoryBlock* mb) { return mb->base + mb->size; }
/* clang-format on */
static inline BOOL
block_arraybuffer(MemoryBlock* mb, JSValueConst ab, JSContext* ctx) {
mb->base = JS_GetArrayBuffer(ctx, &mb->size, ab);
return mb->base != 0;
}
typedef struct {
uint8_t *start, *end;
} PointerRange;
static inline void
range_init(PointerRange* pr) {
pr->end = pr->start = 0;
}
static inline PointerRange
range_from(const MemoryBlock* mb) {
return (PointerRange){mb->base, mb->base + mb->size};
}
typedef struct {
int64_t start, end;
} IndexRange;
typedef struct {
int64_t offset, length;
} OffsetLength;
#define OFFSET_INIT() \
(OffsetLength) { 0, INT64_MAX }
static inline void
offset_init(OffsetLength* ol) {
ol->offset = 0;
ol->length = INT64_MAX;
}
static inline BOOL
offset_is_default(const OffsetLength* ol) {
return ol->offset == 0 && ol->length == INT64_MAX;
}
static inline void*
offset_data(const OffsetLength* ol, const void* x) {
return (uint8_t*)x + ol->offset;
}
static inline size_t
offset_size(const OffsetLength* ol, size_t n) {
if(ol->length == -1)
return (signed long)n - ol->offset;
return MIN_NUM(ol->length, (signed long)n - ol->offset);
}
static inline MemoryBlock
offset_block(const OffsetLength* ol, const void* x, size_t n) {
return (MemoryBlock){offset_data(ol, x), offset_size(ol, n)};
}
static inline PointerRange
offset_range(const OffsetLength* ol, const void* x, size_t n) {
MemoryBlock mb = offset_block(ol, x, n);
return range_from(&mb);
}
static inline OffsetLength
offset_slice(const OffsetLength ol, int64_t start, int64_t end) {
if(start < 0)
start = ol.length + (start % ol.length);
else if(start > ol.length)
start = ol.length;
if(end < 0)
end = ol.length + (end % ol.length);
else if(end > ol.length)
end = ol.length;
return (OffsetLength){start, end - start};
}
static inline OffsetLength
offset_offset(const OffsetLength* ol, const OffsetLength* by) {
OffsetLength ret;
ret.offset = ol->offset + by->offset;
ret.length = MIN_NUM(by->length, ol->length - by->offset);
return ret;
}
static inline OffsetLength
offset_from_indexrange(const IndexRange* ir) {
OffsetLength ret;
ret.offset = ir->start;
ret.length = ir->end - ir->start;
return ret;
}
static inline JSValue
offset_typedarray(OffsetLength* ol, JSValueConst array, JSContext* ctx) {
JSValue ret;
size_t offset, length;
ret = JS_GetTypedArrayBuffer(ctx, array, &offset, &length, NULL);
if(!JS_IsException(ret)) {
ol->offset = offset;
ol->length = length;
}
return ret;
}
static inline IndexRange
indexrange_from_offset(const OffsetLength* ol) {
IndexRange ret;
ret.start = ol->offset;
ret.end = ol->offset + ol->length;
return ret;
}
static inline MemoryBlock
block_range(const MemoryBlock* mb, const OffsetLength* range) {
MemoryBlock ret;
ret.base = mb->base + range->offset;
ret.size = MIN_NUM((size_t)range->length, mb->size - range->offset);
return ret;
}
static inline int
block_realloc(MemoryBlock* mb, size_t new_size, JSContext* ctx) {
if((mb->base = js_realloc(ctx, mb->base, new_size))) {
mb->size = new_size;
return 0;
}
return -1;
}
typedef struct InputBuffer {
union {
MemoryBlock block;
struct {
uint8_t* data;
size_t size;
};
};
size_t pos;
void (*free)(JSContext*, const char*, JSValue);
JSValue value;
OffsetLength range;
} InputBuffer;
static inline void
input_buffer_free_default(JSContext* ctx, const char* str, JSValue val) {
if(JS_IsString(val))
JS_FreeCString(ctx, str);
if(!JS_IsUndefined(val))
JS_FreeValue(ctx, val);
}
InputBuffer js_input_buffer(JSContext* ctx, JSValueConst value);
InputBuffer js_input_chars(JSContext* ctx, JSValueConst value);
InputBuffer js_input_args(JSContext* ctx, int argc, JSValueConst argv[]);
InputBuffer js_output_args(JSContext* ctx, int argc, JSValueConst argv[]);
InputBuffer input_buffer_clone(const InputBuffer* in, JSContext* ctx);
BOOL input_buffer_valid(const InputBuffer* in);
void input_buffer_dump(const InputBuffer* in, DynBuf* db);
void input_buffer_free(InputBuffer* in, JSContext* ctx);
static inline uint8_t*
input_buffer_data(const InputBuffer* in) {
return offset_data(&in->range, in->data);
}
static inline size_t
input_buffer_length(const InputBuffer* in) {
return offset_size(&in->range, in->size);
}
static inline MemoryBlock
input_buffer_block(InputBuffer* in) {
return (MemoryBlock){input_buffer_data(in), input_buffer_length(in)};
}
static inline MemoryBlock*
input_buffer_blockptr(InputBuffer* in) {
return &in->block;
}
const uint8_t* input_buffer_get(InputBuffer* in, size_t* lenp);
const uint8_t* input_buffer_peek(InputBuffer* in, size_t* lenp);
const char* input_buffer_currentline(InputBuffer*, size_t* len);
size_t input_buffer_column(InputBuffer*, size_t* len);
int input_buffer_peekc(InputBuffer* in, size_t* lenp);
int input_buffer_putc(InputBuffer*, unsigned int, JSContext*);
static inline int
input_buffer_getc(InputBuffer* in) {
size_t n;
int ret;
ret = input_buffer_peekc(in, &n);
in->pos += n;
return ret;
}
static inline void*
input_buffer_begin(const InputBuffer* in) {
return input_buffer_data(in);
}
static inline void*
input_buffer_end(const InputBuffer* in) {
return input_buffer_data(in) + input_buffer_length(in);
}
static inline BOOL
input_buffer_eof(const InputBuffer* in) {
return in->pos == input_buffer_length(in);
}
static inline size_t
input_buffer_remain(const InputBuffer* in) {
return input_buffer_length(in) - in->pos;
}
int js_offset_length(JSContext*, int64_t size, int argc, JSValueConst argv[], OffsetLength* off_len_p);
int js_index_range(JSContext*, int64_t size, int argc, JSValueConst argv[], IndexRange* idx_rng_p);
/**
* @}
*/
#endif /* defined(BUFFER_UTILS) */

View File

@@ -1,578 +0,0 @@
#include "char-utils.h"
#include "libutf.h"
#if defined(_WIN32) || defined(__CYGWIN__) || defined(__MSYS__)
#include <winnls.h>
#include <windows.h>
#include <wchar.h>
#endif
/**
* \addtogroup char-utils
* @{
*/
size_t
token_length(const char* str, size_t len, char delim) {
const char *s, *e;
size_t pos;
for(s = str, e = s + len; s < e; s += pos + 1) {
pos = byte_chr(s, e - s, delim);
if(s + pos == e)
break;
if(pos == 0 || s[pos - 1] != '\\') {
s += pos;
break;
}
}
return s - str;
}
size_t
fmt_ulong(char* dest, uint32_t i) {
uint32_t len, tmp, len2;
for(len = 1, tmp = i; tmp > 9; ++len)
tmp /= 10;
if(dest)
for(tmp = i, dest += len, len2 = len + 1; --len2; tmp /= 10)
*--dest = (char)((tmp % 10) + '0');
return len;
}
size_t
fmt_longlong(char* dest, int64_t i) {
if(i < 0) {
if(dest)
*dest++ = '-';
return fmt_ulonglong(dest, (uint64_t)-i) + 1;
}
return fmt_ulonglong(dest, (uint64_t)i);
}
size_t
fmt_ulonglong(char* dest, uint64_t i) {
size_t len;
uint64_t tmp, len2;
for(len = 1, tmp = i; tmp > 9ll; ++len)
tmp /= 10ll;
if(dest)
for(tmp = i, dest += len, len2 = len + 1; --len2; tmp /= 10ll)
*--dest = (tmp % 10ll) + '0';
return len;
}
#define tohex(c) (char)((c) >= 10 ? (c)-10 + 'a' : (c) + '0')
size_t
fmt_xlonglong(char* dest, uint64_t i) {
uint64_t len, tmp;
for(len = 1, tmp = i; tmp > 15ll; ++len)
tmp >>= 4ll;
if(dest)
for(tmp = i, dest += len;;) {
*--dest = tohex(tmp & 15ll);
if(!(tmp >>= 4ll))
break;
}
return len;
}
size_t
fmt_xlonglong0(char* dest, uint64_t num, size_t n) {
size_t i = 0, len;
if((len = fmt_xlonglong(NULL, num)) < n) {
len = n - len;
while(i < len)
dest[i++] = '0';
}
i += fmt_xlonglong(&dest[i], num);
return i;
}
size_t
fmt_8long(char* dest, uint32_t i) {
uint32_t len, tmp;
/* first count the number of bytes needed */
for(len = 1, tmp = i; tmp > 7; ++len)
tmp >>= 3;
if(dest)
for(tmp = i, dest += len;;) {
*--dest = (char)((tmp & 7) + '0');
if(!(tmp >>= 3))
break;
}
return len;
}
#define tohex(c) (char)((c) >= 10 ? (c)-10 + 'a' : (c) + '0')
size_t
fmt_xlong(char* dest, uint32_t i) {
uint32_t len, tmp;
/* first count the number of bytes needed */
for(len = 1, tmp = i; tmp > 15; ++len)
tmp >>= 4;
if(dest)
for(tmp = i, dest += len;;) {
*--dest = tohex(tmp & 15);
if(!(tmp >>= 4))
break;
}
return len;
}
size_t
fmt_xlong0(char* dest, uint32_t num, size_t n) {
size_t i = 0, len;
if((len = fmt_xlong(NULL, num)) < n) {
len = n - len;
while(i < len)
dest[i++] = '0';
}
i += fmt_xlong(&dest[i], num);
return i;
}
size_t
scan_ushort(const char* src, uint16_t* dest) {
const char* cur;
uint16_t l;
for(cur = src, l = 0; *cur >= '0' && *cur <= '9'; ++cur) {
uint32_t tmp = l * 10ul + *cur - '0';
if((uint16_t)tmp != tmp)
break;
l = tmp;
}
if(cur > src)
*dest = l;
return (size_t)(cur - src);
}
size_t
scan_uint(const char* src, uint32_t* dest) {
uint64_t u64;
size_t r = scan_ulonglong(src, &u64);
*dest = u64;
return r;
}
size_t
scan_int(const char* src, int32_t* dest) {
int64_t i64;
size_t r = scan_longlong(src, &i64);
*dest = i64;
return r;
}
#ifndef MAXLONG
#define MAXLONG (((uint32_t)-1) >> 1)
#endif
size_t
scan_longlong(const char* src, int64_t* dest) {
size_t i, o;
uint64_t l;
char c = src[0];
unsigned int neg = c == '-';
o = c == '-' || c == '+';
if((i = scan_ulonglong(src + o, &l))) {
if(i > 0ll && l > MAXLONG + neg) {
l /= 10ll;
--i;
}
if(i + o)
*dest = (int64_t)(c == '-' ? -l : l);
return i + o;
}
return 0;
}
size_t
scan_ulonglong(const char* src, uint64_t* dest) {
const char* tmp = src;
uint64_t l = 0;
unsigned char c;
while((c = (unsigned char)(*tmp - '0')) < 10) {
uint64_t n;
n = l << 3ll;
if((n >> 3ll) != l)
break;
if(n + (l << 1ll) < n)
break;
n += l << 1ll;
if(n + c < n)
break;
l = n + c;
++tmp;
}
if(tmp - src)
*dest = l;
return (size_t)(tmp - src);
}
size_t
scan_xlonglong(const char* src, uint64_t* dest) {
const char* tmp = src;
int64_t l = 0;
unsigned char c;
while((c = scan_fromhex(*tmp)) < 16) {
l = (l << 4) + c;
++tmp;
}
*dest = l;
return tmp - src;
}
size_t
scan_8longn(const char* src, size_t n, uint32_t* dest) {
const char* tmp = src;
uint32_t l = 0;
unsigned char c;
while(n-- > 0 && (c = (unsigned char)(*tmp - '0')) < 8) {
if(l >> (sizeof(l) * 8 - 3))
break;
l = l * 8 + c;
++tmp;
}
*dest = l;
return (size_t)(tmp - src);
}
size_t
scan_whitenskip(const char* s, size_t limit) {
const char *t, *u;
for(t = s, u = t + limit; t < u; ++t)
if(!is_whitespace_char(*t))
break;
return (size_t)(t - s);
}
size_t
scan_nonwhitenskip(const char* s, size_t limit) {
const char *t, *u;
for(t = s, u = t + limit; t < u; ++t)
if(is_whitespace_char(*t))
break;
return (size_t)(t - s);
}
size_t
scan_line(const char* s, size_t limit) {
const char *t, *u;
for(t = s, u = s + limit; t < u; ++t)
if(*t == '\n' || *t == '\r')
break;
return (size_t)(t - s);
}
size_t
scan_lineskip(const char* s, size_t limit) {
const char *t, *u;
for(t = s, u = s + limit; t < u; ++t)
if(*t == '\n') {
++t;
break;
}
return (size_t)(t - s);
}
size_t
scan_lineskip_escaped(const char* s, size_t limit) {
const char *t, *u;
for(t = s, u = s + limit; t < u; ++t) {
if(*t == '\\') {
++t;
continue;
}
if(*t == '\n') {
++t;
break;
}
}
return (size_t)(t - s);
}
size_t
scan_eolskip(const char* s, size_t limit) {
size_t n = 0;
if(n + 1 < limit && s[0] == '\r' && s[1] == '\n')
n += 2;
else if(n < limit && s[0] == '\n')
n += 1;
return n;
}
size_t
utf8_strlen(const void* in, size_t len) {
const uint8_t *pos, *end, *next;
size_t i = 0;
for(pos = (const uint8_t*)in, end = pos + len; pos < end; pos = next, ++i)
unicode_from_utf8(pos, end - pos, &next);
return i;
}
#if defined(_WIN32) || defined(__CYGWIN__) || defined(__MSYS__)
wchar_t*
utf8_towcs(const char* s) {
int len = (int)strlen(s);
int n = MultiByteToWideChar(CP_UTF8, 0, s, len, NULL, 0);
wchar_t* ret;
if((ret = (wchar_t*)malloc((n + 1) * sizeof(wchar_t)))) {
MultiByteToWideChar(CP_UTF8, 0, s, len, ret, n);
ret[n] = L'\0';
}
return ret;
}
char*
utf8_fromwcs(const wchar_t* wstr) {
int len = (int)wcslen(wstr);
int n = WideCharToMultiByte(CP_UTF8, 0, wstr, len, NULL, 0, NULL, NULL);
char* ret;
if((ret = malloc((n + 1)))) {
WideCharToMultiByte(CP_UTF8, 0, wstr, len, ret, n, NULL, NULL);
ret[n] = '\0';
}
return ret;
}
#endif
BOOL
utf16_multiword(const void* in) {
const uint16_t* p16 = in;
LibutfC16Type type = libutf_c16_type(p16[0]);
return !((LIBUTF_UTF16_NOT_SURROGATE == type) || (LIBUTF_UTF16_SURROGATE_HIGH != type || LIBUTF_UTF16_SURROGATE_LOW != libutf_c16_type(p16[1])));
}
int
case_lowerc(int c) {
if(c >= 'A' && c <= 'Z')
c += 'a' - 'A';
return c;
}
int
case_starts(const char* a, const char* b) {
const char *s, *t;
for(s = a, t = b;; ++s, ++t) {
unsigned char x, y;
if(!*t)
return 1;
x = case_lowerc(*s);
y = case_lowerc(*t);
if(x != y)
break;
if(!x)
break;
}
return 0;
}
int
case_diffb(const void* S, size_t len, const void* T) {
unsigned char x, y;
const char *s, *t;
for(s = (const char*)S, t = (const char*)T; len > 0;) {
--len;
x = case_lowerc(*s);
y = case_lowerc(*t);
++s;
++t;
if(x != y)
return ((int)(unsigned int)x) - ((int)(unsigned int)y);
}
return 0;
}
size_t
case_findb(const void* haystack, size_t hlen, const void* what, size_t wlen) {
size_t i, last;
const char* s = haystack;
if(hlen < wlen)
return hlen;
last = hlen - wlen;
for(i = 0; i <= last; i++, s++)
if(!case_diffb(s, wlen, what))
return i;
return hlen;
}
size_t
case_finds(const void* haystack, const char* what) {
return case_findb(haystack, strlen(haystack), what, strlen(what));
}
ssize_t
write_file(const char* file, const void* buf, size_t len) {
FILE* f;
ssize_t ret = -1;
if((f = fopen(file, "w+")))
switch(fwrite(buf, len, 1, f)) {
case 1: {
ret = len;
break;
}
}
fflush(f);
ret = ftell(f);
fclose(f);
return ret;
}
ssize_t
puts_file(const char* file, const char* s) {
return write_file(file, s, strlen(s));
}
size_t
u64toa(char* x, uint64_t num, int base) {
size_t len = 0;
uint64_t n = num;
do {
n /= base;
len++;
x++;
} while(n != 0);
*x-- = '\0';
do {
char c = num % base;
num /= base;
if(c >= 10)
c += 'a' - '0' - 10;
*x-- = c + '0';
} while(num != 0);
return len;
}
size_t
i64toa(char* x, int64_t num, int base) {
size_t pos = 0, len;
if(num < 0) {
x[pos++] = '-';
num = -num;
}
len = u64toa(&x[pos], num, base);
return pos + len;
}
size_t
str_findb(const char* s1, const char* x, size_t n) {
const char* b;
size_t i, j, len = strlen(s1);
if(len >= n) {
size_t end = len - n + 1;
for(i = 0; i < end; i++) {
b = &s1[i];
for(j = 0; x[j] == b[j];)
if(++j == n)
return i;
}
}
return len;
}
size_t
str_find(const void* s, const void* what) {
return str_findb(s, what, strlen(what));
}
/**
* @}
*/

View File

@@ -1,441 +0,0 @@
#ifndef CHAR_UTILS_H
#define CHAR_UTILS_H
#include <cutils.h>
#include <string.h>
#include "debug.h"
/**
* \defgroup char-utils char-utils: Character Utilities
* @{
*/
#define is_control_char(c) ((c) == '\a' || (c) == '\b' || (c) == '\t' || (c) == '\n' || (c) == '\v' || (c) == '\f' || (c) == '\r')
#define is_alphanumeric_char(c) (((c) >= 'A' && (c) <= 'Z') || ((c) >= 'a' && (c) <= 'z'))
#define is_digit_char(c) ((c) >= '0' && (c) <= '9')
#define is_print_char(c) ((c) >= ' ' && (c) <= '\x7f')
#define is_newline_char(c) ((c) == '\n')
#define is_identifier_char(c) (is_alphanumeric_char(c) || is_digit_char(c) || (c) == '$' || (c) == '_')
#define is_whitespace_char(c) ((c) == ' ' || (c) == '\t' || (c) == '\v' || (c) == '\n' || (c) == '\r')
#define str_equal(s, t) (!strcmp((s), (t)))
static inline int
escape_char_pred(int c) {
static const unsigned char table[256] = {
'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 0x62, 0x74, 0x6e, 0x76, 0x66, 0x72, 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x',
'x', 'x', 'x', 0, 0, 0, 0, 0, 0, 0, 0x27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0x5c, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 'x', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
};
return table[(unsigned char)c];
}
static inline int
unescape_char_pred(int c) {
switch(c) {
case 'b': return 8;
case 'f': return 12;
case 'n': return 10;
case 'r': return 13;
case 't': return 9;
case 'v': return 11;
case '\'': return 39;
case '\\': return 92;
}
return 0;
}
static inline int
is_escape_char(int c) {
return is_control_char(c) || c == '\\' || c == '\'' || c == 0x1b || c == 0;
}
static inline int
is_backslash_char(int c) {
return c == '\\';
}
//#define is_dot_char(c) ((c) == '.')0
//#define is_backslash_char(c) ((c) == '\\')
static inline int
is_dot_char(int c) {
return c == '.';
}
static inline int
is_identifier(const char* str) {
if(!((*str >= 'A' && *str <= 'Z') || (*str >= 'a' && *str <= 'z') || *str == '$' || *str == '_'))
return 0;
while(*++str)
if(!is_identifier_char(*str))
return 0;
return 1;
}
static inline int
is_integer(const char* str) {
if(*str == '-')
++str;
if(!(*str >= '1' && *str <= '9') && !(*str == '0' && str[1] == '\0'))
return 0;
while(*++str)
if(!is_digit_char(*str))
return 0;
return 1;
}
static inline size_t
byte_count(const void* s, size_t n, char c) {
const uint8_t* t;
uint8_t ch = (uint8_t)c;
size_t count;
for(t = (uint8_t*)s, count = 0; n; ++t, --n)
if(*t == ch)
++count;
return count;
}
static inline size_t
byte_chr(const void* str, size_t len, char c) {
const char* s = memchr(str, c, len);
if(s)
return s - (const char*)str;
return len;
}
static inline size_t
byte_rchr(const void* haystack, size_t len, char needle) {
const char *s, *t;
for(s = (const char*)haystack, t = s + len;;) {
--t;
if(s > t)
break;
if(*t == needle)
return (size_t)(t - s);
}
return len;
}
/*size_t
byte_rchr(const void* str, size_t len, char c) {
const char* s = memrchr(str, c, len);
if(s)
return s - (const char*)str;
return len;
}*/
static inline size_t
byte_chrs(const void* str, size_t len, const char needle[], size_t nl) {
const char *s, *t;
for(s = str, t = s + len; s != t; s++)
if(byte_chr(needle, nl, *s) < nl)
break;
return s - (const char*)str;
}
static inline int
byte_diff(const void* a, size_t len, const void* b) {
size_t i;
for(i = 0; i < len; ++i) {
int r = ((unsigned char*)a)[i] - ((unsigned char*)b)[i];
if(r)
return r;
}
return 0;
}
static inline int
byte_diff2(const char* a, size_t alen, const char* b, size_t blen) {
if(alen < blen)
return -b[alen];
if(blen < alen)
return a[blen];
return byte_diff(a, alen, b);
}
static inline size_t
str_chr(const char* in, char needle) {
const char *t, c = needle;
for(t = in; *t; ++t)
if(*t == c)
break;
return (size_t)(t - in);
}
static inline size_t
str_chrs(const char* in, const char needles[], size_t nn) {
const char* t;
size_t i;
for(t = in; *t; ++t)
for(i = 0; i < nn; i++)
if(*t == needles[i])
return (size_t)(t - in);
return (size_t)(t - in);
}
static inline size_t
str_rchr(const char* s, char needle) {
const char *in, *found = 0;
for(in = s; *in; ++in)
if(*in == needle)
found = in;
return (size_t)((found ? found : in) - s);
}
static inline size_t
str_rchrs(const char* in, const char needles[], size_t nn) {
const char *s, *found = 0;
size_t i;
for(s = in; *s; ++s)
for(i = 0; i < nn; ++i)
if(*s == needles[i])
found = s;
return (size_t)((found ? found : s) - in);
}
static inline int
str_endb(const char* a, const char* x, size_t n) {
size_t alen = strlen(a);
a += alen - n;
return alen >= n && !memcmp(a, x, n);
}
/* str_ends returns 1 if the b is a suffix of a, 0 otherwise */
static inline int
str_ends(const char* a, const char* b) {
return str_endb(a, b, strlen(b));
}
static inline int
str_startb(const char* a, const char* x, size_t len) {
size_t i;
for(i = 0;; i++) {
if(i == len)
return 1;
if(a[i] != x[i])
break;
}
return 0;
}
static inline int
str_start(const char* a, const char* b) {
return str_startb(a, b, strlen(b));
}
#define str_contains(s, needle) (!!strchr((s), (needle)))
char* str_escape(const char*);
static inline size_t
str_count(const char* s, char c) {
size_t i, count = 0;
for(i = 0; s[i]; i++)
if(s[i] == c)
++count;
return count;
}
static inline size_t
str_copy(char* out, const char* in) {
char* s;
for(s = out; (*s = *in); ++s)
++in;
return (size_t)(s - out);
}
static inline size_t
str_copyn(char* out, const char* in, size_t n) {
char* s;
for(s = out; n-- && (*s = *in); ++s)
++in;
*s = '\0';
return (size_t)(s - out);
}
static inline char*
str_ndup(const char* s, size_t n) {
char* r = malloc(n + 1);
if(r == NULL)
return NULL;
memcpy(r, s, n);
r[n] = '\0';
return r;
}
size_t str_findb(const char*, const char*, size_t);
size_t str_find(const void*, const void*);
static inline size_t
predicate_find(const char* str, size_t len, int (*pred)(int32_t)) {
size_t pos;
for(pos = 0; pos < len; pos++)
if(pred(str[pos]))
break;
return pos;
}
static inline size_t
lookup_find(const char* str, size_t len, const char table[256]) {
size_t pos;
for(pos = 0; pos < len; pos++)
if(table[(unsigned char)str[pos]])
break;
return pos;
}
static inline char
escape_char_letter(char c) {
switch(c) {
case '\0': return '0';
case '\a': return 'a';
case '\b': return 'b';
case '\t': return 't';
case '\n': return 'n';
case '\v': return 'v';
case '\f': return 'f';
case '\r': return 'r';
case '\\': return '\\';
case '\'': return '\'';
}
return 0;
}
#define FMT_LONG 41 /* enough space to hold -2^127 in decimal, plus \0 */
#define FMT_ULONG 40 /* enough space to hold 2^128 - 1 in decimal, plus \0 */
#define FMT_8LONG 44 /* enough space to hold 2^128 - 1 in octal, plus \0 */
#define FMT_XLONG 33 /* enough space to hold 2^128 - 1 in hexadecimal, plus \0 */
size_t token_length(const char*, size_t, char delim);
size_t fmt_ulong(char*, uint32_t);
size_t scan_ushort(const char*, uint16_t*);
size_t fmt_longlong(char*, int64_t);
size_t fmt_ulonglong(char*, uint64_t);
size_t fmt_xlonglong(char*, uint64_t);
size_t fmt_xlonglong0(char*, uint64_t, size_t);
size_t fmt_8long(char* dest, uint32_t i);
size_t fmt_xlong(char* dest, uint32_t num);
size_t fmt_xlong0(char* dest, uint32_t num, size_t n);
size_t scan_longlong(const char*, int64_t*);
size_t scan_int(const char*, int32_t*);
size_t scan_uint(const char*, uint32_t*);
size_t scan_ulonglong(const char*, uint64_t*);
size_t scan_xlonglong(const char*, uint64_t*);
size_t scan_8longn(const char*, size_t, uint32_t* dest);
size_t scan_whitenskip(const char*, size_t);
size_t scan_nonwhitenskip(const char*, size_t);
size_t scan_line(const char*, size_t);
size_t scan_lineskip(const char*, size_t);
size_t scan_lineskip_escaped(const char*, size_t);
size_t scan_eolskip(const char*, size_t);
size_t utf8_strlen(const void*, size_t);
wchar_t* utf8_towcs(const char*);
char* utf8_fromwcs(const wchar_t*);
BOOL utf16_multiword(const void*);
int case_lowerc(int);
int case_starts(const char*, const char*);
int case_diffb(const void*, size_t, const void* T);
size_t case_findb(const void*, size_t, const void* what, size_t wlen);
size_t case_finds(const void*, const char*);
static inline int
scan_fromhex(unsigned char c) {
c -= '0';
if(c <= 9)
return c;
c &= ~0x20;
c -= 'A' - '0';
if(c < 6)
return c + 10;
return -1;
}
static inline size_t
scan_8long(const char* src, uint32_t* dest) {
return scan_8longn(src, (size_t)-1, dest);
}
static inline size_t
utf8_charlen(const char* in, size_t len) {
const uint8_t* next = (const void*)in;
int r = unicode_from_utf8((const uint8_t*)in, len, &next);
return r == -1 ? 0 : next - (const uint8_t*)in;
}
static inline int
utf8_charcode(const char* in, size_t len) {
const uint8_t* next = (const void*)in;
int r = unicode_from_utf8((const uint8_t*)in, len, &next);
return next > in ? r : -1;
}
BOOL utf16_multiword(const void*);
ssize_t write_file(const char* file, const void* buf, size_t len);
ssize_t puts_file(const char* file, const char* s);
size_t u64toa(char*, uint64_t num, int base);
size_t i64toa(char*, int64_t num, int base);
/**
* @}
*/
#endif /* defined(CHAR_UTILS_H) */

View File

@@ -1,631 +0,0 @@
/*
* C utilities
*
* Copyright (c) 2017 Fabrice Bellard
* Copyright (c) 2018 Charlie Gordon
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
#include <stdlib.h>
#include <stdio.h>
#include <stdarg.h>
#include <string.h>
#include "cutils.h"
void pstrcpy(char *buf, int buf_size, const char *str)
{
int c;
char *q = buf;
if (buf_size <= 0)
return;
for(;;) {
c = *str++;
if (c == 0 || q >= buf + buf_size - 1)
break;
*q++ = c;
}
*q = '\0';
}
/* strcat and truncate. */
char *pstrcat(char *buf, int buf_size, const char *s)
{
int len;
len = strlen(buf);
if (len < buf_size)
pstrcpy(buf + len, buf_size - len, s);
return buf;
}
int strstart(const char *str, const char *val, const char **ptr)
{
const char *p, *q;
p = str;
q = val;
while (*q != '\0') {
if (*p != *q)
return 0;
p++;
q++;
}
if (ptr)
*ptr = p;
return 1;
}
int has_suffix(const char *str, const char *suffix)
{
size_t len = strlen(str);
size_t slen = strlen(suffix);
return (len >= slen && !memcmp(str + len - slen, suffix, slen));
}
/* Dynamic buffer package */
static void *dbuf_default_realloc(void *opaque, void *ptr, size_t size)
{
return realloc(ptr, size);
}
void dbuf_init2(DynBuf *s, void *opaque, DynBufReallocFunc *realloc_func)
{
memset(s, 0, sizeof(*s));
if (!realloc_func)
realloc_func = dbuf_default_realloc;
s->opaque = opaque;
s->realloc_func = realloc_func;
}
void dbuf_init(DynBuf *s)
{
dbuf_init2(s, NULL, NULL);
}
/* return < 0 if error */
int dbuf_realloc(DynBuf *s, size_t new_size)
{
size_t size;
uint8_t *new_buf;
if (new_size > s->allocated_size) {
if (s->error)
return -1;
size = s->allocated_size * 3 / 2;
if (size > new_size)
new_size = size;
new_buf = s->realloc_func(s->opaque, s->buf, new_size);
if (!new_buf) {
s->error = TRUE;
return -1;
}
s->buf = new_buf;
s->allocated_size = new_size;
}
return 0;
}
int dbuf_write(DynBuf *s, size_t offset, const uint8_t *data, size_t len)
{
size_t end;
end = offset + len;
if (dbuf_realloc(s, end))
return -1;
memcpy(s->buf + offset, data, len);
if (end > s->size)
s->size = end;
return 0;
}
int dbuf_put(DynBuf *s, const uint8_t *data, size_t len)
{
if (unlikely((s->size + len) > s->allocated_size)) {
if (dbuf_realloc(s, s->size + len))
return -1;
}
memcpy_no_ub(s->buf + s->size, data, len);
s->size += len;
return 0;
}
int dbuf_put_self(DynBuf *s, size_t offset, size_t len)
{
if (unlikely((s->size + len) > s->allocated_size)) {
if (dbuf_realloc(s, s->size + len))
return -1;
}
memcpy(s->buf + s->size, s->buf + offset, len);
s->size += len;
return 0;
}
int dbuf_putc(DynBuf *s, uint8_t c)
{
return dbuf_put(s, &c, 1);
}
int dbuf_putstr(DynBuf *s, const char *str)
{
return dbuf_put(s, (const uint8_t *)str, strlen(str));
}
int __attribute__((format(printf, 2, 3))) dbuf_printf(DynBuf *s,
const char *fmt, ...)
{
va_list ap;
char buf[128];
int len;
va_start(ap, fmt);
len = vsnprintf(buf, sizeof(buf), fmt, ap);
va_end(ap);
if (len < sizeof(buf)) {
/* fast case */
return dbuf_put(s, (uint8_t *)buf, len);
} else {
if (dbuf_realloc(s, s->size + len + 1))
return -1;
va_start(ap, fmt);
vsnprintf((char *)(s->buf + s->size), s->allocated_size - s->size,
fmt, ap);
va_end(ap);
s->size += len;
}
return 0;
}
void dbuf_free(DynBuf *s)
{
/* we test s->buf as a fail safe to avoid crashing if dbuf_free()
is called twice */
if (s->buf) {
s->realloc_func(s->opaque, s->buf, 0);
}
memset(s, 0, sizeof(*s));
}
/* Note: at most 31 bits are encoded. At most UTF8_CHAR_LEN_MAX bytes
are output. */
int unicode_to_utf8(uint8_t *buf, unsigned int c)
{
uint8_t *q = buf;
if (c < 0x80) {
*q++ = c;
} else {
if (c < 0x800) {
*q++ = (c >> 6) | 0xc0;
} else {
if (c < 0x10000) {
*q++ = (c >> 12) | 0xe0;
} else {
if (c < 0x00200000) {
*q++ = (c >> 18) | 0xf0;
} else {
if (c < 0x04000000) {
*q++ = (c >> 24) | 0xf8;
} else if (c < 0x80000000) {
*q++ = (c >> 30) | 0xfc;
*q++ = ((c >> 24) & 0x3f) | 0x80;
} else {
return 0;
}
*q++ = ((c >> 18) & 0x3f) | 0x80;
}
*q++ = ((c >> 12) & 0x3f) | 0x80;
}
*q++ = ((c >> 6) & 0x3f) | 0x80;
}
*q++ = (c & 0x3f) | 0x80;
}
return q - buf;
}
static const unsigned int utf8_min_code[5] = {
0x80, 0x800, 0x10000, 0x00200000, 0x04000000,
};
static const unsigned char utf8_first_code_mask[5] = {
0x1f, 0xf, 0x7, 0x3, 0x1,
};
/* return -1 if error. *pp is not updated in this case. max_len must
be >= 1. The maximum length for a UTF8 byte sequence is 6 bytes. */
int unicode_from_utf8(const uint8_t *p, int max_len, const uint8_t **pp)
{
int l, c, b, i;
c = *p++;
if (c < 0x80) {
*pp = p;
return c;
}
switch(c) {
case 0xc0: case 0xc1: case 0xc2: case 0xc3:
case 0xc4: case 0xc5: case 0xc6: case 0xc7:
case 0xc8: case 0xc9: case 0xca: case 0xcb:
case 0xcc: case 0xcd: case 0xce: case 0xcf:
case 0xd0: case 0xd1: case 0xd2: case 0xd3:
case 0xd4: case 0xd5: case 0xd6: case 0xd7:
case 0xd8: case 0xd9: case 0xda: case 0xdb:
case 0xdc: case 0xdd: case 0xde: case 0xdf:
l = 1;
break;
case 0xe0: case 0xe1: case 0xe2: case 0xe3:
case 0xe4: case 0xe5: case 0xe6: case 0xe7:
case 0xe8: case 0xe9: case 0xea: case 0xeb:
case 0xec: case 0xed: case 0xee: case 0xef:
l = 2;
break;
case 0xf0: case 0xf1: case 0xf2: case 0xf3:
case 0xf4: case 0xf5: case 0xf6: case 0xf7:
l = 3;
break;
case 0xf8: case 0xf9: case 0xfa: case 0xfb:
l = 4;
break;
case 0xfc: case 0xfd:
l = 5;
break;
default:
return -1;
}
/* check that we have enough characters */
if (l > (max_len - 1))
return -1;
c &= utf8_first_code_mask[l - 1];
for(i = 0; i < l; i++) {
b = *p++;
if (b < 0x80 || b >= 0xc0)
return -1;
c = (c << 6) | (b & 0x3f);
}
if (c < utf8_min_code[l - 1])
return -1;
*pp = p;
return c;
}
#if 0
#if defined(EMSCRIPTEN) || defined(__ANDROID__)
static void *rqsort_arg;
static int (*rqsort_cmp)(const void *, const void *, void *);
static int rqsort_cmp2(const void *p1, const void *p2)
{
return rqsort_cmp(p1, p2, rqsort_arg);
}
/* not reentrant, but not needed with emscripten */
void rqsort(void *base, size_t nmemb, size_t size,
int (*cmp)(const void *, const void *, void *),
void *arg)
{
rqsort_arg = arg;
rqsort_cmp = cmp;
qsort(base, nmemb, size, rqsort_cmp2);
}
#endif
#else
typedef void (*exchange_f)(void *a, void *b, size_t size);
typedef int (*cmp_f)(const void *, const void *, void *opaque);
static void exchange_bytes(void *a, void *b, size_t size) {
uint8_t *ap = (uint8_t *)a;
uint8_t *bp = (uint8_t *)b;
while (size-- != 0) {
uint8_t t = *ap;
*ap++ = *bp;
*bp++ = t;
}
}
static void exchange_one_byte(void *a, void *b, size_t size) {
uint8_t *ap = (uint8_t *)a;
uint8_t *bp = (uint8_t *)b;
uint8_t t = *ap;
*ap = *bp;
*bp = t;
}
static void exchange_int16s(void *a, void *b, size_t size) {
uint16_t *ap = (uint16_t *)a;
uint16_t *bp = (uint16_t *)b;
for (size /= sizeof(uint16_t); size-- != 0;) {
uint16_t t = *ap;
*ap++ = *bp;
*bp++ = t;
}
}
static void exchange_one_int16(void *a, void *b, size_t size) {
uint16_t *ap = (uint16_t *)a;
uint16_t *bp = (uint16_t *)b;
uint16_t t = *ap;
*ap = *bp;
*bp = t;
}
static void exchange_int32s(void *a, void *b, size_t size) {
uint32_t *ap = (uint32_t *)a;
uint32_t *bp = (uint32_t *)b;
for (size /= sizeof(uint32_t); size-- != 0;) {
uint32_t t = *ap;
*ap++ = *bp;
*bp++ = t;
}
}
static void exchange_one_int32(void *a, void *b, size_t size) {
uint32_t *ap = (uint32_t *)a;
uint32_t *bp = (uint32_t *)b;
uint32_t t = *ap;
*ap = *bp;
*bp = t;
}
static void exchange_int64s(void *a, void *b, size_t size) {
uint64_t *ap = (uint64_t *)a;
uint64_t *bp = (uint64_t *)b;
for (size /= sizeof(uint64_t); size-- != 0;) {
uint64_t t = *ap;
*ap++ = *bp;
*bp++ = t;
}
}
static void exchange_one_int64(void *a, void *b, size_t size) {
uint64_t *ap = (uint64_t *)a;
uint64_t *bp = (uint64_t *)b;
uint64_t t = *ap;
*ap = *bp;
*bp = t;
}
static void exchange_int128s(void *a, void *b, size_t size) {
uint64_t *ap = (uint64_t *)a;
uint64_t *bp = (uint64_t *)b;
for (size /= sizeof(uint64_t) * 2; size-- != 0; ap += 2, bp += 2) {
uint64_t t = ap[0];
uint64_t u = ap[1];
ap[0] = bp[0];
ap[1] = bp[1];
bp[0] = t;
bp[1] = u;
}
}
static void exchange_one_int128(void *a, void *b, size_t size) {
uint64_t *ap = (uint64_t *)a;
uint64_t *bp = (uint64_t *)b;
uint64_t t = ap[0];
uint64_t u = ap[1];
ap[0] = bp[0];
ap[1] = bp[1];
bp[0] = t;
bp[1] = u;
}
static inline exchange_f exchange_func(const void *base, size_t size) {
switch (((uintptr_t)base | (uintptr_t)size) & 15) {
case 0:
if (size == sizeof(uint64_t) * 2)
return exchange_one_int128;
else
return exchange_int128s;
case 8:
if (size == sizeof(uint64_t))
return exchange_one_int64;
else
return exchange_int64s;
case 4:
case 12:
if (size == sizeof(uint32_t))
return exchange_one_int32;
else
return exchange_int32s;
case 2:
case 6:
case 10:
case 14:
if (size == sizeof(uint16_t))
return exchange_one_int16;
else
return exchange_int16s;
default:
if (size == 1)
return exchange_one_byte;
else
return exchange_bytes;
}
}
static void heapsortx(void *base, size_t nmemb, size_t size, cmp_f cmp, void *opaque)
{
uint8_t *basep = (uint8_t *)base;
size_t i, n, c, r;
exchange_f swap = exchange_func(base, size);
if (nmemb > 1) {
i = (nmemb / 2) * size;
n = nmemb * size;
while (i > 0) {
i -= size;
for (r = i; (c = r * 2 + size) < n; r = c) {
if (c < n - size && cmp(basep + c, basep + c + size, opaque) <= 0)
c += size;
if (cmp(basep + r, basep + c, opaque) > 0)
break;
swap(basep + r, basep + c, size);
}
}
for (i = n - size; i > 0; i -= size) {
swap(basep, basep + i, size);
for (r = 0; (c = r * 2 + size) < i; r = c) {
if (c < i - size && cmp(basep + c, basep + c + size, opaque) <= 0)
c += size;
if (cmp(basep + r, basep + c, opaque) > 0)
break;
swap(basep + r, basep + c, size);
}
}
}
}
static inline void *med3(void *a, void *b, void *c, cmp_f cmp, void *opaque)
{
return cmp(a, b, opaque) < 0 ?
(cmp(b, c, opaque) < 0 ? b : (cmp(a, c, opaque) < 0 ? c : a )) :
(cmp(b, c, opaque) > 0 ? b : (cmp(a, c, opaque) < 0 ? a : c ));
}
/* pointer based version with local stack and insertion sort threshhold */
void rqsort(void *base, size_t nmemb, size_t size, cmp_f cmp, void *opaque)
{
struct { uint8_t *base; size_t count; int depth; } stack[50], *sp = stack;
uint8_t *ptr, *pi, *pj, *plt, *pgt, *top, *m;
size_t m4, i, lt, gt, span, span2;
int c, depth;
exchange_f swap = exchange_func(base, size);
exchange_f swap_block = exchange_func(base, size | 128);
if (nmemb < 2 || size <= 0)
return;
sp->base = (uint8_t *)base;
sp->count = nmemb;
sp->depth = 0;
sp++;
while (sp > stack) {
sp--;
ptr = sp->base;
nmemb = sp->count;
depth = sp->depth;
while (nmemb > 6) {
if (++depth > 50) {
/* depth check to ensure worst case logarithmic time */
heapsortx(ptr, nmemb, size, cmp, opaque);
nmemb = 0;
break;
}
/* select median of 3 from 1/4, 1/2, 3/4 positions */
/* should use median of 5 or 9? */
m4 = (nmemb >> 2) * size;
m = med3(ptr + m4, ptr + 2 * m4, ptr + 3 * m4, cmp, opaque);
swap(ptr, m, size); /* move the pivot to the start or the array */
i = lt = 1;
pi = plt = ptr + size;
gt = nmemb;
pj = pgt = top = ptr + nmemb * size;
for (;;) {
while (pi < pj && (c = cmp(ptr, pi, opaque)) >= 0) {
if (c == 0) {
swap(plt, pi, size);
lt++;
plt += size;
}
i++;
pi += size;
}
while (pi < (pj -= size) && (c = cmp(ptr, pj, opaque)) <= 0) {
if (c == 0) {
gt--;
pgt -= size;
swap(pgt, pj, size);
}
}
if (pi >= pj)
break;
swap(pi, pj, size);
i++;
pi += size;
}
/* array has 4 parts:
* from 0 to lt excluded: elements identical to pivot
* from lt to pi excluded: elements smaller than pivot
* from pi to gt excluded: elements greater than pivot
* from gt to n excluded: elements identical to pivot
*/
/* move elements identical to pivot in the middle of the array: */
/* swap values in ranges [0..lt[ and [i-lt..i[
swapping the smallest span between lt and i-lt is sufficient
*/
span = plt - ptr;
span2 = pi - plt;
lt = i - lt;
if (span > span2)
span = span2;
swap_block(ptr, pi - span, span);
/* swap values in ranges [gt..top[ and [i..top-(top-gt)[
swapping the smallest span between top-gt and gt-i is sufficient
*/
span = top - pgt;
span2 = pgt - pi;
pgt = top - span2;
gt = nmemb - (gt - i);
if (span > span2)
span = span2;
swap_block(pi, top - span, span);
/* now array has 3 parts:
* from 0 to lt excluded: elements smaller than pivot
* from lt to gt excluded: elements identical to pivot
* from gt to n excluded: elements greater than pivot
*/
/* stack the larger segment and keep processing the smaller one
to minimize stack use for pathological distributions */
if (lt > nmemb - gt) {
sp->base = ptr;
sp->count = lt;
sp->depth = depth;
sp++;
ptr = pgt;
nmemb -= gt;
} else {
sp->base = pgt;
sp->count = nmemb - gt;
sp->depth = depth;
sp++;
nmemb = lt;
}
}
/* Use insertion sort for small fragments */
for (pi = ptr + size, top = ptr + nmemb * size; pi < top; pi += size) {
for (pj = pi; pj > ptr && cmp(pj - size, pj, opaque) > 0; pj -= size)
swap(pj, pj - size, size);
}
}
}
#endif

View File

@@ -1,341 +0,0 @@
/*
* C utilities
*
* Copyright (c) 2017 Fabrice Bellard
* Copyright (c) 2018 Charlie Gordon
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
#ifndef CUTILS_H
#define CUTILS_H
#include <stdlib.h>
#include <string.h>
#include <inttypes.h>
#define likely(x) __builtin_expect(!!(x), 1)
#define unlikely(x) __builtin_expect(!!(x), 0)
#define force_inline inline __attribute__((always_inline))
#define no_inline __attribute__((noinline))
#define __maybe_unused __attribute__((unused))
#define xglue(x, y) x ## y
#define glue(x, y) xglue(x, y)
#define stringify(s) tostring(s)
#define tostring(s) #s
#ifndef offsetof
#define offsetof(type, field) ((size_t) &((type *)0)->field)
#endif
#ifndef countof
#define countof(x) (sizeof(x) / sizeof((x)[0]))
#endif
#ifndef container_of
/* return the pointer of type 'type *' containing 'ptr' as field 'member' */
#define container_of(ptr, type, member) ((type *)((uint8_t *)(ptr) - offsetof(type, member)))
#endif
typedef int BOOL;
#ifndef FALSE
enum {
FALSE = 0,
TRUE = 1,
};
#endif
void pstrcpy(char *buf, int buf_size, const char *str);
char *pstrcat(char *buf, int buf_size, const char *s);
int strstart(const char *str, const char *val, const char **ptr);
int has_suffix(const char *str, const char *suffix);
/* Prevent UB when n == 0 and (src == NULL or dest == NULL) */
static inline void memcpy_no_ub(void *dest, const void *src, size_t n) {
if (n)
memcpy(dest, src, n);
}
static inline int max_int(int a, int b)
{
if (a > b)
return a;
else
return b;
}
static inline int min_int(int a, int b)
{
if (a < b)
return a;
else
return b;
}
static inline uint32_t max_uint32(uint32_t a, uint32_t b)
{
if (a > b)
return a;
else
return b;
}
static inline uint32_t min_uint32(uint32_t a, uint32_t b)
{
if (a < b)
return a;
else
return b;
}
static inline int64_t max_int64(int64_t a, int64_t b)
{
if (a > b)
return a;
else
return b;
}
static inline int64_t min_int64(int64_t a, int64_t b)
{
if (a < b)
return a;
else
return b;
}
/* WARNING: undefined if a = 0 */
static inline int clz32(unsigned int a)
{
return __builtin_clz(a);
}
/* WARNING: undefined if a = 0 */
static inline int clz64(uint64_t a)
{
return __builtin_clzll(a);
}
/* WARNING: undefined if a = 0 */
static inline int ctz32(unsigned int a)
{
return __builtin_ctz(a);
}
/* WARNING: undefined if a = 0 */
static inline int ctz64(uint64_t a)
{
return __builtin_ctzll(a);
}
struct __attribute__((packed)) packed_u64 {
uint64_t v;
};
struct __attribute__((packed)) packed_u32 {
uint32_t v;
};
struct __attribute__((packed)) packed_u16 {
uint16_t v;
};
static inline uint64_t get_u64(const uint8_t *tab)
{
return ((const struct packed_u64 *)tab)->v;
}
static inline int64_t get_i64(const uint8_t *tab)
{
return (int64_t)((const struct packed_u64 *)tab)->v;
}
static inline void put_u64(uint8_t *tab, uint64_t val)
{
((struct packed_u64 *)tab)->v = val;
}
static inline uint32_t get_u32(const uint8_t *tab)
{
return ((const struct packed_u32 *)tab)->v;
}
static inline int32_t get_i32(const uint8_t *tab)
{
return (int32_t)((const struct packed_u32 *)tab)->v;
}
static inline void put_u32(uint8_t *tab, uint32_t val)
{
((struct packed_u32 *)tab)->v = val;
}
static inline uint32_t get_u16(const uint8_t *tab)
{
return ((const struct packed_u16 *)tab)->v;
}
static inline int32_t get_i16(const uint8_t *tab)
{
return (int16_t)((const struct packed_u16 *)tab)->v;
}
static inline void put_u16(uint8_t *tab, uint16_t val)
{
((struct packed_u16 *)tab)->v = val;
}
static inline uint32_t get_u8(const uint8_t *tab)
{
return *tab;
}
static inline int32_t get_i8(const uint8_t *tab)
{
return (int8_t)*tab;
}
static inline void put_u8(uint8_t *tab, uint8_t val)
{
*tab = val;
}
#ifndef bswap16
static inline uint16_t bswap16(uint16_t x)
{
return (x >> 8) | (x << 8);
}
#endif
#ifndef bswap32
static inline uint32_t bswap32(uint32_t v)
{
return ((v & 0xff000000) >> 24) | ((v & 0x00ff0000) >> 8) |
((v & 0x0000ff00) << 8) | ((v & 0x000000ff) << 24);
}
#endif
#ifndef bswap64
static inline uint64_t bswap64(uint64_t v)
{
return ((v & ((uint64_t)0xff << (7 * 8))) >> (7 * 8)) |
((v & ((uint64_t)0xff << (6 * 8))) >> (5 * 8)) |
((v & ((uint64_t)0xff << (5 * 8))) >> (3 * 8)) |
((v & ((uint64_t)0xff << (4 * 8))) >> (1 * 8)) |
((v & ((uint64_t)0xff << (3 * 8))) << (1 * 8)) |
((v & ((uint64_t)0xff << (2 * 8))) << (3 * 8)) |
((v & ((uint64_t)0xff << (1 * 8))) << (5 * 8)) |
((v & ((uint64_t)0xff << (0 * 8))) << (7 * 8));
}
#endif
/* XXX: should take an extra argument to pass slack information to the caller */
typedef void *DynBufReallocFunc(void *opaque, void *ptr, size_t size);
typedef struct DynBuf {
uint8_t *buf;
size_t size;
size_t allocated_size;
BOOL error; /* true if a memory allocation error occurred */
DynBufReallocFunc *realloc_func;
void *opaque; /* for realloc_func */
} DynBuf;
void dbuf_init(DynBuf *s);
void dbuf_init2(DynBuf *s, void *opaque, DynBufReallocFunc *realloc_func);
int dbuf_realloc(DynBuf *s, size_t new_size);
int dbuf_write(DynBuf *s, size_t offset, const uint8_t *data, size_t len);
int dbuf_put(DynBuf *s, const uint8_t *data, size_t len);
int dbuf_put_self(DynBuf *s, size_t offset, size_t len);
int dbuf_putc(DynBuf *s, uint8_t c);
int dbuf_putstr(DynBuf *s, const char *str);
static inline int dbuf_put_u16(DynBuf *s, uint16_t val)
{
return dbuf_put(s, (uint8_t *)&val, 2);
}
static inline int dbuf_put_u32(DynBuf *s, uint32_t val)
{
return dbuf_put(s, (uint8_t *)&val, 4);
}
static inline int dbuf_put_u64(DynBuf *s, uint64_t val)
{
return dbuf_put(s, (uint8_t *)&val, 8);
}
int __attribute__((format(printf, 2, 3))) dbuf_printf(DynBuf *s,
const char *fmt, ...);
void dbuf_free(DynBuf *s);
static inline BOOL dbuf_error(DynBuf *s) {
return s->error;
}
static inline void dbuf_set_error(DynBuf *s)
{
s->error = TRUE;
}
#define UTF8_CHAR_LEN_MAX 6
int unicode_to_utf8(uint8_t *buf, unsigned int c);
int unicode_from_utf8(const uint8_t *p, int max_len, const uint8_t **pp);
static inline BOOL is_surrogate(uint32_t c)
{
return (c >> 11) == (0xD800 >> 11); // 0xD800-0xDFFF
}
static inline BOOL is_hi_surrogate(uint32_t c)
{
return (c >> 10) == (0xD800 >> 10); // 0xD800-0xDBFF
}
static inline BOOL is_lo_surrogate(uint32_t c)
{
return (c >> 10) == (0xDC00 >> 10); // 0xDC00-0xDFFF
}
static inline uint32_t get_hi_surrogate(uint32_t c)
{
return (c >> 10) - (0x10000 >> 10) + 0xD800;
}
static inline uint32_t get_lo_surrogate(uint32_t c)
{
return (c & 0x3FF) | 0xDC00;
}
static inline uint32_t from_surrogate(uint32_t hi, uint32_t lo)
{
return 0x10000 + 0x400 * (hi - 0xD800) + (lo - 0xDC00);
}
static inline int from_hex(int c)
{
if (c >= '0' && c <= '9')
return c - '0';
else if (c >= 'A' && c <= 'F')
return c - 'A' + 10;
else if (c >= 'a' && c <= 'f')
return c - 'a' + 10;
else
return -1;
}
void rqsort(void *base, size_t nmemb, size_t size,
int (*cmp)(const void *, const void *, void *),
void *arg);
#endif /* CUTILS_H */

View File

@@ -1,483 +0,0 @@
#define _IN_DEBUG_C 1
#include "debug.h"
#include <quickjs.h>
#include <list.h>
#include <cutils.h>
#include <assert.h>
#include "defines.h"
#include <string.h>
#include <stdlib.h>
/**
* \addtogroup debug
* @{
*/
struct alloc_block {
struct list_head link;
const char* file;
int line;
size_t size;
};
#define ALLOC_BLOCK_SIZE sizeof(struct alloc_block)
#define ALLOC_BLOCK(p) (((struct alloc_block*)(p)) - 1)
#define ALLOC_PTR struct alloc_block*
#undef malloc
#undef calloc
#undef realloc
#undef strdup
#undef free
#undef js_malloc
#undef js_mallocz
#undef js_realloc
#undef js_strdup
#undef js_strndup
#undef js_free
#undef js_malloc_usable_size
#undef js_malloc_rt
#undef js_mallocz_rt
#undef js_realloc_rt
#undef js_free_rt
#undef js_malloc_usable_size_rt
thread_local struct list_head alloc_block_list = {0, 0};
static inline void
add_to_list(struct list_head* el, struct list_head* head) {
if(alloc_block_list.prev == 0 && alloc_block_list.next == 0)
init_list_head(&alloc_block_list);
list_add_tail(el, head);
}
int64_t
check_pointer(void* p) {
ALLOC_PTR ptr = ALLOC_BLOCK(p);
struct list_head* link;
int64_t ret = 0;
list_for_each(link, &alloc_block_list) {
if(link == &ptr->link)
return ret;
ret++;
}
// assert(0);
return -1;
}
void*
debug_malloc(size_t n, const char* file, int line) {
ALLOC_PTR ptr;
if((ptr = malloc(n + ALLOC_BLOCK_SIZE))) {
ptr->file = file;
ptr->line = line;
ptr->size = n;
add_to_list(&ptr->link, &alloc_block_list);
return &ptr[1];
}
return 0;
}
void*
debug_calloc(size_t m, size_t n, const char* file, int line) {
ALLOC_PTR ptr;
m *= n;
n = 1;
if((ptr = calloc(m + ALLOC_BLOCK_SIZE, 1))) {
ptr->file = file;
ptr->line = line;
ptr->size = n;
add_to_list(&ptr->link, &alloc_block_list);
return &ptr[1];
}
return 0;
}
void*
debug_realloc(void* p, size_t n, const char* file, int line) {
ALLOC_PTR ptr;
if(p) {
check_pointer(p);
ptr = ALLOC_BLOCK(p);
list_del(&ptr->link);
if(n == 0) {
free(ptr);
return 0;
}
ptr = realloc(ptr, n + ALLOC_BLOCK_SIZE);
} else {
ptr = malloc(n + ALLOC_BLOCK_SIZE);
}
if(ptr) {
ptr->file = file;
ptr->line = line;
ptr->size = n;
add_to_list(&ptr->link, &alloc_block_list);
return &ptr[1];
}
return 0;
}
void*
debug_strdup(const char* s, const char* file, int line) {
ALLOC_PTR ptr;
size_t len = strlen(s);
if((ptr = malloc(len + 1 + ALLOC_BLOCK_SIZE))) {
ptr->file = file;
ptr->line = line;
ptr->size = len + 1;
add_to_list(&ptr->link, &alloc_block_list);
memcpy(&ptr[1], s, len + 1);
return &ptr[1];
}
return 0;
}
void
debug_free(void* p, const char* file, int line) {
ALLOC_PTR ptr = ALLOC_BLOCK(p);
list_del(&ptr->link);
memset(ptr, 0xff, ALLOC_BLOCK_SIZE);
free(ptr);
}
void*
debug_js_malloc(JSContext* ctx, size_t n, const char* file, int line) {
ALLOC_PTR ptr;
if((ptr = js_malloc(ctx, n + ALLOC_BLOCK_SIZE))) {
ptr->file = file;
ptr->line = line;
ptr->size = n;
add_to_list(&ptr->link, &alloc_block_list);
return &ptr[1];
}
return 0;
}
void*
debug_js_mallocz(JSContext* ctx, size_t n, const char* file, int line) {
ALLOC_PTR ptr;
if((ptr = js_mallocz(ctx, n + ALLOC_BLOCK_SIZE))) {
ptr->file = file;
ptr->line = line;
ptr->size = n;
add_to_list(&ptr->link, &alloc_block_list);
return &ptr[1];
}
return 0;
}
void*
debug_js_realloc(JSContext* ctx, void* p, size_t n, const char* file, int line) {
ALLOC_PTR ptr;
if(p) {
check_pointer(p);
ptr = ALLOC_BLOCK(p);
list_del(&ptr->link);
if(n == 0) {
js_free(ctx, ptr);
return 0;
}
ptr = js_realloc(ctx, ptr, n + ALLOC_BLOCK_SIZE);
} else {
ptr = js_malloc(ctx, n + ALLOC_BLOCK_SIZE);
}
if(ptr) {
ptr->file = file;
ptr->line = line;
ptr->size = n;
add_to_list(&ptr->link, &alloc_block_list);
return &ptr[1];
}
return 0;
}
void*
debug_js_realloc2(JSContext* ctx, void* p, size_t size, size_t* pslack, const char* file, int line) {
void* ptr;
if((ptr = debug_js_realloc(ctx, p, size, file, line))) {
if(pslack) {
size_t new_size = debug_js_malloc_usable_size(ctx, ptr, file, line);
*pslack = (new_size > size) ? new_size - size : 0;
}
}
return ptr;
}
void*
debug_js_strdup(JSContext* ctx, const char* s, const char* file, int line) {
ALLOC_PTR ptr;
size_t len = strlen(s);
if((ptr = js_malloc(ctx, len + 1 + ALLOC_BLOCK_SIZE))) {
char* p = (char*)&ptr[1];
ptr->file = file;
ptr->line = line;
ptr->size = len + 1;
add_to_list(&ptr->link, &alloc_block_list);
memcpy(p, s, len + 1);
return p;
}
return 0;
}
void*
debug_js_strndup(JSContext* ctx, const char* s, size_t len, const char* file, int line) {
ALLOC_PTR ptr;
if((ptr = js_malloc(ctx, len + 1 + ALLOC_BLOCK_SIZE))) {
char* p = (char*)&ptr[1];
ptr->file = file;
ptr->line = line;
ptr->size = len + 1;
add_to_list(&ptr->link, &alloc_block_list);
memcpy(p, s, len);
p[len] = '\0';
return p;
}
return 0;
}
size_t
debug_js_malloc_usable_size(JSContext* ctx, const void* p, const char* file, int line) {
ALLOC_PTR ptr = ALLOC_BLOCK(p);
return js_malloc_usable_size(ctx, ptr) - ALLOC_BLOCK_SIZE;
}
void
debug_js_free(JSContext* ctx, void* p, const char* file, int line) {
ALLOC_PTR ptr;
check_pointer(p);
ptr = ALLOC_BLOCK(p);
list_del(&ptr->link);
memset(ptr, 0xff, ALLOC_BLOCK_SIZE);
js_free(ctx, ptr);
}
void*
debug_js_malloc_rt(JSRuntime* rt, size_t n, const char* file, int line) {
ALLOC_PTR ptr;
if((ptr = js_malloc_rt(rt, n + ALLOC_BLOCK_SIZE))) {
ptr->file = file;
ptr->line = line;
ptr->size = n;
add_to_list(&ptr->link, &alloc_block_list);
return &ptr[1];
}
return 0;
}
void*
debug_js_mallocz_rt(JSRuntime* rt, size_t n, const char* file, int line) {
ALLOC_PTR ptr;
if((ptr = js_mallocz_rt(rt, n + ALLOC_BLOCK_SIZE))) {
ptr->file = file;
ptr->line = line;
ptr->size = n;
add_to_list(&ptr->link, &alloc_block_list);
return &ptr[1];
}
return 0;
}
void*
debug_js_realloc_rt(JSRuntime* rt, void* p, size_t n, const char* file, int line) {
ALLOC_PTR ptr;
if(p) {
check_pointer(p);
ptr = ALLOC_BLOCK(p);
list_del(&ptr->link);
if(n == 0) {
js_free_rt(rt, ptr);
return 0;
}
ptr = js_realloc_rt(rt, ptr, n + ALLOC_BLOCK_SIZE);
} else {
ptr = js_malloc_rt(rt, n + ALLOC_BLOCK_SIZE);
}
if(ptr) {
ptr->file = file;
ptr->line = line;
ptr->size = n;
add_to_list(&ptr->link, &alloc_block_list);
return &ptr[1];
}
return 0;
}
size_t
debug_js_malloc_usable_size_rt(JSRuntime* rt, const void* p, const char* file, int line) {
ALLOC_PTR ptr = ALLOC_BLOCK(p);
return js_malloc_usable_size_rt(rt, ptr) - ALLOC_BLOCK_SIZE;
}
void
debug_js_free_rt(JSRuntime* rt, void* p, const char* file, int line) {
ALLOC_PTR ptr;
check_pointer(p);
ptr = ALLOC_BLOCK(p);
// printf("debug_js_free_rt %p\n", p);
list_del(&ptr->link);
memset(ptr, 0xff, ALLOC_BLOCK_SIZE);
js_free_rt(rt, ptr);
}
#undef malloc
#undef calloc
#undef realloc
#undef strdup
#undef free
#undef js_malloc
#undef js_mallocz
#undef js_realloc
#undef js_strdup
#undef js_strndup
#undef js_malloc_usable_size
#undef js_free
#undef js_malloc_rt
#undef js_mallocz_rt
#undef js_realloc_rt
#undef js_malloc_usable_size_rt
#undef js_free_rt
void*
orig_malloc(size_t size) {
return malloc(size);
}
void*
orig_calloc(size_t nelem, size_t elemsz) {
return calloc(nelem, elemsz);
}
void*
orig_realloc(void* ptr, size_t size) {
return realloc(ptr, size);
}
void*
orig_strdup(const char* str) {
return strdup(str);
}
void
orig_free(void* ptr) {
free(ptr);
}
void*
orig_js_malloc(JSContext* ctx, size_t size) {
return js_malloc(ctx, size);
}
void*
orig_js_mallocz(JSContext* ctx, size_t size) {
return js_mallocz(ctx, size);
}
void*
orig_js_realloc(JSContext* ctx, void* p, size_t size) {
return js_realloc(ctx, p, size);
}
void*
orig_js_strdup(JSContext* ctx, const char* str) {
return js_strdup(ctx, str);
}
void*
orig_js_strndup(JSContext* ctx, const char* str, size_t size) {
return js_strndup(ctx, str, size);
}
size_t
orig_js_malloc_usable_size(JSContext* ctx, const void* p) {
return js_malloc_usable_size(ctx, p);
}
void
orig_js_free(JSContext* ctx, void* p) {
return js_free(ctx, p);
}
void*
orig_js_malloc_rt(JSRuntime* rt, size_t size) {
return js_malloc_rt(rt, size);
}
void*
orig_js_mallocz_rt(JSRuntime* rt, size_t size) {
return js_mallocz_rt(rt, size);
}
void*
orig_js_realloc_rt(JSRuntime* rt, void* p, size_t size) {
return js_realloc_rt(rt, p, size);
}
size_t
orig_js_malloc_usable_size_rt(JSRuntime* rt, const void* p) {
return js_malloc_usable_size_rt(rt, p);
}
void
orig_js_free_rt(JSRuntime* rt, void* p) {
return js_free_rt(rt, p);
}
/**
* @}
*/

View File

@@ -1,165 +0,0 @@
#ifndef DEBUG_H
#define DEBUG_H
#include <quickjs.h>
#include <cutils.h>
#include "defines.h"
#ifndef QUICKJS_H
#error "quickjs.h not included"
#endif
/**
* \defgroup debug debug: Debugging helpers
* @{
*/
extern thread_local struct list_head alloc_block_list;
int64_t check_pointer(void*);
void* debug_malloc(size_t, const char*, int);
void* debug_calloc(size_t, size_t, const char*, int line);
void* debug_realloc(void*, size_t, const char*, int line);
void* debug_strdup(const char*, const char*, int);
void debug_free(void*, const char*, int);
void* debug_js_malloc(JSContext*, size_t, const char*, int line);
void* debug_js_mallocz(JSContext*, size_t, const char*, int line);
void* debug_js_realloc(JSContext*, void*, size_t, const char* file, int line);
void* debug_js_realloc2(JSContext*, void*, size_t, size_t* pslack, const char* file, int line);
void* debug_js_strdup(JSContext*, const char*, const char*, int line);
void* debug_js_strndup(JSContext*, const char*, size_t, const char* file, int line);
size_t debug_js_malloc_usable_size(JSContext*, const void*, const char*, int line);
void debug_js_free(JSContext*, void*, const char*, int line);
void* debug_js_malloc_rt(JSRuntime*, size_t, const char*, int line);
void* debug_js_mallocz_rt(JSRuntime*, size_t, const char*, int line);
void* debug_js_realloc_rt(JSRuntime*, void*, size_t, const char* file, int line);
size_t debug_js_malloc_usable_size_rt(JSRuntime*, const void*, const char*, int line);
void debug_js_free_rt(JSRuntime*, void*, const char*, int line);
#if !defined(_IN_DEBUG_C)
#if defined(DEBUG_ALLOC)
#define malloc(size) debug_malloc(size, __FILE__, __LINE__)
#define calloc(nelem, size) debug_calloc(nelem, size, __FILE__, __LINE__)
#define realloc(ptr, size) debug_realloc(ptr, size, __FILE__, __LINE__)
#define strdup(str) debug_strdup(str, __FILE__, __LINE__)
#define free(ptr) debug_free(ptr, __FILE__, __LINE__)
#define js_malloc(ctx, size) debug_js_malloc(ctx, size, __FILE__, __LINE__)
#define js_mallocz(ctx, size) debug_js_mallocz(ctx, size, __FILE__, __LINE__)
#define js_realloc(ctx, ptr, size) debug_js_realloc(ctx, ptr, size, __FILE__, __LINE__)
#define js_strdup(ctx, str) debug_js_strdup(ctx, str, __FILE__, __LINE__)
#define js_strndup(ctx, str, len) debug_js_strndup(ctx, str, len, __FILE__, __LINE__)
#define js_free(ctx, ptr) debug_js_free(ctx, ptr, __FILE__, __LINE__)
#define js_malloc_usable_size(ctx, ptr) debug_js_malloc_usable_size(ctx, ptr, __FILE__, __LINE__)
#define js_malloc_rt(rt, size) debug_js_malloc_rt(rt, size, __FILE__, __LINE__)
#define js_mallocz_rt(rt, size) debug_js_mallocz_rt(rt, size, __FILE__, __LINE__)
#define js_realloc_rt(rt, ptr, size) debug_js_realloc_rt(rt, ptr, size, __FILE__, __LINE__)
#define js_malloc_usable_size_rt(rt, ptr) debug_js_malloc_usable_size_rt(rt, ptr, __FILE__, __LINE__)
#define js_free_rt(rt, ptr) debug_js_free_rt(rt, ptr, __FILE__, __LINE__)
#endif
#ifdef DEBUG_ALLOC
#define realloc_helper(name) \
void* name(void* ptr, size_t size) { \
if(ptr == 0) \
return debug_malloc(size, __FILE__, __LINE__); \
if(size == 0) \
return debug_free(ptr, __FILE__, __LINE__); \
return debug_realloc(ptr, size, __FILE__, __LINE__); \
}
#define realloc2_helper(name) \
void* name(void* opaque, void* ptr, size_t size) { \
if(ptr == 0) \
return debug_malloc(size, __FILE__, __LINE__); \
if(size == 0) { \
debug_free(ptr, __FILE__, __LINE__); \
return 0; \
} \
return debug_realloc(ptr, size, __FILE__, __LINE__); \
}
#define js_realloc_helper(name) \
void* name(JSContext* ctx, void* ptr, size_t size) { \
if(ptr == 0) \
return debug_js_malloc(ctx, size, __FILE__, __LINE__); \
if(size == 0) { \
debug_js_free(ctx, ptr, __FILE__, __LINE__); \
return 0; \
} \
return debug_js_realloc(ctx, ptr, size, __FILE__, __LINE__); \
}
#define js_realloc_rt_helper(name) \
void* name(JSRuntime* rt, void* ptr, size_t size) { \
if(ptr == 0) \
return debug_js_malloc_rt(rt, size, __FILE__, __LINE__); \
if(size == 0) { \
debug_js_free_rt(rt, ptr, __FILE__, __LINE__); \
return 0; \
} \
return debug_js_realloc_rt(rt, ptr, size, __FILE__, __LINE__); \
}
#else
#define realloc_helper(name) \
void* name(void* ptr, size_t size) { \
if(ptr == 0) \
return malloc(size); \
if(size == 0) { \
free(ptr); \
return 0; \
} \
return realloc(ptr, size); \
}
#define realloc2_helper(name) \
void* name(void* opaque, void* ptr, size_t size) { \
if(ptr == 0) \
return malloc(size); \
if(size == 0) { \
free(ptr); \
return 0; \
} \
return realloc(ptr, size); \
}
#define js_realloc_helper(name) \
void* name(JSContext* ctx, void* ptr, size_t size) { \
if(ptr == 0) \
return orig_js_malloc(ctx, size); \
if(size == 0) { \
orig_js_free(ctx, ptr); \
return 0; \
} \
return orig_js_realloc(ctx, ptr, size); \
}
#define js_realloc_rt_helper(name) \
void* name(JSRuntime* rt, void* ptr, size_t size) { \
if(ptr == 0) \
return orig_js_malloc_rt(rt, size); \
if(size == 0) { \
orig_js_free_rt(rt, ptr); \
return 0; \
} \
return orig_js_realloc_rt(rt, ptr, size); \
}
#endif
#endif
void* orig_malloc(size_t);
void* orig_calloc(size_t, size_t);
void* orig_realloc(void*, size_t);
void* orig_strdup(const char*);
void orig_free(void*);
void* orig_js_malloc(JSContext*, size_t);
void* orig_js_mallocz(JSContext*, size_t);
void* orig_js_realloc(JSContext*, void*, size_t);
void* orig_js_strdup(JSContext*, const char*);
void* orig_js_strndup(JSContext*, const char*, size_t);
size_t orig_js_malloc_usable_size(JSContext*, const void*);
void orig_js_free(JSContext*, void*);
void* orig_js_malloc_rt(JSRuntime*, size_t);
void* orig_js_mallocz_rt(JSRuntime*, size_t);
void* orig_js_realloc_rt(JSRuntime*, void*, size_t);
size_t orig_js_malloc_usable_size_rt(JSRuntime*, const void*);
void orig_js_free_rt(JSRuntime*, void*);
/**
* @}
*/
#endif /* defined(DEBUG_H) */

View File

@@ -1,181 +0,0 @@
#ifndef DEFINES_H
#define DEFINES_H
/**
* \defgroup defines defines: Preprocessor definitions
* @{
*/
#ifdef _WIN32
#include <io.h>
#define FD_TO_SOCKET(fd) ((SOCKET)_get_osfhandle((fd)))
#define SOCKET_TO_FD(fh) (_open_osfhandle((intptr_t)(fh), O_RDWR | O_BINARY))
#else
#define FD_TO_SOCKET(fd) (fd)
#define SOCKET_TO_FD(fh) (fh)
#endif
#ifndef offsetof
#define offsetof(type, field) ((size_t) & ((type*)0)->field)
#endif
#ifndef inrange
#define inrange(value, min, max) ((value) >= (min) && (value) <= (max))
#endif
#define trim_dotslash(str) (!strncmp((str), "./", 2) ? (str) + 2 : (str))
#ifndef thread_local
#ifdef _Thread_local
#define thread_local _Thread_local
#elif defined(__GNUC__) || defined(__INTEL_COMPILER) || defined(__SUNPRO_CC) || defined(__IBMCPP__)
#define thread_local __thread
#elif defined(_WIN32)
#define thread_local __declspec(thread)
#else
#error No TLS implementation found.
#endif
#endif
#if defined(__GNUC__) || defined(__clang__)
#define PACK __attribute__((packed))
#define ENDPACK
#else
#define PACK #pragma pack(push, 1)
#define ENDPACK #pragma pack(pop)
#endif
#define JS_CGETSET_ENUMERABLE_DEF(prop_name, fgetter, fsetter, magic_num) \
{ \
.name = prop_name, .prop_flags = JS_PROP_ENUMERABLE | JS_PROP_CONFIGURABLE, .def_type = JS_DEF_CGETSET_MAGIC, .magic = magic_num, .u = { \
.getset = {.get = {.getter_magic = fgetter}, .set = {.setter_magic = fsetter}} \
} \
}
#define JS_CGETSET_MAGIC_FLAGS_DEF(prop_name, fgetter, fsetter, magic_num, flags) \
{ \
.name = prop_name, .prop_flags = flags, .def_type = JS_DEF_CGETSET_MAGIC, .magic = magic_num, .u = { \
.getset = {.get = {.getter_magic = fgetter}, .set = {.setter_magic = fsetter}} \
} \
}
#define JS_CFUNC_DEF_FLAGS(prop_name, length, func1, flags) \
{ \
.name = prop_name, .prop_flags = flags, .def_type = JS_DEF_CFUNC, .magic = 0, .u = {.func = {length, JS_CFUNC_generic, {.generic = func1}} } \
}
#define JS_CONSTANT_FLAGS(name, flags) JS_PROP_INT32_DEF(#name, name, (flags))
#define JS_CONSTANT(name) JS_PROP_INT32_DEF(#name, name, JS_PROP_CONFIGURABLE | JS_PROP_ENUMERABLE)
#define JS_CONSTANT_NONENUMERABLE(name) JS_PROP_INT32_DEF(#name, name, JS_PROP_CONFIGURABLE)
#ifdef JS_SHARED_LIBRARY
#if defined(_WIN32) || defined(__MINGW32__)
#define VISIBLE __declspec(dllexport)
#define HIDDEN
#else
#define VISIBLE __attribute__((visibility("default")))
#define HIDDEN __attribute__((visibility("hidden")))
#endif
#else
#define VISIBLE
#define HIDDEN
#endif
#ifndef MAX_NUM
#define MAX_NUM(a, b) ((a) > (b) ? (a) : (b))
#endif
#ifndef MIN_NUM
#define MIN_NUM(a, b) ((a) < (b) ? (a) : (b))
#endif
#ifndef ABS_NUM
#define ABS_NUM(n) ((n) < 0 ? -(n) : (n))
#endif
#ifndef MOD_NUM
#define MOD_NUM(n, divisor) ((((n) % (divisor)) + (divisor)) % (divisor))
#endif
#ifndef SIGN_NUM
#define SIGN_NUM(n) ((n) < 0)
#endif
#define COLOR_BLACK "\x1b[0;30m"
#define COLOR_RED "\x1b[0;31m"
#define COLOR_GREEN "\x1b[0;32m"
#define COLOR_BROWN "\x1b[0;33m"
#define COLOR_BLUE "\x1b[0;34m"
#define COLOR_PURPLE "\x1b[0;35m"
#define COLOR_MARINE "\x1b[0;36m"
#define COLOR_LIGHTGRAY "\x1b[0;37m"
#define COLOR_GRAY "\x1b[1;30m"
#define COLOR_NONE "\x1b[0m"
#define COLOR_LIGHTRED "\x1b[1;31m"
#define COLOR_LIGHTGREEN "\x1b[1;32m"
#define COLOR_YELLOW "\x1b[1;33m"
#define COLOR_LIGHTBLUE "\x1b[1;34m"
#define COLOR_MAGENTA "\x1b[1;35m"
#define COLOR_CYAN "\x1b[1;36m"
#define COLOR_WHITE "\x1b[1;37m"
#define BGCOLOR_RED "\x1b[48;5;124m"
#define BGCOLOR_BLUE "\x1b[48;5;20m"
#define BGCOLOR_YELLOW "\x1b[48;5;214m"
#define BGCOLOR_GREEN "\x1b[48;5;28m"
#define BGCOLOR_PINK "\x1b[48;5;165m"
#define JS_VALUE_FREE(ctx, value) \
do { \
JS_FreeValue((ctx), (value)); \
(value) = JS_UNDEFINED; \
} while(0);
#define JS_VALUE_FREE_RT(ctx, value) \
do { \
JS_FreeValueRT((ctx), (value)); \
(value) = JS_UNDEFINED; \
} while(0);
#if 0
#define js_object_tmpmark_set(value) \
do { \
((uint8_t*)JS_VALUE_GET_OBJ((value)))[5] |= 0x40; \
} while(0);
#define js_object_tmpmark_clear(value) \
do { \
((uint8_t*)JS_VALUE_GET_OBJ((value)))[5] &= ~0x40; \
} while(0);
#define js_object_tmpmark_isset(value) (((uint8_t*)JS_VALUE_GET_OBJ((value)))[5] & 0x40)
#else
#define js_object_tmpmark_set(value) \
do { \
JS_VALUE_GET_OBJ((value))->tmp_mark |= 0x40; \
} while(0);
#define js_object_tmpmark_clear(value) \
do { \
JS_VALUE_GET_OBJ((value))->tmp_mark &= ~0x40; \
} while(0);
#define js_object_tmpmark_isset(value) (JS_VALUE_GET_OBJ((value))->tmp_mark & 0x40)
#endif
#define js_runtime_exception_set(rt, value) \
do { \
*(JSValue*)((uint8_t*)(rt) + 216) = value; \
} while(0);
#define js_runtime_exception_get(rt) (*(JSValue*)((uint8_t*)(rt) + 216))
#define js_runtime_exception_clear(rt) \
do { \
if(!JS_IsNull(js_runtime_exception_get(rt))) \
JS_FreeValueRT((rt), js_runtime_exception_get(rt)); \
js_runtime_exception_set(rt, JS_NULL); \
} while(0)
#define JS_ATOM_TAG_INT (1U << 31)
#define JS_ATOM_MAX_INT (JS_ATOM_TAG_INT - 1)
#define JS_ATOM_ISINT(i) ((JSAtom)((i)&JS_ATOM_TAG_INT))
#define JS_ATOM_FROMINT(i) ((JSAtom)((i)&JS_ATOM_MAX_INT) | JS_ATOM_TAG_INT)
#define JS_ATOM_TOINT(i) (unsigned int)(((JSAtom)(i) & (~(JS_ATOM_TAG_INT))))
/**
* @}
*/
#endif /* defined(DEFINES_H) */

View File

@@ -1,45 +0,0 @@
/* File generated automatically by the QuickJS compiler. */
#include "quickjs-libc.h"
const uint32_t qjsc_hello_size = 87;
const uint8_t qjsc_hello[87] = {
0x43, 0x04, 0x0e, 0x63, 0x6f, 0x6e, 0x73, 0x6f,
0x6c, 0x65, 0x06, 0x6c, 0x6f, 0x67, 0x16, 0x48,
0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x57, 0x6f, 0x72,
0x6c, 0x64, 0x22, 0x65, 0x78, 0x61, 0x6d, 0x70,
0x6c, 0x65, 0x73, 0x2f, 0x68, 0x65, 0x6c, 0x6c,
0x6f, 0x2e, 0x6a, 0x73, 0x0c, 0x00, 0x06, 0x00,
0xa2, 0x01, 0x00, 0x01, 0x00, 0x03, 0x00, 0x00,
0x14, 0x01, 0xa4, 0x01, 0x00, 0x00, 0x00, 0x38,
0xe3, 0x00, 0x00, 0x00, 0x42, 0xe4, 0x00, 0x00,
0x00, 0x04, 0xe5, 0x00, 0x00, 0x00, 0x24, 0x01,
0x00, 0xcf, 0x28, 0xcc, 0x03, 0x01, 0x00,
};
static JSContext *JS_NewCustomContext(JSRuntime *rt)
{
JSContext *ctx = JS_NewContextRaw(rt);
if (!ctx)
return NULL;
JS_AddIntrinsicBaseObjects(ctx);
return ctx;
}
int main(int argc, char **argv)
{
JSRuntime *rt;
JSContext *ctx;
rt = JS_NewRuntime();
js_std_set_worker_new_context_func(JS_NewCustomContext);
js_std_init_handlers(rt);
ctx = JS_NewCustomContext(rt);
js_std_add_helpers(ctx, argc, argv);
js_std_eval_binary(ctx, qjsc_hello, qjsc_hello_size, 0);
js_std_loop(ctx);
js_std_free_handlers(rt);
JS_FreeContext(ctx);
JS_FreeRuntime(rt);
return 0;
}

View File

@@ -1,27 +0,0 @@
#ifndef TUTF8E_ISO_8859_1_H
#define TUTF8E_ISO_8859_1_H
#include <tutf8e.h>
static inline int tutf8e_iso_8859_1_string_length(const char *input, size_t *input_length, const char *invalid, size_t *output_length)
{
return tutf8e_encoder_string_length(tutf8e_encoder_iso_8859_1, input, input_length, invalid, output_length);
}
static inline int tutf8e_iso_8859_1_string_encode(const char *input, char *output, const char *invalid, size_t *output_length)
{
return tutf8e_encoder_string_encode(tutf8e_encoder_iso_8859_1, input, invalid, output, output_length);
}
static inline int tutf8e_iso_8859_1_buffer_length(const char *i, size_t input_length, const char *invalid, size_t *length)
{
return tutf8e_encoder_buffer_length(tutf8e_encoder_iso_8859_1, input, input_length, invalid, length);
}
static inline int tutf8e_iso_8859_1_buffer_encode(const char *i, size_t input_length, const char *invalid, char *output, size_t *output_length)
{
return tutf8e_encoder_buffer_encode(tutf8e_encoder_iso_8859_1, input, input_length, invalid, output, output_length);
}
#endif

View File

@@ -1,27 +0,0 @@
#ifndef TUTF8E_ISO_8859_10_H
#define TUTF8E_ISO_8859_10_H
#include <tutf8e.h>
static inline int tutf8e_iso_8859_10_string_length(const char *input, size_t *input_length, const char *invalid, size_t *output_length)
{
return tutf8e_encoder_string_length(tutf8e_encoder_iso_8859_10, input, input_length, invalid, output_length);
}
static inline int tutf8e_iso_8859_10_string_encode(const char *input, char *output, const char *invalid, size_t *output_length)
{
return tutf8e_encoder_string_encode(tutf8e_encoder_iso_8859_10, input, invalid, output, output_length);
}
static inline int tutf8e_iso_8859_10_buffer_length(const char *i, size_t input_length, const char *invalid, size_t *length)
{
return tutf8e_encoder_buffer_length(tutf8e_encoder_iso_8859_10, input, input_length, invalid, length);
}
static inline int tutf8e_iso_8859_10_buffer_encode(const char *i, size_t input_length, const char *invalid, char *output, size_t *output_length)
{
return tutf8e_encoder_buffer_encode(tutf8e_encoder_iso_8859_10, input, input_length, invalid, output, output_length);
}
#endif

View File

@@ -1,27 +0,0 @@
#ifndef TUTF8E_ISO_8859_11_H
#define TUTF8E_ISO_8859_11_H
#include <tutf8e.h>
static inline int tutf8e_iso_8859_11_string_length(const char *input, size_t *input_length, const char *invalid, size_t *output_length)
{
return tutf8e_encoder_string_length(tutf8e_encoder_iso_8859_11, input, input_length, invalid, output_length);
}
static inline int tutf8e_iso_8859_11_string_encode(const char *input, char *output, const char *invalid, size_t *output_length)
{
return tutf8e_encoder_string_encode(tutf8e_encoder_iso_8859_11, input, invalid, output, output_length);
}
static inline int tutf8e_iso_8859_11_buffer_length(const char *i, size_t input_length, const char *invalid, size_t *length)
{
return tutf8e_encoder_buffer_length(tutf8e_encoder_iso_8859_11, input, input_length, invalid, length);
}
static inline int tutf8e_iso_8859_11_buffer_encode(const char *i, size_t input_length, const char *invalid, char *output, size_t *output_length)
{
return tutf8e_encoder_buffer_encode(tutf8e_encoder_iso_8859_11, input, input_length, invalid, output, output_length);
}
#endif

View File

@@ -1,27 +0,0 @@
#ifndef TUTF8E_ISO_8859_13_H
#define TUTF8E_ISO_8859_13_H
#include <tutf8e.h>
static inline int tutf8e_iso_8859_13_string_length(const char *input, size_t *input_length, const char *invalid, size_t *output_length)
{
return tutf8e_encoder_string_length(tutf8e_encoder_iso_8859_13, input, input_length, invalid, output_length);
}
static inline int tutf8e_iso_8859_13_string_encode(const char *input, char *output, const char *invalid, size_t *output_length)
{
return tutf8e_encoder_string_encode(tutf8e_encoder_iso_8859_13, input, invalid, output, output_length);
}
static inline int tutf8e_iso_8859_13_buffer_length(const char *i, size_t input_length, const char *invalid, size_t *length)
{
return tutf8e_encoder_buffer_length(tutf8e_encoder_iso_8859_13, input, input_length, invalid, length);
}
static inline int tutf8e_iso_8859_13_buffer_encode(const char *i, size_t input_length, const char *invalid, char *output, size_t *output_length)
{
return tutf8e_encoder_buffer_encode(tutf8e_encoder_iso_8859_13, input, input_length, invalid, output, output_length);
}
#endif

Some files were not shown because too many files have changed in this diff Show More