develop's new fix-gtest-names pre-commit hook requires a snake_case test-case
name. CI runs it with --all-files, so the existing CamelCase names fail the
check on every branch that carries these files.
develop's new fix-gtest-names pre-commit hook requires a snake_case test-case
name. CI runs it with --all-files, so the existing CamelCase names fail the
check on every branch that carries this file.
gtest's EXPECT_EQ expands to an if/else, so an unbraced if around it is a
dangling-else error with warnings treated as errors. Found by the local
build of xrpl_tests.
A transaction names one or more accounts: the sender in Account, and by
type a Destination, Owner, Issuer, Holder and so on. tx.process now
carries each top-level account-typed field as its own attribute, keyed
tx_<field> in lower snake case (tx_account, tx_destination, ...), so an
account can be searched for in traces whatever role it played.
Addresses are public ledger identifiers and are emitted raw.
The keys live in TxAccountSpanNames.h in libxrpl, with a field-to-key
table in TxAccountSpanNames.cpp. A library test walks TxFormats and the
SField registry: every account field a transaction can carry has a key,
and no field that only ledger entries carry has one. An empty account
field is skipped rather than rendered as the zero address.
Rewrite the privacy policy (design decisions section 2.4.4) and the two
plan summaries that still described account hashing and a configurable
redaction. Add the two account attributes to the path-finding attribute
table and describe pathfind_dest_currency as the rendered asset.
An XRP account address is a public ledger identifier drawn from an
enumerable set. An unsalted hash of it is reversible by lookup, so it
protected nothing and only broke the join against explorers, RPC
responses and logs that show the same address.
- pathfind_source_account and pathfind_dest_account carry the request's
r-address. A value that does not parse as an r-address is not emitted,
so a malformed or mistaken request value never reaches a span. Both
handlers share setAccountAttribute() in PathFindSpanAttributes.h.
- pathfind_dest_currency is to_string(Asset): "XRP", "<issuer>/<CUR>",
or the MPT issuance id.
- The collector's attributes/hash processor is removed. No layer hashes.
- redactAccount() stays available; its header no longer claims to sit in
the emit path.
The scheme guard only ran when tls_client_cert was set, so use_tls=1 against an
http endpoint booted: the CA file was checked for readability, then the exporter
read TLS off the URL scheme alone and sent spans in the clear. An operator who
asks for TLS should get TLS or a startup error.
Widen the guard to every use_tls=1 node. tls_client_cert already requires
use_tls=1, so this strictly widens the old condition. The error message and the
function's description now name use_tls rather than the certificate key.
one_way_tls_on_a_plain_http_endpoint_is_accepted pinned the old behaviour and is
flipped to _throws, asserting the scheme message, the endpoint key and the URL.
Three cases that sent use_tls=1 at an http endpoint while asserting a file error
now set an https endpoint, so the scheme guard cannot be what throws.
All three stop() overrides cleared the global Telemetry instance
unconditionally, which assumes one live Application per process. The unit-test
binary breaks that whenever two environments run, and makeTelemetry() hands it
a NullTelemetryOtel, so the null implementations matter as much as the real one.
Compare against this before clearing, in all three.
An MPT issuance id is a sequence concatenated with the issuer's account id, so
emitting it whole put an account on a span in the clear, while the Issue arm
beside it redacts its issuer. The collector hashes only the two account
attributes, so nothing downstream caught it.
Hash the whole id: one stable token per asset, no issuer. The comment claiming
the id carries no address was wrong and is corrected.
The tx.process span read the fee with STAmount::xrp(), which throws on a
non-native amount. preflight1 rejects such a transaction with temBAD_FEE, but
the attribute runs first, so with tracing on the submit RPC returned
internalSubmit instead. Turning tracing on must not change a response.
Guard on native() and leave the attribute out otherwise. The peer path was
already contained by its own catch.
MonitoredMode::set calls onModeChange on every round start, so the span was
created whether or not the mode moved. A node with a steady mode therefore
emitted one mode_change per round carrying mode_old == mode_new, which a live
sweep confirmed on every round of both instrumented builds. The round span's
own consensus_mode attribute is still written on every call, since that is
where the round learns the mode it is running in.
accept.apply was a plain SpanGuard, so it never became the ambient span of
doAccept. The spans the function goes on to create inherited the activated
accept span instead and came out as accept.apply's siblings, while running
inside its own time window. Every guard was scoped before the SpanGuard split,
so this restores the hierarchy that design had.
Scoped now, so the hierarchy follows the call flow. Drops the parent-context
fallback arm with it: the accept context is captured only while the accept span
is live, and that span is a child of the round context, so an invalid accept
context implies an invalid round context and both arms returned an empty guard.
Only SpanGuard carried addEvent(name, attrs), so a call site holding a scoped
guard could not record an event attribute. Forwarding overload, with the no-op
twin in the telemetry-disabled stub, so a span can be converted between scoped
and unscoped without dropping the attributes on its events.
The find_library hint was pinned to the _RELEASE variable CMakeDeps
generates, so in any other configuration it expanded to nothing. The
archive was then found only via CMAKE_PREFIX_PATH, which can hand a Debug
build the Release archive instead of failing. Derive the suffix from
CMAKE_BUILD_TYPE and ask for the package's lib directory directly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SpanGuardScope.cpp kept both includes: each side added one and both symbols
are used in the merged test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tempo.yaml kept both sides' filter blocks: phase-2's six path-finding
filters ahead of this branch's three transaction filters, matching chain
order, and both header comment lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
RPCHandler.cpp composed both sides: phase-1c's null-guard and reply-aware
status logic, keeping this branch's load_type attribute inside that guard
because its argument allocates.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
propose() and validate() both injected the ambient span context, but no
span is ever activated on the threads that reach them: the round span is
deliberately non-ambient and the validation span is parented through a
stored context. So no TraceContext was written, every receiving peer took
its standalone-span fallback, and the consensus receive spans arrived as
orphan trace roots.
Inject from the span in hand instead, which is the form the propagation
helper documents and the transaction relay path already uses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both pathfind handlers returned on many paths without recording a status,
so a failed request produced a span that read as success. Route every exit
through one helper that reads the rpc error token off the reply, which also
covers the replies built further down the call chain.
The token set is fixed by the error registry, so it is safe as a span
label; raw request text would not be.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
rpc_status is a span-metrics dimension, so leaving it unset on this path
emitted a series with a blank label. Any query selecting on error missed
the failure entirely.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The store swap was undone by trailing statements, which a fatal assertion skips.
The thread store is a function-local static, so it outlives every test rather
than being reset between them: a skipped restore left it owning a stack object
from a dead frame, and the crash then landed in whichever test ran next.
The onCoro flag did not protect it either, because the cleanup function reads
that flag out of the freed object to decide not to delete it.
The guard is declared after both stack stores so it is destroyed before them.
This branch emits nine pathfind attributes and the datasource offered a dropdown
for none of them, so the signal was there but not searchable in Explore. The
file's own header states that each phase adds filters for what it introduces.
Six filters, for the attributes that select a request. The three counts are
measurements read off a span rather than things an operator searches by, so they
get none. ledger_index is dynamic because it takes a new value every ledger.
The case had no assertions at all, so it could not fail. Its comment also said
the attribute constants live in an xrpld-level header a libxrpl test cannot
include. ConsensusSpanNames.h is a libxrpl header, two sibling tests already
include it, and a comment fifty lines above says so correctly.
So the seven literal keys were never forced. They now come from the same
constants the emitter uses, and the case asserts the property it was written
for: the factory decides its verdict once at creation, and writing either
close-time outcome leaves the guard inert and publishing no propagation bytes.
The premise is asserted too, so a failure names which exit produced the null
guard the rest of the case rests on.
The status description was the fixed string error, so a trace recorded that a
request failed but not why.
It now carries the error token from the reply, falling back to the status's own
error code and then to the old string. Every source is a compile-time literal
from the error registry, so no request text reaches it. The rpc_status attribute
stays at success and error: that one is a span-metrics dimension, and widening
it would mint a series per token per command per node.
The work is gated on the span being live, so a build with telemetry compiled out
or a disabled guard pays nothing, matching how the neighbouring helper is
handled. asCString() is null-checked as well as type-checked, because it returns
the raw pointer where asString() guards it.
The comment above resolveCommandSpanName now states the invariant rather than
how it could be abused.
callMethod decided the rpc.command span's status from the Status the handler
returned. Handler.cpp registers 70 of its 72 methods through byRef(), which
returns a default Status whatever happened, because an old-style handler
reports its error in the reply body instead. So a failed account_info or a
refused path_find came out with rpc_status=success and span status Ok.
That is the opposite of what the comment above the code claimed it did, and it
left a {status.code=error} query blind to every non-throwing RPC error.
The status now comes from the reply as well as the Status, which covers both
handler styles. byRef() is unchanged and identical to develop: it behaves
correctly for its own purpose, and no RPC reply changes. Only telemetry was
reading the wrong signal.
setOk() is dropped rather than moved. The specification reserves Ok for an
operator asserting verified success and warns that a tool may treat it as
suppressing errors, so a successful call now leaves the status Unset.
No test accompanies this: the Beast tree has no telemetry fixture and the
in-memory span exporter is linked only into xrpl_tests, which cannot reach
daemon code. Asserting the exported status needs that infrastructure first.
CMake sets the define with add_compile_definitions, which is scoped to this
build, so it reached no consumer. Four public telemetry headers change class
layout on it: with the define SpanGuard holds a unique_ptr and its methods are
out of line in the library, without it SpanGuard is empty and all-inline.
A package compiling against the installed headers therefore built a different
SpanGuard than the one it linked. Exporting the define through cpp_info keeps
the two in step, and it stays inside the existing telemetry option so the Conan
option remains the only switch.
The C-string overload of setAttribute handed its pointer straight to
std::string_view, whose pointer constructor reads the pointer to find the
length. A null one is undefined behaviour.
A null pointer carries no text, and an empty value already means something on
this class, so nothing is recorded rather than an empty string.
The overload itself has to stay. Without it a string literal would bind to the
bool overload, because pointer-to-bool is a standard conversion and beats the
string_view one, so every string attribute would silently record true. The
header now records that.
Four labels in a flowchart wrote a backslash and an n to break a line.
Mermaid has no escape for that, so it printed the two characters inside the
node instead of wrapping. The labels are now markdown strings carrying a real
line break, which also auto-wrap.
The <br/> tags elsewhere in these diagrams are left alone. Mermaid enables HTML
labels by default and this repository overrides no Mermaid config, so they
render correctly.
resolveCommandSpanName() converted command/method to a string with no type
check. json::Value::asString() throws for an array or an object, so a request
whose nested method is [] reached that conversion and the throw replaced a
clean tooBusy reply with internal.
The overloaded path is the only way in: fillHandler() returns tooBusy before
anything has read those fields, and every other exit either converted them
itself or means neither field is present. The effect is that the error code a
client receives depends on whether telemetry was compiled in, which telemetry
must never do.
The span now falls back to its existing unknown-command label when either
present field is not a string. The WebSocket path already validates both
fields before dispatch, so it is left alone.
The test drives a genuinely overloaded job queue, reading the threshold from
the production constant rather than copying it, and asserts the client still
gets tooBusy. It lives in the Beast tree because doCommand is daemon code and
needs jtx, which the gtest binary cannot reach.
The document is internal and is kept outside the repo instead.
This also removes every link to it: the document-index rows in
OpenTelemetryPlan.md and 08-appendix.md, its node, edge and style lines in the
plan's Mermaid map, its own section in OpenTelemetryPlan.md, and the two
cross-reference notes in 02-design-decisions.md and 05-configuration-reference.md.
The compose ports published on every host interface, so the collector's
unauthenticated OTLP receivers, Tempo, and the anonymous-admin Grafana were
reachable by anything that could route to the host.
Nothing consumed the published ports from off-host: containers address each
other by compose service name, and every doc and script uses localhost.
Resolved src/libxrpl/tx/applySteps.cpp. calculateBaseFee now returns
std::expected<XRPAmount, TER>, so invokePreclaim rejects the transaction when
the fee cannot be computed. Kept that inside this branch's preclaimTer wrapper,
so the returned error also reaches the span's ter_result attribute.
Conflicted only on the two adjacent lines this branch had renamed from dotted
to bare attribute keys. Kept the bare keys, which are the convention the naming
check enforces.
The steps described setting a suppressed attribute on a dropped duplicate, in
the pre-rename dotted spelling. The span is now created after the duplicate
check, so there is no span on that path in either spelling.
The steps described setting a suppressed attribute on a dropped duplicate. The
span is now created after the duplicate check, so there is no span on that path.
The tx.receive span is created after the duplicate check, so it carries no
suppressed attribute and a dropped copy produces no span at all. How many were
dropped is the transactions_duplicate traffic category.
A peer relays every transaction it hears, so most inbound copies are ones
handleTransaction() drops. The tx.receive span was created before those checks
ran, so every dropped copy paid for a span, a hex transaction id, and an
open-ledger index read that takes the mutex the apply path needs.
Measured over a four-hour payment run: 83.2M of 101.1M tx.receive spans
described a copy the node dropped. That is 82% of this span and 23% of every
span the node emitted.
Move the span and its attributes below the duplicate check. How many copies
were dropped is already reported as the transactions_duplicate traffic
category, which costs no span. Why a copy was dropped is no longer recorded;
a labelled counter restores it on the branch that carries the metric registry.
The suppressed attribute and the suppressed and rejected_inner_batch status
values go with it. This function was their only user.
CMakeLists conflicted: this branch had rewritten the description to name
SpanGuard, phase-1b removed the option() call. Kept both — this branch's wording
with phase-1b's structure.
CMakeLists declared option(telemetry) with a default of its own, so the setting
had two homes and they were free to disagree. The Conan option already reaches
CMake without it: conanfile.py forwards the option into the generated toolchain,
which sets the variable this file reads, and every build here goes through Conan.
Drop the option() call and keep the if(telemetry) test. docs/build/telemetry.md
told readers a CMake option had to be set as well, so that goes with it.
Conflict in docker/telemetry/otel-collector-config.yaml, in the traces
pipeline: this branch added the attributes/hash processor while upstream
renamed the deprecated otlp exporter to otlp_grpc. Both sides kept — the
processor list keeps attributes/hash and the exporter list takes the new name.
Checked that the exporter definition key was renamed to match the reference,
and that the collector still loads the merged config.
The pinned collector warns on every start that "otlp" is a deprecated alias
for "otlp_grpc". Rename the trace exporter to otlp_grpc/tempo.
Only the exporter is affected. The otlp RECEIVER keeps its name: it serves
both gRPC and HTTP under one component and is not deprecated, verified by
renaming the exporter alone and seeing the warning stop.
This belongs on this branch because it introduces the exporter, and it is the
last of four deprecated aliases in the collector config; the other three are
owned by later branches in the chain.
readability-identifier-naming wants lower_case for a namespace, so
clang-tidy failed on this file under warnings-as-errors. All seven use
sites move with the declaration.
The rename script rewrites "Ripple epoch" to "XRPL epoch", so the old
spelling in a tracked .md makes the check-rename job fail on a dirty tree.
The attribute keys are left alone: the script's pattern needs a space, and
those keys are a cross-layer contract.
The rename script rewrites "Ripple epoch" to "XRPL epoch", so the old
spelling in a tracked .md makes the check-rename job fail on a dirty tree.
The attribute key close_time_ripple_epoch_s is left alone: the script's
pattern needs a space, and that key is a cross-layer contract.
requireReadableFile proved a path readable with getFileContents, which
loads the whole file into a std::string and then drops it. One of the
three paths it checks is tls_client_key, so a private key was loaded to
answer a question that does not need its contents. It now stats the
path, rejects anything that is not a regular file, and opens it without
reading. The message shape is unchanged:
"[telemetry] <key> cannot be read: <path> - <reason>".
A path naming a directory used to escape as an ios failure from the
stream buffer, naming neither the config key nor the path. It is now
rejected as "not a regular file" with both named. The new test covers
that case; it fails against the old implementation and against a copy
with the file-type branch removed.
The runbook's quick start and disable sections both told the reader to
run "cmake --preset default". No presets file is tracked, and the only
preset Conan generates is conan-release, so each of those steps failed
on its first command. Replaced with the flow BUILD.md documents, and
noted that telemetry is the current default while still passing the
flags.
Two conflicts, both additive.
TelemetryConfig.cpp: this branch added requireHttpsEndpoint next to
requireReadableFile; upstream added readConsensusTraceStrategy at the same spot.
Both kept.
05-configuration-reference.md: this branch added the two client-certificate rows
while upstream corrected the consensus strategy value from attribute to random.
Both kept. Also drops the stale "not yet implemented" row for
consensus_trace_strategy, which the merged table now contradicts twice over: the
option is parsed, and its value is no longer spelled attribute.
The plan doc offered `"attribute"` as the alternative to `"deterministic"`
for consensus_trace_strategy. The parser accepts `"random"`; "attribute"
described the correlation mechanism rather than the setting's value. Note
also that the alternative is experimental and not used.
retries_remaining is stamped on the txq.accept_tx span before the
transaction is applied and before the retry counter is decremented, so a
span with txq_status="retried" always shows a non-zero count and exhaustion
shows up as txq_status="failed" with zero. The attribute comment said only
"retries left before discard", which reads as a post-decrement value and
led to a runbook query that could never match.
Also rename the drifted consensus_trace_strategy value in the plan docs
from "attribute" to "random", the spelling the parser accepts, and note that
it is experimental and not used.
The OTLP/HTTP exporter selects TLS from the endpoint URL scheme alone
(HttpSslOptions in the pinned SDK matches "https:" exactly), so a client
certificate handed to it alongside an http:// traces_endpoint is loaded and
never presented. The parser checked the cert/key pairing, use_tls and file
readability, but never the scheme, and the default traces_endpoint is plain
HTTP. makeTelemetrySetup() now requires traces_endpoint to start with
"https://" whenever tls_client_cert is set, including when the key is left
at its default.
Nothing asserted the client options reaching the exporter, so a swapped
certificate and key would have passed every test. Move the options mapping
into makeTraceExporterOptions() and assert it at that boundary with
distinct certificate and key paths, plus a one-way-TLS control and a
use_tls=0 control. One case runs the whole path from a [telemetry] section.
Runbook and example-config fixes:
- tx.included is emitted per transaction of the agreed consensus set,
before buildLCL() applies anything, so it is a superset of the accepted
ledger rather than proof of inclusion.
- the dispute.resolve query used the descendant operator, but the event is
on the consensus.update_positions span itself, so it matched nothing.
- the exhausted-retries query asked for txq_status="retried" with
retries_remaining=0, which cannot occur: the attribute is stamped before
the attempt and the retried branch only runs while retries are left.
Exhaustion is txq_status="failed" with a zero count.
- consensus_round_id is an int64, so the two queries comparing it to a
quoted string matched nothing.
- note that consensus_trace_strategy=random is experimental and not used.
- note that a trailing "| attr = value" is rejected by current Tempo;
attribute filters belong inside the braces.
consensus_trace_strategy was read as a std::string and compared against the
literal "attribute" in startRoundTracing(), while the runbook documented
"deterministic" and "random". The documented value "random" therefore fell
through to the default and did nothing.
Parse the setting once into ConsensusTraceStrategy, so the consensus code
branches on a type. The accepted spellings are now "deterministic" and
"random"; anything else fails at startup instead of silently defaulting.
The behaviour behind the old "attribute" name is unchanged and is now
reached by "random".
Document consensus_trace_strategy in xrpld-example.cfg, stating that
"random" is experimental and not used: it gives each node its own trace id,
so one round arrives as one trace per node.
Also state on the tx.included event that it covers the agreed consensus set
before the ledger is built, so it is a superset of the accepted ledger.
Two conflicts, both additive on each side.
TelemetryConfig.cpp: include blocks only. This branch added FileUtilities.h for
the certificate readability checks; upstream added <limits> and <optional> for
the bounds parser. Both kept.
The TelemetryConfig test: this branch's mutual-TLS cases and upstream's
batch-bounds cases were added at the same positions, so the file is rebuilt from
both stages and carries all 32 tests. Two shared cases were each edited by one
side only, so the edited side wins in each: upstream asserts the batch defaults
in parse_empty_section, and this branch's parse_full_section writes a real
certificate file, which is now required since the parser opens it.
One conflict, in 06-implementation-phases.md: this branch had rewritten the
phase-4 task table with a Status column, a descoping note and a Spans Produced
section, while upstream corrected the class name in the old plain table. This
branch's section is kept and the name correction re-applied to its 4.1 row.
One conflict, in SpanGuard.h: this branch added struct TraceBytes and upstream
added enum SpanRole at the same position after TraceCategory. Unrelated
declarations, so both are kept.
Guards the validation the parser gained upstream: zero rejected for all three
keys, a non-numeric value raising std::runtime_error rather than leaking
boost::bad_lexical_cast, a negative value rejected instead of wrapping to
4294967295, both bounds accepted exactly, and batch_size held at or below
max_queue_size.
The catch is std::runtime_error, not std::exception, on purpose: if the parser
ever stops wrapping, a bad_cast escapes and the suite fails loudly instead of
swallowing it.
Two conflicts, both resolved by composing the sides rather than taking one.
cfg/xrpld-example.cfg: this branch had moved the batch-processor keys under
their own heading while upstream edited them in place, so a merge-both would
have documented them twice. Upstream's range sentences are applied to the
relocated block and the head-sampling note keeps its position.
02-design-decisions.md: the summary table changed on both sides for different
reasons. Upstream renamed ledger_index to current_ledger_seq and ledger_seq;
this branch had corrected the PathFinding row to the keys it actually emits.
Both are kept.
The category mapped every Rpc span to kServer, so one inbound request emitted
several nested server spans. Per the trace spec, SERVER covers server-side
handling of a remote request the client awaits, while INTERNAL is an operation
with a local parent. rpc.process and both rpc.command sites have a local parent,
so they now pass SpanRole::Internal.
The four transport-edge roots keep the category default: rpc.http_request,
rpc.ws_upgrade, rpc.ws_message and the gRPC span each begin a remote call. This
matters to Tempo's service-graph and span-metrics generators, which pair server
spans with client spans and leave a surplus one unpaired.
childSpan() takes the span name verbatim, so a bare op:: suffix names the span
"process" rather than "rpc.process". The same defect was corrected in the
SpanGuard and Telemetry examples; this is the last copy.
startRoundTracing runs as an argument to Consensus::startRound, so it creates
consensus.round before startRoundInternal applies the new mode. Reading mode_
there recorded the previous round's value, and a validator switching from
observing to proposing got a round span labelled observing that nothing corrected.
The attribute is now written in onModeChange, from the mode being applied. All
three MonitoredMode::set paths funnel through there, so round start, a wrong-ledger
switch and a bow-out all correct the parent span with one statement. Every path
reaches it under RCLConsensus::mutex_ on the thread that created the span.
The stale write is removed rather than kept alongside: neither Consensus::startRound
nor startRoundInternal has an early return before mode_.set, so every round span is
stamped. If a future path ever skipped it the attribute would be absent, which reads
as a gap, instead of confidently wrong. onClose also sets consensus_mode, from the
engine's own parameter, and is correct as it stands.
Also tests addEvent's attribute overload on a live span, reading the exported event
name and each value back off the in-memory exporter. It was previously only ever
called on a null guard, so a dropped attribute exported nothing and failed nothing.
tryDirectApply returns an engaged optional whenever the fee bar was cleared,
including when xrpl::apply() failed, so testing the optional labelled failures as
applied. TxQ_test's fail-in-preclaim case hits exactly this: the fee clears the
bar and preclaim then rejects with terINSUF_FEE_B.
The stamp now branches on ApplyResult::applied. A failure reports failed rather
than falling through to the default rejected, because rejected means the
transaction got nowhere, while this one cleared the fee bar and ran through
apply(). ter_code is recorded either way, so the failure is diagnosable. Both
values already existed and are used the same way by the queued-apply path in this
file, so the vocabulary is unchanged.
The value set in the phase-3 task list is updated to match, including a ter_code
row for txq.accept_tx that was already emitted but undocumented.
Span kind was derived from TraceCategory alone, so every Rpc-category span was
kServer. A category cannot tell an inbound handler from the internal work under
it, and trace backends pair kServer with kClient, so internal spans left as
kServer become unpaired edges in a service graph and read as extra inbound
requests.
SpanRole is a new xrpl-owned enum, orthogonal to TraceCategory: the category
names the subsystem and gates the span on config, the role says whether the span
handles a remote call. It is a defaulted fourth parameter on span(), freshRoot()
and the ScopedSpanGuard equivalents, defaulting to SpanRole::FromCategory, so no
existing call site changes. resolveSpanKind() applies an explicit role and falls
back to the category map, which keeps its single responsibility. The
telemetry-disabled stubs mirror all four signatures.
No call site passes a role yet. The two that need it are on a later branch.
Also fixes a ScopedSpanGuard example that passed a bare op:: suffix to
childSpan(), which takes the name verbatim. Naming the child rpc.command made it
a child of rpc.command.<cmd>, inverting the hierarchy, so the example's parent is
now rpc.process and the command attribute moved onto the command span.
Review feedback on the RPC integration PR.
The childSpan examples could not work as written. childSpan() takes its parent
from the ambient context and uses impl_ only as a liveness gate, so an unscoped
SpanGuard parent produced two siblings rather than a parent and child. The parent
is now a ScopedSpanGuard, the child no longer reuses the parent's name, and the
examples pass a full dotted constant because childSpan() takes the name verbatim.
Five of the ten Rule D tests could not fail. Four passed an empty L1 key set,
which makes the rule skip validation altogether; the fifth asserted an empty
result against an escaped-quote selector that extracted no labels at all. Each
now passes a nonempty L1 set and carries a known-bad label in the same
expression, so it asserts both that the intended labels are accepted and that
Rule D ran. Verified by disabling the rule: the old tests stay green, the new
ones all fail.
Span kind is not fixed here. categoryToSpanKind and the span factories belong to
the telemetry library, so the role parameter is routed to that branch, and the
two call sites here follow once it exists.
Two review findings on the telemetry library.
SpanContext::isValid() returned impl_ != nullptr, so it answered true for a
context holding no span. threadLocalContext() wraps whatever GetCurrent()
returns, and that is an empty Context on a thread with no active span, which
contradicted the documented "invalid context if none is active". It now asks the
Context for its span. childSpan(name, ctx) is the one caller whose behaviour
changes: a context with no span used to produce a new root span, and now returns
a null guard as its @return already promised.
The three batch settings went to the OTel BatchSpanProcessor unchecked. Three
ways that failed: zero was accepted for all of them; batch_size could exceed
max_queue_size, which the SDK documents as a precondition and does not enforce;
and a mistyped value let boost::bad_lexical_cast escape, which derives from
std::bad_cast rather than std::runtime_error, so the operator saw a bare "bad
cast" naming no key. Reading unsigned also turned "-1" into 4294967295 instead
of failing, so the value is parsed signed and negatives are rejected.
xrpld-example.cfg now states the ranges.
Review feedback on the plan documents. Four kinds of error:
- Symbols that do not exist: ConsensusProposal::prevLedger_ (it is
previousLedger_), RCLConsensusAdaptor (it is RCLConsensus::Adaptor, and
startRound() is on RCLConsensus itself), and RPCHandler::doCommand (a free
function, xrpl::rpc::doCommand).
- Attribute keys: the tables used ledger_index, which no telemetry code emits.
Same concept as ledger_seq but a different referent, so the code disambiguates
by prefix: current_ledger_seq for the open ledger a transaction targeted,
ledger_seq for a closed or validated one. A note now states which is which.
- TraceQL that does not parse: span-field predicates need braces, status.code
is not an intrinsic (status = error), and avg(duration) does not take a by
clause (avg_over_time does). All five re-tested against Tempo.
- The StatsD comparison omitted the Histogram instrument, which aggregates at
the point of measure, and the when-to-use table had no row for a metric that
spans cannot afford to carry.
The consensus.accept.apply row listed close_time, parent_close_time and
close_time_self. The emitted keys carry the unit and epoch suffix, so
update the row to match.
The emitted keys are close_time_ripple_epoch_s,
parent_close_time_ripple_epoch_s and close_time_self_ripple_epoch_s.
Update the consensus.accept.apply attribute tables and the close-time
attribute descriptions to match.
The emitted key is close_time_ripple_epoch_s, which names its unit and
epoch. Update the ledger attribute table to match.
ledger_index and ledger_tx_count in the same table belong to a separate
rename and are left as they are.
Both sides documented the same six [telemetry] keys, so the automatic merge
duplicated all of them. Resolved by keeping this branch's structure - which
already covers all 14 keys and groups them under TLS and batch-processor
headings - and folding in the corrections from the upstream side:
- endpoint is renamed to traces_endpoint, which is what the parser reads, and
described as used verbatim including its signal path.
- use_tls no longer claims to enable TLS. The exporter's URL scheme selects
TLS; this key only decides whether tls_ca_cert reaches it as a CA bundle.
- tls_ca_cert records that the path is not opened while the config is parsed,
so an unreadable file shows up as an export failure rather than at startup.
- service_instance_id explains that it is normally left unset and filled in
from the node public key during startup.
Section::value_or in 05-configuration-reference.md becomes Section::valueOr;
that member does not exist under the other spelling.
The commented [telemetry] block in cfg/xrpld-example.cfg documented 8 of the 14
keys the parser accepts. Add the six that were missing - service_instance_id,
use_tls, tls_ca_cert, batch_size, batch_delay_ms and max_queue_size - each with
the unit and default read from the parser, and rename the documented endpoint
key to traces_endpoint so it matches what makeTelemetrySetup reads.
use_tls is documented for what it does rather than what its name suggests: it
gates whether tls_ca_cert reaches the exporter as a CA bundle, while the scheme
of traces_endpoint is what selects TLS. The path is not opened during parsing,
so an unreadable file surfaces as an export failure at runtime.
05-configuration-reference.md named three symbols that do not exist:
setup_Telemetry, make_Telemetry and Section::value_or. Correct them to
makeTelemetrySetup, makeTelemetry and Section::valueOr.
NetworkOPs.cpp include block: develop replaced DeliveredAmount.h,
MPTokenIssuanceID.h and NFTSyntheticSerializer.h with the single
rpc/detail/SyntheticFields.h. Kept that and this branch's two telemetry
includes.
Two different facts were sharing one attribute key. The sync-diagnostics
branch already emits validation_status on consensus.validation.accept,
carrying what the validation store did (ValStatus: current, stale,
bad_seq, multiple, conflicting, unknown) with a mapping function and
tests behind it. This branch then added validation_status on
consensus.validation.receive for which exit the receive path took
(queued, dropped_diverged, dropped_load).
One key with two value domains means any aggregation that does not also
filter on span name mixes them. Merging the two branches also produced a
duplicate constexpr declaration in one header, which is how the compiler
surfaced it.
The established accept-path key keeps its name; this one becomes
validation_receive_status, qualified by the span phase it describes. The
values are unchanged.
Conflict in OpenTelemetryPlan/05-configuration-reference.md: phase-5 had
corrected the enabled and use_tls types to 0 or 1 and added the
tls_client_cert and tls_client_key rows, while the incoming side renamed
the endpoint option.
Composed both — phase-5's type corrections and its two mTLS rows are
kept, with the endpoint row renamed to traces_endpoint.
Conflict in OpenTelemetryPlan/05-configuration-reference.md: phase-3 had
widened the options table and added the tx_trace_strategy and
consensus_trace_strategy rows, while the incoming side renamed the
endpoint option.
Composed both — phase-3's wider layout and its two extra rows are kept,
with the endpoint row renamed to traces_endpoint.
The [telemetry] option table documents the config key operators copy.
The key is now traces_endpoint, named for the one OTLP signal it
carries, so the old row pointed at a key the parser no longer reads.
Two prose mentions of "endpoint" further down describe the concept
rather than naming the key, and are left alone.
The rename arrived from phase-1b by merge; the runbook still told
operators to set `endpoint`, which the parser no longer reads. Updates
the Quick Start ini block and the Configuration Reference row.
No metrics_endpoint row is added: this branch exports no metrics, so
documenting the key here would describe something the code ignores.
The rename arrived from phase-1b by merge, which left this test naming a
member that no longer exists. Updates both assertions to tracesEndpoint
and the section key to traces_endpoint.
The key matters as much as the member: had only the member been renamed,
the parse would have fallen back to the default and the test would have
compared the collector URL against localhost.
Two conflicts, both composed rather than taking a side.
SpanGuard.h: phase-4 added the followsFrom parameter and its @param block;
phase-3 added the @return line. Kept both.
Telemetry.cpp: phase-4 added getConsensusTraceStrategy() immediately above
getTracer()'s return type, where phase-3 added [[nodiscard]]. Kept both, in
both implementation classes.
[telemetry] endpoint carried one OTLP signal while its name implied it
covered every signal. That asymmetry is what let the metrics URL be
guessed later by rewriting this one's path suffix, so anything not
ending /v1/traces silently posted metrics to the traces path.
Renames the key to traces_endpoint and Setup::exporterEndpoint to
tracesEndpoint. The default value is unchanged and the URL is still used
verbatim, with no path derived from it. The startup log line and the
compose-file example name the new key, the latter being where an
operator copies it from.
No metrics_endpoint is added here: this branch has no metrics pipeline,
so the key would parse into a member nothing reads.
The trace-context checks validate bytes received from a peer, so a
discarded result means untrusted input was accepted unchecked. hashSpan(),
txReceiveSpan() and txProcessSpan() return an RAII guard; discarding one ends
the span on the same line it began.
The telemetry-disabled twins of both hashSpan overloads already carried the
attribute, so the two #ifdef arms now agree. Both hashSpan overloads also
gain the @return line their siblings already had, now that the result cannot
be dropped.
No caller in the chain discards any of these results.
getInstance(), getTracer(), both startSpan() overloads and networkTypeFromId()
return values that a caller must use. A discarded startSpan() result destroys
the returned span immediately, so the span opens and closes with no content.
Six methods in Telemetry.h already carried the attribute, on the base and on
every override. The new attributes follow that: the overrides in Telemetry.cpp
and NullTelemetry.cpp get it too, because [[nodiscard]] is not inherited and a
call bound to the derived type would otherwise be unchecked.
No caller anywhere in the chain discards any of these results.
consensus.validation.receive is created before the drop decision, so one
span name covered three exits with unrelated cost profiles: two drop
paths that end in microseconds, and a queued path whose handle is moved
into the job and so covers job wait plus checkValidation.
No existing attribute separated them. validation_trusted=true implies
the queued path, but validation_trusted=false spans both drop paths and
the queued path, so any quantile over the span mixed the populations.
Both drop paths are live in the default config, which sets
relayUntrustedValidations.
Adds validation_status, set once on each exit rather than as a default,
following the tx_status precedent in the same file: dropped_diverged,
queued, dropped_load. On the queued path it is set before the handle is
moved into the job.
The three apply-pipeline stages disagreed on span status. preflight set
Error on any non-success TER, preclaim never set it at all, and the apply
stage recorded nothing when a transaction threw.
- preclaim now sets Error for any non-success result, matching preflight.
Routine retryable results (terPRE_SEQ, telINSUF_FEE_P) are included, so
stage=preclaim error rates will rise and track normal queueing.
- The apply span no longer keys Error on canApply. A dry run reports
tesSUCCESS with canApply false, which was recorded as an error.
Every other path with canApply false already has a non-success result,
so !isTesSuccess covers them.
- Exceptions escaping the apply stage set ter_result=tefEXCEPTION and
record an exception event, then rethrow unchanged, mirroring what
invokePreflight and invokePreclaim already do. The caller still maps
the exception to tefEXCEPTION, so behaviour is unchanged.
- Two comments claimed every exit funnels through the logger lambda. A
throw does not, so they now say each return path.
The default setServiceInstanceId() body ignores its argument. Use the
attribute rather than a (void) cast: the codebase already uses it 104 times
and the build is C++23.
Both comments pointed at a state the squash merge does not publish.
- Consensus.h: 'yields a null guard, same as before' had no antecedent in the
round or the function. Say instead that a null guard makes the setAttribute
calls below no-ops, which is what SpanGuard's impl_ guard does.
- ConsensusSpanLabels.h: 'Split from ConsensusSpanNames.h' describes a split
performed entirely within this change; both headers first appear here. The
dependency rationale and the diagram are unchanged.
Comments only, no behaviour change.
The comment said per-asset timing 'is no longer split into individual spans'.
This change introduces the span, so per-asset spans never existed for it to be
split out of, and the comparison resolves against nothing once squash-merged.
Comments only, no behaviour change.
The three close-time span attributes hold NetClock readings, which are whole
seconds since the XRP Ledger epoch of 2000-01-01 rather than the Unix epoch.
Neither the unit nor the epoch was recoverable from the key, so a consumer
rendering one as a wall-clock time without first adding the epoch offset lands
roughly thirty years early. The sibling close_resolution_ms already named its
unit, so the header disagreed with itself.
close_time -> close_time_ripple_epoch_s
parent_close_time -> parent_close_time_ripple_epoch_s
close_time_self -> close_time_self_ripple_epoch_s
Emitted values do not change; only the keys do. The public RPC response fields
of the same name are deliberately untouched, as renaming those would break the
ledger API.
Both sides added a guarded include block at the same point in
PeerImp.cpp: this branch's consensus receive-span header, and phase-3's
tx-tracing header. Kept both in one guard with a comment covering the two
reasons, rather than taking a side and dropping an include the file needs.
TxTracing.h declares txReceiveSpan() and txProcessSpan(). Both call sites
sit inside an XRPL_ENABLE_TELEMETRY guard, so with telemetry off the header
is included but nothing from it is used, which clang-tidy reports as an
unused header. Guard the include the same way so it is still there for the
default build.
startRoundTracing() is the only reader and writer of prevRoundSpanContext_,
and it is compiled out with telemetry, so a telemetry-off build sees an unused
private field and -Werror rejects it.
Marked rather than guarded: the two sibling span contexts are declared in both
configurations -- one of them escapes the warning only because an unguarded
accessor returns it -- and guarding this one alone would make the member set
depend on the build.
The table of contents in this file indexes third- and fourth-level headings,
so a new subsection that is absent from it is a gap rather than a style
choice.
Compiled out, compiled in and tracing, compiled in with no active span, and
compiled in but disabled by config all have to produce the right wire bytes,
and only two of them are obvious. Tabulate them, and record why the predicate
reads the context directly instead of calling GetSpan(), which allocates a
DefaultSpan in the no-span case.
mutable_ on a protobuf optional submessage allocates it and sets its has-bit
at the call site, before the helper can decide there is nothing to write. A
caller that dereferences it ships an empty TraceContext whenever nothing is
recorded, and its peers each take a branch to extract nothing.
Document the rule with the right and wrong forms side by side.
The conditional-compilation section promised zero overhead when telemetry is
not wanted. The span disappears, but the arguments passed to it do not: the
compiled-out guards are ordinary inline functions, so a to_string() or a hash
in an argument list still runs and its result is then discarded.
State that, show the guard that does remove the work, and name the opposite
case -- the metric macros, which discard their arguments and need no guard.
Both broadcast paths passed *msg.mutable_trace_context() to the injector,
which allocates the submessage and sets its has-bit before the injector can
decide there is nothing to write. A compile-time guard covered the
telemetry-off build, but a node with telemetry compiled in and no active
span -- a disabled category, telemetry disabled by config, or a round that
is not being traced -- still broadcast an empty TraceContext to every peer,
and every peer took its has_trace_context() branch to extract nothing.
Add SpanGuard::hasCurrentContext(), a predicate that tests the same two
conditions the injector bails out on without allocating, and an
injectCurrentContext(message) helper that uses it to decide whether to
create the submessage at all. Both consensus call sites now call the helper
unguarded.
Two conflicts, both in the telemetry include blocks.
SpanGuard.h: kept the union. The incoming side moves <memory> inside the
telemetry guard and adds <type_traits>; this branch adds <initializer_list>,
<utility> and the protocol::TraceContext forward declaration. Guarding
<memory> is correct here: the only std::shared_ptr uses are SpanContext's
member and constructor, both inside the guard, and SpanGuardHandle is a
template parameter name rather than a smart-pointer typedef.
NullTelemetry.cpp: took only the incoming guarded Journal.h block. The
incoming hunk also carried <memory> and <utility>, which this branch already
includes below the guarded OpenTelemetry block; taking them as well would
have tripped readability-duplicate-include.
The existing helper takes the TraceContext submessage, so every caller
writes *msg.mutable_trace_context(). On a protobuf optional field that
allocates the submessage and sets its has-bit before the helper runs, so a
message ships an empty TraceContext whenever nothing is recorded and its
peers take their has_trace_context() branch to extract nothing.
Add an overload taking the parent message, which decides whether to create
the submessage at all, and correct the header note that claimed the old
helper was already free.
The three helpers whose bodies are compiled out with telemetry read members
only inside the guard, so in a telemetry-off build they touch no member and
readability-convert-member-functions-to-static fires. WarningsAsErrors makes
that fatal.
Suppress it where the body is gated, matching OverlayImpl::reportDnsResolve.
Making them static instead would give the two configurations different
signatures, which is the hazard the gating pattern avoids.
Five sites on the consensus round path did telemetry-only work whether or
not anything could record it.
onClose set four attributes on the ledger-close span. Two cost real work
once per round: OpenLedger::current() takes currentMutex_ and copies a
shared_ptr just to read txCount, and the mode attribute builds a string.
The block now sits behind if (span).
doAccept read the previous close-time resolution and ran a lambda
returning a std::string, feeding the resolution_direction attribute and
nothing else, once per accepted ledger. Now behind if (doAcceptSpan).
makeAcceptSpan, startRoundTracing and createValidationSpan have wholly
telemetry bodies, so each body sits inside XRPL_ENABLE_TELEMETRY.
makeAcceptSpan then allocates no control block per accepted ledger; an
empty handle is safe because doAccept only passes it to activateIfLive(),
which tests it. Its attributes are additionally guarded on the span being
live. startRoundTracing's early return sits after two virtual Telemetry
calls and a strategy string compare, so the whole body is compiled out
rather than reached each round. createValidationSpan yields std::nullopt,
and its two call sites in validate() test the guard as well as the
optional -- an engaged optional holding a dead guard still turned a
32-byte ledger hash into a 64-character string.
Telemetry.h and SpanNames.h are named only by startRoundTracing, so their
includes are guarded the same way to keep misc-include-cleaner satisfied
when telemetry is off.
onPhaseEvent and onOutcomeEvent are left as they are: the generic
Consensus template calls both and the csf simulator implements both, so
they are part of the adaptor surface.
Both inbound peer-message handlers built a receive span and set its
attributes unconditionally. The proposal path turned two 32-byte hashes
into full hex strings and then took a 16-character substring of each --
four heap allocations per message -- and it ran for every inbound
proposal, trusted or untrusted. The validation path did a field lookup,
two flag reads and a sign-time conversion for every inbound validation,
including the ones dropped just below it for peer divergence or local
load.
The handles are declared empty and only the make_shared sits inside
XRPL_ENABLE_TELEMETRY, so with telemetry compiled out neither path
allocates. The attribute blocks sit behind if (span && *span), which
also skips them when telemetry is compiled in but disabled in config,
and when the consensus trace category is off. It does not skip a span
that exists but was sampled out; that span still pays.
Both job bodies only carry the handle to hold the span alive and never
dereference it, so an empty handle is safe there. The validation span is
still built before the drop decision, so a dropped validation is still
traced; only its cost is removed.
ConsensusReceiveTracing.h has no other user in the file, so its include
is guarded the same way to keep misc-include-cleaner satisfied when
telemetry is off.
The comments claimed the guard skips work for a span that is "not being
recorded", which reads as sampling awareness. It has none: operator bool() is
impl_ != nullptr, and the span factories return an empty guard only when
telemetry is absent, disabled at runtime, or the trace category is off. A span
that exists but was sampled out still pays.
There is no isRecording() in the telemetry API, so the guard is still the
strongest available; only the justification was overstated.
The comments claimed the guard skips work for a span that is "not being
recorded", which reads as sampling awareness. It has none: operator bool() is
impl_ != nullptr, and the span factories return an empty guard only when
telemetry is absent, disabled at runtime, or the trace category is off. A span
that exists but was sampled out still pays.
There is no isRecording() in the telemetry API, so the guard is still the
strongest available; only the justification was overstated.
updateAll's update_all span is wholly telemetry: the optional guard, the
empty-requests test that decides whether to emit at all, and the two
attributes have no reader outside the span. It runs on every ledger close, so
with telemetry compiled out the function still constructed a stub guard and
discarded pathfind_ledger_index and pathfind_num_requests once a close for
nothing. Everything the rest of updateAll depends on, including the
isNewPathRequest() flag reset, is outside the block and unchanged.
No span object exists to test before it is created, so the guard is an #ifdef
over the whole block. The three includes it was the sole user of --
PathFindSpanNames.h, SpanGuard.h and <optional> -- are gated the same way,
because otherwise they would be unused includes in that build and
clang-tidy's misc-include-cleaner would reject them.
Two pieces of pathfinding telemetry ran regardless of the build.
doUpdate fills pathfind_dest_currency by rendering the destination asset for
the pathfind.compute span. For a non-XRP issue that is a base58 check encode
of the issuer, two SHA-256 rounds, then a SHA-512Half over the result and
three string allocations. It is a call argument, so it ran even where
setAttribute's body is empty. doUpdate is not a cold path: besides once per
pathfinding RPC, PathRequestManager::updateAll calls it once per active
path_find subscription on every ledger close, so a node with N subscriptions
paid N times a close. It now sits inside "if (span)" with the cheap
pathfind_fast flag, so it is skipped with telemetry compiled out and also for
any span that is not being recorded.
findPaths keeps a totalPaths counter across its per-source-asset loop. Its
only reader is the pathfind_num_paths attribute at the end of the same
function, so the counter is maintained only when telemetry is compiled in.
That needs an #ifdef rather than "if (span)": the attribute cannot read a
variable that does not exist, and a counter kept up to date but never read is
an unused variable, which fails the build.
doPathFind and doRipplePathFind fill two span attributes from the request's
source and destination accounts. Both values are call arguments, so they are
built whatever the build: asString() copies the address out of the JSON and
redactAccount() takes a SHA-512Half over it and formats 16 hex characters.
That is two copies and two hashes on every pathfinding RPC, for
pathfind_source_account and pathfind_dest_account, which nothing outside the
span reads.
Wrapping the block in "if (span)" drops that work when telemetry is compiled
out, where the guard's operator bool() is a literal false, and also when
telemetry is on but this span is not being recorded. The const-reference read
of context.params moves inside the guard with the code that needs it, so a
telemetry read still never inserts a null into the request.
resolveCommandSpanName() and the error span in doCommand() exist only to
name and label a telemetry span. With telemetry compiled out the
resolver still ran on every RPC that fillHandler() rejects: up to five
json isMember lookups, up to three string copies, a virtual config()
call and a handler-table lookup, all to build a name nobody records. A
storm of malformed requests paid that cost once per request.
A runtime `if (span)` guard cannot work here. The resolver's result is
the span name itself, passed as the third argument of the
ScopedSpanGuard constructor, so no span object exists yet to test. That
leaves `#ifdef XRPL_ENABLE_TELEMETRY`, matching the house style used
elsewhere.
The guard covers the whole telemetry block at the call site and the
helper definition too, so the file-static helper does not become an
unreferenced function, which the build rejects because warnings are
errors. injectError() and the error return stay outside the guard, so
behaviour on the failure path is unchanged. No include is orphaned:
ErrorCodes.h, SpanGuard.h, RpcSpanNames.h and <string_view> all keep
uses outside the guards.
With telemetry on nothing changes: only comments and the four directives
were added.
TxQ::accept set two of its per-transaction span attributes unconditionally, so
every queued transaction the loop tried to apply to the open ledger paid for a
64-char string built from the transaction id and a transToken() lookup that
builds a string, whether or not anything recorded them. That is once per
candidate clearing the required fee level, on every ledger close.
Guard both on the span being active, as the enqueue path in TxQ::apply and the
apply-pipeline spans already do. The transaction apply itself stays outside the
guard; only the attribute that reads its result is telemetry.
The rpc.ws_message span's command attribute was resolved for every
inbound WebSocket message, whether or not anything was recording it.
The resolver does two JSON member tests plus two subscripts, copies the
command into a std::string, calls getAPIVersionNumber and then looks the
name up in the handler multimap. Because it is a call argument to
setAttribute, it ran even with telemetry compiled out, where
setAttribute's body is empty.
Wrapping the call in "if (span)" drops that work entirely when telemetry
is off, and also when telemetry is on but this span is not being
recorded. Nothing outside the attribute reads the resolved value, so no
other behaviour changes; the real request validation further down
computes its own api version, command string and handler role.
updateOurPositions built a 64-character hex string from the transaction
id plus two std::to_string number conversions, then attached them to a
span event. That ran for every dispute that flipped position, on every
establish tick of every consensus round, whether or not anything was
recording.
The three strings and the event have no other consumer; the vote change
itself (mutableSet insert/erase) is untouched.
The block now sits behind if (span). With telemetry compiled out the
stub's operator bool is a literal false, so it is eliminated; with
telemetry on it is also skipped when the span is null because the
establish context was never captured. The guard is inside the function
body, so Consensus<Adaptor>'s adaptor interface is unchanged and the
csf::Peer simulator is unaffected.
doAccept's canonical-tx-set loop built a 64-character hex string from
every transaction hash and added a span event for it, then reported a
txCount that only the span reads. That ran once per transaction in every
accepted ledger, whether or not anything was recording.
txCount and txHash have no other consumer: txCount is only read by the
tx_count attribute, and txHash only by the tx_included event. buildLCL
takes retriableTxs, not the count.
Both now sit behind if (doAcceptSpan). With telemetry compiled out the
stub's operator bool is a literal false, so the blocks are eliminated;
with telemetry on they are also skipped whenever the span is null
because the consensus trace category is off.
TxQ::apply set its enqueue-span attributes unconditionally, so every transaction
paid for two 64-char hash strings and a TxFormats lookup whether or not anything
recorded them. The open-ledger rebuild replays transactions through this same
path, so it was paid more than once each.
Guard the block on the span being active, as the tx apply-pipeline spans do.
Transactor::operator()() set its apply-stage attributes unconditionally, so
every transaction applied paid for a TxFormats::findByType() lookup and a
64-char string built from the view's parent hash, plus one or two transToken()
lookups in the exit funnel, whether or not anything recorded them.
Guard both blocks on the span being active. This is the pattern the sibling
preflight and preclaim spans already use in applySteps.cpp, with a comment
giving this exact reason -- the apply stage was simply missed.
Behaviour is unchanged where the span is live, and setAttribute on an inactive
guard was already a no-op.
The tx.receive and tx.process spans were built with make_shared whatever the
build, so a node without telemetry allocated once per inbound, submitted and
relayed transaction -- duplicates included, since tx.receive runs before the
duplicate check -- to hold an empty object.
Leave the handle null in that build instead. Nothing needed to change to carry
it: both doTransaction* overloads already default the span to nullptr, the job
capture and activateIfLive() accept a null handle, and the apply loop already
tested `e.span && *e.span` before using it. The remaining uses now test the
handle, which they have to do anyway once it can be null.
With telemetry compiled in the behaviour is unchanged, and the attribute block
is still skipped for any span that is not being recorded.
Both the proposal and validation broadcast paths called
injectCurrentContextToProtobuf(*msg.mutable_trace_context()) unconditionally.
The injector is a no-op when telemetry is compiled out, but its argument is not:
mutable_ on an optional submessage allocates it and sets its has-bit. So a node
built without telemetry put an empty TraceContext in every proposal and every
validation it broadcast, and made each receiving peer take its
has_trace_context() branch for nothing.
Guard both calls. The relay path in NetworkOPs already tests its span before
touching mutable_trace_context(), so this brings the two in line.
Both tx.receive and tx.process set their attributes unconditionally, so the
work happened even when nothing consumed it: a 64-char hash string allocation,
a getCurrentLedgerIndex() call that takes the ledger master's lock, a
TxFormats lookup, a peer-version string copy, and in tx.process a fee and
sequence decode. tx.receive runs before the duplicate check, so duplicate
relays paid for it too.
Guard both blocks on the span being live. When telemetry is compiled out the
guard's operator bool() is a literal false and the block is eliminated; when it
is compiled in the block is skipped for any span that is not being recorded,
which the previous code could not do. Behaviour is unchanged where the span is
live, and setAttribute on a null guard was already a no-op.
The remaining attributes at the exit paths keep their unconditional calls: their
arguments are compile-time constants, so there is nothing to save.
isValidTraceContext and std::uint8_t are named only inside the two
telemetry-enabled branches, so with telemetry compiled out both includes are
unused and clang-tidy fails the build on warnings-as-errors.
isValidSpanId and std::uint8_t are named only inside the telemetry-enabled
branch of txReceiveSpan, so with telemetry compiled out both includes are
unused and clang-tidy fails the build on warnings-as-errors.
With telemetry compiled out, <string_view> in Telemetry.h and <memory> in
SpanGuard.h are named only by declarations that are themselves guarded, so
clang-tidy's include-cleaner reports them unused and warnings-as-errors fails
the build. NullTelemetry.cpp has the mirror problem: it names beast::Journal
only in the compiled-out makeTelemetry(), reaching the type transitively.
Guard each include to match the configuration that uses it, and correct three
comments that overstated what the empty destructors cost.
The compiled-out ScopedActivation, SpanGuard and ScopedSpanGuard each used a
defaulted destructor. A defaulted destructor on an empty class is trivial, so
compilers report any guard held only for its scope as an unused variable. With
telemetry compiled out that produced seven -Wunused-variable errors under
-Werror, across the ledger acquire, consensus, ledger master and overlay paths.
Write the three destructors by hand so destruction is not trivial, matching the
telemetry-enabled types, and assert that property so it cannot quietly regress
to `= default`. The bodies are empty, so no code is generated either way.
Also move <memory> inside the telemetry guard: only the telemetry-enabled types
hold a unique_ptr, so the include is unused when telemetry is compiled out.
The block was added on the StatsD branch, but telemetry sources start here.
Codecov config only flows child-ward, so every branch between this one and
that one kept reporting telemetry files as uncovered patch lines.
Also corrects the rationale. The old comment said telemetry is "not enabled
in coverage builds"; it is enabled — conanfile.py and CMakeLists.txt both
default it ON, and the coverage matrix leg does not turn it off. The reason
these files record no coverage is that the unit-test suite never starts an
exporter.
Adds *SpanLabels.h alongside *SpanNames.h: same compile-time-constant
category, and the existing glob did not match it.
This narrows the patch gap but does not close it — instrumentation added to
consensus and overlay files is still counted, so codecov/patch stays red on
the early phases.
Splitting the consensus span labels into their own header moved both
constexpr std::string_view helpers out of ConsensusSpanNames.h, but left
the include they needed behind. The header names string_view nowhere now,
and SpanNames.h already provides the type for the conversion operator.
clang-tidy misc-include-cleaner reports this as an error under
-warnings-as-errors, which fails the clang-tidy job for the whole chain.
close_time_avalanche_state is new; the row also did not say that the other
three are rewritten on every convergence iteration, so a reader could not tell
that the exported value is the last one rather than a series.
This row is byte-identical on phases 5 through 10, so it is edited here and
merges forward. The consensus.phase.open row is empty until phase 9 and is
updated there instead.
The Phase 4a span table had no consensus.phase.open row at all and listed
only three attributes for consensus.establish. Add the row, the eight
attributes added on this branch, and the two label-valued ones to the
attribute inventory.
Leaves the pre-existing dotted names in the consensus.round row alone; they
predate the underscore convention and are not part of this change.
Review of the preceding commits found a clang-tidy failure and a convention
break, both rooted in the same place: the enum-to-label helpers were put in
ConsensusSpanNames.h, which pulled two domain headers into it.
misc-include-cleaner rejected the new test: it used xrpl::LedgerCloseReason
without directly including ConsensusTypes.h, relying on the transitive
include. misc-* is enabled and this path is not in IgnoreHeaders, so it would
have failed CI.
ConsensusSpanNames.h had also become the only one of the eight *SpanNames.h
headers to include anything beyond SpanNames.h. That cost is paid by every
consumer: PeerImp.cpp, ConsensusReceiveTracing.h and RCLConsensus.cpp want
only name and key constants, but were newly compiling ConsensusTypes.h and
DisputedTx.h through it.
Move both helpers to a new ConsensusSpanLabels.h, which owns the domain
includes. ConsensusSpanNames.h is dependency-free again like its siblings, and
the labels reach their only production caller, Consensus.h, directly.
Also from the review:
- phaseOpen() had grown to 81 lines, over the 80-line limit. Extract
annotateOpenStart() and annotateOpenClose(), which also removes the repeated
span guards. phaseOpen is 72 lines; startRoundInternal drops 103 to 93,
still over the limit but it was 99 before this work began.
- Note at the CLOG why the log text keeps the shouldCloseLedger name: existing
consumers match on it.
- whyCloseLedger's doc claimed "both log identically", implying the wrapper
logs too. It delegates, so the logging happens once either way.
- Cross-reference proposers_validated and proposers_finished, which sit eight
lines apart and count different things: validators of the previous ledger
versus those already past it.
- The two static_asserts no longer sit inside TEST bodies with SUCCEED(); they
fire at compile time regardless. Also "consteval-safe" was wrong; they are
constexpr.
- SpanGuardFactory.cpp claimed a libxrpl test cannot include the consensus
span-name header. The new test in the same directory does exactly that, so
the claim is corrected to name the real constraint: the rpc_* constants it
needs live in an xrpld-level header.
Three defects found in review of the two preceding commits.
Drop disputes_count_initial. It claimed to be the dispute count carried in
from the positions held at close, but startEstablishTracing() runs a full
timer tick after closeLedger(): timerEntry() dispatches
`if (phase_ == Open) phaseOpen(); else if (phase_ == Establish)
phaseEstablish();`, and phase_ was Open on the closing tick, so the else-if
cannot run. With ledgerGRANULARITY at 1s the value absorbed up to a second of
dispute growth from peer proposals and arriving tx sets. Making it honest
needs either a member captured at close or moving span creation into
closeLedger(), so it is removed rather than shipped mislabelled.
Record close_time_avalanche_state on recovered rounds. startRoundInternal()
reset establishSpan_ inline, discarding the span before the attribute was
written, so the value was present only on rounds that reached Accepted --
survivor bias in exactly the rounds worth investigating. It now calls
endEstablishTracing(). The comment claiming this avoided "reporting a stale
regime" was wrong: closeTimeAvalancheState_ is not reset until 39 lines
later, so the value was still that span's terminal regime.
Rename avalanche_state to close_time_avalanche_state. DisputedTx carries a
second, per-transaction avalanche tracker; the bare name invited reading a
close-time-only value as the transaction one, which is the tracker that
actually escalates in a stuck round.
Also: both label helpers now fall through to "unknown" instead of a
plausible-looking regime, matching to_string(ConsensusPhase); and the header
now records that the end-of-open attributes are absent on recovered and
simulated rounds, and that tx_sets_acquired can skew either way because
handleWrongLedger clears currPeerPositions_ but not acquired_.
Tests: the minimum-open-time assertion used prevRoundTime=10s, where
openTime=1s trips the too-fast branch as well, so deleting the ledgerMinClose
check entirely left it green. Replaced with prevRoundTime=2s, which isolates
the branch. Added the others-closed boundary, which is strict and was
untested in either direction, its integer truncation for odd prevProposers,
and its precedence over the no-transactions and minimum-open branches.
The open phase ended for one of four distinct reasons, but
shouldCloseLedger() collapsed them into a bool, so a trace could say when a
phase ended and never why. "The network closed without us" and "nothing was
waiting" are the same span today.
Add whyCloseLedger(), which holds the decision and returns
LedgerCloseReason. shouldCloseLedger() keeps its exact signature and becomes
a one-line delegation, so its callers and unit tests are untouched and the
branch logic is not duplicated. phaseOpen() calls whyCloseLedger() directly;
both emit the same journal and CLOG output, so only one is called.
New attributes on consensus.phase.open, both set once on the closing tick:
close_reason anomaly | others_closed | idle | normal
proposers_validated trusted peers that had already validated the prior
ledger, reusing the value the decision was made on
Absent on the simulate() close path, which bypasses the decision rather than
having a reason invented for it.
Skipped has_open_transactions: hasOpenTransactions() is
!getOpenLedger().empty(), which is false on a quiet network for most of a
round, and close_reason=idle already implies it. The sibling
consensus.ledger_close span carries tx_count_open, which is the same fact
with a count instead of a boolean.
shouldCloseLedger() now has no production caller; it stays exported so the
public API and its tests are unchanged.
Tests pin every input vector from should_close_ledger to its literal reason,
including that the anomaly check outranks others-closed, and cover the
inclusive idle boundary either side by one millisecond.
consensus.phase.open and consensus.establish carried almost no state of
their own. Span attributes are not inherited, so the ledger context on the
parent consensus.round span does not describe either child, and the few
attributes the establish span did carry are rewritten on every iteration
and therefore only ever report the final value.
Add seven attributes that are read from state already in scope, are
written exactly once, and are not duplicates of the parent round span:
consensus.phase.open (start)
start_reason initial, or recovered on a handleWrongLedger
re-entry, which emplaces a SECOND phase.open
span under the same round
previous_close_agree feeds the sinceClose branch in phaseOpen()
peer_positions_at_open positions in hand after playbackProposals(),
the head start the round began with
early_close_triggered the round skipped the timer because enough
peers had already closed
consensus.phase.open (end)
tx_sets_acquired candidate tx sets held at close, read before
our own position is added; a low count against
a high peer_positions_at_close means tx-set
fetches did not land, not disagreement
consensus.establish (start)
disputes_count_initial disputes carried in from the positions held at
close, as opposed to disputes_count, which is
overwritten each iteration
consensus.establish (end)
avalanche_state terminal close-time convergence regime; the
derived avalanche_threshold is a weight and
cannot be inverted back to the state
The avalanche label is mapped by a new constexpr avalancheStateLabel() in
ConsensusSpanNames.h rather than an inline switch, so the four labels stay
under the naming check's L1 ownership and are unit-testable.
Deliberately not added: ledger_seq and consensus_mode, which would only
copy the parent round span's values down; tx set size and position hash,
which the TxSet concept does not expose portably across RCLTxSet and the
csf simulator; and the peer-unchanged and dead-node counters, whose
underlying state is reset mid-round and so would report a misleading value.
Behaviour is unchanged. The early-close condition is hoisted into a named
local so the annotation happens before timerEntry(), which can reach
closeLedger() and end the open-phase span.
Tests pin the wire strings for every new key and value and cover all four
enumerators of the avalanche mapping. They need no telemetry runtime: the
csf simulator returns an invalid round span context and a null Telemetry,
so consensus spans there are null guards and attribute writes are no-ops.
Bring the three documentation surfaces in line with the new parse-time check:
- The @throws clause on makeTelemetrySetup now names the third failure
condition and records that an empty path is skipped.
- cfg/xrpld-example.cfg states, under all three TLS keys, that with enabled=1
and use_tls=1 a path that does not exist or cannot be read stops startup. The
tls_ca_cert wording still says that empty selects the system CA store, since
only a path that is set is checked.
- The runbook troubleshooting entry gains a third bullet for the "cannot be
read" message, whose remedy is the path or its permissions rather than the
certificate and key pairing.
Documentation only; no behaviour change.
With telemetry enabled and use_tls=1, makeTelemetrySetup now reads each
non-empty tls_ca_cert / tls_client_cert / tls_client_key path and refuses to
start when the file is missing or cannot be read. The message names the config
key, the path and the OS error, instead of leaving the problem to surface much
later as an opaque TLS handshake failure inside the exporter.
Reading the file with getFileContents, as the gRPC server already does for its
own ssl_cert and ssl_key pair, proves the file is both present and readable; an
existence test alone would miss a permissions problem. The contents are
discarded.
Both gates are deliberate. The check is skipped when enabled is 0, so a stale
cert line still cannot stop a node from booting, and when use_tls is 0, where
the exporter never opens the files. An empty path stays valid; for tls_ca_cert
it selects the system CA store.
Six GTest cases cover the three keys that can fail, the all-readable case, and
each gate on its own.
The configuration reference typed the five trace_* switches as bool with
default true. An xrpld config section carries integers, and these keys are
read with an integer cast, so a literal "true" fails to convert rather
than enabling the switch.
Type them as 0 or 1 with default 1, matching the other integer-valued
keys in the same table.
makeTelemetrySetup() rejects a contradictory [telemetry] mutual-TLS
setup by throwing, but it is called from ApplicationImp's
member-initializer list. A try/catch in the constructor body cannot
reach a throw from there, and nothing further up the stack caught it
either, so a config mistake reached std::terminate: the default handler
printed a terminate dump and raised SIGABRT, leaving a core file
instead of a startup error.
Catch std::exception around makeApplication() in run(), report the
reason on stderr and return -1, so the failure is a clean non-zero exit
with a message an operator can act on. Only the construction is
wrapped. setup() starts subsystems whose shutdown order is delicate and
is left outside deliberately, because unwinding a half-started
Application would skip the normal stop sequence.
Gate both validation guards on enabled. A node with telemetry switched
off previously refused to start over certificate paths that nothing
would read.
Document both throws on makeTelemetrySetup(), state in
cfg/xrpld-example.cfg and the configuration reference that a partial
mutual-TLS setup is fatal and that the checks apply only when
enabled=1, and add a runbook troubleshooting entry keyed on the two
error messages.
Tests cover both guards with the message asserted so the two are told
apart, both enabled=0 paths, and the default plaintext configuration.
These comments pointed at a planning folder and at its rollout phase
numbering, neither of which is part of the shipped tree, so the
references would dangle for any reader of the repository. Each comment
now states the fact it was pointing at.
The trace_state comment pointed at a planning document that is not part
of the shipped tree, so the reference would dangle for any reader of the
repository. State the reserved-and-inert fact on its own.
* upstream/release/3.3.x: (41 commits)
chore: Bump version to 3.3.0
chore: Bump version to 3.3.0-rc7
fix: Increase manifest protocol message size cap and fix manifests relay
fix: Cap untrusted manifests per message and drop oversized ones
chore: Bump version to 3.2.1
chore: Bump version to 3.2.1-rc1
fix: Cap untrusted manifests per message and drop oversized ones
fix: Reject oversized validator manifest before decoding
fix: Reduce untrusted manifest cache cap to 100
fix: Bound untrusted manifest cache
chore: Bump version to 3.3.0-rc6
feat: Package validator-keys inside rippled
chore: Bump version to 3.3.0-rc5
fix: Switch SponsorshipSet to use a delta for sfFeeAmount
fix: Re-revert "fix: Set request size limits and differential pricing for get-object-by-hash calls"
chore: Bump version to 3.3.0-rc4
fix: Revert "fix: Set request size limits and differential pricing for get-object-by-hash calls"
chore: Bump version to 3.3.0-rc3
fix: Reduce untrusted manifest cache cap to 100
fix: Revert "fix: Reject oversized SHAMap nodes in gotStaleData and fetch-pack path"
...
Bumps the three images this branch pins:
otel-collector-contrib 0.121.0 -> 0.158.0
tempo 2.7.2 -> 2.9.4
grafana 11.5.2 -> 13.1.2
Verified locally against the new images: the collector config passes
`validate` under 0.158.0 and tempo.yaml passes `-config.verify=true` under
2.9.4, both unchanged. The collector's log path uses the generic otlphttp
exporter (otlphttp/loki), not the dedicated loki exporter removed upstream in
v0.147.0, so the pipeline is unaffected by that removal.
Grafana crosses two majors. Operator-visible consequences are handled on the
branches that own the affected files: Grafana 13 enables the renderAuthJWT
feature toggle by default, so the image renderer now requires a matching
renderer_token on both the server and the renderer container.
* release/3.2.x:
chore: Bump version to 3.2.1
chore: Bump version to 3.2.1-rc1
fix: Cap untrusted manifests per message and drop oversized ones
fix: Reject oversized validator manifest before decoding
fix: Reduce untrusted manifest cache cap to 100
fix: Bound untrusted manifest cache
Bound the number of manifests carried in a single TMManifests message
(kMaxManifestsPerMessage). Trusted manifests are always included and
processed; untrusted gossip is capped per message on both send and
receive, and the sender is charged only when untrusted entries are
actually skipped. Oversized TMManifests messages are dropped without
penalty at the protocol layer so an unpatched peer is not disconnected.
Complements the cache bound from #276/#323.
telemetry_->start() ran at the end of ApplicationImp::start(), after
overlay_->start(). Spans are emitted well before that, during setup():
beginConsensus() runs the first consensus round there. SpanGuard drops a
span whenever the global Telemetry instance is not yet live, so that
round's spans were never recorded.
Move the start into setup(), behind a new startTelemetry() seam, right
after the node identity is known. getNodeIdentity() needs only the
cmdline, the config, or the wallet DB, and initRelationalDatabase()
has already created the latter -- the adjacent peerReservations_ load
proves it is usable -- so the identity block moves up with it.
The new position is bounded on both sides: after initRelationalDatabase()
because the identity needs the wallet DB and a DB failure aborts setup(),
and before beginConsensus() because that emits the first spans.
Telemetry must read state, never change it. Two defects here did change it,
plus three smaller correctness and privacy fixes.
doPathFind and doRipplePathFind read source_account / destination_account off
context.params to hash them into span attributes. context.params is non-const,
so those reads selected json::Value's non-const operator[], which inserts a
null for a missing key. The same object is later validated by
PathRequest::parseJson, whose first checks are isMember(source_account) and
isMember(destination_account) — so a request that omitted either field looked
present and the client received Malformed instead of Missing. Reads now go
through std::as_const, whose overload returns kNull without inserting.
PathRequest::doUpdate emitted pathfind_dest_currency as
to_string(saDstAmount_.asset()). For a non-XRP asset that renders as
"<issuer>/<currency>" with the issuer as a plaintext Base58 address, so a
plain account address reached the span pipeline even though every other
account here is hashed first. The issuer is now redacted and the currency
kept; an MPT asset renders as its issuance ID and carries no address.
PathRequestManager::updateAll created pathfind.update_all with an unscoped
SpanGuard. An unscoped guard takes the ambient span as its own parent but does
not itself become the ambient parent, so the pathfind.compute spans that
doUpdate creates never nested under it, contradicting the documented hierarchy.
It is now a scoped guard, held in std::optional because ScopedSpanGuard is
deliberately non-movable and so cannot be produced by a ternary. The skip when
there are no active subscriptions is preserved. updateAll is dispatched via
addJob and doUpdate runs synchronously, so the guard is constructed and
destroyed under the same context store, as ScopedSpanGuard requires.
The WebSocket entry point emitted the client-supplied command string directly.
That value becomes a Prometheus label, so arbitrary request input could drive
unbounded label cardinality. It is now resolved against the handler registry,
collapsing anything unrecognized to "unknown", matching what the HTTP path
already does.
Also: the pathfind.discover comment claimed future child spans could be
parented off it, which its unscoped guard cannot do — corrected to say what
would be required instead. Config-reference and task-list docs named the
parser setupTelemetry(); the API is makeTelemetrySetup().
Conflict in processSession(): this branch added the request-payload-size
attribute on the same lines where the incoming change reverted
processRequest() from bool back to void. Kept both — the payload-size
attribute stays, and the call no longer captures a return value. The
trailing status block takes the incoming side, which leaves the
rpc.http_request span's status unset.
Six related defects in the RPC/gRPC span surface, all cases where a failure
was recorded as success or an attribute was missing on an error path.
GRPCServer: the non-exception branch set the span Ok unconditionally, then
sent a possibly-failed grpc::Status. The handler can return a non-OK status
without throwing, so every failed call traced as successful. Status now
follows result.second, with the error message as the span description.
ServerHandler: eight per-item error branches appended an error reply without
recording that the request failed. Batch responses and ripplerpc < 3.0 always
carry HTTP 200, so those failures were invisible and an entirely failed batch
ended its span as successful. Added an appendItemError() helper next to the
existing httpReplyError() lambda and routed all eight sites through it, so the
flag cannot be forgotten at a new call site.
ServerHandler: the early-return validation paths set the span error but not the
rpc_status attribute. Added it to httpReplyError() so every such path gets it.
RPCHandler: the fillHandler error path set only command and rpc_status, while
callMethod sets command, version and rpc_role. Error spans were therefore not
filterable by API version or role. The error path now mirrors that set.
RPCHandler: resolveCommandSpanName() checked only that command/method were
present, not that they agreed, while fillHandler rejects a mismatch as
rpcUNKNOWN_COMMAND. A request supplying both with different values was labelled
with one of the two names, misattributing the error to a command that never
dispatched. It now mirrors fillHandler's rule and collapses to "unknown".
ServerHandler: processRequest returned bool solely so the caller could set its
span status. Telemetry should read state, not shape the signature of the code it
observes, so the signature returns to void and rpc.process sets its own status
from spanHadError. The enclosing rpc.http_request span now leaves status unset:
the OTel spec has instrumentation leave status unset unless the operation itself
errored, and reserves Ok for an operator asserting verified success.
Conflict in src/libxrpl/telemetry/SpanGuard.cpp: both sides added a different
include in the same alphabetical slot — phase-4 added <initializer_list>, the
incoming std::format change added <format>. Both are required, so both are
kept in sorted order.
SpanGuard::span() and freshRoot() joined the prefix and suffix by hand with
reserve() + three append() calls. std::format expresses the same join in one
line; libstdc++ has shipped <format> since GCC 13 and the project floor is
GCC 15.2 (BUILD.md), so it is available on every supported compiler.
Both callers are noexcept and std::format can throw (std::bad_alloc, or
std::format_error on a malformed spec), so an escaping exception would
terminate the process. The call is wrapped in a joinSpanName() helper that
catches and returns std::nullopt; the caller then returns a null guard, which
is the same degrade-to-no-op path already taken when telemetry is disabled.
Telemetry must never bring the node down.
One helper rather than a try/catch at each site keeps both call sites a single
line and puts the exception-safety requirement in one documented place.
Addresses a review comment on PR #6437.
Conflict in OpenTelemetryPlan/03-implementation-strategy.md §3.9: both
branches independently fixed the same defect (stale hand-maintained line
counts) in different ways.
- 1b (0ff947454c) stripped the Lines Added / Lines Changed columns but kept
the §3.9.1 and §3.9.2 tables.
- 1a (3ad525a48a) removed both sections outright.
Resolved in favour of 1a's deletion. The tables were pre-implementation
estimates with no source of truth, so trimming the columns only defers the
next drift; the file/component lists duplicated §3.1's directory tree, which
1b already keeps current (DiscardFlag.h, FilteringSpanProcessor). Keeping the
upstream resolution also stops the same conflict recurring on 1c..10.
1b-specific content verified intact after the merge: §3.1 retains the
DiscardFlag.h entry, the FilteringSpanProcessor and discard() annotations,
and the TracingInstrumentation removal. No references to the deleted
sections remain in any plan doc.
The §3.9.1 "Files Modified Summary" and §3.9.2 "Detailed File Impact"
tables carried hand-maintained per-component line counts that had drifted
from the plan: the Lines Added column summed to 1,565 while the Total row
claimed ~1,670. Files (34) and Lines Changed (120) reconciled, so only the
Added total was stale — residue from expanding Core Telemetry 5 -> 11 files.
Rather than patch one cell, remove both tables. They were pre-implementation
estimates with no source of truth, so any figure in them drifts again on the
next phase. §3.1's directory tree remains the canonical list of the 11 new
telemetry files, and §3.9.3-3.9.7 (risk, architectural impact, backward
compatibility, rollback) carry the assessment without inventing line counts.
No references to the removed sections exist elsewhere in the plan docs.
Both files use telemetry::consensus::span::* but reached the declarations
only transitively, which misc-include-cleaner rejects. Add the direct
include now that the header lives under xrpl/consensus/.
clang-tidy had been skipped on this branch while CMake configure was
failing, so these findings in makeStageSpan surfaced only now:
- brace the three single-statement if bodies (readability-braces-around-statements)
- compare the pointer parameter explicitly against nullptr
(readability-implicit-bool-conversion)
- include xrpl/protocol/Protocol.h for LedgerIndex (misc-include-cleaner)
The consensus headers moved into the isolated xrpl/consensus module and
took a dependency on xrpl/telemetry for the tracing span constants, but
two things were left behind:
- Four includes still pointed at the old src/xrpld/consensus/ location,
which no longer exists, so the build failed with
"fatal error: 'xrpld/consensus/ConsensusParms.h' file not found".
- xrpl.libxrpl.consensus never linked xrpl.libxrpl.telemetry. add_module
isolates each module's headers, so xrpl/telemetry/SpanNames.h was not
on the include path even once the include was repointed.
Repoint the stale includes at xrpl/consensus/, and declare the telemetry
module before consensus so consensus can link it. Regenerate ordering.txt
for the resulting edge.
The phase-2 merge re-added the pre-telemetry `add_module(xrpl tx)` block
without removing it, leaving the module declared twice. CMake's
add_library rejects a repeated target name, so configure failed before
any compilation:
add_library cannot create target "xrpl.libxrpl.tx" because another
target with the same name already exists.
Drop the stale pre-telemetry block and keep the one that follows
add_module(xrpl telemetry), which links both ledger and telemetry.
libxrpl/tx needs the telemetry link for the tx.transactor span, and
levelization already records `libxrpl.tx > xrpl.telemetry`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add current_ledger_seq / current_ledger_hash to the tx.process, tx.receive,
and txq.enqueue span-reference rows, correct the txq.enqueue parent note
(parents to tx.process on the submission path via explicit context; a root on
the open-ledger rebuild path), and add a "Correlating a transaction to the
ledger it was worked on" recipe joining the txID-keyed tx/txq spans to the
ledger trace via current_ledger_seq.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add shared current_ledger_seq / current_ledger_hash span attributes so a
transaction's work can be joined to the ledger trace that produced it, and
fix discrepancy D1 (txq.enqueue was a detached trace root).
- Define current_ledger_seq / current_ledger_hash once in SpanNames.h and
re-export via `using` from TxQ/TxApply/Tx span-name headers. These name the
ledger being worked on (open/tentative apply or in-flight consensus build),
distinct from ledger_seq (the built/validated ledger on ledger.build /
consensus.round). Named after the RPC field ledger_current_index.
- txq.enqueue: set current_ledger_seq/hash from the view, and parent the span
to the caller's tx.process span via an explicit captured SpanContext (new
trailing TxQ::apply param) instead of a detached root. The parent is
explicit, not ambient-inherited, and the ScopedSpanGuard scope is RAII-bound
to the synchronous apply, so it cannot leak onto a reused worker (D1 fix).
On the open-ledger rebuild path no tx.process context exists, so it stays a
root and the attribute provides the correlation.
- tx.preclaim / tx.transactor: set both attributes from their ledger view.
tx.preflight is stateless (no view) and is the documented exception.
- tx.process / tx.receive: set current_ledger_seq from the current open ledger
index at submit/receive time (no hash: not yet applied to a ledger).
- Contract test pins the two new attribute key strings.
Neither key is a spanmetrics dimension, so there is no metric-cardinality
impact. Dashboards/collector/docs land on the later phases per the chain split.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
M1: ~PendingTraceId now increments a process-wide atomic counter when a
pinned deterministic trace_id is destroyed unconsumed, so the silent drop
stays observable in release builds where the existing XRPL_ASSERT is a
no-op. Exposed via unconsumedDeterministicIdDrops() for a future metric.
The deterministic trace_id bytes, the GenerateTraceId() consume logic, and
the unconditional reset() are all unchanged.
M2: tighten the Doxygen on SpanGuard::childSpan(std::string_view) to state
it parents to the current ambient context of this store (meaningful only
when a scope/ScopedSpanGuard/ScopedActivation is active) and to point
callers at childSpan(name, ctx) for explicit cross-store parenting.
Doc-only; no behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Activate the accept span as ambient context at the top of doAccept using the
non-owning ScopedActivation, so doAccept's log lines (and any spans created in
the body) carry the accept span's trace_id. doAccept runs to completion on the
JtAccept worker with no coroutine yield, so the activation is thread-local and
safe; the acceptSpan still owns and ends the span. The accept.apply child span
keeps its explicit childSpan(acceptApply, acceptSpanContext_) parentage --
activation only adds log correlation and does not alter parent linkage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Brings coroutine-aware context storage, scoped rpc.command, coro-store-swap
tests, and the scoped pathfind.request forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add two GTests on the SpanGuardScopeTest fixture and install a
CoroAwareContextStorage so the tests exercise coro-aware ambient context:
- scopedGuard_survives_localvalue_store_swap: a ScopedSpanGuard's scope
is hidden when its LocalValue store is swapped out and visible again
when swapped back in, and pops cleanly under the owning store.
- activate_sets_ambient_without_owning: SpanGuard::activate() makes the
span ambient for the activation's lifetime without ending it; the
owning guard ends it exactly once.
Fix the cross-store death test to match the store-identity assertion
message ("constructing context store", not "constructing thread") after
the A3 refactor; the fixture's storage install is what lets the assert
fire at all.
Revert RipplePathFind's pathfind.request from freshRoot SpanGuard back to
ScopedSpanGuard: the coro-aware storage moves the scope with the
coroutine across yield, so it nests under rpc.command and stays
trace-correlated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
rpc.command.* becomes a scoped child so it nests under rpc.process, correlates
its log lines, and parents pathfind.request. No RPC::Context plumbing needed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Brings coroutine-aware OTel context storage forward: CoroAwareContextStorage,
its install in Telemetry, ScopedSpanGuard same-store assertion, and the
non-owning ScopedActivation helper.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add SpanGuard::activate() returning a ScopedActivation RAII helper that
activates an already-owned span (from a thread-free SpanGuard) as the
current context WITHOUT taking ownership. The activation pushes the span
onto the current LocalValue context store on construction and pops it on
destruction; it never ends the span (its owning SpanGuard does).
This lets a job-handoff span be made ambient for the duration of a
synchronous, non-yielding worker body so log lines there carry the
span's trace_id. Non-copyable and non-movable, mirroring ScopedSpanGuard.
owner is captured after the Scope push via declaration-order member
initialization (scope declared before owner), matching the A3
capture-after-materialization invariant. A #else no-op stub keeps the
API zero-overhead when telemetry is compiled out.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Coroutine-aware storage lets a scope resume on another worker within the same
coroutine store; store-identity is the correct pop-safety invariant. Same-store
equals same-thread for synchronous code, so no safety is lost.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Install CoroAwareContextStorage as the process-wide OTel runtime context
storage in TelemetryImpl::start(), before SetTracerProvider and before any
span is created, so the ambient context follows JobQueue coroutines across
yield/resume. Hold it in a member for the process lifetime; not reset in
stop() because resetting while spans may exist is SDK undefined behaviour.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backs the OTel active-context stack with xrpl::LocalValue so the ambient
context follows a JobQueue::Coro across yield/resume. Not yet installed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Consensus.h includes ConsensusSpanNames.h, which already includes
SpanNames.h (it is built on StaticStr/join()). Consensus.h uses no base
SpanNames.h symbols directly, so misc-include-cleaner flags the direct
include as unused. Remove it; SpanNames.h stays reachable transitively.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Rename file-local thread_local globals to the .clang-tidy convention
(GlobalVariablePrefix "g" + CamelCase), keeping the Tls marker:
tlsPendingTraceId -> gTlsPendingTraceId, tlsPendingConsumed -> gTlsPendingConsumed.
- Add direct includes for opentelemetry trace_id.h / span_id.h (header uses
TraceId/SpanId in signatures) and sdk/trace/id_generator.h (.cpp references
IdGenerator directly) to satisfy misc-include-cleaner.
Both verified clean with clang-tidy against a telemetry-enabled compile DB.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The OTel naming check (Rule F) scans @code doc-comment examples and fails
on string-literal span names; Rule H warns on undefined SpanNames
constants. Replace the literal "subtask" and the undefined
rpc_span::op::dispatch with the defined rpc_span::op::process constant so
the examples model correct API usage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The roundSpan_ / roundSpanContext_ comments still described the old
'detached, Scope stripped' model. roundSpan_ is now a thread-free SpanGuard
(no Scope); children link via the captured roundSpanContext_.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pathfind.request (PathFind.cpp) and pathfind.compute (PathRequest.cpp doUpdate)
were left as unscoped SpanGuard after the type split, so pathfind.compute and
pathfind.discover no longer nested under them. Both handlers run synchronously
with no coroutine yield, so ScopedSpanGuard is safe and restores the
request -> compute -> discover sub-tree. rpc.command.* stays unscoped (callMethod
wraps doRipplePathFind which holds across a yield), so pathfind.request parents
to rpc.process.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With unscoped SpanGuard + spanContext() (own-span capture), the consensus
round/establish/accept sites just capture and drop the detach dance;
openSpan + peer proposal/validation receive are thread-free handoffs (no
detach); rootSpan->freshRoot.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
OTel distinguishes a span's own context from the thread's
current/ambient context; SpanGuard is unscoped so spanContext()
(own span, default) is what cross-thread childSpan parenting needs,
and threadLocalContext() (static) snapshots RuntimeContext::GetCurrent()
for propagation. Renames captureContext.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SpanGuard is unscoped (never pushed onto the thread-local context stack), so
RuntimeContext::GetCurrent() returned an unrelated ambient span. Build the
captured context from impl_->span so captureContext() is correct for every
caller regardless of scoping — childSpan(name, ctx) then parents explicitly to
this span across threads. Matches getTraceBytes()'s own-context behavior.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The generator's doc said it was 'dormant, caller lands on a later branch'.
Reworded to branch-agnostic language: hashSpan() is the primary caller that
forces the root branch to mint deterministic trace roots.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SpanGuard is now thread-free (holds no Scope), so the tx.receive (PeerImp)
and tx.process (NetworkOPs) handoff sites no longer need .detached() before
being stored and ended on a worker thread — just construct the guard. The
stale "Scope leak" comments are replaced accordingly.
Make the six txq.* spans ScopedSpanGuard so their sub-spans nest via the
ambient context: txq.apply_direct/batch_clear under txq.enqueue, and
txq.accept_tx under txq.accept. All six are verified synchronous, ended at
scope, and never moved/handed off, so scoping is safe. applyDirect's span
is pure RAII (no method calls), so it is declared const.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The single-arg hashSpan fabricated a synthetic DefaultSpan parent (random
span_id) to carry the deterministic trace_id. That left every standalone
hash span with a phantom parent_span_id pointing at a span that is never
exported, so Tempo reported "root span not yet received".
Inject the deterministic trace_id through the DeterministicIdGenerator
instead: start the span on the SDK root branch (Context{kIsRootSpanKey,true})
with a PendingTraceId pinning hashData[0:16]. The SDK's no-valid-parent
branch calls GenerateTraceId(), which returns the pinned id, yielding a
TRUE root (empty parent_span_id) whose trace_id == hashData[0:16].
The deterministic bytes are unchanged (still raw hashData[0:16]). The
cross-node overload (real remote parent, remote=true) is untouched and must
keep producing a child of the sender's span. Drop the now-unused
<xrpl/basics/random.h> include; add <array> and DeterministicIdGenerator.h.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrite SpanGuardScope.cpp for the unscoped SpanGuard / scoped
ScopedSpanGuard split and the DeterministicIdGenerator: install the
generator in the TestTelemetry mock (4-arg TracerProviderFactory), drop the
obsolete detached()/detachInPlace() tests, and add freshRoot, scoped-ambient,
scope-handoff, ends-once, deterministic-root, and cross-thread death tests
with exact assertions.
Fix the last detached() caller: RipplePathFind now uses SpanGuard::freshRoot
(SpanGuard is thread-free, so it is held across the coroutine yield and ended
on resume with no scope to strip).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Entry points (http_request/ws_message/ws_upgrade) were inheriting a leaked
ambient span on reused coro workers; now scoped fresh roots via
ScopedSpanGuard::freshRoot. rpc.process stays a scoped child.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a custom OTel IdGenerator that returns a thread-local pending trace_id on
the SDK no-parent (root) branch and a random one otherwise, plus a PendingTraceId
RAII guard that pins that id for the next forced-root span and asserts on
destruction that it was consumed. Wire the generator into
TracerProviderFactory::Create via its 4-arg overload.
This lets hash-derived spans become true trace roots so they line up into one
trace across nodes. It is installed but dormant on this branch: the caller
(hashSpan) arrives on a later branch (phase-3). GenerateSpanId is always random
and is_random_ is false so the W3C random-trace-id flag is not set on
deterministic ids.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Separate the two responsibilities the old SpanGuard fused: span ownership
(thread-free) and scope activation (thread-bound TLS push).
- SpanGuard now owns ONLY the span. Its Impl drops the optional<Scope>,
the owner thread-id, the Detached tag and the scope-less ctor; ~Impl is
just `if (span) span->End()`. The guard never binds a thread-local
context stack, so it may be moved to and destroyed on any thread.
- ScopedSpanGuard is a new pimpl type that wraps a SpanGuard plus an
optional<Scope>. Member order (guard first, scope second) pops the
scope before the span ends. It is non-copyable and non-movable;
factories return unnamed temporaries so guaranteed copy elision covers
`auto s = ScopedSpanGuard::freshRoot(...)`.
- `operator SpanGuard() &&` replaces detached()/detachInPlace(): it pops
the scope eagerly on the origin thread and yields a thread-free
SpanGuard for handoff to a job or another thread. detached(),
detachInPlace() (both overloads) and the Detached apparatus are deleted.
- Move-assignment is re-enabled on SpanGuard (no Scope to re-bind), in
both the real class and the no-op stub.
- rootSpan() renamed to freshRoot() in both classes (behavior unchanged:
still forces kIsRootSpanKey).
- hashSpan behavior is intentionally unchanged (only builds the now
unscoped Impl); the true-root IdGenerator fix is a separate later task.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add two free functions in xrpl::telemetry that wrap the required
"detach a live SpanGuard held in a container" idiom so call sites write
one line instead of the make_shared/emplace rebuild by hand:
void detachInPlace(std::optional<SpanGuard>&);
std::shared_ptr<SpanGuard> detachInPlace(std::shared_ptr<SpanGuard>);
Both are no-ops on an empty/null/inactive guard. The #else branch adds
matching inline no-op stubs so callers compile with telemetry disabled.
Pure API addition, no behavior change; consumer call sites are rewritten
on phase4 in a follow-up.
The unit tests for these helpers land on phase2, where the telemetry
test module (in-memory-exporter harness + SpanGuardScope.cpp) exists —
mirroring how the detached()/rootSpan() tests were placed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
xrpl_branch and xrpl_node_role are stamped by the perf-iac harness --
infrastructure outside this repo's OTel code -- so they have no L1
(*SpanNames.h), L2 (collector config), or L6 (MetricsRegistry.cpp) source
Rule D can derive them from. Unlike the generic builtins set (Prometheus/
Grafana mechanics every OTel setup has), these are repo-specific and
narrow, so they get their own EXTERNAL_INFRA_LABELS constant: a visible,
documented, deliberately narrow exception to the 'no hardcoded allowlist'
design principle, not a silent workaround. Add test coverage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
xrpl_branch and xrpl_node_role are collector/infra-injected resource
labels used by perf-iac dashboards to identify the build under test and
its role in the perf cluster. They have no L1 span-attribute source or
L6 native-metric-label source (like the existing job/instance/job_type
builtins), so Rule D flagged them across every perf-iac dashboard.
Register them in the builtins set, matching the existing pattern for
infra-level dashboard labels. Add test coverage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert leading /// blocks to house-style /** */ across the telemetry
files carried on this branch (using the updated fix_doxy.py). Also
regenerate levelization results. Comment-only: code is byte-identical.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert leading /// blocks in these 9 phase3-originating files to house
style /** */ (using the updated fix_doxy.py that now handles /// ->
block conversion). Comment-only: code is byte-identical.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bring the 5 telemetry files introduced on this branch to the enforced
house Doxygen style (/** alone, ' * ' continuation prefix, no single-line
blocks) so the check-doxygen-style hook passes under CI's --all-files run.
Comment-only: code is byte-identical after comment stripping (verified).
Also regenerate levelization results: the committed ordering.txt carried
stale 'xrpl.telemetry > xrpld.consensus/rpc' edges that the current include
graph no longer produces.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The opentelemetry-cpp::exporter_in_memory component declares no libs (the
Conan recipe ships libopentelemetry_exporter_in_memory.a but does not
associate it with the component), so linking that target put nothing on
the link line and xrpl_tests still failed with an undefined reference to
InMemorySpanExporterFactory::Create. Locate the archive with find_library
(HINTS the OTel package lib dir) and link it by path, ahead of the umbrella
so its undefined SDK references resolve. Verified: the archive now appears
on the generated link line and xrpl_tests links + the SpanGuardScope tests
pass. Supersedes the no-op component-target link in the prior commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The umbrella opentelemetry-cpp target's link set omits the in-memory
exporter library, so xrpl_tests failed to link with an undefined
reference to InMemorySpanExporterFactory::Create. The SpanGuardScope
tests use an InMemorySpanExporter to read exported spans. Link the
opentelemetry-cpp::exporter_in_memory component explicitly. Test-only:
production code never uses this exporter.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Post-review fixes for the SpanGuard scope-leak change:
- SpanGuard.h: restore the injectCurrentContextToProtobuf real-class
declaration, its #else no-op stub, and the protocol::TraceContext
forward-declaration dropped during the 3->4 union merge (compile break).
- Re-parent consensus.phase.open and consensus.establish under the round
span via a new RCLConsensus::Adaptor::roundSpanContext() accessor: the
round span is now detached, so ambient parenting no longer works; the
phase spans link explicitly to the captured round context. csf::Peer
gains a matching no-op accessor so the generic engine still compiles.
- Switch five childSpan(op::X, ctx) sites to the full consensus::span::X
constants: childSpan(name, ctx) takes the name verbatim, so the suffix-
only op:: constants emitted short, non-dotted span names.
- TestTelemetry mock: add getConsensusTraceStrategy() override (base pure
virtual introduced on this branch) so the mock is not abstract.
- validate(): rewrite the trace_context comment to drop attack-surface
framing and the PR-discussion reference.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
roundSpan_, establishSpan_, openSpan_ and the accept span are emplaced on
one job worker and reset()/moved to another, leaking their thread-local OTel
Scope. Each is now detached AFTER its context is captured. Spans that
previously nested under them via the ambient thread context are re-pointed to
the captured SpanContext explicitly:
- round children (proposal.send, ledger_close, mode_change) -> roundSpanContext_
- establish children (update_positions, check) -> establishSpanContext_
- accept.apply -> acceptSpanContext_
onForceAccept (synchronous) keeps the accept span's behaviour; only onAccept
(moved into the JtAccept job) needs the detach. Member move-assignment is
deleted, so re-assignment uses emplace().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The consensus.proposal.receive / consensus.validation.receive guards are
moved into checkPropose/checkValidation jobs, so detach their Scope on the
peer thread before the job hand-off. The standalone (no-propagated-context)
fallbacks in ConsensusReceiveTracing now use rootSpan() so they start a
fresh trace root instead of inheriting an ambient span; the hashSpan
propagated-context branches are unchanged (cross-node linking preserved).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
doRipplePathFind holds its pathfind.request span across context.coro->yield();
the coroutine resumes on a different JobQueue worker. rootSpan() gives it a
fresh trace root (no stale ambient parent), and detached() strips the
thread-local Scope so the guard neither lingers on the worker's context stack
across the yield nor pops the wrong stack on resume.
Adds SpanGuardScope.cpp: an in-memory-exporter test for rootSpan()/detached()
including the rootSpan().detached() combination used here.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both spans are moved into job-queue lambdas and destroyed on a worker
thread. Detaching on the origin thread pops the thread-local OTel Scope
there, so later spans on the peer/RPC thread no longer inherit these as a
leaked ambient parent. Trace_id/parent are unchanged (both are hashSpan).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
rootSpan has the same (cat, prefix, name) signature as span(), so its
prefix/name arguments must be *SpanNames.h constants under Rule F/H. Add
rootSpan to the CALLSITE regex and CONSTANT_ARG_POSITIONS ({1,2}), update
the enumerating docstrings/README, and add two regression tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The otel-naming Rule F check scans @code doc examples as call sites; raw
string literals there trip the rule. Reference *SpanNames.h constants /
placeholder identifiers in the examples so comments follow the naming rules.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
detached() strips the thread-local OTel Scope so a guard can be safely
moved to and destroyed on another thread; it pops the Scope on the origin
thread and moves the span into a scope-less guard. rootSpan() starts a span
as a fresh trace root (kIsRootSpanKey) so inbound entry points never inherit
an ambient span left active on the thread.
Impl now holds an optional<Scope> (nullopt for detached guards). Updated the
SpanGuard class docs and docs/build/telemetry.md with the cross-thread rules.
The unit test lands on phase2 where the telemetry test module exists.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
libstdc++ ships <format> starting with gcc-13, so std::format in
SpanGuard::span() fails to compile on gcc-12. Build the "<prefix>.<name>"
span name via std::string::append instead and drop the <format> include.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: xrplf-ai-reviewer[bot] <266832837+xrplf-ai-reviewer[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Add missing <string_view> include in Consensus.h and RCLConsensus.h.
Remove unused SHAMap.h include from RCLConsensus.h.
Add braces around single-line if/else bodies in RCLConsensus.cpp.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove unused SpanNames.h from Transactor.cpp (transitively included
via TxApplySpanNames.h). Remove unused <ranges> from
TraceContextValidation.h (std::ranges::any_of lives in <algorithm>).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace std::transform with std::ranges::transform in Redaction.cpp.
Replace std::all_of with std::ranges::all_of in Redaction test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add missing <cstdint> include for std::uint32_t in Telemetry.h.
Add braces around single-line if bodies in Telemetry.cpp.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the "Phase 4" tokens, keep descriptions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the "Phase 3" tokens, keep descriptions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the "Phase 2" tokens, keep descriptions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the "Phase 1b" token, keep the filter
description.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add PathFinding Spans section to the runbook Span Reference and
remove the incorrect "not emitted / later-phase" annotations:
pathfind.request/compute/discover/update_all are emitted since
Phase 2 (upstream of Phase 5).
- Runbook label names use the bare spanmetrics dimensions
(command, rpc_status, local, consensus_mode); fix stale docs.
- 05-configuration-reference: note tls_client_cert/key require use_tls=1.
- TelemetryConfig: reject mTLS cert paths when use_tls=0 so telemetry
is not silently sent in plaintext.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Soften the "non-reversible" wording in Redaction.h. XRP account
addresses are a public, enumerable set, so an unsalted hash is
obfuscation (keeps plaintext out of traces/dashboards), not a
secrecy guarantee. Document why a salt is intentionally omitted:
it would break cross-node/restart correlation, the reason for
hashing rather than dropping the attribute.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve the doCommand error-path span suffix and command attribute against
the handler registry instead of using raw request input. A recognized command
yields its canonical handler name; anything else (unregistered command, or
neither command/method present) collapses to the literal "unknown". This bounds
telemetry cardinality to the finite set of handler names plus "unknown" while
preserving per-command error attribution for known-but-rejected commands
(e.g. a submit that hit rpcTOO_BUSY stays rpc.command.submit). The command
attribute is promoted to a Prometheus label by the spanmetrics connector, so
attacker-controlled input previously enabled unbounded label cardinality.
Also add the missing LCOV_EXCL_STOP in the first SpanGuard::linkedSpan overload
so its coverage exclusion no longer bleeds into the second overload's guards.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Change the six remaining `type: static` search filters in tempo.yaml to
`type: dynamic` so Grafana queries Tempo for available tag values instead
of requiring manual free-text entry. Aligns service-name, node-id,
network-type, span-name, span-status, and span-duration with the existing
node-version and network-id filters.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The operator runbook is for people running OpenTelemetry against a node, not
a record of the internal development phases. Strip the "(Phase N)" suffixes
from the span-reference headings and replace the personal-name example
service.name value with a generic one.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a resource/stripsdk processor that deletes telemetry.sdk.language,
telemetry.sdk.name, and telemetry.sdk.version (auto-added by the OTel SDK
to every Resource). Wire it into the traces pipeline; the metrics and
logs pipelines pick it up on the branches that introduce them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a Deployment Tiers section covering the four filtering dimensions
(node, service, network, environment), who owns each attribute, the
collector's upsert-vs-insert rule, per-tier collector values, and how the
tier labels reach metrics. This is the operator-facing home for the
deployment-tier feature.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- 05-configuration-reference: service_name now applies to metrics as well
as traces (service.name resource attribute).
- 07-observability-backends: document that collector enrichment includes
deployment-tier tagging (deployment.environment + fallback
xrpl.network.type) for filtering one Grafana stack by tier.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a resource/tier processor so each collector stamps the deployment
tier onto the signals it forwards, enabling a single Grafana stack to
hold data from many collectors and filter by tier:
- deployment.environment (upsert): the collector is authoritative for
its environment (local/ci/test/prod).
- xrpl.network.type (insert): the xrpld node already stamps its own
chain, so the collector only fills this when absent, never
overwriting a node's real network.
Wired into the traces pipeline (the only pipeline on this branch);
later phases extend it to metrics and logs as those pipelines appear.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CI enabled more clang-tidy checks via the upstream merge:
- DiscardScope discardScope -> const (misc-const-correctness)
- drop the (::max)() Windows macro-guard parens; NOMINMAX is defined
project-wide so the bare form is correct (readability-redundant-parentheses)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 5 added 4 more unit-test cases calling the old setupTelemetry name.
Align them with the renamed makeTelemetrySetup free function so xrpl_tests
builds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The free function was renamed makeTelemetrySetup but this unit test still
called the old name, breaking the xrpl_tests build on arm64.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The followsFrom linkedSpan overload referenced attr::linkType /
attr_val::followsFrom, which don't exist in SpanNames.h — the other two
link sites already use the local kLinkTypeKey / kLinkTypeFollowsFrom
constants. Unify the third site. Fixes arm64 build (-Werror).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The network-type label is derived from [network_id] in TelemetryConfig.cpp;
unmapped/unset IDs fall through to "unknown", not "standalone". Align the
design-doc resource-attribute table with the code and cfg example.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor: Update RPM version scheme; remove auto-update script; service hardening
- **RPM version scheme**: pre-releases now use `~` in the `Version` field instead of the `0.<release>.<suffix>` `Release`-field hack. Matches Debian's `~` convention, so RPM and DEB version strings are symmetric. Requires rpm ≥ 4.10 (RHEL 9 ships 4.17).
Before/after for a pre-release build:
```
# before
xrpld-3.2.0-0.1.rc3+202606011647.d4cb68d5.el9.x86_64.rpm
# after (symmetric with DEB)
xrpld-3.2.0~rc2+202606010139.7679a310-1.el9.x86_64.rpm
xrpld_3.2.0~rc2+202606010139.7679a310-1_amd64.deb
```
- **Auto-update removed**: `update-xrpld`, `update-xrpld.service`, and `update-xrpld.timer` deleted. The `50-xrpld.preset` `disable` line for the timer is dropped too.
- **Service hardening** (two new `[Service]` directives in `xrpld.service`):
- `CapabilityBoundingSet=CAP_NET_BIND_SERVICE` — drops every Linux capability except `CAP_NET_BIND_SERVICE`, capping the privilege ceiling to least-privilege while still letting operators bind ports <1024 (e.g. WS/HTTPS on 443).
- `SystemCallArchitectures=native` — restricts the service to the native syscall ABI, blocking alternate-ABI (32-bit/x32) syscalls used to evade seccomp filtering.
- [ ] Build RPM from a pre-release version (e.g. `3.2.0-b1`) and confirm `rpm -qi` shows `Version: 3.2.0~b1`, `Release: 1`
- [ ] Confirm `3.2.0~b1` sorts before `3.2.0` via `rpmvercmp`
- [ ] Install package and confirm no `update-xrpld*` units appear in `systemctl list-unit-files`
- [ ] Confirm `systemctl show xrpld` reflects the new `CapabilityBoundingSet` and `SystemCallArchitectures`
* fix: Track tmpfiles-created directories in RPM %files as %ghost
The follows-from link attribute used the dotted xrpl.link.type form, which
is reserved for resource attributes. Span attributes use the <domain>_<field>
underscore form, so the key becomes link_type.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The TxQSpanNames.h span-hierarchy comment and the Phase 3 task-list attribute
table showed txq.accept.tx, but the constant emits txq.accept_tx
(op::acceptTx = "accept_tx"). Correct both.
Make consensus span-attribute keys consistent with the peer spans that record
the same concepts, using ONE shared name per concept (told apart by span name),
not an emitter prefix:
- Add the shared ledger_hash and full_validation keys to the base SpanNames.h
(a ledger-object property and an is-full-validation flag, both shared with the
peer validation spans); consensus re-exports them via `using` instead of
defining local copies.
- trusted (bare) -> proposal_trusted on consensus.proposal.receive and
validation_trusted on consensus.validation.receive — the same message-type
qualification the peer.{proposal,validation}.receive spans use.
Also collapse a same-emitter duplicate: establishCounter_ was recorded under
both establish_counter and establish_count on two consensus spans; both now use
establish_count (the establishCounter span-name constant is removed; the
establishCounter_ member variable is unchanged).
The consensus.proposal.receive and consensus.validation.receive spans record
the trust flag under proposal_trusted / validation_trusted (shared with the
peer spans of the same message type), not a bare `trusted`. Update the span
reference table to match.
State the rules so they stay consistent across code, collector, Tempo,
dashboards, and docs:
- Per-span-unique field -> bare name (the span name carries the domain).
- Same concept on more than one span -> ONE shared key, reused verbatim and
distinguished by span name, never tagged with the emitting workflow
(e.g. ledger_hash, full_validation, proposal_trusted/validation_trusted).
Defined once in the base SpanNames.h and re-exported by each domain header.
- Collision qualifier <domain>_<field> only to separate DIFFERENT concepts that
share a word, or the OTel-reserved status key (rpc_status, consensus_state).
- Dotted xrpl.<...> is reserved for resource attributes (xrpl.network.*).
Updates CONTRIBUTING.md (permanent home) and OpenTelemetryPlan §2.3.3.
Rule A silently missed a dotted span attribute (xrpl.ledger.hash) because of
two interacting bugs:
1. attr_keys_from_header resolved each constant via a flat global symbol table
keyed by bare name, so a later header defining a same-named constant (e.g.
consensus attr::ledgerHash = "ledger_hash") clobbered the base header's
attr::ledgerHash = "xrpl.ledger.hash", erasing the real dotted key from L1.
Now each constant is resolved against its own header (the global table only
seeds seg::/join() cross-file references); using-re-exports still resolve
globally.
2. derive_dotted_resource_keys allowlisted any dotted key declared in the base
SpanNames.h. Now it allowlists only the keys actually passed to
Resource::Create() in Telemetry.cpp (semconv service.* + the attr:: constants
set there, e.g. xrpl.network.*). A dotted key declared in a header but never
set as a resource attr is a Rule-A violation.
Adds 4 regression tests (collision, using-re-export, allowlist scope, brace
matching). No allowlist exception is added — the check now catches the
violation so the offending code can be fixed.
Rule C was reading docker/telemetry/tempo.yaml (the Tempo server config), which
has no filter tags, so it always SKIPped — L3 was silently unenforced. The
trace-search filter tags actually live in the Grafana datasource provisioning
file (docker/telemetry/grafana/provisioning/datasources/tempo.yaml) as
search.filters[].{tag,scope}. Point Rule C there (server file as fallback),
pair each tag with its scope, validate only span-scope tags against L1 (resource/
intrinsic tags like service.*/name/status/duration are exempt), and strip the
TraceQL span. prefix.
On phase-9 this turns "SKIP: C" into "OK: C: 24 tempo span-filter tags all in
L1" — L3 is now genuinely guarded. Adds a RuleCTempo test class (4 cases:
span-tag-not-in-L1 flagged, span-tags-pass, resource/intrinsic ignored, skip
when datasource absent). 83 tests total.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 9 surfaced two Rule D gaps (false positives, not data errors):
- TraceQL `span.<attr>` / `resource.<attr>` references: the bare attribute is
in L1, but the scope-prefixed form was flagged. Now strip the
span./resource./event./link/instrumentation_scope. prefix before the L1
lookup.
- Native OTel metric labels (e.g. `job_type`, `reason`) emitted by
MetricsRegistry are valid dashboard labels but are not span attributes. Add
an L6 source: parse `Add(.., {{"label", ...}})` instrument calls and accept
those label keys alongside L1 and builtins.
Verified against phase-9's real dashboards: 6 prior false positives -> 0.
79 tests (7 new for span-prefix stripping and metric-label extraction).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Forward-merge the phase-2/3 BasicConfig.h include-path fix to keep the chain
consistent. (phases 4+ already had the correct include; no file change here.)
Forward-merge the phase-2/3 BasicConfig.h include-path fix to keep the chain
consistent. (phases 4+ already had the correct include; no file change here.)
Forward-merge the BasicConfig.h include-path fix (basics -> config) so the
phase-3 build-test job compiles. phase-3 had the same broken include as phase-2.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
src/tests/libxrpl/telemetry/TelemetryConfig.cpp included
<xrpl/basics/BasicConfig.h>, but the header lives at
<xrpl/config/BasicConfig.h> (the phase-1b layout). This broke the build-test
job on phase-2 (and phase-3) across all platforms with a "file not found"
fatal error. The fix already exists on phase-4+; backport it to the branch
that introduced the test so each PR builds on its own.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rule D (dashboard PromQL labels must exist in L1) flagged `__name__` once the
phase-7 system-*.json dashboards started using `sum by (le, __name__)`.
`__name__` is the Prometheus reserved label for the metric name itself — a
builtin, not a span attribute. Add it to the builtin allowlist and cover it
with a test. (Earlier dashboards only used `__name__` inside `{__name__=~...}`
matchers, which the label regex did not extract, so this surfaced only now.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring phase-4 forward into phase 5 (docs & deployment). Phase 5 owns the
operator runbook (docs/telemetry-runbook.md).
Conflict resolution:
- 05-configuration-reference.md: took phase-4's code-block-free prose for the
config-parser section; the mTLS options (tls_client_cert/tls_client_key)
remain documented in the §5.1.2 options table.
Phase-5-owned naming fix:
- docs/telemetry-runbook.md: converted 20 dotted xrpl.<domain>. attribute keys
in the Span Reference tables to the underscore convention (tx_hash, peer_id,
ledger_seq, consensus_mode/round/round_id/ledger_id, tx_id). Span NAMES stay
dotted (rule 5). `trusted` left as-is — verified it matches the code constant
ConsensusSpanNames.h (the proposal_trusted/validation_trusted split is a
separate, not-yet-applied code change).
End-to-end reconciliation: every span-attribute token in the runbook (L5) now
matches an L1 *SpanNames.h constant. The xrpl_<domain>_<field> tokens are the
spanmetrics-derived Prometheus labels (documented mapping), not span attrs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three entries in Phase4_taskList.md annotated dotted attribute keys
(`xrpl.consensus.round`, `round_id`, `ledger_id`) as "(kept — rule 5)". That
is wrong: rule 5 keeps SPAN NAMES dotted, not attribute keys — the code emits
the underscore form (consensus_round, consensus_round_id, consensus_ledger_id)
per ConsensusSpanNames.h. Remove the false justification; the dotted form in
this task list remains as the illustrative-readability form documented in the
note at the top of the file.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The two "Spans Produced" tables in 06-implementation-phases.md (added by the
Phase 4 consensus work) listed span ATTRIBUTE keys in the dotted
xrpl.consensus.* / ledger.seq / mode.old form. Convert them to the underscore
convention, matching the authoritative ConsensusSpanNames.h constants
(consensus_round, ledger_seq, consensus_mode, consensus_round_id,
consensus_ledger_id, consensus_result, mode_old/new, agree_count/disagree_count,
etc.). Span NAMES in column 1 stay dotted (rule 5). Every key verified to exist
as a constant in ConsensusSpanNames.h / SpanNames.h.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring phase-3 forward into phase 4 (consensus tracing). Phase 4 introduces
ConsensusSpanNames.h (and already did the trusted→proposal/validation rename
and underscore-attr conversion in ffc197b914).
Conflict resolution (reviewed by a code-review agent before commit):
- 02-design-decisions.md §2.4 Consensus Attributes: merge-both — kept phase-3's
underscore table form AND folded in phase-4's richer "Phase 4a" attribute set
(round_id, ledger_id, trace_strategy, converge/establish/disputes counts,
agree/disagree counts, threshold_percent, consensus_result, mode_old/new),
each mapped to its authoritative ConsensusSpanNames.h constant. Dropped the
planned-but-unimplemented proposers_agreed/proposers_total (no code constant;
agree_count/disagree_count serve that role).
- SpanGuardFactory.cpp: kept phase-4's explanatory comment about why a libxrpl
test uses literal keys, plus the converted command/rpc_status keys.
Naming check passes (100 keys across 8 *SpanNames.h headers, including consensus).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring phase-2 forward into phase 3 (transaction tracing). Phase 3 introduces
TxSpanNames.h, TxQSpanNames.h, and TxApplySpanNames.h.
Conflict resolution:
- TxQ.cpp: kept phase-3's txq_span-based instrumentation (phase-2 had none).
Dropped the orphaned `NumberSO{... fixUniversalNumber}` line — develop's
#5962 (Retire fixUniversalNumber) removed that symbol repo-wide; the
conflict block had carried one stale copy that would not compile.
- 05/08/OpenTelemetryPlan.md: dropped the deleted 04-code-samples / POC_taskList
references (carried from phase-2), kept phase-3's new secure-OTel.md doc rows,
section, and Mermaid node/edge/style. Config code block -> prose; merged the
secure-OTel hardening pointer with the authoritative-config prose.
- Phase3_taskList.md: removed the "dotted keys for readability" note that came
from phase-2 — phase 3 already uses the underscore keys.
Reviewed by code-review agents: telemetry instrumentation intact, naming check
green (47 keys across 7 *SpanNames.h headers), no conflict markers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Phase3/4/5 task lists show attribute keys in the older dotted
`xrpl.<domain>.<field>` form because that mirrors how a fully qualified
attribute reads in a Tempo trace view, which keeps the planning prose readable.
Add a note to each clarifying that the implemented keys follow the underscore
convention in CONTRIBUTING.md and that the *SpanNames.h constants (enforced by
the CI naming check) are the single source of truth — so the dotted form here
is illustrative, not the literal key.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SpanGuardFactory.cpp set dotted "xrpl.rpc.command"/"xrpl.rpc.status" attribute
keys. Test files are exempt from the naming check's Rule F (they pass arbitrary
literals to exercise the API), so this slipped through, but the keys should
still illustrate the underscore convention. Use "command"/"rpc_status".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring the hardened OTel naming check forward from phase-1c: unconditional
Rule F, test-file exemption, and the Rule H in-place-constant warning. The
check passes clean on phase 2 (24 keys across 4 *SpanNames.h headers including
PathFind; the SpanGuardFactory.cpp test is correctly exempt from Rule F).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three robustness fixes to check_otel_naming.py, all on phase-1c where the
script lives:
- Rule F now runs UNCONDITIONALLY. It is a purely syntactic check on the
call-sites and does not need the L1 key set, so code that calls
SpanGuard::span/setAttribute directly without ever defining a *SpanNames.h
is still caught (previously it was silently skipped when no header existed).
- Exempt test files from Rule F (tests pass arbitrary literal keys to exercise
the API). The call-site matcher now requires a SpanGuard/`.`/`->` receiver,
so std::span and bare declarations no longer false-positive.
- Add Rule H (warning, non-fatal): a namespace-qualified constant used at a
telemetry call-site but not defined in any *SpanNames.h is flagged, catching
constants defined in-place instead of in the proper header. Bare locals and
std:: names are not warned to avoid noise.
SpanGuard.h / Telemetry.h @code examples updated to reference constants that
exist on this branch. README documents the new behavior.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The script and its README live on phase-1c (where check_otel_naming.py was
introduced). The test-file Rule-F exemption was mistakenly applied here on
phase-2; revert to phase-1c's version verbatim. The exemption and further
script improvements will land on phase-1c and merge forward, keeping the
script's logic on the branch that owns it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring the naming convention, code-sample cleanup, and CI naming check into
phase 2 (RPC tracing). Phase 2 introduces PathFindSpanNames.h.
Conflict resolution:
- 04-code-samples.md, POC_taskList.md: deletion wins.
- 02-design-decisions.md: took the convention-applied tables, but kept phase-2's
accurate PathFinding summary row (pathfind_fast/search_level/num_paths/...,
matching the implemented PathFindSpanNames.h).
- 05/08: took the code-block-free prose; kept phase-2's Phase2-5_taskList.md
index rows (dropping only the deleted POC row). Fixed stale setup_Telemetry/
make_Telemetry doc references to the code-correct setupTelemetry/makeTelemetry.
- Telemetry.h auto-merged to the constant-based @code examples.
check_otel_naming.py change: exempt test files from Rule F (tests pass
arbitrary literal keys to exercise the API). The check passes clean on the
merged tree (24 keys across 4 *SpanNames.h headers, including PathFind).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add Rule G to check_otel_naming.py: every span-attribute key must be
lower_snake_case (^[a-z][a-z0-9_]*$ per dot-separated segment). This catches
camelCase, UPPERCASE, and spaces in keys, which the structural (dotted) and
source (literal) rules did not. Document it in the script README and
CONTRIBUTING.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add check_otel_naming.py and wire it into on-pr.yml so every PR validates
that span-attribute names stay consistent across the code, collector, Tempo,
dashboards, and docs.
- The valid key set is derived dynamically from the *SpanNames.h constants and
the resource attributes the code registers in Telemetry.cpp — no hardcoded
allowlist to drift.
- Each rule is presence-gated: it runs only when the file it needs is in the
tree, so the check is correct whether telemetry changes land in one PR or
several (the collector/Tempo/dashboard/runbook layers arrive in later phases).
- Rule A flags dotted span-attribute keys; Rule F flags string-literal
attribute keys and span-name arguments (values may be runtime data).
- stdlib-only, mirroring the levelization check (bare `python`, no pip step).
- Telemetry.h / SpanGuard.h @code examples now use *SpanNames.h constants so
the strict literal check passes.
- CONTRIBUTING.md documents the check and how to run it locally.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring the phase-1a/1b naming-convention and code-sample cleanup into 1c.
Conflict resolution:
- 04-code-samples.md, POC_taskList.md: deletion wins.
- OpenTelemetryPlan docs (01/02/03/05): took the convention-applied,
code-block-free versions; verified no attribute category, table row, or
section header was lost (the differences were dotted->underscore renames).
- Telemetry.h: kept 1c's RpcSpanNames.h constant-based example
(rpc_span::attr::command) over the string literal.
- 31 non-telemetry files are clean develop carry-forward (identical to 1b).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring the phase-1a code-sample cleanup into phase-1b.
Phase 1a deleted the two code-sample docs (04-code-samples.md,
POC_taskList.md) and stripped C++/config code blocks from the remaining
plan docs, replacing them with prose summaries and pointers to the real
source/config files.
Conflict resolution:
- 04-code-samples.md, POC_taskList.md: deletion wins (phase-1b's SpanGuard
rewrites of these files are intentionally dropped).
- 03/05/OpenTelemetryPlan.md: took phase-1a's code-block-free prose, but
kept phase-1b's accurate descriptions of the real telemetry code
(DiscardFlag.h, FilteringSpanProcessor, SpanGuard factory methods, and
the corrected file-count/file-list tables).
- presentation.md left untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring the span attribute naming convention (phase 1a) into phase 1b.
Conflict resolution kept phase-1b's SpanGuard-based workflow and applied
the underscore naming convention to all non-code-sample text:
- Converted prose, tables, Mermaid labels, and TraceQL/PromQL query
references across the plan docs to the underscore form.
- Converted the two @code attribute-key examples in Telemetry.h
(command, tx_type).
- Left the code-sample files (04-code-samples.md, POC_taskList.md) and
03-implementation-strategy.md code blocks at the phase-1b version; the
code-sample docs are slated for removal on phase-1a.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Establish the single, authoritative naming convention for OpenTelemetry
span attribute keys so the code, collector, Tempo, dashboards, and docs
stay in sync.
- CONTRIBUTING.md: new "Telemetry span attribute naming" section under
the Style guide as the permanent, canonical home for the rules.
- OpenTelemetryPlan/02-design-decisions.md: new section 2.3.3 stating the
decided convention as design, and section 2.4 attribute schema realigned
to the underscore form (exact key spelling defers to the *SpanNames.h
constants).
- Sweep the remaining plan docs: convert dotted xrpl.<domain>.<field> span
attribute keys to the underscore form; leave span names and the
OTel-standard service.*/http.* and xrpl.network.* resource keys dotted.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The xrpl_tests target links only xrpl.libxrpl and cannot include the
xrpld-level ConsensusSpanNames.h / RpcSpanNames.h (CI build failure on
ubuntu-gcc-debug-arm64). Revert the test to underscore-form attribute
literals, restore the SpanNames.h include for seg::consensus, and drop
the now-unused tests.libxrpl -> xrpld.* levelization entries.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- tempo.yaml: align consensus filter tags with emitted keys
(consensus_mode, consensus_round, ledger_seq) instead of dotted form
- haveConsensus(): set span attributes before early-return paths so the
consensus.check span carries diagnostics even when consensus is not reached
- replace hardcoded consensus phase/result/vote literals with
ConsensusSpanNames.h val constants; add val::phaseOpen/Establish/Accepted
- ConsensusReceiveTracing.h: use canonical consensus::span constants instead
of duplicate inline detail:: names
- SpanGuardFactory test: use rpc_span / consensus::span constants now that
levelization permits the dependency
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
BasicConfig.h moved from xrpl/basics/ to xrpl/config/ on develop
(PR #7095 / reorg). Phase 1b's telemetry headers still referenced the
old path, breaking a fresh compile with "BasicConfig.h: No such file or
directory". Point both telemetry includes at the new location.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Head sampling is now fixed at 1.0 and not configurable. Drop the
sampling_ratio config row and redirect volume-reduction guidance to
collector-side tail sampling. trace_peer=0 stays as the high-volume opt-out.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
sampling_ratio is no longer parsed (head sampling fixed at 1.0), so remove
the parse/clamp assertions and the sampling_ratio_clamped test. Update the
default-tracePeer expectations to TRUE to match the enabled-by-default change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document that head sampling is intentionally fixed at 100% and no longer
exposes a sampling_ratio config knob. A per-node ratio let nodes make
divergent keep/drop decisions for the same distributed trace, producing
broken/partial traces; pinning at 1.0 with a ParentBased sampler keeps
decisions coherent across the network. Volume reduction is delegated to
collector-side tail sampling.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reflect the new on-by-default behavior in the config defaults table. The
volume-reduction guidance (High memory usage, Production mainnet tuning)
intentionally keeps trace_peer=0 as an opt-out recommendation for
high-traffic deployments.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Flip the tracePeer Setup default and the trace_peer config parser
default from off to on, and update the example config and build doc to
match. Peer spans record only peer_id (a node-local numeric connection
id) plus trust/ledger metadata — no IP addresses or public keys — so
the privacy concern behind disabling it does not apply. The high-volume
characteristic is retained in the docs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Flip trace_peer default false->true across the Phase-1a plan docs and
correct the rationale: peer spans record only peer_id (numeric local
connection id) plus trust/ledger metadata, never IP addresses or public
keys, so the 'includes addresses' caveat was inaccurate. The high-volume
note is retained.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The tx.transactor span covered only the apply stage; preflight and
preclaim had no telemetry, so a transaction that hard-failed those
stages produced no apply-pipeline span and per-stage latency/failure
was invisible.
Add tx.preflight and tx.preclaim spans in applySteps.cpp via a
makeStageSpan() helper using SpanGuard::hashSpan, so all three stages
share a deterministic trace_id derived from txID[0:16] even though they
run sequentially and often cross-thread. Each span carries stage,
tx_type, and ter_result; exceptions are recorded as tefEXCEPTION before
the public wrappers map them. The type lookup is guarded behind the
span-active check so it costs nothing when tracing is off.
Add a stage="apply" attribute to the tx.transactor span and move its
three hardcoded attribute strings to a new library-safe header
include/xrpl/tx/detail/TxApplySpanNames.h, which mirrors the daemon-side
TxSpanNames.h strings so the collector spanmetrics connector aggregates
both span sets under one dimension set.
A constants-contract test pins the span-name, attribute-key, and
stage-value strings; span content stays covered by the docker
integration test, as the rest of the telemetry suite is.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The OTel C++ SDK's SetAttribute appends rather than overwrites on
in-flight spans. Setting suppressed=false as a default then overriding
to true resulted in both values appearing in the exported span.
Fix: remove the default-false set, place suppressed=false once after
the HashRouter check passes (non-suppressed path), and suppressed=true
remains only in the suppressed path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wraps Transactor::operator() with a span that captures tx_type,
ter_result, and applied. This is the universal dispatch point — every
transaction flows through it, giving per-type latency breakdown.
Adds libxrpl.tx > xrpl.telemetry levelization dependency.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wraps Transactor::operator() with a span that captures tx_type,
ter_result, and applied. This is the universal dispatch point — every
transaction flows through it, giving per-type latency breakdown.
Adds libxrpl.tx > xrpl.telemetry levelization dependency.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds comprehensive "Insights and Sample Queries" section showing operators
what questions they can answer with the newly-added span attributes:
- Transaction workflow analysis (filter by tx_type, fee, ter_result)
- TxQ health (txq_status, ledger_changed)
- RPC debugging (is_batch, request_payload_size, load_type)
- PathFinding performance (dest_currency, num_source_assets)
- Consensus health (consensus_state, is_bow_out, disputes_count)
- Cross-subsystem correlation examples
Also updates all span reference tables with the new attributes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds workflow-critical attributes to consensus spans:
- consensus.proposal.send: is_bow_out (identifies resignation proposals)
- consensus.accept: consensus_state (yes/moved_on/expired), disputes_count
- consensus.validation.send: ledger_hash (correlates validation to ledger)
Enables answering: "Did we reach consensus or time out?", "How many
disputes existed at acceptance?", "Which ledger did we validate?"
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The apply() function doesn't have a `using namespace telemetry` directive
(unlike processTransaction), so tx_span attrs need explicit qualification.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Enriches the tx.process span with final outcome after batch application:
- ter_result: the TER code string (e.g., "tesSUCCESS", "tecPATH_DRY")
- applied: boolean whether the transaction was included in the ledger
These attributes complete the tx.process span lifecycle — it now captures
identity (tx_type, tx_hash), intent (fee, sequence), and outcome
(ter_result, applied) for full workflow traceability.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire up span attributes that enable filtering/grouping traces by request
characteristics: batch detection, payload size, resource cost category,
command name on WS spans, and pathfinding search parameters (destination
amount/currency, source asset count).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two stray "rippled" tokens introduced by 43258e8d ("docs(telemetry):
add secure-OTel pipeline analysis…") were caught by check-rename in
CI. Re-run docs.sh to convert them to xrpld so the rename check
passes on PR #6425 (and downstream PR #6426 once merged up).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brings in pratik/otel-phase3-tx-tracing's two commits that move all
consensus-tracing content off phase-3:
- c9521b97fe refactor(telemetry): pull consensus-tracing scope-leak
out of phase-3
- d6b101069e refactor(telemetry): remove consensus tracing from
phase-3
Phase-4 already owns this content (with the consensus_span -> cons_span
namespace rename, round/accept/proposalSend/validationSend span
construction, and the relocated ConsensusSpanNames.h under
src/xrpld/consensus/), plus its own evolution on top — so the merge is
resolved as a tree-identity merge:
- src/xrpld/app/consensus/ConsensusSpanNames.h: keep phase-4's
renamed/expanded version (modify/delete conflict, resolved with
--ours).
- src/xrpld/app/consensus/RCLConsensus.cpp: keep phase-4's version
with cons_span attribute calls, the trace_context inject blocks
on broadcastPropose/validate, the secure-OTel TODO, and the
full validation/round span instrumentation (content conflict,
resolved with --ours).
- src/xrpld/overlay/detail/PeerImp.cpp: keep phase-4's version with
the proposalReceiveSpan/validationReceiveSpan calls, lambda span
captures, and cons_span::attr::* setAttribute calls (content
conflict, resolved with --ours).
- src/xrpld/telemetry/ConsensusReceiveTracing.h: phase-3 deleted it,
phase-4 still uses it. Restored from phase-4 HEAD (silent
auto-deletion otherwise).
- include/xrpl/telemetry/TraceContextPropagator.h: phase-3 stripped
consensus references and the secure-OTel TODO; phase-4 still has
both. Restored from phase-4 HEAD.
- src/xrpld/telemetry/PropagationHelpers.h: phase-3 swapped the
@see ConsensusReceiveTracing.h cross-reference for TxTracing.h;
phase-4 still wants the consensus reference. Restored from
phase-4 HEAD.
Net tree change on phase-4: zero. Verified via 'git diff <pre-merge-sha>
HEAD' returning empty.
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
Phase-3 (PR #6425) is scoped to transaction tracing only; consensus
tracing belongs to phase-4 (PR #6426). The previous commit on this
branch removed the namespace/attribute scaffolding c6c019ed8b leaked
into phase-3, but phase-3 still carried the consensus span construction
and trace-context propagation introduced in earlier commits
(61cb1faf8f, 93bed03d8d). Move that out too so phase-3 creates and
propagates no consensus spans of any kind.
Removed:
- src/xrpld/telemetry/ConsensusReceiveTracing.h (deleted; phase-4
owns it).
- PeerImp.cpp: remove the std::make_shared<SpanGuard>(
proposalReceiveSpan(...))/validationReceiveSpan(...) constructions
in onMessage(TMProposeSet)/onMessage(TMValidation), drop the
sp = std::move(span) lambda captures, and drop the
#include <xrpld/telemetry/ConsensusReceiveTracing.h>.
- RCLConsensus.cpp: drop the two telemetry::injectToProtobuf() blocks
that injected the active trace context into TMProposeSet (in
Adaptor::propose, after addSuppression) and TMValidation (in
Adaptor::validate, around the broadcast call). Drop the now-unused
#include of TraceContextPropagator.h and the
XRPL_ENABLE_TELEMETRY-gated include of
opentelemetry/context/runtime_context.h.
- TraceContextPropagator.h: update file-level @see comment to drop
the ConsensusReceiveTracing.h reference and to scope the
"wired into the P2P message flow via PropagationHelpers.h"
sentence to TMTransaction only.
- PropagationHelpers.h: replace the
@see ConsensusReceiveTracing.h cross-reference with
@see TxTracing.h.
Inert consensus metadata (TraceCategory::Consensus enum value,
seg::consensus constant, isCategoryEnabled/categoryToSpanKind switch
arms, the SpanGuard.h doc-comment example) is intentionally preserved
on phase-3: nothing references it after this commit, but phase-4
needs it and removing it would widen the phase-3 -> phase-4 merge
surface for no benefit.
Verified via git grep: no remaining phase-3 references to
proposalReceiveSpan, validationReceiveSpan, ConsensusReceiveTracing,
consensus_span::, consensus.proposal, or consensus.validation.
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
Commit c6c019ed8b ("addressed code review comments") bundled tx-tracing
review fixes with consensus-tracing scaffolding that belongs on
pratik/otel-phase4-consensus-tracing (PR #6426). This commit lifts the
consensus-only parts back out of phase-3 so PR #6425 stays scoped to
transaction tracing. Phase-4 already carries the same content (via prior
phase-3 -> phase-4 merges) plus its own evolution on top, so nothing is
moved across — only removed here.
Removed:
- src/xrpld/app/consensus/ConsensusSpanNames.h (deleted; phase-4 owns
it).
- PeerImp.cpp: revert onMessage(TMProposeSet)/onMessage(TMValidation)
consensus-attr setAttribute calls to the hardcoded
"xrpl.consensus.{trusted,round,ledger.seq}" strings used before
c6c019ed8b. Drop the now-unused
#include <xrpld/app/consensus/ConsensusSpanNames.h>.
- RCLConsensus::Adaptor::validate(): remove the
TODO(observability/secure-OTel) block on validation trace_context.
- TraceContextPropagator.h: remove the TODO(observability/secure-OTel)
block on injectToProtobuf().
Tx-tracing parts of c6c019ed8b are intentionally untouched.
No phase-3 caller of telemetry::consensus_span:: remains; verified via
git grep. No test on phase-3 references the removed header.
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
Document the threat model and chosen hardening approach for the OTel
pipeline: mTLS to the collector as primary defense (across-network
deployment), NetworkPolicy as defense-in-depth, and source-side
validation plus per-peer rate limiting for protocol::TraceContext on
peer messages. Skips Basic Auth (wrong shape for multi-operator
fleet) and HTTP-gateway header stripping (rippled is P2P).
Wires the new doc into the master plan ToC, mermaid diagram, and
body section, plus cross-refs from the privacy section in
02-design-decisions.md and the collector config in
05-configuration-reference.md so readers reach it from natural
in-context entry points. Adds a backlink at the top of secure-OTel.md
to the master plan.
Adds 'exfiltration' and 'htpasswd' to cspell dictionary.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Drop xrpl.node.amendment_blocked / xrpl.node.server_state from telemetry
surface (constants in SpanNames.h, two filters in tempo.yaml). Operators
read the same data via server_info / server_state RPC; OTel SDK 1.18.0
cannot refresh resource attrs at runtime so resource-level emission was
not viable either.
- Namespace all pathfind span attributes under pathfind_* (underscore form
per Phase 1c rule 5). Renames in PathFindSpanNames.h and call sites in
PathRequest.cpp, PathRequestManager.cpp, plus the rule-5 retention
xrpl.pathfind.ledger_index -> pathfind_ledger_index.
- Wire pathfind_source_account / pathfind_dest_account on pathfind.request
in doPathFind / doRipplePathFind handlers (only when present + string).
- Collapse per-asset pathfind.discover / pathfind.rank spans into one
pathfind.discover hoisted around the per-source-asset loop in
PathRequest::findPaths. Span count goes from 2N to 1 per RPC call;
per-asset breakdown traded for bounded storage and cardinality. Trade-off
documented inline.
- Fix pathfind_num_paths semantics: now sums getBestPaths().size() across
the loop (paths actually returned) instead of the maxPaths input cap.
- PathRequestManager::updateAll: move span creation after the locked
requests_ snapshot, early-return when no active subscriptions exist
(avoids empty span on every ledger close), set pathfind_num_requests
= requests.size().
- Update Phase2_taskList.md and 02-design-decisions.md to match.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per PR #6438 review thread r3250432621: known-command errors
(rpcTOO_BUSY, rpcNO_PERMISSION, etc.) were collapsing into a
single rpc.command.unknown span name, hiding per-command error
rates in dashboards. Same anti-pattern existed for gRPC, where
every method was bucketed under grpc.request with the method
relegated to an attribute.
- RPCHandler.cpp: doCommand error path uses cmdName as the span
suffix; the rpc_span::val::unknownCommand fallback only applies
when the request truly omits both command and method fields.
- GRPCServer.cpp: gRPC span name is now grpc.<MethodName>
(e.g. grpc.GetLedger). Method also retained as an attribute.
- GrpcSpanNames.h: drop the unused op::request constant; update
the span-hierarchy comment.
- RpcSpanNames.h: update the gRPC span diagram to match.
Dashboards on downstream phases will benefit from per-command
breakdowns without needing TraceQL attribute filters.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: semgrep-companion-app[bot] <218312740+semgrep-companion-app[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: xrplf-ai-reviewer[bot] <266832837+xrplf-ai-reviewer[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Phase 4 added a span catalog in `06-implementation-phases.md` listing the
source location for each consensus span. Line numbers `Consensus.h:707`,
`RCLConsensus.cpp:232/341/492/541/900` drift on every refactor and would
become stale PR after PR. Filename alone is enough for operators to
grep — the RCLConsensus.cpp spans are already unambiguous from the span
name itself.
- RPC Spans table: `rpc.request` was documented but the code actually emits
`rpc.http_request`. Listed the actual emitted names
(`rpc.http_request`, `rpc.ws_upgrade`, `rpc.ws_message`, `rpc.process`)
and their parent/child relationship.
- Drop `:<line>` suffixes from Source File columns in both RPC and
Transaction span tables. Line numbers drift with every refactor; the
filename is enough for operators to grep.
- Summary table: replace the never-emitted `rpc.request` row with the real
entry points so `span_name=` filters in PromQL / TraceQL match.
Phase-1a plan documents advertised OTLP/gRPC on port 4317 as the default
exporter, four unparsed [telemetry] config keys, and "Phase 4a Complete"
status with exit-criteria checkboxes marked done. Every downstream branch
through Phase 5 ships only OTLP/HTTP on port 4318 via OtlpHttpExporterFactory,
never parses the advertised keys, and the Phase 4 work is not yet delivered.
Fixes:
- 02-design-decisions.md: flip §2.1.1 SDK dependency recommendations to
OTLP/HTTP (shipped) with OTLP/gRPC marked Future. Update §2.2 architecture
diagram and text from OTLP/gRPC:4317 to OTLP/HTTP:4318. Rewrite §2.2.1 as
"OTLP/HTTP (Shipped)" and §2.2.2 as "OTLP/gRPC (Future Work — Planned
Upgrade)" with a concrete checklist (Conan dep, config parsing, factory
branch, runbook/dashboard updates) for landing the gRPC transport later.
- 05-configuration-reference.md: drop the fabricated exporter/otlp_grpc key
and the :4317 default from the sample config block and the options-summary
table. Move trace_pathfind, trace_txq, trace_validator, trace_amendment
into a new "Planned (not yet implemented)" table citing the phase that will
add each one. Keep the example config minimal so copy-paste does not produce
a silently-ignored stanza.
- 06-implementation-phases.md: reset Phase 4 Exit Criteria checkboxes from
[x] to [ ] (Phase 4 is not shipped at Phase-1a time). Rename "Phase 4a
Complete" to "Phase 4a Plan" and describe the work as future. Replace the
broken forward link to Phase4_taskList.md (introduced in the Phase 2 PR)
with a sentence pointing readers to where that spec will land. Renumber
the final section 6.12 to 6.11 so it sits directly after 6.10; section 6.11
("Effort Summary") was intentionally removed in earlier edits.
SpanGuard::span() hardcoded SpanKind::kInternal for every span. Tempo's
service-graph and spanmetrics RED calculations rely on kServer /
kConsumer / kClient / kProducer to classify inbound vs outbound vs
internal operations. With kInternal everywhere, the service graph
collapses to a single self-loop and RED metrics attribute all latency
to internal work.
Add categoryToSpanKind() mapping:
- Rpc -> kServer (inbound synchronous request)
- Peer -> kConsumer (inbound async peer message)
- Transactions -> kInternal
- Consensus -> kInternal
- Ledger -> kInternal
Only the single-argument overload is affected; childSpan / linkedSpan
continue to default to kInternal because they represent in-process
continuations of an already-kinded parent.
Consensus span attributes use bare names (close_time_correct,
consensus_state, close_resolution_ms) and shared canonical attrs
(xrpl.ledger.seq) per SpanNames.h. xrpl.consensus.mode and
xrpl.consensus.round are correct (domain-qualified to avoid collision).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Transaction span attributes use bare names (local, tx_status) per
SpanNames.h convention, not xrpl.tx.* qualified names. xrpl.tx.hash
is correct (shared canonical attr defined in SpanNames.h).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
RPC span attributes use bare names (command, rpc_status, rpc_role) per
the naming convention in SpanNames.h, not xrpl.rpc.* qualified names.
Node health attributes (amendment_blocked, server_state) are resource
attributes set at Tracer init, not span attributes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
NetworkOPs.h and SpanNames.h were only needed for per-span
nodeAmendmentBlocked/nodeServerState calls, which were removed
in the attr naming simplification. Fixes clang-tidy CI failure.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Phase2_taskList: update attr refs to bare names, note node-health
attrs moved to resource level.
- 02-design-decisions: strip xrpl.pathfind.* prefix from planned attrs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update OpenTelemetryPlan docs and Telemetry.h doc example to reflect
the renamed per-span attributes: xrpl.rpc.command -> command,
xrpl.rpc.status -> rpc_status, xrpl.grpc.method -> method, etc.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update 31 attribute references in telemetry-runbook.md to match the
simplified naming: drop xrpl.<domain>. prefix on per-span attrs, use
domain-qualified names for collisions (rpc_status, consensus_state,
etc.), and unify cross-domain refs (xrpl.ledger.seq, xrpl.tx.hash).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Drop xrpl.pathfind.* prefix from per-span attrs (source_account,
dest_account, fast, search_level, num_complete_paths, num_paths,
num_requests).
- Keep xrpl.pathfind.ledger_index qualified (rule 5: distinct from
xrpl.ledger.seq).
- Remove per-span nodeAmendmentBlocked/nodeServerState calls from
RPCHandler — promoted to resource-level attrs.
- Mark node-health attrs in SpanNames.h as RESOURCE-ONLY with doc.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Set defaults for tx_span::attr::suppressed (false) and
tx_span::attr::status ("new") immediately after creating the txReceive
span. Without defaults, spans whose suppressed/status attributes would
only be set in the HashRouter-suppressed branch lacked these attributes
entirely, producing incomplete span data in downstream stores.
The suppressed branch still overrides these when the transaction has
already been seen via HashRouter.
- Add [[nodiscard]] to getConsensusTraceStrategy, getYays, getNays
- Add missing <string>, SpanGuard.h, SpanNames.h includes
- Fix widening cast placement (cast before arithmetic, not after)
- Replace nested ternary with lambda for const dir variable
- Add braces to if/else-if chains in Consensus.h
- Concatenate nested namespaces in ConsensusSpanNames.h
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Concatenate nested namespaces in SpanNames.h, RpcSpanNames.h, GrpcSpanNames.h
- Remove unused InfoSub.h and NetworkOPs.h includes from RPCHandler.cpp
- Add missing <string_view> includes in RPCHandler.cpp and GRPCServer.cpp
- Replace nested ternary with if/else-if in RPCHandler.cpp
- Add IWYU pragma keep for json_body.h in ServerHandler.cpp
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Clang-tidy misc-include-cleaner requires direct includes for all used
symbols. Application.cpp calls toBase58(TokenType::NodePublic, ...) at
line 1359 but did not directly include PublicKey.h.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Document the propagation infrastructure: send-side injection in
NetworkOPs/RCLConsensus, receive-side extraction in PeerImp via
PropagationHelpers.h and ConsensusReceiveTracing.h. Update
consensus receive span descriptions to reflect parent extraction.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add [[maybe_unused]] to RAII spans in TxQ.cpp
- Include Telemetry.h in RCLConsensus.cpp for complete type
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix quorum attribute to use actual validator quorum instead of proposer
count, add missing ConsensusState::Expired handling in haveConsensus()
span, move ConsensusSpanNames.h to xrpld/consensus/ to resolve
levelization cycle, remove unused constants, enrich proposal receive
span with sequence, and correct stale documentation references.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add cons_span::event namespace with disputeResolve and txIncluded
constants; replace hardcoded strings in Consensus.h and RCLConsensus.cpp
- Move proposal.receive and validation.receive spans in PeerImp into
shared_ptr captured by job lambdas so they measure checkPropose and
checkValidation timing, not just message parsing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update Phase4_taskList.md and 06-implementation-phases.md to reflect
completed implementation of all remaining Phase 4/4a tasks (4.2-4.6,
4a.5, 4a.6, 4a.8). Update exit criteria and summary tables.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement remaining Phase 4/4a consensus tracing tasks:
- Add consensus.phase.open span (open → closeLedger lifecycle)
- Add consensus.proposal.receive span in PeerImp with trusted attr
- Add consensus.validation.receive span in PeerImp with trusted/seq attrs
- Add tx_count attr on accept.apply, disputes_count on update_positions
- Add tx.included events with txId in doAccept transaction loop
- Enhance dispute.resolve event with yays/nays fields
- Add avalanche_threshold attr on update_positions span
- Reparent accept/accept.apply as children of round span via childSpan()
Also adds compile-time constants in ConsensusSpanNames.h and updates
the span hierarchy diagram.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The 4-arg hashSpan overload was duplicated during a prior rebase
cascade — it appeared at both line 240 and line 305 in SpanGuard.cpp.
This would cause a linker error (multiple definition).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Record the close time voting threshold and consensus state on
consensus.update_positions and consensus.check spans:
- xrpl.consensus.close_time_threshold: the avCT_CONSENSUS_PCT (75%)
threshold required for close time agreement
- xrpl.consensus.have_close_time_consensus: whether validators
reached close time consensus in this iteration
These attributes enable dashboards to show how the close time
voting process converges (or stalls) across consensus iterations.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove the span-replacement logic in startRoundTracing() that was
discarding the hash-derived round span and replacing it with a linked
span (which gets a random trace_id). The deterministic trace_id from
the ledger hash is the key feature enabling cross-node correlation —
replacing it broke correlation on all rounds after the first.
Also: use thread_local mt19937 for hashSpan() span IDs (same fix as
phase-3 txSpan), add Doxygen to establish tracing method declarations
in Consensus.h, and update SpanGuard.h diagram with hashSpan/addEvent.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Instrument the consensus subsystem with OpenTelemetry spans covering
the full round lifecycle: round start, establish phase, proposal send,
ledger close, position updates, consensus check, accept, validation
send, and mode changes.
Key design choices adapted from the original Phase 4 implementation
to the new SpanGuard factory pattern introduced in Phase 3:
- Add SpanGuard::hashSpan() for category-gated hash-derived trace IDs
(consensus round spans share trace_id across validators via ledger hash)
- Add SpanGuard::addEvent() overload with key-value attribute pairs
(used for dispute.resolve events during position updates)
- Add ConsensusSpanNames.h with compile-time span name constants
following the colocated *SpanNames.h pattern from Phase 3
- Add consensusTraceStrategy config option ("deterministic"/"attribute")
for cross-node trace correlation strategy selection
- Use SpanGuard::linkedSpan() for follows-from relationships between
consecutive rounds and cross-thread validation spans
- Use SpanGuard::captureContext() for thread-safe context propagation
from consensus thread to jtACCEPT worker thread
Spans produced: consensus.round, consensus.proposal.send,
consensus.ledger_close, consensus.establish, consensus.update_positions,
consensus.check, consensus.accept, consensus.accept.apply,
consensus.validation.send, consensus.mode_change
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wire trace context into P2P message flow so distributed traces
link across nodes. TX relay injects SpanGuard context via
PropagationHelpers.h; consensus propose/validate injects via
TraceContextPropagator.h. Receive-side extraction in PeerImp
creates child spans for proposals and validations.
- Add TraceBytes struct and SpanGuard::getTraceBytes() for
extracting raw trace context without OTel type dependencies
- Add PropagationHelpers.h: injectSpanContext(SpanGuard, proto)
- Add ConsensusReceiveTracing.h: proposalReceiveSpan(),
validationReceiveSpan() with parent context extraction
- NetworkOPs::apply(): inject tx.process context before relay
- RCLConsensus::propose()/validate(): inject active span context
- PeerImp: create receive spans for proposals and validations
with sender's trace context as parent
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move TxQSpanNames.h include to correct alphabetical position, update
levelization results for new xrpld.telemetry module dependencies,
and apply rename script to docs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- tx.receive span in PeerImp: convert to shared_ptr, capture in
checkTransaction lambda so it measures actual processing, not just
message parsing
- tx.process span in NetworkOPs: convert to shared_ptr, store in
TransactionStatus so it lives until the batch job processes the entry;
sync path unchanged (span destructs on function return)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace SpanGuard::txSpan(prefix, name, hash) with the generic
SpanGuard::hashSpan(TraceCategory, name, hash) that accepts a
TraceCategory parameter instead of hardcoding Transactions. This
enables reuse for consensus round spans (Phase 4) and any future
subsystem needing deterministic cross-node trace correlation via
hash-derived trace IDs.
Both overloads are replaced:
- hashSpan(cat, name, hash, size) — standalone with random span_id
- hashSpan(cat, name, hash, size, parentSpanId, parentSize, flags)
— with remote parent from protobuf context propagation
Add full span name constants (tx_span::receive, tx_span::process)
to TxSpanNames.h following the ConsensusSpanNames.h pattern.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mark local variables in extractFromProtobuf() and injectToProtobuf()
as const since they are not modified after initialization: traceId,
spanId, flags, spanCtx, and span.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace thread_local mt19937 with xrpl::default_prng() for span ID
generation — uses the project's existing thread-local xor-shift engine.
One call yields a uint64_t (8 bytes), filling the span ID in a single
memcpy without loops.
Fix compilation failure when XRPL_ENABLE_TELEMETRY is not defined:
move xrpl.pb.h include outside the #ifdef guard in TxTracing.h since
protocol::TMTransaction is used unconditionally in the function
signature.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace per-call std::random_device with thread_local std::mt19937 in
txSpan() for span ID generation. random_device is ~423x slower due to
/dev/urandom syscalls on each construction; mt19937 is seeded once per
thread and reused for all subsequent span IDs.
Update the SpanGuard class ASCII diagram to include txSpan factory
methods that were added in the hash-derived trace ID commit.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move TxSpanNames.h and TxQSpanNames.h from src/xrpld/telemetry/ to sit
next to the classes they instrument, matching the PathFindSpanNames.h
convention.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Derive trace_id from txHash[0:16] so all nodes handling the same
transaction produce spans under the same trace. Protobuf span_id
propagation provides parent-child relay ordering when available.
- Add SpanGuard::txSpan() factory methods (hash-derived trace ID)
- Add TxTracing.h helpers: txReceiveSpan(), txProcessSpan()
- Update PeerImp and NetworkOPs to use the new helpers
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add trace_id = txHash[0:16] strategy so all nodes handling the same
transaction independently produce spans under the same trace_id,
combined with protobuf span_id propagation for parent-child ordering.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move scattered string literals from PeerImp.cpp and NetworkOPs.cpp into
compile-time constants in src/xrpld/telemetry/TxSpanNames.h. Follows
the same StaticStr/join() pattern established in Phase 1c for RPC spans.
Constants cover: span prefixes (tx), operations (receive, process),
attribute keys (hash, local, path, suppressed, status, peerId,
peerVersion), and values (sync, async, knownBad).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace references to old XRPL_TRACE_TX/CONSENSUS macros with
SpanGuard::span(TraceCategory, ...) factory calls introduced in Phase 1c.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds xrpl.peer.version attribute to tx.receive spans for version-mismatch
correlation during network upgrades.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add "hicpp" to cspell dictionary for NOLINT annotations
- Concatenate nested namespaces in RpcSpanNames.h
- Fix include hygiene and nested ternary in RPCHandler.cpp
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add [[maybe_unused]] to RAII span variables in PathFind/RipplePathFind
handlers (Clang -Wunused-variable with -Werror)
- Restore over-renamed values: rippledb, rippled.cfg, historical GitHub URL
- Concatenate nested namespaces in SpanNames.h and PathFindSpanNames.h
(modernize-concat-nested-namespaces)
- Add missing includes and const qualifiers in test files
- Suppress intentional use-after-move in SpanGuardFactory move test
- Remove unused NetworkOPs.h include from PathRequest.cpp
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Document service_instance_id, use_tls, tls_ca_cert, batch_size,
batch_delay_ms, and max_queue_size in the [telemetry] section.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add unknownCommand and wsUpgrade span name constants to RpcSpanNames.h,
fix SpanGuardFactory tests to use the 3-argument SpanGuard::span() API,
update levelization results, and apply rename script to docs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use C++17 concatenated namespaces, add [[nodiscard]] to query methods,
add missing direct includes, and use pass-by-value + std::move in
NullTelemetry constructor.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sort PathFindSpanNames.h after AssetCache.h alphabetically in
PathRequestManager.cpp and Pathfinder.cpp to satisfy the project's
include-order convention enforced by pre-commit hooks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move SpanGuard.h (associated header) to first include position,
separated by blank line from other project includes, per the
project's include-order convention enforced by pre-commit hooks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move Telemetry.h (associated header) to first include position in
Telemetry.cpp per the project's include-order convention. Trim
trailing whitespace from POC_taskList.md markdown table columns.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Instrument the path finding subsystem with full span coverage:
- pathfind.request: wraps doPathFind() and doRipplePathFind() RPC handlers
- pathfind.compute: wraps PathRequest::doUpdate() with fast/normal attr
- pathfind.update_all: wraps PathRequestManager::updateAll() on ledger
close with ledger_index attr
- pathfind.discover: wraps Pathfinder::findPaths() graph exploration
with search_level attr
- pathfind.rank: wraps Pathfinder::computePathRanks() liquidity
validation with num_paths attr
New file: PathFindSpanNames.h with compile-time constants following
the StaticStr/join() pattern from Phase 1c.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Move node health attribute strings to compile-time constants in
SpanNames.h (attr::nodeAmendmentBlocked, attr::nodeServerState)
- Add Tempo search filters for node health attributes
- Remove unnecessary .c_str() on strOperatingMode() return
- Add samplingRatio clamping test (values > 1.0 and < 0.0)
- Fix Task 2.3 status: delivered in Phase 1c, not Phase 2
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- serviceName default is "xrpld" not "rippled"
- Remove references to nonexistent exporterType field
- Pass networkId (4th param) to setup_Telemetry()
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mark deferred tasks (2.1→Phase 3, 2.5→low priority) with rationale.
Mark superseded tasks (2.2→Phase 1c SpanGuard factory). Add Task 2.7
for Grafana search filters. Update summary table with status column.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Grafana Tempo datasource: add rpc-command, rpc-status, rpc-role
search filters for the Explore UI
- Unit tests: TelemetryConfig (config parsing defaults and sections),
SpanGuardFactory (null guard safety, move semantics, discard, all
factory methods)
- Test CMake registration with optional OTel linking
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add amendment_blocked and server_state span attributes to every
rpc.command.* span so operators can correlate RPC behavior with node state.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduces task list documents for Phases 2 through 5, with Tempo
references (replacing Jaeger) and Task 2.8 dashboard parity spec.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Centralise scattered string literals into compile-time constants using
StaticStr<N> and join() for dot-separated composition. Shared primitives
live in SpanNames.h; RPC-specific names in RpcSpanNames.h. Future modules
(consensus, peer, ledger) add their own *SpanNames.h without bloating
the central header.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add [[maybe_unused]] to the RAII span in processSession() — the
variable is not read but its lifetime scopes the active OTel context
for child spans created in processRequest()
- Regenerate levelization: remove premature xrpld.telemetry entries
that reference a module not yet present on this branch
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use grpc_span::val::resourceExhausted constant instead of raw
"resource_exhausted" string in GRPCServer.cpp
- Fix unbounded span name cardinality in RPCHandler.cpp error path:
use fixed rpc_span::val::unknownCommand as span name instead of
user-supplied cmdName (attacker-controlled input). The actual
command is still captured in the xrpl.rpc.command attribute.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Without this, cloned CallData instances (created for the next incoming
gRPC request) would have an empty name_, making subsequent span attrs
blank.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Follow project convention (PerfLog.h, SpanGuard.h) for documentation
diagrams. Show HTTP single, HTTP batch, and WebSocket span nesting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document the span hierarchy, covered paths, and known instrumentation
gaps directly in the header that developers reference when adding spans.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Centralise scattered string literals into compile-time constants using
StaticStr<N> and join() for dot-separated composition. Shared primitives
live in SpanNames.h; RPC-specific names in RpcSpanNames.h. Future modules
(consensus, peer, ledger) add their own *SpanNames.h without bloating
the central header.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace rpcSpan(fullName) calls with span(TraceCategory::Rpc, prefix,
name). Add 'using namespace telemetry' to both RPC files so call sites
read cleanly without repeated namespace qualifiers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Delete TracingInstrumentation.h and replace all XRPL_TRACE_* macro
invocations with direct SpanGuard::rpcSpan() calls. SpanGuard's pimpl
design and global Telemetry accessor eliminate the need for macro
wrappers and explicit Telemetry instance passing at call sites.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
TracingInstrumentation.h:
- Unify all span-creation macros to use std::optional<SpanGuard>
(fixes type mismatch between XRPL_TRACE_SPAN and SET_ATTR)
- Wrap XRPL_TRACE_SET_ATTR/EXCEPTION in do-while(0) (dangling-else)
- Move macros outside namespace blocks (macros are global)
- Cache telemetry reference to avoid double-evaluation
- Remove leaked _xrpl_span_ intermediate variable
- Add @note tags for thread safety, scope, and usage constraints
- Add 3 usage examples per CLAUDE.md requirements
ServerHandler.cpp:
- Remove misleading rpc.request span from onRequest() (span ended
before coroutine runs, producing orphan spans)
- Add rpc.http_request span to HTTP processSession() (runs inside
the coroutine, correct parent for rpc.process/rpc.command spans)
- Add XRPL_TRACE_EXCEPTION and error status in both catch blocks
(WS processSession and processRequest)
SpanGuard.h:
- Add null guards to all mutating methods (setOk, setStatus,
setAttribute, addEvent, recordException) for safety after discard()
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Concatenate nested namespaces (modernize-concat-nested-namespaces)
- Add [[nodiscard]] to factory and accessor methods
- NOLINT no-op stub instance methods that must stay non-static for API
parity with the real implementation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove the single-arg span(name) factory that creates unconditional
spans without category gating. All call sites use the 3-arg
span(TraceCategory, prefix, name) variant which checks whether the
category is enabled in config before creating a span. The 1-arg form
was dead code with no callers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace references to non-existent TracingInstrumentation.h with
SpanGuard.cpp pimpl implementation that actually exists on this branch.
Update conditional compilation section to describe the pimpl approach.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace rpcSpan(), txSpan(), consensusSpan(), peerSpan(), ledgerSpan()
with a single span(TraceCategory, prefix, name) factory method. Adding
a new traceable subsystem now requires only a new enum value and one
switch case — no new methods or header changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- SpanContext::isValid(): add inline no-op when XRPL_ENABLE_TELEMETRY
is not defined, preventing a linker error if called in that path
- linkedSpan(): set kIsRootSpanKey on the StartSpanOptions parent
context so linked spans start a genuinely independent sub-tree
instead of silently becoming children of the current active span
- Telemetry::instance_: use std::atomic with acquire/release ordering
to avoid a data race between start()/stop() and factory methods
called from worker threads
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Redesign SpanGuard with pimpl idiom to hide all OpenTelemetry types
from public headers. Add global Telemetry accessor so SpanGuard factory
methods work without explicit Telemetry references. Add child/linked
span creation and cross-thread context propagation. Update plan docs
to reflect macro removal in favor of SpanGuard factory pattern.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add DiscardFlag.h and FilteringSpanProcessor references to the file
tree, key files table, and implementation summary in OpenTelemetryPlan.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add span discard mechanism that drops unwanted spans before they enter
the batch export queue, saving both network bandwidth and storage.
FilteringSpanProcessor is a custom SpanProcessor decorator that wraps
BatchSpanProcessor. SpanGuard::discard() sets a thread-local flag
(tl_discardCurrentSpan) before calling Span::End(). The OTel SDK calls
OnEnd() synchronously on the same thread, where the flag is checked and
cleared to drop the span.
New file: DiscardFlag.h — zero-dependency header for the thread-local
flag, avoiding transitive include bloat from Telemetry.h.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove duplicate otlp/tempo exporter block, duplicate tempo service
definition, and jaeger dependency from docker-compose example.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tempo is now the sole trace backend. Remove Jaeger all-in-one service
from docker-compose, otlp/jaeger exporter from OTel Collector config,
and Jaeger Grafana datasource provisioning file.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Run .github/scripts/rename/docs.sh to replace rippled → xrpld
references in all plan documentation files, fixing the check-rename
CI failure.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Plan documents referenced Application.h and app_ for getTelemetry()
but the codebase now uses ServiceRegistry as the interface. Updated:
- 05-configuration-reference.md: getTelemetry() on ServiceRegistry,
deferred serviceInstanceId pattern in ApplicationImp
- POC_taskList.md Task 4: target ServiceRegistry.h not Application.h,
correct config file path and constructor pattern
- 04-code-samples.md: fix overlay() -> getOverlay(), rewrite JobQueue
sample to reflect actual architecture (no app_ member)
- 03-implementation-strategy.md: fix file impact table path
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Strip effort/risk columns from task tables and remove the §6.9 Effort
Summary section with its pie chart and resource requirements table.
Renumber §6.10 Quick Wins → §6.9.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split document index into Plan Documents and Task Lists sections.
These files were introduced in this branch but missing from the index.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This change:
* Removes a set of unnecessary brackets in the initialization of an `std::uint32_t`.
* Fixes a couple of incorrect flags (same value, just wrong variables - so no amendment needed).
This change replaces all instances of `<variable> != tesSUCCESS` with `!isTesSuccess(<variable>)` and `<variable> == tesSUCCESS` with `isTesSuccess(<variable>)`.
This change fixes delegation:
* If the Delegate object is not present, we should disallow empty permission list in DelegateSet preclaim.
* Empty permission list is only allowed to delete the existing Delegate object.
* In `doApply`, permission list being empty returns `tecINTERNAL`, which should not happen.
This change:
* Makes `addSLE` in `DIDSet` a static function, instead of a free function.
* Renames `Attestation` to `Data` everywhere (an artifact of a previous name for the field).
* Actually runs a set of tests that were not included in the `run` function of `DID_test`.
This change:
* Introduces a new helper function on `STTx`, `getFeePayer`.
* Removes the usage of `mSourceBalance` and replaces it with SLE balance lookups.
* Renames `mPriorBalance` to `preFeeBalance_`
This simplifies some of the code in the transactors and makes it a lot more readable.
Throwing exceptions from code sometime confuses ASAN, as it cannot keep track of stack frames. This change therefore adds a macro to skip instrumentation around the `Throw` function.
This change moves the sanitizer runtime options out to dedicated files, such that they can be used in multiple places (CI, local runs) without any need to rewrite them.
Per [XLS-0095](https://xls.xrpl.org/xls/XLS-0095-rename-rippled-to-xrpld.html), we are taking steps to rename ripple(d) to xrpl(d). This change modifies the system name from `rippled` to `xrpld`.
The system name is used in limited places:
* When no explicit config file is passed via the `--config` flag, then the system name is used to construct the path where the config file and database may be stored, via the `$XDG_CONFIG_HOME` and `$XDG_DATA_HOME` directories, respectively.
* It is used in the metadata and user-agent as part of RPC calls.
* It is newly used in the full version string.
ASAN wasn't able to keep track of `boost::coroutine` context switches, and would lead to many false positives being detected. By switching to `boost::coroutine2` and `ucontext`, ASAN is able to know about the context switches advertised by the `boost::fiber` class, which in turn leads to more cleaner ASAN analysis.
The `HTTPClient` class initializes a global SSL context via `initializeSSLContext()`. However, it had no way to release it, which caused memory leaks flagged by the LeakSanitizer. Multiple LSAN suppressions in the sanitizers' suppressions file were masking these leaks. Our test code also manually called `initializeSSLContext()` in each test without guaranteed cleanup on failure paths.
This change fixes these memory leaks by adding a `cleanupSSLContext()` method to properly release the global SSL context, and removes the corresponding LSAN suppressions. The change further refactors the `HTTPClient` tests to use a Google Test fixture (`HTTPClientTest`) that manages the SSL context lifecycle via RAII (SetUp/TearDown), making it impossible for tests to leak the context.
This change deletes the `SecretKey` equality/inequality operators from the public library header and moves the comparison logic into test-only code.
Specifically, the `operator==` and `operator!=` free functions on `SecretKey` have been removed from `include/xrpl/protocol/SecretKey.h` and have been replaced with explicitly deleted member functions to prevent accidental use in production code. A named `test::equal()` helper has also been added in `src/test/unit_test/utils.h` for test assertions that need to compare secret keys.
This change removes an unnecessary newline in a logging statement. Namely, `std::endl` is unneeded in `JLOG`, since it automatically places a newline at the end of the string.
This change enforces a maximum length of 63 characters on feature names, as well as not permitting an exactly 32 character long feature name to avoid confusion with those that use a `uint256` hex representation, as that is an alternative way to specify a feature. This change further prevents the use of Unicode characters in feature names, because some can be confused with regular ASCII characters despite being valid in identifiers.
The `setFieldU32` call is currently used to set the credential's expiry using a `getFieldU32` call to obtain the expiration time. However, that value was already obtained previously and can thus be reused.
DID, Escrows, PaymentChannels, and Credentials previously contained multiple unrelated transactor classes in a single header/implementation pair. This change splits each into one class per file, following the same pattern established by the rest of the codebase.
Now that prefixes in PR titles are being validated as part of CI, the "Type of Change" section in the PR template is no longer needed. The prefixes and descriptions in the `CONTRIBUTING.md` file have been updated to reflect the currently supported list.
Subscribe tests have a problem that there is no way to synchronize application running in background threads and test threads. Threads are communicating via websocket messages. When the code is compiled in debug mode with code coverage enabled it executes quite slow, so receiving websocket messages by the client in subscribe tests may time out.
This change does 2 things to fix the problem:
* Increases timeout for receiving a websocket message.
* Decreases the number of tests running in parallel.
While testing the fix for subscribe test another flaky test in ledger replay was found, which has also been addressed.
The refs as previously used pointed to the source branch, not the target branch. However, determining the target branch is different depending on the GitHub event. The pull request logic was incorrect and needed to be fixed, and the logic inside the workflow could be simplified. Both modifications have been made in this commit.
This change reorganizes the `tx/transactors` directory for consistency and discoverability. There are no behavioral changes, this is a pure refactor. Underscores were chosen as the way to separate multi-words as this is the more popular option in C++ projects.
Specific changes:
- Rename all subdirectories to lowercase/snake_case (`AMM` → `amm`, `Check` → `check`, `NFT` → `nft`, `PermissionedDomain` → `permissioned_domain`, etc.)
- Merge `AMM/` and `Offer/` into `dex/`, including `PermissionedDEXHelpers`
- Rename `MPT/` → `token/`, absorbing `SetTrust` and `Clawback`
- Move top-level transactors into named groups: `account/`, `bridge/`, `credentials/`, `did/`, `escrow/`, `oracle/`, `payment/`, `payment_channel/`, `system/`
- Update all include paths across the codebase and `transactions.macro`
The existing code added the git commit info (`GIT_COMMIT_HASH` and `GIT_BRANCH`) to every file, which was a problem for leveraging `ccache` to cache build objects. This change adds a separate C++ file from where these compile-time variables are propagated to wherever they are needed. A new CMake file is added to set the commit info if the `git` binary is available.
When `gateway_balances` gets called on an account that is involved in the `EscrowCreate` transaction (with MPT being escrowed), the method returns internal error. This change fixes this case by excluding the MPT type when totaling escrow amount.
The `Subscribe` tests were flaky, because each test performs some operations (e.g. sends transactions) and waits for messages to appear in subscription with a 100ms timeout. If tests are slow (e.g. compiled in debug mode or a slow machine) then some of them could fail. This change adds an attempt to synchronize the background Env's thread and the test's thread by ensuring that all the scheduled operations are started before the test's thread starts to wait for a websocket message. This is done by limiting I/O threads of the app inside Env to 1 and adding a synchronization barrier after closing the ledger.
The invariant check system had grown into a single monolithic file pair containing 24 invariant checker classes. The large `InvariantCheck.cpp` file was a frequent source of merge conflicts and difficult to navigate. This refactoring improves maintainability and readability with zero behavioral changes.
In particular, this change:
- Splits `InvariantCheck.h` and `InvariantCheck.cpp` into 10 focused header/source pairs organized by domain under a new `invariants/` subdirectory.
- Extracts the shared `Privilege` enum and `hasPrivilege()` function into a dedicated `InvariantCheckPrivilege.h` header, so domain-specific files can reference them independently.
This change enables all clang-tidy checks that are already passing. It also modifies the clang-tidy CI job, so it runs against all files if .clang-tidy changed.
This change replaces `void const*` by `uint256 const&` for database fetches.
Object hashes are expressed using the `uint256` data type, and are converted to `void *` when calling the `fetch` or `fetchBatch` functions. However, in these fetch functions they are converted back to `uint256`, making the conversion process unnecessary. In a few cases the underlying pointer is needed, but that can then be easy obtained via `[hash variable].data()`.
Updates XRPLF/actions prepare-runner to version 2cbf48101 which fixes
pip upgrade failures on macOS runners with Homebrew-managed Python.
* This commit was cherry-picked from "release-3.1", but ended up empty
because the changes are already present. It is included only for
accounting - to indicate that all changes/commits from the previous
release will be in the next one.
For coverage builds, we try to link against the `gcov` library (specific to the environment). But as macOS doesn't have this library and thus doesn't have the coverage tools to generate reports, the coverage builds on that platform were failing on linking.
We actually don't need to explicitly force this linking, as the `CodeCoverage` file already has correct detection logic (currently on lines 177-193), which is invoked when the `--coverage` flag is provided:
* AppleClang: Uses `xcrun -f llvm-cov` to set `GCOV_TOOL="llvm-cov gcov"`.
* Clang: Finds `llvm-cov` to set `GCOV_TOOL="llvm-cov gcov"`.
* GCC: Finds `gcov` to set `GCOV_TOOL="gcov"`.
The `GCOV_TOOL` is then passed to `gcovr` on line 416, so the correct tool is used for processing coverage data.
This change therefore removes the `gcov` suffix from lines 473 and 475 in the `CodeCoverage.cmake` file.
The rdb module was not properly designed, which is fixed in this change. The module had three classes:
1) The abstract class `RelationalDB`.
2) The abstract class `SQLiteDatabase`, which inherited from `RelationalDB` and added some pure virtual methods.
3) The concrete class `SQLiteDatabaseImp`, which inherited from `SQLiteDatabase` and implemented all methods.
The updated code simplifies this as follows:
* The `SQLiteDatabaseImp` has become `SQLiteDatabase`, and
* The former `SQLiteDatabase `has merged with `RelationalDatabase`.
This change modularizes the `WalletDB` and `Manifest`. Note that the wallet db has nothing to do with account wallets and it stores node configuration, which is why it depends on the manifest code.
This change removes the cache in `DatabaseNodeImp` and simplifies the caching logic in `SHAMapStoreImp`. As NuDB and RocksDB internally already use caches, additional caches in the code are not very valuable or may even be unnecessary, as also confirmed during preliminary performance analyses.
In certain cases, such as when modifying headers used by many compilation units, performing a unity build is slower than when performing a regular build with `ccache` enabled. There is also a benefit to a unity build in that it can detect things such as macro redefinitions within the group of files that are compiled together as a unit. This change therefore restores the ability to perform unity builds. However, instead of running every configuration with and without unity enabled, it is now only enabled for a single configuration to maintain lower computational use.
As part of restoring the code, it became clear that currently two configurations have coverage enabled, since the check doesn't focus specifically on Debian Bookworm so it also applies to Debian Trixie. This has been fixed too in this change.
The `ManifestCache::applyManifest` function was returning early without incrementing `seq_`. `OverlayImpl `uses this sequence to identify/invalidate a cached `TMManifests` message, which is exchanged with peers on connection. Depending on network size, startup sequencing, and topology, this can cause syncing issues. This change therefore increments `seq_` when a new manifest is accepted.
Unity builds were intended to speed up builds, by bundling multiple files into compilation units. However, now that ccache is available on all platforms, there is no need for unity builds anymore, as ccache stores compiled individual build objects for reuse. This change therefore removes the ability to make unity builds.
Currently we're passing the `Application` object around, whereby the `Application` class acts more like a service registry that gives other classes access to other services. In order to allow modularization, we should replace `Application` with a service registry class so that modules depending on `Application` for other services can be moved easily. This change adds the `ServiceRegistry` class.
This change introduces the `fixExpiredNFTokenOfferRemoval` amendment that allows expired offers to pass through `preclaim()` and be deleted in `doApply()`, following the same pattern used for expired credentials.
This change adds the project configuration directory to `.gitignore` for the `zed` editor.
As per the [documentation](https://zed.dev/docs/remote-development?highlight=.zed#zed-settings), the project configuration files are stored in the `.zed` directory at the project root dir.
This change cleans up the `API-CHANGELOG.md` file. It moves the version-specific documentation to other files and fleshes out the changelog with all the API-related changes in each version.
When support was added for `xrpld.cfg` in addition to `rippled.cfg` in https://github.com/XRPLF/rippled/pull/6098, as part of an effort to rename occurrences of ripple(d) to xrpl(d), the clearing and creation of the data directory were modified for what, at the time, seemed to result in an equivalent code flow. This has turned out to not be true, which is why this change restores two modifications to `Config.cpp` that currently break running the binary in standalone mode.
This change adds `cmake-format` as. a pre-commit hook. The style file closely matches that in Clio, and they will be made to be equivalent over time. For now, some files have been excluded, as those need some manual adjustments, which will be done in future changes.
This change makes the `read` function call in `handleConnection` async, adds a new class `TestSink` to help debugging, and adds a new target `xrpl.tests.helpers` to put the helper class in.
To allow developers to consume the latest unstable and (near-)stable versions of our `xrpl` Conan recipe, we should export and upload it whenever a push occurs to the corresponding branch or a release tag has been created. This way, developers do not have to figure out themselves what the most recent shortened commit hash was to determine the latest unstable recipe version (e.g. `3.2.0-b0+a1b2c3d`) or what the most recent release (candidate) was to determine the latest (near-)stable recipe version (e.g. `3.1.0-rc2`).
Now, pushes to the `develop` branch will produce the `develop` recipe version, pushes to the `release` branch will produce the `rc` recipe version, and creation of versioned tags will produce the `release` recipe version.
This change replaces the mutex `stoppingMutex_`, the `atomic_bool` variable `isTimeToStop`, and the conditional variable `stoppingCondition_` with an `atomic_flag` variable.
When `xrpld` is running the embedded tests as a child process, it has a control thread (the app bundle thread) that starts the application, and an application thread (the thread that executes `app_->run()`). Due to the relaxed memory ordering on ARM, it's not guaranteed that the application thread can see the change of the value resulting from the `isTimeToStop.exchange(true)` call before it is notified by `stoppingCondition_.notify_all()`, even though they do happen in the right order in the app bundle thread in `ApplicationImp::signalStop`. We therefore often get into the situation where `isTimeToStop` is `true`, but the application thread is waiting for `stoppingCondition_` to notify, because the app bundle thread may have already notified before the application thread actually starts waiting.
Switching to a single `atomic_flag` variable makes sure that there's only one synchronisation object and then the memory order guarantee provided by c++ can make sure that `notify_all` gets synchronised after `test_and_set` does.
Fixing this issue will stop the unit tests hanging forever and then we should see less (or hopefully no) time out errors in daily github action runs
Since the minimum Clang version we support is 16, the checks for version < 15 are no longer necessary. This change therefore removes the macros checking if the clang version is < 15 and simplifies uses of `std::source_location`.
The `upload-conan-deps` workflow that's triggered on push is supposed to upload the Conan dependencies to our remote, so future PR commits can pull those dependencies from the remote. However, as the `sanitize` argument is missing, it was building different dependencies than what the PRs are building for the asan/tsan/ubsan job, so the latter would not find anything in the remote that they could use. This change sets the missing `sanitizers` input variable when running the `build-deps` action.
Separately, the `setup-conan` action showed the default profile, while we are using the `ci` profile. To ensure the profile is correctly printed when sanitizers are enabled, the environment variable the profile uses is set before calling the action.
The export and upload steps were initially in a separate action, where GitHub Actions does not support the `secrets` keyword, but only `inputs` for the credentials. After they were moved to a reusable workflow, only part of the references to the credentials were updated. This change correctly references to the Conan credentials via `secrets` instead of `inputs`.
By default the Conan recipe extracts the version from `BuildInfo.cpp`, but in some of the cases we want to upload a recipe with a suffix derived from the commit hash. This currently then results in the uploading to fail, since there is a version mismatch.
Here we explicitly set the version, and then simplify the steps in the upload workflow since we now need the recipe name (embedded within the conanfile.py but also needed when uploading), the recipe version, and the recipe ref (name/version).
Conan recipes use semantic versioning, and since our version already contains a hyphen the second hyphen causes Conan to ignore it. The plus sign is a valid separator we can use instead, so this change uses a `+` to separate a version suffix (commit hash) instead of a `-`.
There were a few uninitialized variables in CMake files. This change will make sure we always check if a variable has been initialized before using them, or in come cases initialize them by default. This change will raise an error on CI if a developer introduced an uninitialized variable in CMake files.
This change continues the thread naming work from #5691 and #5758, which enables more useful lock contention profiling by ensuring threads/jobs have short, stable, human-readable names (rather than being truncated/failing due to OS limits). This changes diagnostic naming only (thread names and job/load-event labels), not behavior.
Specific modifications are:
* Shortens all thread/job names used with `beast::setCurrentThreadName`, so the effective Linux thread name stays within the 15-character limit.
* Removes per-ledger sequence numbers from job/thread names to avoid long labels. This improves aggregation in lock contention profiling for short-lived job executions.
The Ripple Bug Bounty program recently changed the public keys that security researchers can use to encrypt vulnerabilities and messages for submission to the program. This information was updated on https://ripple.com/legal/bug-bounty/ and this PR updates the `SECURITY.md` to align.
During several iterations of development of https://github.com/XRPLF/rippled/pull/6235, the commit hash was supposed to be moved into the `run:` statement, but it slipped through the cracks and did not get added. This change adds the commit hash as suffix to the Conan recipe version.
The `Number.h` header file now has `std::reference_wrapper` from `<functional>`, but the include is missing, causing downstream build problems. This change adds the header.
This change uploads the `libxrpl` library as a Conan recipe to our remote when (i) merging into the `develop` branch, (ii) committing to a PR that targets a `release*` branch, and (iii) a versioned tag is applied. Clio is only notified in the second case. The user and channel are no longer used when uploading the recipe.
Specific changes are:
* A `generate-version` action is added, which extracts the build version from `BuildInfo.cpp` and appends the short 7-character commit hash to it for merges into the `develop` branch and for commits to a PR that targets a `release*` branch. When a tag is applied, however, the tag itself is used as the version. This functionality has been turned into a separate action as we will use the same versioning logic for creating .rpm and .deb packages, as well as Docker images.
* An `upload-recipe` action is added, which calls the `generate-version` action and further handles the uploading of the recipe to Conan.
* This action is called by both the `on-pr` and `on-trigger` workflows, and a new `on-tag` workflow.
The reason for this change is that we have downstream uses for the `libxrpl` library, but currently only upload the recipe to check for compatibility with Clio when making commits to a PR that targets the release branch.
`PeerImp` processes `TMGetObjectByHash` queries with an unbounded per-request loop, which performs a `NodeStore` fetch and then appends retrieved data to the reply for each queried object without a local count cap or reply-byte budget. However, the `Nodestore` fetches are expensive when high in numbers, which might slow down the process overall. Hence this code change adds an upper cap on the response size.
This change removes the `master` branch as a trigger for the CI pipelines, and updates comments accordingly. It also fixes the pre-commit workflow, so it will run on all release branches.
These "fixed location" objects can be found in multiple ways:
1. The lookup parameters use the same format as other ledger objects, but the only valid value is true or the valid index of the object:
- Amendments: "amendments" : true
- FeeSettings: "fee" : true
- NegativeUNL: "nunl" : true
- LedgerHashes: "hashes" : true (For the "short" list. See below.)
2. With RPC API >= 3, using special case values to "index", such as "index" : "amendments". Uses the same names as above. Note that for "hashes", this option will only return the recent ledger hashes / "short" skip list.
3. LedgerHashes has two types: "short", which stores recent ledger hashes, and "long", which stores the flag ledger hashes for a particular ledger range.
- To find a "long" LedgerHashes object, request '"hashes" : <ledger sequence>'. <ledger sequence> must be a number that evaluates to an unsigned integer.
- To find the "short" LedgerHashes object, request "hashes": true as with the other fixed objects.
The following queries are all functionally equivalent:
- "amendments" : true
- "index" : "amendments" (API >=3 only)
- "amendments" : "7DB0788C020F02780A673DC74757F23823FA3014C1866E72CC4CD8B226CD6EF4"
- "index" : "7DB0788C020F02780A673DC74757F23823FA3014C1866E72CC4CD8B226CD6EF4"
Finally, whether the object is found or not, if a valid index is computed, that index will be returned. This can be used to confirm the query was valid, or to save the index for future use.
This change adds support for sanitizer build options in CI builds workflow. Currently `asan+ubsan` is enabled, while `tsan+ubsan` is left disabled as more changes are required.
The word `failed` in the test case makes it hard to search through the test logs when an actual test failure occurs, so this change renames the word to just `fail` instead.
- Refactor Number internals away from int64 to uint64 & a sign flag
- ctors and accessors use `rep`. Very few things expose
`internalrep`.
- An exception is "unchecked" and the new "normalized", which explicitly
take an internalrep. But with those special control flags, it's easier
to distinguish and control when they are used.
- For now, skip the larger mantissas in AMM transactions and tests
- Remove trailing zeros from scientific notation Number strings
- Update tests. This has the happy side effect of making some of the string
representations _more_ consistent between the small and large
mantissa ranges.
- Add semi-automatic rounding of STNumbers based on Asset types
- Create a new SField metadata enum, sMD_NeedsAsset, which indicates
the field should be associated with an Asset so it can be rounded.
- Add a new STTakesAsset intermediate class to handle the Asset
association to a derived ST class. Currently only used in STNumber,
but could be used by other types in the future.
- Add "associateAsset" which takes an SLE and an Asset, finds the
sMD_NeedsAsset fields, and associates the Asset to them. In the case
of STNumber, that both stores the Asset, and rounds the value
immediately.
- Transactors only need to add a call to associateAsset _after_ all of
the STNumbers have been set. Unfortunately, the inner workings of
STObject do not do the association correctly with uninitialized
fields.
- When serializing an STNumber that has an Asset, round it before
serializing.
- Add an override of roundToAsset, which rounds a Number value in place
to an Asset, but without any additional scale.
- Update and fix a bunch of Loan-related tests to accommodate the
expanded Number class.
---------
Co-authored-by: Vito <5780819+Tapanito@users.noreply.github.com>
- Spec: XLS-66
Fix overpayment asserts (#6084)
MPTTester::operator() parameter should be std::int64_t
- Originally defined as uint64_t, but the testIssuerLoan() test called
it with a negative number, causing an overflow to a very large number
that in some circumstances could be silently cast back to an int64_t,
but might not be. I believe this is UB, and we don't want to rely on
that.
Review feedback from @Tapanito: overpayment value change
- In overpayment results, the management fee was being calculated twice:
once as part of the value change, and as part of the fees paid.
Exclude it from the value change.
Fix Overpayment Calculation (#6087)
- Adds additional unit tests to cover math calculations.
- Removes unused methods.
Review feedback from @shawnxie999: even more rounding
- Round the initial total value computation upward, unless there is
0-interest.
- Rename getVaultScale to getAssetsTotalScale, and convert one incorrect
computation to use it.
- Use adjustImpreciseNumber for LossUnrealized.
- Add some logging to computeLoanProperties.
Fix LoanBrokerSet debtMaximum limits (#6116)
Fix some minor bugs in Lending Protocol (#6101)
- add nodiscard to unimpairLoan, and check result in LoanPay
- add a check to verify that issuer exists
- improve LoanManage error code for dust amounts
Check permissions in LoanSet and LoanPay (#6108)
Disallow pseudo accounts to be Destination for LoanBrokerCoverWithdraw (#6106)
Ensure vault asset cap is not exceeded (#6124)
Fix Overpayment ValueChange calculation in Lending Protocol (#6114)
- Adds loan state to LoanProperties.
- Cleans up computeLoanProperties.
- Fixes missing management fee from overpayment.
fix: Enable LP Deposits when the broker is the asset issuer (#6119)
* Replace accountHolds with accountSpendable when checking
for account funds in VaultDeposit and LoanBrokerCoverDeposit
Add a few minor changes (#6158)
- Updates or fixes a couple of things I noticed while reviewing changes
to the spec.
- Rename sfPreviousPaymentDate to sfPreviousPaymentDueDate.
- Make the vault asset cap check added in #6124 a little more robust:
1. Check in preflight if the vault is _already_ over the limit.
2. Prevent overflow when checking with the loan value. (Subtract
instead of adding, in case the values are near maxint. Both return
the same result. Also add a unit test so each case is covered.
Add minimum grace period validation (#6133)
Fix bugs: frozen pseudo-account, and FLC cutoff (#6170)
refactor: Rename raw state to theoretical state (#6187)
Check if a withdrawal amount exceeds any applicable receiving limit. (#6117)
Fix overpayment result calculation (#6195)
Address review feedback from Lending Protocol re-review (#6161)
---------
Co-authored-by: Gregory Tsipenyuk <gregtatcam@users.noreply.github.com>
Co-authored-by: Bronek Kozicki <brok@incorrekt.com>
Co-authored-by: Vito Tumas <5780819+Tapanito@users.noreply.github.com>
Co-authored-by: Shawn Xie <35279399+shawnxie999@users.noreply.github.com>
Co-authored-by: Jingchen <a1q123456@users.noreply.github.com>
This change removes unnecessary version numbers in the OpenSSL and Boost `find_package` CMake statements. An unnecessary OpenSSL definition is removed, while Conan options for SSL are updated to disable insecure ciphers. Moreover, the statements are now ordered alphabetically and more logically.
- Introduces amendment `fixBatchInnerSigs`
- Update Batch unit tests
- Fix all the Env instantiations to _use_ the "features" parameter.
- testInnerSubmitRPC runs with Batch enabled and disabled.
- Add a test to testInnerSubmitRPC for a correctly signed tx incorrectly
using the tfInnerBatchTxn flag.
- Generalize the submitAndValidate lambda in testInnerSubmitRPC.
- With the fix amendment, a transaction never reaches the transaction
engine (Transactor and derived classes.)
- Test submitting a pseudo-transaction. Stopped before reaching the
transaction engine, but with different errors.
- The tests verify that without the amendment, a transaction with
tfInnerBatchTxn is immediately rejected. Without the amendment, things
are safe. The amendment just makes things safer and more future-proof.
- Adds a mechanism for the vault owner to burn user shares when the vault is stuck. If the Vault has 0 AssetsAvailable and Total, the owner may submit a VaultClawback to reclaim the worthless fees, and thus allow the Vault to be deleted. The Amount must be left off (unless the owner is the asset issuer), specified as 0 Shares, or specified as the number of Shares held.
This change:
* Truncates thread names if more than 15 chars with `snprintf`.
* Adds warnings for truncated thread names if `-DTRUNCATED_THREAD_NAME_LOGS=ON`.
* Add a static assert for string literals to stop compiling if > 15 chars.
* Shortens `Resource::Manager` to `Resource::Mngr` to fix the static assert failure.
* Updates `CurrentThreadName_test` unit test specifically for Linux to verify truncation.
This change updates the XRPLF pre-commit workflow and prepare-runner action to their latest versions. For naming consistency the prepare-runner action changed the disable_ccache variable into enable_ccache, which matches our naming.
This change fixes the last of the spelling issues, and enables the pre-commit (and CI) check for spelling. There are no functionality changes, but it does rename some enum values.
Right now, each pipeline invocation builds the source code from scratch. Although compiled Conan dependencies are cached in a remote server, the source build objects are not. We are able to further speed up our builds by leveraging `ccache`. This change enables caching of build objects using `ccache` on Linux, macOS, and Windows.
This change adds some basic tests for all the `ledger_entry` helper functions, so each ledger entry type is covered. There are further some minor refactors in `parseAMM` to provide better error messages. Finally, to improve readability, alphabetization was applied in the helper functions.
This change renames all occurrences of `rippled.cfg` to `xrpld.cfg`. It also provides a script to allow developers to replicate the changes in their local branch or fork to avoid conflicts. For the time being it maintains support for `rippled.cfg` as config file, if `xrpld.cfg` does not exist.
This change modifies the build directory structure from `build/build/xxx` or `.build/build/xxx` to just `build/xxx`. Namely, the `conanfile.py` has the CMake generators build directory hardcoded to `build/generators`. We may as well leverage the top-level build directory without introducing another layer of directory nesting.
`Json::Object` and related objects are not used at all, so this change removes `include/xrpl/json/Object.h` and all downstream files. There are a number of minor downstream changes as well.
Full list of deleted classes and functions:
* `Json::Collections`
* `Json::Object`
* `Json::Array`
* `Json::WriterObject`
* `Json::setArray`
* `Json::addObject`
* `Json::appendArray`
* `Json::appendObject`
The last helper function, `copyFrom`, seemed a bit more complex and was actually used in a few places, so it was moved to `LedgerToJson.h` instead of deleting it.
The latest update to `cleanup-workspace`, `get-nproc`, and `prepare-runner` moved the action to the repository root directory, and also includes some ccache changes. In response, this change updates the various shared actions to the latest commit hash.
This change renames all occurrences of `namespace ripple` and `ripple::` to `namespace xrpl` and `xrpl::`, respectively, as well as the names of test suites. It also provides a script to allow developers to replicate the changes in their local branch or fork to avoid conflicts.
Per [XLS-0095](https://xls.xrpl.org/xls/XLS-0095-rename-rippled-to-xrpld.html), we are taking steps to rename ripple(d) to xrpl(d).
This change modifies the binary name from `rippled` to `xrpld`, and creates a symlink named `rippled` that points to the `xrpld` binary.
Note that https://github.com/XRPLF/rippled/pull/5975 renamed any references to `rippled` in the CMake files and their contents, but explicitly maintained the `rippled` binary name by adding an exception. This change now undoes this exception and adds an explicit symlink instead.
This change renames all the `info()` functions to `header()`, since they return `LedgerHeader` structs. It also renames the underlying variables from `info_` to `header_`.
This PR renames `LedgerInfo` to `LedgerHeader`. Namely, `LedgerInfo` was already an alias for `LedgerHeader`, and the comments next to the alias suggested that it would make sense to rename it, since that makes it clearer what it is.
This PR cleans up `RPCHelpers.h` and `RPCHelpers.cpp`. It splits out all the fetch-ledger functions to a new set of files, `RPCLedgerHelpers.h`/`RPCLedgerHelpers.cpp`, and moves the general-API functions to `ApiVersion.h`. There is no functionality change.
The .gitignore and .gitattributes files contain references to files and directories that the current build no longer produces, so this change removes obsolete entries in these files, and does some general reorganizing of the remaining entries.
This change updates the secp256k1 recipe that defines the SECP256K1_STATIC, so it no longer needs to be defined in the code here. Running the Conan update script also updated two other recipes in the lock file.
This PR updates protobuf and grpc to their latest versions. The latest protobuf version no longer requires patches, so we can use it directly from the official Conan Center Index, while the latest grpc still needed a patch, which was added to our own Conan Center Index fork in XRPLF/conan-center-index#8.
This change substitutes the secp256k1 source code copy by the Conan recipe added in XRPLF/conan-center-index#24, which updates the version of the library to 0.7.0.
- Spec: XLS-66
- Introduces amendment "LendingProtocol", but leaves it UNSUPPORTED to
allow for standalone testing, future development work, and potential
bug fixes.
- AccountInfo RPC will indicate the type of pseudo-account when
appropriate.
- Refactors and improves several existing classes and functional areas,
including Number, STAmount, STObject, json_value, Asset, directory
handling, View helper functions, and unit test helpers.
This change moves the lockfile instructions into a script, and instead of removing packages it sets the Conan home directory to a temporary directory.
There are several advantages, such as:
* Not affecting the user's Conan home directory, so there is no need to remove packages.
* Only the script needs to be run, rather than several commands.
This change updates the instructions for how to generate a Conan lockfile that is compatible with Linux, macOS, and Windows.
Whenever Conan dependencies change, the Conan lock file needs to be regenerated. However, different OSes have slightly different requirements, and thus require slightly different dependencies. Luckily a single `conan.lock` file can be used, as long as the union of dependencies is included.
The current instructions in the `BUILD.md` file are insufficient to regenerate the lock file such that it is compatible with all OSes, which this change addresses. The three profiles contain the bare minimum needed to generate the lockfile; it does not particularly matter whether the build type is Debug or Release, for instance.
This change triggers the Clio pipeline on PRs that target any of the `release*` branches (in addition to the `master` branch), as opposed to only the `release` branch.
If the config disables SQL db usage, such as a validator:
```
[ledger_tx_tables]
use_tx_tables = 0
```
then the pointer to DB engine is null, but it was still resolved during startup. Although it didn't crash in Release mode, possibly due to the compiler optimizing it away, it did crash in Debug mode. This change explicitly checks for the validity of the pointer and generates a runtime error if not set.
The `ledger_entry` and `deposit_preauth` requests require an array of credentials. However, the array size is not checked before is gets processing. This fix adds checks and return errors in case array size is too big.
This PR splits `RPCHelpers.h` into two files, by moving out all the ledger-fetching-related functions into a separate file, `RPCLedgerHelpers.h`. It also moves `getAccountObjects` to `AccountObjects.h`, since it is only used in that one place.
Technically b0 is not a release, so no "release" prefix here. It marks the point at which we moved the preceding release (3.0.0 in this case) from Beta to Release Candidate.
This change fixes floating point errors in conversion of shares to assets and other way, used in `VaultDeposit`, `VaultWithdraw` and `VaultClawback`. In the floating point calculations the division introduces a larger error than multiplication. If we do division first, then the error introduced will be increased by the multiplication that follows, which is therefore the wrong order to perform these two operations. This change flips the order of arithmetic operations, which minimizes the error.
Rather than having a single `XRPL_RETIRE` macro that applies to both feature and fix amendments, this change replaces it by new `XRPL_RETIRE_FIX` and `XRPL_RETIRE_FEATURE` macros that avoids confusion between whether to prefix the amendment name with `feature` or `fix`.
This updates the CI image hashes after following change: https://github.com/XRPLF/ci/pull/81. And, since we use latest Conan, we can have `conan.lock` with a newline at the end, and we don't need to exclude it from `pre-commit` hooks any longer.
This change fixes JSON parsing of negative `int` input in `STNumber` and `STAmount`. The conversion of JSON to `STNumber` or `STAmount` may trigger a condition where we negate smallest possible `int` value, which is undefined behaviour. We use a temporary storage as `int64_t` to avoid this bug. Note that this only affects RPC, because we do not parse JSON in the protocol layer, and hence no amendment is needed.
This change removes unused definitions from the CMake files, moves variable definitions from `XrplSanity` to `XrplSettings` where they better belong, and updates the minimum GCC and Clang versions to match what we actually minimally support.
This change unifies the build and test jobs into a single job, and adds `ctest` to coverage reporting.
The mechanics of coverage reporting is slightly complex and most of it is encapsulated in the `coverage` target. The status quo way of preparing coverage reports involves running a single target `cmake --build . --target coverage`, which does three things:
* Build the `rippled` binary (via target dependency)
* Prepare coverage reports:
* Run `./rippled -u` unit tests.
* Gather test output and build reports.
This makes it awkward to add an additional `ctest` step between build and coverage reporting steps. The better solution is to split `coverage` target into separate build, followed by `ctest`, followed by test generation. Luckily, the `coverage` target has been designed specifically to support such case; it does not need to build `rippled`, it's just a dependency. Similarly it allows additional tests to be run before gathering test outputs; in principle we could even strip it from running tests and run them separately instead. This means we can keep build, `ctest` and generation of coverage reports as separate steps, as long as the state of build directory is fully (including file timestamps, additional coverage files etc.) preserved between the steps. This means that in order to run `ctest` for coverage reporting we need to integrate build and test into a single job, which this change does.
As part of renaming ripple(d) to xrpl(d), the xrpld symlink was made to point to itself instead of to the rippled binary. This change fixes the symlink.
Per XLS-0095, we are taking steps to rename ripple(d) to xrpl(d).
This change updates the CMake files and definitions therein, plus a handful of related modifications. Specifically, the compiler files are renamed from `RippleXXX.cmake` or `RippledXXX.cmake` to `XrplXXX.cmake`, and any references to `ripple` and `rippled` (with or without capital letters) are renamed to `xrpl` and `xrpld`, respectively. The name of the binary, currently `rippled`, remains unchanged and will be updated in a separate PR. This change is purely cosmetic and does not affect the functioning of the binary.
Per XLS-0095, we are taking steps to rename ripple(d) to xrpl(d).
This change specifically removes all copyright notices referencing Ripple, XRPLF, and certain affiliated contributors upon mutual agreement, so the notice in the LICENSE.md file applies throughout. Copyright notices referencing external contributions remain as-is. Duplicate verbiage is also removed.
Per XLS-0095, we are taking steps to rename ripple(d) to xrpl(d).
C++ include guards are used to prevent the contents of a header file from being included multiple times in a single compilation unit. This change renames all `RIPPLE_` and `RIPPLED_` definitions, primarily include guards, to `XRPL_`. It also provides a script to allow developers to replicate the changes in their local branch or fork to avoid conflicts.
Amendments activated for more than 2 years can be retired, and obsolete retirements that were never activated can also be removed after 2 years. This change retires the NonFungibleTokensV1_1, fixNonFungibleTokensV1_2, and fixNFTokenRemint amendments, and removes the NonFungibleTokensV1, fixNFTokenNegOffer, and fixNFTokenDirV1 amendments.
To debug test failures we would like to use `netstat`, but that package wasn't installed yet in the CI images. This change uses the new CI images created by https://github.com/XRPLF/ci/pull/79.
This change:
* Simplifies the `TxMeta` constructors - both were setting the same set of fields, and to make it harder for future bugs to arise and keep the code DRY, we can combine those into one helper function.
* Removes an unused constructor.
* Renames the variables to avoid Hungarian naming.
* Removes a bunch of now-unnecessary helper functions.
This change introduces the `featurePermissionDelegationV1_1` amendment, which is designed to supersede both `featurePermissionDelegation` and `fixDelegateV1_1 amendments, which should be considered deprecated. The `checkPermission` function will now return `terNO_DELEGATE_PERMISSION` when a delegate transaction lacks the necessary permissions.
This change introduces the `fixDirectoryLimit` amendment to remove the directory pages limit. We found that the directory size limit is easier to hit than originally assumed, and there is no good reason to keep this limit, since the object reserve provides the necessary incentive to avoid creating unnecessary objects on the ledger.
Field `sfSubjectNode` is not populated by `CredentialCreate` in self-issued credentials. Rather than fixup the Credentials already on the ledger, we can in this case safely change the object template for this field from `soeREQUIRED` to `soeOPTIONAL`.
- Restructures `STTx` signature checking code to be able to handle
a `sigObject`, which may be the full transaction, or may be an object
field containing a separate signature. Either way, the `sigObject` can
be a single- or multi-sign signature.
- This is distinct from 550f90a75e (#5594), which changed the check in
Transactor, which validates whether a given account is allowed to sign
for the given transaction. This cryptographically checks the signature
validity.
This change adds an extra step to the CI test job that outputs network info, which may allow us to confirm whether random test failures are caused by port exhaustion.
We are on an amendment retiring spree, but each change results in conflicts in `features.macro` because currently they all add the retired amendment to the end of the list. By sorting the list the number of conflicts should be reduced, making it easier to merge them.
Amendments activated for more than 2 years can be retired. This change retires the fix1571 amendment.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
To protect the identity of UNL validators, the IP addresses are redacted from the log messages sent to the common Grafana instance. However, without such identifying information it is challenging to debug issues. This change adds a node's public key to logs to improve our ability to debug issues.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
* Retired fix1781 amendment
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
* refactor: Retire fix1781 amendment
Amendments activated for more than 2 years can be retired. This change retires the fix1781 amendment.
---------
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
This change reduces the number of cores used to build and test, as using all cores may be contributing to occasional build and test failures.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
This change changes the CI concurrency group for pushes to the `develop` branch to use the commit hash instead of the target branch.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
There are separate steps for logging into Conan and uploading packages. However, at the moment sometimes the login step is executed even though no packages will be uploaded. The condition for performing both steps should be the same.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
This changes fixes an invariant error where the amount withdrawn is equal to the transaction fee.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
This change fixes the `account_tx` RPC method to properly validate malformed limit parameter values. Previously, invalid values like `0`, `1.2`, `"10"`, `true`, `false`, `-1`, `[]`, `{}`, etc. were either accepted without errors or caused internal errors. Now all malformed values correctly return the `invalidParams` error.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
Amendments activated for more than 2 years can be retired. This change retires the fix1543 amendment.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
The `-Wno-missing-template-arg-list-after-template-kw` flag is only needed for the grpc library. Use `+=` for the default build flags to make it easier to extend in the future.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
Amendments activated for more than 2 years can be retired. This change retires the fix1515 amendment.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
As XRPL network demand grows and ledger sizes increase, the default 4K NuDB block size becomes a performance bottleneck, especially on high-performance storage systems. Modern SSDs and enterprise storage often perform better with larger block sizes, but rippled previously had no way to configure this parameter. This change therefore implements configurable NuDB block size support, allowing operators to optimize storage performance based on their hardware configuration. The feature adds a new `nudb_block_size` configuration parameter that enables block sizes from 4K to 32K bytes, with comprehensive validation and backward compatibility.
Specific changes are:
- Implements `parseBlockSize()` function with validation.
- Adds `nudb_block_size` configuration parameter.
- Supports block sizes from 4K to 32K (power of 2).
- Adds comprehensive logging and error handling.
- Maintains backward compatibility with 4K default.
- Adds unit tests for block size validation.
- Updates configuration documentation with performance guidance.
- Marks feature as experimental.
- Applies code formatting fixes.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
Similarly to other transaction typed that can create a trust line or MPToken for the transaction submitter (e.g. CashCheck #5285, EscrowFinish #5185 ), VaultWithdraw should enforce reserve before creating a new object. Additionally, the lsfRequireDestTag account flag should be enforced for the transaction submitter.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
The default job timeout is 5 hours, while build times are anywhere between 4-20 mins and test times between 2-10. As a runner occasionally gets stuck, we should fail much quicker.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
This PR sets the fail-fast strategy option to false (it defaults to true), unless it is run by a merge group.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
This change sanitizes inputs by setting them as environment variables, and adjusts the number of CPUs used for building. Namely, GitHub inputs should be sanitized, per recommendation by Semgrep, as using them directly poses a security risk. A recent change further overrode the global configuration by having builds use all cores, but as we have noticed an increased number of job cancelation this change updates it to use all cores less one.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
There are some tests that write out JSONs as a string instead of using the Json::Value library, which are cleaned up by this change.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
A regression was introduced in #5669 which would cause rippled to potentially dereference a disengaged std::optional when connecting to a peer. This would cause UB in release build and crash in debug.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
This change replaces boost::lexical_cast<std::string> with to_string in some of the tests to make them more readable.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
This change replaces instances of JSON LastLedgerSequence with last_ledger_seq, which makes the tests a bit simpler and easier to read.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
Windows is extremely chatty and generates tons of logs when building, making it practically impossible to use the build logs to debug issues. This change sets the verbosity to 'quiet' on Windows.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
This PR changes fee().accountReserve(0) to fee().reserve, as the current network reserve amount should be used instead of the account reserve.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
This change updates the Docker image hashes of the tools-rippled images to fix a missing dependency.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
This change adds a paychan namespace to the TestHelpers and implementation files, improving organization and clarity. Additionally, it updates the AMM test to use the new `paychan::create` function for payment channel creation.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
This change excludes from Codecov unreachable/difficult-to-test transaction code (such as `tecINTERNAL`) and old code (from amendments that have been enabled for a long time that are only around for ledger replay reasons). This removes about 200 lines of misses and increases the Codecov coverage by 0.3% (79.2% to 79.5%).
This change adds a wildcard to the release branch in the CI pipeline spec. Namely, after adopting an improved release process, with release branches that now look like release-X.Y, the trigger pipeline was no longer running as it only searched for an exact match to release.
This change downgrades OpenSSL 3.6.0 to 3.5.4. To avoid potential zero-day issues in a new major version of OpenSSL, 3.6.0, it is safer to stick with 3.5.4. While 3.6.0 has some nice new features, such as improved SHA512 hashing, it also introduces new features that could contain bugs. In contrast, 3.5.4 has seen quite a few bug fixes over 3.5.0 and has been used in the wild for a while now.
This change fixes a release build error with GCC 15.2.
The `fields` variable is only used in `XRPL_ASSERT`, which evaluates to nothing in a Release build, leaving the variable unused. This change silences the build warning.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
This change uses the new RHEL 9 and 10 images to build and test the binary, and adds support for having different Docker image SHAs per distro-compiler combination.
Instead of supporting RHEL each minor version, we are simplifying our pipelines by only supporting RHEL major versions. Our CI Docker images have already been updated accordingly, and we recently added support for RHEL 10 as well. Up until now, the CI Docker images had all been rebuilt at the same time, but that is not necessarily true as the most recent push to the CI repo has shown where the RHEL images now have a different SHA than the Debian and Ubuntu ones.
Co-authored-by: Bart Thomee <11445373+bthomee@users.noreply.github.com>
* Restructures Transactor signature checking code to be able to handle a `sigObject`, which may be the full transaction, or may be an object field containing a separate signature. Either way, the `sigObject` can be a single- or multi-sign signature.
* Restructures `Transactor::preflight` to create several functions that will remove the need for error-prone boilerplate code in derived classes' implementations of `preflight`.
This change adds `STInt32` as a new `SType` under the `STInteger` umbrella, with `SType` value `12`. This is the first and only `STInteger` type that supports negative values.
This change adds more comprehensive tests for the `FeeVote` module, which previously only checked the basics, and not the more comprehensive flows in that class.
This change makes the regex in `HttpClient.cpp` that matches the content-length http header case insensitive to improve compatibility, as http headers are case insensitive.
This change raises logging severity from `INFO` to `WARN` when handling UNL manifest signed with an unexpected / invalid key. It also changes the internal error code for an invalid format of UNL manifest to `invalid` (from `untrusted`).
This is a follow up to problems experienced by an UNL node due to old manifest key configured in `validators.txt`, which would be easier to diagnose with improved logging.
It also replaces a log line with `UNREACHABLE` for an impossible situation when we match UNL manifest key against a configured key which has an invalid type (we cannot configure such a key because of checks when loading configured keys).
This adds a comment to avoid using `std::counting_semaphore` until the minimum compiler versions of GCC and Clang have been updated to no longer contain the bug that is present in older compilers.
This change reverts #5617, because it will require extensive testing that will take up more time than we have before the next scheduled release.
Reverting this change does not mean we are abandoning it. We aim to pick it back up once there's a sufficient time window to allow for testing on multiple distros running a mixture of OpenSSL 1.x and 3.x.
This change is to improve code coverage (and to simplify #5720 and #5725); there is otherwise no change in functionality. The change adds basic tests for `STInteger` and `STParsedJSON`, so it becomes easier to test smaller changes to the types, as well as removes `STParsedJSONArray`, since it is not used anywhere (including in Clio).
- Added a new Invariant: `ValidPseudoAccounts` which checks that all pseudo-accounts behave consistently through creation and updates, and that no "real" accounts look like pseudo-accounts (which means they don't have a 0 sequence).
- `to_short_string(base_uint)`. Like `to_string`, but only returns the first 8 characters. (Similar to how a git commit ID can be abbreviated.) Used as a wrapped sink to prefix most transaction-related messages. More can be added later.
- `XRPL_ASSERT_PARTS`. Convenience wrapper for `XRPL_ASSERT`, which takes the `function` and `description` as separate parameters.
- `SField::sMD_PseudoAccount`. Metadata option for `SField` definitions to indicate that the field, if set in an `AccountRoot` indicates that account is a pseudo-account. Removes the need for hard-coded field lists all over the place. Added the flag to `AMMID` and `VaultID`.
- Added functionality to `SField` ctor to detect both code and name collisions using asserts. And require all SFields to have a name
- Convenience type aliases `STLedgerEntry::const_pointer` and `STLedgerEntry::const_ref`. (`SLE` is an alias to `STLedgerEntry`.)
- Generalized `feeunit.h` (`TaggedFee`) into `unit.h` (`ValueUnit`) and added new "BIPS"-related tags for future use. Also refactored the type restrictions to use Concepts.
- Restructured `transactions.macro` to do two big things
1. Include the `#include` directives for transactor header files directly in the macro file. Removes the need to update `applySteps.cpp` and the resulting conflicts.
2. Added a `privileges` parameter to the `TRANSACTION` macro, which specifies some of the operations a transaction is allowed to do. These `privileges` are enforced by invariant checks. Again, removed the need to update scattered lists of transaction types in various checks.
- Unit tests:
1. Moved more helper functions into `TestHelpers.h` and `.cpp`.
2. Cleaned up the namespaces to prevent / mitigate random collisions and ambiguous symbols, particularly in unity builds.
3. Generalized `Env::balance` to add support for `MPTIssue` and `Asset`.
4. Added a set of helper classes to simplify `Env` transaction parameter classes: `JTxField`, `JTxFieldWrapper`, and a bunch of classes derived or aliased from it. For an example of how awesome it is, check the changes `src/test/jtx/escrow.h` for how much simpler the definitions are for `finish_time`, `cancel_time`, `condition`, and `fulfillment`.
5. Generalized several of the amount-related helper classes to understand `Asset`s.
6. `env.balance` for an MPT issuer will return a negative number (or 0) for consistency with IOUs.
This change re-enables building and testing all configurations, but only for the daily scheduled run. Previously all configurations were run for each merge into the develop branch, but that overwhelmed both the GitHub runners and the Conan remote, and thus they were limited to just a subset of configurations. Now that the number of jobs is limited via `max-parallel: 10`, we should be able to safely enable building all configurations again. However, building them all once a day instead of for each PR merge should be sufficient.
- Ensures the commits don't get orphaned, even though the relevant code
changes are already included.
* tag '2.5.1':
Set version to 2.5.1
Fix: Don't flag consensus as stalled prematurely (#5658)
GitHub runners have a limit on how many concurrent jobs they can actually process (even though they will try to run them all at the same time), and similarly the Conan remote cannot handle hundreds of concurrent requests. Previously, the Conan dependency uploading was already limited to max 10 jobs running in parallel, and this change makes the same change to the build+test workflow.
This change adds a fix amendment (`fixIncludeKeyletFields`) that adds:
* `sfSequence` to `Escrow` and `PayChannel`
* `sfOwner` to `SignerList`
* `sfOracleDocumentID` to `Oracle`
This ensures that all ledger entries hold all the information needed to determine their keylet.
The XRPL establishes connections in three stages: first a TCP connection, then a TLS/SSL handshake to secure the connection, and finally an upgrade to the bespoke XRP Ledger peer-to-peer protocol. During connection termination, xrpld directly closes the TCP connection, bypassing the TLS/SSL shutdown handshake. This makes peer disconnection diagnostics more difficult - abrupt TCP termination appears as if the peer crashed rather than disconnected gracefully.
This change refactors the connection lifecycle with the following changes:
- Enhanced outgoing connection logic with granular timeouts for each connection stage (TCP, TLS, XRPL handshake) to improve diagnostic capabilities
- Updated both PeerImp and ConnectAttempt to use proper asynchronous TLS shutdown procedures for graceful connection termination
* extends the functionality of the MPTokenIssuanceSet transaction, allowing the issuer to update fields or flags that were explicitly marked as mutable during creation.
Clio should only be notified when releases are about to be made, instead of for all PR, so this change only notifies Clio when a PR targets the release or master branch.
This change wraps all GitHub conditionals in `${{ .. }}`, both for consistency and to reduce unexpected failures, because it was previously noticed that not all conditionals work without those curly braces.
- Amendment: fixDelegateV1_1
- In DelegateSet, disallow invalid PermissionValues like 0, and transaction values when the transaction's amendment is not enabled. Acts as if the transaction doesn't exist, which is the same thing older versions without the amendment will do.
- Payment burn/mint should disallow DEX currency exchange.
- Support MPT for Payment burn/mint.
- Don't run upload-conan-deps in PRs, unless the PR changes the workflow file.
- Change cron schedule for uploading Conan dependencies to run after work hours for most dev.
- This should prevent Artifactory from being overloaded by too many requests at a time.
- Uses "max-parallel" to limit the build job to 10 simultaneous instances.
- Only run the minimal matrix on PRs.
For the purposes of being able to merge a PR, Github Actions jobs count as passed if they ran and passed, or were skipped.
With this change, if any of the jobs that "passed" depends on fail or are cancelled, then "passed" will fail. If they all succeed or are skipped, then "passed" is skipped, which does not prevent a merge.
This saves spinning up a runner in the usual case where things work, and will simplify our branch protection rules, so that only "passed" will need to be checked.
* Add and Scale to VaultCreate
* Add round-trip calculation to VaultDeposit VaultWithdraw and VaultClawback
* Implement Number::truncate() for VaultClawback
* Add rounding to DepositWithdraw
* Disallow zero shares withdraw or deposit with tecPRECISION_LOSS
* Return tecPATH_DRY on overflow when converting shares/assets
* Remove empty shares MPToken in clawback or withdraw (except for vault owner)
* Implicitly create shares MPToken for vault owner in VaultCreate
* Review feedback: defensive checks in shares/assets calculations
---------
Co-authored-by: Ed Hennis <ed@ripple.com>
Fix stalled consensus detection to prevent false positives in situations where there are no disputed transactions.
Stalled consensus detection was added to 2.5.0 in response to a network consensus halt that caused a round to run for over an hour. However, it has a flaw that makes it very easy to have false positives. Those false positives are usually mitigated by other checks that prevent them from having an effect, but there have been several instances of validators "running ahead" because there are circumstances where the other checks are "successful", allowing the stall state to be checked.
* chore: Use conan lockfile
* Add windows-specific dependencies as well
* Add more info about lockfiles
* Update lockfile to latest version
* Update BUILD.md with conan install note
This is a major refactor of LedgerEntry.cpp. It adds a number of helper functions to make the code easier to maintain.
It also splits up the ledger and ledger_entry tests into different files, and cleans up the ledger_entry tests to make them easier to write and maintain.
This refactor also caught a few bugs in some of the other RPC processing, so those are fixed along the way.
This is a follow-up to PR #5664 that further improves the specificity of logging for refused peer connections. The previous changes did not account for several key scenarios, leading to potentially misleading log messages.
It addresses the following
- Inbound Disabled: Connections are now explicitly logged as rejected when the server is not configured to accept inbound peers. Previously, this was logged as the server being "full," which was technically correct but lacked diagnostic clarity.
- Duplicate Connections: The logging now distinguishes between two types of duplicate connection refusals:
- When a peer with the same node public key is already connected (duplicate connection).
- When a connection is rejected because the limit for connections from a single IP address has been reached.
These changes provide more accurate and actionable diagnostic information when analyzing peer connection behavior.
Test jobs will run if
* Either the PR is non-draft or has the "DraftRunCI" label set *AND*
* One of the following:
* Certain files were changed *OR*
* The PR is non-draft and has the "Ready to merge" flag *OR*
* The workflow is being run from the merge queue.
Additionally, a meta "passed" job was added that is dependent on all the other test jobs, so the required jobs list under branch protection rules only needs to specify "passed" to ensure that *either* all the test jobs pass *or* all the test jobs are skipped because they don't need to be run.
This allows PRs that don't affect the build or binary to be merged without overriding.
This updates Boost to 1.88, which is needed because Clio wants to move to 1.88 as that fixes several ASAN false positives around coroutine usage. In order for Clio to move to newer boost, libXRPL needs to move too. Hence the changes in this PR. A lot has changed between 1.83 and 1.88 so there are lots of changes in the diff, especially in regards to Boost.Asio and coroutines in particular.
This change removes `labeled` and `unlabeled` as pipeline trigger actions, and instead adds `reopened` and `ready_for_review`. The logic whether to run the pipeline jobs is then simplified, although to get a draft PR with the `DraftCIRun` label to run it can be necessary to close and reopen a PR.
The change updates how clang-format is called in CI and locally, and adds prettier to the pre-commit hook. Proto files are now also formatted, while external files are excluded.
This change will skip running the notify-clio job when a PR is created from a fork, and reorders the strategy matrix configuration fields so GitHub will more clearly show which configuration is running.
This change reverts the formatting applied to external files and adds formatting of proto files.
As clang-format will complain if a proto file is modified or moved, since the .clang-format file does not explicitly contain a section for proto files, the change has been included in this PR as well.
This change updates OpenSSL from 1.1.1w to 3.5.2. The code works as-is, but many functions have been marked as deprecated and thus will need to be rewritten. For now we explicitly add the `-DOPENSSL_SUPPRESS_DEPRECATED` to give us time to do so, while providing us with the benefits of the updated version.
This change modifies the `build_only` check used to determine whether to run tests. For easier debugging in the future it also prints out the contents of the strategy matrix.
Reduce log noise by changing two log statements from error/warn level to debug level. These logs occur during normal operation when AMM offers are not available or when IOU authorization checks fail, which are expected scenarios that don't require an elevated log level.
Currently, all peer connection rejections are logged with the reason "slots full". This is inaccurate, as the PeerFinder can also reject connections if they are a duplicate. This change updates the logging logic to correctly report the specific reason (full or duplicate) for a rejected peer connection, providing more accurate diagnostic information.
This change fixes the suite names all around the test files, to make them match to the folder name in which this test files are located. Also, the RCL test files are relocated to the consensus folder, because they are testing consensus functionality.
This change replaces the configuration variable with the hardcoded `https://conan.ripplex.io`, making it possible for PRs from forks to use our Conan remote containing workarounds.
We're currently calling `XXH3_createState` and `XXH3_freeState` when hashing an object. However, it may be slow because they call `malloc` and `free`, which may affect the performance. This change avoids the use of the streaming API as much as possible by using an internal buffer.
Fix stalled consensus detection to prevent false positives in situations where there are no disputed transactions.
Stalled consensus detection was added to 2.5.0 in response to a network consensus halt that caused a round to run for over an hour. However, it has a flaw that makes it very easy to have false positives. Those false positives are usually mitigated by other checks that prevent them from having an effect, but there have been several instances of validators "running ahead" because there are circumstances where the other checks are "successful", allowing the stall state to be checked.
This change updates some incorrect Conan commands for Conan 2. As some flags do not exist in Conan 2, such as --settings build_type=[configuration], the commands have been adjusted accordingly. This change further uses the org-level variables and secrets rather than the repo-level ones.
This change introduces two key optimizations:
* Mutex scope reduction: Limits the lock to individual partitions within `TaggedCache`, reducing contention.
* Decoupling: Removes the tight coupling between `LedgerHistory` and `TaggedCache`, improving modularity and testability.
Lock contention analysis based on eBPF showed significant improvements as a result of this change.
This change uploads built Conan dependencies to the Conan remote upon merge into the develop branch.
At the moment, whenever Conan dependencies change, we need to remember to manually push them to our Conan remote, so they are cached for future reuse. If we forget to do so, these changed dependencies need to be rebuilt over and over again, which can take a long time.
This change:
* Removes the patched Conan recipes from the `external/` directory.
* Adds instructions for contributors how to obtain our patched recipes.
* Updates the Conan remote name and remote URL (the underlying package repository isn't changed).
* If the remote already exists, updates the URL instead of removing and re-adding.
* This is not done for the libXRPL job as it still uses Conan 1. This job will be switched to Conan 2 soon.
* Removes duplicate Conan remote CI pipeline steps.
* Overwrites the existing global.conf on MacOS and Windows machines, as those do not run CI pipelines in isolation but all share the same Conan installation; appending the same config over and over bloats the file.
This change updates BUILD.md for Conan 2, add fixes/workarounds for Apple Clang 17, Clang 20 and CMake 4. This also removes (from BUILD.md only) workarounds for compiler versions which we no longer support e.g. Clang 15 and adds compilation flag -Wno-deprecated-declarations to enable building with Clang 20 on Linux.
This change fixes an issue where the order of `PriceDataSeries` was out of sync between when `PriceOracle` was created and when it was updated. Although they are registered in the canonical order when updated, they are created using the order specified in the transaction; this change ensures that they are also registered in the canonical order when created.
This change decouples `ledger` from `xrpld/app`, and therefore fully clears the path to the modularisation of the ledger component. Before this change, `View.cpp` relied on `MPTokenAuthorize::authorize; this change moves `MPTokenAuthorize::authorize` to `View.cpp` to invert the dependency, making ledger a standalone module.
The Payment transaction metadata is missing the `DeliveredAmount` field that displays the actual amount delivered to the destination excluding transfer fees. This amendment fixes this problem.
#5224 added (among other things) a `VaultWithdraw` transaction that allows setting the recipient of the withdrawn funds in the `Destination` transaction field. This technically turns this transaction into a payment, and in some respect the implementation does follow payment rules, e.g. enforcement of `lsfRequireDestTag` or `lsfDepositAuth`, or that MPT transfer has destination `MPToken`. However for IOUs, it missed verification that the destination account has a trust line to the asset issuer. Since the default behavior of `accountSendIOU` is to create this trust line (if missing), this is what `VaultWithdraw` currently does. This is incorrect, since the `Destination` might not be interested in holding the asset in question; this basically enables spammy transfers. This change, therefore, removes automatic creation of a trust line to the `Destination` account in `VaultWithdraw`.
This change updates RocksDB to its latest version. RocksDB is backward-compatible, so even though this is a major version bump, databases created with previous versions will continue to function.
The external RocksDB folder is removed, as the latest version available via Conan Center no longer needs custom patches.
Before `XRPLF/ci` images, we did not have a `dependencies:` job for clang-16, so `instrumentation:` had to build its own dependencies. Now we have clang-16 Conan dependencies built in a separate job that can be used.
This change includes `network_id` data in the validations and ledger subscription stream responses, as well as unit tests to validate the response fields. Fixes#4783
This change adds support for `DomainID` to existing transactions `MPTokenIssuanceCreate` and `MPTokenIssuanceSet`.
In #5224 `DomainID` was added as an access control mechanism for `SingleAssetVault`. The actual implementation of this feature lies in `MPToken` and `MPTokenIssuance`, hence it makes sense to enable the use of `DomainID` also in `MPTokenIssuanceCreate` and `MPTokenIssuanceSet`, following same rules as in Vault:
* `MPTokenIssuanceCreate` and `MPTokenIssuanceSet` can only set `DomainID` if flag `MPTRequireAuth` is set.
* `MPTokenIssuanceCreate` requires that `DomainID` be a non-zero, uint256 number.
* `MPTokenIssuanceSet` allows `DomainID` to be zero (or empty) in which case it will remove `DomainID` from the `MPTokenIssuance` object.
The change is amendment-gated by `SingleAssetVault`. This is a non-breaking change because `SingleAssetVault` amendment is `Supported::no`, i.e. at this moment considered a work in progress, which cannot be enabled on the network.
For jobs running in containers, $GITHUB_WORKSPACE and ${{ github.workspace }} might not be the same directory. The actions/checkout step is supposed to checkout into `$GITHUB_WORKSPACE` and then add it to safe.directory (see instructions at https://github.com/actions/checkout), but that's apparently not happening for some container images. We can't be sure what is actually happening, so we preemptively add both directories to `safe.directory`. See also the GitHub issue opened in 2022 that still has not been resolved https://github.com/actions/runner/issues/2058.
The current implementation of rngfill is prone to false warnings from GCC about array bounds violations. Looking at the code, the implementation naively manipulates both the bytes count and the buffer pointer directly to ensure the trailing memcpy doesn't overrun the buffer. As expressed, there is a data dependency on both fields between loop iterations.
Now, ideally, an optimizing compiler would realize that these dependencies were unnecessary and end up restructuring its intermediate representation into a functionally equivalent form with them absent. However, the point at which this occurs may be disjoint from when warning analyses are performed, potentially rendering them more difficult to
determine precisely.
In addition, it may also consume a portion of the budget the optimizer has allocated to attempting to improve a translation unit's performance. Given this is a function template which requires context-sensitive instantiation, this code would be more prone than most to being inlined, with a decrease in optimization budget corresponding to the effort the optimizer has already expended, having already optimized one or more calling functions. Thus, the scope for impacting the the ultimate quality of the code generated is elevated.
For this change, we rearrange things so that the location and contents of each memcpy can be computed independently, relying on a simple loop iteration counter as the only changing input between iterations.
Remove `include(default)` from `conan/profiles/libxrpl`. This means that we will now rely on compiler workarounds stored elsewhere e.g. in global.conf.
This change reverts the usage of boost::shared_mutex back to std::shared_mutex. The change was originally introduced as a workaround for a bug in glibc 2.28 and older versions, which could cause threads using std::shared_mutex to stall. This issue primarily affected Ubuntu 18.04 and earlier distributions, which we no longer support.
This change fixes the MacOS pipeline issue by limiting GitHub to choose the existing runners, ensuring the new experimental runners are excluded until they are ready.
This issue was reported on the Javascript client library: XRPLF/xrpl.js#2611
The type filter (Note: as of the latest version of rippled, type parameter is deprecated) does not work as expected. This PR removes the type filter from the ledger command.
This PR updates several dependencies to their latest versions. Not all dependencies have been updated, as some need to be patched and some require additional code changes due to backward incompatibilities introduced by the version bump.
Due to rounding, the LPTokenBalance of the last LP might not match the LP's trustline balance. This was fixed for `AMMWithdraw` in `fixAMMv1_1` by adjusting the LPTokenBalance to be the same as the trustline balance. Since `AMMClawback` is also performing a withdrawal, we need to adjust LPTokenBalance as well in `AMMClawback.`
This change includes:
1. Refactored `verifyAndAdjustLPTokenBalance` function in `AMMUtils`, which both`AMMWithdraw` and `AMMClawback` call to adjust LPTokenBalance.
2. Added the unit test `testLastHolderLPTokenBalance` to test the scenario.
3. Modify the existing unit tests for `fixAMMClawbackRounding`.
Currently there is no easy way to track MPT related transactions for the issuer. This change allows MPT transactions to show up on issuer's AccountTx RPC (to align with how IOUs work).
* Update the `account_info` API so that the `allowTrustLineLocking` flag is included in the response.
* The proposed `TokenEscrow` amendment added an `allowTrustLineLocking` flag in the `AccountRoot` object.
* In the API response, under `account_flags`, there is now an `allowTrustLineLocking` field with a boolean (`true` or `false`) value.
* For reference, the XLS-85 Token-Enabled Escrows implementation can be found in https://github.com/XRPLF/rippled/pull/5185
The current version was copied from `antithesis-sdk-cpp` but there is no logical reason to require this specific version of CMake. This change downgrades the version to make the project build with older CMake versions.
Having `boost::boost` in `self.requires` makes clio link with all boost libraries. There are additionally several Boost stacktrace backends that are both linked with, which violate ODR.
This change fixes the problem.
This PR refactors `CredentialHelpers` and removes some unnecessary dependencies as a step of modularization.
The ledger component is almost independent except that it references `MPTokenAuthorize` and `CredentialHelpers.h`, and the latter further references `Transactor.h`. This PR partially clears the path to modularizing the ledger component and decouples `CredentialHelpers` from xrpld.
This PR fixes a crash in tests when the test `Env is run at trace/debug log level.
This issue only affects tests, and only if logging at trace/debug level, so really only relevant during rippled development, and does not affect production servers.
The tests that ensure `tfInnerBatchTxn` won't block delegated transactions silently fail in `Delegate_test.cpp`. This change removes these cases from that file and adds them to `Batch_test.cpp` instead where they do not silently fail, because there the batch delegate results are explicitly checked. Moving them to that file further avoids refactoring many helper functions.
This change allows users to submit simulate requests from a multi-sign account without needing to specify the accounts that are doing the multi-signing, and fixes an error with simulate that allowed double-"signed" (both single-sign and multi-sign public keys are provided) transactions.
Multi-line log messages are hard to work with. Writing these handful of related messages as one message should make the log a tiny bit easier to manage.
The CMake statements that make it seem as if the number of cores used to build external project dependencies is halved don't actually do anything. This change removes these statements.
* Adds `tecNO_DELEGATE_PERMISSION` for unauthorized transactions sent by a delegated account.
* Returns `tecNO_TARGET` instead of `terNO_ACCOUNT` for the `DelegateSet` transaction if the delegated account does not exist.
* Fixes `tfFullyCanonicalSig` and `tfInnerBatchTxn` blocking transactions issue by adding `tfUniversal` in the permission related masks in `txFlags.h`
The change increases the default network I/O worker thread pool size from 2 to 6. This will improve stability, as worker thread saturation correlates to desyncs, particularly on high-traffic peers, such as hubs.
To be able to consume `rippled` in Conan 2, the recipe should specify transitive_headers for external libraries that are present in the exported header files. This change remains compatibility with Conan 1, where this flag was not present.
- Specification: https://github.com/XRPLF/XRPL-Standards/pull/272
- Amendment: `TokenEscrow`
- Enables escrowing of IOU and MPT tokens in addition to native XRP.
- Allows accounts to lock issued tokens (IOU/MPT) in escrow objects, with support for freeze, authorization, and transfer rates.
- Adds new ledger fields (`sfLockedAmount`, `sfIssuerNode`, etc.) to track locked balances for IOU and MPT escrows.
- Updates EscrowCreate, EscrowFinish, and EscrowCancel transaction logic to support IOU and MPT assets, including proper handling of trustlines and MPT authorization, transfer rates, and locked balances.
- Enforces invariant checks for escrowed IOU/MPT amounts.
- Extends GatewayBalances RPC to report locked (escrowed) balances.
The changes are focused on fixing NFT transactions bypassing the trustline authorization requirement and potential invariant violation when interacting with deep frozen trustlines.
* Add AMM bid/create/deposit/swap/withdraw/vote invariants:
- Deposit, Withdrawal invariants: `sqrt(asset1Balance * asset2Balance) >= LPTokens`.
- Bid: `sqrt(asset1Balance * asset2Balance) > LPTokens` and the pool balances don't change.
- Create: `sqrt(asset1Balance * assetBalance2) == LPTokens`.
- Swap: `asset1BalanceAfter * asset2BalanceAfter >= asset1BalanceBefore * asset2BalanceBefore`
and `LPTokens` don't change.
- Vote: `LPTokens` and pool balances don't change.
- All AMM and swap transactions: amounts and tokens are greater than zero, except on withdrawal if all tokens
are withdrawn.
* Add AMM deposit and withdraw rounding to ensure AMM invariant:
- On deposit, tokens out are rounded downward and deposit amount is rounded upward.
- On withdrawal, tokens in are rounded upward and withdrawal amount is rounded downward.
* Add Order Book Offer invariant to verify consumed amounts. Consumed amounts are less than the offer.
* Fix Bid validation. `AuthAccount` can't have duplicate accounts or the submitter account.
This commit changes the ledger close in env.meta to be conditional on if it hasn't already been closed (i.e. the current ledger doesn't have any transactions in it). This change will make it a bit easier to use, as it will still work if you close the ledger outside of this usage. Previously, if you accidentally closed the ledger outside of the meta function, it would segfault and it was incredibly difficult to debug.
This commit introduces the following changes:
* Renames `vp_enable config` option to `vp_base_squelch_enable` to enable squelching for validators.
* Removes `vp_squelch` config option which was used to configure whether to send squelch messages to peers or not. With this flag removed, if squelching is enabled, squelch messages will be sent. This was an option used for debugging.
* Introduces a temporary `vp_base_squelch_max_trusted_peers` config option to change the max number of peers who are selected as validator message sources. This is a temporary option, which will be removed once a good value is found.
* Adds a traffic counter to count the number of times peers ignored squelch messages and kept sending messages for squelched validators.
* Moves the decision whether squelching is enabled and ready into Slot.h.
- Specification: [XRPLF/XRPL-Standards 56](https://github.com/XRPLF/XRPL-Standards/blob/master/XLS-0056d-batch/README.md)
- Amendment: `Batch`
- Implements execution of multiple transactions within a single batch transaction with four execution modes: `tfAllOrNothing`, `tfOnlyOne`, `tfUntilFailure`, and `tfIndependent`.
- Enables atomic multi-party transactions where multiple accounts can participate in a single batch, with up to 8 inner transactions and 8 batch signers per batch transaction.
- Inner transactions use `tfInnerBatchTxn` flag with zero fees, no signature, and empty signing public key.
- Inner transactions are applied after the outer batch succeeds via the `applyBatchTransactions` function in apply.cpp.
- Network layer prevents relay of transactions with `tfInnerBatchTxn` flag - each peer applies inner transactions locally from the batch.
- Batch transactions are excluded from AccountDelegate permissions but inner transactions retain full delegation support.
- Metadata includes `ParentBatchID` linking inner transactions to their containing batch for traceability and auditing.
- Extended STTx with batch-specific signature verification methods and added protocol structures (`sfRawTransactions`, `sfBatchSigners`).
Before #5224, the pseudoaccount ID was calculated using prefix expressed in `std::uint16_t`. The refactoring to move the pseudoaccount ID calculation to View.cpp had accidentally changed the prefix type to `int` (derived from `auto i = 0`) which in turn changed the length of the input to `sha512Half` from 2 bytes to 4, altering the result.
This resulted in a different ID of the pseudoaccount calculated from the function after the refactoring, breaking the ledger. This impacts AMMCreate, even when the `SingleAssetVault` amendment is not active. This change restores the prefix type to `std::uint16_t`.
- Specification: XRPLF/XRPL-Standards#239
- Amendment: `SingleAssetVault`
- Implements a vault feature used to store a fungible asset (XRP, IOU, or MPT, but not NFT) and to receive shares in the vault (an MPT) in exchange.
- A vault can be private or public.
- A private vault can use permissioned domains, subject to the `PermissionedDomains` amendment.
- Shares can be exchanged back into asset with `VaultWithdraw`.
- Permissions on the asset in the vault are transitively applied on shares in the vault.
- Issuer of the asset in the vault can clawback with `VaultClawback`.
- Extended `MPTokenIssuance` with `DomainID`, used by the permissioned domain on the vault shares.
Co-authored-by: John Freeman <jfreeman08@gmail.com>
Using std::barrier performs extremely poorly (~1 hour vs ~1 minute to run the test suite) in certain macOS environments.
To unblock our macOS CI pipeline, std::barrier has been replaced with a custom mutex-based barrier (Barrier) that significantly improves performance without compromising correctness.
Running unit tests in parallel and multiple threads can write into one file can corrupt output files, and then gcovr won't be able to parse the corrupted file. This change adds -fprofile-update=atomic as instructed by https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68080.
This change implements the account permission delegation described in XLS-75d, see https://github.com/XRPLF/XRPL-Standards/pull/257.
* Introduces transaction-level and granular permissions that can be delegated to other accounts.
* Adds `DelegateSet` transaction to grant specified permissions to another account.
* Adds `ltDelegate` ledger object to maintain the permission list for delegating/delegated account pair.
* Adds an optional `Delegate` field in common fields, allowing a delegated account to send transactions on behalf of the delegating account within the granted permission scope. The `Account` field remains the delegating account; the `Delegate` field specifies the delegated account. The transaction is signed by the delegated account.
This change updates the squelching logic to accept squelch messages for untrusted validators. As a result, servers will also squelch untrusted validator messages reducing duplicate traffic they generate.
In particular:
* Updates squelch message handling logic to squelch messages for all validators, not only trusted ones.
* Updates the logic to send squelch messages to peers that don't squelch themselves
* Increases the threshold for the number of messages that a peer has to deliver to consider it as a candidate for validator messages.
Combines four related changes:
1. "Decrease `shouldRelay` limit to 30s." Pretty self-explanatory. Currently, the limit is 5 minutes, by which point the `HashRouter` entry could have expired, making this transaction look brand new (and thus causing it to be relayed back to peers which have sent it to us recently).
2. "Give a transaction more chances to be retried." Will put a transaction into `LedgerMaster`'s held transactions if the transaction gets a `ter`, `tel`, or `tef` result. Old behavior was just `ter`.
* Additionally, to prevent a transaction from being repeatedly held indefinitely, it must meet some extra conditions. (Documented in a comment in the code.)
3. "Pop all transactions with sequential sequences, or tickets." When a transaction is processed successfully, currently, one held transaction for the same account (if any) will be popped out of the held transactions list, and queued up for the next transaction batch. This change pops all transactions for the account, but only if they have sequential sequences (for non-ticket transactions) or use a ticket. This issue was identified from interactions with @mtrippled's #4504, which was merged, but unfortunately reverted later by #4852. When the batches were spaced out, it could potentially take a very long time for a large number of held transactions for an account to get processed through. However, whether batched or not, this change will help get held transactions cleared out, particularly if a missing earlier transaction is what held them up.
4. "Process held transactions through existing NetworkOPs batching." In the current processing, at the end of each consensus round, all held transactions are directly applied to the open ledger, then the held list is reset. This bypasses all of the logic in `NetworkOPs::apply` which, among other things, broadcasts successful transactions to peers. This means that the transaction may not get broadcast to peers for a really long time (5 minutes in the current implementation, or 30 seconds with this first commit). If the node is a bottleneck (either due to network configuration, or because the transaction was submitted locally), the transaction may not be seen by any other nodes or validators before it expires or causes other problems.
This change addresses an issue where `rippled` attempts to connect to an IPv6 address, even when the local network lacks IPv6 support, resulting in a "Network is unreachable" error.
The fix replaces the custom endpoint selection logic with `boost::async_connect`, which sequentially attempts to connect to available endpoints until one succeeds or all fail.
This PR replaces the word `failed` with `failure` in any test names and renames some test files to fix MSVC warnings, so that it is easier to search through the test output to find tests that failed.
This change fixes a number of issues involved with CTID:
* CTID is not present on all RPC tx transactions.
* rpcWRONG_NETWORK is missing in the ErrorCodes.cpp
When using subscribe at admin RPC port to send webhooks for the transaction stream to a backend, on large(r) ledgers the endpoint receives fewer HTTP POSTs with TX information than the amount of transactions in a ledger. This change removes the hardcoded queue length to avoid dropping TX notifications for the admin-only command. In addition, the per-request TTL for outgoing RPC HTTP calls has been reduced from 10 minutes to 30 seconds.
This change introduces a new fix amendment (`fixPayChanV1`) that prevents the creation of new `PaymentChannelCreate` transaction with a `CancelAfter` time less than the current ledger time. It piggy backs off of fix1571.
Once the amendment is activated, creating a new `PaymentChannel` will require that if you specify the `CancelAfter` time/value, that value must be greater than or equal to the current ledger time.
Currently users can create a payment channel where the `CancelAfter` time is before the current ledger time. This results in the payment channel being immediately closed on the next PaymentChannel transaction.
This PR splits out `ledger_entry` tests into its own file (`LedgerEntry_test.cpp`) and alphabetizes the helper functions in `LedgerEntry.cpp`. These commits were split out of #5237 to make that PR a little more manageable, since these basic trivial changes are most of the diff. There is no code change, just moving code around.
Adds metric counters for the following P2P message types:
* Untrusted proposal and validation messages
* Duplicate proposal, validation and transaction messages
It’s possible for this to happen legitimately if a set of peers, including a validator, are connected in a cycle, and the latency and message processing time between those peers is significantly less than the latency between the validator and the last peer. It’s unlikely in the real world, but obviously easy to simulate with Antithesis.
The ci pipelines are constantly hitting Docker Hub's public rate limiting since increasing the number of jobs we're running. This change switches over to images hosted in GitHub's registry.
As part of import optimization, a transitive include had been removed that defined `BOOST_COMP_MSVC` on Windows. In unity builds, this definition was pulled in, but in non-unity builds it was not - causing a compilation error. An inspection of the Boost code revealed that we can just gate the statements by `_MS_VER` instead. A `#pragma message` is added to verify that the statement is only printed on Windows builds.
The main goal of this optimisation is memory reduction in SHAMapTreeNodes by introducing intrusive pointers instead of standard std::shared_ptr and std::weak_ptr.
In preparation for a potential reference fee change we would like to verify that fee change works as expected. The first step is to fix all unit tests to be able to work with different reference fee values.
- Detects if the consensus process is "stalled". If it is, then we can declare a
consensus and end successfully even if we do not have 80% agreement on
our proposal.
- "Stalled" is defined as:
- We have a close time consensus
- Each disputed transaction is individually stalled:
- It has been in the final "stuck" 95% requirement for at least 2
(avMIN_ROUNDS) "inner rounds" of phaseEstablish,
- and either all of the other trusted proposers or this validator, if proposing,
have had the same vote(s) for at least 4 (avSTALLED_ROUNDS) "inner
rounds", and at least 80% of the validators (including this one, if
appropriate) agree about the vote (whether yes or no).
- If we have been in the establish phase for more than 10x the previous
consensus establish phase's time, then consensus is considered "expired",
and we will leave the round, which sends a partial validation (indicating
that the node is moving on without validating). Two restrictions avoid
prematurely exiting, or having an extended exit in extreme situations.
- The 10x time is clamped to be within a range of 15s
(ledgerMAX_CONSENSUS) to 120s (ledgerABANDON_CONSENSUS).
- If consensus has not had an opportunity to walk through all avalanche
states (defined as not going through 8 "inner rounds" of phaseEstablish),
then ConsensusState::Expired is treated as ConsensusState::No.
- When enough nodes leave the round, any remaining nodes will see they've
fallen behind, and move on, too, generally before hitting the timeout. Any
validations or partial validations sent during this time will help the
consensus process bring the nodes back together.
This change removes the existing undefined behavior from `LogicError`, so we can be certain that there will be always a stacktrace.
De-referencing a null pointer is an old trick to generate `SIGSEGV`, which would typically also create a stacktrace. However it is also an undefined behaviour and compilers can do something else. A more robust way to create a stacktrace while crashing the program is to use `std::abort`, which we have also used in this location for a long time. If we combine the two, we might not get the expected behaviour - namely, the nullpointer deref followed by `std::abort`, as handled in certain compiler versions may not immediately cause a crash. We have observed stacktrace being wiped instead, and thread put in indeterminate state, then stacktrace created without any useful information.
The Trustline RPC `no_ripple` flag gets set depending on `lsfDefaultRipple` flag, which is not a flag of a trustline but of the account root. The `lsfDefaultRipple` flag does not provide any insight if this particular trust line has `lsfLowNoRipple` or `lsfHighNoRipple` flag set, so it should not be used here at all. This change simplifies the logic.
The `end_marker` is used to limit the range of ledger entries to fetch. If `end_marker` is less than `marker`, a crash can occur. This change adds an additional check.
Changes the error to `malformedAddress` for `permissioned_domain` in the `ledger_entry` rpc, when the account is not a string. This change makes it more clear to a user what is wrong with their request.
What the LoadManager class does is stall detection, which is not the same as deadlock detection. In the condition of severe CPU starvation, LoadManager will currently intentionally crash rippled reporting `LogicError: Deadlock detected`. This error message is misleading as the condition being detected is not a deadlock. This change fixes and refactors the code in response.
Requiring manual updates of numFeatures is an annoying manual process that is easily forgotten, and leads to frequent merge conflicts. This change takes advantage of the `XRPL_FEATURE` and `XRPL_FIX` macros, and adds a new `XRPL_RETIRE` macro to automatically set `numFeatures`.
The codebase is filled with includes that are unused, and which thus can be removed. At the same time, the files often do not include all headers that contain the definitions used in those files. This change uses clang-format and clang-tidy to clean up the includes, with minor manual intervention to ensure the code compiles on all platforms.
- PR #5228 added assert=TRUE and werr=TRUE CMake flags to the
build/action.yml script which is used by all CI jobs to build rippled,
ensuring those flags were always set. The assumption was that only the
CI jobs used that script, so any extra time cost was offset by the
benefit of the extra checks. That assumption was incorrect. That
script is used by other downstream projects. Therefore, those flags
have been moved into the individual CI jobs' "cmake-args" parameter
passed to build/action.yml. This will have the same effect for CI jobs
without any side effects.
Combine multiple related debug log data points into a single
message. Allows quick correlation of events that
previously were either not logged or, if logged, strewn
across multiple lines, making correlation difficult.
The Heartbeat Timer and consensus ledger accept processing
each have this capability.
Also guarantees that log entries will be written if the
node is a validator, regardless of log severity level.
Otherwise, the level of these messages is at INFO severity.
* Add logging for amendment voting decision process
* When counting "received validations" to determine quorum, count the number of validators actually voting, not the total number of possible votes.
The current comment in the example cfg file incorrectly mentions both "may" and "must". This change fixes this comment to clarify that the default port of hosts is 2459 and that specifying it is therefore optional. It further sets the default port to 2459 instead of the legacy 51235.
- Drop duplicate outgoing TMGetLedger messages per peer
- Allow a retry after 30s in case of peer or network congestion.
- Addresses RIPD-1870
- (Changes levelization. That is not desirable, and will need to be fixed.)
- Drop duplicate incoming TMGetLedger messages per peer
- Allow a retry after 15s in case of peer or network congestion.
- The requestCookie is ignored when computing the hash, thus increasing
the chances of detecting duplicate messages.
- With duplicate messages, keep track of the different requestCookies
(or lack of cookie). When work is finally done for a given request,
send the response to all the peers that are waiting on the request,
sending one message per peer, including all the cookies and
a "directResponse" flag indicating the data is intended for the
sender, too.
- Addresses RIPD-1871
- Drop duplicate incoming TMLedgerData messages
- Addresses RIPD-1869
- Improve logging related to ledger acquisition
- Class "CanProcess" to keep track of processing of distinct items
---------
Co-authored-by: Valentin Balaschenko <13349202+vlntb@users.noreply.github.com>
This change enhances the filtering in the ledger, ledger_data, and account_objects methods by also supporting filtering by the canonical name of the LedgerEntryType using case-insensitive matching.
Rewrites the code so that the lock is not held during the callback. Instead it locks twice, once before, and once after. This is safe due to the structure of the code, but is checked after the second lock. This allows mutex_ to be changed back to a regular mutex.
In PeerImpl.cpp, if the function is a message handler (onMessage) or called directly from a message handler, then it should use fee_, since when the handler returns (OnMessageEnd) then the charge function is called. If the function is not a message handler, such as a job queue item, it should remain charge.
- Rename the job in missing-commits.yml from "check" to "up_to_date",
because other jobs named "check" prevent merges, but this one should
not prevent merges. How else are branches going to get caught up?
- Move the job in instrumentation.yml to nix.yml, but keep it entirely
independent.
If the permissioned domains amendment XLS-80 is enabled before credentials XLS-70, then the permissioned domain users will not be able to match any credentials. The changes here prevent the creation of any permissioned domain objects if credentials are not enabled.
Make `simulate` RPC easier to use:
* Prevent the use of `seed`, `secret`, `seed_hex`, and `passphrase` fields (to avoid confusing with the signing methods).
* Add autofilling of the `NetworkID` field.
- Also get the branch name.
- Use rev-parse instead of describe to get a clean hash.
- Return the git hash and branch name in server_info for admin
connections.
- Include git hash and branch name on separate lines in --version.
- spec: XRPLF/XRPL-Standards#220
- amendment: "DeepFreeze"
- implemented deep freeze spec to allow token issuers to prevent currency holders from being able to acquire more of these tokens.
- in combination with normal freeze, deep freeze effectively prevents any balance trust line balance change of a currency holder (except direct issuer <-> holder payments).
- added 2 new invariant checks to verify that deep freeze cannot be enacted without normal freeze and transfer is not frozen.
- made some fixes to existing freeze handling.
Co-authored-by: Ed Hennis <ed@ripple.com>
Co-authored-by: Howard Hinnant <howard.hinnant@gmail.com>
- Fix an erroneous high fee penalty that peers could incur for sending
older transactions.
- Update to the fees charged for imposing a load on the server.
- Prevent the relaying of internal pseudo-transactions.
- Before: Pseudo-transactions received from a peer will fail the signature
check, even if they were requested (using TMGetObjectByHash), because
they have no signature. This causes the peer to be charge for an
invalid signature.
- After: Pseudo-transactions, are put into the global cache
(TransactionMaster) only. If the transaction is not part of
a TMTransactions batch, the peer is charged an unwanted data fee.
These fees will not be a problem in the normal course of operations,
but should dissuade peers from behaving badly by sending a bunch of
junk.
- Improve logging: include the reason for fees charged to a peer.
Co-authored-by: Ed Hennis <ed@ripple.com>
* Has more steps, but allows merges to develop to continue when a
beta / RC is pending, increasing developer velocity.
* Add a CI job to check that no reverse merges have been missed.
* Add some useful scripts in bin/git:
* Set up upstreams as expected for safer pushes
* Squash a bunch of branches
* Set the version number
* Resolves an issue introduced in #5111, which inadvertently removed the
-Wno-maybe-uninitialized compiler option from some xrpl.libxrpl
modules. This resulted in new "may be used uninitialized" build
warnings, first noticed in the "protocol" module. When compiling with
derr=TRUE, those warnings became errors, which made the build fail.
* Github CI actions will build with the assert and werr options turned
on. This will cause CI jobs to fail if a developer introduces a new
compiler warning, or causes an assert to fail in release builds.
* Includes the OS and compiler version in the linux dependencies jobs in
the "check environment" step.
* Translates the `unity` build option into `CMAKE_UNITY_BUILD` setting.
The LEDGER_ENTRY macro now takes an additional parameter, which makes it easier to avoid missing including the new field in jss.h and to the list of account_objects/ledger_data filters.
Replace Issue in STIssue with Asset. STIssue with MPTIssue is only used in MPT tests.
Will be used in Vault and in transactions with STIssue fields once MPT is integrated into DEX.
* Rename ASSERT to XRPL_ASSERT
* Upgrade to Anthithesis SDK 0.4.4, and use new 0.4.4 features
* automatic cast to bool, like assert
* Add instrumentation workflow to verify build with instrumentation enabled
Adds two CMake functions:
* add_module(library subdirectory): Declares an OBJECT "library" (a CMake abstraction for a collection of object files) with sources from the given subdirectory of the given library, representing a module. Isolates the module's headers by creating a subdirectory in the build directory, e.g. .build/tmp123, that contains just a symlink, e.g. .build/tmp123/basics, to the module's header directory, e.g. include/xrpl/basics, in the source directory, and putting .build/tmp123 (but not include/xrpl) on the include path of the module sources. This prevents the module sources from including headers not explicitly linked to the module in CMake with target_link_libraries.
* target_link_modules(library scope modules...): Links the library target to each of the module targets, and removes their sources from its source list (so they are not compiled and linked twice).
Uses these functions to separate and explicitly link modules in libxrpl:
Level 01: beast
Level 02: basics
Level 03: json, crypto
Level 04: protocol
Level 05: resource, server
* Copy Antithesis SDK version 0.4.0 to directory external/
* Add build option `voidstar` to enable instrumentation with Antithesis SDK
* Define instrumentation macros ASSERT and UNREACHABLE in terms of regular C assert
* Replace asserts with named ASSERT or UNREACHABLE
* Add UNREACHABLE to LogicError
* Document instrumentation macros in CONTRIBUTING.md
`STNumber` lets objects and transactions contain multiple fields for
quantities of XRP, IOU, or MPT without duplicating information about the
"issue" (represented by `STIssue`). It is a straightforward serialization of
the `Number` type that uniformly represents those quantities.
---------
Co-authored-by: John Freeman <jfreeman08@gmail.com>
Co-authored-by: Howard Hinnant <howard.hinnant@gmail.com>
* 2.2.2 changed functions acquireAsync and NetworkOPsImp::recvValidation to add an item to a collection under lock, unlock, do some work, then lock again to do remove the item. It will deadlock if an exception is thrown while adding the item - before unlocking.
* Replace ScopedUnlock with scope_unlock.
Move the newest information to the top, i.e., use reverse chronological order within each of the two sections ("API Versions" and "XRP Ledger server versions")
The page_size will soon be made configurable with #5135, making this
re-ordering necessary.
When opening SQLite connection, there are specific pragmas set with
commonPragmas.
In particular, PRAGMA journal_mode creates journal file and locks the
page_size; as of this commit, this sets the page size to the default
value of 4096. Coincidentally, the hardcoded page_size was also 4096, so
no issue was noticed.
Update book_changes RPC to reduce latency, add "validated" field, and accept shortcut strings (current, closed, validated) for ledger_index.
`"validated": true` indicates that the transaction has been included in a validated ledger so the result of the transaction is immutable.
Fix#5033Fix#5034Fix#5035Fix#5036
---------
Co-authored-by: Bronek Kozicki <brok@incorrekt.com>
When rippled initiates a connection to SQLite3, rippled sends a "PRAGMA"
statement defining the maximum number of pages allowed in the database.
Update the max_page_count so it is consistent with the default for newer
versions of SQLite3. Increasing max_page_count is critical for keeping
full history servers online.
Fix#5102
* Retry some failed RPC connections / commands in unit tests
* Remove orphaned `getAccounts` function
Co-authored-by: John Freeman <jfreeman08@gmail.com>
* upstream/master:
Set version to 2.2.2
Allow only 1 job queue slot for each validation ledger check
Allow only 1 job queue slot for acquiring inbound ledger.
Track latencies of certain code blocks, and log if they take too long
* refactor filtering of validations to specifically avoid
concurrent checkAccept() calls for the same validation ledger hash.
* Log when duplicate concurrent validation requests are filtered.
* RAII for containers that track concurrent validation requests.
* Log when duplicate concurrent inbound ledger are filtered.
* RAII for containers that track concurrent inbound ledger.
* Comment on when to asynchronously acquire inbound ledgers, which
is possible to be always OK, but should have further review.
* Other small logging changes
Co-authored-by: Ed Hennis <ed@ripple.com>
Implements a CI workflow that detects when a new version of libxrpl is
proposed, uploads it to artifactory under the `clio` channel and
notifies Clio's CI to check this newly proposed version.
* Add fixNFTokenPageLinks amendment:
It was discovered that under rare circumstances the links between
NFTokenPages could be removed. If this happens, then the
account_objects and account_nfts RPC commands under-report the
NFTokens owned by an account.
The fixNFTokenPageLinks amendment does the following to address
the problem:
- It fixes the underlying problem so no further broken links
should be created.
- It adds Invariants so, if such damage were introduced in the
future, an invariant would stop it.
- It adds a new FixLedgerState transaction that repairs
directories that were damaged in this fashion.
- It adds unit tests for all of it.
* `account_objects` returns an invalid field error if `type` is not supported.
This includes objects an account can't own, or which are unsupported by `account_objects`
* Includes:
* Amendments
* Directory Node
* Fee Settings
* Ledger Hashes
* Negative UNL
The names of the files should reflect the name of the Dir class.
Co-authored-by: Zack Brunson <Zshooter@gmail.com>
Co-authored-by: Ed Hennis <ed@ripple.com>
* fix CTID in tx command returns invalidParams on lowercase hex
* test mixed case and change auto to explicit type
* add header cctype because std::tolower is called
* remove unused local variable
* change test case comment from 'lowercase' to 'mixed case'
---------
Co-authored-by: Zack Brunson <Zshooter@gmail.com>
* Add feature / amendment "InvariantsV1_1"
* Adds invariant AccountRootsDeletedClean:
* Checks that a deleted account doesn't leave any directly
accessible artifacts behind.
* Always tests, but only changes the transaction result if
featureInvariantsV1_1 is enabled.
* Unit tests.
* Resolves#4638
* [FOLD] Review feedback from @gregtatcam:
* Fix unused variable warning
* Improve Invariant test const correctness
* [FOLD] Review feedback from @mvadari:
* Centralize the account keylet function list, and some optimization
* [FOLD] Some structured binding doesn't work in clang
* [FOLD] Review feedback 2 from @mvadari:
* Clean up and clarify some comments.
* [FOLD] Change InvariantsV1_1 to unsupported
* Will allow multiple PRs to be merged over time using the same amendment.
* fixup! [FOLD] Change InvariantsV1_1 to unsupported
* [FOLD] Update and clarify some comments. No code changes.
* Move CMake directory
* Rearrange sources
* Rewrite includes
* Recompute loops
* Fix merge issue and formatting
---------
Co-authored-by: Pretty Printer <cpp@ripple.com>
* fix "account_nfts" with unassociated marker returning issue
* create unit test for fixing nft page invalid marker not returning error
add more test
change test name
create unit test
* fix "account_nfts" with unassociated marker returning issue
* fix "account_nfts" with unassociated marker returning issue
* fix "account_nfts" with unassociated marker returning issue
* fix "account_nfts" with unassociated marker returning issue
* fix "account_nfts" with unassociated marker returning issue
* fix "account_nfts" with unassociated marker returning issue
* fix "account_nfts" with unassociated marker returning issue
* fix "account_nfts" with unassociated marker returning issue
* [FOLD] accumulated review suggestions
* move BEAST check out of lambda function
---------
Authored-by: Scott Schurr <scott@ripple.com>
* fixInnerObjTemplate2 amendment:
Apply inner object templates to all remaining (non-AMM)
inner objects.
Adds a unit test for applying the template to sfMajorities.
Other remaining inner objects showed no problems having
templates applied.
* Move CMake directory
* Rearrange sources
* Rewrite includes
* Recompute loops
---------
Co-authored-by: Pretty Printer <cpp@ripple.com>
Fix interactions between NFTokenOffers and trust lines.
Since the NFTokenAcceptOffer does not check the trust line that
the issuer receives as a transfer fee in the NFTokenAcceptOffer,
if the issuer deletes the trust line after NFTokenCreateOffer,
the trust line is created for the issuer by the
NFTokenAcceptOffer. That's fixed.
Resolves#4925.
Fixes issue #4937.
The fixReducedOffersV1 amendment fixed certain forms of offer
modification that could lead to blocked order books. Reduced
offers can block order books if the effective quality of the
reduced offer is worse than the quality of the original offer
(from the perspective of the taker). It turns out that, for
small values, the quality of the reduced offer can be
significantly affected by the rounding mode used during
scaling computations.
Issue #4937 identified an additional code path that modified
offers in a way that could lead to blocked order books. This
commit changes the rounding in that newly located code path so
the quality of the modified offer is never worse than the
quality of the offer as it was originally placed.
It is possible that additional ways of producing blocking
offers will come to light. Therefore there may be a future
need for a V3 amendment.
* Add trap_tx_hash command line option
This new option can be used only if replay is also enabled. It takes a transaction hash from the ledger loaded for replay, and will cause a specific line to be hit in Transactor.cpp, right before the selected transaction is applied.
Price Oracle data-series logic uses `unordered_map` to update the Oracle object.
This results in different servers disagreeing on the order of that hash table.
Consequently, the generated ledgers will have different hashes.
The fix uses `map` instead to guarantee the order of the token pairs
in the data-series.
Due to the rounding, LPTokenBalance of the last
Liquidity Provider (LP), might not match this LP's
trustline balance. This fix sets LPTokenBalance on
last LP withdrawal to this LP's LPToken trustline
balance.
Single path AMM offer has to factor in the transfer in rate
when calculating the upper bound quality and the quality function
because single path AMM's offer quality is not constant.
This fix factors in the transfer fee in
BookStep::adjustQualityWithFees().
* Fix AMM offer rounding and low quality LOB offer blocking AMM:
A single-path AMM offer with account offer on DEX, is always generated
starting with the takerPays first, which is rounded up, and then
the takerGets, which is rounded down. This rounding ensures that the pool's
product invariant is maintained. However, when one of the offer's side
is XRP, this rounding can result in the AMM offer having a lower
quality, potentially causing offer generation to fail if the quality
is lower than the account's offer quality.
To address this issue, the proposed fix adjusts the offer generation process
to start with the XRP side first and always rounds it down. This results
in a smaller offer size, improving the offer's quality. Regardless if the offer
has XRP or not, the rounding is done so that the offer size is minimized.
This change still ensures the product invariant, as the other generated
side is the exact result of the swap-in or swap-out equations.
If a liquidity can be provided by both AMM and LOB offer on offer crossing
then AMM offer is generated so that it matches LOB offer quality. If LOB
offer quality is less than limit quality then generated AMM offer quality
is also less than limit quality and the offer doesn't cross. To address
this issue, if LOB quality is better than limit quality then use LOB
quality to generate AMM offer. Otherwise, don't use the quality to generate
AMM offer. In this case, limitOut() function in StrandFlow limits
the out amount to match strand's quality to limit quality and consume
maximum AMM liquidity.
Rounding in the payment engine is causing an assert to sometimes fire
with "dust" amounts. This is causing issues when running debug builds of
rippled. This issue will be addressed, but the assert is no longer
serving its purpose.
I am resigning from my role as maintainer of the `rippled` codebase.
Please update repository permissions accordingly, prior to merging this pull request.
Thanks to everyone who has contributed, especially those whom I had the opportunity to closely collaborate with.
The AMM has an invariant for swaps where:
new_balance_1*new_balance_2 >= old_balance_1*old_balance_2
Due to rounding, this invariant could sometimes be violated (although by
very small amounts).
This patch introduces an amendment `fixAMMRounding` that changes the
rounding to always favor the AMM. Doing this should maintain the
invariant.
Co-authored-by: Bronek Kozicki
Co-authored-by: thejohnfreeman
It can be difficult to make transaction breaking changes to low level
code because the low level code does not have access to a ledger and the
current activated amendments in that ledger (the "rules"). This patch
adds global access to the current ledger rules as a `std::optional`. If
the optional is not seated, then there is no active transaction.
The `rotateWithLock` function holds a lock while it calls a callback
function that's passed in by the caller. This is a problematic design
that needs to be used very carefully. In this case, at least one caller
passed in a callback that eventually relocks the mutex on the same
thread, causing UB (a deadlock was observed). The caller was from
SHAMapStoreImpl, and it called `clearCaches`. This `clearCaches` can
potentially call `fetchNodeObject`, which tried to relock the mutex.
This patch resolves the issue by changing the mutex type to a
`recursive_mutex`. Ideally, the code should be rewritten so it doesn't
hold the mutex during the callback and the mutex should be changed back
to a regular mutex.
Co-authored-by: Ed Hennis <ed@ripple.com>
* Amend `.codecov.yml` to disable coverage reporting of test sources
and explicitly set most parameters
* Increase codecov upload retry time to 210s (from 35s)
* Upgrade gcovr adding support for more coverage formats (lcov, clover, jacoco)
* Upgrade github actions in coverage workflow
* Explicitly disable codecov plugins (also removing `gcov` coverage, which is not
correctly handled by codecov https://github.com/codecov/feedback/issues/334)
This amendment, `fixPreviousTxnID`, adds `PreviousTxnID` and
`PreviousTxnLgrSequence` as fields to all ledger objects that did
not already have them included (`DirectoryNode`, `Amendments`,
`FeeSettings`, `NegativeUNL`, and `AMM`). This makes it much easier
to go through the history of these ledger objects.
Github Actions for the build/test jobs (nix.yml, mac.yml, windows.yml) will only run on branches that build packages (develop, release, master), and branches with names starting with "ci/". This is intended as a compromise between disabling CI jobs on personal forks entirely, and having the jobs run as a free-for-all. Note that it will not affect PR jobs at all.
A large synthetic offer was not handled correctly in the payment engine.
This patch fixes that issue and introduces a new invariant check while
processing synthetic offers.
When calculating reward shares, the amount should always be rounded
down. If the `fixUniversalNumber` amendment is not active, this works
correctly. If it is not active, then the amount is incorrectly rounded
up. This patch introduces an amendment so it will be rounded down.
This fixes a case where a peer can desync under a certain timing
circumstance--if it reaches a certain point in consensus before it receives
proposals.
This was noticed under high transaction volumes. Namely, when we arrive at the
point of deciding whether consensus is reached after minimum establish phase
duration but before having received any proposals. This could be caused by
finishing the previous round slightly faster and/or having some delay in
receiving proposals. Existing behavior arrives at consensus immediately after
the minimum establish duration with no proposals. This causes us to desync
because we then close a non-validated ledger. The change in this PR causes us to
wait for a configured threshold before making the decision to arrive at
consensus with no proposals. This allows validators to catch up and for brief
delays in receiving proposals to be absorbed. There should be no drawback since,
with no proposals coming in, we needn't be in a huge rush to jump ahead.
Remove the zaphod.alloy.ee hubs from the bootstrap and default configuration after 5 years. It has been an honor to run these servers, but it is now time for another entity to step into this role.
The zaphod servers will be taken offline in a phased manner keeping all those who have peering arrangements informed.
These would be the preferred attributes of a boostrap set of hubs:
1. Commitment to run the hubs for a minimum of 2 years
2. Highly available
3. Geographically dispersed
4. Secure and up to date
5. Committed to ensure that peering information is kept private
We do not currently enforce that incoming peer connection does not have
remote_endpoint which is already used (either by incoming or outgoing
connection), hence already stored in slots_. If we happen to receive a
connection from such a duplicate remote_endpoint, it will eventually result in a
crash (when disconnecting) or weird behavior (when updating slot state), as a
result of an apparently matching remote_endpoint in slots_ being used by a
different connection.
This amendment fixes an edge case where an empty DID object can be
created. It adds an additional check to ensure that DIDs are
non-empty when created, and returns a `tecEMPTY_DID` error if the DID
would be empty.
The witness server makes heavily use of the `account_tx` RPC command. Perf
testing showed that the SQL query used by `account_tx` became unacceptably slow
when the DB was large and there was a `marker` parameter. The plan for the query
showed only indexed reads. This appears to be an issue with the internal SQLite
optimizer. This patch rewrote the query to use `UNION` instead of `OR` and
significantly improves performance. See RXI-896 and RIPD-1847 for more details.
- Update container for Doxygen workflow. Matches Linux workflow, with newer GLIBC version required by newer actions.
- Fixes macOS workflow to install and configure Conan correctly. Still fails on tests, but that does not seem attributable to the workflow.
* telENV_RPC_FAILED is a new code, reserved exclusively
for unit tests when RPC fails. This will
make those types of errors distinct and easier to test
for when expected and/or diagnose when not.
* Output RPC command result when result is not expected.
We are currently using old version 0.6.2 of `xxhash`, as a verbatim copy and paste of its header file `xxhash.h`. Switch to the more recent version 0.8.2. Since this version is in Conan Center (and properly protects its ABI by keeping the state object incomplete), add it as a Conan requirement. Switch to the SIMD instructions (in the new `XXH3` family) supported by the new version.
This algorithm is about an order of magnitude faster than the existing
algorithm (about 10x faster for encoding and about 15x faster for
decoding - including the double hash for the checksum). The algorithms
use gcc's int128 (fast MS version will have to wait, in the meantime MS
falls back to the slow code).
* It is now an invariant that all constructed Public Keys are valid,
non-empty and contain 33 bytes of data.
* Additionally, the memory footprint of the PublicKey class is reduced.
The size_ data member is declared as static.
* Distinguish and identify the PublisherList retrieved from the local
config file, versus the ones obtained from other validators.
* Fixes#2942
The compilation fails due to an issue in the initializer list
of an optional argument, which holds a vector of pairs.
The code compiles correctly on earlier gcc versions, but fails on gcc 13.
Implement native support for Price Oracles.
A Price Oracle is used to bring real-world data, such as market prices,
onto the blockchain, enabling dApps to access and utilize information
that resides outside the blockchain.
Add Price Oracle functionality:
- OracleSet: create or update the Oracle object
- OracleDelete: delete the Oracle object
To support this functionality add:
- New RPC method, `get_aggregate_price`, to calculate aggregate price for a token pair of the specified oracles
- `ltOracle` object
The `ltOracle` object maintains:
- Oracle Owner's account
- Oracle's metadata
- Up to ten token pairs with the scaled price
- The last update time the token pairs were updated
Add Oracle unit-tests
* Commit 01c37fe introduced a change to the STTx unit test where a local
"defaultRules" object was created with a temporary inline "presets"
value provided to the ctor. Rules::Impl stores a const ref to the
presets provided to the ctor. This particular call provided an inline
temp variable, which goes out of scope as soon as the object is
created. On Windows, attempting to use the presets (e.g. via the
enabled() function) causes an access violation, which crashes the test
run.
* An audit of the code indicates that all other instances of Rules use
the Application's config.features list, which will have a sufficient
lifetime.
Add `STObject` constructor to explicitly set the inner object template.
This allows certain AMM transactions to apply in the same ledger:
There is no issue if the trading fee is greater than or equal to 0.01%.
If the trading fee is less than 0.01%, then:
- After AMM create, AMM transactions must wait for one ledger to close
(3-5 seconds).
- After one ledger is validated, all AMM transactions succeed, as
appropriate, except for AMMVote.
- The first AMMVote which votes for a 0 trading fee in a ledger will
succeed. Subsequent AMMVote transactions which vote for a 0 trading
fee will wait for the next ledger (3-5 seconds). This behavior repeats
for each ledger.
This has no effect on the ultimate correctness of AMM. This amendment
will allow the transactions described above to succeed as expected, even
if the trading fee is 0 and the transactions are applied within one
ledger (block).
Prior to this commit, `port_grpc` could not be added to the [server]
stanza. Instead of validating gRPC IP/Port/Protocol information in
ServerHandler, validate grpc port info in GRPCServer constructor. This
should not break backwards compatibility.
gRPC-related config info must be in a section (stanza) called
[port_gprc].
* Close#4015 - That was an alternate solution. It was decided that with
relaxed validation, it is not necessary to rename port_grpc.
* Fix#4557
These headers are required in the xrpl Conan package in order for
xbridge witness server (xbwd) to build. This change to libxrpl may help
any dependents of libxrpl. This addition does not change any C++ code.
Without this amendment, an NFTokenAcceptOffer transaction can succeed
even when the NFToken recipient does not have sufficient reserves for
the new NFTokenPage. This allowed accounts to accept NFT sell offers
without having a sufficient reserve. (However, there was no issue in
brokered mode or when a buy offer is involved.)
Instead, the transaction should fail with `tecINSUFFICIENT_RESERVE` as
appropriate. The `fixNFTokenReserve` amendment adds checks in the
NFTokenAcceptOffer transactor to check if the OwnerCount changed. If it
did, then it checks the new reserve requirement.
Fix#4679
Use consistent platform-agnostic library names on all platforms.
Fix an issue that prevents dependents like validator-keys-tool from
linking to libxrpl on Windows.
It is bad practice to change the binary base name depending on the
platform. CMake already manipulates the base name into a final name that
fits the conventions of the platform. Linkers accept base names on the
command line and then look for conventional names on disk.
* Disable the Windows CI unit tests "allowed to fail" workaround which
was previously introduced in #4596.
* The runner hardware was upgraded, and the unit tests have been passing
since then.
Update to #4849, using a workaround for spurious codecov upload errors.
Spurious codecov upload errors are expected in public repos which rely
on PRs via forks. Retrying uploads is a decent and easy workaround.
Clients subscribed to `transactions` over WebSocket are being
disconnected because the traffic exceeds the default `send_queue_limit`
of 100.
This commit changes the default configuration, not the default in code.
Fix#4866
Resolves a warning that was emitted from the clang compiler. Switches
usage of the sprintf function to the recommended snprintf function.
Warning was observed in Apple clang version 15.0.0 (clang-1500.0.40.1).
Fix#4569
* Add logging for Application.cpp sweep()
* Improve lifetime management of ledger objects (`SLE`s)
* Only store SLE digest in CachedView; get SLEs from CachedSLEs
* Also force release of last ledger used for path finding if there are
no path finding requests to process
* Count more ST objects (derive from `CountedObject`)
* Track CachedView stats in CountedObjects
* Rename the CachedView counters
* Fix the scope of the digest lookup lock
Before this patch, if you asked "is it caching?" It was always caching.
Prevent WebSocket connections from trying to close twice.
The issue only occurs in debug builds (assertions are disabled in
release builds, including published packages), and when the WebSocket
connections are unprivileged. The assert (and WRN log) occurs when a
client drives up the resource balance enough to be forcibly disconnected
while there are still messages pending to be sent.
Thanks to @lathanbritz for discovering this issue in #4822.
This reverts commit 002893f280.
There were two files with conflicts in the automated revert:
- src/ripple/rpc/impl/RPCHelpers.h and
- src/test/rpc/JSONRPC_test.cpp
Those files were manually resolved.
Workaround for compilation errors with gcc-13 and other compilers
relying on `libstdc++` version 13. This is temporary until actual fix is
available for us to use: https://github.com/boostorg/beast/pull/2682
Some boost.beast files (which we do use) rely on an old gcc-12 behaviour
where `#include <cstdint>` was not needed even though types from this
header were used. This was broken by a change in libstdc++ version 13:
https://gcc.gnu.org/gcc-13/porting_to.html#header-dep-changes
The necessary fix was implemented in boost.beast, however it is not yet
available. Until it is available, we can use this workaround to enable
compilation of `rippled` with gcc-13, clang-16, etc.
* Revert "Optimize calculation of close time to avoid impasse and minimize gratuitous proposal changes (#4760)"
This reverts commit 8ce85a9750.
* Revert "Several changes to improve Consensus stability: (#4505)"
This reverts commit f259cc1ab6.
* Add missing include
---------
Co-authored-by: seelabs <scott.determan@yahoo.com>
* Optimize the calculation of close time to avoid
impasse and minimize gratuitous proposal changes.
* git apply clang-format.patch
* Review (Howard) fixes.
* Review fix for impasse discovered by John.
* Review fixes (comments) from John.
* Scott S review fixes. Also clang-format.
* Promote API version 2 to supported
* Switch command line to API version 1
* Fix LedgerRequestRPC test
* Remove obsolete tx_account method
This method is not implemented, the only parts which are removed are related to command-line parsing
* Fix RPCCall test
* Reduce diff size, small test improvements
* Minor fixes
* Support for the mold linker
* [fold] handle case where both mold and gold are installed
* [fold] Use first non-default linker
* Fix TransactionEntry_test
* Fix AccountTx_test
---------
Co-authored-by: seelabs <scott.determan@yahoo.com>
* Support for the mold linker (#4807)
* Promote API version 2 to supported (#4803)
* Promote API version 2 to be supported
* Switch the command line to API version 1
* Fix LedgerRequestRPC test
* Remove obsolete tx_account method
This method is not implemented, the only parts which are removed are related to command-line parsing
* Fix RPCCall test
* Reduce diff size, small test improvements
* Minor fixes
* Support for the mold linker
* Fix TransactionEntry_test
* Fix AccountTx_test
---------
Co-authored-by: seelabs <scott.determan@yahoo.com>
* Update Linux smoketest distros (#4813)
* Fix 2.0 regression in tx method with binary output (#4812)
* Fix binary output from tx method
* Formatting fix
* Minor test improvement
* Minor test improvements
* Optimize calculation of close time to avoid impasse and minimize gratuitous proposal changes (#4760)
* Optimize the calculation of close time to avoid
impasse and minimize gratuitous proposal changes.
* git apply clang-format.patch
* Scott S review fixes. Also clang-format.
* Set version to 2.0.0-rc2
---------
Co-authored-by: manoj <mdoshi@ripple.com>
Co-authored-by: Scott Determan <scott.determan@yahoo.com>
Co-authored-by: Bronek Kozicki <brok@incorrekt.com>
Co-authored-by: Michael Legleux <legleux@users.noreply.github.com>
Co-authored-by: Mark Travis <mtrippled@users.noreply.github.com>
* Optimize the calculation of close time to avoid
impasse and minimize gratuitous proposal changes.
* git apply clang-format.patch
* Review (Howard) fixes.
* Review fix for impasse discovered by John.
* Review fixes (comments) from John.
* Scott S review fixes. Also clang-format.
* Promote API version 2 to supported
* Switch command line to API version 1
* Fix LedgerRequestRPC test
* Remove obsolete tx_account method
This method is not implemented, the only parts which are removed are related to command-line parsing
* Fix RPCCall test
* Reduce diff size, small test improvements
* Minor fixes
* Support for the mold linker
* [fold] handle case where both mold and gold are installed
* [fold] Use first non-default linker
* Fix TransactionEntry_test
* Fix AccountTx_test
---------
Co-authored-by: seelabs <scott.determan@yahoo.com>
* Remove include <ranges>
* Formatting fix
* Output for subscriptions
* Output from sign, submit etc.
* Output from ledger
* Output from account_tx
* Output from transaction_entry
* Output from tx
* Store close_time_iso in API v2 output
* Add small APIv2 unit test for subscribe
* Add unit test for transaction_entry
* Add unit test for tx
* Remove inLedger from API version 2
* Set ledger_hash and ledger_index
* Move isValidated from RPCHelpers to LedgerMaster
* Store closeTime in LedgerFill
* Time formatting fix
* additional tests for Subscribe unit tests
* Improved comments
* Rename mInLedger to mLedgerIndex
* Minor fixes
* Set ledger_hash on closed ledger, even if not validated
* Update API-CHANGELOG.md
* Add ledger_hash, ledger_index to transaction_entry
* Fix validated and close_time_iso in account_tx
* Fix typos
* Improve getJson for Transaction and STTx
* Minor improvements
* Replace class enum JsonOptions with struct
We may consider turning this into a general-purpose template and using it elsewhere
* simplify the extraction of transactionID from Transaction object
* Remove obsolete comments
* Unconditionally set validated in account_tx output
* Minor improvements
* Minor fixes
---------
Co-authored-by: Chenna Keshava <ckeshavabs@gmail.com>
The command line API still uses `apiMaximumSupportedVersion`.
The unit test RPCs use `apiMinimumSupportedVersion` if unspecified.
Context:
- #4568
- #4552
With clang 15, an unused-but-set-variable warning was emitted:
PostgresDatabase.cpp:178:14: warning: variable 'expNumResults' set but
not used [-Wunused-but-set-variable]
uint32_t expNumResults = 1;
Introduce the `fixFillOrKill` amendment.
Fix an edge case occurring when an offer with `tfFillOrKill` set (but
without `tfSell` set) fails to cross an offer with a better rate. If
`tfFillOrKill` is set, then the owner must receive the full TakerPays.
Without this amendment, an offer fails if the entire `TakerGets` is not
spent. With this amendment, when `tfSell` is not set, the entire
`TakerGets` does not have to be spent.
For details about OfferCreate, see: https://xrpl.org/offercreate.htmlFix#4684
---------
Co-authored-by: Scott Schurr <scott@ripple.com>
Remove dependency on `<ranges>` header, since it is not implemented by
all compilers which we want to support.
This code change only affects unit tests.
Resolve https://github.com/XRPLF/rippled/issues/4787
Remove `tx_history` and `ledger_header` methods from API version 2.
Update `RPC::Handler` to allow for methods (or method implementations)
to be API version specific. This partially resolves#4727. We can now
store multiple handlers with the same name, as long as they belong to
different (non-overlapping) API versions. This necessarily impacts the
handler lookup algorithm and its complexity; however, there is no
performance loss on x86_64 architecture, and only minimal performance
loss on arm64 (around 10ns). This design change gives us extra
flexibility evolving the API in the future, including other parts of
#4727.
In API version 2, `tx_history` and `ledger_header` are no longer
recognised; if they are called, `rippled` will return error
`unknownCmd`
Resolve#3638Resolve#3539
Using the "Amount" field in Payment transactions can cause incorrect
interpretation. There continue to be problems from the use of this
field. "Amount" is rarely the correct field to use; instead,
"delivered_amount" (or "DeliveredAmount") should be used.
Rename the "Amount" field to "DeliverMax", a less misleading name. With
api_version: 2, remove the "Amount" field from Payment transactions.
- Input: "DeliverMax" in `tx_json` is an alias for "Amount"
- sign
- submit (in sign-and-submit mode)
- submit_multisigned
- sign_for
- Output: Add "DeliverMax" where transactions are provided by the API
- ledger
- tx
- tx_history
- account_tx
- transaction_entry
- subscribe (transactions stream)
- Output: Remove "Amount" from API version 2
Fix#3484Fix#3902
The unity build speeds up compilation by bundling multiple source files
into one larger file. This reduces Windows CI build time by up to 50%.
As described in #4596, the automatic Windows builds take a very long
time. Unity builds are significantly faster - currently about 45 min,
much closer to the typical MacOS (35-40 minutes) and nix (~30 minutes)
run times.
This is intended as a stopgap solution until a more resourced and
reliable runner is available.
No C++ code was changed. This only affects CI.
Add a new RPC / WS call for `server_definitions`, which returns an
SDK-compatible `definitions.json` (binary enum definitions) generated by
the server. This enables clients/libraries to dynamically work with new
fields and features, such as ones that may become available on side
chains. Clients query `server_definitions` on a node from the network
they want to work with, and immediately know how to speak that node's
binary "language", even if new features are added to it in the future
(as long as there are no new serialized types that the software doesn't
know how to serialize/deserialize).
Example:
```js
> {"command": "server_definitions"}
< {
"result": {
"FIELDS": [
[
"Generic",
{
"isSerialized": false,
"isSigningField": false,
"isVLEncoded": false,
"nth": 0,
"type": "Unknown"
}
],
[
"Invalid",
{
"isSerialized": false,
"isSigningField": false,
"isVLEncoded": false,
"nth": -1,
"type": "Unknown"
}
],
[
"ObjectEndMarker",
{
"isSerialized": false,
"isSigningField": true,
"isVLEncoded": false,
"nth": 1,
"type": "STObject"
}
],
...
```
Close#3657
---------
Co-authored-by: Richard Holland <richard.holland@starstone.co.nz>
Implement native support for W3C DIDs.
Add a new ledger object: `DID`.
Add two new transactions:
1. `DIDSet`: create or update the `DID` object.
2. `DIDDelete`: delete the `DID` object.
This meets the requirements specified in the DID v1.0 specification
currently recommended by the W3C Credentials Community Group.
The DID format for the XRP Ledger conforms to W3C DID standards.
The objects can be created and owned by any XRPL account holder.
The transactions can be integrated by any service, wallet, or application.
It might be possible for the server code to indirect through certain
`end()` iterators. While a debug build would catch this problem with
`assert()`s, a release build would crash. If there are problems in this
area in the future, it is best to get a definitive indication of the
nature of the error regardless of whether it's a debug or release build.
To accomplish this, these `assert`s are converted into `LogicError`s
that will produce a reasonable error message when they fire.
In Windows, we need to call `python` in order for the `pip` upgrade
command to work.
This changes the GitHub Actions Windows CI job to use the correct
command to upgrade PIP, fixing this error:
```
ERROR: To modify pip, please run the following command:
C:\hostedtoolcache\windows\Python\3.9.13\x64\python.exe -m pip install --upgrade pip
```
A future task is to make job run on heavy Windows runners so that it
doesn't take so long.
Context: #4596
The assert is saying that the only reason `pathFinder` would be null is
if the request was aborted (connection dropped, etc.). That's what
`continueCallback()` checks. But that is very clearly not true if you
look at `getPathFinder`, which calls `findPaths`, which can return false
for many reasons.
Fix#4744
Update the nix CI runner. This commit does not modify any source code
files. The unix builds were successful, but the binaries were not
uploaded to the internal artifactory. This PR borrows an idea from
@ximinez to attempt to fix this issue.
After successful authentication, the `outcome` variable contains a
string. In the upload step, we are checking if outcome == 'success' as a
prerequisite for uploading the binary.
This commit updates the contents of the `outcome` variable.
Artifactory support was added to the `nix` builds with #4556. This
extends that support to the Windows build. Now the Windows build works;
CI will build and test a Windows release build. This only affects CI and
does not change any C++ code.
* Copy the remote setup step outcome fix from #4716 discussion
* Allow the Windows job to succeed if tests fail:
* Currently the tests do not always pass, even on a single threaded
run on the GitHub runners. So we are using parallel runs and mark
the test step as allowed to fail (continue-on-error).
* At this point, it's more important that the build succeeds than that
the tests succeed, because:
* We've got plenty of test coverage on the other jobs.
* Test failures are much rarer than build failures because of
cross-platform issues.
* Having a test failure locally doesn't interrupt a workflow nearly as
much as a build failure.
Note that Conan Center cannot hold the binaries we need. They do not
build the configurations we need, and they will not add them.
## Future Tasks
This introduces a new bottleneck since the build and test takes over an
hour. Speed up the job by:
* Making this job run on heavy Windows runners.
* Increasing the number of hardware threads.
P2P link compression is a feature added in 1.6.0 by #3287.
https://xrpl.org/enable-link-compression.html
If the default changes in the future - for example, as currently
proposed by #4387 - the comment will be updated at that time.
Fix#4656
Context: The `DisallowIncoming` amendment provides an option to block
incoming trust lines from reaching your account. The
asfDisallowIncomingTrustline AccountSet Flag, when enabled, prevents any
incoming trust line from being created. However, it was too restrictive:
it would block an issuer from authorizing a trust line, even if the
trust line already exists. Consider:
1. Issuer sets asfRequireAuth on their account.
2. User sets asfDisallowIncomingTrustline on their account.
3. User submits tx to SetTrust to Issuer.
At this point, without `fixDisallowIncomingV1` active, the issuer would
not be able to authorize the trust line.
The `fixDisallowIncomingV1` amendment, once activated, allows an issuer
to authorize a trust line even after the user sets the
asfDisallowIncomingTrustline flag, as long as the trust line already
exists.
When a new transactor is added, there are several places in applySteps
that need to be modified. This patch refactors the code so only one
function needs to be modified.
Make transactions and pseudo-transactions share the same commonFields
again. This regularizes the code in a nice way.
While this technically allows pseudo-transactions to have a
TicketSequence field, pseudo-transactions are only ever constructed by
code paths that don't add such a field, so this is not a transaction
processing change. It may be possible to add a separate check to ensure
TicketSequence (and other fields that don't make sense on
pseudo-transactions) are never added to pseudo-transactions, but that
should not be necessary. (TicketSequence is not the only common field
that can not and does not appear in pseudo-transactions.) Note:
TicketSequence is already documented as a common field.
Related: #4637Fix#4714
Address a stack-use-after-scope issue when using rvalues with
`soci::use`. Replace rvalues with lvalues to ensure the scope extends
beyond the end of the expression.
The issue arises from `soci` taking a reference to the rvalue without
copying its value or extending its lifetime. `soci` references rvalues
in `soci::use_container` and then the address in `soci_use_type`. For
types like `int`, memory access post-lifetime is unlikely to cause
issues. However, for `std::string`, the backing heap memory can be freed
and potentially reused, leading to a potential segmentation fault.
This was detected on x86_64 using clang-15 with asan. asan confirms
resolution of the issue.
Fix#4675
Update minimum compiler requirement for building the codebase. The
feature "using enum" is required. This feature was introduced in C++20.
Updating the C++ compiler to version 11 or later fixes this error:
```
Building CXX object CMakeFiles/xrpl_core.dir/src/ripple/protocol/impl/STAmount.cpp.o
/build/ripple/binary/src/ripple/protocol/impl/STAmount.cpp: In lambda function:
/build/ripple/binary/src/ripple/protocol/impl/STAmount.cpp:1577:15: error: expected nested-name-specifier before 'enum'
1577 | using enum Number::rounding_mode;
| ^~~~
```
Fix#4693
Modify the `XChainBridge` amendment.
Before this patch, two door accounts on the same chain could could own
the same bridge spec (of course, one would have to be the issuer and one
would have to be the locker). While this is silly, it does not violate
any bridge invariants. However, on further review, if we allow this then
the `claim` transactions would need to change. Since it's hard to see a
use case for two doors to own the same bridge, this patch disallows
it. (The transaction will return tecDUPLICATE).
Amendment "flapping" (an amendment repeatedly gaining and losing
majority) usually occurs when an amendment is on the verge of gaining
majority, and a validator not in favor of the amendment goes offline or
loses sync. This fix makes two changes:
1. The number of validators in the UNL determines the threshold required
for an amendment to gain majority.
2. The AmendmentTable keeps a record of the most recent Amendment vote
received from each trusted validator (and, with `trustChanged`, stays
up-to-date when the set of trusted validators changes). If no
validation arrives from a given validator, then the AmendmentTable
assumes that the previously-received vote has not changed.
In other words, when missing an `STValidation` from a remote validator,
each server now uses the last vote seen. There is a 24 hour timeout for
recorded validator votes.
These changes do not require an amendment because they do not impact
transaction processing, but only the threshold at which each individual
validator decides to propose an EnableAmendment pseudo-transaction.
Fix#4350
Fix the Windows build by using `unsigned int` (instead of `uint`).
The error, introduced by #4618, looks something like:
rpc\impl\RPCHelpers.h(299,5): error C2061: syntax error: identifier
'uint' (compiling source file app\ledger\Ledger.cpp)
A few methods, including `book_offers`, take currency codes as
parameters. The XRPL doesn't care if the letters in those codes are
lowercase or uppercase, as long as they come from an alphabet defined
internally. rippled doesn't care either, when they are submitted in a
hex representation. When they are submitted in an ASCII string
representation, rippled, but not XRPL, is more restrictive, preventing
clients from interacting with some currencies already in the XRPL.
This change gets rippled out of the way and lets clients submit currency
codes in ASCII using the full alphabet.
Fixes#4112
Currently, the `BUILD.md` instructions suggest using `.build` as the
build directory, so this change helps to reduce confusion.
An alternative would be to instruct developers to add `/.build/` to
`.git/info/exclude` or to user-level `.gitignore` (although the latter
is very intrusive). However, it is being added here because it is a good
practice to have a sensible default that's consistent with the build
instructions.
Clean up the peer-to-peer protocol start/close sequences by introducing
START_PROTOCOL and GRACEFUL_CLOSE messages, which sync inbound/outbound
peer send/receive. The GRACEFUL_CLOSE message differentiates application
and link layer failures.
* Introduce the `InboundHandoff` class to manage inbound peer
instantiation and synchronize the send/receive protocol messages
between peers.
* Update `OverlayImpl` to utilize the `InboundHandoff` class to manage
inbound handshakes.
* Update `PeerImp` for improved handling of protocol messages.
* Modify the `Message` class for better maintainability.
* Introduce P2P protocol version `2.3`.
gateway_balances
* When `account` does not exist in the ledger, return `actNotFound`
* (Previously, a normal response was returned)
* Fix#4290
* When required field(s) are missing, return `invalidParams`
* (Previously, `invalidHotWallet` was incorrectly returned)
* Fix#4548
channel_authorize
* When the specified `key_type` is invalid, return `badKeyType`
* (Previously, `invalidParams` was returned)
* Fix#4289
Since these are breaking changes, they apply only to API version 2.
Supersedes #4577
Copy the new code to `src/secp256k1` without changes:
`src/secp256k1` is identical to bitcoin-core/secp256k1@acf5c55 (v0.3.2).
We could consider changing to a Git submodule, though that would require
changes to the build instructions because we are not using submodules
anywhere else.
Remove the `verify` and `message` function declarations. The explicit
instantiation requests could not be completed because there were no
implementations for those two member functions. It is helpful that the
Microsoft (MSVC) compiler on Windows appears to be strict when it comes
to template instantiation.
This resolves the warning:
XChainAttestations.h(450): warning C4661: 'bool
ripple::XChainAttestationsBase<ripple::XChainClaimAttestation>::verify(void)
const': no suitable definition provided for explicit template
instantiation request
* For example, without this change, to run the TxQ tests, must specify
`--unittest=TxQ1,TxQ2` on the command line. With this change, can use
`--unittest=TxQ`, and both will be run.
* An exact match will prevent any further partial matching.
* This could have some side effects for different tests with a common
name beginning. For example, NFToken, NFTokenBurn, NFTokenDir. This
might be useful. If not, the shorter-named test(s) can be renamed. For
example, NFToken to NFTokens.
* Split the NFToken, NFTokenBurn, and Offer test classes. Potentially speeds
up parallel tests by a factor of 5.
A bridge connects two blockchains: a locking chain and an issuing
chain (also called a mainchain and a sidechain). Both are independent
ledgers, with their own validators and potentially their own custom
transactions. Importantly, there is a way to move assets from the
locking chain to the issuing chain and a way to return those assets from
the issuing chain back to the locking chain: the bridge. This key
operation is called a cross-chain transfer. A cross-chain transfer is
not a single transaction. It happens on two chains, requires multiple
transactions, and involves an additional server type called a "witness".
A bridge does not exchange assets between two ledgers. Instead, it locks
assets on one ledger (the "locking chain") and represents those assets
with wrapped assets on another chain (the "issuing chain"). A good model
to keep in mind is a box with an infinite supply of wrapped assets.
Putting an asset from the locking chain into the box will release a
wrapped asset onto the issuing chain. Putting a wrapped asset from the
issuing chain back into the box will release one of the existing locking
chain assets back onto the locking chain. There is no other way to get
assets into or out of the box. Note that there is no way for the box to
"run out of" wrapped assets - it has an infinite supply.
Co-authored-by: Gregory Popovitch <greg7mdp@gmail.com>
For the `account_tx` and `noripple_check` methods, perform input
validation for optional parameters such as "binary", "forward",
"strict", "transactions". Previously, when these parameters had invalid
values (e.g. not a bool), no error would be returned. Now, it returns an
`invalidParams` error.
* This updates the behavior to match Clio
(https://github.com/XRPLF/clio).
* Since this is potentially a breaking change, it only applies to
requests specifying api_version: 2.
* Fix#4543.
* Verify accepted ledger becomes validated, and retry
with a new consensus transaction set if not.
* Always store proposals.
* Track proposals by ledger sequence. This helps slow peers catch
up with the rest of the network.
* Acquire transaction sets for proposals with future ledger sequences.
This also helps slow peers catch up.
* Optimize timer delay for establish phase to wait based on how
long validators have been sending proposals. This also helps slow
peers to catch up.
* Fix impasse achieving close time consensus.
* Don't wait between open and establish phases.
Add new transaction submission API field, "sync", which
determines behavior of the server while submitting transactions:
1) sync (default): Process transactions in a batch immediately,
and return only once the transaction has been processed.
2) async: Put transaction into the batch for the next processing
interval and return immediately.
3) wait: Put transaction into the batch for the next processing
interval and return only after it is processed.
Minor refactor to `TxFormats.cpp`:
- Rename `commonFields` to `pseudoCommonFields` (since it is the common fields
that all pseudo-transactions need)
- Add a new static variable, `commonFields`, which represents all the common
fields that non-pseudo transactions need (essentially everything that
`pseudoCommonFields` contains, plus `sfTicketSequence`)
This makes it harder to accidentally leave out `sfTicketSequence` in a new
transaction.
- Verify "check", used to retrieve a Check object, is a string.
- Verify "nft_page", used to retrieve an NFT Page, is a string.
- Verify "index", used to retrieve any type of ledger object by its
unique ID, is a string.
- Verify "directory", used to retrieve a DirectoryNode, is a string or
an object.
This change only impacts api_version 2 since it is a breaking change.
https://xrpl.org/ledger_entry.htmlFix#4550
* In namespace ripple, introduces get_name function that takes a
std:🧵:native_handle_type and returns a std::string.
* In namespace ripple, introduces get_name function that takes a
std::thread or std::jthread and returns a std::string.
* In namespace ripple::this_thread, introduces get_name function
that takes no parameters and returns the name of the current
thread as a std::string.
* In namespace ripple::this_thread, introduces set_name function
that takes a std::string_view and sets the name of the current
thread.
* Intended to replace the beast utilities setCurrentThreadName
and getCurrentThreadName.
- Update amm_info to fetch AMM by amm account id.
- This is an additional way to retrieve an AMM object.
- Alternatively, AMM can still be fetched by the asset pair as well.
- Add owner directory entry for AMM object.
Context:
- Add back the AMM object directory entry, which was deleted by #4626.
- This fixes `account_objects` for `amm` type.
Modify two error cases in AMMBid transactor to return `tecINTERNAL` to
more clearly indicate that these errors should not be possible unless
operating in unforeseen circumstances. It likely indicates a bug.
The log level has been updated to `fatal()` since it indicates a
(potentially network-wide) unexpected condition when either of these
errors occurs.
Details:
The two specific transaction error cases changed are:
- `tecAMM_BALANCE` - In this case, this error (total LP Tokens
outstanding is lower than the amount to be burned for the bid) is a
subset of the case where the user doesn't have enough LP Tokens to pay
for the bid. When this case is reached, the bidder's LP Tokens balance
has already been checked first. The user's LP Tokens should always be
a subset of total LP Tokens issued, so this should be impossible.
- `tecINSUFFICIENT_PAYMENT` - In this case, the amount to be refunded as
a result of the bid is greater than the price paid for the auction
slot. This should never occur unless something is wrong with the math
for calculating the refund amount.
Both error cases in question are "defense in depth" measures meant to
protect against making things worse if the code has already reached a
state that is supposed to be impossible, likely due to a bug elsewhere.
Such "shouldn't ever occur" checks should use an error code that
categorically indicates a larger problem. This is similar to how
`tecINVARIANT_FAILED` is a warning sign that something went wrong and
likely could've been worse, but since there isn't an Invariant Check
applying here, `tecINTERNAL` is the appropriate error code.
This is "debatably" a transaction processing change since it could
hypothetically change how transactions are processed if there's a bug we
don't know about.
* Verify accepted ledger becomes validated, and retry
with a new consensus transaction set if not.
* Always store proposals.
* Track proposals by ledger sequence. This helps slow peers catch
up with the rest of the network.
* Acquire transaction sets for proposals with future ledger sequences.
This also helps slow peers catch up.
* Optimize timer delay for establish phase to wait based on how
long validators have been sending proposals. This also helps slow
peers to catch up.
* Fix impasse achieving close time consensus.
* Don't wait between open and establish phases.
Add new transaction submission API field, "sync", which
determines behavior of the server while submitting transactions:
1) sync (default): Process transactions in a batch immediately,
and return only once the transaction has been processed.
2) async: Put transaction into the batch for the next processing
interval and return immediately.
3) wait: Put transaction into the batch for the next processing
interval and return only after it is processed.
Introduce a new variadic template helper function, `forAllApiVersions`,
that accepts callables to execute a set of functions over a range of
versions - from RPC::apiMinimumSupportedVersion to RPC::apiBetaVersion.
This avoids the duplication of code.
Context: #4552
- "Rename" the type `LedgerInfo` to `LedgerHeader` (but leave an alias
for `LedgerInfo` to not yet disturb existing uses). Put it in its own
public header, named after itself, so that it is more easily found.
- Move the type `Fees` and NFT serialization functions into public
(installed) headers.
- Compile the XRPL and gRPC protocol buffers directly into `libxrpl` and
install their headers. Fix the Conan recipe to correctly export these
types.
Addresses change (2) in
https://github.com/XRPLF/XRPL-Standards/discussions/121.
For context: This work supports Clio's dependence on libxrpl. Clio is
just an example consumer. These changes should benefit all current and
future consumers.
---------
Co-authored-by: cyan317 <120398799+cindyyan317@users.noreply.github.com>
Signed-off-by: Manoj Doshi <mdoshi@ripple.com>
Improve the checking of the path lengths during Payments. Previously,
the code that did the check of the payment path lengths was sometimes
executed, but without any effect. This changes it to only check when it
matters, and to not make unnecessary copies of the path vectors.
Signed-off-by: Manoj Doshi <mdoshi@ripple.com>
- Replace custom popcnt16 implementation with std::popcount from C++20
- Maintain compatibility with older compilers and MacOS by providing a
conditional compilation fallback to __builtin_popcount and a lookup
table method
- Move and inline related functions within SHAMapInnerNode for
performance and readability
Signed-off-by: Manoj Doshi <mdoshi@ripple.com>
When an AMM account is deleted, the owner directory entries must be
deleted in order to ensure consistent ledger state.
* When deleting AMM account:
* Clean up AMM owner dir, linking AMM account and AMM object
* Delete trust lines to AMM
* Disallow `CheckCreate` to AMM accounts
* AMM cannot cash a check
* Constrain entries in AuthAccounts array to be accounts
* AuthAccounts is an array of objects for the AMMBid transaction
* SetTrust (TrustSet): Allow on AMM only for LP tokens
* If the destination is an AMM account and the trust line doesn't
exist, then:
* If the asset is not the AMM LP token, then fail the tx with
`tecNO_PERMISSION`
* If the AMM is in empty state, then fail the tx with `tecAMM_EMPTY`
* This disallows trustlines to AMM in empty state
* Add AMMID to AMM root account
* Remove lsfAMM flag and use sfAMMID instead
* Remove owner dir entry for ltAMM
* Add `AMMDelete` transaction type to handle amortized deletion
* Limit number of trust lines to delete on final withdraw + AMMDelete
* Put AMM in empty state when LPTokens is 0 upon final withdraw
* Add `tfTwoAssetIfEmpty` deposit option in AMM empty state
* Fail all AMM transactions in AMM empty state except special deposit
* Add `tecINCOMPLETE` to indicate that not all AMM trust lines are
deleted (i.e. partial deletion)
* This is handled in Transactor similar to deleted offers
* Fail AMMDelete with `tecINTERNAL` if AMM root account is nullptr
* Don't validate for invalid asset pair in AMMDelete
* AMMWithdraw deletes AMM trust lines and AMM account/object only if the
number of trust lines is less than max
* Current `maxDeletableAMMTrustLines` = 512
* Check no directory left after AMM trust lines are deleted
* Enable partial trustline deletion in AMMWithdraw
* Add `tecAMM_NOT_EMPTY` to fail any transaction that expects an AMM in
empty state
* Clawback considerations
* Disallow clawback out of AMM account
* Disallow AMM create if issuer can claw back
This patch applies to the AMM implementation in #4294.
Acknowledgements:
Richard Holland and Nik Bougalis for responsibly disclosing this issue.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the project code and urge researchers to
responsibly disclose any issues they may find.
To report a bug, please send a detailed report to:
bugs@xrpl.org
Signed-off-by: Manoj Doshi <mdoshi@ripple.com>
Introduce an API Changelog, which logs the changes that have been made
to the API.
Without this changelog, it is difficult to keep track of the changes and
additions that are made to the API. While all changes are surfaced in
PRs and Release Notes, these are mixed in with other non-API-affecting
changes. PRs that affect the API have the `API Change` label applied,
but it is hard to identify which PRs have been included in each release.
Furthermore, some API changes will take effect based on `api_version`
(starting with rippled version 1.12, which will introduce `api_version:
2`), while others are based on the `rippled` version.
The API Changelog clarifies the details of the changes in a way that is
easily understood by API consumers, and breaks down the changes to be
clear which ones are gated by api_version (versus `rippled` version).
From now on, all PR authors are responsible for updating the API
Changelog according to the additions/changes that their PR makes to the
APIs.
Reason for this change is here XRPLF/XRPL-Standards#119
We would want to be explicit that this flag is exclusively for trustline. For new token types(eg. CFT), they will not utilize this flag for clawback, instead, they will turn clawback on/off on the token-level, which is more versatile.
Use the most recent versions in ConanCenter.
* Due to a bug in Clang 16, you may get a compile error:
"call to 'async_teardown' is ambiguous"
* A compiler flag workaround is documented in `BUILD.md`.
* At this time, building this with gcc 13 may require editing some files
in `.conan/data`
* A patch to support gcc13 may be added in a later PR.
---------
Co-authored-by: Scott Schurr <scott@ripple.com>
Add AMM functionality:
- InstanceCreate
- Deposit
- Withdraw
- Governance
- Auctioning
- payment engine integration
To support this functionality, add:
- New RPC method, `amm_info`, to fetch pool and LPT balances
- AMM Root Account
- trust line for each IOU AMM token
- trust line to track Liquidity Provider Tokens (LPT)
- `ltAMM` object
The `ltAMM` object tracks:
- fee votes
- auction slot bids
- AMM tokens pair
- total outstanding tokens balance
- `AMMID` to AMM `RootAccountID` mapping
Add new classes to facilitate AMM integration into the payment engine.
`BookStep` uses these classes to infer if AMM liquidity can be consumed.
The AMM formula implementation uses the new Number class added in #4192.
IOUAmount and STAmount use Number arithmetic.
Add AMM unit tests for all features.
AMM requires the following amendments:
- featureAMM
- fixUniversalNumber
- featureFlowCross
Notes:
- Current trading fee threshold is 1%
- AMM currency is generated by: 0x03 + 152 bits of sha256{cur1, cur2}
- Current max AMM Offers is 30
---------
Co-authored-by: Howard Hinnant <howard.hinnant@gmail.com>
Improve error handling for ledger_entry by returning an "invalidParams"
error when one or more request fields are specified incorrectly, or one
or more required fields are missing.
For example, if none of of the following fields is provided, then the
API should return an invalidParams error:
* index, account_root, directory, offer, ripple_state, check, escrow,
payment_channel, deposit_preauth, ticket
Prior to this commit, the API returned an "unknownOption" error instead.
Since the error was actually due to invalid parameters, rather than
unknown options, this error was misleading.
Since this is an API breaking change, the "invalidParams" error is only
returned for requests using api_version: 2 and above. To maintain
backward compatibility, the "unknownOption" error is still returned for
api_version: 1.
Related: #4573Fix#4303
- Previously, mulDiv had `std::pair<bool, uint64_t>` as the output type.
- This is an error-prone interface as it is easy to ignore when
overflow occurs.
- Using a return type of `std::optional` should decrease the likelihood
of ignoring overflow.
- It also allows for the use of optional::value_or() as a way to
explicitly recover from overflow.
- Include limits.h header file preprocessing directive in order to
satisfy gcc's numeric_limits incomplete_type requirement.
Fix#3495
---------
Co-authored-by: John Freeman <jfreeman08@gmail.com>
* Update the `account_info` API so that the `allowClawback` flag is
included in the response.
* The proposed `Clawback` amendement added an `allowClawback` flag in
the `AccountRoot` object.
* In the API response, under `account_flags`, there is now an
`allowClawback` field with a boolean (`true` or `false`) value.
* For reference, the XLS-39 Clawback implementation can be found in
#4553Fix#4588
Enhance security during the build process:
* The '-fstack-protector' flag enables stack protection for preventing
buffer overflow vulnerabilities. If an attempt is made to overflow the
buffer, the program will terminate, thus protecting the integrity of
the stack.
* The '-Wl,-z,relro,-z,now' linker flag enables Read-only Relocations
(RELRO), a feature that helps harden the binary against certain types
of exploits, particularly those that involve overwriting the Global
Offset Table (GOT).
* This flag is only set for Linux builds, due to compatibility issues
with apple-clang.
* The `relro` option makes certain sections of memory read-only after
initialization to prevent them from being overwritten, while `now`
ensures that all dynamic symbols are resolved immediately on program
start, reducing the window of opportunity for attacks.
* Add a new YAML file (.pre-commit-config.yaml) to set up pre-commit
hook for clang-format
* The pre-commit hook is opt-in and needs to be installed in order to
run automatically
* Update CONTRIBUTING.md with instructions on how to set up and use the
clang-format linter
Automating the process of running clang-format before committing code
helps to save time by removing the need to fix formatting issues later.
This is a tooling improvement and doesn't change C++ code.
* Update the version of the checkout action (for GitHub Actions) in
`clang-format.yml` and `levelization.yml`.
* The previous version, v2, was raising deprecation warnings due to
its reliance on Node.js 12.
* The latest checkout action version, v3, uses Node.js 16.
When requesting `account_info` with an invalid `signer_lists` value, the
API should return an "invalidParams" error.
`signer_lists` should have a value of type boolean. If it is not a
boolean, then it is invalid input. The response now indicates that.
* This is an API breaking change, so the change is only reflected for
requests containing `"api_version": 2`
* Fix#4539
The debug packages were named with the extension ".ddeb", but due to a
bug in Artifactory, they need to have the ".deb" extension. Debug symbol
packages with ".ddeb" extensions are not indexed, and thus are not
visible in apt clients.
* Fix the issue by renaming the debug packages in the build script.
* Use GCC-11 and update GCC Conan profile.
* This software requires GCC 11 and C++20. However, reporting mode is
built with C++17.
This is a quick band-aid to fix the build. Later, it will be better to
remove this package-building code.
For context, a Debian (deb) package contains bundled software and
resources necessary for installing and managing software on a
Debian-based system, including Ubuntu and derivatives.
- Use powers of two to clearly indicate the bitmask
- Replace bitmask with explicit if-conditions to better indicate predicates
Change enum values to be powers of two (fix#3417) #4239
Implement the simplified condition evaluation
removes the complex bitwise and(&) operator
Implement the second proposed solution in Nik Bougalis's comment - Software does not distinguish between different Conditions (Version: 1.5) #3417 (comment)
I have tested this code change by performing RPC calls with the commands server_info, server_state, peers and validation_info. These commands worked as expected.
Certain inputs for the AccountTx method should return an error. In other
words, an invalid request from a user or client now results in an error
message.
Since this can change the response from the API, it is an API breaking
change. This commit maintains backward compatibility by keeping the
existing behavior for existing requests. When clients specify
"api_version": 2, they will be able to get the updated error messages.
Update unit tests to check the error based on the API version.
* Fix#4288
* Fix#4545
* Commits 0b812cd (#4427) and 11e914f (#4516) conflict. The first added
references to `ServerHandlerImp` in files outside of that class's
organizational unit (which is technically incorrect). The second
removed `ServerHandlerImp`, but was not up to date with develop. This
results in the build failing.
* Fixes the build by changing references to `ServerHandlerImp` to
the more correct `ServerHandler`.
Rename `ServerHandlerImp` to `ServerHandler`. There was no other
ServerHandler definition despite the existence of a header suggesting
that there was.
This resolves a piece of historical confusion in the code, which was
identified during a code review.
The changes in the diff may look more extensive than they actually are.
The contents of `impl/ServerHandlerImp.h` were merged into
`ServerHandler.h`, making the latter file appear to have undergone
significant modifications. However, this is a non-breaking refactor that
only restructures code.
Fix the libxrpl library target for consumers using Conan.
* Fix installation issues and update includes.
* Update requirements in the Conan package info.
* libxrpl requires openssl::crypto.
(Conan is a software package manager for C++.)
Replace hand-rolled code with std::from_chars for better
maintainability.
The C++ std::from_chars function is intended to be as fast as possible,
so it is unlikely to be slower than the code it replaces. This change is
a net gain because it reduces the amount of hand-rolled code.
Apply a minor cleanup in `TypedField`:
* Remove a non-working and unused move constructor.
* Constrain the remaining constructor to not be overly generic enough as
to be used as a copy or move constructor.
There is now an Artifactory (thanks @shichengripple001 and team!) to
hold dependency binaries for the builds.
* Rewrite the `nix` workflow to use it and cut the time down to a mere
21 minutes.
* This workflow should continue to work (just more slowly) for forks
that do not have access to the Artifactory.
Enhance the /crawl endpoint by publishing WebSocket/RPC ports in the
server_info response. The function processing requests to the /crawl
endpoint actually calls server_info internally, so this change enables a
server to advertise its WebSocket/RPC port(s) to peers via the /crawl
endpoint. `grpc` and `peer` ports are included as well.
The new `ports` array contains objects, each containing a `port` for the
listening port (number string), and an array `protocol` listing the
supported protocol(s).
This allows crawlers to build a richer topology without needing to
port-scan nodes. For non-admin users (including peers), the info about
*admin* ports is excluded.
Also increase test coverage for RPC ServerInfo.
Fix#2837.
Curtail the occurrence of order books that are blocked by reduced offers
with the implementation of the fixReducedOffersV1 amendment.
This commit identifies three ways in which offers can be reduced:
1. A new offer can be partially crossed by existing offers, so the new
offer is reduced when placed in the ledger.
2. An in-ledger offer can be partially crossed by a new offer in a
transaction. So the in-ledger offer is reduced by the new offer.
3. An in-ledger offer may be under-funded. In this case the in-ledger
offer is scaled down to match the available funds.
Reduced offers can block order books if the effective quality of the
reduced offer is worse than the quality of the original offer (from the
perspective of the taker). It turns out that, for small values, the
quality of the reduced offer can be significantly affected by the
rounding mode used during scaling computations.
This commit adjusts some rounding modes so that the quality of a reduced
offer is always at least as good (from the taker's perspective) as the
original offer.
The amendment is titled fixReducedOffersV1 because additional ways of
producing reduced offers may come to light. Therefore, there may be a
future need for a V2 amendment.
* Enable api_version 2, which is currently in beta. It is expected to be
marked stable by the next stable release.
* This does not change any defaults.
* The only existing tests changed were one that set the same flag, which
was now redundant, and a couple that tested versioning explicitly.
- Resolve gcc compiler warning:
AccountObjects.cpp:182:47: warning: redundant move in initialization [-Wredundant-move]
- The std::move() operation on trivially copyable types may generate a
compile warning in newer versions of gcc.
- Remove extraneous header (unused imports) from a unit test file.
Fix a bug in the `NODE_SIZE` auto-detection feature in `Config.cpp`.
Specifically, this patch corrects the calculation for the total amount
of RAM available, which was previously returned in bytes, but is now
being returned in units of the system's memory unit. Additionally, the
patch adjusts the node size based on the number of available hardware
threads of execution.
Misaligned load and store operations are supported by both Intel and ARM
CPUs. However, in C++, these operations are undefined behavior (UB).
Substituting these operations with a `memcpy` fixes this UB. The
compiled assembly code is equivalent to the original, so there is no
performance penalty to using memcpy.
For context: The unaligned load and store operations fixed here were
originally introduced in the slab allocator (#4218).
This assert was put in the wrong place, but it only triggers if shards
are configured. This change moves the assert to the right place and
updates it to ensure correctness.
The assert could be hit after the server downloads some shards. It may
be necessary to restart after the shards are downloaded.
Note that asserts are normally checked only in debug builds, so release
packages should not be affected.
Introduced in: #4319 (66627b26cf)
Global variables in different TUs are initialized in an undefined order.
At least one global variable was accessing a global switchover variable.
This caused the switchover variable to be accessed in an uninitialized
state.
Since the switchover is always explicitly set before transaction
processing, this bug can not effect transaction processing, but could
effect unit tests (and potentially the value of some global variables).
Note: at the time of this patch the offending bug is not yet in
production.
Three new fields are added to the `Tx` responses for NFTs:
1. `nftoken_id`: This field is included in the `Tx` responses for
`NFTokenMint` and `NFTokenAcceptOffer`. This field indicates the
`NFTokenID` for the `NFToken` that was modified on the ledger by the
transaction.
2. `nftoken_ids`: This array is included in the `Tx` response for
`NFTokenCancelOffer`. This field provides a list of all the
`NFTokenID`s for the `NFToken`s that were modified on the ledger by
the transaction.
3. `offer_id`: This field is included in the `Tx` response for
`NFTokenCreateOffer` transactions and shows the OfferID of the
`NFTokenOffer` created.
The fields make it easier to track specific tokens and offers. The
implementation includes code (by @ledhed2222) from the Clio project to
extract NFTokenIDs from mint transactions.
The API would allow seeds (and public keys) to be used in place of
accounts at several locations in the API. For example, when calling
account_info, you could pass `"account": "foo"`. The string "foo" is
treated like a seed, so the method returns `actNotFound` (instead of
`actMalformed`, as most developers would expect). In the early days,
this was a convenience to make testing easier. However, it allows for
poor security practices, so it is no longer a good idea. Allowing a
secret or passphrase is now considered a bug. Previously, it was
controlled by the `strict` option on some methods. With this commit,
since the API does not interpret `account` as `seed`, the option
`strict` is no longer needed and is removed.
Removing this behavior from the API is a [breaking
change](https://xrpl.org/request-formatting.html#breaking-changes). One
could argue that it shouldn't be done without bumping the API version;
however, in this instance, there is no evidence that anyone is using the
API in the "legacy" way. Furthermore, it is a potential security hole,
as it allows users to send secrets to places where they are not needed,
where they could end up in logs, error messages, etc. There's no reason
to take such a risk with a seed/secret, since only the public address is
needed.
Resolves: #3329, #3330, #4337
BREAKING CHANGE: Remove non-strict account parsing (#3330)
On macOS, if you have not installed something that depends on `xz`, then your
system may lack `lzma`, resulting in a build error similar to:
```
Downloading libarchive-3.6.0.tar.xz completed [6250.61k]
libarchive/3.6.0:
ERROR: libarchive/3.6.0: Error in source() method, line 120
get(self, **self.conan_data["sources"][self.version], strip_root=True)
ReadError: file could not be opened successfully:
- method gz: ReadError('not a gzip file')
- method bz2: ReadError('not a bzip2 file')
- method xz: CompressionError('lzma module is not available')
- method tar: ReadError('invalid header')
```
The solution is to ensure that `lzma` is installed by installing `xz`.
SOCI is the C++ database access library. The SOCI recipe was updated in
Conan Center Index (CCI), and it breaks for our choice of options. This
breakage occurs when you build with a fresh Conan cache (e.g. when you
submit a PR, or delete `~/.conan/data`).
* Add a custom Conan recipe for SOCI v4.0.3
* Update dependency building to handle exporting and installing Snappy
and SOCI
* Fix workflows to use custom SOCI recipe
* Update BUILD.md to include instruction for exporting the SOCI Conan
recipe:
* `conan export external/soci soci/4.0.3@`
This solution has been verified on Ubuntu 20.04 and macOS.
Context:
* There is a compiler error that the `sqlite3.h` header is not available
when building soci.
* When package B depends on package A, it finds the pieces it needs by
importing the Package Configuration File (PCF) that Conan generates
for package A.
* Read the CMake written by package B to check that it is importing
the PCF correctly and linking its exports correctly.
* Since this can be difficult, it is often more efficient to check
https://github.com/conan-io/conan-center-index/issues for package B
to see if anyone else has seen a similar problem.
* One of the issues points to a problem area in soci's CMake. To
confirm the diagnosis, review soci's CMake (after any patches are
applied) in the Conan build directory `build/$buildId/src/`.
* Review the Conan-generated PCF in
`build/$buildId/build/$buildType/generators/`.
* In this case, the problem was likely (re)introduced by
https://github.com/conan-io/conan-center-index/pull/17026
* If there is a problem in the source or in the Conan recipe, the
fastest fix is to copy the recipe and either:
* Add a source patch to fix any problems in the source.
* Change the recipe to fix any problems in the recipe.
* In this case, this can be done by finding soci's Conan recipe at
https://github.com/conan-io/conan-center-index/tree/master/recipes/soci
and then copying the `all` directory as `external/$packageName` in our
project. Then, make any changes.
* Test packages can be removed from the recipe folder as they are not
needed.
* If adding a patch in the `patches` directory, add a description for
it to `conandata.yml`.
* Since `conanfile.py` has no `version` property on the recipe class,
builders need to pass a version on the command line (like they do
for our `snappy` recipe).
* Add an example command to `BUILD.md`.
Future work: It may make sense to refer to recipes by revision, by
checking in a lockfile.
This change makes progress on the plan in #4371. It does not replicate
the full [matrix] implemented in #3851, but it does replicate the 1.ii
section of the Linux matrix. It leverages "heavy" self-hosted runners,
and demonstrates a repeatable pattern for future matrices.
[matrix]: d794a0f3f1/.github/README.md (continuous-integration)
Address issues related to the removal of `std::{u,bi}nary_function` in
C++17 and some warnings with Clang 16. Some warnings appeared with the
upgrade to Apple clang version 14.0.3 (clang-1403.0.22.14.1).
- `std::{u,bi}nary_function` were removed in C++17. They were empty
classes with a few associated types. We already have conditional code
to define the types. Just make it unconditional.
- libc++ checks a cast in an unevaluated context to see if a type
inherits from a binary function class in the standard library, e.g.
`std::equal_to`, and this causes an error when the type privately
inherits from such a class. Change these instances to public
inheritance.
- We don't need a middle-man for the empty base optimization. Prefer to
inherit directly from an empty class than from
`beast::detail::empty_base_optimization`.
- Clang warns when all the uses of a variable are removed by conditional
compilation of assertions. Add a `[[maybe_unused]]` annotation to
suppress it.
- As a drive-by clean-up, remove commented code.
See related work in #4486.
If `--quorum` setting is present on the command line, use the specified
value as the minimum quorum. This allows for the use of a potentially
fork-unsafe quorum, but it is sometimes necessary for small and test
networks.
Fix#4488.
---------
Co-authored-by: RichardAH <richard.holland@starstone.co.nz>
Add instructions for installing rippled using the package managers APT
and YUM. Some steps were adapted from xrpl.org.
---------
Co-authored-by: Michael Legleux <mlegleux@ripple.com>
Newer compilers, such as Apple Clang 15.0, have removed `std::result_of`
as part of C++20. The build instructions provided a fix for this (by
adding a preprocessor definition), but the fix was broken.
This fixes the fix by:
* Adding the `conf` prefix for tool configurations (which had been
forgotten).
* Passing `extra_b2_flags` to `boost` package to fix its build.
* Define `BOOST_ASIO_HAS_STD_INVOKE_RESULT` in order to build boost
1.77 with a newer compiler.
Add a `NetworkID` field to help prevent replay attacks on and from
side-chains.
The new field must be used when the server is using a network id > 1024.
To preserve legacy behavior, all chains with a network ID less than 1025
retain the existing behavior. This includes Mainnet, Testnet, Devnet,
and hooks-testnet. If `sfNetworkID` is present in any transaction
submitted to any of the nodes on one of these chains, then
`telNETWORK_ID_MAKES_TX_NON_CANONICAL` is returned.
Since chains with a network ID less than 1025, including Mainnet, retain
the existing behavior, there is no need for an amendment.
The `NetworkID` helps to prevent replay attacks because users specify a
`NetworkID` field in every transaction for that chain.
This change introduces a new UINT32 field, `sfNetworkID` ("NetworkID").
There are also three new local error codes for transaction results:
- `telNETWORK_ID_MAKES_TX_NON_CANONICAL`
- `telREQUIRES_NETWORK_ID`
- `telWRONG_NETWORK`
To learn about the other transaction result codes, see:
https://xrpl.org/transaction-results.html
Local error codes were chosen because a transaction is not necessarily
malformed if it is submitted to a node running on the incorrect chain.
This is a local error specific to that node and could be corrected by
switching to a different node or by changing the `network_id` on that
node. See:
https://xrpl.org/connect-your-rippled-to-the-xrp-test-net.html
In addition to using `NetworkID`, it is still generally recommended to
use different accounts and keys on side-chains. However, people will
undoubtedly use the same keys on multiple chains; for example, this is
common practice on other blockchain networks. There are also some
legitimate use cases for this.
A `app.NetworkID` test suite has been added, and `core.Config` was
updated to include some network_id tests.
The `Ledger` class contains two `SHAMap` instances: the state and
transaction maps. Previously, the maps were dynamically allocated using
`std::make_shared` despite the fact that they did not require lifetime
management separate from the lifetime of the `Ledger` instance to which
they belong.
The two `SHAMap` instances are now regular member variables. Some smart
pointers and dynamic memory allocation was avoided by using stack-based
alternatives.
Commit 3 of 3 in #4218.
The `SHAMapItem` class contains a variable-sized buffer that
holds the serialized data associated with a particular item
inside a `SHAMap`.
Prior to this commit, the buffer for the serialized data was
allocated separately. Coupled with the fact that most instances
of `SHAMapItem` were wrapped around a `std::shared_ptr` meant
that an instantiation might result in up to three separate
memory allocations.
This commit switches away from `std::shared_ptr` for `SHAMapItem`
and uses `boost::intrusive_ptr` instead, allowing the reference
count for an instance to live inside the instance itself. Coupled
with using a slab-based allocator to optimize memory allocation
for the most commonly sized buffers, the net result is significant
memory savings. In testing, the reduction in memory usage hovers
between 400MB and 650MB. Other scenarios might result in larger
savings.
In performance testing with NFTs, this commit reduces memory size by
about 15% sustained over long duration.
Commit 2 of 3 in #4218.
When instantiating a large amount of fixed-sized objects on the heap
the overhead that dynamic memory allocation APIs impose will quickly
become significant.
In some cases, allocating a large amount of memory at once and using
a slabbing allocator to carve the large block into fixed-sized units
that are used to service requests for memory out will help to reduce
memory fragmentation significantly and, potentially, improve overall
performance.
This commit introduces a new `SlabAllocator<>` class that exposes an
API that is _similar_ to the C++ concept of an `Allocator` but it is
not meant to be a general-purpose allocator.
It should not be used unless profiling and analysis of specific memory
allocation patterns indicates that the additional complexity introduced
will improve the performance of the system overall, and subsequent
profiling proves it.
A helper class, `SlabAllocatorSet<>` simplifies handling of variably
sized objects that benefit from slab allocations.
This commit incorporates improvements suggested by Greg Popovitch
(@greg7mdp).
Commit 1 of 3 in #4218.
Add Clio-specific JSS constants to ensure a common vocabulary of
keywords in Clio and this project. By providing visibility of the full
API keyword namespace, it reduces the likelihood of developers
introducing minor variations on names used by Clio, or unknowingly
claiming a keyword that Clio has already claimed. This change moves this
project slightly away from having only the code necessary for running
the core server, but it is a step toward the goal of keeping this
server's and Clio's APIs similar. The added JSS constants are annotated
to indicate their relevance to Clio.
Clio can be found here: https://github.com/XRPLF/clio
Signed-off-by: ledhed2222 <ledhed2222@users.noreply.github.com>
- Include NFTokenPages in account_objects to make it easier to
understand an account's Owner Reserve and simplify app development.
- Update related tests and documentation.
- Fix#4347.
For info about the Owner Reserve, see https://xrpl.org/reserves.html
---------
Co-authored-by: Scott Schurr <scott@ripple.com>
Co-authored-by: Ed Hennis <ed@ripple.com>
Change `ledger_data` to return an empty list when all entries are
filtered out.
When the `type` field is specified for the `ledger_data` method, it is
possible that no objects of the specified type are found. This can even
occur if those objects exist, but not in the section that the server
checked while serving your request. Previously, the `state` field of the
response has the value `null`, instead of an empty array `[]`. By
changing this to an empty array, the response is the same data type so
that clients can handle it consistently.
For example, in Python, `for entry in state` should now work correctly.
It would raise an exception if `state` is `null` (or `None`).
This could break client code that explicitly checks for null. However,
this fix aligns the response with the documentation, where the `state`
field is an array.
Fix#4392.
Previously, the object `account_data` in the `account_info` response
contained a single field `Flags` that contains flags of an account. API
consumers must perform bitwise operations on this field to retrieve the
account flags.
This change adds a new object, `account_flags`, at the top level of the
`account_info` response `result`. The object contains relevant flags of
the account. This makes it easier to write simple code to check a flag's
value.
The flags included may depend on the amendments that are enabled.
Fix#2457.
Log exception messages at several locations.
Previously, these were locations where an exception was caught, but the
exception message was not logged. Logging the exception messages can be
useful for analysis or debugging. The additional logging could have a
small negative performance impact.
Fix#3213.
In the release notes (current and historical), there is a link to the
`Builds` directory. By creating `Builds/README.md` with a link to
`BUILD.md`, it is easier to find the build instructions.
* Create the FeeSettings object in genesis ledger.
* Initialize with default values from the config. Removes the need to
pass a Config down into the Ledger initialization functions, including
setup().
* Drop the undocumented fee config settings in favor of the [voting]
section.
* Fix#3734.
* If you previously used fee_account_reserve and/or fee_owner_reserve,
you should change to using the [voting] section instead. Example:
```
[voting]
account_reserve=10000000
owner_reserve=2000000
```
* Because old Mainnet ledgers (prior to 562177 - yes, I looked it up)
don't have FeeSettings, some of the other ctors will default them to
the config values before setup() tries to load the object.
* Update default Config fee values to match Mainnet.
* Fix unit tests:
* Updated fees: Some tests are converted to use computed values of fee
object, but the default Env config was also updated to fix the rest.
* Unit tests that check the structure of the ledger have updated
hashes and counts.
Add the ability to mark amendments as obsolete. There are some known
amendments that should not be voted for because they are broken (or
similar reasons).
This commit marks four amendments as obsolete:
1. `CryptoConditionsSuite`
2. `NonFungibleTokensV1`
3. `fixNFTokenDirV1`
4. `fixNFTokenNegOffer`
When an amendment is `Obsolete`, voting for the amendment is prevented.
A determined operator can still vote for the amendment by changing the
source, and doing so does not break any protocol rules.
The "feature" command now does not modify the vote for obsolete
amendments.
Before this change, there were two options for an amendment's
`DefaultVote` behavior: yes and no.
After this change, there are three options for an amendment's
`VoteBehavior`: DefaultYes, DefaultNo, and Obsolete.
To be clear, if an obsolete amendment were to (somehow) be activated by
consensus, the server still has the code to process transactions
according to that amendment, and would not be amendment blocked. It
would function the same as if it had been voting "no" on the amendment.
Resolves#4014.
Incorporates review feedback from @scottschurr.
Fix a case where `ripple::Expected` returned a json array, not a value.
The problem was that `Expected` invoked the wrong constructor for the
expected type, which resulted in a constructor that took multiple
arguments being interpreted as an array.
A proposed fix was provided by @godexsoft, which involved a minor
adjustment to three constructors that replaces the use of curly braces
with parentheses. This makes `Expected` usable for
[Clio](https://github.com/XRPLF/clio).
A unit test is also included to ensure that the issue doesn't occur
again in the future.
Make it easy for projects to depend on libxrpl by adding an `ALIAS`
target named `xrpl::libxrpl` for projects to link.
The name was chosen because:
* The current library target is named `xrpl_core`. There is no other
"non-core" library target against which we need to distinguish the
"core" library. We only export one library target, and it should just
be named after the project to keep things simple and predictable.
* Underscores in target or library names are generally discouraged.
* Every target exported in CMake should be prefixed with the project
name.
By adding an `ALIAS` target, existing consumers who use the `xrpl_core`
target will not be affected.
* In the future, there can be a migration plan to make `xrpl_core` the
`ALIAS` target (and `libxrpl` the "real" target, which will affect the
filename of the compiled binary), and eventually remove it entirely.
Also:
* Fix the Conan recipe so that consumers using Conan import a target
named `xrpl::libxrpl`. This way, every consumer can use the same
instructions.
* Document the two easiest methods to depend on libxrpl. Both have been
tested.
* See #4443.
* Remove obsolete build instructions.
* By using Conan, builders can choose which dependencies specifically to
build and link as shared objects.
* Refactor the build instructions based on the plan in #4433.
Without the protocol amendment introduced by this commit, an NFT ID can
be reminted in this manner:
1. Alice creates an account and mints an NFT.
2. Alice burns the NFT with an `NFTokenBurn` transaction.
3. Alice deletes her account with an `AccountDelete` transaction.
4. Alice re-creates her account.
5. Alice mints an NFT with an `NFTokenMint` transaction with params:
`NFTokenTaxon` = 0, `Flags` = 9).
This will mint a NFT with the same `NFTokenID` as the one minted in step
1. The params that construct the NFT ID will cause a collision in
`NFTokenID` if their values are equal before and after the remint.
With the `fixNFTokenRemint` amendment, there is a new sequence number
construct which avoids this scenario:
- A new `AccountRoot` field, `FirstNFTSequence`, stays constant over
time.
- This field is set to the current account sequence when the account
issues their first NFT.
- Otherwise, it is not set.
- The sequence of a newly-minted NFT is computed by: `FirstNFTSequence +
MintedNFTokens`.
- `MintedNFTokens` is then incremented by 1 for each mint.
Furthermore, there is a new account deletion restriction:
- An account can only be deleted if `FirstNFTSequence + MintedNFTokens +
256` is less than the current ledger sequence.
- 256 was chosen because it already exists in the current account
deletion constraint.
Without this restriction, an NFT may still be remintable. Example
scenario:
1. Alice's account sequence is at 1.
2. Bob is Alice's authorized minter.
3. Bob mints 500 NFTs for Alice. The NFTs will have sequences 1-501, as
NFT sequence is computed by `FirstNFTokenSequence + MintedNFTokens`).
4. Alice deletes her account at ledger 257 (as required by the existing
`AccountDelete` amendment).
5. Alice re-creates her account at ledger 258.
6. Alice mints an NFT. `FirstNFTokenSequence` initializes to her account
sequence (258), and `MintedNFTokens` initializes as 0. This
newly-minted NFT would have a sequence number of 258, which is a
duplicate of what she issued through authorized minting before she
deleted her account.
---------
Signed-off-by: Shawn Xie <shawnxie920@gmail.com>
There were situations where `marker`s returned by `account_lines` did
not work on subsequent requests, returning "Invalid Parameters".
This was caused by the optimization implemented in "Enforce account RPC
limits by account objects traversed":
e28989638d
Previously, the ledger traversal would find up to `limit` account lines,
and if there were more, the marker would be derived from the key of the
next account line. After the change, ledger traversal would _consider_
up to `limit` account objects of any kind found in the account's
directory structure. If there were more, the marker would be derived
from the key of the next object, regardless of type.
With this optimization, it is expected that `account_lines` may return
fewer than `limit` account lines - even 0 - along with a marker
indicating that there are may be more available.
The problem is that this optimization did not update the
`RPC::isOwnedByAccount` helper function to handle those other object
types. Additionally, XLS-20 added `ltNFTOKEN_OFFER` ledger objects to
objects that have been added to the account's directory structure, but
did not update `RPC::isOwnedByAccount` to be able to handle those
objects. The `marker` provided in the example for #4354 includes the key
for an `ltNFTOKEN_OFFER`. When that `marker` is used on subsequent
calls, it is not recognized as valid, and so the request fails.
* Add unit test that walks all the object types and verifies that all of
their indexes can work as a marker.
* Fix#4340
* Fix#4354
When writing objects to the NodeStore, we need to convert them from
the in-memory format to the binary format used by the node store.
The conversion is handled by the `EncodedBlob` class, which is only
instantiated on the stack. Coupled with the fact that most objects
are under 1024 bytes in size, this presents an opportunity to elide
a memory allocation in a critical path.
This commit also simplifies the interface of `EncodedBlob` and
eliminates a subtle corner case that could result in dangling
pointers.
These changes are not expected to cause a significant reduction in
memory usage. The change avoids the use of a `std::shared_ptr` when
unnecessary and tries to use stack-based memory allocation instead
of the heap whenever possible.
This is a net gain both in terms of memory usage (lower
fragmentation) and performance (less work to do at runtime).
In rare circumstances, both `onRequestTimeout` and the response handler
(`onSiteFetch` or `onTextFetch`) can get queued and processed. In all
observed cases, the response handler processes a network error.
`onRequestTimeout` usually runs first, but on rare occasions, the
response handler runs first, which leaves `activeResource` empty.
* Prevent internal error by catching overflow exception in `gateway_balances`.
* Treat `gateway_balances` obligations overflow as max (largest valid) `STAmount`.
* Note that very large sums of STAmount are approximations regardless.
---------
Co-authored-by: Scott Schurr <scott@ripple.com>
- MSVC 19.x reported a warning about import paths in boost for
function_output_iterator class (boost::function_output_iterator).
- Eliminate that warning by updating the import paths, as suggested by
the compiler warnings.
Port numbers can now be specified using either a colon or a space.
Examples:
1.2.3.4:51235
1.2.3.4 51235
- In the configuration file, an annoying "gotcha" for node operators is
accidentally specifying IP:PORT combinations using a colon. The code
previously expected a space, not a colon. It also does not provide
good feedback when this operator error is made.
- This change simply allows this mistake (using a colon) to be fixed
automatically, preserving the intention of the operator.
- Add unit tests, which test the functionality when specifying IP:PORT
in the configuration file.
- The RPCCall test regime is not specific enough to test this
functionality, it has been tested by hand.
- Ensure IPv6 addresses are not confused for ip:port
---------
Co-authored-by: Elliot Lee <github.public@intelliot.com>
- Implement the `operator==` and the `operator<=>` (aka the spaceship
operator) in `base_uint`, `Issue`, and `Book`.
- C++20-compliant compilers automatically provide the remaining
comparison operators (e.g. `operator<`, `operator<=`, ...).
- Remove the function compare() because it is no longer needed.
- Maintain the same semantics as the existing code.
- Add some unit tests to gain further confidence.
- Fix#2525.
In Reporting Mode, a server would core dump when it is not able to read
from Cassandra. This patch prevents the core dump when Cassandra is down
for reporting mode servers. This does not fix the root cause, but it
cuts down on some of the resulting noise.
* Follow-up to #4336
* NFToken is the naming convention in the codebase (rather than NFT)
* Rename `lsfDisallowIncomingNFTOffer` to `lsfDisallowIncomingNFTokenOffer`
* Rename `asfDisallowIncomingNFTOffer` to `asfDisallowIncomingNFTokenOffer`
Partially revert the functionality introduced
with #4195 / 5a15229 (part of 1.10.0-b1).
Acknowledgements:
Aaron Hook for responsibly disclosing this issue.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers to
responsibly disclose any issues they may find.
To report a bug, please send a detailed report to:
bugs@xrpl.org
---------
Co-authored-by: Nik Bougalis <nikb@bougalis.net>
- Copies the recipe for Snappy from Conan Center, but removes three
lines that explicitly link the standard library, which prevents
builders from statically linking it.
- Removes the recipe for RocksDB now that an official recipe for version
6.27.3 is in Conan Center.
Developers will likely need to remove cached versions of both RocksDB
and Snappy:
```
conan remove -f rocksdb
conan remove -f snappy
```
---------
Co-authored-by: John Freeman <jfreeman08@gmail.com>
* Set "fail-fast: false" so that multiple jobs in one workflow can
finish independently. By default, if one job fails, other running jobs
will be aborted, even if the other jobs are working fine and are
almost done. This leads to wasted time and resources if the failure
is, for example, OS specific, or due to a flaky unit test, and the
failed job needs to be re-run, because all the jobs end up re-running.
* Put conditions back into the windows.yml job (manual, and for
a specific branch name and that job). This prevents Github Actions
from sending "No jobs were run" failure emails on every commit.
Without this amendment, for NFTs using broker mode, if the sell offer contains a destination and that destination is the buyer account, anyone can broker the transaction. Also, if a buy offer contains a destination and that destination is the seller account, anyone can broker the transaction. This is not ideal and is misleading.
Instead, with this amendment: If you set a destination, that destination needs to be the account settling the transaction. So, the broker must be the destination if they want to settle. If the buyer is the destination, then the buyer must accept the sell offer, as you cannot broker your own offers.
If users want their offers open to the public, then they should not set a destination. On the other hand, if users want to limit who can settle the offers, then they would set a destination.
Unit tests:
1. The broker cannot broker a destination offer to the buyer and the buyer must accept the sell offer. (0 transfer)
2. If the broker is the destination, the broker will take the difference. (broker mode)
Fixes#4374
It was possible for a broker to combine a sell and a buy offer from an account that already owns an NFT. Such brokering extracts money from the NFT owner and provides no benefit in return.
With this amendment, the code detects when a broker is returning an NFToken to its initial owner and prohibits the transaction. This forbids a broker from selling an NFToken to the account that already owns the token. This fixes a bug in the original implementation of XLS-20.
Thanks to @nixer89 for suggesting this fix.
Fixes 3 issues:
In the following scenario, an account cannot perform NFTokenAcceptOffer even though it should be allowed to:
- BROKER has < S
- ALICE offers to sell token for S
- BOB offers to buy token for > S
- BROKER tries to bridge the two offers
This currently results in `tecINSUFFICIENT_FUNDS`, but should not because BROKER is not spending any funds in this transaction, beyond the transaction fee.
When trading an NFT using IOUs, and when the issuer of the IOU has any non-zero value set for TransferFee on their account via AccountSet (not a TransferFee on the NFT), and when the sale amount is equal to the total balance of that IOU that the buyer has, the resulting balance for the issuer of the IOU will become positive. This means that the buyer of the NFT was supposed to have caused a certain amount of IOU to be burned. That amount was unable to be burned because the buyer couldn't cover it. This results in the buyer owing this amount back to the issuer. In a real world scenario, this is appropriate and can be settled off-chain.
Currency issuers could not make offers for NFTs using their own currency, receiving `tecINSUFFICIENT_FUNDS` if they tried to do so.
With this fix, they are now able to buy/sell NFTs using their own currency.
Three static member functions are introduced with
definitions consistent with std::numeric_limits:
static constexpr Number min() noexcept;
Returns: The minimum positive value. This is the value closest to zero.
static constexpr Number max() noexcept;
Returns: The maximum possible value.
static constexpr Number lowest() noexcept;
Returns: The negative value which is less than all other values.
You can set a thread-local flag to direct Number how to round
non-exact results with the syntax:
Number::rounding_mode prev_mode = Number::setround(Number::towards_zero);
This flag will stay in effect for this thread only until another call
to setround. The previously set rounding mode is returned.
You can also retrieve the current rounding mode with:
Number::rounding_mode current_mode = Number::getround();
The available rounding modes are:
* to_nearest : Rounds to nearest representable value. On tie, rounds
to even.
* towards_zero : Rounds towards zero.
* downward : Rounds towards negative infinity.
* upward : Rounds towards positive infinity.
The default rounding mode is to_nearest.
* Conversions to Number are implicit
* Conversions away from Number are explicit and potentially lossy
* If lossy, round to nearest, and to even on tie
* Introduces amendment `XRPFees`
* Convert fee voting and protocol messages to use XRPAmounts
* Includes Validations, Change transactions, the "Fees" ledger object,
and subscription messages
* Improve handling of 0 drop reference fee with TxQ. For use with networks that do not want to require fees
* Note that fee escalation logic is still in place, which may cause the
open ledger fee to rise if the network is busy. 0 drop transactions
will still queue, and fee escalation can be effectively disabled by
modifying the configuration on all nodes
* Change default network reserves to match Mainnet
* Name the new SFields *Drops (not *XRP)
* Reserve SField IDs for Hooks
* Clarify comments explaining the ttFEE transaction field validation
Fixes#4005
Makes it possible for internal RPC Error Codes to associate
themselves with a non-OK (200) HTTP status code. There are
quite a number of RPC responses in addition to tooBusy that
now have non-OK HTTP status codes.
The new return HTTP return codes are only enabled by including
"ripplerpc": "3.0" or higher in the original request.
Otherwise the historical value, 200, continues to be returned.
This ensures that this is not a breaking change.
featureDisallowIncoming is a new amendment that would allow users to opt-out of incoming Checks, Payment Channels, NFTokenOffers, and trust lines. This commit includes tests.
Adds four new AccountSet Flags:
1. asfDisallowIncomingNFTOffer
2. asfDisallowIncomingCheck
3. asfDisallowIncomingPayChan
4. asfDisallowIncomingTrustline
Introduces a conanfile.py (and a Conan recipe for RocksDB) to enable building the package with Conan, choosing more recent default versions of dependencies. It removes almost all of the CMake build files related to dependencies, and the configurations for Travis CI and GitLab CI. A new set of cross-platform build instructions are written in BUILD.md.
Includes example GitHub Actions workflow for each of Linux, macOS, Windows.
* Test on macos-12
We use the <concepts> library which was not added to Apple Clang until
version 13.1.6. The default Clang on macos-11 (the sometimes current
version of macos-latest) is 13.0.0, and the default Clang on macos-12 is
14.0.0.
Closes#4223.
Clang warned about the code removed in this patch with the warning:
```
warning: out-of-line definition of constexpr static data member is
redundant in C++17 and is deprecated [-Wdeprecated]
```
There's a bug in gdb where unsigned template parameters cause issues with
RTTI. This patch changes a template parameter from `size_t` to `int` to
work around this gdb bug.
Reduce the reserve requirements from 20/5 to 10/2 in line with the current network votes. The requirements of 10/2 have been on the network long enough that new nodes should not still have the old reserve amount.
Co-authored-by: Richard Holland <richard.holland@starstone.co.nz>
* Per actions/runner-images#6002, ubuntu-18.04 is being deprecated. If
latest ever fails in the future, we'll need to fix the jobs anyway, so
catch it early.
* Use long option names
* Force clang-format to ubuntu-20.04 because LLVM 10 is not available for 22.04
* Improve move semantics in Expected:
This patch unconditionally moves an `Unexpected<U>` value parameter as
long as `U` is not a reference. If `U` is a reference the code should
not compile. An error type that holds a reference is a strange use-case,
and an overload is not provided. If it is required in the future it can
be added.
The `Expected(U r)` overload should take a forwarding ref.
* Replace enable_if with concepts in Expected
* Removed a reference to the default number of workers varying based on whether a node has validation enabled. Workers default to the number of processor cores + 2: https://github.com/ripple/rippled/blob/develop/src/ripple/core/impl/JobQueue.cpp#L166
* Protobuf v2 and Ubuntu 16.04 are no longer supported.
* Updated protobuf version as v3 is now supported, fixed typos, automatically sent number of processors when building boost & rippled.
It turns out that the feature enabled by the tfTrustLine flag
on an NFTokenMint transaction could be used as a means to
attack the NFToken issuer. Details are in
https://github.com/XRPLF/rippled/issues/4300
The fixRemoveNFTokenAutoTrustLine amendment removes the
ability to set the tfTrustLine flag on an NFTokenMint
transaction.
Closes 4300.
When starting, the code generates a new ephemeral private key and
a corresponding certificate to go along with it. This process can
take time and, while this is unlikely to matter for normal server
operations, it can have a significant impact for unit testing and
development. Profiling data suggests that ~20% of the time needed
for a unit test run can be attributed to this.
This commit does several things:
1. It restructures the code so that a new self-signed certificate
and its corresponding private key are only initialized once at
startup; this has minimal impact on the operation of a regular
server.
2. It provides new default DH parameters. This doesn't impact the
security of the connection, but those who compile from scratch
can generate new parameters if they so choose.
3. It properly sets the version number in the certificate, fixing
issue #4007; thanks to @donovanhide for the report.
4. It uses SHA-256 instead of SHA-1 as the hash algorithm for the
certificate and adds some X.509 extensions as well as a random
128-bit serial number.
5. It rounds the certificate's "start of validity" period so that
the server's precise startup time cannot be easily deduced and
limits the validity period to two years, down from ten years.
6. It removes some CBC-based ciphers from the default cipher list
to avoid some potential security issues, such as CVE-2016-2107
and CVE-2013-0169.
Caching the base58check encoded version of an `AccountID` has
performance advantages, because because of the computationally
heavy cost associated with the conversion, which requires the
application of SHA-256 twice.
This commit makes the cache significantly more efficient in terms
of memory used: it eliminates the map, using a vector with a size
that is determined by the configured size of the node, and a hash
function to directly map any given `AccountID` to a specific slot
in the cache; the eviction policy is simple: in case of collision
the existing entry is removed and replaced with the new data.
Previously, use of the cache was optional and required additional
effort by the programmer. Now the cache is automatic and does not
require any additional work or information.
The new cache also utilizes a 64-way spinlock, to help reduce any
contention that the pressure on the cache would impose.
Each node on the network is supposed to have a unique cryptographic
identity. Typically, this identity is generated randomly at startup
and stored for later reuse in the (poorly named) file `wallet.db`.
If the file is copied, it is possible for two nodes to share the
same node identity. This is generally not desirable and existing
servers will detect and reject connections to other servers that
have the same key.
This commit achives three things:
1. It improves the detection code to pinpoint instances where two
distinct servers with the same key connect with each other. In
that case, servers will log an appropriate error and shut down
pending intervention by the server's operator.
2. It makes it possible for server administrators to securely and
easily generate new cryptographic identities for servers using
the new `--newnodeid` command line arguments. When a server is
started using this command, it will generate and save a random
secure identity.
3. It makes it possible to configure the identity using a command
line option, which makes it possible to derive it from data or
parameters associated with the container or hardware where the
instance is running by passing the `--nodeid` option, followed
by a single argument identifying the infomation from which the
node's identity is derived. For example, the following command
will result in nodes with different hostnames having different
node identities: `rippled --nodeid $HOSTNAME`
The last option is particularly useful for automated cloud-based
deployments that minimize the need for storing state and provide
unique deployment identifiers.
**Important note for server operators:**
Depending on variables outside of the the control of this code,
such as operating system version or configuration, permissions,
and more, it may be possible for other users or programs to be
able to access the command line arguments of other processes
on the system.
If you are operating in a shared environment, you should avoid
using this option, preferring instead to use the `[node_seed]`
option in the configuration file, and use permissions to limit
exposure of the node seed.
A user who gains access to the value used to derive the node's
unique identity could impersonate that node.
The commit also updates the minimum supported server protocol
version to `XRPL/2.1`, which has been supported since version
1.5.0 and eliminates support for `XPRL/2.0`.
We profiled different algorithms and data structures to understand which
strategy is best from a performance standpoint:
- Linear search on an array;
- Binary search on a sorted array;
- Using `std::map`; and
- Using `std::unordered_map`.
Both linear search and std::unordered_map outperformed the other alternatives
so no change to the existing data structure is justified. If more handers are
added, this should be revisited.
For some additional details and timings, please see:
https://github.com/XRPLF/rippled/issues/3298#issuecomment-1185946010
Trustlines must be between two different accounts but two trustlines exist
where an account extends trust to itself. They were created in the early
days, likely because of bugs that have been fixed. The new fixTrustLinesToSelf
amendment will remove those trustlines when it activates.
The existing spinlock code, used to protect SHAMapInnerNode
child lists, has a mistake that can allow the same child to
be repeatedly locked under some circumstances.
The bug was in the `SpinBitLock::lock` loop condition check
and would result in the loop terminating early.
This commit fixes this and further simplifies the lock loop
making the correctness of the code easier to verify without
sacrificing performance.
It also promotes the spinlock class from an implementation
detail to a more general purpose, easier to use lock class
with clearer semantics. Two different lock types now allow
developers to easily grab either a single spinlock from an
a group of spinlocks (packed in an unsigned integer) or to
grab all of the spinlocks at once.
While this commit makes spinlocks more widely available to
developers, they are rarely the best tool for the job. Use
them judiciously and only after careful consideration.
The XLS-20 implementation contained two bugs that would require the
introduction of amendments. This complicates the adoption of XLS-20
by requiring a staggered amendment activation, first of the two fix
amendments, followed by the `NonFungibleTokensV1` amendment.
After consideration, the consensus among node operators is that the
process should be simplified by the introduction of a new amendment
that, if enabled, would behaves as if the `NonFungibleTokensV1` and
the two fix amendments (`fixNFTokenDirV1` and `fixNFTokenNegOffer`)
were activated at once.
This commit implements this proposal; it does not introduce any new
functionality or additional features, above and beyond that offered
by the existing amendments.
The peer discovery protocol depends on peers exchanging messages
listing IP addresses for other peers.
Under normal circumstances, these messages should not be sent
frequently; the existing code would track the earliest time a
new message should be processed, but did not actually enforce
that limit.
An incorrect SQL query could cause the server to improperly
configure its voting state after a restart; typically, this
would manifest as an apparent failure to store a vote which
the administrator of the server had configured.
This commit fixes the broken SQL and ensures that amendment
votes are properly reloaded post-restart and closes#4220.
The existing code would, incorrectly, allow negative amounts in offers
for non-fungible tokens. Such offers would be handled very differently
depending on the context: a direct offer would fail with an error code
indicating an internal processing error, whereas brokered offers would
improperly succeed.
This commit introduces the `fixNFTokenNegOffer` amendment that detects
such offers during creation and returns an appropriate error code.
The commit also extends the existing code to allow for buy offers that
contain a `Destination` field, so that a specific broker can be set in
the offer.
A few unit tests have historically generated a lot of noise
to the console from log writes. This noise was not useful
and made it harder to locate actual test failures.
By changing the log level of these tests from
- severities::kError to
- severities::kDisabled
it was possible to remove that noise coming from the logs.
While there should never be a missing node when copying the SHAMap,
rippled should not terminate when there's an error rotating the
database. This patch aborts the database rotation rather than aborting rippled.
ThreadSafetyAnalysis was used to identify race conditions in this file.
This analysis was modivated by a (rare) crash while running unit tests.
Add locks to Shard flagged by ThreadSafetyAnalysis
The existing code properly parses the network_id parameter from the
the configuration file, but it does not properly set up the code to
use the value correctly. As a result the configured `network_id` is
ignored.
o Fixes an off-by-one when determining which NFTokenPage an
NFToken belongs on.
o Improves handling of packed sets of 32 NFTs with
identical low 96-bits.
o Fixes marker handling by the account_nfts RPC command.
o Tightens constraints of NFTokenPage invariant checks.
Adds unit tests to exercise the fixed cases as well as tests
for previously untested functionality.
The amendment increases the maximum sign of an account's signer
list from 8 to 32.
Like all new features, the associated amendment is configured with
a default vote of "no" and server operators will have to vote for
it explicitly if they believe it is useful.
* "A path is considered invalid if and only if it enters and exits an
address node through trust lines where No Ripple has been enabled for
that address." (https://xrpl.org/rippling.html#specifics)
* When loading trust lines for an account "Alice" which was reached
via a trust line that has the No Ripple flag set on Alice's side, do
not use or cache any of Alice's trust lines which have the No Ripple
flag set on Alice's side. For typical "end-user" accounts, this will
return no trust lines.
One of the two versions of the `rngfill` function accepts a pointer
to a buffer and a size (in bytes). The function aims to fill the
provided `buffer` with `size` random bytes. It does this in chunks
of 8 bytes, for long as possible, and then fills any left-over gap
one byte at a time.
To avoid an annoying and incorrect warning about a potential buffer
overflow in the "trailing write", commit 78bc2727f7
used a `#pragma` to instruct the compiler to not generate the incorrect
diagnostic. Unfortunately, this change _also_ eliminated the trailing
write code, which means that, under some cases, the `rngfill` function
would generate between 1 and 7 fewer random bytes than requested.
This problem would only manifest on builds that do not define `__GNUC__`
which, as of this writing, means MSVC.
Several hard-coded parameters control the behavior of the ledger
acquisition engine. The values of many of these parameters where
set by intuition and have complex and non-intuitive interactions
with each other and other parts of the code.
An earlier commit attempted to adjust several of these parameters
to improve syncing performance; initial testing was promising but
a number of operators reported experiencing syncing and stability
issues with their servers. As a result, this commit reverts parts
of commit 18235067af.
This commit further adjusts some tunables so as to increase the
aggressiveness of the ledger acquisition engine.
This commit addresses minor bugs introduced with commit
6faaa91850:
- The number of threads used by the database engine was
incorrectly clamped to the lower possible value, such
that the database was effectively operating in single
threaded mode.
- The number of requests to extract at once was so high
that it could result in increased latency. The bundle
size is now limited to 4 and can be adjusted by a new
configuration option `rq_bundle` in the `[node_db]`
stanza. This is an advanced tunable and adjusting it
should not be needed.
This commit modernizes the `AcceptedLedger` and `AcceptedLedgerTx`
classes, reduces their memory footprint and reduces unnecessary
dynamic memory allocations.
This commit optimizes the way asynchronous nodestore operations are
processed both by reducing the amount of time locks are held and by
minimizing the number of memory allocations and data copying.
Adds support to TaggedCache to support smart replacement
(Needed to avoid race conditions with negative caching.)
Create a "hotDUMMY" object that represents the absence
of an object.
Allow DatabaseNodeImp::asyncFetch to complete immediately
if object is in cache (positive or negative).
Fix a bug in asyncFetch where the object returned may not
be the correct canonical version because we stash the
object in the results array before we canonicalize it.
When fetching ledgers, the existing code would isolate the peer
that sent the most useful responses and issue follow up queries
only to that peer.
This commit increases the query aggressiveness, and changes the
mechanism used to select which peers to issue follow-up queries
to so as to more evenly spread the load along those peers which
provided useful responses.
The `SHAMapInnerNode` class had a global mutex to protect the
array of node children. Profiling suggested that around 4% of
all attempts to lock the global would block.
This commit removes that global mutex, and replaces it with a
new per-node 16-way spinlock (implemented so as not to effect
the size of an inner node objet), effectively eliminating the
lock contention.
* Txs with the same fee level will sort by TxID XORed with the parent
ledger hash.
* The TxQ is re-sorted after every ledger.
* Attempt to future-proof the TxQ tie breaking test
* Abort background path finding when closed or disconnected
* Exit pathfinding job thread if there are no requests left
* Don't bother creating the path find job if there are no requests
* Refactor to remove circular dependency between InfoSub and PathRequest
The existing trust line caching code was suboptimal in that it stored
redundant information, pinned SLEs into memory and required multiple
memory allocations per cached object.
This commit eliminates redundant data, reducing the size of cached
objects and unpinning SLEs from memory, and uses value_types to
avoid the need for `std::shared_ptr`. As a result of these changes, the
effective size of a cached object, includes the overhead of the memory
allocator and the `std::shared_ptr` should be reduced by at least 64
bytes. This is significant, as there can easily be tens of millions
of these objects.
This commit combines the `apply_mutex` and `read_mutex` into a single `mutex_`
var. This new `mutex_` var is a `shared_mutex`, and most operations only need to
lock it with a `shared_lock`. The only exception is `applyMutex`, which may need
a `unique_lock`.
One consequence of removing the `apply_mutex` is more than one `applyMutex`
function can run at the same time. To help reduce lock contention that a
`unique_lock` would cause, checks that only require reading data are run a
`shared_lock` (call these the "prewriteChecks"), then the lock is released, then
a `unique_lock` is acquired. Since a currently running `applyManifest` may write
data between the time a `shared_lock` is released and the `write_lock` is
acquired, the "prewriteChecks" need to be rerun. Duplicating this work isn't
ideal, but the "prewirteChecks" are relatively inexpensive.
A couple of other designs were considered. We could restrict more than one
`applyMutex` function from running concurrently - either with a `applyMutex` or
my setting the max number of manifest jobs on the job queue to one. The biggest
issue with this is if any other function ever adds a write lock for any reason,
`applyManifest` would not be broken - data could be written between the release
of the `shared_lock` and the acquisition of the `unique_lock`. Note: it is
tempting to solve this problem by not releasing the `shared_mutex` and simply
upgrading the lock. In the presence of concurrently running `applyManifest`
functions, this will deadlock (both function need to wait for the other to
release their read locks before they can acquire a write lock).
In order to preserve the Hooks ABI, it is important that field
values used for hooks be stable going forward.
This commit reserves the required codes so that they will not
be repurposed before Hooks can be proposed for inclusion in
the codebase.
* Remove Application & Database dependency in PerfLog. Replace it with
a callback passed into the constructor.
* Fixes the circular dependency between ripple/nodestore and ripple/basics
If fast loading is enabled but the last persisted ledger is not
entirely on disk, the server would fail to start without manual
intervention by the server operator.
This commit allows the server to detect this scenario and attempt
to automatically recover.
This is a refactor aimed at cleaning up and simplifying the existing
job queue.
As of now, all jobs are cancelled at the same time and in the same
way, so this commit removes the per-job cancellation token. If the
need for such support is demonstrated, support can be re-added.
* Revise documentation for ClosureCounter and Workers.
* Simplify code, removing unnecessary function arguments and
deduplicating expressions
* Restructure job handlers to no longer need to pass a job's
handle to the job.
A typographical error would mishandle the case where a caller explicitly
tries to remove a child that is not actually part of the node. This case
is never invoked in practice, and so the bug would will never trigger.
Commit bf013c02ad added support
for incorporating a commit ID into the compiled version string
but did so in a way that did not follow the semantic versioning
standard.
This commit corrects that flaw by moving the commit ID into the
"metadata" part of the version string and properly handles the
case where the commit hash cannot be retrieved.
The pathfinding engine built into the code has several configurable
parameters to adjust the depth of the paths indexed and explored.
These parameters can dramatically impact the performance and memory
consumption of a server; higher values can result in resource usage
increasing exponentially.
These default values were decided early and somewhat arbitrarily at
a time when the network and the size of the network state were much
smaller.
This commit adjusts the default values to reduce the depth of paths
to more reasonable levels; unless explicitly overriden, the changes
mean that pathfinding operations will return fewer, shallower paths
than previous versions of the software.
This commit corrects a technical flaw that was introduced with commit
7c12f01358: as written, a mutex that is
intended to help provide synchronization for multiple threads as they
are each walking the map, is declared so that each thread is passed a
dangling reference to a unique mutex.
This commit hoists the mutex outside the thread creation loop, so all
threads use a single mutex and eliminating the dangling reference.
The nodestore includes a built-in cache to reduce the disk I/O
load but, by default, this cache was not initialized unless it
was explicitly configured by the server operator.
This commit introduces sensible defaults based on the server's
configured node size.
It remains possible to completely disable the cache if desired
by explicitly configuring it the cache size and age parameters
to 0:
[node_db]
...
cache_size = 0
cache_age = 0
- Only duplicate records from archive to writable during online_delete.
- Log duration of nodestore reads.
- Include nodestore counters in perf_log output.
- Remove gratuitous nodestore activity counting.
- Report initial sync duration in server_info and perfLog.
- Report state_accounting in perfLog.
- Make state_accounting durations more accurate.
- Parallel ledger loader.
- Config parameter to load ledgers on start.
1) Don't acquire so many nodes per pass. It's likely
far more than we need.
2) Right-size the finishedReads_ vector on passes other
than just the first.
* Sort by fee level (which is the current behavior) then by transaction
ID (hash).
* Edge case when the account at the end of the queue submits a higher
paying transaction to walk backwards and compare against the cheapest
transaction from a different account.
* Use std::if_any to simplify the JobQueue::isOverloaded loop.
* Log load fee values (at debug) received from validations.
* Log remote and cluster fee values (at trace) when changed.
* Refactor JobQueue::isOverloaded to return sooner if overloaded.
* Refactor Transactor::checkFee to only compute fee if ledger is open.
This flag, if present, suppresses the output of incoming
trustlines in default state.
This is primarily motivated by observing that users of Xumm often
have many unwanted incoming trustlines in a default state, which are
not useful in the vast majority of cases.
Being able to suppress those when doing `account_lines` saves bandwidth
and resources.
This commit implements partitioned unordered maps and makes it possible
to traverse such a map in parallel, allowing for more efficient use of
CPU resources.
The `CachedSLEs`, `TaggedCache`, and `KeyCache` classes make use of the
new functionality, which should improve performance.
The pathfinding engine requires pre-building large tables which is a
resource-intensive operation. Typically, one would not expect that a
server configured as a validator would also support pathfinding APIs
and so, building those tables by default wastes resources.
This commit, if merged, will disable pathfinding on servers that are
configured as validators, unless the server operator opts to support
it explicitly, by configuring the `[path_search_max]` parameter.
Validator operators that wish to support pathfinding on a validator
and want to use the default values can add the following stanza to
their server's configuration file:
[path_search_max]
7
The priority of different types of jobs was set back in the early
days of development, based on insight and observations that don't
necessarily apply any longer.
Specifically, job types used by the server to sync to the network
were being treated as lower priority than client requests, making
it more difficult to regain sync.
This commit adjusts the priority of several jobs and should allow
servers to prioritize resynchronizing to the network over serving
clients.
The existing calculation would limit the maximum number of threads
that would be created by default to at most 6; this may have been
reasonable a few years ago, but given both the load on the network
as of today and the increase in the number of CPU cores, the value
should be revisited.
This commit, if merged, changes the default calculation for nodes
that are configured as `large` or `huge` to allow for up to twelve
threads.
The "sweep interval" is the amount of time between successive sweeps of
of various in-memory data structures to remove stale items.
Prior to this commit, the interval was automatically adjusted, based on
the value of the `[node_size]` option in a server's configuration file.
If merged, this commit introduces a new configuration option that makes
it possible for a server operator to adjust the sweep interval and make
a CPU/memory tradeoff:
[sweep_interval]
<integer>
The specified value represents the number of seconds between successive
sweeps. The range of valid values is between 10 and 600.
Important operator notes:
This is an advanced configuration option that should not be used unless
there is empirical data which suggests that the default sweep frequency
is either resulting in performance problems or is causing undue load to
the server.
Note that adjusting the sweep interval may not have the intended effect
on the server. Lower values will not always translate to a reduction of
memory usage and higher values will not always translate to a reduction
of CPU usage and/or load.
The performance characteristics of `std::unordered_map` are better
than `std::map` and the former should be preferred when the strict
ordering of the latter is not required.
* Only require adding the new feature names in one place. (Also need to
increment a counter, but a check on startup will catch that.)
* Allows rippled to have the code to support a given amendment, but
not vote for it by default. This allows the amendment to be enabled in
a future version without necessarily amendment blocking these older
versions.
* The default vote is carried with the amendment name in the list of
supported amendments.
* The amendment table is constructed with the amendment and default
vote.
* Also clean up some formatting in the Windows instructions
* Changed the recommended version for Windows to 1.1.1L after deeper
checking uncovered some build issues.
* Patch the soci unsigned-types.h file. If no changes are made, delete
the patched file and exit. If there are changes, backup the original
and replace it with the patched file.
* Fixes#3885
Patch Rocksdb only once:
* The repeated patches do not appear to affect build times, but avoiding
unnecessary copies is good for its own sake.
There are two mutexes in ValidatorSite: `site_mutex_` and `state_mutex_`. Some
function end up locking both mutexes. However, depending on the call, the
mutexes could be locked in different orders, resulting in deadlocks.
If both mutexes are locked, this patch always locks the `sites_mutex_` first.
The existing logic involves every server sending every transaction
that it receives to all its peers (except the one that it received
a transaction from).
This commit instead uses a randomized algorithm, where a node will
randomly select peers to relay a given transaction to, caching the
list of transaction hashes that are not relayed and forwading them
to peers once every second. Peers can then determine whether there
are transactions that they have not seen and can request them from
the node which has them.
It is expected that this feature will further reduce the bandwidth
needed to operate a server.
The existing license file contains copyright statements and license
snippets referencing code that has long since been removed from the
codebase and which are not necessary any longer.
The general copyright statement for the ISC License is also changed
from "Ripple Labs" which is one contributor to the codebase, to the
more general "XRP Ledger Developers".
Remaining code which was originally taken from Bitcoin includes the
relevant copyright statement(s) inline.
To aid in the automated detection of the license type by GitHub and
tools like `licensee`, the following statement which referenced the
ISC license, and which was listed at the bottom of the file, is now
incorporated at the top:
>The accompanying files incorporate work covered by the following copyright
>and previous license notice:
>
>Copyright (c) 2011 Arthur Britto, David Schwartz, Jed McCaleb,
>Vinnie Falco, Bob Way, Eric Lombrozo, Nikolaos D. Bougalis, Howard Hinnant
This commit removes the `ltINVALID` pseudo-type identifier from
`LedgerEntryType` and the `ttINVALID` pseudo-type identifier from
`TxType` and includes several small additional improvements that
help to simplify the code base.
It also improves the documentation `LedgerEntryType` and `TxType`,
which was all over the place, and highlights some important caveats
associated with making changes to the ledger and transaction type
identifiers.
The commit also adds a safety check to the `KnownFormats<>` class,
that will catch the the accidental reuse of format identifiers.
Ideally, this should be done at compile time but C++ does not (yet?)
allow for the sort of introspection that would enable this.
The legacy functions `cdirFirst` and `dirFirst` were mostly
identical; the differences were only type-related. The same
situation existed with `cdirNext` and `dirNext`.
This commit removes the duplicated code by introducing new
template functions that abstract away the differences that
are present between each pair of functions.
This commit also improves the naming of function arguments,
helping to elucidate their purpose & use and to make the
code self-documenting.
The Negative UNL is a feature of the XRP Ledger consensus protocol that
improves liveness (the network's ability to make forward progress) during
a partial outage. Using the Negative UNL, servers adjust their effective
UNLs based on which validators are currently online and operational, so
that a new ledger version can be declared validated even if several trusted
validators are offline.
The Negative UNL has no impact on how the network processes transactions
or what transactions' outcomes are, except that it improves the network's
ability to declare outcomes final during some types of partial outages.
The feature was originally introduced with version **1.6.0** but it was
only possible to manually enable this. If merged, this commit introduces
the amendment associated with the feature so that server operators can
vote on whether to enable this feature.
For more details, please see https://xrpl.org/negative-unl.html
This commit closes#3898.
Under some circumstances, it is possible to induce an out-of-bounds
memory read in the base58 decoder.
This commit addresses this issue.
Acknowledgements:
Guido Vranken for discovering and responsibly disclosing this issue.
Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers to
responsibly disclose any issues they may find.
Ripple is generously sponsoring a bug bounty program for the
rippled project. For more information please visit:
https://ripple.com/bug-bounty
The HardenedValidations amendment introduces additional fields
in validations:
- `sfValidatedHash`, if present, is the hash the of last ledger that
the validator considers to be fully validated.
- `sfCookie`, if present, is a 64-bit cookie (the default
implementation selects it randomly at startup but other
implementations are possible), which can be used to improve the
detection and classification of duplicate validations.
- `sfServerVersion`, if present, reports the version of the software
that the validator is running. By surfacing this information,
server operators gain additional insight about variety of software
on the network.
If merged, this commit fixes#3797 by adding the fields to the
`validations` stream as shown below:
- `sfValidateHash` as `validated_hash`: a 256-bit hex string;
- `sfCookie` as `cookie`: a 64-bit integer as a string; and
- `sfServerVersion` as `server_version`: a 64-bit integer as
a string.
With this amendment, the CheckCash transaction creates a TrustLine
if needed. The change is modeled after offer crossing. And,
similar to offer crossing, cashing a check allows an account to
exceed its trust line limit.
The following changes were made:
- Removed dependency on template defined in beast detail namespace.
- Removed Section::find() method which had an obsolete interface.
- Made Section::get<>() easier to use for the common case of
retrieving a std::string. The revised get() method replaces old
calls to Section::find().
- Provided a default template parameter to free function
get<>(Section config, std::string name) so it stays similar to
Section::get<>().
Then the rest of the code was adapted to these changes.
- Calls to Section::find() were replaced with calls to Section::get.
- Unnecessary get<std::string>() arguments were reduced to get().
These changes dug up an interesting artifact in the SHAMap unit
tests. I'm not sure why the tests were working before, but there
was a problem with the case of a Section key. The unit test is
fixed.
The `[node_size]` configuration parameter is used to tune various
parameters based on the hardware that the code is running on. The
parameter can take five distinct values: `tiny`, `small`, `medium`,
`large` and `huge`.
The default value in the code is `tiny` but the default configuration
file sets the value to `medium`. This commit attempts to detect the
amount of RAM on the system and adjusts the node size default value
based on the amount of RAM and the number of hardware execution
threads on the system.
The decision matrix currently used is:
| | 1 | 2 or 3 | ≥ 4 |
|:-------:|:----:|:------:|:------:|
| > ~8GB | tiny | tiny | tiny |
| > ~12GB | tiny | small | small |
| > ~16GB | tiny | small | medium |
| > ~24GB | tiny | small | large |
| > ~32GB | tiny | small | huge |
Some systems exclude memory reserved by the the hardware, the kernel
or the underlying hypervisor so the automatic detection code may end
up determining the node_size to be one less than "appropriate" given
the above table.
The detection algorithm is simplistic and does not take into account
other relevant factors. Therefore, for production-quality servers it
is recommended that server operators examine the system holistically
and determine what the appropriate size is instead of relying on the
automatic detection code.
To aid server operators, the node size will now be reported in the
`server_info` API as `node_size` when the command is invoked in
'admin' mode.
A recent version of clang notes a number of places in range
for loops where the code base was making unnecessary copies
or using const lvalue references to extend lifetimes. This
fixes the places that clang identified.
* Create SQLite database for mapping transaction IDs to shard indexes
* Create SQLite database for mapping ledger hashes to shard indexes
* Create additional test cases for the shard database
* Add instructions to the workflow at the point where the failure would
occur, which is where someone experiencing a failure is most likely to
look. To keep things simple, the instructions are always printed. The
assumption is that if the job succeeds, nobody is likely to look
anyway.
* Provides the diff of the failure as an artifact, so the user can apply
it directly to their repo.
* Also update the levelization/README.md to clarify the levels a little
bit.
This updates the build process to use the local Artifactory server as a docker image cache to avoid being rate limited by docker hub during the build process.
While most of the code associated with secp256k1 operations had
been migrated to libsecp256k1, the deterministic key derivation
code was still using calls to OpenSSL.
If merged, this commit replaces the OpenSSL-based routines with
new libsecp256k1-based implementations. No functional change is
expected and the change should be transparent.
This commit also removes several support classes and utility
functions that wrapped or adapted various OpenSSL types that
are no longer needed.
A tip of the hat to the original author of this truly superb
library, Dr. Pieter Wuille, and to all other contributors.
This commit expands the detection capabilities of the Byzantine
validation detector. Prior to this commit, only validators that
were on a server's UNL were monitored. Now, all the validations
that a server receives are passed through the detector.
The existing class offered several constructors which were mostly
unnecessary. This commit eliminates all existing constructors and
introduces a single new one, taking a `Slice`.
The internal buffer is switched from `std::vector` to `Buffer` to
save a minimum of 8 bytes (plus the buffer slack that is inherent
in `std::vector`) per SHAMapItem instance.
Add support to allow multiple indepedent nodes to produce a binary identical
shard for a given range of ledgers. The advantage is that servers can use
content-addressable storage, and can more efficiently retrieve shards by
downloading from multiple peers at once and then verifying the integrity of
a shard by cross-checking its checksum with the checksum other servers report.
# This file is sorted in reverse chronological order, with the most recent commits at the top.
# The commits listed here are ignored by git blame, which is useful for formatting-only commits that would otherwise obscure the history of changes to a file.
*If there is any impact to the public API methods (HTTP / WebSocket), please update https://github.com/xrplf/rippled/blob/develop/API-CHANGELOG.md
*Update API-CHANGELOG.md and add the change directly in this PR by pushing to your PR branch.
*libxrpl: See https://github.com/XRPLF/rippled/blob/develop/docs/build/depend.md
*Peer Protocol: See https://xrpl.org/peer-protocol.html
-->
- [ ]Bug fix (non-breaking change which fixes an issue)
- [ ]New feature (non-breaking change which adds functionality)
- [ ]Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ]Refactor (non-breaking change that only restructures code)
- [ ] Tests (You added tests for code that already exists, or your new feature included in this PR)
- [ ] Documentation Updates
- [ ] Release
- [ ]Public API: New feature (new methods and/or new fields)
- [ ]Public API: Breaking change (in general, breaking changes should only impact the next api_version)
- [ ]`libxrpl` change (any change that may affect `libxrpl` or dependents of `libxrpl`)
- [ ]Peer protocol change (must be backward compatible or bump the peer protocol version)
<!--
## Before / After
If relevant, use this section for an English description of the change at a technical level.
If this change affects an API, examples should be included here.
For performance-impacting changes, please provide these details:
1. Is this a new feature, bug fix, or improvement to existing functionality?
2. What behavior/functionality does the change impact?
3. In what processing can the impact be measured? Be as specific as possible - e.g. RPC client call, payment transaction that involves LOB, AMM, caching, DB operations, etc.
4. Does this change affect concurrent processing - e.g. does it involve acquiring locks, multi-threaded processing, or async processing?
-->
<!--
@@ -51,5 +65,5 @@ This section may not be needed if your change includes thoroughly commented unit
| A | No stray dotted span-attribute key (only the derived resource keys may be dotted). |
| G | Attribute keys are `lower_snake_case` (`^[a-z][a-z0-9_]*$` per dot-segment) — no camelCase, UPPERCASE, or spaces. |
| F | No string literals as attribute keys or span-name arguments in `setAttribute`/`addEvent`/`span`/`rootSpan`/`childSpan` (`rootSpan` shares `span`'s `(cat, prefix, name)` signature). Attribute _values_ are exempt (runtime data); `*SpanNames.h` definitions and test files are exempt. |
| B | Every collector `spanmetrics.dimensions` name exists in the L1 key set. |
| C | Every Tempo span-filter tag exists in the L1 key set. |
| D | Every dashboard label resolves to an L1 span attribute, a native-metric label (L6, emitted by MetricsRegistry), or a Prometheus/Grafana builtin. TraceQL scope prefixes (`span.`/`resource.`/…) are stripped before the L1 lookup. |
| E | No dotted `xrpl.<domain>.<field>` attribute key in the runbook (only the L1 resource attrs `xrpl.network.*` may be dotted). Span names, filenames, OTel-standard keys, and metric labels are not flagged. |
Rule F runs **unconditionally** (it is a purely syntactic check on the
call-sites and needs no `*SpanNames.h`), so a code path that calls
`SpanGuard::span`/`setAttribute` directly without ever defining a header is
| H | A namespace-qualified constant (e.g. `foo::bar::myKey`) used at a telemetry call-site is not defined in any `*SpanNames.h`. The constant should live in the proper header; defining it in-place bypasses rules A/G/F. Warns rather than fails — the argument may be a legitimately dynamic value, and the header may live on a later branch. Bare locals and `std::` names are not warned. |
## Presence-gated
Every rule runs **only when the source files it needs are present** in the tree
and is otherwise skipped (printed as `SKIP: <rule> — <reason>`), never failed.
This keeps the check correct no matter how telemetry work is split across PRs —
a stacked chain, one large PR, or independent per-stage PRs where (for example)
the collector config lands before the dashboards. The collector/Tempo/dashboard/
runbook layers are introduced in later phases; on a branch without them, only
# Restore copyright notices that were removed from specific files, without
# restoring the verbiage that is already present in LICENSE.md. Ensure that if
# the script is run multiple times, duplicate notices are not added.
if ! grep -q 'Raw Material Software' include/xrpl/beast/core/CurrentThreadName.h;then
echo -e "// Portions of this file are from JUCE (http://www.juce.com).\n// Copyright (c) 2013 - Raw Material Software Ltd.\n// Please visit http://www.juce.com\n\n$(cat include/xrpl/beast/core/CurrentThreadName.h)" >include/xrpl/beast/core/CurrentThreadName.h
fi
if ! grep -q 'Dev Null' src/test/app/NetworkID_test.cpp;then
description:"Whether to smoke-run the benchmarks after testing."
required:false
type:boolean
default:false
build_only:
description:'Whether to only build or to build and test the code ("true", "false").'
required:true
type:boolean
build_type:
description:'The build type to use ("Debug", "Release").'
required:true
type:string
ccache_enabled:
description:"Whether to enable ccache."
required:false
type:boolean
default:false
cmake_args:
description:"Additional arguments to pass to CMake."
required:false
type:string
default:""
cmake_target:
description:"The CMake target to build."
required:true
type:string
runs_on:
description:Runner to run the job on as a JSON string
required:true
type:string
image:
description:"The image to run in (leave empty to run natively)"
required:true
type:string
config_name:
description:"The configuration string (used for naming artifacts and such)."
required:true
type:string
nproc_subtract:
description:"The number of processors to subtract when calculating parallelism."
required:false
type:number
default:2
sanitizers:
description:"The sanitizers to enable."
required:false
type:string
default:""
compiler:
description:'The compiler to use ("gcc" or "clang"). Leave empty for macOS/Windows (uses system default).'
required:false
type:string
default:""
toolchain:
description:'Where the toolchain comes from ("nix" to build the flake CI environment on the runner, empty for the system one). macOS only: Linux always builds in a Nix image, and Nix has no Windows support.'
required:false
type:string
default:""
secrets:
CODECOV_TOKEN:
description:"The Codecov token to use for uploading coverage reports."
required:true
defaults:
run:
shell:bash
env:
# Conan installs the generators in the build/generators directory, see the
# layout() method in conanfile.py. We then run CMake from the build directory.
# No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
# Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#build-times-out-because-no-output-was-received
This file provides guidance to AI coding agents (Claude Code, and other AGENTS.md-compatible tools) when working with code in this repository.
## Build
Required on Linux/macOS: use the Nix devshell, which sets up the compiler, Conan, ccache, and (optionally) Rust automatically.
```bash
nix develop
```
For alternate devshell variants (specific compiler, no-compiler, coverage), see [docs/build/nix.md](./docs/build/nix.md). For the manual build steps, CMake options, and protocol codegen commands, see [BUILD.md](./BUILD.md) (`## Steps`, `## Options`, `## Code generation`).
Rust crate tests (independent of the CMake build): `cargo test --manifest-path crates/Cargo.toml --workspace` (CI uses `cargo nextest`).
## Testing
Unit tests are a custom framework built into the `xrpld` binary itself (not Boost.Test/GTest/Catch); see [CONTRIBUTING.md](./CONTRIBUTING.md#unit-tests) for the basic invocation. Notes not covered there:
- A suite's `--unittest` name is built from the arguments to its `BEAST_DEFINE_TESTSUITE`/`BEAST_DEFINE_TESTSUITE_PRIO` macro (usually at the bottom of the test file), in reverse order and joined with `.`: `BEAST_DEFINE_TESTSUITE(Credentials, app, xrpl)` → `xrpl.app.Credentials`.
-`--unittest-arg` does nothing — don't use it.
- Tests that run offline in under a minute should be automatic `--unittest` suites; anything else is a manual/integration test.
- New tests should be written using `gtest` under `src/tests/` unless that isn't possible, in which case fall back to the legacy Beast framework under `src/test/`. `tests/` (top-level) holds integration tests exercised against `libxrpl`/`xrpld`.
## Lint/Format
See [CONTRIBUTING.md](./CONTRIBUTING.md#pre-commit-hooks) for `pre-commit` setup and [CONTRIBUTING.md](./CONTRIBUTING.md#clang-tidy) for `clang-tidy` (opt-in, needs local `clang-tidy` and generated headers).
## Code Style
New file placement and header levelization: see [CONTRIBUTING.md](./CONTRIBUTING.md#before-making-a-pull-request). Braces, whitespace, member order, and other conventions: see [docs/CodingStyle.md](./docs/CodingStyle.md). `XRPL_ASSERT`/`UNREACHABLE` contracts: see [CONTRIBUTING.md](./CONTRIBUTING.md#contracts-and-instrumentation). Commit messages: see [CONTRIBUTING.md](./CONTRIBUTING.md#good-commit-messages).
## Architecture
Paths below reflect the current layout; update this section if modularization moves a subsystem to a different directory.
-`include/xrpl/` + `src/libxrpl/` — the core protocol library: ledger, shamap, consensus, crypto, json, resource, nodestore, rdb, peerfinder, and `tx/` (transaction application: `Transactor.cpp`, `applySteps.cpp`, invariants, payment paths). `tx/transactors/` has one file per transaction type, grouped by subsystem: `escrow/`, `vault/`, `lending/`, `sponsor/`, `nft/`, `token/` (MPT), `payment_channel/`, `permissioned_domain/`, `dex/`, `oracle/`, `did/`, `credentials/`, `bridge/`, `check/`, `delegate/`, `account/`, `system/`. Any change to transaction-processing behavior must be gated behind an Amendment.
-`src/xrpld/` — the server application built on top of `libxrpl`: `app`, `core`, `overlay` (P2P networking), `peerfinder`, `perflog`, `rpc`, `shamap`. `main` builds an `ApplicationImp` implementing `Application`; most components hold a reference to it (`app_`), giving broad cross-component access — expect to trace call chains through `Application&`.
-`src/test/` — unit tests mirroring the subsystems above, plus `jtx/` (the transaction-building test DSL — e.g. `jtx/escrow.h`, `jtx/vault.h`, `jtx/sponsor.h`, `jtx/permissioned_dex.h`) and `unit_test/` (the custom test framework itself, derived from Beast).
-`src/tests/` — unit tests for `libxrpl` written in `gtest`, gradually replacing the `src/test` equivalents.
-`crates/` — a Rust workspace (only built with `-Dxrpld -Drust=ON`) bridged into C++ via `cxxbridge`/the `cxx` crate; currently just a `hello_world` interop scaffold. Requires the Rust toolchain pinned in `rust-toolchain.toml` (the Nix devshell provides it automatically).
This changelog is intended to list all updates to the [public API methods](https://xrpl.org/public-api-methods.html).
For info about how [API versioning](https://xrpl.org/request-formatting.html#api-versioning) works, including examples, please view the [XLS-22d spec](https://github.com/XRPLF/XRPL-Standards/discussions/54). For details about the implementation of API versioning, view the [implementation PR](https://github.com/XRPLF/rippled/pull/3155). API versioning ensures existing integrations and users continue to receive existing behavior, while those that request a higher API version will experience new behavior.
The API version controls the API behavior you see. This includes what properties you see in responses, what parameters you're permitted to send in requests, and so on. You specify the API version in each of your requests. When a breaking change is introduced to the `xrpld` API, a new version is released. To avoid breaking your code, you should set (or increase) your version when you're ready to upgrade.
The [commandline](https://xrpl.org/docs/references/http-websocket-apis/api-conventions/request-formatting/#commandline-format) always uses the latest API version. The command line is intended for ad-hoc usage by humans, not programs or automated scripts. The command line is not meant for use in production code.
For a log of breaking changes, see the **API Version [number]** headings. In general, breaking changes are associated with a particular API Version number. For non-breaking changes, scroll to the **XRP Ledger version [x.y.z]** headings. Non-breaking changes are associated with a particular XRP Ledger (`xrpld`) release.
## API Version 3 (Beta)
API version 3 is currently a beta API. It requires enabling `[beta_rpc_api]` in the xrpld configuration to use. See [API-VERSION-3.md](API-VERSION-3.md) for the full list of changes in API version 3.
## API Version 2
API version 2 is available in `xrpld` version 2.0.0 and later. See [API-VERSION-2.md](API-VERSION-2.md) for the full list of changes in API version 2.
## API Version 1
This version is supported by all `xrpld` versions. For WebSocket and HTTP JSON-RPC requests, it is currently the default API version used when no `api_version` is specified.
## XRP Ledger server version 3.4.0
Version 3.4.0 is not yet released. These changes are available in the 3.4.0 beta releases.
### Additions in 3.4.0
-`ledger`: `nftoken_id`, `nftoken_ids`, and `offer_id` are now included in transaction metadata when transactions are expanded (`expand`, or admin-only `full`), matching the `tx`, `account_tx`, and `subscribe` (`transactions` stream) responses. ([#5706](https://github.com/XRPLF/rippled/pull/5706))
### Bugfixes in 3.4.0
-`sign`, `sign_for`, `submit`: `signature_target` now returns `invalidParams` unless it names `CounterpartySignature` or `SponsorSignature`. It previously accepted any inner object field, such as `Book` or `NFToken`, and signed into it.
-`sign`, `sign_for`, `submit`, `submit_multisigned`: With `fixCleanup3_4_0` enabled, a signature in `CounterpartySignature` or `SponsorSignature` covers a different prefix than the transaction's own signature, so a signature can no longer be moved from one of those roles into another. Clients that build these signatures themselves must use the new prefixes: `CPT` and `CPM` (single- and multi-signing) for `CounterpartySignature`, and `SPN` and `SPM` for `SponsorSignature`.
-`get_aggregate_price`: Duplicate entries in the `oracles` request array are now ignored. [#6586](https://github.com/XRPLF/rippled/pull/6586)
-`vault_info`: Errors now identify what the request got wrong instead of reporting every failure as the unregistered token `malformedRequest`, and the `error`, `error_code` and `error_message` fields now agree with each other. An invalid `vault_id` or `seq` returns `invalidParams`, an invalid `owner` returns `actMalformed`, and a request that mixes `vault_id` with `owner`/`seq` or supplies neither returns `invalidParams` with a message naming the accepted combinations. [#8015](https://github.com/XRPLF/rippled/pull/8015)
-`vault_info`: A well-formed all-zero `vault_id` now returns `entryNotFound` instead of being rejected as malformed, and `entryNotFound` responses now include `error_code` and `error_message`. Clients that request `ripplerpc` 3.0 or above therefore receive HTTP 400 with that error rather than HTTP 200. [#8015](https://github.com/XRPLF/rippled/pull/8015)
-`vault_info`: `vault_id` and `owner` must now be strings, matching how `ledger_entry` reads the same fields. An object or an array in either field previously produced an internal error, and a number was silently converted to its decimal text; `vault_id` now returns `invalidParams` and `owner` returns `actMalformed`. [#8015](https://github.com/XRPLF/rippled/pull/8015)
-`gateway_balances`: The `account` and `ident` fields now return an `invalidParams` error if the value is not a string, instead of an `internal` error. [#7655](https://github.com/XRPLF/rippled/pull/7655)
-`account_lines`: The `peer` field now returns an error if the value is not a string. [#7728](https://github.com/XRPLF/rippled/pull/7728)
-`ledger`: `delivered_amount` is now included in the metadata of successful `AccountDelete` transactions when transactions are expanded (`expand`, or admin-only `full`). Previously it was only added for `Payment` and `CheckCash`, which made `ledger` inconsistent with `tx` and `account_tx`. [#5706](https://github.com/XRPLF/rippled/pull/5706)
-`noripple_check`: The `transactions` field is no longer included in error responses; it is still returned (possibly as an empty array) whenever `transactions` is `true` and the request succeeds. A malformed `account` is now rejected before the ledger is looked up, so that error response no longer carries the `ledger_hash`, `ledger_index`, and `validated` fields ([#6303](https://github.com/XRPLF/rippled/pull/6303)).
## XRP Ledger server version 3.3.0
[Version 3.3.0](https://github.com/XRPLF/rippled/releases/tag/3.3.0) was released on Aug 6, 2026.
### Additions in 3.3.0
-`account_tx`: Added an optional `delegate` request object to filter delegated transactions. The object requires `delegate_filter`, which must be either `actor` for transactions owned by the requested account but signed by another account, or `authorizer` for transactions signed by the requested account on behalf of another account. The optional `counter_party` account narrows the results to a specific signer/delegate for `actor` or a specific owner/delegator for `authorizer`. Malformed `delegate`, `delegate_filter`, and `counter_party` values return standard invalid field errors, and invalid account IDs return `actMalformed`. When paginating delegate-filtered queries, a marker from a delegate-filtered query includes a `delegate` flag and is only valid for follow-up requests that also supply `delegate` (mixing marker conventions returns `invalidParams`). Because filtering is applied after the ledger scan, a page may contain fewer results than `limit` (possibly zero) while still returning a marker, so callers must continue until no marker is present. ([#6126](https://github.com/XRPLF/rippled/pull/6126))
## XRP Ledger server version 3.2.1
[Version 3.2.1](https://github.com/XRPLF/rippled/releases/tag/3.2.1) was released on Aug 1, 2026.
This release contains bug fixes only and no API changes.
## XRP Ledger server version 3.2.0
[Version 3.2.0](https://github.com/XRPLF/rippled/releases/tag/3.2.0) was released on Jun 16, 2026.
### Additions in 3.2.0
-`ledger_entry`, `account_objects`: The `Delegate` ledger entry now includes an optional `DestinationNode` field, which stores the index into the authorized account's owner directory. This field is present on entries created after bidirectional directory tracking was introduced and may appear in RPC responses for those entries. ([#6681](https://github.com/XRPLF/rippled/pull/6681))
-`server_definitions`: Added the following new sections to the response ([#6321](https://github.com/XRPLF/rippled/pull/6321)):
-`TRANSACTION_FORMATS`: Describes the fields and their optionality for each transaction type, including common fields shared across all transactions.
-`LEDGER_ENTRY_FORMATS`: Describes the fields and their optionality for each ledger entry type, including common fields shared across all ledger entries.
-`TRANSACTION_FLAGS`: Maps transaction type names to their supported flags and flag values.
-`LEDGER_ENTRY_FLAGS`: Maps ledger entry type names to their flags and flag values.
-`ACCOUNT_SET_FLAGS`: Maps AccountSet flag names (asf flags) to their numeric values.
### Bugfixes in 3.2.0
- Peer Crawler: The `port` field in `overlay.active[]` now consistently returns an integer instead of a string for outbound peers. [#6318](https://github.com/XRPLF/rippled/pull/6318)
-`ping`: The `ip` field is no longer returned as an empty string for proxied connections without a forwarded-for header. It is now omitted, consistent with the behavior for identified connections. [#6730](https://github.com/XRPLF/rippled/pull/6730)
- gRPC `GetLedgerDiff`: Fixed error message that incorrectly said "base ledger not validated" when the desired ledger was not validated. [#6730](https://github.com/XRPLF/rippled/pull/6730)
-`account_channels`: The `destination_account` field now returns an error if the value is not a string. [#6529](https://github.com/XRPLF/rippled/pull/6529)
-`subscribe`: The `taker` field in the `books` array now returns an error if the value is not a string. [#6529](https://github.com/XRPLF/rippled/pull/6529)
-`account_info`: The `urlgravatar` field now uses HTTPS instead of HTTP. [#6529](https://github.com/XRPLF/rippled/pull/6529)
-`ledger`: The `full`, `accounts`, `transactions`, `expand`, `binary`, `owner_funds`, and `queue` fields now return an error if the value is not a boolean. [#6529](https://github.com/XRPLF/rippled/pull/6529)
-`ledger_data`: The `binary` field now returns an error if the value is not a boolean. [#6529](https://github.com/XRPLF/rippled/pull/6529)
-`submit`: The `fail_hard` field now returns an error if the value is not a boolean. [#6529](https://github.com/XRPLF/rippled/pull/6529)
-`subscribe`: The `taker` field in the `books` array now returns `actMalformed` instead of `badIssuer` if the value is not a valid account. [#6529](https://github.com/XRPLF/rippled/pull/6529)
- Fixed a bug in `Forwarded` HTTP header parsing where the extracted IP address could be incorrect when no comma or semicolon delimiter follows the address. This could cause the server to misidentify a client's IP address when operating behind a reverse proxy. [#6529](https://github.com/XRPLF/rippled/pull/6529)
## XRP Ledger server version 3.1.3
[Version 3.1.3](https://github.com/XRPLF/rippled/releases/tag/3.1.3) was released on May 8, 2026.
This release contains bug fixes only and no API changes.
## XRP Ledger server version 3.1.2
[Version 3.1.2](https://github.com/XRPLF/rippled/releases/tag/3.1.2) was released on Mar 12, 2026.
This release contains bug fixes only and no API changes.
## XRP Ledger server version 3.1.1
[Version 3.1.1](https://github.com/XRPLF/rippled/releases/tag/3.1.1) was released on Feb 23, 2026.
This release contains bug fixes only and no API changes.
## XRP Ledger server version 3.1.0
[Version 3.1.0](https://github.com/XRPLF/rippled/releases/tag/3.1.0) was released on Jan 27, 2026.
### Additions in 3.1.0
-`vault_info`: New RPC method to retrieve information about a specific vault (part of XLS-66 Lending Protocol). ([#6156](https://github.com/XRPLF/rippled/pull/6156))
## XRP Ledger server version 3.0.0
[Version 3.0.0](https://github.com/XRPLF/rippled/releases/tag/3.0.0) was released on Dec 9, 2025.
### Additions in 3.0.0
-`ledger_entry`: Supports all ledger entry types with dedicated parsers. ([#5237](https://github.com/XRPLF/rippled/pull/5237))
-`ledger_entry`: New error codes `entryNotFound` and `unexpectedLedgerType` for more specific error handling. ([#5237](https://github.com/XRPLF/rippled/pull/5237))
-`ledger_entry`: Improved error messages with more context (e.g., specifying which field is invalid or missing). ([#5237](https://github.com/XRPLF/rippled/pull/5237))
-`ledger_entry`: Assorted bug fixes in RPC processing. ([#5237](https://github.com/XRPLF/rippled/pull/5237))
-`simulate`: Supports additional metadata in the response. ([#5754](https://github.com/XRPLF/rippled/pull/5754))
## XRP Ledger server version 2.6.2
[Version 2.6.2](https://github.com/XRPLF/rippled/releases/tag/2.6.2) was released on Nov 19, 2025.
This release contains bug fixes only and no API changes.
## XRP Ledger server version 2.6.1
[Version 2.6.1](https://github.com/XRPLF/rippled/releases/tag/2.6.1) was released on Sep 30, 2025.
This release contains bug fixes only and no API changes.
## XRP Ledger server version 2.6.0
[Version 2.6.0](https://github.com/XRPLF/rippled/releases/tag/2.6.0) was released on Aug 27, 2025.
### Additions in 2.6.0
-`account_info`: Added `allowTrustLineLocking` flag in response. ([#5525](https://github.com/XRPLF/rippled/pull/5525))
-`ledger`: Removed the type filter from the RPC command. ([#4934](https://github.com/XRPLF/rippled/pull/4934))
-`subscribe` (`validations` stream): `network_id` is now included. ([#5579](https://github.com/XRPLF/rippled/pull/5579))
-`subscribe` (`transactions` stream): `nftoken_id`, `nftoken_ids`, and `offer_id` are now included in transaction metadata. ([#5230](https://github.com/XRPLF/rippled/pull/5230))
## XRP Ledger server version 2.5.1
[Version 2.5.1](https://github.com/XRPLF/rippled/releases/tag/2.5.1) was released on Sep 17, 2025.
This release contains bug fixes only and no API changes.
## XRP Ledger server version 2.5.0
[Version 2.5.0](https://github.com/XRPLF/rippled/releases/tag/2.5.0) was released on Jun 24, 2025.
### Additions and bugfixes in 2.5.0
-`tx`: Added `ctid` field to the response and improved error handling. ([#4738](https://github.com/XRPLF/rippled/pull/4738))
-`ledger_entry`: Improved error messages in `permissioned_domain`. ([#5344](https://github.com/XRPLF/rippled/pull/5344))
-`channel_authorize`: If `signing_support` is not enabled in the config, the RPC is disabled. ([#5385](https://github.com/XRPLF/rippled/pull/5385))
-`subscribe` (admin): Removed webhook queue limit to prevent dropping notifications; reduced HTTP timeout from 10 minutes to 30 seconds. ([#5163](https://github.com/XRPLF/rippled/pull/5163))
-`ledger_data` (gRPC): Fixed crashing issue with some invalid markers. ([#5137](https://github.com/XRPLF/rippled/pull/5137))
-`account_lines`: Fixed error with `no_ripple` and `no_ripple_peer` sometimes showing up incorrectly. ([#5345](https://github.com/XRPLF/rippled/pull/5345))
-`account_tx`: Fixed issue with incorrect CTIDs. ([#5408](https://github.com/XRPLF/rippled/pull/5408))
## XRP Ledger server version 2.4.0
[Version 2.4.0](https://github.com/XRPLF/rippled/releases/tag/2.4.0) was released on March 4, 2025.
### Additions and bugfixes in 2.4.0
-`simulate`: A new RPC that executes a [dry run of a transaction submission](https://github.com/XRPLF/XRPL-Standards/tree/master/XLS-0069d-simulate#2-rpc-simulate). ([#5069](https://github.com/XRPLF/rippled/pull/5069))
- Signing methods (`sign`, `sign_for`, `submit`): Autofill fees better, properly handle transactions without a base fee, and autofill the `NetworkID` field. ([#5069](https://github.com/XRPLF/rippled/pull/5069))
-`ledger_entry`: `state` is added as an alias for `ripple_state`. ([#5199](https://github.com/XRPLF/rippled/pull/5199))
-`ledger`, `ledger_data`, `account_objects`: Support filtering ledger entry types by their canonical names (case-insensitive). ([#5271](https://github.com/XRPLF/rippled/pull/5271))
-`validators`: Added new field `validator_list_threshold` in response. ([#5112](https://github.com/XRPLF/rippled/pull/5112))
-`server_info`: Added git commit hash info on admin connection. ([#5225](https://github.com/XRPLF/rippled/pull/5225))
-`server_definitions`: Changed larger `UInt` serialized types to `Hash`. ([#5231](https://github.com/XRPLF/rippled/pull/5231))
## XRP Ledger server version 2.3.1
[Version 2.3.1](https://github.com/XRPLF/rippled/releases/tag/2.3.1) was released on Jan 29, 2025.
This release contains bug fixes only and no API changes.
## XRP Ledger server version 2.3.0
[Version 2.3.0](https://github.com/XRPLF/rippled/releases/tag/2.3.0) was released on Nov 25, 2024.
### Breaking changes in 2.3.0
-`book_changes`: If the requested ledger version is not available on this node, a `ledgerNotFound` error is returned and the node does not attempt to acquire the ledger from the p2p network (as with other non-admin RPCs). Admins can still attempt to retrieve old ledgers with the `ledger_request` RPC.
### Additions and bugfixes in 2.3.0
-`book_changes`: Returns a `validated` field in its response. ([#5096](https://github.com/XRPLF/rippled/pull/5096))
-`book_changes`: Accepts shortcut strings (`current`, `closed`, `validated`) for the `ledger_index` parameter. ([#5096](https://github.com/XRPLF/rippled/pull/5096))
-`server_definitions`: Include `index` in response. ([#5190](https://github.com/XRPLF/rippled/pull/5190))
-`account_nfts`: Fix issue where unassociated marker would return incorrect results. ([#5045](https://github.com/XRPLF/rippled/pull/5045))
-`account_objects`: Fix issue where invalid marker would not return an error. ([#5046](https://github.com/XRPLF/rippled/pull/5046))
-`account_objects`: Disallow filtering by ledger entry types that an account cannot hold. ([#5056](https://github.com/XRPLF/rippled/pull/5056))
-`feature`: Better error handling for invalid values of `feature`. ([#5063](https://github.com/XRPLF/rippled/pull/5063))
## XRP Ledger server version 2.2.0
[Version 2.2.0](https://github.com/XRPLF/rippled/releases/tag/2.2.0) was released on Jun 5, 2024. The following additions are non-breaking (because they are purely additive):
-`feature`: Add a non-admin mode for users. (It was previously only available to admin connections.) The method returns an updated list of amendments, including their names and other information. ([#4781](https://github.com/XRPLF/rippled/pull/4781))
## XRP Ledger server version 2.0.1
[Version 2.0.1](https://github.com/XRPLF/rippled/releases/tag/2.0.1) was released on Jan 29, 2024. The following additions are non-breaking:
[Version 2.0.0](https://github.com/XRPLF/rippled/releases/tag/2.0.0) was released on Jan 9, 2024. The following additions are non-breaking (because they are purely additive):
-`server_definitions`: A new RPC that generates a `definitions.json`-like output that can be used in XRPL libraries.
- In `Payment` transactions, `DeliverMax` has been added. This is a replacement for the `Amount` field, which should not be used. Typically, the `delivered_amount` (in transaction metadata) should be used. To ease the transition, `DeliverMax` is present regardless of API version, since adding a field is non-breaking.
- API version 2 has been moved from beta to supported, meaning that it is generally available (regardless of the `beta_rpc_api` setting). The full list of changes is in [API-VERSION-2.md](API-VERSION-2.md).
## XRP Ledger server version 1.12.0
[Version 1.12.0](https://github.com/XRPLF/rippled/releases/tag/1.12.0) was released on Sep 6, 2023. The following additions are non-breaking (because they are purely additive):
-`server_info`: Added `ports`, an array which advertises the RPC and WebSocket ports. This information is also included in the `/crawl` endpoint (which calls `server_info` internally). `grpc` and `peer` ports are also included. ([#4427](https://github.com/XRPLF/rippled/pull/4427))
-`ports` contains objects, each containing a `port` for the listening port (a number string), and a `protocol` array listing the supported protocols on that port.
- This allows crawlers to build a more detailed topology without needing to port-scan nodes.
- (For peers and other non-admin clients, the info about admin ports is excluded.)
- Clawback: The following additions are gated by the Clawback amendment (`featureClawback`). ([#4553](https://github.com/XRPLF/rippled/pull/4553))
- Adds an [AccountRoot flag](https://xrpl.org/accountroot.html#accountroot-flags) called `lsfAllowTrustLineClawback`. ([#4617](https://github.com/XRPLF/rippled/pull/4617))
- Adds the corresponding `asfAllowTrustLineClawback` [AccountSet Flag](https://xrpl.org/accountset.html#accountset-flags) as well.
- Clawback is disabled by default, so if an issuer desires the ability to claw back funds, they must use an `AccountSet` transaction to set the AllowTrustLineClawback flag. They must do this before creating any trust lines, offers, escrows, payment channels, or checks.
- Adds the [Clawback transaction type](https://github.com/XRPLF/XRPL-Standards/blob/master/XLS-39d-clawback/README.md#331-clawback-transaction), containing these fields:
-`Account`: The issuer of the asset being clawed back. Must also be the sender of the transaction.
-`Amount`: The amount being clawed back, with the `Amount.issuer` being the token holder's address.
`tfWithdrawAll`, `tfOneAssetWithdrawAll` which allow a trader to specify different fields combination
for `AMMDeposit` and `AMMWithdraw` transactions.
- Adds new transaction result codes:
- tecUNFUNDED_AMM: insufficient balance to fund AMM. The account does not have funds for liquidity provision.
- tecAMM_BALANCE: AMM has invalid balance. Calculated balances greater than the current pool balances.
- tecAMM_FAILED: AMM transaction failed. Fails due to a processing failure.
- tecAMM_INVALID_TOKENS: AMM invalid LP tokens. Invalid input values, format, or calculated values.
- tecAMM_EMPTY: AMM is in empty state. Transaction requires AMM in non-empty state (LP tokens > 0).
- tecAMM_NOT_EMPTY: AMM is not in empty state. Transaction requires AMM in empty state (LP tokens == 0).
- tecAMM_ACCOUNT: AMM account. Clawback of AMM account.
- tecINCOMPLETE: Some work was completed, but more submissions required to finish. AMMDelete partially deletes the trustlines.
## XRP Ledger server version 1.11.0
[Version 1.11.0](https://github.com/XRPLF/rippled/releases/tag/1.11.0) was released on Jun 20, 2023.
### Breaking changes in 1.11
- Added the ability to mark amendments as obsolete. For the `feature` admin API, there is a new possible value for the `vetoed` field. ([#4291](https://github.com/XRPLF/rippled/pull/4291))
- The value of `vetoed` can now be `true`, `false`, or `"Obsolete"`.
- Removed the acceptance of seeds or public keys in place of account addresses. ([#4404](https://github.com/XRPLF/rippled/pull/4404))
- This simplifies the API and encourages better security practices (i.e. seeds should never be sent over the network).
- For the `ledger_data` method, when all entries are filtered out, the `state` field of the response is now an empty list (in other words, an empty array, `[]`). (Previously, it would return `null`.) While this is technically a breaking change, the new behavior is consistent with the documentation, so this is considered only a bug fix. ([#4398](https://github.com/XRPLF/rippled/pull/4398))
- If and when the `fixNFTokenRemint` amendment activates, there will be a new AccountRoot field, `FirstNFTSequence`. This field is set to the current account sequence when the account issues their first NFT. If an account has not issued any NFTs, then the field is not set. ([#4406](https://github.com/XRPLF/rippled/pull/4406))
- There is a new account deletion restriction: an account can only be deleted if `FirstNFTSequence` + `MintedNFTokens` + `256` is less than the current ledger sequence.
- This is potentially a breaking change if clients have logic for determining whether an account can be deleted.
- NetworkID
- For sidechains and networks with a network ID greater than 1024, there is a new [transaction common field](https://xrpl.org/transaction-common-fields.html), `NetworkID`. ([#4370](https://github.com/XRPLF/rippled/pull/4370))
- This field helps to prevent replay attacks and is now required for chains whose network ID is 1025 or higher.
- The field must be omitted for Mainnet, so there is no change for Mainnet users.
- There are three new local error codes:
-`telNETWORK_ID_MAKES_TX_NON_CANONICAL`: a `NetworkID` is present but the chain's network ID is less than 1025. Remove the field from the transaction, and try again.
-`telREQUIRES_NETWORK_ID`: a `NetworkID` is required, but is not present. Add the field to the transaction, and try again.
-`telWRONG_NETWORK`: a `NetworkID` is specified, but it is for a different network. Submit the transaction to a different server which is connected to the correct network.
### Additions and bug fixes in 1.11
- Added `nftoken_id`, `nftoken_ids` and `offer_id` meta fields into NFT `tx` and `account_tx` responses. ([#4447](https://github.com/XRPLF/rippled/pull/4447))
- Added an `account_flags` object to the `account_info` method response. ([#4459](https://github.com/XRPLF/rippled/pull/4459))
- Added `NFTokenPages` to the `account_objects` RPC. ([#4352](https://github.com/XRPLF/rippled/pull/4352))
- Fixed: `marker` returned from the `account_lines` command would not work on subsequent commands. ([#4361](https://github.com/XRPLF/rippled/pull/4361))
- If the `XRPFees` feature is enabled, the `fee_ref` field will be
removed from the [ledger subscription stream](https://xrpl.org/subscribe.html#ledger-stream), because it will no longer
have any meaning.
# Unit tests for API changes
The following information is useful to developers contributing to this project:
The purpose of unit tests is to catch bugs and prevent regressions. In general, it often makes sense to create a test function when there is a breaking change to the API. For APIs that have changed in a new API version, the tests should be modified so that both the prior version and the new version are properly tested.
To take one example: for `account_info` version 1, WebSocket and JSON-RPC behavior should be tested. The latest API version, i.e. API version 2, should be tested over WebSocket, JSON-RPC, and command line.
API version 2 is available in `xrpld` version 2.0.0 and later. To use this API, clients specify `"api_version" : 2` in each request.
For info about how [API versioning](https://xrpl.org/request-formatting.html#api-versioning) works, including examples, please view the [XLS-22d spec](https://github.com/XRPLF/XRPL-Standards/discussions/54). For details about the implementation of API versioning, view the [implementation PR](https://github.com/XRPLF/rippled/pull/3155). API versioning ensures existing integrations and users continue to receive existing behavior, while those that request a higher API version will experience new behavior.
## Removed methods
In API version 2, the following deprecated methods are no longer available: ([#4759](https://github.com/XRPLF/rippled/pull/4759))
-`tx_history` - Instead, use other methods such as `account_tx` or `ledger` with the `transactions` field set to `true`.
-`ledger_header` - Instead, use the `ledger` method.
## Modifications to JSON transaction element in API version 2
In API version 2, JSON elements for transaction output have been changed and made consistent for all methods which output transactions. ([#4775](https://github.com/XRPLF/rippled/pull/4775))
This helps to unify the JSON serialization format of transactions. ([clio#722](https://github.com/XRPLF/clio/issues/722), [#4727](https://github.com/XRPLF/rippled/issues/4727))
- JSON transaction element is named `tx_json`
- Binary transaction element is named `tx_blob`
- JSON transaction metadata element is named `meta`
- Binary transaction metadata element is named `meta_blob`
Additionally, these elements are now consistently available next to `tx_json` (i.e. sibling elements), where possible:
-`hash` - Transaction ID. This data was stored inside transaction output in API version 1, but in API version 2 is a sibling element.
-`ledger_index` - Ledger index (only set on validated ledgers)
-`ledger_hash` - Ledger hash (only set on closed or validated ledgers)
-`close_time_iso` - Ledger close time expressed in ISO 8601 time format (only set on validated ledgers)
-`validated` - Bool element set to `true` if the transaction is in a validated ledger, otherwise `false`
This change affects the following methods:
-`tx` - Transaction data moved into element `tx_json` (was inline inside `result`) or, if binary output was requested, moved from `tx` to `tx_blob`. Renamed binary transaction metadata element (if it was requested) from `meta` to `meta_blob`. Changed location of `hash` and added new elements
-`account_tx` - Renamed transaction element from `tx` to `tx_json`. Renamed binary transaction metadata element (if it was requested) from `meta` to `meta_blob`. Changed location of `hash` and added new elements
-`transaction_entry` - Renamed transaction metadata element from `metadata` to `meta`. Changed location of `hash` and added new elements
-`subscribe` - Renamed transaction element from `transaction` to `tx_json`. Changed location of `hash` and added new elements
-`sign`, `sign_for`, `submit` and `submit_multisigned` - Changed location of `hash` element.
## Modifications to `Payment` transaction JSON schema
When reading Payments, the `Amount` field should generally **not** be used. Instead, use [delivered_amount](https://xrpl.org/partial-payments.html#the-delivered_amount-field) to see the amount that the Payment delivered. To clarify its meaning, the `Amount` field is being renamed to `DeliverMax`. ([#4733](https://github.com/XRPLF/rippled/pull/4733))
- In `Payment` transaction type, JSON RPC field `Amount` is renamed to `DeliverMax`. To enable smooth client transition, `Amount` is still handled, as described below: ([#4733](https://github.com/XRPLF/rippled/pull/4733))
- On JSON RPC input (e.g. `submit_multisigned` etc. methods), `Amount` is recognized as an alias to `DeliverMax` for both API version 1 and version 2 clients.
- On JSON RPC input, submitting both `Amount` and `DeliverMax` fields is allowed _only_ if they are identical; otherwise such input is rejected with `rpcINVALID_PARAMS` error.
- On JSON RPC output (e.g. `subscribe`, `account_tx` etc. methods), `DeliverMax` is present in both API version 1 and version 2.
- On JSON RPC output, `Amount` is only present in API version 1 and _not_ in version 2.
## Modifications to account_info response
-`signer_lists` is returned in the root of the response. (In API version 1, it was nested under `account_data`.) ([#3770](https://github.com/XRPLF/rippled/pull/3770))
- When using an invalid `signer_lists` value, the API now returns an "invalidParams" error. ([#4585](https://github.com/XRPLF/rippled/pull/4585))
- (`signer_lists` must be a boolean. In API version 1, strings were accepted and may return a normal response - i.e. as if `signer_lists` were `true`.)
## Modifications to [account_tx](https://xrpl.org/account_tx.html#account_tx) response
- Using `ledger_index_min`, `ledger_index_max`, and `ledger_index` returns `invalidParams` because if you use `ledger_index_min` or `ledger_index_max`, then it does not make sense to also specify `ledger_index`. In API version 1, no error was returned. ([#4571](https://github.com/XRPLF/rippled/pull/4571))
- The same applies for `ledger_index_min`, `ledger_index_max`, and `ledger_hash`. ([#4545](https://github.com/XRPLF/rippled/issues/4545#issuecomment-1565065579))
- Using a `ledger_index_min` or `ledger_index_max` beyond the range of ledgers that the server has:
- returns `lgrIdxMalformed` in API version 2. Previously, in API version 1, no error was returned. ([#4288](https://github.com/XRPLF/rippled/issues/4288))
- Attempting to use a non-boolean value (such as a string) for the `binary` or `forward` parameters returns `invalidParams` (`rpcINVALID_PARAMS`). Previously, in API version 1, no error was returned. ([#4620](https://github.com/XRPLF/rippled/pull/4620))
## Modifications to [noripple_check](https://xrpl.org/noripple_check.html#noripple_check) response
- Attempting to use a non-boolean value (such as a string) for the `transactions` parameter returns `invalidParams` (`rpcINVALID_PARAMS`). Previously, in API version 1, no error was returned. ([#4620](https://github.com/XRPLF/rippled/pull/4620))
API version 3 is currently a **beta API**. It requires enabling `[beta_rpc_api]` in the xrpld configuration to use. To use this API, clients specify `"api_version" : 3` in each request.
For info about how [API versioning](https://xrpl.org/request-formatting.html#api-versioning) works, including examples, please view the [XLS-22d spec](https://github.com/XRPLF/XRPL-Standards/discussions/54). For details about the implementation of API versioning, view the [implementation PR](https://github.com/XRPLF/rippled/pull/3155). API versioning ensures existing integrations and users continue to receive existing behavior, while those that request a higher API version will experience new behavior.
## Breaking Changes
### Modifications to `amm_info`
The order of error checks has been changed to provide more specific error messages. ([#4924](https://github.com/XRPLF/rippled/pull/4924))
- **Before (API v2)**: When sending an invalid account or asset to `amm_info` while other parameters are not set as expected, the method returns a generic `rpcINVALID_PARAMS` error.
- **After (API v3)**: The same scenario returns a more specific error: `rpcISSUE_MALFORMED` for malformed assets or `rpcACT_MALFORMED` for malformed accounts.
### Modifications to `ledger_entry`
Added support for string shortcuts to look up fixed-location ledger entries using the `"index"` parameter. ([#5644](https://github.com/XRPLF/rippled/pull/5644))
In API version 3, the following string values can be used with the `"index"` parameter:
-`"index": "amendments"` - Returns the `Amendments` ledger entry
-`"index": "fee"` - Returns the `FeeSettings` ledger entry
-`"index": "nunl"` - Returns the `NegativeUNL` ledger entry
| These instructions assume you have a C++ development environment ready with Git, Python, Conan, CMake, and a C++ compiler. For help setting one up on Linux, macOS, or Windows, [see this guide](./docs/build/environment.md).<br><br>These instructions also assume a basic familiarity with Conan and CMake. If you are unfamiliar with Conan, you can read our [crash course](./docs/build/conan.md) or the official [Getting Started][conan-getting-started] walkthrough. |
The Ubuntu Linux distribution has received the highest level of quality
assurance, testing, and support. We also support Red Hat and use Debian
internally.
Our Linux CI tooling is distro-independent and uses a Nix-based environment, so it should be possible to build on other Linux distributions as well, although we have not tested them.
### macOS
Many `xrpld` engineers use macOS for development.
The minimum supported version is macOS 15 (Sequoia).
CI testing is done in macOS 26 (Tahoe), but the build defaults `CMAKE_OSX_DEPLOYMENT_TARGET` to 15.
### Windows
Windows is used by some engineers for development only, and is not recommended
for production use.
## Steps
### Branches
For the latest set of untested features, or to contribute, choose the `develop`
branch.
```bash
git checkout develop
```
For a release candidate, choose the relevant release branch, e.g.
You can then build and test as usual, with the generated `xrpld` binary containing the sanitizer instrumentation. When you run it, it will report any sanitizer errors it detects in the console output.
See [Sanitizers docs](./docs/build/sanitizers.md) for more details.
If you're seeing an error like `ERROR: Package 'snappy/1.1.10' not resolved: Unable to find 'snappy/1.1.10#968fef506ff261592ec30c574d4a7809%1756234314.246' in remotes.`,
please [set Conan up](#set-up-conan) so the `xrplf` remote is configured, or re-run `conan export` for [patched recipes](./docs/build/advanced_conan.md#patched-recipes).
### `protobuf/port_def.inc` file not found
If `cmake --build .` results in an error due to a missing a protobuf file, then
you might have generated CMake files for a different `build_type` than the
`CMAKE_BUILD_TYPE` you passed to Conan.
```
/xrpld/.build/pb-xrpl.libpb/xrpl/proto/xrpl.pb.h:10:10: fatal error: 'google/protobuf/port_def.inc' file not found
10 | #include <google/protobuf/port_def.inc>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
```
For example, if you want to build Debug:
1. For conan install, pass `--settings build_type=Debug`
message(FATAL_ERROR"unable to determine rippled version")
endif()
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
