Four files conflicted.
xrpl.proto and TraceContextPropagator.h: phase-9's note is a semantic
superset of phase-8's and already carries no plan-folder pointer, so
phase-9's text is kept and nothing phase-8 said is lost.
node-health.json: the only change phase-8 makes to this file is renaming a
row that phase-9 had already deleted, so phase-9's file stands. Panel count
holds at 59 and both Validated Ledger Seq panels survive.
TESTING.md: resolved per hunk rather than by side. Phase-9's replacement of
the drifted span table and its new Grafana Cloud section are kept, and
phase-8's de-numbered "Test 3: Log-Trace Correlation" heading is taken --
keeping phase-9 wholesale would have reinstated the phase number that
phase-8 removed.
These comments were indexed by task, use-case and limitation numbers that
are defined only in planning documents outside the shipped tree. Nothing
in the repository defined them, so the cross-references resolved nowhere.
Each comment now states what the code does.
Two comment-only notes.
MetricsRegistry.cpp: record why the app and overlay includes exist and
what they cost. They are what makes loops.txt carry
xrpld.app <-> xrpld.telemetry and xrpld.overlay <-> xrpld.telemetry, where
ordering.txt previously had telemetry strictly below both. The observable
gauges are pull-model, so their callbacks need the concrete types to
sample live state. The cycle is confined to this translation unit: no
telemetry header includes app or overlay, and all of src/xrpld builds
into one target, so there is no header or link cycle. Inverting it needs
a metrics-source interface below overlay, which is left as follow-up.
Also note loops.txt is generated and must never be hand-edited.
docker-compose.yml: state the Compose >= 2.24.0 floor. The grafana
service uses the long-form env_file mapping, which older Compose cannot
parse, and it fails for the whole file rather than that one service. The
long form is required because .env.alerting is gitignored and absent in a
fresh clone, and the short form treats a missing env file as an error.
The test carried both <xrpl/proto/xrpl.pb.h> and the bare <xrpl.pb.h>.
Both resolve to the same generated header, because the proto helper puts
the generated tree and its prefixed subdirectory on the target, so the
second include expanded to nothing behind the header guard.
The bare form arrived from merging two same-day clang-tidy commits that
added the include with different spellings. Keep the prefixed spelling,
which is what the telemetry headers and the upstream phase branches use.
An InboundLedger destroyed while !isDone() recorded recordAbort() in the
metrics but left ledger.acquire carrying only the attributes set at
construction, so an abandoned acquisition was indistinguishable from one
still in flight.
Set outcome=aborted plus timeouts on that path, and add the val::aborted
constant. peer_count is deliberately omitted: reading it goes through
Overlay, and a destructor must not depend on Overlay still existing.
The status stays Unset for an abort, because InboundLedgers::stop()
clears every in-flight acquisition, so a clean shutdown would otherwise
report errors. done() now sets Error when failed_, which is an
unambiguous failure of the operation. Success is left Unset rather than
Ok, per the OpenTelemetry guidance that instrumentation should not
assert Ok.
rpcStart, jobQueue, jobStart and jobFinish each acquired a lock without
braces, so it stayed held to the end of the function and covered the OTel
recording calls this branch added. counters_.jobsMutex and
counters_.methodsMutex are process-wide, so every worker thread starting
or finishing a job serialised on the SDK's work.
That work is not a bare atomic add: each record builds a map-backed
attribute set and takes a spin lock inside the SDK, whose backoff reaches
a millisecond-scale sleep under contention.
Brace the lock plus the state it guards, then record after it releases.
None of the metric calls read lock-protected state, so this is
semantics-preserving, and rpcEnd() in the same file already had this
shape.
These comments pointed at a planning folder and at its rollout phase
numbering, neither of which is part of the shipped tree, so the
references would dangle for any reader of the repository. Each comment
now states the fact it was pointing at.
The reference docs had drifted from the code in ways that break the reader
rather than merely misinform: PromQL examples that return no data, a rollback
flag that is a no-op, a sampling knob that does not exist, and two span parents
that moved. Code is treated as the truth throughout; where the code is the
defective side, the doc now records it as a known issue instead of describing
the bug as intent.
Renames the docs missed: histogram names gain the exporter's unit suffix
(ios_latency_milliseconds_bucket and four siblings), ledger_history_mismatch
gains _total, the StatsD-era quantile label gives way to le buckets,
rpc.request becomes rpc.http_request, traces_spanmetrics_calls_total becomes
span_calls_total, and the nine dotted xrpl.* span attributes are recorded as
renamed rather than left as live keys.
Re-parenting: consensus.update_positions and consensus.check are children of
consensus.establish, not of consensus.round.
Units and labels: state_accounting_*_duration is microseconds, not seconds;
cache_metrics label values are case-sensitive; object_count carries demangled
C++ type names. Nodestore read and write latency stays microseconds -- the
nanosecond accumulator change did not move the exported unit.
Adds what shipped but was undocumented: the ledger.acquire span, seven
consensus.round events, twelve span attributes, node_writes_duration_us, the
7-day validation-agreement window, the TxQ admission and reduce-relay metric
families, metrics_endpoint, and the phase-10 validation workflow.
Corrects claims that never held: 10% head sampling (it is fixed at 100%),
configurable redaction (it is unconditional), -DXRPL_ENABLE_TELEMETRY=OFF
(the flag is -Dtelemetry=OFF, default ON), FindOpenTelemetry.cmake and the
xrpl_telemetry target (neither exists), Promtail and a StatsD exporter in the
pipeline (neither exists), and Loki stream selection on job= (only
service_name is a stream label).
Phase 9 is marked complete, its provisioned alerting is attributed to the
branch that shipped it, and Phase 11 stays at zero except the one prerequisite
its code closes. Counts are reconciled repo-wide: 41 emitted span families,
15 dashboards on disk with 14 asserted, 13 alert rules in 5 groups.
Hardens the gate that let this drift through: Rule E of the naming check now
covers the reference docs, its allow-dotted marker is key-scoped and warns on
stale or empty use, a missing checked file is reported instead of silently
skipped, the test suite runs in CI, and doc paths trigger the check.
C++ and CMake changes are comment-only: three MetricsRegistry instrument names,
eight OTelCollector claims of a metric-name prefix that formatName never adds,
and the telemetry option's inverted default.
The fetch and store duration counters converted each sample to
microseconds before adding it, so any backend call finishing in under a
microsecond contributed zero. A warm nudb read answers in a few hundred
nanoseconds, so on fast hardware every read floored and the totals stayed
at zero no matter how many reads happened -- the same loss of resolution
the microsecond report was introduced to avoid, one decade lower.
Both accumulators now hold nanoseconds, the clock's own resolution, and
convert once in getFetchDurationUs() and getStoreDurationUs(). The public
accessors, the node_reads_duration_us and node_writes_duration_us JSON
fields, and the metrics that read them all keep microseconds, so nothing
downstream changes unit. storeDurationStats() takes the raw duration
instead of a pre-converted integer so no caller can round early, and
updateFetchMetrics() scales its microsecond input to match.
FetchReport::elapsed stays microseconds: it carries one fetch, not a
total, and that is the unit it declares. The reported sum is therefore the
accumulated total minus a sub-microsecond remainder per fetch, so the two
tests that asserted exact equality between them now assert that bound.
Both had assertions that depended on how fast the host reads; the bound
holds on any hardware.
develop renamed xrpl::Resource to xrpl::resource. MetricsRegistry.cpp
declared `namespace resource = opentelemetry::sdk::resource` at file
scope, but both use sites are inside namespace xrpl::telemetry, where
inner-scope lookup finds the enclosing xrpl::resource first and never
reaches the alias. That namespace has no ResourceAttributes or Resource,
so the build failed on all four platforms:
error: no type named 'ResourceAttributes' in namespace 'xrpl::resource'
error: no member named 'Create' in namespace 'xrpl::resource';
did you mean 'creat'?
Rename the alias to otel_resource. Moving it inside xrpl::telemetry would
also compile, but only by shadowing xrpl::resource -- a reader at the use
site still could not tell which namespace `resource::` meant, and a later
using-directive would reintroduce the ambiguity. A distinct name removes
the collision by construction and matches the metric_sdk / otlp_http
aliases already in this file.
09-data-collection-reference.md: keep phase-9's structure, which relocated
the §2a call-site-metrics content into §5b (3860c93db2). The incoming §2a
block is superseded; its snake_case namespace fix is already applied to
phase-9's copy of the same text.
