beast::insight instruments are created during ApplicationImp's member-init
list, and opentelemetry-cpp 1.28 never rebinds an already-vended Meter, so an
instrument created before the MeterProvider is published records nothing for
the rest of the process. Observable instruments carry the opposite constraint:
registering one arms the SDK reader thread, and its callbacks run hook handlers
that read services which do not exist that early.
Publish the provider in Telemetry's constructor, ahead of every producer, and
defer only the observables. Collector gains onCollectionReady() and
onCollectionStopping(); OTelCollector arms and disarms its gauges in response.
StatsDCollector starts its polling thread in its own constructor and had the
same hazard, so it uses the pair to gate that thread.
The metrics resource carries service.instance.id and is immutable once built,
so the node public key is resolved in Main.cpp, where a config error can still
be reported, and passed to makeApplication(). getNodeIdentity() remains
authoritative; both paths now share readNodeIdentity(), so telemetry cannot
report a key the node has abandoned.
An explicit ~ApplicationImp stops observing and stops telemetry, covering the
setup() failure paths that never reach run(). Telemetry::stop() is once-only
and no longer clears another instance's global pointer. The histogram view's
meter selector now matches the meter actually in use, so its bucket boundaries
apply for the first time.
makeTelemetrySetup() rejects a contradictory [telemetry] mutual-TLS
setup by throwing, but it is called from ApplicationImp's
member-initializer list. A try/catch in the constructor body cannot
reach a throw from there, and nothing further up the stack caught it
either, so a config mistake reached std::terminate: the default handler
printed a terminate dump and raised SIGABRT, leaving a core file
instead of a startup error.
Catch std::exception around makeApplication() in run(), report the
reason on stderr and return -1, so the failure is a clean non-zero exit
with a message an operator can act on. Only the construction is
wrapped. setup() starts subsystems whose shutdown order is delicate and
is left outside deliberately, because unwinding a half-started
Application would skip the normal stop sequence.
Gate both validation guards on enabled. A node with telemetry switched
off previously refused to start over certificate paths that nothing
would read.
Document both throws on makeTelemetrySetup(), state in
cfg/xrpld-example.cfg and the configuration reference that a partial
mutual-TLS setup is fatal and that the checks apply only when
enabled=1, and add a runbook troubleshooting entry keyed on the two
error messages.
Tests cover both guards with the message asserted so the two are told
apart, both enabled=0 paths, and the default plaintext configuration.
Node identity reached the OTel resource only as service.instance.id, which is
config-overridable and carries a deployment-chosen label rather than the node's
own identity. Add xrpl.node.id, set unconditionally from the node public key
(base58, TokenType::NodePublic), so traces and metrics share a stable per-node
key independent of [telemetry] service_instance_id.
Set on the tracer resource via Telemetry::setNodeId(), called from
ApplicationImp::setup() once nodeIdentity_ is known, and on the MetricsRegistry
resource via an added start() parameter. The beast::insight meter provider is
built in TelemetryImpl's constructor, before the wallet DB exists, so its
resource cannot carry the value; that path is left for later and the attribute
is omitted rather than stamped blank.
Also drops the transform/spanidentity collector processor added in
4a361a496d: per-node identity belongs on the resource, not copied onto every
span.
These comments pointed at a planning folder and at its rollout phase
numbering, neither of which is part of the shipped tree, so the
references would dangle for any reader of the repository. Each comment
now states the fact it was pointing at.
The trace_state comment pointed at a planning document that is not part
of the shipped tree, so the reference would dangle for any reader of the
repository. State the reserved-and-inert fact on its own.
The reference docs had drifted from the code in ways that break the reader
rather than merely misinform: PromQL examples that return no data, a rollback
flag that is a no-op, a sampling knob that does not exist, and two span parents
that moved. Code is treated as the truth throughout; where the code is the
defective side, the doc now records it as a known issue instead of describing
the bug as intent.
Renames the docs missed: histogram names gain the exporter's unit suffix
(ios_latency_milliseconds_bucket and four siblings), ledger_history_mismatch
gains _total, the StatsD-era quantile label gives way to le buckets,
rpc.request becomes rpc.http_request, traces_spanmetrics_calls_total becomes
span_calls_total, and the nine dotted xrpl.* span attributes are recorded as
renamed rather than left as live keys.
Re-parenting: consensus.update_positions and consensus.check are children of
consensus.establish, not of consensus.round.
Units and labels: state_accounting_*_duration is microseconds, not seconds;
cache_metrics label values are case-sensitive; object_count carries demangled
C++ type names. Nodestore read and write latency stays microseconds -- the
nanosecond accumulator change did not move the exported unit.
Adds what shipped but was undocumented: the ledger.acquire span, seven
consensus.round events, twelve span attributes, node_writes_duration_us, the
7-day validation-agreement window, the TxQ admission and reduce-relay metric
families, metrics_endpoint, and the phase-10 validation workflow.
Corrects claims that never held: 10% head sampling (it is fixed at 100%),
configurable redaction (it is unconditional), -DXRPL_ENABLE_TELEMETRY=OFF
(the flag is -Dtelemetry=OFF, default ON), FindOpenTelemetry.cmake and the
xrpl_telemetry target (neither exists), Promtail and a StatsD exporter in the
pipeline (neither exists), and Loki stream selection on job= (only
service_name is a stream label).
Phase 9 is marked complete, its provisioned alerting is attributed to the
branch that shipped it, and Phase 11 stays at zero except the one prerequisite
its code closes. Counts are reconciled repo-wide: 41 emitted span families,
15 dashboards on disk with 14 asserted, 13 alert rules in 5 groups.
Hardens the gate that let this drift through: Rule E of the naming check now
covers the reference docs, its allow-dotted marker is key-scoped and warns on
stale or empty use, a missing checked file is reported instead of silently
skipped, the test suite runs in CI, and doc paths trigger the check.
C++ and CMake changes are comment-only: three MetricsRegistry instrument names,
eight OTelCollector claims of a metric-name prefix that formatName never adds,
and the telemetry option's inverted default.
The fetch and store duration counters converted each sample to
microseconds before adding it, so any backend call finishing in under a
microsecond contributed zero. A warm nudb read answers in a few hundred
nanoseconds, so on fast hardware every read floored and the totals stayed
at zero no matter how many reads happened -- the same loss of resolution
the microsecond report was introduced to avoid, one decade lower.
Both accumulators now hold nanoseconds, the clock's own resolution, and
convert once in getFetchDurationUs() and getStoreDurationUs(). The public
accessors, the node_reads_duration_us and node_writes_duration_us JSON
fields, and the metrics that read them all keep microseconds, so nothing
downstream changes unit. storeDurationStats() takes the raw duration
instead of a pre-converted integer so no caller can round early, and
updateFetchMetrics() scales its microsecond input to match.
FetchReport::elapsed stays microseconds: it carries one fetch, not a
total, and that is the unit it declares. The reported sum is therefore the
accumulated total minus a sub-microsecond remainder per fetch, so the two
tests that asserted exact equality between them now assert that bound.
Both had assertions that depended on how fast the host reads; the bound
holds on any hardware.
* upstream/release/3.3.x: (41 commits)
chore: Bump version to 3.3.0
chore: Bump version to 3.3.0-rc7
fix: Increase manifest protocol message size cap and fix manifests relay
fix: Cap untrusted manifests per message and drop oversized ones
chore: Bump version to 3.2.1
chore: Bump version to 3.2.1-rc1
fix: Cap untrusted manifests per message and drop oversized ones
fix: Reject oversized validator manifest before decoding
fix: Reduce untrusted manifest cache cap to 100
fix: Bound untrusted manifest cache
chore: Bump version to 3.3.0-rc6
feat: Package validator-keys inside rippled
chore: Bump version to 3.3.0-rc5
fix: Switch SponsorshipSet to use a delta for sfFeeAmount
fix: Re-revert "fix: Set request size limits and differential pricing for get-object-by-hash calls"
chore: Bump version to 3.3.0-rc4
fix: Revert "fix: Set request size limits and differential pricing for get-object-by-hash calls"
chore: Bump version to 3.3.0-rc3
fix: Reduce untrusted manifest cache cap to 100
fix: Revert "fix: Reject oversized SHAMap nodes in gotStaleData and fetch-pack path"
...
