Telemetry must read state, never change it. Two defects here did change it,
plus three smaller correctness and privacy fixes.
doPathFind and doRipplePathFind read source_account / destination_account off
context.params to hash them into span attributes. context.params is non-const,
so those reads selected json::Value's non-const operator[], which inserts a
null for a missing key. The same object is later validated by
PathRequest::parseJson, whose first checks are isMember(source_account) and
isMember(destination_account) — so a request that omitted either field looked
present and the client received Malformed instead of Missing. Reads now go
through std::as_const, whose overload returns kNull without inserting.
PathRequest::doUpdate emitted pathfind_dest_currency as
to_string(saDstAmount_.asset()). For a non-XRP asset that renders as
"<issuer>/<currency>" with the issuer as a plaintext Base58 address, so a
plain account address reached the span pipeline even though every other
account here is hashed first. The issuer is now redacted and the currency
kept; an MPT asset renders as its issuance ID and carries no address.
PathRequestManager::updateAll created pathfind.update_all with an unscoped
SpanGuard. An unscoped guard takes the ambient span as its own parent but does
not itself become the ambient parent, so the pathfind.compute spans that
doUpdate creates never nested under it, contradicting the documented hierarchy.
It is now a scoped guard, held in std::optional because ScopedSpanGuard is
deliberately non-movable and so cannot be produced by a ternary. The skip when
there are no active subscriptions is preserved. updateAll is dispatched via
addJob and doUpdate runs synchronously, so the guard is constructed and
destroyed under the same context store, as ScopedSpanGuard requires.
The WebSocket entry point emitted the client-supplied command string directly.
That value becomes a Prometheus label, so arbitrary request input could drive
unbounded label cardinality. It is now resolved against the handler registry,
collapsing anything unrecognized to "unknown", matching what the HTTP path
already does.
Also: the pathfind.discover comment claimed future child spans could be
parented off it, which its unscoped guard cannot do — corrected to say what
would be required instead. Config-reference and task-list docs named the
parser setupTelemetry(); the API is makeTelemetrySetup().
- Add PathFinding Spans section to the runbook Span Reference and
remove the incorrect "not emitted / later-phase" annotations:
pathfind.request/compute/discover/update_all are emitted since
Phase 2 (upstream of Phase 5).
- Runbook label names use the bare spanmetrics dimensions
(command, rpc_status, local, consensus_mode); fix stale docs.
- 05-configuration-reference: note tls_client_cert/key require use_tls=1.
- TelemetryConfig: reject mTLS cert paths when use_tls=0 so telemetry
is not silently sent in plaintext.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- 05-configuration-reference: service_name now applies to metrics as well
as traces (service.name resource attribute).
- 07-observability-backends: document that collector enrichment includes
deployment-tier tagging (deployment.environment + fallback
xrpl.network.type) for filtering one Grafana stack by tier.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring the hardened OTel naming check (Rule E fix + 71 tests) and the phase 1-6
convention work forward into phase 7 (native metrics).
Conflict resolution:
- 05-configuration-reference.md: kept phase-7's native-OTLP metrics story
(server=otel, /v1/metrics) over phase-6's superseded StatsD scrape job, in
prose form (no code block).
- ordering.txt/loops.txt: regenerated via generate.py (not hand-edited).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring phase-4 forward into phase 5 (docs & deployment). Phase 5 owns the
operator runbook (docs/telemetry-runbook.md).
Conflict resolution:
- 05-configuration-reference.md: took phase-4's code-block-free prose for the
config-parser section; the mTLS options (tls_client_cert/tls_client_key)
remain documented in the §5.1.2 options table.
Phase-5-owned naming fix:
- docs/telemetry-runbook.md: converted 20 dotted xrpl.<domain>. attribute keys
in the Span Reference tables to the underscore convention (tx_hash, peer_id,
ledger_seq, consensus_mode/round/round_id/ledger_id, tx_id). Span NAMES stay
dotted (rule 5). `trusted` left as-is — verified it matches the code constant
ConsensusSpanNames.h (the proposal_trusted/validation_trusted split is a
separate, not-yet-applied code change).
End-to-end reconciliation: every span-attribute token in the runbook (L5) now
matches an L1 *SpanNames.h constant. The xrpl_<domain>_<field> tokens are the
spanmetrics-derived Prometheus labels (documented mapping), not span attrs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring phase-2 forward into phase 3 (transaction tracing). Phase 3 introduces
TxSpanNames.h, TxQSpanNames.h, and TxApplySpanNames.h.
Conflict resolution:
- TxQ.cpp: kept phase-3's txq_span-based instrumentation (phase-2 had none).
Dropped the orphaned `NumberSO{... fixUniversalNumber}` line — develop's
#5962 (Retire fixUniversalNumber) removed that symbol repo-wide; the
conflict block had carried one stale copy that would not compile.
- 05/08/OpenTelemetryPlan.md: dropped the deleted 04-code-samples / POC_taskList
references (carried from phase-2), kept phase-3's new secure-OTel.md doc rows,
section, and Mermaid node/edge/style. Config code block -> prose; merged the
secure-OTel hardening pointer with the authoritative-config prose.
- Phase3_taskList.md: removed the "dotted keys for readability" note that came
from phase-2 — phase 3 already uses the underscore keys.
Reviewed by code-review agents: telemetry instrumentation intact, naming check
green (47 keys across 7 *SpanNames.h headers), no conflict markers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring the naming convention, code-sample cleanup, and CI naming check into
phase 2 (RPC tracing). Phase 2 introduces PathFindSpanNames.h.
Conflict resolution:
- 04-code-samples.md, POC_taskList.md: deletion wins.
- 02-design-decisions.md: took the convention-applied tables, but kept phase-2's
accurate PathFinding summary row (pathfind_fast/search_level/num_paths/...,
matching the implemented PathFindSpanNames.h).
- 05/08: took the code-block-free prose; kept phase-2's Phase2-5_taskList.md
index rows (dropping only the deleted POC row). Fixed stale setup_Telemetry/
make_Telemetry doc references to the code-correct setupTelemetry/makeTelemetry.
- Telemetry.h auto-merged to the constant-based @code examples.
check_otel_naming.py change: exempt test files from Rule F (tests pass
arbitrary literal keys to exercise the API). The check passes clean on the
merged tree (24 keys across 4 *SpanNames.h headers, including PathFind).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Establish the single, authoritative naming convention for OpenTelemetry
span attribute keys so the code, collector, Tempo, dashboards, and docs
stay in sync.
- CONTRIBUTING.md: new "Telemetry span attribute naming" section under
the Style guide as the permanent, canonical home for the rules.
- OpenTelemetryPlan/02-design-decisions.md: new section 2.3.3 stating the
decided convention as design, and section 2.4 attribute schema realigned
to the underscore form (exact key spelling defers to the *SpanNames.h
constants).
- Sweep the remaining plan docs: convert dotted xrpl.<domain>.<field> span
attribute keys to the underscore form; leave span names and the
OTel-standard service.*/http.* and xrpl.network.* resource keys dotted.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document that head sampling is intentionally fixed at 100% and no longer
exposes a sampling_ratio config knob. A per-node ratio let nodes make
divergent keep/drop decisions for the same distributed trace, producing
broken/partial traces; pinning at 1.0 with a ParentBased sampler keeps
decisions coherent across the network. Volume reduction is delegated to
collector-side tail sampling.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Flip trace_peer default false->true across the Phase-1a plan docs and
correct the rationale: peer spans record only peer_id (numeric local
connection id) plus trust/ledger metadata, never IP addresses or public
keys, so the 'includes addresses' caveat was inaccurate. The high-volume
note is retained.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two stray "rippled" tokens introduced by 43258e8d ("docs(telemetry):
add secure-OTel pipeline analysis…") were caught by check-rename in
CI. Re-run docs.sh to convert them to xrpld so the rename check
passes on PR #6425 (and downstream PR #6426 once merged up).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Document the threat model and chosen hardening approach for the OTel
pipeline: mTLS to the collector as primary defense (across-network
deployment), NetworkPolicy as defense-in-depth, and source-side
validation plus per-peer rate limiting for protocol::TraceContext on
peer messages. Skips Basic Auth (wrong shape for multi-operator
fleet) and HTTP-gateway header stripping (rippled is P2P).
Wires the new doc into the master plan ToC, mermaid diagram, and
body section, plus cross-refs from the privacy section in
02-design-decisions.md and the collector config in
05-configuration-reference.md so readers reach it from natural
in-context entry points. Adds a backlink at the top of secure-OTel.md
to the master plan.
Adds 'exfiltration' and 'htpasswd' to cspell dictionary.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
