Commit Graph

1180 Commits

Author SHA1 Message Date
Pratik Mankawde
dc696d260e Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-09-23 18:10:45 +01:00
Pratik Mankawde
18181f3c31 docs(telemetry): Drop the remaining account-hashing claims 2026-09-23 18:08:58 +01:00
Pratik Mankawde
2969965d8c Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
Conflicts, all resolved by keeping phase-9's rewritten text and applying
the upstream change to it:
- PathRequest.cpp: drop the Redaction.h include, keep MetricMacros.h.
- XrplCore.cmake: keep the PUBLIC-link comment, note the SField table use.
- OpenTelemetryPlan.md, 02-design-decisions.md, 05-configuration-reference.md,
  telemetry-runbook.md: account addresses are emitted raw; no hashing at
  any layer; tx.process lists the per-role account attributes.
2026-09-23 18:08:07 +01:00
Pratik Mankawde
1381b8f684 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-23 18:06:19 +01:00
Pratik Mankawde
331a26cb4c Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
Conflicts in src/libxrpl/telemetry/TelemetryConfig.cpp and its test:
phase-5's "require an https endpoint whenever use_tls=1" met phase-7's
version of the same guard, which already covers both the traces and the
metrics endpoint. Kept phase-7's side.
2026-09-23 18:05:31 +01:00
Pratik Mankawde
bf0670991a Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Conflicts: docker/telemetry/otel-collector-config.yaml (kept phase-6's
config, removed the attributes/hash processor as phase-2 did) and
docs/telemetry-runbook.md (kept phase-6's span table, applied phase-5's
tx.process attribute additions to its row).
2026-09-23 18:04:35 +01:00
Pratik Mankawde
a9771dcd30 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-23 18:02:58 +01:00
Pratik Mankawde
52a7d52acd Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Conflict in cmake/XrplCore.cmake: kept phase-4's comment block and applied
phase-3's two-line note about the SField table.
2026-09-23 18:02:45 +01:00
Pratik Mankawde
3761dee234 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-23 18:01:54 +01:00
Pratik Mankawde
509fc7853e feat(telemetry): Emit every account a transaction names on tx.process
A transaction names one or more accounts: the sender in Account, and by
type a Destination, Owner, Issuer, Holder and so on. tx.process now
carries each top-level account-typed field as its own attribute, keyed
tx_<field> in lower snake case (tx_account, tx_destination, ...), so an
account can be searched for in traces whatever role it played.
Addresses are public ledger identifiers and are emitted raw.

The keys live in TxAccountSpanNames.h in libxrpl, with a field-to-key
table in TxAccountSpanNames.cpp. A library test walks TxFormats and the
SField registry: every account field a transaction can carry has a key,
and no field that only ledger entries carry has one. An empty account
field is skipped rather than rendered as the zero address.
2026-09-23 18:01:14 +01:00
Pratik Mankawde
91b440820a fix(telemetry): Emit path-finding accounts as raw r-addresses
An XRP account address is a public ledger identifier drawn from an
enumerable set. An unsalted hash of it is reversible by lookup, so it
protected nothing and only broke the join against explorers, RPC
responses and logs that show the same address.

- pathfind_source_account and pathfind_dest_account carry the request's
  r-address. A value that does not parse as an r-address is not emitted,
  so a malformed or mistaken request value never reaches a span. Both
  handlers share setAccountAttribute() in PathFindSpanAttributes.h.
- pathfind_dest_currency is to_string(Asset): "XRP", "<issuer>/<CUR>",
  or the MPT issuance id.
- The collector's attributes/hash processor is removed. No layer hashes.
- redactAccount() stays available; its header no longer claims to sit in
  the emit path.
2026-09-23 18:00:32 +01:00
Pratik Mankawde
c156ad1b37 docs(insight): State when collector instruments must be created
The Collector interface said instruments are created in the owning class's
constructor, but not why, or what the boundary is. onCollectionReady() runs
once, after every service the hooks read exists; a collector starts polling and
arms its observable instruments at that point. A gauge made after it is never
armed, so it is never exported, and nothing reports that.

Name the boundary on the interface, and the consequence on makeGauge(), the one
instrument where being late fails silently.
2026-09-23 15:31:00 +01:00
Pratik Mankawde
f23b66ca4d Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics
expected_spans.json: composed both sides. Kept this branch's 16 extra spans,
relationships and the per_ledger join group; took phase-10's allowed_parents
gate for the shared entries and derived allowed_parents for the 16 from their
factories. ledger.store, ledger.validate and ledger.acquire are hashSpan roots
on this branch, so they list ROOT alone rather than consensus.accept.apply.
2026-09-23 15:18:25 +01:00
Pratik Mankawde
5ce7e4e855 fix(telemetry): Warn on server=otel with telemetry off, and widen the TLS guard
Three fixes in the telemetry wiring this branch owns.

An OTel collector records through the global meter provider, which only the
telemetry module installs. With [telemetry] enabled=0 the collector attached to
the SDK's noop provider, dropped every metric, and still logged that it had
started. makeCollectorManager now takes whether telemetry is on and warns; what
gets built is unchanged. telemetry_ is constructed before collectorManager_, so
the value is available.

The default metrics endpoint was spelled three times. Name it once as
kDefaultMetricsEndpoint, beside the export-cadence defaults that already live in
Telemetry.h for the same reason, and read it from the Setup member, the config
parser's default and the collector.

The scheme guard ran only when tls_client_cert was set, so use_tls=1 on an http
endpoint validated the certificate files and then exported in the clear. It now
covers every use_tls=1 node, for both signals. Two cases that pinned the old
behaviour are flipped to _throws, each asserting which endpoint key the message
names, and three more get https endpoints so the scheme guard is not what fires.
2026-09-23 14:29:24 +01:00
Pratik Mankawde
3328bd2323 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-23 13:57:48 +01:00
Pratik Mankawde
769325d162 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-23 13:57:48 +01:00
Pratik Mankawde
360cde04ef Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-23 13:57:48 +01:00
Pratik Mankawde
3a9eae3998 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-23 13:57:47 +01:00
Pratik Mankawde
f7f8aa79c3 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-23 13:57:47 +01:00
Pratik Mankawde
6fadd0e2ee fix(telemetry): emit consensus.mode_change only on a real transition
MonitoredMode::set calls onModeChange on every round start, so the span was
created whether or not the mode moved. A node with a steady mode therefore
emitted one mode_change per round carrying mode_old == mode_new, which a live
sweep confirmed on every round of both instrumented builds. The round span's
own consensus_mode attribute is still written on every call, since that is
where the round learns the mode it is running in.
2026-09-23 13:49:54 +01:00
Pratik Mankawde
59bae37688 fix(telemetry): nest the accept work under consensus.accept.apply
accept.apply was a plain SpanGuard, so it never became the ambient span of
doAccept. The spans the function goes on to create inherited the activated
accept span instead and came out as accept.apply's siblings, while running
inside its own time window. Every guard was scoped before the SpanGuard split,
so this restores the hierarchy that design had.

Scoped now, so the hierarchy follows the call flow. Drops the parent-context
fallback arm with it: the accept context is captured only while the accept span
is live, and that span is a child of the round context, so an invalid accept
context implies an invalid round context and both arms returned an empty guard.
2026-09-23 13:49:42 +01:00
Pratik Mankawde
7d21baf558 feat(telemetry): add event-with-attributes overload to ScopedSpanGuard
Only SpanGuard carried addEvent(name, attrs), so a call site holding a scoped
guard could not record an event attribute. Forwarding overload, with the no-op
twin in the telemetry-disabled stub, so a span can be converted between scoped
and unscoped without dropping the attributes on its events.
2026-09-23 13:49:03 +01:00
Pratik Mankawde
881faf5620 Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics
The runbook's nodestore_state table kept AppMetricGauges.cpp as the source
column, which is where those gauges are actually registered on every branch
in the chain; the incoming side named MetricsRegistry.cpp, which registers
none of them. Took the incoming side's clearer wording for the sweep row.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 21:19:08 +01:00
Pratik Mankawde
aeac95b1d7 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-22 21:17:58 +01:00
Pratik Mankawde
a9541a7300 fix(telemetry): Install the OTel context storage before any thread starts
SetRuntimeContextStorage() writes a process-global shared_ptr that every log
line reads through RuntimeContext::GetCurrent(). Neither side is atomic, and
assigning the wrapper destroys it and placement-news a replacement over the
same buffer, so a concurrent reader could make a virtual call through an
indeterminate vptr. Install it in main() while the process is still
single-threaded instead, and drop the member that held it.

Gated on the telemetry section being enabled. Only that one key is read
here, because parsing the whole section can throw on a contradictory TLS
combination and that error belongs where it already reports.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 21:11:58 +01:00
Pratik Mankawde
2d3eb7a981 fix(telemetry): Stop unl_expiry_days wrapping, and count only unfinished sweep evictions
unl_expiry_days subtracted two NetClock time points, whose rep is uint32_t, so
the subtraction wrapped before the duration_cast ran. A list expired by one day
read about +49709 days. The panel is green above 30 while its own description
promises red at expiry, so an expired validator list rendered healthy.

daysUntil() widens both endpoints to int64_t first, which makes the wrap
impossible rather than checked for. It deliberately does not clamp at zero: a
negative reading is the signal that expiry has passed. A config-listed list,
which uses time_point::max(), now reports positive infinity, because any finite
sentinel could not be told apart from the wrap this removes. -1 keeps its
existing meaning of no published list fetched.

The sweep counter told a second story it could not support. It counted every
entry the 1-minute sweep evicted, including acquisitions that had already
completed or failed and were merely still in the map. Those were counted when
they ended, so the metric buried the wasteful case in ordinary cleanup while
the runbook, the reference doc and the panel description all described only the
unfinished population. It now counts what those three already claimed.

isComplete()/isFailed() are used rather than isDone(), which is protected on
TimeoutCounter and not callable here.
2026-09-22 19:07:41 +01:00
Pratik Mankawde
6b1ced940e Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-09-22 15:43:12 +01:00
Pratik Mankawde
8bca6a8351 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-22 15:38:47 +01:00
Pratik Mankawde
f68375213e Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-22 15:38:36 +01:00
Pratik Mankawde
d4f8f7cfbd Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-22 15:38:30 +01:00
Pratik Mankawde
344d453292 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-22 15:38:26 +01:00
Pratik Mankawde
a570216c94 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-22 15:38:21 +01:00
Pratik Mankawde
7b275f6c0d Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-22 15:38:17 +01:00
Pratik Mankawde
dad8838c31 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-22 15:38:12 +01:00
Pratik Mankawde
3b2192a48b Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-22 15:38:06 +01:00
Pratik Mankawde
359ae64170 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-22 15:38:01 +01:00
Bart
00606bec1a fix: Derive traversal node IDs from the branch actually descended (#7942)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-09-22 14:02:11 +00:00
Mayukha Vadari
2bc17c3e73 refactor: Add initial wrapper classes for all SLEs (#7886) 2026-09-21 19:47:23 +00:00
Pratik Mankawde
f86d5e408b Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-09-21 15:18:00 +01:00
Pratik Mankawde
82aeac5c0e fix(core): include <string> for PerfLog's job hooks
jobQueue, jobStart and jobFinish take their name as std::string const&, so this
header has to provide <string> itself. The rpc hooks moved to string_view and
the include went with them, which include-cleaner reports as an error.
2026-09-21 15:17:51 +01:00
Pratik Mankawde
143538581c Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-09-21 12:52:31 +01:00
Pratik Mankawde
5e398cd6b7 merge: bring develop forward into phase-9 metric gap fill
Resolved five files.

PerfLogImp.cpp: PerfLog now takes its method names as string_view, so the
methods map stores the map key rather than a char const*. Kept this branch's
scoped lock so the OTel recording still runs after methodsMutex is released,
and kept both the NetworkOPs and metric-macro includes.

PeerImp.h: took develop's earlier `protected:`, which also covers
processLedgerRequest, and kept recordGetObjectMetrics inside it. Dropped the
comment naming one test, since four test files now rely on that access.

TMGetObjectByHash_test.cpp: kept this branch's suite. It already runs develop's
three bound cases (limit+1, limit, limit-1) as testReplyLimit, plus the charge
and hit/miss cases. develop's extracted PeerTest.h stays, used by its four new
overlay tests.

ordering.txt: regenerated. Both perflog edges are real, to xrpl.server and to
xrpl.telemetry.

.cspell.config.yaml: kept both added words.
2026-09-21 12:51:31 +01:00
Pratik Mankawde
05ce773671 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-21 12:45:59 +01:00
Pratik Mankawde
243913250b Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-21 12:45:59 +01:00
Pratik Mankawde
c8a8da6fd6 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-21 12:45:55 +01:00
Pratik Mankawde
e5d28ccbc4 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-21 12:45:54 +01:00
Pratik Mankawde
870367d71e merge: bring develop forward into phase-3 tx tracing
Resolved src/libxrpl/tx/applySteps.cpp. calculateBaseFee now returns
std::expected<XRPAmount, TER>, so invokePreclaim rejects the transaction when
the fee cannot be computed. Kept that inside this branch's preclaimTer wrapper,
so the returned error also reaches the span's ter_result attribute.
2026-09-21 12:45:45 +01:00
Pratik Mankawde
c030880a36 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-21 12:44:43 +01:00
Pratik Mankawde
d16f2c4181 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-21 12:43:48 +01:00
Pratik Mankawde
f48a2b4871 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-21 12:43:44 +01:00