Commit Graph

17686 Commits

Author SHA1 Message Date
Pratik Mankawde
1748178cfa Merge branch 'pratik/otel-sync-diagnostics' into pratik/otel-sync-diagnostics-freshen-fix 2026-09-23 18:51:01 +01:00
Pratik Mankawde
b6072f2677 Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-09-23 18:46:25 +01:00
Pratik Mankawde
8ec0044bca Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-09-23 18:44:57 +01:00
Pratik Mankawde
26db3eff82 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-23 18:44:45 +01:00
Pratik Mankawde
ca601ce616 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-23 18:42:12 +01:00
Pratik Mankawde
e8b5322c5b Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-23 18:42:00 +01:00
Pratik Mankawde
89cfda8089 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-23 18:39:12 +01:00
Pratik Mankawde
c24670e007 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-23 18:39:12 +01:00
Pratik Mankawde
6bf8cbf265 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-23 18:39:12 +01:00
Pratik Mankawde
334d2346b0 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-23 18:39:12 +01:00
Pratik Mankawde
7aaa985c0c Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-23 18:39:12 +01:00
Pratik Mankawde
5f0d3a4186 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-23 18:39:12 +01:00
Pratik Mankawde
1db1ae9b54 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-23 18:39:12 +01:00
Pratik Mankawde
bd490c694b Merge remote-tracking branch 'origin/develop' into pratik/otel-phase1a-plan-docs 2026-09-23 18:38:44 +01:00
Pratik Mankawde
4efd228e07 Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-09-23 18:30:54 +01:00
Pratik Mankawde
fb281af485 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-09-23 18:30:54 +01:00
Pratik Mankawde
31984bd432 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-23 18:30:54 +01:00
Pratik Mankawde
56f6b9b0ec Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-23 18:30:54 +01:00
Pratik Mankawde
ac8eee1bab Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-23 18:30:54 +01:00
Pratik Mankawde
3b14147a83 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-23 18:30:54 +01:00
Pratik Mankawde
71b538b2e6 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-23 18:30:54 +01:00
Pratik Mankawde
bc252e2f15 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-23 18:30:54 +01:00
Pratik Mankawde
010fe0c7c9 fix(tests): Brace the EXPECT inside an if in the tx account key test
gtest's EXPECT_EQ expands to an if/else, so an unbraced if around it is a
dangling-else error with warnings treated as errors. Found by the local
build of xrpl_tests.
2026-09-23 18:30:52 +01:00
Pratik Mankawde
dc696d260e Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-09-23 18:10:45 +01:00
Pratik Mankawde
67ec4071a2 docs(telemetry): Note the per-role account attributes on tx.process in the harness 2026-09-23 18:10:16 +01:00
Pratik Mankawde
64760545eb Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-09-23 18:09:26 +01:00
Pratik Mankawde
18181f3c31 docs(telemetry): Drop the remaining account-hashing claims 2026-09-23 18:08:58 +01:00
Pratik Mankawde
2969965d8c Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
Conflicts, all resolved by keeping phase-9's rewritten text and applying
the upstream change to it:
- PathRequest.cpp: drop the Redaction.h include, keep MetricMacros.h.
- XrplCore.cmake: keep the PUBLIC-link comment, note the SField table use.
- OpenTelemetryPlan.md, 02-design-decisions.md, 05-configuration-reference.md,
  telemetry-runbook.md: account addresses are emitted raw; no hashing at
  any layer; tx.process lists the per-role account attributes.
2026-09-23 18:08:07 +01:00
Pratik Mankawde
1381b8f684 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-23 18:06:19 +01:00
Pratik Mankawde
b98b4e0c5a docs(telemetry): Account attributes in the data-collection reference 2026-09-23 18:06:17 +01:00
Pratik Mankawde
331a26cb4c Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
Conflicts in src/libxrpl/telemetry/TelemetryConfig.cpp and its test:
phase-5's "require an https endpoint whenever use_tls=1" met phase-7's
version of the same guard, which already covers both the traces and the
metrics endpoint. Kept phase-7's side.
2026-09-23 18:05:31 +01:00
Pratik Mankawde
bf0670991a Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Conflicts: docker/telemetry/otel-collector-config.yaml (kept phase-6's
config, removed the attributes/hash processor as phase-2 did) and
docs/telemetry-runbook.md (kept phase-6's span table, applied phase-5's
tx.process attribute additions to its row).
2026-09-23 18:04:35 +01:00
Pratik Mankawde
021f20b311 docs(telemetry): Path-find spans carry raw account addresses 2026-09-23 18:03:36 +01:00
Pratik Mankawde
c13e83d7f4 docs(telemetry): List the per-role account attributes in the runbook span table 2026-09-23 18:03:18 +01:00
Pratik Mankawde
a9771dcd30 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-23 18:02:58 +01:00
Pratik Mankawde
52a7d52acd Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Conflict in cmake/XrplCore.cmake: kept phase-4's comment block and applied
phase-3's two-line note about the SField table.
2026-09-23 18:02:45 +01:00
Pratik Mankawde
dd8a35ec36 docs(telemetry): List the per-role account attributes of tx.process 2026-09-23 18:02:15 +01:00
Pratik Mankawde
3761dee234 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-23 18:01:54 +01:00
Pratik Mankawde
509fc7853e feat(telemetry): Emit every account a transaction names on tx.process
A transaction names one or more accounts: the sender in Account, and by
type a Destination, Owner, Issuer, Holder and so on. tx.process now
carries each top-level account-typed field as its own attribute, keyed
tx_<field> in lower snake case (tx_account, tx_destination, ...), so an
account can be searched for in traces whatever role it played.
Addresses are public ledger identifiers and are emitted raw.

The keys live in TxAccountSpanNames.h in libxrpl, with a field-to-key
table in TxAccountSpanNames.cpp. A library test walks TxFormats and the
SField registry: every account field a transaction can carry has a key,
and no field that only ledger entries carry has one. An empty account
field is skipped rather than rendered as the zero address.
2026-09-23 18:01:14 +01:00
Pratik Mankawde
9bfa3cc32c docs(telemetry): Account addresses are public and not hashed
Rewrite the privacy policy (design decisions section 2.4.4) and the two
plan summaries that still described account hashing and a configurable
redaction. Add the two account attributes to the path-finding attribute
table and describe pathfind_dest_currency as the rendered asset.
2026-09-23 18:00:34 +01:00
Pratik Mankawde
91b440820a fix(telemetry): Emit path-finding accounts as raw r-addresses
An XRP account address is a public ledger identifier drawn from an
enumerable set. An unsalted hash of it is reversible by lookup, so it
protected nothing and only broke the join against explorers, RPC
responses and logs that show the same address.

- pathfind_source_account and pathfind_dest_account carry the request's
  r-address. A value that does not parse as an r-address is not emitted,
  so a malformed or mistaken request value never reaches a span. Both
  handlers share setAccountAttribute() in PathFindSpanAttributes.h.
- pathfind_dest_currency is to_string(Asset): "XRP", "<issuer>/<CUR>",
  or the MPT issuance id.
- The collector's attributes/hash processor is removed. No layer hashes.
- redactAccount() stays available; its header no longer claims to sit in
  the emit path.
2026-09-23 18:00:32 +01:00
Pratik Mankawde
fc98dac0f1 fix(telemetry): name each rule's query steps, and let email keep its own layout
The alert notification ended with `Values: A=0 B=0 C=1`, the internal ids of the
three query steps, which mean nothing to a reader. The previous attempt removed
that line by replacing the whole email body with a custom template. That worked,
but it threw away everything else Grafana's default email gives: bold section
headings, a styled label table, annotations rendered as clickable links, and the
View and Silence buttons. The result read as a flat block of text and raw URLs.

Fix the line instead of the body. Rename all 13 rules' steps from A/B/C to names
that say what they hold -- close_rate_5m, close_rate_now, is_stalled -- so the
default body's value line reads close_rate_5m=0 close_rate_now=0 is_stalled=1.
The threshold step is named is_* so a 0/1 verdict is obviously a verdict and not
a rate.

Drop the email subject/message override so the default layout returns, and drop
the now-unused xrpld.email.body template. Slack keeps its own body: Grafana
escapes HTML in an email message, so a custom email body can only ever be plain
text, while Slack needs mrkdwn and <url|label> links. One body cannot serve both,
and only Slack benefits from having one at all.

Verified by provisioning the real files into Grafana: all 13 rules load with the
new step names and evaluate health=ok, and the Slack template renders with no
parse error.
2026-09-23 17:40:10 +01:00
yinyiqian1
30640a626f test: Clean up ConfidentialTransfer test helpers (#8265) 2026-09-23 16:24:41 +00:00
Pratik Mankawde
c156ad1b37 docs(insight): State when collector instruments must be created
The Collector interface said instruments are created in the owning class's
constructor, but not why, or what the boundary is. onCollectionReady() runs
once, after every service the hooks read exists; a collector starts polling and
arms its observable instruments at that point. A gauge made after it is never
armed, so it is never exported, and nothing reports that.

Name the boundary on the interface, and the consequence on makeGauge(), the one
instrument where being late fails silently.
2026-09-23 15:31:00 +01:00
Pratik Mankawde
bca04bc2c3 fix(telemetry): give email its own notification body, not the Slack one
Sharing one body between the Slack and email receivers made the email
unreadable. Email cannot render Slack markup, so the bold asterisks, the
backticks and the `🚨` shortcodes all arrived as literal
characters, and a `<url|label>` link could not become a link at all -- it dumped
the whole dashboard URL inline. Four panel links then buried the prose.

Split the body in two. The Slack body keeps mrkdwn. The email body is plain
text, one fact per line, with each dashboard link on its own labelled line.
Both still share the title and the node-identity fallback.

Measured while fixing this, and now recorded in templates.yaml: email escapes
any HTML in the message, so `<br>` arrives as `&lt;br&gt;` and no tag or anchor
is possible; but a newline in the template does become a real `<br>`, so line
breaks are the only layout tool email has.

Also correct two comments that were wrong. A missing template define does not
ship raw template text: it logs one warning and silently delivers Grafana's
default body, the value dump this file exists to remove, while the rule still
reports health=ok. And the Cloud contact point is not email-only; it holds a
Slack receiver and an email receiver, on an instance shared with other teams.
2026-09-23 15:19:51 +01:00
Pratik Mankawde
9dedf3e22a docs(telemetry): acquire, store and validate are hashSpan roots on this branch
The parenting section still described them as ambient spans that could land
under an RPC command or under consensus.accept.apply. On this branch all three
derive their trace id from the ledger hash and are roots of the per-ledger
trace, whatever thread creates them.
2026-09-23 15:19:41 +01:00
Pratik Mankawde
f23b66ca4d Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics
expected_spans.json: composed both sides. Kept this branch's 16 extra spans,
relationships and the per_ledger join group; took phase-10's allowed_parents
gate for the shared entries and derived allowed_parents for the 16 from their
factories. ledger.store, ledger.validate and ledger.acquire are hashSpan roots
on this branch, so they list ROOT alone rather than consensus.accept.apply.
2026-09-23 15:18:25 +01:00
Pratik Mankawde
fd29e92a0e Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-09-23 15:15:28 +01:00
Pratik Mankawde
9e635516da docs(telemetry): the accept log pair correlates to consensus.accept.apply
doAccept opens accept.apply as a scoped guard, so the CNF Val / CNF buildLCL
lines carry that span's id, not the accept span's. Same trace either way.
2026-09-23 15:15:01 +01:00
Pratik Mankawde
0764aa01aa docs(telemetry): correct the span-parenting claims the code has outgrown
tx.apply carries its own ledger_seq. On the consensus path ledger.build,
ledger.store, ledger.validate and the queue spans nest under
consensus.accept.apply, which doAccept opens as a scoped guard, so the passage
that said no ambient span exists there is replaced by the edge it now has. The
per-stage failure-rate comment no longer claims the stages leave status unset
on a failing result; all three set an error status.
2026-09-23 15:14:57 +01:00