Commit Graph

430 Commits

Author SHA1 Message Date
Pratik Mankawde
1381b8f684 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-23 18:06:19 +01:00
Pratik Mankawde
331a26cb4c Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
Conflicts in src/libxrpl/telemetry/TelemetryConfig.cpp and its test:
phase-5's "require an https endpoint whenever use_tls=1" met phase-7's
version of the same guard, which already covers both the traces and the
metrics endpoint. Kept phase-7's side.
2026-09-23 18:05:31 +01:00
Pratik Mankawde
bf0670991a Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Conflicts: docker/telemetry/otel-collector-config.yaml (kept phase-6's
config, removed the attributes/hash processor as phase-2 did) and
docs/telemetry-runbook.md (kept phase-6's span table, applied phase-5's
tx.process attribute additions to its row).
2026-09-23 18:04:35 +01:00
Pratik Mankawde
52a7d52acd Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Conflict in cmake/XrplCore.cmake: kept phase-4's comment block and applied
phase-3's two-line note about the SField table.
2026-09-23 18:02:45 +01:00
Pratik Mankawde
3761dee234 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-23 18:01:54 +01:00
Pratik Mankawde
91b440820a fix(telemetry): Emit path-finding accounts as raw r-addresses
An XRP account address is a public ledger identifier drawn from an
enumerable set. An unsalted hash of it is reversible by lookup, so it
protected nothing and only broke the join against explorers, RPC
responses and logs that show the same address.

- pathfind_source_account and pathfind_dest_account carry the request's
  r-address. A value that does not parse as an r-address is not emitted,
  so a malformed or mistaken request value never reaches a span. Both
  handlers share setAccountAttribute() in PathFindSpanAttributes.h.
- pathfind_dest_currency is to_string(Asset): "XRP", "<issuer>/<CUR>",
  or the MPT issuance id.
- The collector's attributes/hash processor is removed. No layer hashes.
- redactAccount() stays available; its header no longer claims to sit in
  the emit path.
2026-09-23 18:00:32 +01:00
Pratik Mankawde
e06fdd266e Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-23 12:03:08 +01:00
Pratik Mankawde
bcc44dedf1 docs(telemetry): drop the example ticket id from the work-item filter
The $xrpl_work_item variable's description carried a real ticket id as its
example. The filter needs no example, so the id is gone and the wording stays.
2026-09-23 12:00:29 +01:00
Pratik Mankawde
fcdc4f9f66 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-22 21:17:58 +01:00
Pratik Mankawde
e48b0286c2 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
StatsDCollector test kept both sides: this branch's onCollectionReady() call,
which enables polling and only exists from here on, followed by the upstream
branch's control assertion that reads the resulting datagram.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 21:17:48 +01:00
Pratik Mankawde
e9d9cffe3b Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Both doc indexes kept this branch's 09-data-collection-reference.md rows,
which only exist here, and dropped every secure-OTel.md reference because
the upstream branch removed that file. No dangling link remains.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 21:16:44 +01:00
Pratik Mankawde
aea4f56505 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
SpanGuardScope.cpp kept both includes: each side added one and both symbols
are used in the merged test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 21:15:05 +01:00
Pratik Mankawde
bba92767a6 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
tempo.yaml kept both sides' filter blocks: phase-2's six path-finding
filters ahead of this branch's three transaction filters, matching chain
order, and both header comment lines.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 21:14:20 +01:00
Pratik Mankawde
3cfe8d139e feat(telemetry): Add Tempo search filters for the path-finding attributes
This branch emits nine pathfind attributes and the datasource offered a dropdown
for none of them, so the signal was there but not searchable in Explore. The
file's own header states that each phase adds filters for what it introduces.

Six filters, for the attributes that select a request. The three counts are
measurements read off a span rather than things an operator searches by, so they
get none. ledger_index is dynamic because it takes a new value every ledger.
2026-09-22 20:31:14 +01:00
Pratik Mankawde
87fad43d9a Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
Conflict in docker/telemetry/docker-compose.yml, on the collector service's
command key. This branch layers a second --config for file_log offset
persistence; upstream added a comment above the ports block explaining the
loopback binding.

Kept both. The two changes are unrelated: one is the command list, the other is
a comment. The layered --config and all nine loopback-bound ports are present,
and the file still parses.
2026-09-22 15:13:54 +01:00
Pratik Mankawde
4308f967f3 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
Conflict in docker/telemetry/docker-compose.yml, in the collector's ports
block. Upstream bound every port to the host loopback, including the StatsD
UDP port. This branch had already retired that port, because beast::insight
moved to native OTLP, leaving it commented out as a fallback example.

Not resolved as a union, which would have resurrected the retired port.
Composed instead: this branch's shape is kept, so StatsD stays commented, and
the loopback binding is applied to the active ports and to the commented
example, so uncommenting it yields the safe form.
2026-09-22 15:13:14 +01:00
Pratik Mankawde
18df80c5ec Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Conflict in docker/telemetry/docker-compose.yml, in the collector's ports
block. Both sides bound ports to the host loopback: upstream did 4317, 4318
and 13133, this branch did 8125, 8889 and 9090.

Resolved as the union. All eight published ports are loopback-bound, and both
explanatory comments are kept: upstream's, which covers the whole block and
cites the upstream guidance, and this branch's, which is specific to the
unauthenticated StatsD receiver.
2026-09-22 15:12:32 +01:00
Pratik Mankawde
6ecd5f24a9 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-22 15:11:47 +01:00
Pratik Mankawde
0e1e335f42 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-22 15:11:47 +01:00
Pratik Mankawde
5a148ff55e Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-22 15:11:47 +01:00
Pratik Mankawde
46ca2daf2f Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-22 14:07:04 +01:00
Pratik Mankawde
2de992f334 fix(telemetry): give Ledger Data & Sync an auto-refresh interval
The dashboard had no top-level refresh key at all, so auto-refresh was off
while every other dashboard refreshes at 30s.
2026-09-22 14:07:02 +01:00
Pratik Mankawde
0dccf29b64 fix(telemetry): publish the Loki port on the host loopback
Grafana reaches Loki as loki:3100 over the compose network and the collector
exports to it in-network, so the published port was reachable off-host for no
consumer.
2026-09-22 13:58:20 +01:00
Pratik Mankawde
3cc9f33041 fix(telemetry): publish the Prometheus ports on the host loopback
The StatsD port beside them was already loopback-bound with a comment saying
why; the Prometheus exporter and Prometheus itself were not. Prometheus
scrapes the collector as otel-collector:8889 over the compose network, so the
published port had no consumer.
2026-09-22 13:58:18 +01:00
Pratik Mankawde
3f510ecf44 fix(telemetry): publish local stack ports on the host loopback
The compose ports published on every host interface, so the collector's
unauthenticated OTLP receivers, Tempo, and the anonymous-admin Grafana were
reachable by anything that could route to the host.

Nothing consumed the published ports from off-host: containers address each
other by compose service name, and every doc and script uses localhost.
2026-09-22 13:58:17 +01:00
Pratik Mankawde
fb92c78c22 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-21 19:08:04 +01:00
Pratik Mankawde
826802a35f merge: bring the count-unit changes forward from phase-6
Resolved consensus-health.json. This branch re-exported that dashboard, so
phase-6's unit changes land in lines this branch had already rewritten. Took
this branch's file from the clean baseline merge, then applied si: and
decimals 0 to every count-unit panel left in it. The panel set matches that
baseline merge exactly.
2026-09-21 19:07:53 +01:00
Pratik Mankawde
5c8495e230 feat(telemetry): scale count units and drop meaningless decimals
Panels whose unit counts whole things switch from a custom `suffix:` unit to
`si:`, so a large value reads 15 k nodes/s instead of 15000 nodes/s. A custom
suffix unit is formatted by toFixedUnit, which never scales; si: goes through
SIPrefix and does.

decimals is set to 0 on the same panels, because a fraction of a ledger or a
node says nothing. The two settings need each other: on its own decimals 0
would round a slow rate down to zero, and si: shifts it to the milli step
instead.

Left alone: the one panel whose unit is a seconds-per-second ratio.

26 panels across 9 dashboards, plus 3 legends and one name placement.
2026-09-21 19:06:03 +01:00
Pratik Mankawde
202fefe896 feat(telemetry): scale count units and drop meaningless decimals
Panels whose unit counts whole things switch from a custom `suffix:` unit to
`si:`, so a large value reads 15 k nodes/s instead of 15000 nodes/s. A custom
suffix unit is formatted by toFixedUnit, which never scales; si: goes through
SIPrefix and does.

decimals is set to 0 on the same panels, because a fraction of a ledger or a
node says nothing. The two settings need each other: on its own decimals 0
would round a slow rate down to zero, and si: shifts it to the milli step
instead.

Left alone: the one panel whose unit is a seconds-per-second ratio.

20 panels across 5 dashboards.
2026-09-21 19:06:01 +01:00
Pratik Mankawde
b2aa831621 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-21 18:08:26 +01:00
Pratik Mankawde
6b8462cddf feat(telemetry): tidy dashboard legends and bar gauge styling
A panel legend drops ${__field.labels.series} where the panel title already
carries that name, and the query drops the label_replace that set the label,
since nothing reads it any more.

Horizontal bar gauges put the series name above the bar.

3 legends across 2 dashboards, and one bar gauge name placement.
2026-09-21 18:08:14 +01:00
Pratik Mankawde
4cbb2199e5 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-21 17:06:18 +01:00
Pratik Mankawde
94bb5313a2 merge: bring the dashboard annotation settings forward from phase-6
Resolved three dashboards. phase-7 rewrote consensus-health, rpc-performance
and transaction-overview, so both sides added the annotations block next to
lines that had already diverged. Rebuilt each file from the clean pre-change
merge of the two branches, then set the option on every panel, which is what
both sides intended. Panel sets match that baseline merge exactly.

All 107 timeseries and state-timeline panels on this branch now set multiLane,
and both state timelines set perPage.
2026-09-21 17:06:07 +01:00
Pratik Mankawde
0e4db42a34 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-21 17:04:05 +01:00
Pratik Mankawde
18e7333bbe feat(telemetry): add state-timeline pagination and multi-row annotations
Every timeseries and state-timeline panel sets
options.annotations.multiLane. The state-timeline panels set
options.perPage to 5.

62 panels across 5 dashboards.
2026-09-21 17:00:54 +01:00
Pratik Mankawde
5b4f328489 feat(telemetry): add state-timeline pagination and multi-row annotations
Every timeseries and state-timeline panel sets
options.annotations.multiLane. The state-timeline panels set
options.perPage to 5.

46 panels across 5 dashboards.
2026-09-21 17:00:52 +01:00
Pratik Mankawde
f666427570 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-18 09:26:33 +01:00
Pratik Mankawde
b972682955 merge: bring the suppressed-reference removal forward from phase-6
Resolved transaction-overview.json by composing both sides: this branch's
renamed span_calls_total metric and its 15s interval, with phase-6's removal of
the suppressed grouping and its new legend.
2026-09-18 09:26:17 +01:00
Pratik Mankawde
e87432fe3c fix(telemetry): drop the suppressed references the span no longer emits
tx.receive is created after the duplicate check, so it never carries a
suppressed attribute. The Tempo tag filter and the dashboard panel still
referenced it, and check_otel_naming rules C and D fail on a reference with no
constant behind it.

The panel keeps its remaining signal as a plain receive rate. How many relayed
copies were dropped is the transactions_duplicate traffic category, which does
not depend on a span.

The spanmetrics dimension goes too; a dimension for an attribute nothing sets
only widens the label set.
2026-09-18 09:24:56 +01:00
Pratik Mankawde
0c2b002206 merge: bring phase-7's gauges weak-ref refactor forward
Merges pratik/otel-phase7-native-metrics into pratik/otel-phase8-log-correlation.

Conflict was one TESTING.md hunk under "Nodes not reaching proposing state":
this branch renamed the node directories to Node-N in integration-test.sh,
phase-7 kept nodeN and expanded the [peer_private] explanation. Resolution
keeps this branch's Node-1 path (its own script uses that naming) and
phase-7's fuller prose citing peerfinder/Config.cpp.

Non-conflicting phase-7 changes come through: OTelCollector's gauges_ list
becomes weak_ptr, matching the earlier hooks_ change; the phase-6 revert of
the StatsD-test onCollectionReady() calls resolved against phase-7's version
that keeps them.
2026-09-14 21:09:56 +01:00
Pratik Mankawde
28773e903b merge: bring the phase-6 revert forward, keeping phase-7's collection lifecycle
Merges pratik/otel-phase6-statsd into pratik/otel-phase7-native-metrics.
Phase-6 dropped the three onCollectionReady() calls that had been added to
its StatsD test, because that method is only declared here on phase-7.
This branch's own copy of the file was unchanged from the merge base, so
the default merge would have silently deleted the calls from here too —
where they are needed, because this branch gates polling behind
onCollectionReady() in OTelCollectorImp::onTimer.

Resolution keeps both sides: phase-6's two new include lines
(Counter.h, Gauge.h) and phase-7's three onCollectionReady() calls plus
their doxygen and inline explanations. The merged file is exactly
phase-7's tip plus those two includes.

TESTING.md auto-merged cleanly; both sides added text under Test 1 in
different regions.
2026-09-14 21:07:59 +01:00
Pratik Mankawde
c92bb3e02e Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-14 20:34:51 +01:00
Pratik Mankawde
975238d7d4 docs(telemetry): fix the node log path, log level and Grafana span link
The config template wrote each node's log to a lowercase node{N} directory
while setting service_instance_id=Node-{N}. The collector takes the node name
from the log file's parent directory and stamps it as the Loki
service_instance_id label, so the logs carried a name no trace or metric
shared and nothing joined. Use Node-{N} and state the rule.

The template also set log_level to warning. Nothing in the pipeline filters on
severity; the constraint is that a log line carries trace context only when it
is emitted inside an active span. At warning the only such statements in the
consensus accept span are a catch path a healthy round never takes and a
periodic censorship warning. At info the CNF Val / CNF buildLCL pair writes one
line per accepted ledger, which is what makes this test's Step 1 findable.

Grafana 13 offers the link per span, labelled "Logs for this span", in the
span's Links row — not per trace. Fix the step and the expected-results row.

The example log line quoted a message that does not exist. The real in-span
RPC statement logs at debug, so the severity code is DBG; say which line to
look for under each test, since Test 2 now logs at info.

Drop the reference to workload/validate_telemetry.py: that file is not part of
this branch, and its instant-endpoint call uses seconds, so the nanoseconds
claim applied only to query_range.
2026-09-11 11:30:39 +01:00
Pratik Mankawde
e1ef6ba183 docs(telemetry): drop the inert insight endpoint from the test config template
On the OTel path only [insight] server is load-bearing. CollectorManager reads
endpoint and hands it to OTelCollector, which logs it at startup and routes
nothing with it; the real export endpoint is [telemetry] metrics_endpoint,
which the template already sets. service_instance_id and service_name in that
section are read and discarded.

Leaving the line invited an operator to reconcile a mismatch that has no
effect. integration-test.sh already emits only server=otel with the same
explanation, so the two now agree.
2026-09-11 11:30:17 +01:00
Pratik Mankawde
e0b9810a08 docs(telemetry): correct the standalone span table and bound the Tempo queries
The Payment destination was not a valid XRPL address — its base58 checksum
does not match — so Test 1 Step 4 and Test 2 Step 7 could never have returned
the tesSUCCESS they claim. Use a valid one and note that the destination does
not need to exist.

The Tempo search loop had no -G, so curl posted the parameters as a body,
Tempo answered 200 while ignoring the query, and every span name came back
non-zero. It also had no time bound, and Tempo keeps blocks for an hour, so a
re-run was answered by the previous run's traces. Add -G, RUN_START, and
start/end, matching what integration-test.sh already does.

Split the query list in two: 35 names that should be present, and 8 that need
a trigger neither test performs, where zero is the expected answer. Previously
two of the latter sat in the pass/fail list and read as failures.

Correct the standalone span table. consensus.mode_change fires once per round
start whether or not the mode changes, ledger.validate cannot fire because
checkAccept is unreachable in standalone, and the apply-stage, TxQ and ledger
families were missing rows. Give each "No" row the reason that actually
applies: the establish phase, a missing validator key, or no peers.

Also: ledger_accept is not required before submit, the teardown pgrep matched
more than this node, [peer_private] also disables the inbound listener, and
the 15-second wait covers Tempo but not Prometheus.
2026-09-11 11:30:00 +01:00
Pratik Mankawde
6f28d46682 merge: bring the component renames forward from phase7-native-metrics
Two conflicts, both where this branch's log-pipeline additions sat next to the
upstream spanmetrics -> span_metrics rename: the config header comment, which
this branch extended with a logs line, and the integration test, where the
log-correlation step precedes the span-metrics step. This branch's content is
kept in both and the rename carried into it.
2026-09-09 19:20:45 +01:00
Pratik Mankawde
366bcaa328 merge: bring the component renames forward from phase6-statsd
Four conflicts, all where this branch's replacement of the StatsD path with
native OTLP met the upstream spanmetrics -> span_metrics rename. This branch's
design wins in every case; the rename is carried into its text rather than
reverting it, so the connector, its pipeline references, the header comment,
the TESTING.md summary and the runbook all use span_metrics while keeping the
native-OTLP wording.

One addition beyond a straight take-a-side: publish the collector's health
check port. This branch restored the health_check extension and its own
TESTING.md polls http://localhost:13133/ to decide the collector is ready, but
the port was never published on this side of docker-compose.yml, so that check
could not pass from the host. Verified the merged config loads with no
deprecation warnings and that 13133 is published exactly once.

No metric name changed: traces_span_metrics_* already read that way before the
rename, which only ever touched component names and prose.
2026-09-09 19:19:22 +01:00
Pratik Mankawde
988015dc65 merge: bring the OTLP gRPC exporter rename forward from phase5-docs-deployment
Conflict in docker/telemetry/otel-collector-config.yaml, in the service
pipelines: this branch renamed the deprecated spanmetrics connector to
span_metrics and adds the statsd metrics pipeline, while upstream renamed the
deprecated otlp exporter to otlp_grpc. Both kept.

With both renames present the collector now starts with no deprecation
warnings at all, which was the point of the pair.
2026-09-09 19:17:07 +01:00
Pratik Mankawde
c0024c7c57 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-09 19:16:08 +01:00
Pratik Mankawde
b98a9d3d13 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-09 19:16:08 +01:00