Commit Graph

16837 Commits

Author SHA1 Message Date
Pratik Mankawde
4bf76ce2ff metrics_flush_interval = "15s"
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-09-22 15:11:28 +01:00
Pratik Mankawde
6ed6189ba9 docs(telemetry): re-apply the comment rewrite lost to a concurrent commit
Commit 54809960ff removed the site measurements from the three collector
comment blocks, but a later commit on this branch was built from an index
holding the pre-rewrite blobs, so the tree reverted while the commit stayed in
the log. History is append-only, so this re-applies the same three blocks.

No non-comment line changes. The bucket lists are untouched and still agree
across the two collector configs and the Alloy config.
2026-09-22 15:07:10 +01:00
Pratik Mankawde
d3f9fa1f67 fix(telemetry): make the job latency panel readable at any time range
The gauge showed a p99 pooled across every job type, so a slow rare job could
not surface: sweep sat at 5 s while the dial read 20 ms, because a
count-weighted p99 follows the frequent jobs and those are fast.

It is now a bar gauge over per-job-type p99, which is what the panel is for,
and a bar gauge holds tens of values where a dial holds one. The series keep
job_type so each bar names its job.

max is set to the red threshold, so every bar shares one 0 to 1 s scale
instead of each dial auto-scaling to its own data.

The targets are instant queries. Left as range queries the bar count followed
the dashboard's time range rather than the current series count, reaching 531
series over a week.
2026-09-22 15:04:40 +01:00
Pratik Mankawde
54809960ff docs(telemetry): remove site measurements from the collector comments
The repo is public, so a comment must not carry one-site measurements or
claim a prior state of the source. Three comments did both.

The tail-sampling comment called itself "Box-only (uncommitted)", which names
our deployment and was untrue. It now states what the sampling does, and adds
the consequence that matters: Log.cpp writes a trace id only for a sampled
span, so sampling after that decision leaves log-to-trace links resolving to
nothing.

The histogram bucket comments justified each edge with measured percentages
and durations, and said what the buckets "used to" do. The mechanism is
unchanged and now stated without the figures. Also drops a duplicated line in
the Alloy copy. The bucket lists themselves are untouched and still agree
across all three files.
2026-09-22 15:00:30 +01:00
Pratik Mankawde
a1b3b4486a docs(telemetry): stop tracking the validator list files
The two xrpld-telemetry configs name a file in [validators_file], and xrpld
exits at startup when that file is missing. Neither list ships now: each
developer creates their own from cfg/validators-example.txt.

The runbook says so in "Run against a live network", ahead of the run command.
2026-09-22 14:56:57 +01:00
Pratik Mankawde
318fe0d10d Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-22 14:07:04 +01:00
Pratik Mankawde
46ca2daf2f Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-22 14:07:04 +01:00
Pratik Mankawde
2de992f334 fix(telemetry): give Ledger Data & Sync an auto-refresh interval
The dashboard had no top-level refresh key at all, so auto-refresh was off
while every other dashboard refreshes at 30s.
2026-09-22 14:07:02 +01:00
Pratik Mankawde
07ec48d717 fix(telemetry): point template variables at the datasource variable
Sixteen template variables named a datasource by the literal uid "prometheus"
or "loki". Neither exists on the Grafana instance, so those variables resolved
to nothing and their pickers offered only All.

That also broke the panels. With an empty variable the selector becomes
job_type=~"" rather than job_type=~".*", which matches no series, so the
Current Job Latency gauge had no data and its renderer raised the plugin error
banner on every refresh.

Every working dashboard already references ${DS_PROMETHEUS} or ${DS_LOKI},
which the dashboard declares in templating.list. These now match.

job-queue 6, ledger-data-sync 5, log-derived-insights 3, overlay-traffic-detail 2.
2026-09-22 14:06:49 +01:00
Pratik Mankawde
850ed7ec2e fix(telemetry): slow the Job Queue dashboard refresh to 30s
Every other dashboard refreshes at 30s. This one was at 5s, which re-ran all
of its queries twelve times a minute for no benefit.
2026-09-22 14:06:47 +01:00
Pratik Mankawde
0dccf29b64 fix(telemetry): publish the Loki port on the host loopback
Grafana reaches Loki as loki:3100 over the compose network and the collector
exports to it in-network, so the published port was reachable off-host for no
consumer.
2026-09-22 13:58:20 +01:00
Pratik Mankawde
a42f68cf89 merge: bring the count-unit changes forward from phase-8
Resolved nine dashboards. This branch re-exported all of them, so the upstream
unit changes land in lines this branch had already rewritten. Took this
branch's files from the clean baseline merge, then applied si: and decimals 0
to every count-unit panel left in them. Every panel set matches that baseline
merge exactly.

The three si:drops panels keep no decimals setting, as before.
2026-09-21 19:09:03 +01:00
Pratik Mankawde
fb92c78c22 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-21 19:08:04 +01:00
Pratik Mankawde
826802a35f merge: bring the count-unit changes forward from phase-6
Resolved consensus-health.json. This branch re-exported that dashboard, so
phase-6's unit changes land in lines this branch had already rewritten. Took
this branch's file from the clean baseline merge, then applied si: and
decimals 0 to every count-unit panel left in it. The panel set matches that
baseline merge exactly.
2026-09-21 19:07:53 +01:00
Pratik Mankawde
7d38d1bff8 feat(telemetry): scale count units and drop meaningless decimals
Panels whose unit counts whole things switch from a custom `suffix:` unit to
`si:`, so a large value reads 15 k nodes/s instead of 15000 nodes/s. A custom
suffix unit is formatted by toFixedUnit, which never scales; si: goes through
SIPrefix and does.

decimals is set to 0 on the same panels, because a fraction of a ledger or a
node says nothing. The two settings need each other: on its own decimals 0
would round a slow rate down to zero, and si: shifts it to the milli step
instead.

Left alone: the one panel whose unit is a seconds-per-second ratio.

42 panels across 10 dashboards, plus 5 name placements and one title.
2026-09-21 19:06:05 +01:00
Pratik Mankawde
5c8495e230 feat(telemetry): scale count units and drop meaningless decimals
Panels whose unit counts whole things switch from a custom `suffix:` unit to
`si:`, so a large value reads 15 k nodes/s instead of 15000 nodes/s. A custom
suffix unit is formatted by toFixedUnit, which never scales; si: goes through
SIPrefix and does.

decimals is set to 0 on the same panels, because a fraction of a ledger or a
node says nothing. The two settings need each other: on its own decimals 0
would round a slow rate down to zero, and si: shifts it to the milli step
instead.

Left alone: the one panel whose unit is a seconds-per-second ratio.

26 panels across 9 dashboards, plus 3 legends and one name placement.
2026-09-21 19:06:03 +01:00
Pratik Mankawde
202fefe896 feat(telemetry): scale count units and drop meaningless decimals
Panels whose unit counts whole things switch from a custom `suffix:` unit to
`si:`, so a large value reads 15 k nodes/s instead of 15000 nodes/s. A custom
suffix unit is formatted by toFixedUnit, which never scales; si: goes through
SIPrefix and does.

decimals is set to 0 on the same panels, because a fraction of a ledger or a
node says nothing. The two settings need each other: on its own decimals 0
would round a slow rate down to zero, and si: shifts it to the milli step
instead.

Left alone: the one panel whose unit is a seconds-per-second ratio.

20 panels across 5 dashboards.
2026-09-21 19:06:01 +01:00
Pratik Mankawde
b69c7631de merge: bring the legend and bar gauge changes forward from phase-8
Resolved network-traffic.json. Both branches own a version of the Peer
Disconnects legend: phase-7 had the older "[instance branch role work_item]"
identity form, and this branch rewrote the same line to use xrpl_ident. Kept
this branch's line, then applied the same change to it, so the legend drops
the series token and the query drops the label_replace that set it.

The incoming side's only edit to that file was those two lines, so nothing
else was superseded.
2026-09-21 18:10:35 +01:00
Pratik Mankawde
b2aa831621 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-21 18:08:26 +01:00
Pratik Mankawde
3036ada2e7 feat(telemetry): tidy dashboard legends and bar gauge styling
A panel legend drops ${__field.labels.series} where the panel title already
carries that name, and the query drops the label_replace that set the label,
since nothing reads it any more.

Horizontal bar gauges put the series name above the bar.

40 legends across 5 dashboards, 2 bar gauges to the LCD style, 5 name placements.
2026-09-21 18:08:16 +01:00
Pratik Mankawde
6b8462cddf feat(telemetry): tidy dashboard legends and bar gauge styling
A panel legend drops ${__field.labels.series} where the panel title already
carries that name, and the query drops the label_replace that set the label,
since nothing reads it any more.

Horizontal bar gauges put the series name above the bar.

3 legends across 2 dashboards, and one bar gauge name placement.
2026-09-21 18:08:14 +01:00
Pratik Mankawde
81a2406092 merge: bring the dashboard annotation settings forward from phase-8
Resolved four dashboards. This branch re-exported consensus-health, node-health,
rpc-performance and transaction-overview, so both sides touched the same option
blocks: upstream added the annotations block where this branch's export already
carried it set to false. Rebuilt each file from the clean pre-change merge of the
two branches, then set the option on every panel. Panel sets match that baseline
merge exactly, so the panels this branch's export drops are unchanged by the
resolution.

All 189 timeseries and state-timeline panels on this branch now set multiLane,
and all four state timelines set perPage.
2026-09-21 17:07:06 +01:00
Pratik Mankawde
4cbb2199e5 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-21 17:06:18 +01:00
Pratik Mankawde
94bb5313a2 merge: bring the dashboard annotation settings forward from phase-6
Resolved three dashboards. phase-7 rewrote consensus-health, rpc-performance
and transaction-overview, so both sides added the annotations block next to
lines that had already diverged. Rebuilt each file from the clean pre-change
merge of the two branches, then set the option on every panel, which is what
both sides intended. Panel sets match that baseline merge exactly.

All 107 timeseries and state-timeline panels on this branch now set multiLane,
and both state timelines set perPage.
2026-09-21 17:06:07 +01:00
Pratik Mankawde
0e4db42a34 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-21 17:04:05 +01:00
Pratik Mankawde
73eb8336cc feat(telemetry): add state-timeline pagination and multi-row annotations
Every timeseries and state-timeline panel sets
options.annotations.multiLane. The state-timeline panels set
options.perPage to 5.

99 panels across 13 dashboards. Seven of those already carried the option from
this branch's node-health export, so they flip here and not upstream.
2026-09-21 17:03:53 +01:00
Pratik Mankawde
18e7333bbe feat(telemetry): add state-timeline pagination and multi-row annotations
Every timeseries and state-timeline panel sets
options.annotations.multiLane. The state-timeline panels set
options.perPage to 5.

62 panels across 5 dashboards.
2026-09-21 17:00:54 +01:00
Pratik Mankawde
5b4f328489 feat(telemetry): add state-timeline pagination and multi-row annotations
Every timeseries and state-timeline panel sets
options.annotations.multiLane. The state-timeline panels set
options.perPage to 5.

46 panels across 5 dashboards.
2026-09-21 17:00:52 +01:00
Pratik Mankawde
82aeac5c0e fix(core): include <string> for PerfLog's job hooks
jobQueue, jobStart and jobFinish take their name as std::string const&, so this
header has to provide <string> itself. The rpc hooks moved to string_view and
the include went with them, which include-cleaner reports as an error.
2026-09-21 15:17:51 +01:00
Pratik Mankawde
1bffe264f0 fix(test): follow PerfLog's rpc hooks to string_view
PerfLog takes the method name as a std::string_view, so SilentPerfLog's three
rpc overrides stopped overriding anything and left the class abstract.
2026-09-21 12:52:13 +01:00
Pratik Mankawde
5e398cd6b7 merge: bring develop forward into phase-9 metric gap fill
Resolved five files.

PerfLogImp.cpp: PerfLog now takes its method names as string_view, so the
methods map stores the map key rather than a char const*. Kept this branch's
scoped lock so the OTel recording still runs after methodsMutex is released,
and kept both the NetworkOPs and metric-macro includes.

PeerImp.h: took develop's earlier `protected:`, which also covers
processLedgerRequest, and kept recordGetObjectMetrics inside it. Dropped the
comment naming one test, since four test files now rely on that access.

TMGetObjectByHash_test.cpp: kept this branch's suite. It already runs develop's
three bound cases (limit+1, limit, limit-1) as testReplyLimit, plus the charge
and hit/miss cases. develop's extracted PeerTest.h stays, used by its four new
overlay tests.

ordering.txt: regenerated. Both perflog edges are real, to xrpl.server and to
xrpl.telemetry.

.cspell.config.yaml: kept both added words.
2026-09-21 12:51:31 +01:00
Pratik Mankawde
6174d03ce0 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-21 12:46:00 +01:00
Pratik Mankawde
05ce773671 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-21 12:45:59 +01:00
Pratik Mankawde
243913250b Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-21 12:45:59 +01:00
Pratik Mankawde
c8a8da6fd6 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-21 12:45:55 +01:00
Pratik Mankawde
e5d28ccbc4 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-21 12:45:54 +01:00
Pratik Mankawde
870367d71e merge: bring develop forward into phase-3 tx tracing
Resolved src/libxrpl/tx/applySteps.cpp. calculateBaseFee now returns
std::expected<XRPAmount, TER>, so invokePreclaim rejects the transaction when
the fee cannot be computed. Kept that inside this branch's preclaimTer wrapper,
so the returned error also reaches the span's ter_result attribute.
2026-09-21 12:45:45 +01:00
Pratik Mankawde
c030880a36 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-21 12:44:43 +01:00
Pratik Mankawde
806473a9e9 fix(rpc): pass the command name to the span as a string_view
callMethod now takes its name as a std::string_view, which has no c_str().
setAttribute already accepts a string_view, so hand it the name directly.
2026-09-21 12:44:34 +01:00
Pratik Mankawde
d16f2c4181 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-21 12:43:48 +01:00
Pratik Mankawde
f48a2b4871 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-21 12:43:44 +01:00
Pratik Mankawde
e25c1a5920 Merge remote-tracking branch 'origin/develop' into pratik/otel-phase1a-plan-docs 2026-09-21 12:43:23 +01:00
Bart
f6b51f0b8b ci: Say when a package publish is a dry run (#8247)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-09-18 18:31:07 +00:00
Pratik Mankawde
9d462dfcaf Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-18 10:09:50 +01:00
Pratik Mankawde
8fef4b3cc6 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-18 10:09:50 +01:00
Pratik Mankawde
8a5e831448 docs(telemetry): drop the suppressed row from the second attribute table
The same attribute was listed in two tables in this file. tx.receive does not
set it, because the span starts after the duplicate check.
2026-09-18 10:09:26 +01:00
Pratik Mankawde
80b737a41a Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-18 10:07:53 +01:00
Pratik Mankawde
3dc08bf4ce Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-18 10:07:53 +01:00
Pratik Mankawde
44447e341b Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-18 10:07:53 +01:00
Pratik Mankawde
017e4bad42 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-18 10:07:53 +01:00