Commit Graph

549 Commits

Author SHA1 Message Date
Pratik Mankawde
216a9c6c1b added annotations section to dashboards
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-15 13:36:38 +01:00
Pratik Mankawde
49e44259f7 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-15 11:45:46 +01:00
Pratik Mankawde
352c85d945 fix(telemetry): wrap legend identity labels in a single bracket
Group the node + perf-iac identity labels inside one pair of brackets in the
legend Display name so a series reads e.g. "Queue Depth [validator-0
test:pr:abc123 validator RIPD-7455]" on a perf node and "Queue Depth
[aws-dev-xrpl-1]" on a plain node. Absent perf-iac labels collapse to a single
space before the closing bracket; no empty "[]", no data impact. Covers the
eleven dashboards this branch owns.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 11:45:26 +01:00
Pratik Mankawde
e08b29016d Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-15 11:33:32 +01:00
Pratik Mankawde
d764238474 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-15 11:33:26 +01:00
Pratik Mankawde
e3452f9e30 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
# Conflicts:
#	docker/telemetry/otel-collector-config.yaml
2026-07-15 11:33:24 +01:00
Pratik Mankawde
5e37c5fae9 fix(telemetry): revert collector group-label build (cloud + Alloy configs)
The legend now uses raw labels in the dashboard Display name (see the paired
dashboard commit), so the collector no longer needs to build resource_group /
iac_group. Those derived labels also could not survive the dashboards' sum by()
aggregation, which is why the grouped approach was dropped.

Revert transform/legendgroups and its transform/cloudlabels datapoint copies
from the Grafana Cloud collector config and the in-repo Alloy config; both files
are now identical to their pre-change state.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 11:32:32 +01:00
Pratik Mankawde
b6433aa91f fix(telemetry): use raw-label legend Display name; drop collector group labels
The collector-built resource_group / iac_group labels did not survive the
dashboards' own PromQL aggregation: panels that sum by(service_instance_id,
xrpl_branch, xrpl_node_role, ...) drop any label not in the by() clause, so the
derived group labels were aggregated away and the Display name rendered empty.

Switch the legend Display name to the raw labels the panels already group by,
which are always present by construction:
  ${__field.labels.series} ${__field.labels.service_instance_id}
  ${__field.labels.xrpl_branch} ${__field.labels.xrpl_node_role}
  ${__field.labels.xrpl_work_item}
Absent labels render empty (no brackets, no gaps). Covers the eleven dashboards
this branch owns. The matching collector-config revert lands on the branches
that introduced those configs (phase-7 local, phase-10 cloud/alloy).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 11:32:22 +01:00
Pratik Mankawde
1f43baa0bb fix(telemetry): use raw-label legend Display name; drop collector group labels
The collector-built resource_group / iac_group labels did not survive the
dashboards' own PromQL aggregation: panels that sum by(service_instance_id,
xrpl_branch, xrpl_node_role, ...) drop any label not named in the by() clause,
so the derived group labels were aggregated away and the Display name rendered
empty ("Observing" with no node). Building them in the collector also coupled
every dashboard to a lockstep collector redeploy on every node.

Switch the legend Display name to reference the raw labels the panels already
group by, which are always present by construction and need no collector
support:
  ${__field.labels.series} ${__field.labels.service_instance_id}
  ${__field.labels.xrpl_branch} ${__field.labels.xrpl_node_role}
  ${__field.labels.xrpl_work_item}
${__field.labels.X} renders empty for a label a series lacks (verified in
Grafana), so dev/mainnet nodes show just "<name> <node>" and perf nodes add the
branch/role/work-item, with no empty brackets and no double spaces. Verified
live across timeseries, stat, bargauge, piechart, gauge and cross-node
aggregation panels.

Revert the transform/legendgroups processor from the local collector config;
this file is now identical to its pre-change state.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 11:32:09 +01:00
Pratik Mankawde
b4af979a42 docs(telemetry): refresh legend-group comment for bracketed label values
Update the config.alloy metric_statements comment to describe the current
bracket-baked, ", "-joined group-label values ("[node, mainnet]"), matching the
implementation. Comment-only; no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 19:14:37 +01:00
Pratik Mankawde
f18b03b8ae Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
# Conflicts:
#	docker/telemetry/grafana/dashboards/ledger-data-sync.json
2026-07-14 19:09:32 +01:00
Pratik Mankawde
c18f9cbf58 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-14 19:08:53 +01:00
Pratik Mankawde
ec487ff25d Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-14 19:08:53 +01:00
Pratik Mankawde
1d8176918c fix(telemetry): bake legend brackets into cloud + Alloy collector labels
Mirror the local collector change on the Grafana Cloud collector config and the
in-repo Alloy config: build the group-label VALUES with brackets and comma
separators (resource_group="[node, mainnet]",
iac_group="[branch, role, work-item]") in both trace and metric resource
statements, so span-derived and native metrics carry the same bracketed values.

This lets the dashboards use a bracket-free Display name and avoids the empty
"[]" that a templated bracket would leave when a group is absent (Grafana's
renameByRegex runs pre-interpolation and cannot strip it). Separator changes
from "-" to ", ".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 19:08:44 +01:00
Pratik Mankawde
53289203ed fix(telemetry): move legend brackets into collector label, drop empty []
The Display name templated the brackets: "${series} [${resource_group}]
[${iac_group}]". When a group label was absent the "${}" rendered empty,
leaving a literal "[]" in the legend, and the cleanup renameByRegex could not
remove it (Grafana runs that transform on the pre-interpolation template, which
has no "[]" -- it only appears after interpolation at render time).

Bake the brackets and comma separators into the group-label VALUES in the
collector instead (resource_group="[node, mainnet]",
iac_group="[branch, role, work-item]"), so the Display name is bracket-free
("${series} ${resource_group} ${iac_group}") and an absent group renders as
nothing. Separator changes from "-" to ", ". Drop the dead cleanup transform.

Covers this branch's local collector config and its three dashboards
(ledger-data-sync, network-traffic, overlay-traffic-detail).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 19:08:25 +01:00
Pratik Mankawde
dfcc091e7f fix(telemetry): move legend brackets into collector label, drop empty []
The Display name templated the brackets: "${series} [${resource_group}]
[${iac_group}]". When a group label was absent (e.g. iac_group on a non-perf
node) the "${}" rendered empty, leaving a literal "[]" in the legend. The
empty-bracket cleanup renameByRegex could not remove it: Grafana runs that
transform on the pre-interpolation Display name template, which contains no
"[]" -- the "[]" only appears after variable interpolation, at render time.

Fix it at the source: the collector now bakes the brackets and comma separators
into the group-label VALUES (e.g. resource_group="[node, mainnet]",
iac_group="[branch, role, work-item]"). The Display name becomes bracket-free --
"${series} ${resource_group} ${iac_group}" -- so an absent group is simply an
absent label and renders as nothing. Also switches the intra-group separator
from "-" to ", " (e.g. "[node, mainnet]"). Drop the now-dead cleanup transform
from every panel; value-mapping renameByRegex rules are preserved.

Collector-side label building lives in the paired commits on the collector
config branches. This commit is the dashboard half for the eleven dashboards
this branch owns.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 19:08:14 +01:00
Pratik Mankawde
2754cd888a Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-14 18:57:26 +01:00
Pratik Mankawde
a984055d92 fix(telemetry): drop redundant dimension bracket from grouped legends
Panels that group by a distinguishing dimension (consensus_mode, tx_type,
command, method, stage, ...) already fold that dimension's value into the
curated series name via label_replace(..., "series", "<prefix> $1", "<dim>",
"(.*)"). The migration also added a separate [${__field.labels.<dim>}] bracket,
so those legends rendered the value twice, e.g. "proposing [proposing] [...]"
on Consensus Mode Over Time.

Remove the dimension bracket from all 29 affected panel Display names, leaving
${__field.labels.series} [resource_group] [iac_group]; the series name already
conveys the dimension. The migration script no longer emits the bracket either.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 18:57:17 +01:00
Pratik Mankawde
8dbaf4effb fix(telemetry): build legend groups on spans so span metrics get them
In the Alloy config the spanmetrics connector derives span_calls_total /
span_duration_* from the tagged TRACES and sends them straight to the batch
processor, bypassing the metric resource statements. The grouped legend labels
were built only in metric_statements, so span-derived RED metrics carried no
resource_group / iac_group and span-based dashboard panels rendered empty group
legends.

Build the same labels in trace_statements too (mirroring how the tier attrs are
already set in both), so spans carry them before spanmetrics derives from them.
The reference otel collector is unaffected: there spanmetrics is a connector
feeding the metrics pipeline, which runs transform/legendgroups on all inputs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 18:42:01 +01:00
Pratik Mankawde
2bb9ebef65 fix(telemetry): build grouped legend labels in the in-repo Alloy config
The docker/telemetry/alloy/config.alloy node config feeds Grafana Cloud via the
OTLP gateway (which promotes resource attributes to labels server-side). Add the
resource_group / iac_group build to its metric resource statements so nodes
using this Alloy path get the same grouped legend labels the dashboards expect.

This is a generic node config (no perf-iac attrs), so iac_group's guard never
fires here and only resource_group is built; the iac_group statement is kept for
parity and activates automatically on a perf-comparison node. Guards check != ""
as well as != nil since OTTL Concat emits a stray separator for an empty element.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 18:36:22 +01:00
Pratik Mankawde
e93be29ccb Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-14 18:33:42 +01:00
Pratik Mankawde
61e6d0089f Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-14 18:33:35 +01:00
Pratik Mankawde
e33e6d3a40 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-14 18:33:35 +01:00
Pratik Mankawde
a0f0872ca6 fix(telemetry): set tooltip maxHeight to 600 per dashboard guideline
Eighteen timeseries panels in consensus-health and transaction-overview
carried tooltip maxHeight 500; the dashboard guideline specifies 600. Normalize
them so every panel's tooltip matches Mode:All / Sort:Desc / Max Height:600.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 18:33:25 +01:00
Pratik Mankawde
6ebb21d8a3 fix(telemetry): keep category names on Overlay Traffic bargauge
The "Overlay Traffic by Category (Bytes In)" bargauge names each bar by
category via 22 byName field-override displayNames (Transactions, Proposals,
...). The legend migration had also added a generic
${__field.labels.series} [...] Display name, which competed with those
curated names. Remove the generic Display name (and its now-unneeded
empty-bracket cleanup transform) from this panel so the category names stand;
this panel's identity is the traffic category, not the node/perf-iac grouping.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 18:33:16 +01:00
Pratik Mankawde
50b7cc67b2 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-14 18:29:05 +01:00
Pratik Mankawde
914e291dfc Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-14 18:29:05 +01:00
Pratik Mankawde
4f9c507c71 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-14 18:28:59 +01:00
Pratik Mankawde
6ad9b7108e fix(telemetry): guard cloud legend-group build against empty-string attributes
Mirror the local-config guard hardening on the Grafana Cloud collector config:
add != "" to the transform/legendgroups presence guards so an empty-string
perf-iac attribute (work_item | default('') on a no-work-item run) does not
produce iac_group="--". Empty-string sources now leave the group unset.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 18:28:50 +01:00
Pratik Mankawde
72db55233e fix(telemetry): guard legend-group build against empty-string attributes
The transform/legendgroups guards checked only != nil. The perf-iac collector
stamps xrpl.branch / xrpl.node.role / xrpl.work.item with an empty-string
default (work_item | default('')) when a run has no work item, and an empty
string is not nil. OTTL Concat renders a stray separator for an empty element,
so an all-empty perf-iac stamp would have produced iac_group="--".

Add != "" to every presence guard (iac_group sources, service.instance.id,
xrpl.network.type) so a group is built only from genuinely-present values and
left unset otherwise. Verified on a live collector: empty-string perf-iac attrs
leave iac_group absent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 18:28:39 +01:00
Pratik Mankawde
4a533cb41b fix(telemetry): build grouped legend labels on the Grafana Cloud path
Add the transform/legendgroups processor (introduced on phase-7 for the local
config) to the dual-export grafanacloud collector config, and copy the two
group labels onto datapoint attributes in transform/cloudlabels.

The local Prometheus scrape promotes resource attributes to labels via
resource_to_telemetry_conversion, but Grafana Cloud's OTLP ingest does not, so
resource_group / iac_group must be copied onto datapoint attributes the same
way service_instance_id / deployment_environment / xrpl_network_type already
are. legendgroups runs before cloudlabels in both metrics pipelines so the
built resource attributes are available to copy; the copies are presence-guarded
so an absent group is not created as an empty datapoint label.

Verified on a live collector (debug exporter): a perf node emits datapoint
attributes resource_group and iac_group; a non-perf node emits resource_group
only, with iac_group correctly absent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 18:08:57 +01:00
Pratik Mankawde
d2f52c578c Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
# Conflicts:
#	docker/telemetry/grafana/dashboards/network-traffic.json
#	docker/telemetry/grafana/dashboards/node-health.json
2026-07-14 18:07:15 +01:00
Pratik Mankawde
f682d0dc98 fix(telemetry): migrate remaining dashboards to grouped Display-name legends
Apply the collector-built grouped-label legend approach (introduced on
phase-7) to the eleven dashboards this branch and its ancestors own:
consensus-health, fee-market, job-queue, ledger-operations, node-health,
peer-network, peer-quality, rpc-pathfinding, rpc-performance,
transaction-overview, validator-health.

Per legend panel: drop the 11-rule renameByRegex chain and legendFormat
"__auto"; set a Standard-option Display name referencing the pre-built
resource_group / iac_group labels plus, when the panel groups by a
distinguishing dimension (e.g. command, tx_type, consensus_mode, stage,
method), that dimension as a leading bracket; keep a single generic
empty-bracket cleanup rule; set tooltip maxHeight to 600.

Value-mapping renameByRegex rules (e.g. consensus close-time
increased->Coarser) are preserved. Only labels a panel groups by appear in
its legend; filter-only selectors ($command with no by(command)) narrow data
but do not become legend groups. All 14 dashboards validate; no legend
renameByRegex chain remains.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 18:06:02 +01:00
Pratik Mankawde
8296012f71 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-14 18:05:01 +01:00
Pratik Mankawde
9b9e4787db Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
# Conflicts:
#	docker/telemetry/otel-collector-config.yaml
2026-07-14 18:04:52 +01:00
Pratik Mankawde
6911963939 fix(telemetry): grouped legend labels via collector, drop regex chain
The per-panel legend used legendFormat "__auto" reshaped by an 11-rule
renameByRegex chain. __auto sorts labels alphabetically and cannot group,
so the chain existed only to bolt grouping onto its output -- unmaintainable
and non-compliant with the dashboard legend guideline.

Build the grouped labels once at ingestion instead. A new transform/legendgroups
processor derives two resource attributes, promoted to Prometheus labels by
resource_to_telemetry_conversion:
  - resource_group: service_instance_id[-xrpl_network_type]
  - iac_group:      xrpl_branch-xrpl_node_role-xrpl_work_item (perf runs only)
Each is left unset when its sources are absent, so Prometheus drops the empty
label and the legend shows nothing for it. Concat is presence-guarded because
OTTL Concat renders "<nil>" for an absent attribute.

Dashboards now use a Standard-option Display name that references the pre-built
labels (${__field.labels.series} [resource_group] [iac_group]); ${__field.labels.X}
renders empty for a missing label, so absent groups vanish. A single generic
renameByRegex strips the resulting empty "[]" bracket. legendFormat is dropped
(Display name supersedes it). Tooltip maxHeight set to 600 per guideline.

Applied here to the three dashboards this branch introduced (ledger-data-sync,
network-traffic, overlay-traffic-detail); value-mapping renameByRegex rules are
preserved. Verified end-to-end on a live local stack (collector -> Prometheus ->
Grafana): perf and dev nodes render correct, clutter-free legends.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 18:04:03 +01:00
Pratik Mankawde
9bbc71bda4 chore(telemetry): stop tracking local grafanacloud dashboard copies
The docker/telemetry/grafanacloud/*.json files are local deploy variants
(they pin the Grafana Cloud datasource UID) and are regenerated from the
canonical grafana/dashboards/*.json. Untrack them and gitignore so they
stay local-only and never drift in version control.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 12:32:26 +01:00
Pratik Mankawde
49fee77ca9 fix(telemetry): conditional filter-dimension legends + 30m span-nulls
Panel legends only showed the node (service_instance_id); the perf-iac
filter dimensions (xrpl_branch, xrpl_node_role) never appeared, so
baseline-vs-test series were indistinguishable.

Each node-referencing target injects its curated name via
label_replace(...,"series",...) and uses the "__auto" legend, which shows
only labels that vary across the displayed series. A word-boundary-anchored
renameByRegex chain reshapes the auto output into
"Name [node, role]-[branch, work_item]", collapsing empty groups so a
dimension appears only when the filter widens it. Aggregating queries gain
xrpl_branch/xrpl_node_role in their by() so those labels survive; multi-
dimension legends keep their rich label set and append branch/role.

Quantile panels surface the quantile via the panel title (q$quantile),
since {{quantile}} is a dashboard variable, not a series label.

Timeseries panels connect only null gaps shorter than 30m
(spanNulls=1800000): brief scrape gaps bridge, genuine outages stay broken.

Non-node panels (heatmaps, rows, tables, state timelines, category/bucket
series) are left unchanged. Applied to all 14 dashboards.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 12:18:52 +01:00
Pratik Mankawde
af9a2e8bea Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-14 12:18:19 +01:00
Pratik Mankawde
f8bc8dee48 fix(telemetry): conditional filter-dimension legends + 30m span-nulls
Panel legends only showed the node (service_instance_id); the perf-iac
filter dimensions (xrpl_branch, xrpl_node_role) never appeared, so
baseline-vs-test series were indistinguishable.

Each node-referencing target injects its curated name via
label_replace(...,"series",...) and uses the "__auto" legend, which shows
only labels that vary across the displayed series. A word-boundary-anchored
renameByRegex chain reshapes the auto output into
"Name [node, role]-[branch, work_item]", collapsing empty groups so a
dimension appears only when the filter widens it. Aggregating queries gain
xrpl_branch/xrpl_node_role in their by() so those labels survive; multi-
dimension legends keep their rich label set and append branch/role.

Quantile panels surface the quantile via the panel title (q$quantile),
since {{quantile}} is a dashboard variable, not a series label.

Timeseries panels connect only null gaps shorter than 30m
(spanNulls=1800000): brief scrape gaps bridge, genuine outages stay broken.

Non-node panels (heatmaps, rows, tables, state timelines, category/bucket
series) are left unchanged. Applied to all 14 dashboards.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 12:18:04 +01:00
Pratik Mankawde
8c34fd9f5a fix(telemetry): conditional filter-dimension legends + 30m span-nulls
Panel legends only showed the node (service_instance_id); the perf-iac
filter dimensions (xrpl_branch, xrpl_node_role) never appeared, so
baseline-vs-test series were indistinguishable.

Each node-referencing target injects its curated name via
label_replace(...,"series",...) and uses the "__auto" legend, which shows
only labels that vary across the displayed series. A word-boundary-anchored
renameByRegex chain reshapes the auto output into
"Name [node, role]-[branch, work_item]", collapsing empty groups so a
dimension appears only when the filter widens it.

Timeseries panels connect only null gaps shorter than 30m
(spanNulls=1800000): brief scrape gaps bridge, genuine outages stay broken.

Dashboards (stable panel set): ledger-data-sync, network-traffic,
overlay-traffic-detail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 12:17:39 +01:00
Pratik Mankawde
cb1ca17442 feat(telemetry): add Grafana Cloud dashboard copies
Grafana Cloud variants of the 14 xrpld dashboards. Identical to the local
grafana/dashboards/ originals except the DS_PROMETHEUS / DS_TEMPO datasource
template variables have their `current` value pinned to the Grafana Cloud
datasource UIDs (grafanacloud-ripplex-prometheus / -traces).

Includes the ledger-data-sync data-type split and node-health mixed-unit
split + Cache Hit Ratio fix merged forward from phases 7 and 9.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-13 21:12:54 +01:00
Pratik Mankawde
8919769721 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
# Conflicts:
#	docker/telemetry/grafana/dashboards/ledger-data-sync.json
#	docker/telemetry/grafana/dashboards/node-health.json
2026-07-13 20:57:35 +01:00
Pratik Mankawde
1a2b210214 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-13 20:51:40 +01:00
Pratik Mankawde
71686b0d52 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-13 20:51:34 +01:00
Pratik Mankawde
674c9a8496 fix(telemetry): correct NodeStore Cache Hit Ratio PromQL vector matching
The Cache Hit Ratio panel divided two nodestore_state series that carry
different `metric` label values (node_reads_hit / node_reads_total). Without
ignoring(metric) the two vectors share no common label set, so the division
matched nothing and the panel showed "No data".

Add `ignoring(metric)` to the division. Verified live against mainnet: now
returns a valid ratio (~0.11-0.23). Pre-existing bug, surfaced when the panel
was split out from NodeStore Read/Write Totals.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-13 20:48:51 +01:00
Pratik Mankawde
d06faec407 fix(telemetry): split mixed-unit node-health panels into single-unit panels
Three panels mixed incompatible units on one Y-axis (count vs ms, ops vs
percent), causing the small-magnitude series to flatline to zero.

- Last Close Info -> Last Close — Proposers (short) + Converge Time (ms)
- NodeStore Read/Write Totals -> Throughput (ops) + Cache Hit Ratio (%)
- NodeStore Read Threads & Duration -> Read Threads (short) + Busy Ratio (%)

Each new panel has a single unit; per-series unit overrides removed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-13 20:29:52 +01:00
Pratik Mankawde
6f9319e319 refactor(telemetry): split ledger-data-sync panels by data type
Each family panel (7-8 series) was overloaded. Split by Ledger/Transaction/
Account State so each panel shows only related series. Get+Share stay
together within a type.

- Ledger Data Exchange: 8 targets -> 3 panels
- Ledger Share/Get Traffic: 8 targets -> 3 panels
- GetObject Traffic by Type: 8 targets -> 3 panels
- GetObject Messages by Type: 7 targets -> 3 + specials panel
- GetObject Aggregate & Special: renamed, kept as one
- Overlay Traffic Heatmap: unchanged

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-13 20:29:38 +01:00
Pratik Mankawde
e103bd32f6 fix(telemetry): flush spanmetrics every 15s so Cloud rate panels render
Span-derived metrics (span_calls_total, span_duration_milliseconds_*) were
emitted at the spanmetrics connector default of 60s. Grafana Cloud stores one
datapoint per flush, but dashboard panels use rate(...[$__rate_interval]),
whose window at short ranges (<=1h) falls below 60s — leaving <2 samples per
window, so those panels rendered "No data" on the Cloud datasource. (Local
Prometheus scrapes at 15s, so local dashboards were unaffected.)

Set metrics_flush_interval: 15s so every rate window holds >=2 samples at any
dashboard range. Trade-off: ~4x span-metric datapoints pushed to Cloud. Native
insight metrics (ledgermaster_*, jobq_*, peer_quality) already push faster and
were never affected.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-07-13 18:08:29 +01:00
Pratik Mankawde
4f722052d6 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
Brings dashboard panel units, validator-health full-width layout, and filter
description polish into phase 10. The validator-health conflict was resolved
by keeping phase 10's per-target datasource uids and re-applying the unit,
layout, and description changes on top.
2026-07-13 15:46:34 +01:00