Commit Graph

253 Commits

Author SHA1 Message Date
Pratik Mankawde
36e4056869 feat(telemetry): merge native per-method RPC panels into RPC Performance
Fold the 7 panels from the separate RPC Performance (OTel) dashboard into
RPC Performance as a labelled 'Per-Method Detail' section, and retire the
duplicate dashboard. The two showed the same subject (RPC) from two data
sources; one dashboard with two sections is clearer and loses no panel.
Adds the $method template variable for per-method filtering.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 22:26:19 +01:00
Pratik Mankawde
adb4c7cdb4 docs(telemetry): re-clean 5 reworked dashboards on phase9 structure, retag, strip suffixes
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 22:23:25 +01:00
Pratik Mankawde
cc3cb31cd9 docs(telemetry): drop pipeline prefixes from node-health section headers
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 22:14:07 +01:00
Pratik Mankawde
10499f9f1a docs(telemetry): re-clean consensus and node-health descriptions on phase9 structure
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 22:12:51 +01:00
Pratik Mankawde
42abeb70a7 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/grafana/dashboards/consensus-health.json
#	docker/telemetry/grafana/dashboards/ledger-operations.json
#	docker/telemetry/grafana/dashboards/peer-network.json
#	docker/telemetry/grafana/dashboards/rpc-performance.json
#	docker/telemetry/grafana/dashboards/system-ledger-data-sync.json
#	docker/telemetry/grafana/dashboards/system-network-traffic.json
#	docker/telemetry/grafana/dashboards/system-node-health.json
#	docker/telemetry/grafana/dashboards/system-overlay-traffic-detail.json
#	docker/telemetry/grafana/dashboards/system-rpc-pathfinding.json
#	docker/telemetry/grafana/dashboards/transaction-overview.json
2026-07-03 22:06:39 +01:00
Pratik Mankawde
958e379e0a docs(telemetry): retag and rewrite native dashboard descriptions to describe data not pipeline
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 22:02:05 +01:00
Pratik Mankawde
e86c1adc5d Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-03 21:57:45 +01:00
Pratik Mankawde
b0d46af0ca Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-03 21:57:27 +01:00
Pratik Mankawde
50377df271 docs(telemetry): rewrite dashboard descriptions and tags to describe data not pipeline
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 21:56:08 +01:00
Pratik Mankawde
7befb16705 docs(telemetry): drop (System Metrics) suffix, retag, rewrite descriptions to describe data not pipeline
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 21:54:07 +01:00
Pratik Mankawde
9a4a97cdec docs(telemetry): add dashboard-level descriptions for the info button
Consensus Health, Ledger Operations, and Transaction Overview had no
top-level description, so Grafana showed no info button. Add a one-line
summary of what each dashboard shows and its data source.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 21:24:36 +01:00
Pratik Mankawde
55712c1388 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-03 21:18:47 +01:00
Pratik Mankawde
b0197105ea Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-03 21:18:47 +01:00
Pratik Mankawde
400f84aafe Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-03 21:18:38 +01:00
Pratik Mankawde
b59b3c1bb4 fix(telemetry): correct service.name to xrpld and restore rename README
The integration-test.sh trace queries used resource.service.name="rippled",
but the OTel resource attribute is "xrpld" (TelemetryConfig.cpp, MetricsRegistry.cpp),
so every span check and the service-registration check returned no results.

Also restore .github/scripts/rename/README.md, which an overzealous
find-and-replace had corrupted into circular no-ops (e.g. rename "xrpld" to
"xrpld"). The file documents the rename process and must retain the source
names (rippled/ripple); restored to match develop.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 21:18:22 +01:00
Pratik Mankawde
1a279e3edc Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-03 20:42:12 +01:00
Pratik Mankawde
5ade9431ef Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	docker/telemetry/grafana/dashboards/statsd-ledger-data-sync.json
#	docker/telemetry/grafana/dashboards/statsd-rpc-pathfinding.json
2026-07-03 20:42:07 +01:00
Pratik Mankawde
c669c20bc0 chore(telemetry): drop redundant rippled dashboard tag
Every dashboard already lives in the rippled/xrpld dashboard folder and
carries domain-specific tags (consensus, rpc, statsd, etc.), so the
leading "rippled" tag added no filtering value and only cluttered the
tag list.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 20:41:27 +01:00
Pratik Mankawde
754894c6a0 chore(telemetry): drop redundant xrpld dashboard tag
Every dashboard already lives in the xrpld folder and carries a
domain-specific tag (rpc, consensus, statsd, etc.), so the leading
"xrpld" tag added no filtering value and only cluttered the tag list.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 20:31:47 +01:00
Pratik Mankawde
93f776fda4 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-03 20:01:54 +01:00
Pratik Mankawde
84eb08eb8c Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-03 20:01:54 +01:00
Pratik Mankawde
fdfa651006 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	OpenTelemetryPlan/09-data-collection-reference.md
2026-07-03 20:01:44 +01:00
Pratik Mankawde
f81919af05 docs(telemetry): document tier template variables and label checks
- 09-data-collection-reference: add $service_name, $deployment_environment,
  $xrpl_network_type to the template-variable table; note they appear on all
  dashboards and point to the runbook's Deployment Tiers section.
- TESTING.md: add a Prometheus check verifying the tier labels
  (deployment_environment, xrpl_network_type, service_name) are present on
  metric series.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 19:56:15 +01:00
Pratik Mankawde
55404b7011 feat(telemetry): add deployment-tier filters to phase 9 dashboards
Add $service_name, $deployment_environment, $xrpl_network_type template
variables to the metric-gap-fill dashboards and wire them into every
panel query that filters by $node, so these dashboards can be sliced by
tier like the rest of the set.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 19:44:05 +01:00
Pratik Mankawde
8f14b40fbe Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/grafana/dashboards/consensus-health.json
#	docker/telemetry/grafana/dashboards/ledger-operations.json
#	docker/telemetry/grafana/dashboards/system-node-health.json
#	docker/telemetry/grafana/dashboards/system-rpc-pathfinding.json
#	docker/telemetry/grafana/dashboards/transaction-overview.json
2026-07-03 19:44:00 +01:00
Pratik Mankawde
20179dc0ed feat(telemetry): tag logs with deployment tier at the collector
Add resource/tier to the logs pipeline so log records carry the same
deployment.environment and xrpl.network.type attributes as traces and
metrics. This lets a single Grafana stack filter logs by tier alongside
the other signals. resource/logs still runs first to set service.name
and the job label; resource/tier then adds the tier attributes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 19:42:41 +01:00
Pratik Mankawde
294b167355 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
# Conflicts:
#	docker/telemetry/otel-collector-config.yaml
2026-07-03 19:42:25 +01:00
Pratik Mankawde
f55f8c59fc feat(telemetry): stamp service.name and network on native metrics
The native OTel metrics path hard-coded service.name="xrpld" and stamped
no network attribute, while traces stamped a configurable service.name
and xrpl.network.type. Metrics therefore could not be filtered by service
or network. Align the two paths:

- OTelCollector::New / OTelCollectorImp gain serviceName + networkType
  params. service.name uses the configured value (default "xrpld" when
  unset, preserving today's behavior); xrpl.network.type is stamped when
  provided. The key is a string literal because beast/insight sits below
  the telemetry module and cannot include its SpanNames const.
- CollectorManager reads service_name from [insight], falling back to the
  [telemetry] value, and receives the network type from the caller.
- Application derives the network type once via the shared
  telemetry::networkTypeFromId, now declared in Telemetry.h and moved out
  of an anonymous namespace so the trace and metric paths reuse a single
  0/1/2 -> mainnet/testnet/devnet mapping (no duplication).

Dashboards (5 system-* files): add $service_name, $deployment_environment,
$xrpl_network_type template variables and wire them into every panel query
that filters by $node.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 19:40:59 +01:00
Pratik Mankawde
3c0ac8f257 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	docker/telemetry/otel-collector-config.yaml
2026-07-03 19:31:40 +01:00
Pratik Mankawde
3d4119a3ba feat(telemetry): filter metrics by deployment tier across dashboards
Extend the deployment-tier tagging to the metrics path and expose it in
the dashboards so a single Grafana stack can be sliced by environment,
network, and service:

Collector (otel-collector-config.yaml):
- prometheus exporter: enable resource_to_telemetry_conversion so
  resource attributes (deployment.environment, xrpl.network.type,
  service.name, service.instance.id) become Prometheus labels.
- metrics pipeline: run resource/tier so statsd metrics are tagged too.
- spanmetrics: add the tier keys to the resource-metrics grouping key
  so per-tier series stay grouped separately.

Dashboards (5 files):
- add $service_name, $deployment_environment, $xrpl_network_type
  template variables, named to match their Prometheus labels.
- inject the matching label filters into every panel query that
  already filters by $node.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 19:30:53 +01:00
Pratik Mankawde
50c7d672d4 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	docker/telemetry/otel-collector-config.yaml
2026-07-03 19:12:11 +01:00
Pratik Mankawde
04df7fd92c Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-07-03 19:11:30 +01:00
Pratik Mankawde
d79a41dde6 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-07-03 19:11:30 +01:00
Pratik Mankawde
78f3595bc1 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-07-03 19:11:30 +01:00
Pratik Mankawde
ebc355a465 feat(telemetry): tag traces with deployment tier at the collector
Add a resource/tier processor so each collector stamps the deployment
tier onto the signals it forwards, enabling a single Grafana stack to
hold data from many collectors and filter by tier:

- deployment.environment (upsert): the collector is authoritative for
  its environment (local/ci/test/prod).
- xrpl.network.type (insert): the xrpld node already stamps its own
  chain, so the collector only fills this when absent, never
  overwriting a node's real network.

Wired into the traces pipeline (the only pipeline on this branch);
later phases extend it to metrics and logs as those pipelines appear.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 19:10:56 +01:00
Pratik Mankawde
266b8db897 fix(telemetry): widen remaining right-side table-legend timeseries to full width
Apply the same width rule from the earlier transaction-overview fix to the
job-queue, rpc-perf-otel, and rpc-performance dashboards. Timeseries panels
whose legend is a right-side table need the horizontal room, so they go to
full width (w=24); each stacks onto its own row and panels below shift down.
Pie charts, heatmaps, and bottom-legend panels stay half width.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 19:03:53 +01:00
Pratik Mankawde
f08c098049 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-06-11 23:28:41 +01:00
Pratik Mankawde
bef40b37e7 fix(telemetry): use bare consensus_ledger_id tempo filter tag
The Tempo datasource search filter used the stale dotted xrpl.consensus.ledger_id
tag, but the consensus.round span emits the bare consensus_ledger_id key
(ConsensusSpanNames.h). The dotted tag matched no spans and failed the Rule C
naming check. Align it with L1.
2026-06-11 23:28:29 +01:00
Pratik Mankawde
25227141b3 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
Bring phase-8 (log correlation) and the naming-check + BasicConfig fixes forward
into phase 9 (native-metrics gap fill). Phase 9 adds MetricsRegistry and
LedgerSpanNames.h.

Conflict resolution:
- presentation.md: took phase-9's restructured deck — sampling now lives in
  Slide 8 (Head vs Tail) and the Phase 10/11/12 future-phase slides; phase-8's
  older standalone sampling section is superseded (no content lost).
- ordering.txt/loops.txt: regenerated via generate.py.

Naming check: A/B/E/F/G green. Rule D reports 6 findings that are all check
gaps, not data errors (verified every dashboard label traces to code):
  - 3 are TraceQL `span.<attr>` references (bare attr is in L1); Rule D should
    strip the span./resource. scope prefix.
  - 2 (`reason`, `job_type`) are native OTel metric labels emitted by
    MetricsRegistry.cpp — a new source of truth the check does not yet model.
The Rule D enhancement lands on phase-1c and merges forward.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 19:11:54 +01:00
Pratik Mankawde
3236daf08e Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
Bring native-metrics (phase 7) and the naming-check work forward into phase 8
(log correlation).

Conflict resolution — tempo.yaml datasource provisioning (search.filters):
- Took phase-7's complete 30-filter list as the base (it is the comprehensive
  set), preserved phase-8's tracesToLogs Loki-correlation block, and added
  phase-8's two unique filters (tx-type, ledger-hash). Verified the result is
  the full union (32 filters) with no filter dropped and no duplicate ids.
- Fixed a pre-existing dotted tag carried in from phase-7: the
  consensus-ledger-id filter used `xrpl.consensus.ledger_id`; the code emits
  the underscore key `consensus_ledger_id` (ConsensusSpanNames.h), so the
  dotted form was a dead filter. Now corrected.

Every span-scope filter tag verified to exist in the L1 *SpanNames.h key set.
Naming check green; 72 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 18:48:20 +01:00
Pratik Mankawde
2efc932165 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
Bring the hardened OTel naming check (Rule E fix + 71 tests) and the phase 1-6
convention work forward into phase 7 (native metrics).

Conflict resolution:
- 05-configuration-reference.md: kept phase-7's native-OTLP metrics story
  (server=otel, /v1/metrics) over phase-6's superseded StatsD scrape job, in
  prose form (no code block).
- ordering.txt/loops.txt: regenerated via generate.py (not hand-edited).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 18:32:21 +01:00
Pratik Mankawde
5705c4dc02 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-11 17:49:55 +01:00
Pratik Mankawde
ffc197b914 fix(telemetry): use underscore attr names and SpanNames constants in Phase 4
- tempo.yaml: align consensus filter tags with emitted keys
  (consensus_mode, consensus_round, ledger_seq) instead of dotted form
- haveConsensus(): set span attributes before early-return paths so the
  consensus.check span carries diagnostics even when consensus is not reached
- replace hardcoded consensus phase/result/vote literals with
  ConsensusSpanNames.h val constants; add val::phaseOpen/Establish/Accepted
- ConsensusReceiveTracing.h: use canonical consensus::span constants instead
  of duplicate inline detail:: names
- SpanGuardFactory test: use rpc_span / consensus::span constants now that
  levelization permits the dependency

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 17:07:10 +01:00
Pratik Mankawde
4a0994209e Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-10 16:26:23 +01:00
Pratik Mankawde
f37589b1f5 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-06-10 16:05:29 +01:00
Pratik Mankawde
4ed409f35a addressed code review comments
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-10 16:05:04 +01:00
Pratik Mankawde
81beeb6747 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-10 10:21:12 +01:00
Pratik Mankawde
21c177c8f9 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-10 10:20:20 +01:00
Pratik Mankawde
a7c3cd8a79 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-06-10 10:20:03 +01:00
Pratik Mankawde
a7ada14010 docs(telemetry): drop sampling_ratio from mainnet sample config
Head sampling is fixed at 1.0 and not configurable. Remove the
sampling_ratio line and its stale tuning comment; point volume reduction
at collector-side tail sampling.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 10:04:59 +01:00