Commit Graph

80 Commits

Author SHA1 Message Date
Pratik Mankawde
829cad81af docs(telemetry): update metric names across docs, tests, alert rules, and plan docs
Strip xrpld_ prefix, lowercase beast::insight names, and replace
traces_span_metrics_ with span_ in all remaining tracked files:
alert rules, integration tests, workload validation, TESTING.md,
OpenTelemetryPlan docs, code comments, and config templates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 18:01:53 +01:00
Pratik Mankawde
e64afb53d0 fix(telemetry): use service_instance_id in Grafana Cloud transform and docs
Update the transform/cloudlabels processor to set service_instance_id
instead of exported_instance, matching the local Prometheus path where
resource_to_telemetry_conversion promotes service.instance.id with that
exact label name. Also update source comments and documentation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 14:55:15 +01:00
Pratik Mankawde
53b1751384 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-07 20:00:54 +01:00
Pratik Mankawde
6a2570e26e Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/docker-compose.yml
2026-07-07 20:00:36 +01:00
Pratik Mankawde
9e5f1caff2 docs(telemetry): address review feedback on phase-8 log correlation
- Correct log-timestamp examples to real Logs::format() output
  (2024-Jan-15 10:30:45.123456 UTC) in docs, TESTING.md, and reference.
- Update Loki to v3.4.2 and switch the collector exporter to
  otlphttp/loki on the native /otlp endpoint (docs + task list).
- Make the collector log-mount path configurable via XRPLD_LOG_DIR.
- Remove implementation-phase references from shipped config/script
  comments, keeping the functional descriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:55:26 +01:00
Pratik Mankawde
6736ce363e Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-07 13:13:09 +01:00
Pratik Mankawde
81d3db3a1f docs(telemetry): repoint remaining parity-spec links to appendix
The 09-data-collection-reference and Phase9 task list still linked the moved
External Dashboard Parity spec at its old docs/superpowers/ path. Point them
at the appendix in 06-implementation-phases.md where the content now lives.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 13:12:49 +01:00
Pratik Mankawde
21bcedfd4a Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-04 02:49:47 +01:00
Pratik Mankawde
85ce6467d0 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-04 02:47:50 +01:00
Pratik Mankawde
25868f2740 refactor(telemetry): rename xrpld-* dashboards to bare names, bare uids across all dashboards
Rename the 4 xrpld-* dashboard files to bare names and set every
dashboard uid to its bare filename stem; update doc/plan references.
Removes the redundant xrpld-/system- prefixes now that titles are clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 02:46:37 +01:00
Pratik Mankawde
2c590a47c5 refactor(telemetry): rename system-* dashboards to bare names, drop redundant prefix
Rename the 5 system-* dashboard files to bare domain names (uid ==
filename stem) and update all doc references. The system- prefix was
redundant now that titles no longer carry a recorder-pipeline suffix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 02:45:39 +01:00
Pratik Mankawde
628726b51b Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
# Conflicts:
#	docker/telemetry/grafana/dashboards/rpc-performance.json
#	docker/telemetry/grafana/dashboards/xrpld-rpc-perf-otel.json
2026-07-03 22:31:15 +01:00
Pratik Mankawde
94791ef924 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-03 22:29:24 +01:00
Pratik Mankawde
2a90b582ff Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-03 22:29:18 +01:00
Pratik Mankawde
6961727ba7 docs(telemetry): drop (System Metrics) suffix from dashboard names in reference
Match the reader-facing dashboard catalog to the actual dashboard titles,
which no longer carry the recorder-pipeline suffix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 22:28:56 +01:00
Pratik Mankawde
0f43675440 docs(telemetry): reflect RPC dashboard merge in data-collection reference
The separate RPC Performance (OTel) dashboard was folded into RPC
Performance as a per-method section, so describe it as an addition to the
existing dashboard rather than a new one.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 22:28:02 +01:00
Pratik Mankawde
4e566949da Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
# Conflicts:
#	docker/telemetry/TESTING.md
2026-07-03 20:02:37 +01:00
Pratik Mankawde
93f776fda4 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-03 20:01:54 +01:00
Pratik Mankawde
84eb08eb8c Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-03 20:01:54 +01:00
Pratik Mankawde
fdfa651006 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	OpenTelemetryPlan/09-data-collection-reference.md
2026-07-03 20:01:44 +01:00
Pratik Mankawde
f81919af05 docs(telemetry): document tier template variables and label checks
- 09-data-collection-reference: add $service_name, $deployment_environment,
  $xrpl_network_type to the template-variable table; note they appear on all
  dashboards and point to the runbook's Deployment Tiers section.
- TESTING.md: add a Prometheus check verifying the tier labels
  (deployment_environment, xrpl_network_type, service_name) are present on
  metric series.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 19:56:15 +01:00
Pratik Mankawde
03b92bfca7 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-06-11 23:21:14 +01:00
Pratik Mankawde
c30123796e Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-11 23:21:14 +01:00
Pratik Mankawde
5a6a00c458 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-06-11 23:21:14 +01:00
Pratik Mankawde
f29f32816b docs(telemetry): fix stale txq.accept_tx span name in 09 doc 2026-06-11 23:20:09 +01:00
Pratik Mankawde
c8c540e388 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-06-11 23:18:37 +01:00
Pratik Mankawde
22bb4a74a9 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-06-11 23:18:37 +01:00
Pratik Mankawde
663ef989eb Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-11 23:18:36 +01:00
Pratik Mankawde
725b4fb5cb docs(telemetry): drop the dotted ledger-hash exception in 09 doc
The peer.validation.receive span now records the ledger hash and full-validation
flag under the shared bare keys ledger_hash / full_validation (same keys the
consensus validation spans use), so there is no longer a dotted span attribute.
Remove the "only dotted span attribute" note and update the peer attribute table.
2026-06-11 23:12:42 +01:00
Pratik Mankawde
81919612f8 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-06-10 10:22:07 +01:00
Pratik Mankawde
81beeb6747 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-10 10:21:12 +01:00
Pratik Mankawde
21c177c8f9 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-10 10:20:20 +01:00
Pratik Mankawde
a7c3cd8a79 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-06-10 10:20:03 +01:00
Pratik Mankawde
55a674f042 docs(telemetry): remove sampling_ratio knob from phase-6 docs and configs
Head sampling is fixed at 1.0 and not configurable. Drop sampling_ratio
from the docker sample/test configs and the data-collection reference;
rewrite the span-metric sampling caveat and volume guidance to point at
collector-side tail sampling.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 10:03:31 +01:00
Pratik Mankawde
010b0d1bd6 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
# Conflicts:
#	src/tests/libxrpl/CMakeLists.txt
2026-06-09 16:46:24 +01:00
Pratik Mankawde
51003bc285 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	src/tests/libxrpl/CMakeLists.txt
2026-06-09 16:45:14 +01:00
Pratik Mankawde
3e4b1eb60f Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-09 16:43:45 +01:00
Pratik Mankawde
3633ffc4c7 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	src/tests/libxrpl/CMakeLists.txt
2026-06-09 16:43:15 +01:00
Pratik Mankawde
07599778d4 docs(telemetry): reflect peer tracing enabled by default in data reference
Update the Peer Spans note, the known-gaps caveat, and the trace category
toggle table to show trace_peer enabled by default. The Production Setup
example keeps trace_peer=0 as a deliberate high-volume opt-out alongside
sampling_ratio=0.01.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 14:57:01 +01:00
Pratik Mankawde
758a3fec29 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
# Conflicts:
#	OpenTelemetryPlan/09-data-collection-reference.md
2026-06-05 19:42:53 +01:00
Pratik Mankawde
a23d83f393 docs(telemetry): add ledger.acquire to 09-doc + fix peer-quality dashboard metric prefix
Phase 9 introduces the ledger.acquire span (InboundLedger fetch) that phases 7-8
do not have, so the forward-merged 09-data-collection-reference inventory is
extended here:
- §1.1: add ledger.acquire to the Ledger span table.
- §1.2: add its attributes (acquire_reason, timeouts, peer_count, outcome) and
  note it also sets ledger_seq; bump the span count.

Also fix two stale StatsD metric references in the Peer Quality dashboard
(xrpld-peer-quality.json): rippled_Peer_Finder_Active_{Inbound,Outbound}_Peers
-> xrpld_Peer_Finder_* to match the xrpld_ metric prefix the rest of the stack uses.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 19:36:34 +01:00
Pratik Mankawde
22b533ac51 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-06-05 19:33:13 +01:00
Pratik Mankawde
8046a30e9b Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-05 19:28:58 +01:00
Pratik Mankawde
4a8aa9e514 docs(telemetry): reconcile 09-data-collection-reference span/attribute inventory
The §1 span and attribute inventory had regressed to an older 16-span snapshot
that uses the pre-2026-05-13 dotted attribute keys, while phase-7's code emits
~36 spans with bare/underscore attribute keys. The §Data Flow Overview and §2
System Metrics sections (native OTLP transport — phase-7's migration) were already
correct and are left unchanged.

- §1.1: expand the span inventory to the full surface — add gRPC (grpc.<MethodName>),
  TxQ (txq.*), PathFind (pathfind.*), and the full consensus set (round/phase.open/
  establish/update_positions/check/mode_change/proposal.receive/validation.receive).
  Fix the phantom rpc.request -> rpc.http_request, add rpc.ws_upgrade. No grpc.request,
  no pathfind.rank, no ledger.acquire (the latter is added in phase-9, not yet present here).
- §1.2: convert every span-attribute key from dotted xrpl.<domain>.<field> to the
  bare/underscore form. The sole span-attr dotted exception is xrpl.ledger.hash on
  peer.validation.receive (shared constant); consensus.validation.send uses bare
  ledger_hash. Resource attrs xrpl.network.id/type stay dotted. Fix tx_count/tx_failed
  placement (on tx.apply, not ledger.build). Add attribute tables for the new families.
- §1.3: list the full set of spanmetrics dimension labels (bare keys, from the collector
  config) instead of the stale xrpl_rpc_command-style names.
- §4/§5: convert Tempo TraceQL and PromQL examples to the bare attribute/label forms.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 19:28:45 +01:00
Pratik Mankawde
dc5bb4b35c feat(telemetry): emit xrpld_validation_{agreements,missed}_total counters
Wire the two previously-registered-but-never-incremented validation
counters to ValidationTracker's gross lifetime tallies, exported as
monotonic ObservableCounters. New gross atomics count each ledger once at
first classification and are never adjusted on late repair, keeping the
_total counters monotonic and additive (agreements_total + missed_total ==
ledgers reconciled); the repair-aware windowed view stays on the existing
xrpld_validation_agreement gauge. The validator-health dashboard panels
that already query these names now render data instead of "No data".

Also de-stale 09-data-collection-reference.md: §5b documented flat metric
names (xrpld_cache_SLE_hit_rate, ...) that the code never emits — it emits
labeled gauges (xrpld_cache_metrics{metric="SLE_hit_rate"}). Replace the
stale flat-name tables with a pointer to the canonical labeled section,
reconcile the contradictory headline counts, and correct xrpld_job_count
to its real exported name xrpld_jobq_job_count.

Adds two GTests asserting gross tallies stay frozen on repair while net
totals move, plus the additive invariant.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 18:29:29 +01:00
Pratik Mankawde
db5b93e2c4 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-06-05 12:50:09 +01:00
Pratik Mankawde
f37a4a1022 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	src/xrpld/app/misc/detail/TxQ.cpp
2026-06-05 12:49:38 +01:00
Pratik Mankawde
8f3974c094 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-05 12:48:40 +01:00
Pratik Mankawde
283fbaa54f Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	OpenTelemetryPlan/09-data-collection-reference.md
2026-06-05 12:48:31 +01:00
Pratik Mankawde
3167a49f41 feat(telemetry): derive per-stage tx metrics from apply-pipeline spans
Wire the apply-pipeline stage spans (tx.preflight, tx.preclaim,
tx.transactor) added on phase-3 through the observability stack so the
spanmetrics connector produces per-stage RED metrics without any native
instruments.

- collector: add the `stage` dimension to the spanmetrics connector so
  the three stages split into separate metric series (3 bounded values).
- dashboard: add a "Tx Apply Pipeline" section to transaction-overview
  with rate, p95 latency, and failure-rate panels grouped by stage, plus
  a `stage` template variable. Panels follow the existing config (node
  filter, exported_instance legends, Title Case, axis labels).
- The failure panel filters ter_result != tesSUCCESS rather than span
  status, because a failing ter code completes the span normally — only
  thrown exceptions set an error status. This matches the existing
  "Transaction Results by Type" panel convention.
- docs: document the spans, attributes, and stage dimension in the data
  collection reference and runbook, including the sampling caveat that
  span-derived metrics inherit tracer head-sampling and undercount at
  sampling_ratio < 1.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 12:42:53 +01:00