Commit Graph

219 Commits

Author SHA1 Message Date
Pratik Mankawde
379241c593 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-07-29 16:05:13 +01:00
Pratik Mankawde
c373165882 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-29 16:04:51 +01:00
Pratik Mankawde
35a0896fc5 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Conflict in src/libxrpl/telemetry/SpanGuard.cpp: both sides added a different
include in the same alphabetical slot — phase-4 added <initializer_list>, the
incoming std::format change added <format>. Both are required, so both are
kept in sorted order.
2026-07-29 15:08:52 +01:00
Pratik Mankawde
5e60f1b842 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-07-29 15:07:59 +01:00
Pratik Mankawde
fb24fa67e2 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-07-29 15:07:59 +01:00
Pratik Mankawde
87078dca6e Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra
Conflict in OpenTelemetryPlan/03-implementation-strategy.md §3.9: both
branches independently fixed the same defect (stale hand-maintained line
counts) in different ways.

- 1b (0ff947454c) stripped the Lines Added / Lines Changed columns but kept
  the §3.9.1 and §3.9.2 tables.
- 1a (3ad525a48a) removed both sections outright.

Resolved in favour of 1a's deletion. The tables were pre-implementation
estimates with no source of truth, so trimming the columns only defers the
next drift; the file/component lists duplicated §3.1's directory tree, which
1b already keeps current (DiscardFlag.h, FilteringSpanProcessor). Keeping the
upstream resolution also stops the same conflict recurring on 1c..10.

1b-specific content verified intact after the merge: §3.1 retains the
DiscardFlag.h entry, the FilteringSpanProcessor and discard() annotations,
and the TracingInstrumentation removal. No references to the deleted
sections remain in any plan doc.
2026-07-29 14:35:33 +01:00
Pratik Mankawde
3ad525a48a docs(telemetry): drop speculative file/line estimate tables from §3.9
The §3.9.1 "Files Modified Summary" and §3.9.2 "Detailed File Impact"
tables carried hand-maintained per-component line counts that had drifted
from the plan: the Lines Added column summed to 1,565 while the Total row
claimed ~1,670. Files (34) and Lines Changed (120) reconciled, so only the
Added total was stale — residue from expanding Core Telemetry 5 -> 11 files.

Rather than patch one cell, remove both tables. They were pre-implementation
estimates with no source of truth, so any figure in them drifts again on the
next phase. §3.1's directory tree remains the canonical list of the 11 new
telemetry files, and §3.9.3-3.9.7 (risk, architectural impact, backward
compatibility, rollback) carry the assessment without inventing line counts.

No references to the removed sections exist elsewhere in the plan docs.
2026-07-29 14:33:05 +01:00
Pratik Mankawde
6768aa4c07 docs(telemetry): document handler label, GetObject and queue-saturation metrics
Add reference entries for the observability surface introduced on
phase-9: the `handler` label on the job instruments, the five
`getobject_*` request metrics, and the per-job-type queue saturation
gauges.

Names here follow this branch's StatsD pipeline, which preserves case
and carries the `xrpld_` prefix, so they differ from the lowercased
OTel-native names used from phase-7 onward. The sections state where
the implementing code lives, since it is introduced downstream.

Also correct pre-existing entries: `job_count` exports as
`jobq_job_count` via the collector group prefix, the non-special job
type count is 35 (not 36), and `JtLedgerData` has five producers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 11:58:16 +01:00
Pratik Mankawde
21d9b2f8f9 docs(telemetry): add current_ledger_seq/hash to 09 attribute inventory
Document the two new ledger-correlation attributes in the data-collection
reference: add them to the transaction and TxQ attribute tables (with per-span
coverage), update the attribute count, and correct the txq.enqueue parent note
(parents to tx.process on the submission path via explicit context; a root on
the open-ledger rebuild path, correlated via current_ledger_seq).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 16:11:18 +01:00
Pratik Mankawde
e0c4272f5b docs(otel): coro-aware context storage; RPC/tx/consensus log correlation retained
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 22:40:13 +01:00
Pratik Mankawde
2f627e8e8e docs(telemetry): 09-doc — pathfind.request parents to rpc.process, not rpc.command
Reflects the C1 fix: rpc.command.* stays unscoped (its dispatch wraps
doRipplePathFind which yields), so pathfind.request nests under rpc.process.
The request -> compute -> discover sub-tree nests correctly via ScopedSpanGuard.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:12:55 +01:00
Pratik Mankawde
7bdc14eee5 docs(telemetry): 09-doc — deterministic roots are true roots; rootSpan->freshRoot
Document that deterministic-trace_id spans (tx.* apply pipeline, tx.process,
tx.receive, consensus.round) are now genuine trace roots with empty
parent_span_id via the custom DeterministicIdGenerator, superseding the old
synthetic-parent behavior that showed 'root span not yet received' in Tempo.
Also update the fresh-root note: peer entry spans use ScopedSpanGuard::freshRoot().

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 10:58:35 +01:00
Pratik Mankawde
56c03ff24e Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	src/xrpld/overlay/detail/PeerImp.cpp
2026-07-20 12:19:11 +01:00
Pratik Mankawde
777aa9b467 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-20 12:17:52 +01:00
Pratik Mankawde
c3b1f69ba8 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
# Conflicts:
#	include/xrpl/telemetry/SpanGuard.h
#	src/xrpld/app/consensus/RCLConsensus.h
#	src/xrpld/consensus/ConsensusTypes.h
2026-07-20 12:10:44 +01:00
Pratik Mankawde
ea7f585337 fix(telemetry): root peer.validation.receive and peer.proposal.receive spans
These inbound peer-message entry points (kConsumer) used span(), which
inherits whatever span is active on the peer thread — including a leaked
tx.receive scope — so validations/proposals were wrongly nested under
unrelated transaction traces. rootSpan() starts a fresh trace root.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 22:24:52 +01:00
Pratik Mankawde
6c4e0e6ed4 fix(telemetry): detach tx.receive and tx.process spans before job hand-off
Both spans are moved into job-queue lambdas and destroyed on a worker
thread. Detaching on the origin thread pops the thread-local OTel Scope
there, so later spans on the peer/RPC thread no longer inherit these as a
leaked ambient parent. Trace_id/parent are unchanged (both are hashSpan).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 22:24:38 +01:00
Pratik Mankawde
f4981a1907 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
#	docker/telemetry/grafana/provisioning/datasources/tempo.yaml
#	docker/telemetry/otel-collector-config.yaml
#	docs/telemetry-runbook.md
2026-07-06 21:47:17 +01:00
Pratik Mankawde
7cc963f494 fix(telemetry): phase-6 statsd dashboards, config, and doc corrections
- Fix invalid PromQL in statsd dashboards (rate/topk/nested-__name__
  label selectors); add missing $node filter on a binary-op operand
  in statsd-node-health "Ledger Publish Gap".
- Wrap cumulative counters in rate(); correct Bytes_In metric.
- Pin prometheus image; add [insight] server=statsd stanza (prefix
  rippled matches the rippled_* StatsD metric names the dashboards
  query); bind statsd port to loopback; fix Jaeger->Grafana comment.
- Correct dotted attribute keys to underscore form in
  09-data-collection-reference (peer_id, tx_hash, consensus_mode, ...).
- Reconcile span count (11) and proposers attribute in the
  integration-test task list.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 21:36:59 +01:00
Pratik Mankawde
1938b9e377 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-06 21:05:06 +01:00
Pratik Mankawde
b7d0b700c6 fix(telemetry): correct runbook pathfind spans + mTLS validation
- Add PathFinding Spans section to the runbook Span Reference and
  remove the incorrect "not emitted / later-phase" annotations:
  pathfind.request/compute/discover/update_all are emitted since
  Phase 2 (upstream of Phase 5).
- Runbook label names use the bare spanmetrics dimensions
  (command, rpc_status, local, consensus_mode); fix stale docs.
- 05-configuration-reference: note tls_client_cert/key require use_tls=1.
- TelemetryConfig: reject mTLS cert paths when use_tls=0 so telemetry
  is not silently sent in plaintext.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 21:04:07 +01:00
Pratik Mankawde
ea9c639ee9 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:50:24 +01:00
Pratik Mankawde
7998d01dc9 code review comments.
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:49:51 +01:00
Pratik Mankawde
e02b7968ad Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:41:14 +01:00
Pratik Mankawde
38f6afbf4c code review comments
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:40:12 +01:00
Pratik Mankawde
092afb7aae Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:39:27 +01:00
Pratik Mankawde
7a6ff619e5 code review changes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:37:08 +01:00
Pratik Mankawde
3b08c2d90a Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-07-06 18:15:41 +01:00
Pratik Mankawde
26a85c764e code review comment addressed
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 18:15:23 +01:00
Pratik Mankawde
d5d9d1d8da refactor(telemetry): drop redundant xrpld- prefix from dashboard uids
Make dashboard uids bare domain names (uid == filename stem) and update
doc references. Grafana links/bookmarks to the old uids will need
refreshing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 02:44:54 +01:00
Pratik Mankawde
f1f0189c11 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	docs/telemetry-runbook.md
2026-07-03 20:00:19 +01:00
Pratik Mankawde
1ed08d09d0 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-07-03 19:59:01 +01:00
Pratik Mankawde
8e4ab5bd04 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-03 19:59:01 +01:00
Pratik Mankawde
73edc2dd58 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
# Conflicts:
#	OpenTelemetryPlan/05-configuration-reference.md
2026-07-03 19:58:52 +01:00
Pratik Mankawde
a2e8f3a6b2 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-07-03 19:57:45 +01:00
Pratik Mankawde
6e9d71c5eb Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-07-03 19:57:45 +01:00
Pratik Mankawde
f81919af05 docs(telemetry): document tier template variables and label checks
- 09-data-collection-reference: add $service_name, $deployment_environment,
  $xrpl_network_type to the template-variable table; note they appear on all
  dashboards and point to the runbook's Deployment Tiers section.
- TESTING.md: add a Prometheus check verifying the tier labels
  (deployment_environment, xrpl_network_type, service_name) are present on
  metric series.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 19:56:15 +01:00
Pratik Mankawde
d2c7a00584 docs(telemetry): note service.name on metrics and collector tier tagging
- 05-configuration-reference: service_name now applies to metrics as well
  as traces (service.name resource attribute).
- 07-observability-backends: document that collector enrichment includes
  deployment-tier tagging (deployment.environment + fallback
  xrpl.network.type) for filtering one Grafana stack by tier.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 19:53:54 +01:00
Pratik Mankawde
394a64f437 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-06-26 16:31:37 +01:00
Pratik Mankawde
ea2e6787c6 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-26 16:31:37 +01:00
Pratik Mankawde
b3c5f8860c Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-06-26 16:31:37 +01:00
Pratik Mankawde
f48b212a1c Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-06-26 16:31:37 +01:00
Pratik Mankawde
0f1bd86ba2 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-26 16:31:37 +01:00
Pratik Mankawde
cf629e2a76 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-06-26 16:31:36 +01:00
Pratik Mankawde
190b9470a4 docs(telemetry): fix xrpl.network.type value (standalone → unknown)
The network-type label is derived from [network_id] in TelemetryConfig.cpp;
unmapped/unset IDs fall through to "unknown", not "standalone". Align the
design-doc resource-attribute table with the code and cfg example.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:31:06 +01:00
Pratik Mankawde
ce7fe95715 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-26 14:49:43 +01:00
Pratik Mankawde
3463a85943 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-26 14:46:10 +01:00
Pratik Mankawde
bc7cff230c Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-26 14:45:43 +01:00
Pratik Mankawde
54d92ff973 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-26 14:42:55 +01:00
Pratik Mankawde
9a2d682a42 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-06-26 14:33:16 +01:00