Commit Graph

64 Commits

Author SHA1 Message Date
Pratik Mankawde
070d29b465 feat(telemetry): add the xrpl.node.id resource attribute
Node identity reached the OTel resource only as service.instance.id, which is
config-overridable and carries a deployment-chosen label rather than the node's
own identity. Add xrpl.node.id, set unconditionally from the node public key
(base58, TokenType::NodePublic), so traces and metrics share a stable per-node
key independent of [telemetry] service_instance_id.

Set on the tracer resource via Telemetry::setNodeId(), called from
ApplicationImp::setup() once nodeIdentity_ is known, and on the MetricsRegistry
resource via an added start() parameter. The beast::insight meter provider is
built in TelemetryImpl's constructor, before the wallet DB exists, so its
resource cannot carry the value; that path is left for later and the attribute
is omitted rather than stamped blank.

Also drops the transform/spanidentity collector processor added in
4a361a496d: per-node identity belongs on the resource, not copied onto every
span.
2026-08-19 19:50:40 +01:00
Pratik Mankawde
3860c93db2 refactor(telemetry): route dashboards, runbook and collector work to phase-9
These changes were developed on the phase-10 branch but belong to content this
branch and its upstreams introduced. Carrying them on phase-10 made its PR diff
report churn in files phase-10 does not own, and left each PR claiming a scope
that did not match its contents.

Moved here from phase-10 (identical content, no functional change):

- Dashboards: all 14 existing boards plus the new log-derived-insights board.
- Docs: telemetry-runbook.md (minus the workload/benchmark sections, which
  describe phase-10 tooling) and the new telemetry-glossary.md.
- Grafana Cloud + Alloy export path: collector config, compose override, the
  two .env examples and alloy/config.alloy.
- Local stack: otel-collector-config.yaml gains sub-millisecond and
  second-scale spanmetrics buckets, pins unit=ms, and promotes
  close_time_correct; integration-test.sh and TESTING.md follow.
- Node configs: exported_instance -> service_instance_id in comments; the
  mainnet sample now logs at warning to bound log volume.
- Metrics code: Telemetry.cpp builds the metrics pipeline in the constructor
  via initMetrics() so the global MeterProvider is published before any
  subsystem creates a beast::insight instrument, and the histogram view keeps
  each instrument's own name instead of collapsing them under one series.
  MetricsRegistry gains a last_close_time gauge and skips negative job-queue
  durations. OTelCollector drops an unused accessor.
- Naming CI: xrpl_work_item joins EXTERNAL_INFRA_LABELS and Rule E accepts the
  dotted perf-iac resource-attribute form. This must travel with the
  dashboards and runbook that reference those labels, or the rules fail.
- Doxygen input glob no longer recurses dot-directories.

Sections describing phase-10 tooling stay on phase-10 and keep their
"Future Enhancement" / "Planned, not yet implemented" markers here; phase-10
removes those markers when it lands the tooling.
2026-08-04 16:10:04 +01:00
Pratik Mankawde
3d495f9622 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
Brings coroutine-aware context storage + tx/consensus worker-body activation.
Resolved: Telemetry.cpp keeps both meterProvider_ (phase-7) and contextStorage_
(coro-aware); doc-09 keeps phase-7 structure and applies the pathfind.request →
rpc.command.<name> correction to phase-7's own PathFind section.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 22:46:54 +01:00
Pratik Mankawde
0441d50793 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
Brings coroutine-aware context storage + tx/consensus worker-body activation
forward. Preserves phase-5's mTLS exporter config in Telemetry.cpp.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 22:29:11 +01:00
Pratik Mankawde
7de0229450 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Brings coroutine-aware context storage + tx worker-body activation forward.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 22:15:38 +01:00
Pratik Mankawde
6a9eaedeac Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
Brings coroutine-aware OTel context storage forward: CoroAwareContextStorage,
its install in Telemetry, ScopedSpanGuard same-store assertion, and the
non-owning ScopedActivation helper.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 19:49:26 +01:00
Pratik Mankawde
9fa58067fa feat(telemetry): install coroutine-aware context storage at start
Install CoroAwareContextStorage as the process-wide OTel runtime context
storage in TelemetryImpl::start(), before SetTracerProvider and before any
span is created, so the ambient context follows JobQueue coroutines across
yield/resume. Hold it in a member for the process lifetime; not reset in
stop() because resetting while spans may exist is SDK undefined behaviour.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 17:12:11 +01:00
Pratik Mankawde
eec3062c9c Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-22 10:58:50 +01:00
Pratik Mankawde
b2d6f06a9f Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-21 22:14:36 +01:00
Pratik Mankawde
fbb79d73fb Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
# Conflicts:
#	src/libxrpl/telemetry/SpanGuard.cpp
2026-07-21 20:43:10 +01:00
Pratik Mankawde
d2d114595b Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-07-21 18:43:26 +01:00
Pratik Mankawde
4d95f455f4 feat(telemetry): add DeterministicIdGenerator + PendingTraceId for true-root deterministic trace_ids
Add a custom OTel IdGenerator that returns a thread-local pending trace_id on
the SDK no-parent (root) branch and a random one otherwise, plus a PendingTraceId
RAII guard that pins that id for the next forced-root span and asserts on
destruction that it was consumed. Wire the generator into
TracerProviderFactory::Create via its 4-arg overload.

This lets hash-derived spans become true trace roots so they line up into one
trace across nodes. It is installed but dormant on this branch: the caller
(hashSpan) arrives on a later branch (phase-3). GenerateSpanId is always random
and is_random_ is false so the W3C random-trace-id flag is not set on
deterministic ids.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 18:42:54 +01:00
Pratik Mankawde
c1f004ec22 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	include/xrpl/telemetry/Telemetry.h
#	src/libxrpl/telemetry/TelemetryConfig.cpp
2026-07-20 17:40:30 +01:00
Pratik Mankawde
c155a1e242 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
# Conflicts:
#	include/xrpl/telemetry/Telemetry.h
2026-07-20 17:22:02 +01:00
Pratik Mankawde
84729d8eaa Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-07-20 17:19:52 +01:00
Pratik Mankawde
daea7fa34f Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
# Conflicts:
#	include/xrpl/telemetry/Telemetry.h
2026-07-20 17:17:20 +01:00
Pratik Mankawde
8b9cb43824 chore(telemetry): normalize Doxygen comment style + refresh levelization
Bring the 5 telemetry files introduced on this branch to the enforced
house Doxygen style (/** alone, ' * ' continuation prefix, no single-line
blocks) so the check-doxygen-style hook passes under CI's --all-files run.
Comment-only: code is byte-identical after comment stripping (verified).

Also regenerate levelization results: the committed ordering.txt carried
stale 'xrpl.telemetry > xrpld.consensus/rpc' edges that the current include
graph no longer produces.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 17:09:43 +01:00
Pratik Mankawde
7ab34d2e37 fix(telemetry): resolve phase-7 clang-tidy findings
- OTelCollector::formatName made static (no member state); braces added;
  <cctype> added for std::tolower.
- Telemetry.cpp getMeter(): braces around single-statement if.
- GetMeter.cpp: drop unused <opentelemetry/metrics/meter.h> and
  sync_instruments.h; add <xrpl/beast/utility/Journal.h> and <memory>;
  make sdkProvider const.
- ValidationTracker.h: add <cstddef> for std::size_t.
- detail/ValidationTracker.cpp: use std::views::reverse range-based loop;
  add <ranges>.
- test ValidationTracker.cpp: use auto for cast result.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 15:09:43 +01:00
Pratik Mankawde
a396fe073f Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-08 15:38:26 +01:00
Pratik Mankawde
b1091a482c Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
# Conflicts:
#	src/libxrpl/telemetry/Telemetry.cpp
2026-07-08 15:23:03 +01:00
Pratik Mankawde
87c003230c Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-07-08 15:20:07 +01:00
Pratik Mankawde
ba8785dc16 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-07-08 15:18:06 +01:00
Pratik Mankawde
c6be7cf83a fix(telemetry): resolve clang-tidy CI errors (include-cleaner, braces)
Add missing <cstdint> include for std::uint32_t in Telemetry.h.
Add braces around single-line if bodies in Telemetry.cpp.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 15:17:58 +01:00
Pratik Mankawde
ab0d8249fd feat(telemetry): expose direct OTel metrics API via getMeter()
Give developers direct access to the full OpenTelemetry metrics API,
symmetric with getTracer(), so all seven OTel instrument types
(including UpDownCounter, sync Gauge, and the observable variants) are
reachable — not just the four beast::insight models.

- Telemetry: build and own the metrics pipeline (OTLP HTTP metric
  exporter + PeriodicExportingMetricReader + SpanMetrics histogram view)
  alongside the tracer, sharing the same resource attributes and TLS
  config. Register it globally via metrics::Provider::SetMeterProvider
  and expose Telemetry::getMeter(). Metrics enable with [telemetry];
  the metrics endpoint is derived from the trace endpoint.
- beast OTelCollector: no longer owns a pipeline. It fetches the global
  Meter, becoming a thin shim over the shared provider (legacy path
  during beast deprecation). This also resolves the review note that the
  metric exporter ignored [telemetry] use_tls — TLS now comes from the
  shared telemetry pipeline.
- Add a libxrpl unit test covering getMeter() on the enabled (global
  provider) and disabled (noop) paths, exercising an UpDownCounter.

Design: docs/superpowers/specs/2026-07-07-direct-otel-metrics-api-design.md

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 16:52:45 +01:00
Pratik Mankawde
074fb0ff22 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-26 17:37:54 +01:00
Pratik Mankawde
b8a5361e28 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-06-26 17:37:54 +01:00
Pratik Mankawde
5cc837289a Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-26 17:37:54 +01:00
Pratik Mankawde
498bc3ee96 fix(telemetry): satisfy clang-tidy const-correctness and redundant-parens
CI enabled more clang-tidy checks via the upstream merge:
- DiscardScope discardScope -> const (misc-const-correctness)
- drop the (::max)() Windows macro-guard parens; NOMINMAX is defined
  project-wide so the bare form is correct (readability-redundant-parentheses)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 17:37:37 +01:00
Pratik Mankawde
3463a85943 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-26 14:46:10 +01:00
Pratik Mankawde
bc7cff230c Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-26 14:45:43 +01:00
Pratik Mankawde
f9092e7948 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-15 20:39:08 +01:00
Pratik Mankawde
ba85aecf00 hide gTlDiscardCurrentSpan better
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-15 17:21:35 +01:00
Pratik Mankawde
37a4c9deb8 core review comments
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-15 17:10:23 +01:00
Pratik Mankawde
53ebd7a8c2 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-10 17:47:27 +01:00
Pratik Mankawde
4a0994209e Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-10 16:26:23 +01:00
Pratik Mankawde
8908036b11 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-06-10 14:55:29 +01:00
Pratik Mankawde
e205d0ef8e handle gTlDiscardCurrentSpan change
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-10 14:00:52 +01:00
Pratik Mankawde
b9de9d0abe Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-09 19:10:37 +01:00
Pratik Mankawde
f12c896ac7 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-09 19:05:52 +01:00
Pratik Mankawde
142e8c5b36 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-09 18:58:11 +01:00
Pratik Mankawde
fa71280795 removed head sampling ratio from config
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-09 18:45:52 +01:00
Pratik Mankawde
cf71dc0a01 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-09 14:53:27 +01:00
Pratik Mankawde
9512930b0f Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-09 14:52:29 +01:00
Pratik Mankawde
4a3ba35c1d Added mTls config for Node-Collector verification
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-08 16:26:09 +01:00
Pratik Mankawde
bdd7dea4b4 code coverage fixes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-03 14:23:24 +01:00
Pratik Mankawde
3cd3d5e80e clang-tidy fixes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-01 14:45:29 +01:00
Pratik Mankawde
9918803333 clang tidy fixes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-01 13:18:27 +01:00
Pratik Mankawde
523bfdbbe1 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-01 11:39:37 +01:00
Pratik Mankawde
5d1e3f207c clang-tidy fixes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-01 11:35:09 +01:00
Pratik Mankawde
2f96c6547c Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 16:51:31 +01:00