Commit Graph

386 Commits

Author SHA1 Message Date
Pratik Mankawde
cb9fce6890 fix(telemetry): align Phase 10 workload harness with current OTel recording surface + fix CI
The Phase 10 validation harness had drifted from the code's recording surface
and the telemetry-validation CI job was failing before it could build.

CI fix (telemetry-validation.yml):
- Replace nonexistent local action ./.github/actions/print-env with the remote
  XRPLF/actions/print-build-env (the build-xrpld job failed in 56s on this).
- Sync prepare-runner and upload-artifact action SHAs to the canonical workflow.

Recording-surface reconciliation (docker/telemetry/workload/):
- Migrate span attributes from dotted xrpl.<domain>.<field> to the bare/underscore
  form introduced by the 2026-05-13 span-attr naming redesign (tx_hash, peer_id,
  ledger_seq, consensus_mode, consensus_round, full_validation, quorum, ...).
  Dotted xrpl.ledger.hash is retained only on peer.validation.receive (shared
  constant), while consensus.validation.send uses bare ledger_hash.
- Fix attribute placement: tx.apply carries tx_count/tx_failed (not ledger_seq);
  ledger.build carries ledger_seq/close_* (not tx_count/tx_failed).
- Replace the phantom rpc.request span with the real WS root rpc.ws_message; drop
  the never-emitted duration_ms; rebuild the parent-child map accordingly.
- Add the new spans the code emits: apply-pipeline stage spans
  (tx.preflight/preclaim/transactor with stage/tx_type/ter_result), txq.*,
  consensus sub-spans (round/establish/update_positions/check/phase.open),
  ledger.acquire, grpc.*, pathfind.*. Conditional spans are marked optional so
  they are skipped (not failed) when the workload does not exercise them.
- validate_telemetry.py: service.name and Loki job label rippled -> xrpld; fix
  PARITY_SPAN_ATTRS (rename the 4 real attrs, drop the 3 that are metrics not span
  attrs); add optional-span handling that skips missing optional spans while still
  validating attributes when present.
- expected_metrics.json: rippled_ -> xrpld_ on all beast::insight/overlay metrics,
  xrpld_job_count, the 15 on-disk xrpld-* dashboard UIDs, and the real bare
  spanmetrics dimension labels.
- regression-metrics.json + baseline-timings.json: rpc.request -> rpc.ws_message.

Metrics pipeline fix:
- Switch node [insight] config from server=statsd/prefix=rippled to server=otel +
  /v1/metrics endpoint + prefix=xrpld across run-full-validation.sh,
  xrpld-validator.cfg.template, benchmark.sh and the workload compose. The
  collector has no StatsD receiver, so system metrics only reach Prometheus over
  OTLP.

Synthetic load for new spans:
- Add ripple_path_find to the RPC load generator (drives pathfind.* spans).
- Add a high-TPS txq-burst workload phase to force fee escalation (drives txq.*).

All facts verified against the *SpanNames.h headers and a live xrpld node +
collector (Tempo service.name=xrpld, tx.preflight attrs [stage,ter_result,tx_type],
279 xrpld_ Prometheus metrics and zero rippled_).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 17:08:58 +01:00
Pratik Mankawde
db5b93e2c4 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-06-05 12:50:09 +01:00
Pratik Mankawde
f37a4a1022 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	src/xrpld/app/misc/detail/TxQ.cpp
2026-06-05 12:49:38 +01:00
Pratik Mankawde
283fbaa54f Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	OpenTelemetryPlan/09-data-collection-reference.md
2026-06-05 12:48:31 +01:00
Pratik Mankawde
759d3506b2 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-06-05 11:58:59 +01:00
Pratik Mankawde
a71d6635e6 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-05 11:58:43 +01:00
Pratik Mankawde
6428c9f13c feat(telemetry): add preflight/preclaim stage spans and stage attribute
The tx.transactor span covered only the apply stage; preflight and
preclaim had no telemetry, so a transaction that hard-failed those
stages produced no apply-pipeline span and per-stage latency/failure
was invisible.

Add tx.preflight and tx.preclaim spans in applySteps.cpp via a
makeStageSpan() helper using SpanGuard::hashSpan, so all three stages
share a deterministic trace_id derived from txID[0:16] even though they
run sequentially and often cross-thread. Each span carries stage,
tx_type, and ter_result; exceptions are recorded as tefEXCEPTION before
the public wrappers map them. The type lookup is guarded behind the
span-active check so it costs nothing when tracing is off.

Add a stage="apply" attribute to the tx.transactor span and move its
three hardcoded attribute strings to a new library-safe header
include/xrpl/tx/detail/TxApplySpanNames.h, which mirrors the daemon-side
TxSpanNames.h strings so the collector spanmetrics connector aggregates
both span sets under one dimension set.

A constants-contract test pins the span-name, attribute-key, and
stage-value strings; span content stays covered by the docker
integration test, as the rest of the telemetry suite is.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 11:11:55 +01:00
Pratik Mankawde
478b58395b loop levelization
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-04 17:54:52 +01:00
Pratik Mankawde
a13a858112 feat(telemetry): add tx.transactor span for per-transactor execution timing
Wraps Transactor::operator() with a span that captures tx_type,
ter_result, and applied. This is the universal dispatch point — every
transaction flows through it, giving per-type latency breakdown.

Adds libxrpl.tx > xrpl.telemetry levelization dependency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-03 16:40:10 +01:00
Pratik Mankawde
f6b4d945d8 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-06-03 16:39:07 +01:00
Pratik Mankawde
146ea1455b Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-06-03 16:32:37 +01:00
Pratik Mankawde
d6fe31442e Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-06-03 16:32:36 +01:00
Pratik Mankawde
8adb5d03da Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-03 16:32:31 +01:00
Pratik Mankawde
2264a8427a Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-06-03 16:32:31 +01:00
Pratik Mankawde
c5bdaafc39 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-06-03 16:32:31 +01:00
Pratik Mankawde
4b6c1c270f feat(telemetry): add tx.transactor span for per-transactor execution timing
Wraps Transactor::operator() with a span that captures tx_type,
ter_result, and applied. This is the universal dispatch point — every
transaction flows through it, giving per-type latency breakdown.

Adds libxrpl.tx > xrpl.telemetry levelization dependency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-03 16:32:16 +01:00
Pratik Mankawde
860b1601c7 formatting updates
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-02 14:34:27 +01:00
Pratik Mankawde
e07a0c347f Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-06-02 11:13:17 +01:00
Pratik Mankawde
66e6310b56 more clang-tidy fixes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-01 19:24:20 +01:00
Pratik Mankawde
98fc939851 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-01 15:01:19 +01:00
Pratik Mankawde
4d6ddb5f1f Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-01 14:56:09 +01:00
Pratik Mankawde
e7dea147cd Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 18:18:36 +01:00
Pratik Mankawde
8d730b8b9a Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 18:16:35 +01:00
Pratik Mankawde
2f96c6547c Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 16:51:31 +01:00
Pratik Mankawde
c187a62353 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 16:47:15 +01:00
Pratik Mankawde
c848e51e13 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 16:44:07 +01:00
Pratik Mankawde
8f9057729c Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 16:14:21 +01:00
Pratik Mankawde
f031befc6e compilation fixes and levelization fixes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 16:04:19 +01:00
Pratik Mankawde
3a1f22583f Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 15:34:22 +01:00
Bart
2f3558c610 ci: Run PR title and description checks on staging and release branches (#7331)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-05-28 14:57:29 +00:00
Ayaz Salikhov
f9551ac5ca style: Run shfmt on workflows, actions and markdown bash code (#7333) 2026-05-27 19:24:18 +00:00
Ayaz Salikhov
1162371def ci: Only push docker images in XRPLF/rippled (#7330) 2026-05-26 20:03:04 +00:00
dependabot[bot]
2a0feca46b ci: [DEPENDABOT] bump docker/setup-buildx-action from 4.0.0 to 4.1.0 (#7322)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-26 19:36:32 +00:00
dependabot[bot]
108a4c8217 ci: [DEPENDABOT] bump codecov/codecov-action from 6.0.0 to 6.0.1 (#7321)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-26 19:36:21 +00:00
dependabot[bot]
4584b01bde ci: [DEPENDABOT] bump docker/build-push-action from 7.1.0 to 7.2.0 (#7320)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-26 19:36:13 +00:00
dependabot[bot]
7c59786565 ci: [DEPENDABOT] bump docker/metadata-action from 6.0.0 to 6.1.0 (#7319)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-26 19:36:00 +00:00
dependabot[bot]
9623e67b76 ci: [DEPENDABOT] bump docker/login-action from 4.1.0 to 4.2.0 (#7318)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-26 19:35:52 +00:00
Ayaz Salikhov
23d0812827 style: Use shfmt instead of bashate (#7326) 2026-05-26 18:28:23 +00:00
Ayaz Salikhov
49cb3f45a4 ci: Add clang to nix images (#7308)
Co-authored-by: semgrep-companion-app[bot] <218312740+semgrep-companion-app[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-26 15:45:33 +00:00
Pratik Mankawde
e9d885bd9b fix: Fix clang-tidy pre-commit hook to locate compile_commands.json from repo root (#7325)
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-26 13:50:18 +00:00
Michael Legleux
a37afe13ff ci: Re-enable full nproc for Linux (#7315) 2026-05-22 11:30:37 +00:00
dependabot[bot]
d9a3af8207 ci: [DEPENDABOT] bump actions/upload-artifact from 7.0.0 to 7.0.1 (#7286)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-19 16:35:38 +00:00
Ayaz Salikhov
8d1083e5ea ci: Only run reusable package in public repos (#7293) 2026-05-19 13:15:11 +00:00
Michael Legleux
93836f22db ci: Add Linux package builds (DEB + RPM) to CI (#6639) 2026-05-16 05:08:37 +00:00
Alex Kremer
5b6e8b6f93 refactor: Rename static constants (#7120)
Co-authored-by: Bart <bthomee@users.noreply.github.com>
2026-05-15 15:32:19 +00:00
Ayaz Salikhov
15b3ed1ae7 ci: Update XRPLF/actions (#7281) 2026-05-15 11:21:29 +00:00
Pratik Mankawde
3f8aa47224 fix(telemetry): drop duplicate Beast MetricsRegistry test + remove author-local symlink
- `src/test/telemetry/MetricsRegistry_test.cpp` (Beast `unit_test::suite`
  format under `src/test/`) duplicates the GTest version already
  maintained at `src/tests/libxrpl/telemetry/MetricsRegistry.cpp`.
  Project rule (`tasks/lessons.md` §Test Format): all new tests use
  GTest under `src/tests/libxrpl/`. The GTest version exercises the
  same four cases (disabled construction, start/stop lifecycle, recording
  no-op, destructor-calls-stop). Deleting the Beast duplicate eliminates
  drift and keeps the test authoritative in one place.
- Drop the matching `test.telemetry > xrpl.basics/xrpl.core/xrpld.telemetry`
  entries from `.github/scripts/levelization/results/ordering.txt`
  because `xrpl.test.telemetry` (the GTest binary) retains its own
  entries; the removed ones belonged to the deleted Beast suite.

- `.claude/instructions.md` was committed as a symlink to an
  author-local absolute path (`/home/pratik/sourceCode/personal/Rippled/
  instructions.md`) that does not exist for any other contributor or in
  CI. Remove the symlink from git tracking and add `.claude/` to
  `.gitignore` so future agent commits do not re-add per-developer
  settings.
2026-05-14 17:27:28 +01:00
Pratik Mankawde
a9e4006591 fix(telemetry): address clang-tidy CI failures on phase10 beast test
- Add missing direct includes (contract.h, Log.h, Journal.h, uint256.h,
  suite.h, io_context.hpp, optional, stdexcept, string).
- Replace broad unit_test.h with specific unit_test/suite.h.
- Concatenate nested namespaces (xrpl::test).
- Add [[nodiscard]] to getOpenLedger/isStopping/getTrapTxID overrides.
- Make const-eligible variables const (Journal j, registry in
  disabled_construction test).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-13 18:27:05 +01:00
Ayaz Salikhov
648ec747f2 feat: Implement nix-based Dockerfile for CI (#7083) 2026-05-13 15:10:53 +00:00
Pratik Mankawde
4470ae7bc9 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-05-13 12:13:35 +01:00