Commit Graph

346 Commits

Author SHA1 Message Date
Pratik Mankawde
5e398cd6b7 merge: bring develop forward into phase-9 metric gap fill
Resolved five files.

PerfLogImp.cpp: PerfLog now takes its method names as string_view, so the
methods map stores the map key rather than a char const*. Kept this branch's
scoped lock so the OTel recording still runs after methodsMutex is released,
and kept both the NetworkOPs and metric-macro includes.

PeerImp.h: took develop's earlier `protected:`, which also covers
processLedgerRequest, and kept recordGetObjectMetrics inside it. Dropped the
comment naming one test, since four test files now rely on that access.

TMGetObjectByHash_test.cpp: kept this branch's suite. It already runs develop's
three bound cases (limit+1, limit, limit-1) as testReplyLimit, plus the charge
and hit/miss cases. develop's extracted PeerTest.h stays, used by its four new
overlay tests.

ordering.txt: regenerated. Both perflog edges are real, to xrpl.server and to
xrpl.telemetry.

.cspell.config.yaml: kept both added words.
2026-09-21 12:51:31 +01:00
Pratik Mankawde
d16f2c4181 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-21 12:43:48 +01:00
Bart
f6b51f0b8b ci: Say when a package publish is a dry run (#8247)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-09-18 18:31:07 +00:00
Denis Angell
07b36871c7 ci: Publish packages through packages-upload.xrplf.org (#8241) 2026-09-17 18:05:51 +00:00
Ayaz Salikhov
d6022fbc4d build: Fix test installation on debian:11 due to EOL 2026-09-17 13:33:06 +02:00
Ayaz Salikhov
295b74da1c build: Add assert-enabled builds and packages 2026-09-17 13:33:05 +02:00
Pratik Mankawde
8bd043ecb7 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-15 16:17:44 +01:00
Pratik Mankawde
6a0642817d Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-15 16:17:43 +01:00
Sergey Kuznetsov
9403736199 ci: Exclude Rust unit tests from code coverage (#8203) 2026-09-10 11:38:25 +00:00
Ayaz Salikhov
1381483c7a build: Fix test installation on debian:11 due to EOL (#8200) 2026-09-08 22:49:16 +00:00
Pratik Mankawde
bae31955db merge: bring the CI fixes forward from phase8-log-correlation
One conflicted file, docs/telemetry-runbook.md, with three spots:

- Build section: both sides added different text at one point. Kept both,
  incoming sentence first, then this branch's "Run against a live network".
- Disabling section, first spot: this branch's wording names the config
  section and says no rebuild is needed, so it already covers the incoming
  sentence.
- Disabling section, second spot: kept this branch's paragraph and folded in
  the one point it lacked, that both flags have to be passed.
2026-09-08 17:22:04 +01:00
Pratik Mankawde
31ace1a357 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-08 16:02:42 +01:00
dependabot[bot]
3e4bdf2782 ci: [DEPENDABOT] bump actions/deploy-pages from 5.0.0 to 5.0.1 in the github-actions group across 1 directory (#8180)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-07 16:07:56 +00:00
Ayaz Salikhov
3e54e7d00b fix: Don't use github.workspace as it slow downs gcovr (#8173) 2026-09-03 19:20:38 +00:00
Pratik Mankawde
54d6e80e2e merge: bring develop forward from phase8-log-correlation
Two conflicts, both additions at the same spot:
NetworkOPs.cpp include block kept develop's rpc/detail/SyntheticFields.h
alongside this branch's telemetry/MetricsRegistry.h; .cspell.config.yaml
kept both new words.
2026-09-03 17:39:21 +01:00
Pratik Mankawde
5eb8fa9f96 merge: bring develop forward from phase1b-telemetry-infra 2026-09-03 17:31:23 +01:00
Bart
2ddf6ee148 ci: Update pre-commit SHA (#8169)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-09-03 14:52:42 +00:00
Bart
6e1eb88e6e ci: Update prepare-runner SHA (#8168)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-09-03 11:52:08 +00:00
Ayaz Salikhov
49cdc105de fix: Correct and simplify Linux packaging (#8165) 2026-09-02 21:18:13 +00:00
Pratik Mankawde
279ccd84d7 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-08-27 16:04:58 +01:00
Pratik Mankawde
5dfee8564b Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-08-27 16:00:26 +01:00
Ayaz Salikhov
3967ed6d54 build: Update release-info to get better pkg_release (#8131) 2026-08-27 13:34:38 +00:00
Ayaz Salikhov
b6a899583b build: Make packaging reusable (#8126) 2026-08-26 19:44:31 +00:00
Ayaz Salikhov
c28d389e0e build: Refactor generate.py to make packaging_config part of config (#8115) 2026-08-26 14:04:58 +00:00
Sergey Kuznetsov
42502e4263 ci: Update CI image (#8121) 2026-08-26 13:42:19 +00:00
Ayaz Salikhov
fee4bfc22e build: Implement packaging in Python (#8109) 2026-08-25 19:32:03 +00:00
Ayaz Salikhov
45e4b8899d build: Update packaging images; add Python (#8106) 2026-08-25 14:34:41 +00:00
Pratik Mankawde
6e2b2da772 fix(telemetry): resolve microsecond latencies below 100us
The microsecond ladder's first edge was 100us, which sat ABOVE the mass of
every instrument using it. Measured on devnet: 99.3% of job_queued_us
samples, 92.5% of job_running_us and 90.4% of getobject_lookup_us fell in
that first bucket. histogram_quantile then interpolated inside bucket 0 and
returned `quantile / fraction_in_bucket_0 x first_edge` -- p75/p95/p99 of
job_queued_us read 75.52/95.66/99.69us against a prediction of
75.53/95.67/99.70. Three-decimal agreement: those panels were reporting
arithmetic on the bucket edge, not latency.

The fix was already half-written. kSubMillisecondBoundaries had been parked
in MetricsRegistry.cpp as [[maybe_unused]] with a comment noting exactly this
problem for nodestore reads. Its edges are now folded into kMicrosecondBuckets
rather than deleted, so the parked intent is carried forward: 1..1000us
resolution where the mass is, upper edges unchanged so multi-second stalls
stay measurable.

Also moves the GetObject count and charge ladders into HistogramBuckets.h, so
all five ladders have one owner and one set of invariant tests (29 now).

Adds check_bucket_parity.py, wired into the existing OTel naming workflow.
The C++ millisecond ladder and the collector's spanmetrics ladder are
specified to agree over their shared range; they were identical when shipped,
then the collector side alone was extended and nothing noticed for eleven
phases. The check asserts containment rather than equality, because jobs
outlive spans -- jobq_updatepaths averages ~60s, which no span approaches, so
demanding equality would force a ceiling that censors it. Verified it rejects
a missing collector edge, a bogus in-range edge, and a return to the 5s
ceiling.

ledger-data-sync's "Job Queue Wait p95 By Type" moves off the beast
jobq_*_q_milliseconds pair onto job_queued_us filtered by job_type. Those
beast metrics are ms-quantised at the source (Event rounds up to a whole
millisecond), so 94-100% of their samples sat in the first bucket and no
ladder change could fix them. Note the label values are camelCase
(job_type="ledgerData"), not the lowercase metric-name fragments.

Both histogram-fed alert thresholds re-validated and left unchanged, with the
measured basis recorded so neither gets tuned against the old artefact: only
0.0022% of job_queued_us samples exceed the 1s threshold, and every edge
bracketing the 1000ms ios_latency threshold survived the ladder change.

Docs: the rpc_size "known issue -- tracked separately" notes in the runbook
and 09-data-collection-reference are now resolved notes, the stale 10-edge
span_duration bucket list is corrected to the collector's real 20, and the
runbook gains a "Reading A Histogram Percentile" section covering both
saturation traps and the expected discontinuity after a ladder change.
2026-08-21 12:46:56 +01:00
Sergey Kuznetsov
422e5245e4 ci: Save cargo cache only from develop by default (#8063) 2026-08-20 16:02:26 +00:00
Ayaz Salikhov
b1ac891210 ci: Do not cache cargo binaries (#8062) 2026-08-20 11:47:27 +00:00
Pratik Mankawde
5ce2bad4a9 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	.cspell.config.yaml
#	src/tests/libxrpl/nodestore/NuDBFactory.cpp
2026-08-20 12:14:58 +01:00
Pratik Mankawde
1135470656 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
# Conflicts:
#	src/xrpld/app/main/GRPCServer.cpp
2026-08-20 12:05:54 +01:00
Sergey Kuznetsov
f370289733 chore: Rust-C++ cmake and CI integration (#7034) 2026-08-19 14:30:06 +00:00
Ayaz Salikhov
1be4868875 build: Sign RPM packages (#8046) 2026-08-19 13:46:11 +00:00
Ayaz Salikhov
a6983f8bf3 build: Use AlmaLinux for the RHEL packaging image (#8045) 2026-08-19 00:25:46 +00:00
Ayaz Salikhov
4113b105a5 build: Run nix macos builds in CI; deny nix store references (#8023) 2026-08-18 23:34:16 +00:00
Ayaz Salikhov
f5f47f1cf5 chore: Publish debian/rpm packages from GitHub directly (#8031) 2026-08-18 15:03:45 +00:00
Ayaz Salikhov
bd87edfc75 test: Check versioned tools in check-tools & print nicely (#8030) 2026-08-14 14:07:55 +00:00
Pratik Mankawde
3153f3ef56 docs(telemetry): align runbook and plan docs with the shipped phase-9/10 code
The reference docs had drifted from the code in ways that break the reader
rather than merely misinform: PromQL examples that return no data, a rollback
flag that is a no-op, a sampling knob that does not exist, and two span parents
that moved. Code is treated as the truth throughout; where the code is the
defective side, the doc now records it as a known issue instead of describing
the bug as intent.

Renames the docs missed: histogram names gain the exporter's unit suffix
(ios_latency_milliseconds_bucket and four siblings), ledger_history_mismatch
gains _total, the StatsD-era quantile label gives way to le buckets,
rpc.request becomes rpc.http_request, traces_spanmetrics_calls_total becomes
span_calls_total, and the nine dotted xrpl.* span attributes are recorded as
renamed rather than left as live keys.

Re-parenting: consensus.update_positions and consensus.check are children of
consensus.establish, not of consensus.round.

Units and labels: state_accounting_*_duration is microseconds, not seconds;
cache_metrics label values are case-sensitive; object_count carries demangled
C++ type names. Nodestore read and write latency stays microseconds -- the
nanosecond accumulator change did not move the exported unit.

Adds what shipped but was undocumented: the ledger.acquire span, seven
consensus.round events, twelve span attributes, node_writes_duration_us, the
7-day validation-agreement window, the TxQ admission and reduce-relay metric
families, metrics_endpoint, and the phase-10 validation workflow.

Corrects claims that never held: 10% head sampling (it is fixed at 100%),
configurable redaction (it is unconditional), -DXRPL_ENABLE_TELEMETRY=OFF
(the flag is -Dtelemetry=OFF, default ON), FindOpenTelemetry.cmake and the
xrpl_telemetry target (neither exists), Promtail and a StatsD exporter in the
pipeline (neither exists), and Loki stream selection on job= (only
service_name is a stream label).

Phase 9 is marked complete, its provisioned alerting is attributed to the
branch that shipped it, and Phase 11 stays at zero except the one prerequisite
its code closes. Counts are reconciled repo-wide: 41 emitted span families,
15 dashboards on disk with 14 asserted, 13 alert rules in 5 groups.

Hardens the gate that let this drift through: Rule E of the naming check now
covers the reference docs, its allow-dotted marker is key-scoped and warns on
stale or empty use, a missing checked file is reported instead of silently
skipped, the test suite runs in CI, and doc paths trigger the check.

C++ and CMake changes are comment-only: three MetricsRegistry instrument names,
eight OTelCollector claims of a metric-name prefix that formatName never adds,
and the telemetry option's inverted default.
2026-08-13 18:55:32 +01:00
Ayaz Salikhov
9859e5ceda Merge remote-tracking branch 'upstream/release/3.3.x' into mathbunnyru/merge-3.3.0-to-develop
* upstream/release/3.3.x: (41 commits)
  chore: Bump version to 3.3.0
  chore: Bump version to 3.3.0-rc7
  fix: Increase manifest protocol message size cap and fix manifests relay
  fix: Cap untrusted manifests per message and drop oversized ones
  chore: Bump version to 3.2.1
  chore: Bump version to 3.2.1-rc1
  fix: Cap untrusted manifests per message and drop oversized ones
  fix: Reject oversized validator manifest before decoding
  fix: Reduce untrusted manifest cache cap to 100
  fix: Bound untrusted manifest cache
  chore: Bump version to 3.3.0-rc6
  feat: Package validator-keys inside rippled
  chore: Bump version to 3.3.0-rc5
  fix: Switch SponsorshipSet to use a delta for sfFeeAmount
  fix: Re-revert "fix: Set request size limits and differential pricing for get-object-by-hash calls"
  chore: Bump version to 3.3.0-rc4
  fix: Revert "fix: Set request size limits and differential pricing for get-object-by-hash calls"
  chore: Bump version to 3.3.0-rc3
  fix: Reduce untrusted manifest cache cap to 100
  fix: Revert "fix: Reject oversized SHAMap nodes in gotStaleData and fetch-pack path"
  ...
2026-08-07 16:00:25 +01:00
Pratik Mankawde
379f4ff60b Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-08-07 14:34:47 +01:00
Ayaz Salikhov
cb425647a4 ci: Generate protocol_autogen only once in CI (#7918) 2026-08-06 13:25:28 +00:00
Ayaz Salikhov
5bf24c4045 ci: Use a separate benchmark filter (#7919) 2026-08-06 11:04:41 +00:00
Ayaz Salikhov
3ad6ce236e feat: Package validator-keys inside rippled 2026-07-30 11:29:37 -04:00
Pratik Mankawde
6b6a23df4e Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-07-29 14:09:48 +01:00
Ayaz Salikhov
ff7fff2cf2 style: Make clang-tidy format files using clang-format rules (#7880) 2026-07-27 18:39:53 +00:00
Mayukha Vadari
2db631c915 ci: Change server_definitions upload config name (#7878) 2026-07-27 17:42:07 +00:00
Pratik Mankawde
7e5f726388 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-07-27 16:56:15 +01:00
Sergey Kuznetsov
e0aa50c4be ci: Update CI image and prepare-runner action (#7874) 2026-07-27 14:18:01 +00:00
dependabot[bot]
b878818e80 ci: [DEPENDABOT] bump actions/checkout from 7.0.0 to 7.0.1 (#7871)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 09:55:01 +00:00