Commit Graph

16220 Commits

Author SHA1 Message Date
Pratik Mankawde
e54d81a644 style(telemetry): dim the Locust annotation colour
Use `rgb(15, 122, 102)` instead of `rgb(25, 158, 112)`: the brighter step drew
too much attention for a background band.

This is the darkest teal that still separates from the JMeter grey by a readable
margin -- normal-vision dE 15.6 against a floor of 15, CVD dE 12.3 against a
target of 8, and at least 3:1 on the dark surface. Dimmer steps fail: rgb(25,
100, 90) lands at dE 9.5, and a grey-derived rgb(25, 70, 70) at dE 5.8, which is
indistinguishable from the JMeter grey even with full colour vision.
2026-08-20 19:16:21 +01:00
Pratik Mankawde
3ed5baf485 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-08-20 19:11:28 +01:00
Pratik Mankawde
1a280603b8 style(telemetry): keep the JMeter annotation colour as it was
The driver split changed the existing perf-run regions from grey to violet,
which was not asked for. Restore `rgb(70, 70, 70)` on `Perf Runs (JMeter)` so
every region that rendered before keeps its colour; `Perf Runs (Locust)` stays
aqua, since it is new.

Grey separates from aqua well (dE 22.8 deutan, 25.9 tritan, 26.1 normal), but it
sits at 1.98:1 against the dark-theme surface, below the 3:1 floor, so its region
edges read faint there. Noted in the runbook.
2026-08-20 19:10:00 +01:00
Pratik Mankawde
62ef753804 feat(telemetry): split perf-run annotations by load driver
A single "Annotate perf-iac runs" layer matched only `perf-iac`, so a Locust
load window was indistinguishable from a JMeter one. perf-iac now tags every
region with its load driver, so each driver can have its own layer and colour.

- Replace that layer with `Perf Runs (JMeter)` and `Perf Runs (Locust)`, each
  matching ["perf-iac", "<driver>"] with matchAny:false, on 12 dashboards.
- job-queue, ledger-data-sync and log-derived-insights had an empty annotations
  list and drew no perf regions at all; they now carry the builtIn layer plus
  both driver layers.
- Grafana tag matching is a superset AND with no negation, so a generic
  `perf-iac` layer also matches every driver region. Keeping one alongside the
  driver layers would draw each load window twice, so it is replaced, not kept.
- Document the layers in the telemetry runbook, including two rendering limits:
  annotations draw only on timeseries, state-timeline and candlestick panels,
  and the shaded fill is 10% opacity so the region edges carry the colour.
- Add `jmeter` to the cspell dictionary; the hook rejects the bare word.
2026-08-20 18:33:19 +01:00
Pratik Mankawde
74d0bb4ae7 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-08-20 18:19:34 +01:00
Pratik Mankawde
d8035a35aa fix(telemetry): include the headers the new code depends on
clang-tidy runs misc-include-cleaner with WarningsAsErrors, so a symbol
reached only transitively fails CI. Add the direct includes for JLOG,
beast::Journal, StartUpType, TokenType, toBase58, std::exception and
std::size_t.
2026-08-20 18:19:22 +01:00
Pratik Mankawde
308e77d003 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-08-20 16:54:11 +01:00
Pratik Mankawde
dbef6a705e Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-08-20 16:54:11 +01:00
Pratik Mankawde
c6491eef27 fix(telemetry): log without JLOG in the beast insight collector
JLOG is defined in xrpl/basics/Log.h, and libxrpl.beast cannot include
xrpl.basics -- basics depends on beast, not the reverse. Use the journal
stream idiom the rest of the file already uses.
2026-08-20 16:54:00 +01:00
Pratik Mankawde
f572aedeec Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	OpenTelemetryPlan/05-configuration-reference.md
#	docs/telemetry-runbook.md
2026-08-20 16:50:46 +01:00
Pratik Mankawde
b2ba81a56c Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
# Conflicts:
#	docs/telemetry-runbook.md
2026-08-20 16:46:56 +01:00
Pratik Mankawde
466660564f Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	src/xrpld/app/main/Main.cpp
2026-08-20 16:45:37 +01:00
Pratik Mankawde
597b0c2bab Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/xrpld-telemetry.cfg
#	src/libxrpl/beast/insight/OTelCollector.cpp
#	src/libxrpl/telemetry/Telemetry.cpp
#	src/xrpld/app/main/Application.cpp
2026-08-20 16:43:32 +01:00
Pratik Mankawde
2cc6a5f4f2 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-08-20 16:43:00 +01:00
Pratik Mankawde
8fac7fc08c Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-08-20 16:38:03 +01:00
Pratik Mankawde
9f85564172 fix(telemetry): give the metrics shutdown a barrier, not a flag
detachCallbacks() flips a flag that each observable callback checks on entry,
which leaves a callback already past that check running while the twelve
service stops below it tear down the state it reads. Stop the provider at the
same point instead: that joins the reader thread, so once it returns no
callback is running and none can start. Metrics recorded during the remaining
shutdown steps are no longer exported, which is the cost of the guarantee.

Build metricsRegistry_ in the member-init list rather than assigning it in
setup(). getMetricsRegistry() is read from the job queue and io threads, which
are running by then, so the later assignment was an unsynchronised write to the
handle those reads follow.
2026-08-20 16:36:59 +01:00
Pratik Mankawde
4278014ab0 fix(telemetry): order the metrics pipeline by instrument kind
beast::insight instruments are created during ApplicationImp's member-init
list, and opentelemetry-cpp 1.28 never rebinds an already-vended Meter, so an
instrument created before the MeterProvider is published records nothing for
the rest of the process. Observable instruments carry the opposite constraint:
registering one arms the SDK reader thread, and its callbacks run hook handlers
that read services which do not exist that early.

Publish the provider in Telemetry's constructor, ahead of every producer, and
defer only the observables. Collector gains onCollectionReady() and
onCollectionStopping(); OTelCollector arms and disarms its gauges in response.
StatsDCollector starts its polling thread in its own constructor and had the
same hazard, so it uses the pair to gate that thread.

The metrics resource carries service.instance.id and is immutable once built,
so the node public key is resolved in Main.cpp, where a config error can still
be reported, and passed to makeApplication(). getNodeIdentity() remains
authoritative; both paths now share readNodeIdentity(), so telemetry cannot
report a key the node has abandoned.

An explicit ~ApplicationImp stops observing and stops telemetry, covering the
setup() failure paths that never reach run(). Telemetry::stop() is once-only
and no longer clears another instance's global pointer. The histogram view's
meter selector now matches the meter actually in use, so its bucket boundaries
apply for the first time.
2026-08-20 16:36:41 +01:00
Pratik Mankawde
89b58da1e8 fix: Report telemetry config errors instead of aborting at startup
makeTelemetrySetup() rejects a contradictory [telemetry] mutual-TLS
setup by throwing, but it is called from ApplicationImp's
member-initializer list. A try/catch in the constructor body cannot
reach a throw from there, and nothing further up the stack caught it
either, so a config mistake reached std::terminate: the default handler
printed a terminate dump and raised SIGABRT, leaving a core file
instead of a startup error.

Catch std::exception around makeApplication() in run(), report the
reason on stderr and return -1, so the failure is a clean non-zero exit
with a message an operator can act on. Only the construction is
wrapped. setup() starts subsystems whose shutdown order is delicate and
is left outside deliberately, because unwinding a half-started
Application would skip the normal stop sequence.

Gate both validation guards on enabled. A node with telemetry switched
off previously refused to start over certificate paths that nothing
would read.

Document both throws on makeTelemetrySetup(), state in
cfg/xrpld-example.cfg and the configuration reference that a partial
mutual-TLS setup is fatal and that the checks apply only when
enabled=1, and add a runbook troubleshooting entry keyed on the two
error messages.

Tests cover both guards with the message asserted so the two are told
apart, both enabled=0 paths, and the default plaintext configuration.
2026-08-20 16:14:56 +01:00
Pratik Mankawde
4f4f0cf613 fix(test): follow TempDir out of beast:: after the develop merge
develop moved TempDir from beast:: to xrpl::, deleting
include/xrpl/beast/utility/temp_dir.h in favour of
include/xrpl/basics/FileUtilities.h. The merge kept this branch's
references to the old API, so the tree no longer compiled: the missing
header is a fatal include, and because DatabaseConfig_test.cpp lands in
the xrpld unity blob it broke the xrpld target itself, not just the tests.

Swap the include and drop the stale beast:: qualifier on 17 uses. The
three src/tests/libxrpl/nodestore files already include FileUtilities.h
and already spell TempDir unqualified elsewhere, so only the qualifier
was wrong there. DatabaseConfig_test.cpp needed the include as well; it
sits in namespace xrpl::node_store, so unqualified TempDir resolves to
xrpl::TempDir through the enclosing namespace.

Two further uses exist only on the sync-diagnostics tip and are fixed
there rather than here.
2026-08-20 13:55:07 +01:00
Pratik Mankawde
7d9392aeb6 makeMetricsResource added
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-08-20 13:36:46 +01:00
Pratik Mankawde
5ce2bad4a9 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	.cspell.config.yaml
#	src/tests/libxrpl/nodestore/NuDBFactory.cpp
2026-08-20 12:14:58 +01:00
Pratik Mankawde
ed36578077 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
# Conflicts:
#	.cspell.config.yaml
2026-08-20 12:13:09 +01:00
Pratik Mankawde
e6688d8a0b Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	.cspell.config.yaml
2026-08-20 12:12:36 +01:00
Pratik Mankawde
35c3c31b38 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	.cspell.config.yaml
2026-08-20 12:10:49 +01:00
Pratik Mankawde
45ad80c57a Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-08-20 12:10:18 +01:00
Pratik Mankawde
074f71034b Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-08-20 12:10:08 +01:00
Pratik Mankawde
687cc57595 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
# Conflicts:
#	src/libxrpl/tx/Transactor.cpp
2026-08-20 12:09:58 +01:00
Pratik Mankawde
91596e2c7b Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
# Conflicts:
#	.cspell.config.yaml
#	src/tests/libxrpl/CMakeLists.txt
2026-08-20 12:07:11 +01:00
Pratik Mankawde
1135470656 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
# Conflicts:
#	src/xrpld/app/main/GRPCServer.cpp
2026-08-20 12:05:54 +01:00
Pratik Mankawde
8c9a79e4ae Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra
# Conflicts:
#	.gitignore
#	conan.lock
2026-08-20 12:05:03 +01:00
Pratik Mankawde
33ebccef15 Merge branch 'develop' into pratik/otel-phase1a-plan-docs
# Conflicts:
#	.cspell.config.yaml
2026-08-20 12:03:10 +01:00
Pratik Mankawde
070d29b465 feat(telemetry): add the xrpl.node.id resource attribute
Node identity reached the OTel resource only as service.instance.id, which is
config-overridable and carries a deployment-chosen label rather than the node's
own identity. Add xrpl.node.id, set unconditionally from the node public key
(base58, TokenType::NodePublic), so traces and metrics share a stable per-node
key independent of [telemetry] service_instance_id.

Set on the tracer resource via Telemetry::setNodeId(), called from
ApplicationImp::setup() once nodeIdentity_ is known, and on the MetricsRegistry
resource via an added start() parameter. The beast::insight meter provider is
built in TelemetryImpl's constructor, before the wallet DB exists, so its
resource cannot carry the value; that path is left for later and the attribute
is omitted rather than stamped blank.

Also drops the transform/spanidentity collector processor added in
4a361a496d: per-node identity belongs on the resource, not copied onto every
span.
2026-08-19 19:50:40 +01:00
Ayaz Salikhov
da57183e0c build: Compress the RPM payload with zstd (#8047) 2026-08-19 15:05:04 +00:00
Pratik Mankawde
4a361a496d fix(telemetry): carry the per-node id on spans, not only the resource
Consensus spans share one deterministic, ledger-derived trace_id, so a
single trace holds spans from every node and the resource-level node id is
not a reliable per-span discriminator in stored traces.

Add transform/spanidentity to both collector configs, copying
service.instance.id onto every span as service_instance_id so TraceQL can
filter per node with the same value the $node dashboard variable already
uses on the metrics side. Wired into the traces pipeline locally and into
traces/store (after tail_sampling) on the Grafana Cloud variant.
2026-08-19 15:44:32 +01:00
Sergey Kuznetsov
f370289733 chore: Rust-C++ cmake and CI integration (#7034) 2026-08-19 14:30:06 +00:00
Vito Tumas
d1dc7a6ccf refactor: Extract invariant invocation into free checkInvariants runner (#7404)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-19 14:10:11 +00:00
Ayaz Salikhov
5639863715 docs: Rewrite the install guide (#8048) 2026-08-19 14:02:42 +00:00
Ayaz Salikhov
1be4868875 build: Sign RPM packages (#8046) 2026-08-19 13:46:11 +00:00
Timur Yalymov
368ff1afce fix: Exempt loan default from asset freeze (#7932)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Vito Tumas <5780819+Tapanito@users.noreply.github.com>
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2026-08-19 13:43:40 +00:00
Vito Tumas
3adf2d40b5 fix: Reject VaultWithdraw fixed-share amounts that round to zero (#7950) 2026-08-19 13:09:38 +00:00
Ayaz Salikhov
a6983f8bf3 build: Use AlmaLinux for the RHEL packaging image (#8045) 2026-08-19 00:25:46 +00:00
Ayaz Salikhov
4113b105a5 build: Run nix macos builds in CI; deny nix store references (#8023) 2026-08-18 23:34:16 +00:00
Olek
7442ff2dec fix: Enable reserve checking on ending sponsorship (#8044) 2026-08-18 22:39:32 +00:00
Kassaking7
666e77b22c fix: Add ValidPermissionedDEX invariant track for fully consumed offer (#6736) 2026-08-18 21:08:02 +00:00
Vito Tumas
8c12de6c56 test: Split Vault_test into topical suites under src/test/app/vault/ (#8041) 2026-08-18 17:57:12 +00:00
Shawn Xie
b21fd86f6e fix: Fix assorted NFT and pDEX bugs (#7749) 2026-08-18 17:56:33 +00:00
Ayaz Salikhov
f5f47f1cf5 chore: Publish debian/rpm packages from GitHub directly (#8031) 2026-08-18 15:03:45 +00:00
Bart
ca39bff3c8 refactor: Add SHAMapNodeID::isPrefixOf (#7939)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-08-18 12:35:32 +00:00
Vito Tumas
dd0edc19a0 fix: Conserve funds correctly when LoanPay fee payee is below reserve (#7843) 2026-08-18 11:09:33 +00:00
Copilot
820ca5b332 refactor: Convert boost::beast::string_view to std::string_view (#6306)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mvadari <8029314+mvadari@users.noreply.github.com>
Co-authored-by: Mayukha Vadari <mvadari@ripple.com>
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
Co-authored-by: xrplf-ai-reviewer[bot] <266832837+xrplf-ai-reviewer[bot]@users.noreply.github.com>
Co-authored-by: Mayukha Vadari <mvadari@gmail.com>
Co-authored-by: Timur Yalymov <36795566+tyalymov@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Vito Tumas <5780819+Tapanito@users.noreply.github.com>
2026-08-17 23:19:56 +00:00