merge: bring the traces_endpoint rename forward from phase-9

This commit is contained in:
Pratik Mankawde
2026-09-03 16:07:14 +01:00
45 changed files with 883 additions and 553 deletions

View File

@@ -2,13 +2,13 @@
"""Assert the C++ millisecond ladder agrees with the collector's spanmetrics ladder.
The two are specified to match so a span-derived latency panel and a native
histogram panel can be read on the same scale. They *were* identical when first
shipped. Then the collector ladder alone was extended -- sub-millisecond edges
below 1ms and second-scale edges up to 30s -- and nothing checked the other
side, so the C++ ladder stayed capped at 5s. Every quantile above 5s then read
back as a flat 5000, because Prometheus returns the second-highest edge for a
quantile landing in the `+Inf` bucket. That looks like a measurement rather
than an error, which is why it survived for eleven phases.
histogram panel can be read on the same scale. Nothing else couples them, so
extending one ladder alone -- sub-millisecond edges below 1ms, second-scale
edges up to 30s -- silently leaves the other short. That failure is quiet:
Prometheus returns the second-highest edge for a quantile landing in the
`+Inf` bucket, so every quantile above a too-low ceiling reads back as a flat
number that looks like a measurement rather than an error. This check is what
makes the drift loud.
The rule is containment, not equality:
@@ -17,7 +17,7 @@ The rule is containment, not equality:
* the C++ ladder MAY carry extra edges ABOVE the collector's highest edge,
because jobs outlive spans -- the updatepaths job type was measured
averaging ~60s, which no span approaches. Demanding equality would force a
ceiling that censors it, reintroducing the bug this guards against;
ceiling that censors it, recreating the failure this guards against;
* collector edges below 1ms are expected to be ABSENT rather than missing:
beast::insight::Event rounds every duration up to a whole millisecond
before it reaches the histogram, so those edges could never collect a
@@ -117,8 +117,10 @@ def main():
)
print(
"\nThe two ladders must agree over their shared range. Extra C++ edges are\n"
"permitted only ABOVE the collector's highest edge. Change both sides, or\n"
"change the spec in OpenTelemetryPlan/Phase7_taskList.md.",
"permitted only ABOVE the collector's highest edge. To re-price the shared\n"
f"range, edit the ladder in {HEADER} and the\n"
f"spanmetrics 'buckets:' list in {COLLECTOR}\n"
"in the same change, so both sides stay in step.",
file=sys.stderr,
)
return 1

View File

@@ -69,7 +69,7 @@ The authoritative `[telemetry]` example lives in `cfg/xrpld-example.cfg`. Teleme
| Option | Type | Default | Description |
| -------------------------- | ------ | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `enabled` | 0 or 1 | `0` | Enable/disable telemetry |
| `endpoint` | string | `http://localhost:4318/v1/traces` | OTLP/HTTP collector endpoint for **traces** |
| `traces_endpoint` | string | `http://localhost:4318/v1/traces` | OTLP/HTTP collector endpoint for **traces** |
| `metrics_endpoint` | string | `http://localhost:4318/v1/metrics` | OTLP/HTTP collector endpoint for the native metrics pipeline (`MetricsRegistry`). Read in `Application.cpp:1670` |
| `use_tls` | 0 or 1 | `0` | Enable TLS for exporter connection |
| `tls_ca_cert` | string | `""` | Path to CA certificate file |
@@ -131,10 +131,10 @@ The parser `makeTelemetrySetup()` in `src/libxrpl/telemetry/TelemetryConfig.cpp`
`metrics_endpoint` is deliberately **not** handled here: it is read separately in `ApplicationImp::startTelemetry()` (`Application.cpp:1670`) and passed to `MetricsRegistry::start()`. Note the consequence — the two metric exporters resolve their URL differently:
| Metric source | Exporter built by | URL comes from |
| ------------------------------------------ | -------------------------------------------- | -------------------------------------------------------------------- |
| `beast::insight` (`[insight] server=otel`) | `Telemetry::initMetrics()` (global provider) | `endpoint` with a trailing `/v1/traces` rewritten to `/v1/metrics` |
| Native `XRPL_METRIC_*` (`MetricsRegistry`) | `MetricsRegistry::initExporterAndProvider()` | `metrics_endpoint`, defaulting to `http://localhost:4318/v1/metrics` |
| Metric source | Exporter built by | URL comes from |
| ------------------------------------------ | -------------------------------------------- | ------------------------------------------------------------------------- |
| `beast::insight` (`[insight] server=otel`) | `Telemetry::initMetrics()` (global provider) | `traces_endpoint` with a trailing `/v1/traces` rewritten to `/v1/metrics` |
| Native `XRPL_METRIC_*` (`MetricsRegistry`) | `MetricsRegistry::initExporterAndProvider()` | `metrics_endpoint`, defaulting to `http://localhost:4318/v1/metrics` |
Setting a non-default `endpoint` therefore moves the insight metrics with it, but leaves the native metrics on localhost unless `metrics_endpoint` is set too.

View File

@@ -2209,7 +2209,7 @@ endpoint=http://localhost:4318/v1/metrics
```ini
[telemetry]
enabled=1
endpoint=http://otel-collector:4318/v1/traces
traces_endpoint=http://otel-collector:4318/v1/traces
trace_peer=0
batch_size=1024
max_queue_size=4096

View File

@@ -38,8 +38,10 @@ The binary is at `.build/xrpld`.
## Test 1: Single-Node Standalone (Quick Verification)
This test verifies RPC and transaction spans in standalone mode. Consensus
spans will not fire because standalone mode does not run consensus.
This test verifies RPC and transaction spans in standalone mode, plus the
consensus spans that a simulated round still produces. The proposal, voting
and peer-facing consensus spans do not fire — see the expected-spans table at
the end of this test for which do and which do not.
### Step 1: Start the observability stack
@@ -125,32 +127,7 @@ curl -s http://localhost:5005 -d '{"method":"ledger_accept"}'
### Step 5: Verify traces in Tempo
Wait 5 seconds for the batch export, then:
```bash
TEMPO="http://localhost:3200"
# Check xrpld service is registered
curl -s "$TEMPO/api/v2/search/tag/resource.service.name/values" | jq '.tagValues[].value'
# Check RPC spans
curl -s "$TEMPO/api/search" \
--data-urlencode 'q={resource.service.name="xrpld" && name="rpc.http_request"}' \
--data-urlencode 'limit=5' | jq '.traces | length'
curl -s "$TEMPO/api/search" \
--data-urlencode 'q={resource.service.name="xrpld" && name="rpc.process"}' \
--data-urlencode 'limit=5' | jq '.traces | length'
curl -s "$TEMPO/api/search" \
--data-urlencode 'q={resource.service.name="xrpld" && name="rpc.command.server_info"}' \
--data-urlencode 'limit=5' | jq '.traces | length'
# Check transaction spans
curl -s "$TEMPO/api/search" \
--data-urlencode 'q={resource.service.name="xrpld" && name="tx.process"}' \
--data-urlencode 'limit=5' | jq '.traces | length'
```
Wait 5 seconds for the batch export, then see the "Verification Queries" section below. Its span loop is a superset of what standalone mode produces, so compare its output against the "Expected spans (standalone mode)" table above rather than running a second, narrower set of queries here.
Or open Grafana Explore with Tempo datasource: http://localhost:3000
@@ -164,23 +141,24 @@ kill $(pgrep -f 'xrpld.*xrpld-telemetry')
docker compose -f docker/telemetry/docker-compose.yml down
# Clean xrpld data
rm -rf data/
rm -rf docker/telemetry/data/
```
### Expected spans (standalone mode)
| Span Name | Expected | Notes |
| --------------------------- | -------- | ----------------------------- |
| `rpc.http_request` | Yes | Every HTTP RPC call |
| `rpc.process` | Yes | Every RPC processing |
| `rpc.command.server_info` | Yes | server_info RPC |
| `rpc.command.server_state` | Yes | server_state RPC |
| `rpc.command.ledger` | Yes | ledger RPC |
| `rpc.command.submit` | Yes | submit RPC |
| `rpc.command.ledger_accept` | Yes | ledger_accept RPC |
| `tx.process` | Yes | Transaction submission |
| `tx.receive` | No | No peers in standalone |
| `consensus.*` | No | Consensus disabled standalone |
| Span Name | Expected | Notes |
| ---------------------------------------------------------------------------------------------------------- | -------- | ------------------------------------------------- |
| `rpc.http_request` | Yes | Every HTTP RPC call |
| `rpc.process` | Yes | Every RPC processing |
| `rpc.command.server_info` | Yes | server_info RPC |
| `rpc.command.server_state` | Yes | server_state RPC |
| `rpc.command.ledger` | Yes | ledger RPC |
| `rpc.command.submit` | Yes | submit RPC |
| `rpc.command.ledger_accept` | Yes | ledger_accept RPC |
| `tx.process` | Yes | Transaction submission |
| `tx.receive` | No | No peers in standalone |
| `consensus.round`, `.phase.open`, `.ledger_close`, `.accept`, `.accept.apply` | Yes | `ledger_accept` drives a simulated round |
| `consensus.establish`, `.update_positions`, `.check`, `.proposal.*`, `.validation.receive`, `.mode_change` | No | `simulate` jumps straight to `Accepted`; no peers |
---
@@ -197,17 +175,11 @@ Run the integration test script:
bash docker/telemetry/integration-test.sh
```
The script will:
It checks prerequisites, clears the previous run, brings up the observability stack, generates six validator key pairs and their node configs, starts the nodes, waits for consensus and then for a validated ledger, exercises RPC and submits a transaction, verifies traces in Tempo and both the spanmetrics and the StatsD-derived metrics in Prometheus, then prints a summary and leaves the stack running.
1. Start the observability stack
2. Generate 6 validator key pairs
3. Create config files for each node
4. Start all 6 nodes
5. Wait for consensus ("proposing" state)
6. Exercise RPC, submit transactions
7. Verify all span categories in Tempo
8. Verify spanmetrics in Prometheus
9. Print results and leave the stack running
The script announces each step as it runs, so read its `Step N:` headers for the authoritative sequence — they are not restated here, because a numbered copy of them drifts as soon as a step is added.
Its Tempo checks cover the RPC, transaction, consensus, ledger and peer span categories from a fixed list, which is narrower than the loop in the "Verification Queries" section below.
### Manual
@@ -243,7 +215,7 @@ Kill the temporary node:
```bash
kill $TEMP_PID
rm -rf data/
rm -rf docker/telemetry/data/
```
#### Step 3: Create node configs
@@ -296,7 +268,7 @@ online_delete=256
[telemetry]
enabled=1
endpoint=http://localhost:4318/v1/traces
traces_endpoint=http://localhost:4318/v1/traces
batch_size=512
batch_delay_ms=2000
max_queue_size=2048
@@ -371,9 +343,11 @@ curl -s http://localhost:5005 -d '{
"Amount": "10000000"
}
}]
}'
}' | jq .result.engine_result
```
Expected result: `"tesSUCCESS"`, the same as Test 1 Step 4.
Wait 15 seconds for consensus and batch export.
#### Step 8: Verify in Tempo

View File

@@ -24,7 +24,7 @@
# Configure xrpld to export traces by adding to xrpld.cfg:
# [telemetry]
# enabled=1
# endpoint=http://localhost:4318/v1/traces
# traces_endpoint=http://localhost:4318/v1/traces
services:
# One-shot init for the collector's offset store. Docker creates a fresh

View File

@@ -384,7 +384,7 @@ ${IPS_FIXED}
[telemetry]
enabled=1
service_instance_id=Node-${i}
endpoint=http://localhost:4318/v1/traces
traces_endpoint=http://localhost:4318/v1/traces
exporter=otlp_http
batch_size=512
batch_delay_ms=2000
@@ -619,6 +619,9 @@ log "--- Spanmetrics ---"
log "Waiting 20s for Prometheus scrape cycle..."
sleep 20
# Names come from the spanmetrics connector's `namespace: "span"` in
# otel-collector-config.yaml. Without that namespace the connector emits
# traces_span_metrics_*, so these queries must move whenever it changes.
calls_count=$(curl -sf "$PROM/api/v1/query?query=span_calls_total" |
jq '.data.result | length' 2>/dev/null || echo 0)
if [ "$calls_count" -gt 0 ]; then
@@ -662,6 +665,12 @@ check_otel_metric() {
fi
}
# Names are what OTelCollector::formatName() produces: the beast::insight
# name lowercased with '.' and ' ' mapped to '_', any group() segment kept, and
# no prefix. The [insight] prefix knob is logged at startup and never applied on
# this path, and the collector's prometheus exporter sets no namespace, so a
# name carrying a product prefix or capitals cannot match any exported series.
# Node health gauges (ObservableGauge — no _total suffix)
check_otel_metric "ledgermaster_validated_ledger_age"
check_otel_metric "ledgermaster_published_ledger_age"
@@ -677,7 +686,8 @@ check_otel_metric "peer_finder_active_outbound_peers"
# RPC counters (Counter — Prometheus adds _total suffix automatically)
check_otel_metric "rpc_requests_total"
# Overlay traffic
# Overlay traffic — one series per TrafficCount category; "total" is the
# aggregate category.
check_otel_metric "total_bytes_in"
# Verify StatsD receiver is NOT required (no statsd receiver in pipeline)

View File

@@ -121,7 +121,7 @@ endpoint=http://localhost:4318/v1/metrics
[telemetry]
enabled=1
service_instance_id=xrpld-devnet
endpoint=http://localhost:4318/v1/traces
traces_endpoint=http://localhost:4318/v1/traces
metrics_endpoint=http://localhost:4318/v1/metrics
batch_size=512
batch_delay_ms=5000

View File

@@ -76,7 +76,7 @@ Add to your `xrpld.cfg`:
```ini
[telemetry]
enabled=1
endpoint=http://localhost:4318/v1/traces
traces_endpoint=http://localhost:4318/v1/traces
```
### 3. Build with telemetry support
@@ -129,7 +129,7 @@ curl -s http://localhost:5015 -d '{"method":"server_info"}' |
| Option | Default | Description |
| -------------------------- | --------------------------------- | ------------------------------------------------------------ |
| `enabled` | `0` | Master switch for telemetry |
| `endpoint` | `http://localhost:4318/v1/traces` | OTLP/HTTP endpoint |
| `traces_endpoint` | `http://localhost:4318/v1/traces` | OTLP/HTTP endpoint |
| `service_name` | `xrpld` | OpenTelemetry service name resource attribute |
| `service_instance_id` | node public key | OpenTelemetry service instance ID resource attribute |
| `trace_rpc` | `1` | Enable RPC request tracing |
@@ -1561,6 +1561,8 @@ The OTel Collector's spanmetrics connector automatically derives RED (Rate, Erro
### Generated Metric Names
These names are deliberately generic: the connector emits **one** metric family covering every span, not a metric per span. Which span a series belongs to comes from the `span_name` label, and the rest of the breakdown from the `dimensions` list in `otel-collector-config.yaml`. So a query always names the span in a label selector rather than in the metric name — `span_calls_total{span_name="ledger.build"}`, never a `ledger_build_calls_total`.
| Prometheus Metric | Type | Description |
| ----------------------------------- | --------- | ---------------------------- |
| `span_calls_total` | Counter | Total span invocations |
@@ -1568,6 +1570,19 @@ The OTel Collector's spanmetrics connector automatically derives RED (Rate, Erro
| `span_duration_milliseconds_count` | Histogram | Latency observation count |
| `span_duration_milliseconds_sum` | Histogram | Cumulative latency |
Only one part of those names is ours to choose. Reading a name left to right:
| Part | Set by |
| ------------------------------------- | ------------------------------------------------------------------------------------ |
| `span_` | the connector's `namespace: "span"` in `otel-collector-config.yaml` — **our choice** |
| `calls`, `duration` | the spanmetrics connector's own metric names |
| `_milliseconds` | the Prometheus exporter, expanding the metric's declared unit |
| `_total`, `_bucket`, `_count`, `_sum` | Prometheus conventions for counters and histograms |
`_milliseconds` rather than `_ms` is therefore not a style decision taken here. The config declares the histogram in milliseconds (`buckets: [1ms, 5ms, ...]`) and never contains the string `milliseconds`; the exporter writes the unit out in full when it translates OTLP to Prometheus. Shortening it would mean renaming the series after export, which would break every dashboard and leave the exported name and the queried name disagreeing.
Drop the `namespace` setting and these become `traces_span_metrics_*` instead — the connector's default. Any query, dashboard panel or test that names one of these metrics has to move with that setting.
### Metric Labels
Every metric carries these standard labels:
@@ -3231,7 +3246,7 @@ Then read the answer off the pair:
| Expensive | Depth over ~1.2 | **Both paths queueing.** Rarer, and neither fix on its own will be enough. Treat the larger of the two costs as the lead. |
**Why the rule is shaped this way.** Three points about the thresholds, each
learned from a dataset that an earlier version of this table got wrong:
grounded in a measured dataset rather than a round number:
- **Read cost is a relative judgement, so the band has a floor and a ceiling, not
one cut.** A cold read on our box measured 31.8 µs mean; a cold read on the

View File

@@ -116,9 +116,8 @@ public:
* @param unit What the samples measure.
*/
virtual Event
makeEvent(std::string const& name, Unit unit)
makeEvent(std::string const& name, [[maybe_unused]] Unit unit)
{
(void)unit;
return makeEvent(name);
}
@@ -130,7 +129,7 @@ public:
return makeEvent(prefix + "." + name);
}
Event
[[nodiscard]] Event
makeEvent(std::string const& prefix, std::string const& name, Unit unit)
{
if (prefix.empty())

View File

@@ -39,14 +39,27 @@
#include <memory>
#include <string>
#include <string_view>
namespace beast::insight {
/**
* Instrumentation scope this collector fetches its Meter under.
*
* Must equal xrpl::telemetry::kMeterName and kMeterVersion, or instruments land
* on a different scope than the views. Duplicated because beast sits below the
* telemetry module and cannot include its header; Telemetry.cpp static_asserts
* the two agree.
*/
inline constexpr std::string_view kOTelMeterName{"xrpld"};
inline constexpr std::string_view kOTelMeterVersion{"1.0.0"};
/**
* @brief A Collector that exports metrics via OpenTelemetry OTLP/HTTP.
*
* Replaces StatsD-based metric collection with native OTel Metrics SDK
* instruments. Each beast::insight instrument maps to an OTel equivalent:
* Selected by `[insight] server=otel`, as an alternative to StatsDCollector:
* it exports through the native OTel Metrics SDK rather than the StatsD wire
* format. Each beast::insight instrument maps to an OTel equivalent:
*
* - Counter -> OTel Counter<uint64_t>
* - Gauge -> OTel ObservableGauge<int64_t> (async callback)
@@ -126,7 +139,7 @@ public:
* @param journal Journal for logging.
* @return Shared pointer to the created Collector.
*/
static std::shared_ptr<Collector>
[[nodiscard]] static std::shared_ptr<Collector>
// NOLINTNEXTLINE(readability-identifier-naming)
New(std::string const& endpoint,
std::string const& prefix,

View File

@@ -7,24 +7,24 @@ namespace beast::insight {
/**
* @brief What an Event's samples measure.
*
* `Event` documents itself as carrying "a millisecond time, or other integral
* value", but both backends used to assume the first case: the OTel bridge
* declared every instrument with unit `ms`, and StatsD tagged every sample
* `|ms`. A size metric therefore exported under a `_milliseconds` name and
* inherited a latency bucket ladder, which censored a quarter of its samples
* and pinned its p95 to a constant.
*
* Naming the unit at creation time is what lets the OTel bridge pick both the
* instrument unit and the matching bucket ladder:
* `Event` carries "a millisecond time, or other integral value", so the unit
* cannot be inferred from the sample. Naming it at creation time is what lets
* the OTel bridge pick both the instrument unit and the matching bucket
* ladder:
*
* makeEvent("time", Unit::Millis) --> OTel unit "ms" --> millisecond ladder
* makeEvent("size", Unit::Bytes) --> OTel unit "By" --> byte ladder
*
* The StatsD backend deliberately ignores this and keeps emitting `|ms` for
* every Event. That path is retired here -- its UDP port is commented out of
* the compose file and the integration test fails if anything is listening on
* 8125 -- so changing its wire format would alter a legacy contract for no
* local benefit and with no way to verify it.
* Without an explicit unit every instrument declares `ms`, so a size metric
* exports under a `_milliseconds` name and inherits a latency bucket ladder.
* For RPC response sizes that ladder censors about a quarter of the samples
* and pins the p95 to a constant.
*
* The StatsD backend deliberately ignores this and emits `|ms` for every
* Event. That path is out of service -- its UDP port is commented out of the
* compose file and the integration test fails if anything is listening on
* 8125 -- so changing its wire format would alter an external protocol
* contract for no local benefit and with no way to verify it.
*
* @note Adding a member requires extending otelUnitCode(), which switches
* exhaustively so a new member is a compile error rather than a silent
@@ -53,7 +53,7 @@ enum class Unit : std::uint8_t {
* @param unit The unit to translate.
* @return A static, null-terminated UCUM code.
*/
constexpr char const*
[[nodiscard]] constexpr char const*
otelUnitCode(Unit unit) noexcept
{
switch (unit)
@@ -78,7 +78,7 @@ otelUnitCode(Unit unit) noexcept
* @param unit The unit to describe.
* @return A static, null-terminated description.
*/
constexpr char const*
[[nodiscard]] constexpr char const*
otelUnitDescription(Unit unit) noexcept
{
switch (unit)

View File

@@ -1689,8 +1689,9 @@ Consensus<Adaptor>::updateOurPositions(std::unique_ptr<std::stringstream> const&
// NOLINTBEGIN(bugprone-unchecked-optional-access) assert above
using namespace telemetry;
// Child of the establish span via its captured context (establishSpan_ is
// a thread-free SpanGuard, so parent explicitly via its context). Null
// context (establish not started) yields a null guard, same as before.
// a thread-free SpanGuard, so parent explicitly via its context). A null
// context — the establish phase has not started — yields a null guard, so
// the setAttribute calls below are no-ops.
auto span = SpanGuard::childSpan(consensus::span::updatePositions, establishSpanContext_);
span.setAttribute(
consensus::span::attr::convergePercent, static_cast<int64_t>(convergePercent_));

View File

@@ -3,10 +3,11 @@
/**
* Enum-to-label mappings for consensus span attribute values.
*
* Split from ConsensusSpanNames.h so that header stays dependency-free like
* its siblings: the span-name and attribute-key constants are included by
* overlay and app translation units that have no use for the consensus
* enums, while these mappings are needed only by Consensus.h.
* These mappings live in their own header so ConsensusSpanNames.h stays
* dependency-free like its siblings: the span-name and attribute-key
* constants are included by overlay and app translation units that have no
* use for the consensus enums, while these mappings are needed only by
* Consensus.h.
*
* ConsensusSpanNames.h (constants only, no domain deps)
* ^

View File

@@ -59,10 +59,10 @@
* | |
* | +-- consensus.accept.apply [jtACCEPT thread, child of accept]
* | Created: Adaptor::doAccept()
* | Attrs: ledger_seq, close_time, close_time_correct,
* | Attrs: ledger_seq, close_time_ripple_epoch_s, close_time_correct,
* | close_resolution_ms, consensus_state, proposing, round_time_ms,
* | parent_close_time, close_time_self, close_time_vote_bins,
* | resolution_direction, tx_count
* | parent_close_time_ripple_epoch_s, close_time_self_ripple_epoch_s,
* | close_time_vote_bins, resolution_direction, tx_count
* | Events: tx.included (per tx, attrs: tx_id)
* |
* +~~~ consensus.validation.send [jtACCEPT thread, linked]
@@ -140,8 +140,8 @@ namespace attr {
* concept, same key, distinguished by span name (not an emitter prefix).
*/
using ::xrpl::telemetry::attr::closeResolutionMs;
using ::xrpl::telemetry::attr::closeTime;
using ::xrpl::telemetry::attr::closeTimeCorrect;
using ::xrpl::telemetry::attr::closeTimeRippleEpochS;
using ::xrpl::telemetry::attr::fullValidation;
using ::xrpl::telemetry::attr::ledgerHash;
using ::xrpl::telemetry::attr::ledgerSeq;
@@ -232,8 +232,20 @@ inline constexpr auto positionHashPrefix = makeStr("position_hash_prefix");
* "consensus_state" — domain-qualified (collides with other domains' state).
*/
inline constexpr auto consensusState = makeStr("consensus_state");
inline constexpr auto parentCloseTime = makeStr("parent_close_time");
inline constexpr auto closeTimeSelf = makeStr("close_time_self");
/**
* Close-time instants, both NetClock readings in whole seconds since the XRP
* Ledger epoch (2000-01-01T00:00:00Z) — see `closeTimeRippleEpochS` in
* SpanNames.h for why the epoch is spelled into the key.
*
* `parentCloseTimeRippleEpochS` is the previous ledger's close time;
* `closeTimeSelfRippleEpochS` is this node's own close-time vote for the round,
* so the pair shows how far the node's position sat from the ledger it built on.
*
* `closeTimeVoteBins` is not a time: it holds the number of distinct close-time
* positions seen from peers this round.
*/
inline constexpr auto parentCloseTimeRippleEpochS = makeStr("parent_close_time_ripple_epoch_s");
inline constexpr auto closeTimeSelfRippleEpochS = makeStr("close_time_self_ripple_epoch_s");
inline constexpr auto closeTimeVoteBins = makeStr("close_time_vote_bins");
inline constexpr auto resolutionDirection = makeStr("resolution_direction");
inline constexpr auto convergePercent = makeStr("converge_percent");
@@ -275,6 +287,14 @@ inline constexpr auto disputesCount = makeStr("disputes_count");
*/
inline constexpr auto proposalTrusted = makeStr("proposal_trusted");
inline constexpr auto validationTrusted = makeStr("validation_trusted");
/**
* "validation_status" — which exit the inbound validation took. Set once per
* exit, so a dropped validation (microseconds) is separable from a queued one
* (job wait plus checkValidation). Without it the span name reports two
* unrelated latency distributions and every quantile over it is meaningless.
*/
inline constexpr auto validationStatus = makeStr("validation_status");
} // namespace attr
// ===== Event names ===========================================================
@@ -340,6 +360,10 @@ inline constexpr auto closeAnomaly = makeStr("anomaly");
inline constexpr auto closeOthersClosed = makeStr("others_closed");
inline constexpr auto closeIdle = makeStr("idle");
inline constexpr auto closeNormal = makeStr("normal");
// validation_status values, one per exit of the inbound validation path.
inline constexpr auto validationQueued = makeStr("queued");
inline constexpr auto validationDroppedDiverged = makeStr("dropped_diverged");
inline constexpr auto validationDroppedLoad = makeStr("dropped_load");
} // namespace val
} // namespace xrpl::telemetry::consensus::span

View File

@@ -11,10 +11,8 @@ namespace xrpl::telemetry::buckets {
* @file HistogramBuckets.h
* @brief Explicit histogram bucket edges for xrpld's OTel instruments.
*
* One header owns every ladder so a reviewer sees all of them at once and a
* test can assert their invariants. Before this existed the edges lived as
* file-local `namespace {}` constants, unreachable from any test, and they
* drifted apart.
* One header owns every ladder, so a reviewer sees all of them together and
* a test can assert their invariants.
*
* Why a ladder is worth this much care: when a quantile falls in the `+Inf`
* bucket, Prometheus returns the *second-highest* edge, not `+Inf`. A
@@ -68,11 +66,10 @@ namespace xrpl::telemetry::buckets {
* **This list must contain every representable edge of the collector's
* spanmetrics ladder, and may extend above it.** Agreement over the shared
* range is deliberate: it lets a span-derived latency panel and a native
* histogram panel be read on the same scale. It was specified that way
* originally, then silently broken when the collector ladder alone was
* extended, which left this side capped at 5 s while spans reached 30 s and
* censored every quantile above 5 s. `check_bucket_parity.py` now enforces
* the containment -- add a collector edge, add it here too.
* histogram panel be read on the same scale. Drop an edge the collector
* carries and every quantile above it reads back as the top edge instead of
* failing. `check_bucket_parity.py` enforces the containment -- add a
* collector edge, add it here too.
*
* The sub-millisecond edges the collector carries (0.01 to 0.5 ms) are
* deliberately absent. `beast::insight::Event` rounds every duration up to
@@ -82,12 +79,12 @@ namespace xrpl::telemetry::buckets {
*
* The 60 s and 120 s edges exceed the collector's 30 s top on purpose,
* because jobs outlive spans: the updatepaths job type was measured
* averaging about 60 s, so a 30 s ceiling would censor its quantiles just
* as 5 s censors them today. All these Events share one ladder, so its
* ceiling has to cover the slowest member rather than the typical one.
* averaging about 60 s, so a 30 s ceiling would censor its quantiles. All
* these Events share one ladder, so its ceiling has to cover the slowest
* member rather than the typical one.
*
* The 2, 3 and 4 s edges resolve second-scale work that previously had to
* interpolate across a single four-second-wide bucket.
* The 2, 3 and 4 s edges resolve second-scale work, which a single
* four-second-wide bucket can only interpolate across.
*/
inline constexpr std::array kMillisecondBuckets{
1.0,
@@ -209,7 +206,7 @@ inline constexpr std::array
* @return true when the ladder is non-empty, starts at or above zero, and
* every later edge is strictly greater than its predecessor.
*/
constexpr bool
[[nodiscard]] constexpr bool
isAscendingNonNegative(std::span<double const> ladder) noexcept
{
if (ladder.empty() || ladder.front() < 0.0)
@@ -235,7 +232,7 @@ static_assert(isAscendingNonNegative(kChargeBuckets));
* @param ladder Bucket upper bounds.
* @return A vector holding the same edges in the same order.
*/
inline std::vector<double>
[[nodiscard]] inline std::vector<double>
toVector(std::span<double const> ladder)
{
return std::vector<double>(ladder.begin(), ladder.end());

View File

@@ -6,12 +6,12 @@
* Each type below holds real state when telemetry is compiled in and is an
* empty type with no-op methods when it is not. The member is declared in
* both configurations, so a class's member set and public API never differ
* between builds -- a difference that has previously made a test mock
* abstract. The compiled-out forms are empty types, so such a member costs a
* byte of padding rather than nothing. `[[no_unique_address]]` would remove
* even that, but MSVC ignores the standard spelling for ABI compatibility, so
* it is deliberately not used. What these types buy is work not being done,
* not a smaller struct.
* between builds -- a difference that leaves a test mock complete in one
* configuration and abstract in the other, where it then fails to compile.
* The compiled-out forms are empty types, so such a member costs a byte of
* padding rather than nothing. `[[no_unique_address]]` would remove even that, but MSVC ignores
* the standard spelling for ABI compatibility, so it is deliberately not
* used. What these types buy is work not being done, not a smaller struct.
*
* kEnabled ---- if constexpr ---- telemetry-only blocks
* |
@@ -174,12 +174,10 @@ public:
* @param n How much to add; defaults to 1.
*/
void
add(T const n = 1) noexcept
add([[maybe_unused]] T const n = 1) noexcept
{
#ifdef XRPL_ENABLE_TELEMETRY
value_.fetch_add(n, std::memory_order_relaxed);
#else
(void)n;
#endif
}

View File

@@ -410,8 +410,10 @@ public:
* follows-from link. Use to stitch sequential
* top-level spans (e.g. consecutive consensus
* rounds). Ignored if nullptr or invalid.
* @return An active guard, or a null guard when the category is
* disabled or hashSize is under 16.
*/
static SpanGuard
[[nodiscard]] static SpanGuard
hashSpan(
TraceCategory const cat,
std::string_view const name,
@@ -431,8 +433,10 @@ public:
* @param parentSpanId Pointer to 8 bytes of parent span ID.
* @param parentSpanSize Size of parent span ID buffer (must be 8).
* @param traceFlags Trace flags from remote context.
* @return An active guard, or a null guard when the category is
* disabled, hashSize is under 16, or parentSpanSize is not 8.
*/
static SpanGuard
[[nodiscard]] static SpanGuard
hashSpan(
TraceCategory const cat,
std::string_view const name,

View File

@@ -144,8 +144,19 @@ inline constexpr auto ledgerSeq = makeStr("ledger_seq");
/**
* Shared close-time attrs — bare names, reused by consensus and ledger.
*
* `closeTimeRippleEpochS` carries a NetClock reading: whole seconds since the
* XRP Ledger epoch (2000-01-01T00:00:00Z), never the Unix epoch. The key names
* both the unit and the epoch because neither is recoverable from the value.
* A consumer rendering it as wall-clock time must first add kEpochOffset
* (946684800 seconds, see basics/chrono.h); read as a Unix timestamp instead,
* it lands roughly 30 years early.
*
* `closeResolutionMs` is a duration, not an instant — the granularity the
* close time is rounded to. NetClock resolution is whole seconds, so this
* value is always a multiple of 1000.
*/
inline constexpr auto closeTime = makeStr("close_time");
inline constexpr auto closeTimeRippleEpochS = makeStr("close_time_ripple_epoch_s");
inline constexpr auto closeTimeCorrect = makeStr("close_time_correct");
inline constexpr auto closeResolutionMs = makeStr("close_resolution_ms");
/**

View File

@@ -149,7 +149,7 @@ public:
* Get the global Telemetry instance.
* @return Pointer to the active instance, or nullptr if not started.
*/
static Telemetry*
[[nodiscard]] static Telemetry*
getInstance()
{
return instance.load(std::memory_order_acquire);
@@ -205,9 +205,10 @@ public:
std::string nodeId;
/**
* OTLP/HTTP endpoint URL where spans are sent.
* Full OTLP/HTTP URL where spans are sent, including the signal path.
* Used verbatim: no other endpoint is derived from it.
*/
std::string exporterEndpoint = "http://localhost:4318/v1/traces";
std::string tracesEndpoint = "http://localhost:4318/v1/traces";
/**
* Whether to use TLS for the exporter connection.
@@ -318,10 +319,9 @@ public:
* @param id The node's base58-encoded public key or custom identifier.
*/
virtual void
setServiceInstanceId(std::string const& id)
setServiceInstanceId([[maybe_unused]] std::string const& id)
{
// Default no-op for NullTelemetry implementations.
(void)id;
}
/**
@@ -336,10 +336,9 @@ public:
* @param id The node's base58-encoded public key.
*/
virtual void
setNodeId(std::string const& id)
setNodeId([[maybe_unused]] std::string const& id)
{
// Default no-op for NullTelemetry implementations.
(void)id;
}
/**
@@ -405,7 +404,7 @@ public:
* @param name Tracer name used to identify the instrumentation library.
* @return A shared pointer to the Tracer.
*/
virtual opentelemetry::nostd::shared_ptr<opentelemetry::trace::Tracer>
[[nodiscard]] virtual opentelemetry::nostd::shared_ptr<opentelemetry::trace::Tracer>
getTracer(std::string_view name = kTracerName) = 0;
/**
@@ -420,7 +419,7 @@ public:
* @param name Meter name used to identify the instrumentation scope.
* @return A shared pointer to the Meter.
*/
virtual opentelemetry::nostd::shared_ptr<opentelemetry::metrics::Meter>
[[nodiscard]] virtual opentelemetry::nostd::shared_ptr<opentelemetry::metrics::Meter>
getMeter(std::string_view name = kMeterName) = 0;
/**
@@ -438,7 +437,7 @@ public:
* - kConsumer: async message receive
* @return A shared pointer to the new Span.
*/
virtual opentelemetry::nostd::shared_ptr<opentelemetry::trace::Span>
[[nodiscard]] virtual opentelemetry::nostd::shared_ptr<opentelemetry::trace::Span>
startSpan(
std::string_view name,
opentelemetry::trace::SpanKind kind = opentelemetry::trace::SpanKind::kInternal) = 0;
@@ -454,7 +453,7 @@ public:
* @param kind The span kind (defaults to kInternal).
* @return A shared pointer to the new Span.
*/
virtual opentelemetry::nostd::shared_ptr<opentelemetry::trace::Span>
[[nodiscard]] virtual opentelemetry::nostd::shared_ptr<opentelemetry::trace::Span>
startSpan(
std::string_view name,
opentelemetry::context::Context const& parentContext,
@@ -513,7 +512,7 @@ makeTelemetrySetup(
* @param networkId The network identifier from [network_id] config.
* @return "mainnet" (0), "testnet" (1), "devnet" (2), or "unknown".
*/
std::string
[[nodiscard]] std::string
networkTypeFromId(std::uint32_t networkId);
} // namespace xrpl::telemetry

View File

@@ -43,7 +43,7 @@ namespace xrpl::telemetry {
* @return An OTel Context with the extracted parent span, or an empty
* context if the protobuf fields are missing or invalid.
*/
inline opentelemetry::context::Context
[[nodiscard]] inline opentelemetry::context::Context
extractFromProtobuf(protocol::TraceContext const& proto)
{
namespace trace = opentelemetry::trace;

View File

@@ -57,7 +57,7 @@ namespace xrpl::telemetry {
* @param traceId The raw trace_id bytes from a protobuf TraceContext.
* @return true if usable as a trace identifier, false otherwise.
*/
inline bool
[[nodiscard]] inline bool
isValidTraceId(std::string const& traceId)
{
return traceId.size() == 16 && std::ranges::any_of(traceId, [](char c) { return c != 0; });
@@ -69,7 +69,7 @@ isValidTraceId(std::string const& traceId)
* @param spanId The raw span_id bytes from a protobuf TraceContext.
* @return true if usable as a span identifier, false otherwise.
*/
inline bool
[[nodiscard]] inline bool
isValidSpanId(std::string const& spanId)
{
return spanId.size() == 8 && std::ranges::any_of(spanId, [](char c) { return c != 0; });
@@ -86,7 +86,7 @@ isValidSpanId(std::string const& spanId)
* @param tc The protobuf TraceContext received from a peer.
* @return true if both ids are present and valid, false otherwise.
*/
inline bool
[[nodiscard]] inline bool
isValidTraceContext(protocol::TraceContext const& tc)
{
return tc.has_trace_id() && isValidTraceId(tc.trace_id()) && tc.has_span_id() &&

View File

@@ -4,10 +4,9 @@
*
* Compiled only when XRPL_ENABLE_TELEMETRY is defined (via CMake
* telemetry=ON). Maps beast::insight instruments to OTel SDK instruments
* created on the GLOBAL Meter published by the telemetry module. This class
* is an adapter only: it owns no export pipeline. The MeterProvider,
* PeriodicExportingMetricReader, OTLP exporter and histogram view all live in
* xrpl::telemetry::Telemetry.
* created on the GLOBAL Meter published by the telemetry module. It owns no
* export pipeline of its own: the MeterProvider, PeriodicExportingMetricReader,
* OTLP exporter and histogram view all live in xrpl::telemetry::Telemetry.
*
* When XRPL_ENABLE_TELEMETRY is not defined, OTelCollector::New() returns
* a NullCollector so the build succeeds without OTel dependencies.
@@ -43,6 +42,7 @@
#include <xrpl/beast/insight/MeterImpl.h>
#include <xrpl/beast/insight/Unit.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/beast/utility/instrumentation.h>
#include <opentelemetry/metrics/async_instruments.h>
#include <opentelemetry/metrics/meter.h>
@@ -61,9 +61,12 @@
#include <cstddef>
#include <cstdint>
#include <exception>
#include <limits>
#include <memory>
#include <mutex>
#include <ranges>
#include <string>
#include <string_view>
#include <utility>
#include <vector>
@@ -175,11 +178,10 @@ private:
* The instrument's declared unit is what selects its bucket ladder: the
* histogram views registered in Telemetry.cpp match on unit, so a `ms`
* instrument gets the millisecond ladder and a `By` instrument the byte
* ladder. The edges themselves live in xrpl/telemetry/HistogramBuckets.h --
* do not restate them here. An earlier version of this comment listed
* `[1, 5, ..., 1000, 5000] ms` as "matching the SpanMetrics connector"; that
* was true when written and silently became false when the connector's
* ladder was extended, which is why the edges now have one owner.
* ladder. The edges themselves live in xrpl/telemetry/HistogramBuckets.h,
* which is their single owner -- do not restate them here. An edge list copied
* into a comment reads as authoritative and goes stale the moment the
* collector's SpanMetrics ladder is extended, with nothing to flag the drift.
*
* Thread safety: OTel Histogram::Record() is thread-safe by specification.
*/
@@ -271,7 +273,7 @@ public:
* @brief Return the current gauge value for the OTel callback.
* @return The most recently set/incremented value.
*/
int64_t
[[nodiscard]] int64_t
currentValue() const;
OTelGaugeImpl&
@@ -284,10 +286,13 @@ public:
gaugeCallback(opentelemetry::metrics::ObserverResult result, void* state);
/**
* Create the observable instrument and register the callback, once.
* Create the observable instrument and register the callback.
*
* Called when the collector is told collection is ready, because the
* callback reads live application state.
*
* Idempotent. Arming twice would register the callback twice, so callers
* need not check; onCollectionReady() iterates a snapshot and may re-arm.
*/
void
arm();
@@ -379,7 +384,7 @@ private:
//------------------------------------------------------------------------------
/**
* @brief Main OTel Collector implementation (adapter over the global Meter).
* @brief Main OTel Collector implementation.
*
* Obtains its Meter from the GLOBAL MeterProvider owned and published by the
* telemetry module (xrpl::telemetry::Telemetry), rather than building its own
@@ -388,7 +393,7 @@ private:
*
* The metrics pipeline (MeterProvider + PeriodicExportingMetricReader + OTLP
* HTTP exporter + histogram view) lives in the telemetry module. This class is
* a thin adapter kept for beast::insight callers during deprecation.
* the thin adapter that lets beast::insight callers reach it.
*
* Class diagram:
*
@@ -444,14 +449,12 @@ public:
/**
* @brief Construct the OTel collector over the global MeterProvider.
*
* @param endpoint OTLP/HTTP metrics endpoint URL, recorded in the
* collector's startup log line. Export uses the
* endpoint configured on the global telemetry
* pipeline.
* @param prefix Label for the collector's startup log line
* (e.g. "xrpld"). Exported metric names come from
* formatName(); the service is identified by the
* service.name resource attribute.
* @param endpoint OTLP/HTTP metrics endpoint URL. Informational only:
* the global telemetry pipeline is authoritative for
* the actual export endpoint. Used only in the startup
* log line.
* @param prefix Metric-name prefix. Not applied to metric names;
* used only in the startup log line.
* @param instanceId Value for the service.instance.id resource attribute.
* When empty, the attribute is omitted.
* @param serviceName Value for the service.name resource attribute.
@@ -555,7 +558,7 @@ public:
* @brief The shared Meter, for gauges creating their instrument in arm().
* @return The Meter this collector resolved at construction.
*/
opentelemetry::nostd::shared_ptr<metrics_api::Meter> const&
[[nodiscard]] opentelemetry::nostd::shared_ptr<metrics_api::Meter> const&
otelMeter() const;
/**
@@ -570,8 +573,8 @@ public:
* @param name Raw metric name from beast::insight callers.
* @return Fully-qualified metric name.
*/
static std::string
formatName(std::string const& name);
[[nodiscard]] static std::string
formatName(std::string_view name);
private:
/**
@@ -655,8 +658,10 @@ OTelCounterImpl::OTelCounterImpl(
void
OTelCounterImpl::increment(value_type amount)
{
// OTel counters require non-negative values. beast::insight CounterImpl
// uses int64_t, so clamp negative values to 0 and cast to uint64_t.
// OTel counters take unsigned deltas only. Assert to catch a decrementing
// caller; skip the Add so a release build under-counts instead of wrapping.
XRPL_ASSERT(
amount >= 0, "beast::insight::detail::OTelCounterImpl::increment : non-negative amount");
if (amount > 0)
counter_->Add(static_cast<uint64_t>(amount));
}
@@ -743,20 +748,25 @@ OTelGaugeImpl::~OTelGaugeImpl()
void
OTelGaugeImpl::set(value_type value)
{
value_.store(static_cast<int64_t>(value), std::memory_order_relaxed);
// value_type is uint64_t, the gauge reports int64_t. Clamp instead of
// wrapping to a negative, which increment() would then floor to 0.
constexpr auto kMax = static_cast<value_type>(std::numeric_limits<int64_t>::max());
value_.store(static_cast<int64_t>(std::min(value, kMax)), std::memory_order_relaxed);
}
void
OTelGaugeImpl::increment(difference_type amount)
{
// Use compare-exchange loop to safely clamp to [0, MAX].
// Saturate in [0, INT64_MAX]. Signed overflow is UB, so check the headroom
// before adding. A negative amount cannot underflow: current is never
// negative, so the lowest sum is 0 + INT64_MIN.
constexpr auto kMax = std::numeric_limits<int64_t>::max();
int64_t current = value_.load(std::memory_order_relaxed);
int64_t desired = 0;
do
{
desired = current + amount;
// Clamp to 0 on underflow.
desired = std::max(desired, int64_t{0});
desired =
(amount > 0 && current > kMax - amount) ? kMax : std::max(current + amount, int64_t{0});
} while (!value_.compare_exchange_weak(current, desired, std::memory_order_relaxed));
}
@@ -790,23 +800,19 @@ OTelMeterImpl::increment(value_type amount)
OTelCollectorImp::OTelCollectorImp(
std::string const& endpoint,
std::string prefix,
std::string const& instanceId,
std::string const& serviceName,
std::string const& networkType,
// instanceId/serviceName/networkType are accepted so the New() signature
// stays uniform for callers, but they are not read here: the telemetry
// module owns the resource attributes for the shared metrics pipeline.
[[maybe_unused]] std::string const& instanceId,
[[maybe_unused]] std::string const& serviceName,
[[maybe_unused]] std::string const& networkType,
Journal journal)
: journal_(journal), prefix_(std::move(prefix))
{
// instanceId/serviceName/networkType are accepted but unused here: the
// telemetry module owns the resource attributes for the shared metrics
// pipeline, so setting them from this collector would have no effect.
(void)instanceId;
(void)serviceName;
(void)networkType;
if (journal_.info())
{
// endpoint is logged for diagnostics only: the global telemetry
// pipeline owns the exporter that actually sends the metrics.
// endpoint is informational: the global telemetry pipeline owns the
// real exporter. It is logged here purely as a startup diagnostic.
journal_.info() << "OTelCollector starting: endpoint=" << endpoint << " prefix=" << prefix_;
}
@@ -815,14 +821,10 @@ OTelCollectorImp::OTelCollectorImp(
// periodic reader, histogram view, resource attributes) and registers it
// via metrics::Provider::SetMeterProvider() during start(). beast metrics
// ride that shared pipeline, so both direct-API and beast-sourced metrics
// export under one resource identity.
//
// The name/version literals MUST match the telemetry module's kMeterName
// ("xrpld") and kMeterVersion ("1.0.0"). They are written as literals (not
// referenced from Telemetry.h) because beast/insight sits below the
// telemetry module in the layering and cannot include its header.
// export under one resource identity. The scope must match the telemetry
// module's; see kOTelMeterName in the header.
otelMeter_ = metrics_api::Provider::GetMeterProvider()->GetMeter(
std::string{"xrpld"}, std::string{"1.0.0"});
std::string{kOTelMeterName}, std::string{kOTelMeterVersion});
if (journal_.info())
{
@@ -832,12 +834,8 @@ OTelCollectorImp::OTelCollectorImp(
OTelCollectorImp::~OTelCollectorImp()
{
if (journal_.info())
{
journal_.info() << "OTelCollector shutting down";
}
// No pipeline teardown here: the telemetry module owns the global
// MeterProvider lifecycle (ForceFlush/Shutdown happen in Telemetry::stop()).
// Nothing to tear down: the telemetry module owns the global MeterProvider,
// so ForceFlush and Shutdown happen in Telemetry::stop().
if (journal_.info())
{
journal_.info() << "OTelCollector stopped";
@@ -1003,25 +1001,16 @@ OTelCollectorImp::otelMeter() const
}
std::string
OTelCollectorImp::formatName(std::string const& name)
OTelCollectorImp::formatName(std::string_view name)
{
// Produce a lowercase, Prometheus-compatible metric name: dots and
// spaces become underscores. Service identity travels in the
// service.name resource attribute, not in the metric name.
std::string result;
result.reserve(name.size());
for (char const c : name)
{
if (c == '.' || c == ' ')
{
result += '_';
}
else
{
result += static_cast<char>(std::tolower(static_cast<unsigned char>(c)));
}
}
return result;
// Lowercase, with '.' and ' ' mapped to '_'. No prefix: the service.name
// resource attribute identifies the service.
return name | std::views::transform([](char c) {
return (c == '.' || c == ' ')
? '_'
: static_cast<char>(std::tolower(static_cast<unsigned char>(c)));
}) |
std::ranges::to<std::string>();
}
} // namespace detail

View File

@@ -167,6 +167,9 @@ private:
std::string name_;
GaugeImpl::value_type lastValue_{0};
GaugeImpl::value_type value_{0};
// Start dirty so the initial value (0) is emitted on the first flush.
// Without this, gauges whose value never changes from 0 would never
// appear in downstream metric stores (e.g. Prometheus via StatsD).
bool dirty_{true};
};
@@ -599,9 +602,6 @@ StatsDEventImpl::doNotify(EventImpl::value_type const& value)
StatsDGaugeImpl::StatsDGaugeImpl(std::string name, std::shared_ptr<StatsDCollectorImp> impl)
: impl_(std::move(impl)), name_(std::move(name))
{
// Start dirty so the initial value (0) is emitted on the first flush.
// Without this, gauges whose value never changes from 0 would never
// appear in downstream metric stores (e.g. Prometheus via StatsD).
impl_->add(*this);
}

View File

@@ -116,7 +116,7 @@ public:
}
#ifdef XRPL_ENABLE_TELEMETRY
opentelemetry::nostd::shared_ptr<opentelemetry::trace::Tracer>
[[nodiscard]] opentelemetry::nostd::shared_ptr<opentelemetry::trace::Tracer>
getTracer(std::string_view) override
{
static auto noopTracer = opentelemetry::nostd::shared_ptr<opentelemetry::trace::Tracer>(
@@ -124,14 +124,14 @@ public:
return noopTracer;
}
opentelemetry::nostd::shared_ptr<opentelemetry::trace::Span>
[[nodiscard]] opentelemetry::nostd::shared_ptr<opentelemetry::trace::Span>
startSpan(std::string_view, opentelemetry::trace::SpanKind) override
{
return opentelemetry::nostd::shared_ptr<opentelemetry::trace::Span>(
new opentelemetry::trace::NoopSpan(nullptr));
}
opentelemetry::nostd::shared_ptr<opentelemetry::trace::Span>
[[nodiscard]] opentelemetry::nostd::shared_ptr<opentelemetry::trace::Span>
startSpan(
std::string_view,
opentelemetry::context::Context const&,

View File

@@ -19,6 +19,7 @@
#include <xrpl/telemetry/Telemetry.h>
#include <xrpl/basics/Log.h>
#include <xrpl/beast/insight/OTelCollector.h>
#include <xrpl/beast/insight/Unit.h>
#include <xrpl/beast/utility/Journal.h>
#include <xrpl/telemetry/CoroAwareContextStorage.h>
@@ -77,6 +78,24 @@
namespace xrpl::telemetry {
// beast cannot include this header, so it duplicates the meter scope. Fail the
// build if the copies drift: instruments would land off the views' scope.
static_assert(kMeterName == beast::insight::kOTelMeterName);
static_assert(kMeterVersion == beast::insight::kOTelMeterVersion);
/**
* OTLP/HTTP path per signal, appended by signalEndpoint().
*/
constexpr std::string_view kTracesPath{"/v1/traces"};
constexpr std::string_view kMetricsPath{"/v1/metrics"};
/**
* Metric export cadence. The interval matches the 1 s scrape the dashboards
* assume; the timeout bounds a stalled collector.
*/
constexpr auto kMetricExportInterval = std::chrono::milliseconds{1000};
constexpr auto kMetricExportTimeout = std::chrono::milliseconds{500};
namespace {
namespace trace_api = opentelemetry::trace;
@@ -244,7 +263,7 @@ public:
return setup_.consensusTraceStrategy;
}
opentelemetry::nostd::shared_ptr<trace_api::Tracer>
[[nodiscard]] opentelemetry::nostd::shared_ptr<trace_api::Tracer>
getTracer(std::string_view) override
{
static auto noopTracer =
@@ -252,7 +271,7 @@ public:
return noopTracer;
}
opentelemetry::nostd::shared_ptr<metrics_api::Meter>
[[nodiscard]] opentelemetry::nostd::shared_ptr<metrics_api::Meter>
getMeter(std::string_view name) override
{
// Serve a meter from a process-wide noop provider, mirroring the
@@ -262,13 +281,13 @@ public:
return noopProvider->GetMeter(std::string(name), std::string(kMeterVersion));
}
opentelemetry::nostd::shared_ptr<trace_api::Span>
[[nodiscard]] opentelemetry::nostd::shared_ptr<trace_api::Span>
startSpan(std::string_view, trace_api::SpanKind) override
{
return opentelemetry::nostd::shared_ptr<trace_api::Span>(new trace_api::NoopSpan(nullptr));
}
opentelemetry::nostd::shared_ptr<trace_api::Span>
[[nodiscard]] opentelemetry::nostd::shared_ptr<trace_api::Span>
startSpan(std::string_view, opentelemetry::context::Context const&, trace_api::SpanKind)
override
{
@@ -370,12 +389,12 @@ public:
void
start() override
{
JLOG(journal_.info()) << "Telemetry starting: endpoint=" << setup_.exporterEndpoint
JLOG(journal_.info()) << "Telemetry starting: traces_endpoint=" << setup_.tracesEndpoint
<< " sampling=" << setup_.samplingRatio;
// Configure OTLP HTTP exporter
otlp_http::OtlpHttpExporterOptions exporterOpts;
exporterOpts.url = setup_.exporterEndpoint;
exporterOpts.url = signalEndpoint(setup_.tracesEndpoint, kTracesPath);
if (setup_.useTls)
{
exporterOpts.ssl_ca_cert_path = setup_.tlsCertPath;
@@ -518,7 +537,7 @@ public:
// Derive the metrics endpoint from the trace endpoint by swapping
// the trailing "/v1/traces" path for "/v1/metrics". Any other URL
// shape is used as-is.
std::string metricsEndpoint = setup_.exporterEndpoint;
std::string metricsEndpoint = setup_.tracesEndpoint;
constexpr std::string_view tracesPath{"/v1/traces"};
if (metricsEndpoint.ends_with(tracesPath))
{
@@ -690,7 +709,7 @@ public:
return setup_.consensusTraceStrategy;
}
opentelemetry::nostd::shared_ptr<trace_api::Tracer>
[[nodiscard]] opentelemetry::nostd::shared_ptr<trace_api::Tracer>
getTracer(std::string_view name = kTracerName) override
{
if (!sdkProvider_)
@@ -700,7 +719,7 @@ public:
return sdkProvider_->GetTracer(std::string(name));
}
opentelemetry::nostd::shared_ptr<metrics_api::Meter>
[[nodiscard]] opentelemetry::nostd::shared_ptr<metrics_api::Meter>
getMeter(std::string_view name = kMeterName) override
{
if (!meterProvider_)
@@ -711,7 +730,7 @@ public:
return meterProvider_->GetMeter(std::string(name), std::string(kMeterVersion));
}
opentelemetry::nostd::shared_ptr<trace_api::Span>
[[nodiscard]] opentelemetry::nostd::shared_ptr<trace_api::Span>
startSpan(std::string_view name, trace_api::SpanKind kind) override
{
auto tracer = getTracer();
@@ -720,7 +739,7 @@ public:
return tracer->StartSpan(std::string(name), opts);
}
opentelemetry::nostd::shared_ptr<trace_api::Span>
[[nodiscard]] opentelemetry::nostd::shared_ptr<trace_api::Span>
startSpan(
std::string_view name,
opentelemetry::context::Context const& parentContext,

View File

@@ -35,7 +35,7 @@ namespace key {
constexpr char const* enabled = "enabled";
constexpr char const* serviceName = "service_name";
constexpr char const* serviceInstanceId = "service_instance_id";
constexpr char const* endpoint = "endpoint";
constexpr char const* tracesEndpoint = "traces_endpoint";
constexpr char const* useTls = "use_tls";
constexpr char const* tlsCaCert = "tls_ca_cert";
constexpr char const* tlsClientCert = "tls_client_cert";
@@ -60,7 +60,7 @@ constexpr char const* traceLedger = "trace_ledger";
*/
namespace dflt {
constexpr char const* serviceName = "xrpld";
constexpr char const* endpoint = "http://localhost:4318/v1/traces";
constexpr char const* tracesEndpoint = "http://localhost:4318/v1/traces";
constexpr std::uint32_t batchSize = 512u;
constexpr std::uint32_t batchDelayMs = 5000u;
constexpr std::uint32_t maxQueueSize = 2048u;
@@ -136,7 +136,7 @@ makeTelemetrySetup(
setup.serviceVersion = version;
setup.serviceInstanceId = section.valueOr<std::string>(key::serviceInstanceId, nodePublicKey);
setup.exporterEndpoint = section.valueOr<std::string>(key::endpoint, dflt::endpoint);
setup.tracesEndpoint = section.valueOr<std::string>(key::tracesEndpoint, dflt::tracesEndpoint);
setup.useTls = section.valueOr<int>(key::useTls, 0) != 0;
setup.tlsCertPath = section.valueOr<std::string>(key::tlsCaCert, "");

View File

@@ -49,6 +49,7 @@
#include <algorithm>
#include <cstddef>
#include <cstdint>
#include <exception>
#include <functional>
#include <map>
#include <optional>
@@ -1624,126 +1625,140 @@ Transactor::operator()()
trapTransaction(*trap);
}
auto result = ctx_.preclaimResult;
if (isTesSuccess(result))
result = apply();
// No transaction can return temUNKNOWN from apply,
// and it can't be passed in from a preclaim.
XRPL_ASSERT(result != temUNKNOWN, "xrpl::Transactor::operator() : result is not temUNKNOWN");
if (auto stream = j_.trace())
stream << "preclaim result: " << transToken(result);
auto fee = ctx_.tx.getFieldAmount(sfFee).xrp();
bool const canApply = std::invoke([&result, &fee, this] {
bool canApplyTmp = isTesSuccess(result);
if (ctx_.size() > kOversizeMetaDataCap)
result = tecOVERSIZE;
if (isTecClaim(result) && ((view().flags() & TapFailHard) != 0u))
{
// If the TapFailHard flag is set, a tec result
// must not do anything
ctx_.discard();
canApplyTmp = false;
}
else if (
(result == tecOVERSIZE) || (result == tecKILLED) || (result == tecINCOMPLETE) ||
(result == tecEXPIRED) || (isTecClaimHardFail(result, view().flags())))
{
// This is and must remain the only place where `canApplyTmp` can change from false to
// true. Changing from true to false is no problem.
std::tie(result, fee, canApplyTmp) = processPersistentChanges(result, fee);
}
return canApplyTmp;
});
// Every exit from this function funnels through here, so this is also where
// the apply span records its outcome: each return path reports the engine
// result and whether the transaction was applied.
auto const logger = [this, &span](
TER result,
bool canApply,
std::optional<TxMeta>&& metadata = std::nullopt) -> ApplyResult {
JLOG(j_.trace()) << (canApply ? "applied " : "not applied ") << transToken(result);
// Also guarded: transToken() is a lookup returning a string, and this
// funnel runs on every exit path.
if (span)
{
span.setAttribute(
telemetry::tx_apply_span::attr::terResult, transToken(result).c_str());
span.setAttribute(telemetry::tx_apply_span::attr::applied, canApply);
// Mark the span as errored when the transaction was not applied or
// the engine result is not a success, so failed applies surface in
// span-status error counts alongside preflight and preclaim.
if (!canApply || !isTesSuccess(result))
span.setError(transToken(result));
}
return {result, canApply, std::move(metadata)};
};
if (!canApply)
return logger(result, canApply);
// First invariant pass: both protocol and transaction-specific
// checks run against the transaction's tentative outcome. If it
// does not return tecINVARIANT_FAILED, we can proceed to apply the
// tx.
result = checkInvariants(result, fee, InvariantScope::Full);
if (result == tecINVARIANT_FAILED)
try
{
// Fee-claim reset: roll the transaction's effects back so that
// only the fee deduction remains. This is the reset referenced
// by InvariantScope::ProtocolOnly.
auto const resetResult = reset(fee);
if (!isTesSuccess(resetResult.first))
result = resetResult.first;
auto result = ctx_.preclaimResult;
if (isTesSuccess(result))
result = apply();
fee = resetResult.second;
// No transaction can return temUNKNOWN from apply,
// and it can't be passed in from a preclaim.
XRPL_ASSERT(
result != temUNKNOWN, "xrpl::Transactor::operator() : result is not temUNKNOWN");
// Re-check invariants against the post-reset (fee-claim only)
// state. The transaction's effects are gone, so the
// transaction-specific invariants no longer apply and only the
// protocol invariants are re-run. A failure here escalates to
// tefINVARIANT_FAILED and excludes the tx from the ledger.
if (isTesSuccess(result) || isTecClaim(result))
result = checkInvariants(result, fee, InvariantScope::ProtocolOnly);
if (auto stream = j_.trace())
stream << "preclaim result: " << transToken(result);
auto fee = ctx_.tx.getFieldAmount(sfFee).xrp();
bool const canApply = std::invoke([&result, &fee, this] {
bool canApplyTmp = isTesSuccess(result);
if (ctx_.size() > kOversizeMetaDataCap)
result = tecOVERSIZE;
if (isTecClaim(result) && ((view().flags() & TapFailHard) != 0u))
{
// If the TapFailHard flag is set, a tec result
// must not do anything
ctx_.discard();
canApplyTmp = false;
}
else if (
(result == tecOVERSIZE) || (result == tecKILLED) || (result == tecINCOMPLETE) ||
(result == tecEXPIRED) || (isTecClaimHardFail(result, view().flags())))
{
// This is and must remain the only place where `canApplyTmp` can change from false
// to true. Changing from true to false is no problem.
std::tie(result, fee, canApplyTmp) = processPersistentChanges(result, fee);
}
return canApplyTmp;
});
// Each return path funnels through here, so this is also where the apply
// span records its outcome: the engine result and whether the transaction
// was applied. A throw bypasses it and ends the span with no outcome.
auto const logger = [this, &span](
TER result,
bool canApply,
std::optional<TxMeta>&& metadata = std::nullopt) -> ApplyResult {
JLOG(j_.trace()) << (canApply ? "applied " : "not applied ") << transToken(result);
// Also guarded: transToken() is a lookup returning a string, and this
// funnel runs on every return path.
if (span)
{
span.setAttribute(
telemetry::tx_apply_span::attr::terResult, transToken(result).c_str());
span.setAttribute(telemetry::tx_apply_span::attr::applied, canApply);
// Mark the span as errored when the engine result is not a success,
// so failed applies surface alongside preflight and preclaim. Not
// keyed on `canApply`: a dry run reports tesSUCCESS with canApply
// false, and that is not a failure.
if (!isTesSuccess(result))
span.setError(transToken(result));
}
return {result, canApply, std::move(metadata)};
};
if (!canApply)
return logger(result, canApply);
// First invariant pass: both protocol and transaction-specific
// checks run against the transaction's tentative outcome. If it
// does not return tecINVARIANT_FAILED, we can proceed to apply the
// tx.
result = checkInvariants(result, fee, InvariantScope::Full);
if (result == tecINVARIANT_FAILED)
{
// Fee-claim reset: roll the transaction's effects back so that
// only the fee deduction remains. This is the reset referenced
// by InvariantScope::ProtocolOnly.
auto const resetResult = reset(fee);
if (!isTesSuccess(resetResult.first))
result = resetResult.first;
fee = resetResult.second;
// Re-check invariants against the post-reset (fee-claim only)
// state. The transaction's effects are gone, so the
// transaction-specific invariants no longer apply and only the
// protocol invariants are re-run. A failure here escalates to
// tefINVARIANT_FAILED and excludes the tx from the ledger.
if (isTesSuccess(result) || isTecClaim(result))
result = checkInvariants(result, fee, InvariantScope::ProtocolOnly);
}
// We ran through the invariant checker, which can, in some cases,
// return a tef error code. Don't apply the transaction in that case.
if (!isTecClaim(result) && !isTesSuccess(result))
return logger(result, false);
std::optional<TxMeta> metadata;
// Transaction succeeded fully or (retries are not allowed and the
// transaction could claim a fee)
// The transactor and invariant checkers guarantee that this will
// *never* trigger but if it, somehow, happens, don't allow a tx
// that charges a negative fee.
if (fee < beast::kZero)
Throw<std::logic_error>("fee charged is negative!");
// Charge whatever fee they specified. The fee has already been
// deducted from the balance of the account that issued the
// transaction. We just need to account for it in the ledger
// header.
if (!view().open() && fee != beast::kZero)
ctx_.destroyXRP(fee);
// Once we call apply, we will no longer be able to look at view()
metadata = ctx_.apply(result);
if ((ctx_.flags() & TapDryRun) != 0u)
return logger(result, false, std::move(metadata));
return logger(result, canApply, std::move(metadata));
}
catch (std::exception const& e)
{
// The caller's doApply() maps this to tefEXCEPTION. Record it on the
// span before unwinding so per-stage error counts include exceptions.
span.setAttribute(
telemetry::tx_apply_span::attr::terResult, transToken(tefEXCEPTION).c_str());
span.recordException(e);
throw;
}
// We ran through the invariant checker, which can, in some cases,
// return a tef error code. Don't apply the transaction in that case.
if (!isTecClaim(result) && !isTesSuccess(result))
return logger(result, false);
std::optional<TxMeta> metadata;
// Transaction succeeded fully or (retries are not allowed and the
// transaction could claim a fee)
// The transactor and invariant checkers guarantee that this will
// *never* trigger but if it, somehow, happens, don't allow a tx
// that charges a negative fee.
if (fee < beast::kZero)
Throw<std::logic_error>("fee charged is negative!");
// Charge whatever fee they specified. The fee has already been
// deducted from the balance of the account that issued the
// transaction. We just need to account for it in the ledger
// header.
if (!view().open() && fee != beast::kZero)
ctx_.destroyXRP(fee);
// Once we call apply, we will no longer be able to look at view()
metadata = ctx_.apply(result);
if ((ctx_.flags() & TapDryRun) != 0u)
return logger(result, false, std::move(metadata));
return logger(result, canApply, std::move(metadata));
}
} // namespace xrpl

View File

@@ -314,6 +314,10 @@ invokePreclaim(PreclaimContext const& ctx)
{
span.setAttribute(
telemetry::tx_apply_span::attr::terResult, transToken(preclaimTer).c_str());
// Mark the span as errored when preclaim rejects the transaction so
// failed stages surface in span-status error counts.
if (!isTesSuccess(preclaimTer))
span.setError(transToken(preclaimTer));
}
return preclaimTer;
}

View File

@@ -723,9 +723,9 @@ class TMGetObjectByHash_test : public beast::unit_test::Suite
* One pricing case: inputs, the derived expectation, and the literal.
*
* Both expectations are kept. `derived` is written from the Tuning
* constants so a deliberate re-pricing needs one edit; `literal` is the
* number as of this branch so a re-pricing cannot pass unnoticed by
* being self-consistently wrong.
* constants so a deliberate re-pricing needs one edit; `literal` pins the
* number those constants currently produce, so a re-pricing cannot pass
* unnoticed by being self-consistently wrong.
*/
struct FeeCase
{

View File

@@ -1,7 +1,7 @@
/**
* GTest unit tests for MetricsRegistry.
*
* Three independent groups, split by what they can link:
* Four independent groups, split by what they can link:
*
* 1. sanitiseHandler() — the `handler` label sanitiser. Runs in **both**
* builds. sanitiseHandler() is a public static constexpr defined inline
@@ -14,7 +14,13 @@
* on the nodestore_state gauge. Also a public static constexpr inline,
* so it runs in both builds for the same reason.
*
* 3. The no-op / telemetry-disabled path — construction, the two-phase
* 3. parseLedgerRange() — reads one segment of the complete-ledger range
* string the complete_ledgers gauge publishes. A public static inline, so
* it runs in both builds for the same reason. The last case drives the
* real producer, xrpl::to_string(RangeSet), rather than restating its
* format.
*
* 4. The no-op / telemetry-disabled path — construction, the two-phase
* start() / startAsyncGauges() / stop() lifecycle, and the synchronous
* record*() methods. Guarded, because when XRPL_ENABLE_TELEMETRY is
* defined MetricsRegistry.cpp is not compiled into this binary (see
@@ -24,6 +30,8 @@
#include <xrpld/telemetry/MetricsRegistry.h>
#include <xrpl/basics/RangeSet.h>
#include <gtest/gtest.h>
#include <algorithm>
@@ -33,7 +41,10 @@
#include <limits>
#include <optional>
#include <set>
#include <string>
#include <string_view>
#include <utility>
#include <vector>
namespace {
@@ -204,9 +215,9 @@ allFoldToOther()
static_assert(allPassThroughUnchanged());
static_assert(allFoldToOther());
// The verified size of the pass-through set as of this branch: 43 all-letter
// job-name literals. Pinned so that adding or removing a job name without
// revisiting the label-cardinality budget fails the build here.
// The pass-through set is every all-letter job-name literal in the tree: 43
// of them. Pinned so that adding or removing a job name without revisiting
// the label-cardinality budget fails the build here.
static_assert(kPassThroughHandlers.size() == 43);
/**
@@ -417,6 +428,155 @@ TEST(MetricsRegistryScaledMean, default_scale_is_one)
EXPECT_EQ(Registry::scaledMean(360, 8), 45);
}
namespace {
/**
* Segments the producer can emit, paired with the range each denotes.
*
* Both shapes come from xrpl::to_string(ClosedInterval): `first-last`, and a
* bare number when first equals last.
*/
constexpr std::array<std::pair<std::string_view, std::pair<std::uint32_t, std::uint32_t>>, 6>
kProducibleSegments{{
{"32570-50000", {32570, 50000}},
{"50005-75891421", {50005, 75891421}},
{"0-1", {0, 1}},
{"5000", {5000, 5000}},
{"0", {0, 0}},
{"1-1", {1, 1}},
}};
/**
* Segments no producer emits and the parser must refuse.
*
* `5-6 ` and `0x10` are the two that pin the consumed-everything check: they
* start with digits from_chars can read, so only the `ptr != end` test rejects
* them. from_chars refuses the other ten on its own. Keep those two.
*/
constexpr std::array<std::string_view, 12> kUnreadableSegments{
"",
"-",
"-5",
"5-",
"abc",
"5-a",
"a-5",
"5--6",
" 5-6",
"5-6 ",
"+5",
"0x10",
};
} // namespace
TEST(MetricsRegistryParseLedgerRange, dashed_segment_yields_both_bounds)
{
// The ordinary shape. Both bounds must survive, because the gauge publishes
// them as separate `start` and `end` series and a dashboard subtracts them.
EXPECT_EQ(
Registry::parseLedgerRange("32570-50000"),
(std::pair<std::uint32_t, std::uint32_t>{32570, 50000}));
EXPECT_EQ(
Registry::parseLedgerRange("50005-75891421"),
(std::pair<std::uint32_t, std::uint32_t>{50005, 75891421}));
}
TEST(MetricsRegistryParseLedgerRange, single_ledger_segment_is_a_range_not_a_reject)
{
// A node holding exactly one complete ledger renders as a bare number, so
// treating a dashless segment as malformed reports nothing at all for that
// node -- the reading an operator most needs while a node is catching up.
auto const one = Registry::parseLedgerRange("5000");
ASSERT_TRUE(one.has_value());
EXPECT_EQ(one->first, 5000u);
EXPECT_EQ(one->second, 5000u);
// Cause, not just state: acceptance is specific to an all-digit segment.
// These two prove the dashless branch is not simply accepting everything,
// so the test above would still fail if the guard were removed outright.
EXPECT_FALSE(Registry::parseLedgerRange("abc").has_value());
EXPECT_FALSE(Registry::parseLedgerRange("5-").has_value());
}
TEST(MetricsRegistryParseLedgerRange, every_producible_segment_parses_exactly)
{
for (auto const& [segment, expected] : kProducibleSegments)
{
auto const parsed = Registry::parseLedgerRange(segment);
ASSERT_TRUE(parsed.has_value()) << "rejected a producible segment: " << segment;
EXPECT_EQ(*parsed, expected) << "wrong bounds for segment: " << segment;
}
}
TEST(MetricsRegistryParseLedgerRange, unreadable_segments_are_refused)
{
for (auto const segment : kUnreadableSegments)
{
EXPECT_FALSE(Registry::parseLedgerRange(segment).has_value())
<< "accepted an unreadable segment: [" << segment << "]";
}
}
TEST(MetricsRegistryParseLedgerRange, bounds_are_exact_at_the_sequence_limits)
{
// The width comes from the function's own return type, so widening the
// sequence cannot leave this asserting against a stale boundary.
using Seq = decltype(Registry::parseLedgerRange("0"))::value_type::first_type;
constexpr auto kMaxSeq = std::numeric_limits<Seq>::max();
auto const maxText = std::to_string(kMaxSeq);
auto const atLimit = Registry::parseLedgerRange(maxText);
ASSERT_TRUE(atLimit.has_value()) << "rejected the largest representable sequence";
EXPECT_EQ(atLimit->first, kMaxSeq);
EXPECT_EQ(atLimit->second, kMaxSeq);
// One past the limit does not wrap to a small, believable sequence.
auto const pastLimit = std::to_string(static_cast<std::uint64_t>(kMaxSeq) + 1);
EXPECT_FALSE(Registry::parseLedgerRange(pastLimit).has_value())
<< "overflowed instead of refusing: " << pastLimit;
}
TEST(MetricsRegistryParseLedgerRange, reads_back_what_the_real_producer_wrote)
{
// Drives the actual producer rather than a restatement of its format, so a
// change to to_string() fails here instead of silently changing what the
// gauge reports. The middle interval is one ledger wide on purpose: that is
// the shape that renders without a dash.
xrpl::RangeSet<std::uint32_t> ledgers;
ledgers.insert(xrpl::range<std::uint32_t>(32570, 50000));
ledgers.insert(xrpl::range<std::uint32_t>(60000, 60000));
ledgers.insert(xrpl::range<std::uint32_t>(70000, 75891421));
auto const rendered = xrpl::to_string(ledgers);
std::vector<std::pair<std::uint32_t, std::uint32_t>> recovered;
std::string_view rest{rendered};
while (!rest.empty())
{
auto const comma = rest.find(',');
auto const segment = rest.substr(0, comma);
auto const parsed = Registry::parseLedgerRange(segment);
ASSERT_TRUE(parsed.has_value()) << "producer emitted a segment the parser refuses: ["
<< segment << "] from " << rendered;
recovered.push_back(*parsed);
rest = (comma == std::string_view::npos) ? std::string_view{} : rest.substr(comma + 1);
}
std::vector<std::pair<std::uint32_t, std::uint32_t>> const expected{
{32570, 50000},
{60000, 60000},
{70000, 75891421},
};
EXPECT_EQ(recovered, expected) << "rendered as: " << rendered;
// Cause, not just state: every interval survived the round trip, so none
// was dropped and no later index shifted down to fill a gap.
EXPECT_EQ(recovered.size(), ledgers.iterative_size());
}
// When telemetry is globally enabled, MetricsRegistry.cpp requires xrpld
// link dependencies we cannot satisfy in a standalone GTest binary.
#ifndef XRPL_ENABLE_TELEMETRY

View File

@@ -99,7 +99,7 @@ TEST(SpanGuardFactory, consensus_close_time_attributes)
auto span = telemetry::SpanGuard::span(
telemetry::TraceCategory::Consensus, telemetry::seg::consensus, "accept.apply");
span.setAttribute("ledger_seq", static_cast<int64_t>(42));
span.setAttribute("close_time", static_cast<int64_t>(780000000));
span.setAttribute("close_time_ripple_epoch_s", static_cast<int64_t>(780000000));
span.setAttribute("close_time_correct", true);
span.setAttribute("close_resolution_ms", static_cast<int64_t>(30000));
span.setAttribute("consensus_state", std::string("finished"));

View File

@@ -116,7 +116,7 @@ TEST(TelemetryConfig, setup_defaults)
EXPECT_TRUE(s.serviceVersion.empty());
EXPECT_TRUE(s.serviceInstanceId.empty());
EXPECT_TRUE(s.nodeId.empty());
EXPECT_EQ(s.exporterEndpoint, "http://localhost:4318/v1/traces");
EXPECT_EQ(s.tracesEndpoint, "http://localhost:4318/v1/traces");
EXPECT_FALSE(s.useTls);
EXPECT_TRUE(s.tlsCertPath.empty());
EXPECT_DOUBLE_EQ(s.samplingRatio, 1.0);
@@ -160,7 +160,7 @@ TEST(TelemetryConfig, parse_full_section)
section.set("service_name", "my-rippled");
section.set("service_instance_id", "custom-id");
section.set("exporter", "otlp_http");
section.set("endpoint", "http://collector:4318/v1/traces");
section.set("traces_endpoint", "http://collector:4318/v1/traces");
section.set("use_tls", "1");
section.set("tls_ca_cert", caCert);
section.set("batch_size", "256");
@@ -177,7 +177,7 @@ TEST(TelemetryConfig, parse_full_section)
EXPECT_TRUE(setup.enabled);
EXPECT_EQ(setup.serviceName, "my-rippled");
EXPECT_EQ(setup.serviceInstanceId, "custom-id");
EXPECT_EQ(setup.exporterEndpoint, "http://collector:4318/v1/traces");
EXPECT_EQ(setup.tracesEndpoint, "http://collector:4318/v1/traces");
EXPECT_TRUE(setup.useTls);
EXPECT_EQ(setup.tlsCertPath, caCert);
EXPECT_EQ(setup.batchSize, 256u);

View File

@@ -637,7 +637,8 @@ RCLConsensus::Adaptor::doAccept(
: telemetry::SpanGuard::childSpan(cs::acceptApply, roundSpanContext_);
doAcceptSpan.setAttribute(cs::attr::ledgerSeq, static_cast<int64_t>(prevLedger.seq()) + 1);
doAcceptSpan.setAttribute(
cs::attr::closeTime, static_cast<int64_t>(consensusCloseTime.time_since_epoch().count()));
cs::attr::closeTimeRippleEpochS,
static_cast<int64_t>(consensusCloseTime.time_since_epoch().count()));
doAcceptSpan.setAttribute(cs::attr::closeTimeCorrect, closeTimeCorrect);
doAcceptSpan.setAttribute(
cs::attr::closeResolutionMs,
@@ -650,10 +651,10 @@ RCLConsensus::Adaptor::doAccept(
doAcceptSpan.setAttribute(
cs::attr::roundTimeMs, static_cast<int64_t>(result.roundTime.read().count()));
doAcceptSpan.setAttribute(
cs::attr::parentCloseTime,
cs::attr::parentCloseTimeRippleEpochS,
static_cast<int64_t>(prevLedger.closeTime().time_since_epoch().count()));
doAcceptSpan.setAttribute(
cs::attr::closeTimeSelf,
cs::attr::closeTimeSelfRippleEpochS,
static_cast<int64_t>(rawCloseTimes.self.time_since_epoch().count()));
doAcceptSpan.setAttribute(
cs::attr::closeTimeVoteBins, static_cast<int64_t>(rawCloseTimes.peers.size()));

View File

@@ -226,8 +226,8 @@ private:
/**
* Spans the acquire lifecycle: started in init(), finalized in done()
* with the outcome (complete/failed), timeout count, and peer count.
* Gives operators visibility into back-fill / fork-recovery cost, which
* previously emitted no span or metric.
* This span is the only signal for back-fill / fork-recovery cost; no
* other span or metric covers it.
* Thread-free: emplaced by the acquiring thread, reset on a JtLedgerData
* worker. A SpanGuard owns no thread-local Scope, so it can be destroyed
* on the worker without corrupting the origin thread's context stack.

View File

@@ -188,6 +188,12 @@ applyTransactions(
// If there are any transactions left, we must have
// tried them in at least one final pass
XRPL_ASSERT(txns.empty() || !certainRetry, "xrpl::applyTransactions : retry transactions");
// Repeated from the parent ledger.build span on purpose: TraceQL cannot
// reach a parent's attributes from a child, so without it no query can
// select this span by ledger. `view` is the accumulator over the ledger
// being built and copies its header, so this is the same sequence number
// the parent reports.
applySpan.setAttribute(ledger_span::attr::ledgerSeq, static_cast<int64_t>(view.seq()));
applySpan.setAttribute(ledger_span::attr::txCount, static_cast<int64_t>(count));
applySpan.setAttribute(ledger_span::attr::txFailed, static_cast<int64_t>(failed.size()));
return count;

View File

@@ -467,8 +467,9 @@ bool
LedgerMaster::storeLedger(std::shared_ptr<Ledger const> ledger)
{
using namespace telemetry;
auto span = SpanGuard::span(TraceCategory::Ledger, seg::ledger, ledger_span::op::store);
span.setAttribute(ledger_span::attr::ledgerSeq, static_cast<int64_t>(ledger->header().seq));
auto storeSpan = SpanGuard::span(TraceCategory::Ledger, seg::ledger, ledger_span::op::store);
storeSpan.setAttribute(
ledger_span::attr::ledgerSeq, static_cast<int64_t>(ledger->header().seq));
bool const validated = ledger->header().validated;
// Returns true if we already had the ledger
@@ -990,55 +991,65 @@ LedgerMaster::checkAccept(std::shared_ptr<Ledger const> const& ledger)
return;
}
using namespace telemetry;
auto valSpan = SpanGuard::span(TraceCategory::Ledger, seg::ledger, ledger_span::op::validate);
valSpan.setAttribute(ledger_span::attr::ledgerSeq, static_cast<int64_t>(ledger->header().seq));
valSpan.setAttribute(ledger_span::attr::validations, static_cast<int64_t>(tvc));
JLOG(journal_.info()) << "Advancing accepted ledger to " << ledger->header().seq
<< " with >= " << minVal << " validations";
ledger->setValidated();
ledger->setFull();
setValidLedger(ledger);
if (!pubLedger_)
// Scoped so ledger.validate measures only the promotion itself. The
// flag-ledger upgrade-warning check below runs on one ledger in 256 and
// reads every trusted validation of the parent, so leaving it inside would
// make every 256th span a duration outlier for work that is not part of
// promoting a ledger. tryAdvance() stays inside: it only sets a flag and
// posts a job, so it adds no measurable time.
{
pendSaveValidated(app_, ledger, true, true);
setPubLedger(ledger);
app_.getOrderBookDB().setup(ledger);
}
using namespace telemetry;
auto validateSpan =
SpanGuard::span(TraceCategory::Ledger, seg::ledger, ledger_span::op::validate);
validateSpan.setAttribute(
ledger_span::attr::ledgerSeq, static_cast<int64_t>(ledger->header().seq));
validateSpan.setAttribute(ledger_span::attr::validations, static_cast<int64_t>(tvc));
std::uint32_t const base = app_.getFeeTrack().getLoadBase();
auto fees = app_.getValidations().fees(ledger->header().hash, base);
{
auto fees2 = app_.getValidations().fees(ledger->header().parentHash, base);
fees.reserve(fees.size() + fees2.size());
std::ranges::copy(fees2, std::back_inserter(fees));
}
std::uint32_t fee = 0;
if (!fees.empty())
{
std::ranges::sort(fees);
if (auto stream = journal_.debug())
JLOG(journal_.info()) << "Advancing accepted ledger to " << ledger->header().seq
<< " with >= " << minVal << " validations";
ledger->setValidated();
ledger->setFull();
setValidLedger(ledger);
if (!pubLedger_)
{
std::stringstream s;
s << "Received fees from validations: (" << fees.size() << ") ";
for (auto const fee1 : fees)
{
s << " " << fee1;
}
stream << s.str();
pendSaveValidated(app_, ledger, true, true);
setPubLedger(ledger);
app_.getOrderBookDB().setup(ledger);
}
fee = fees[fees.size() / 2]; // median
}
else
{
fee = base;
}
app_.getFeeTrack().setRemoteFee(fee);
std::uint32_t const base = app_.getFeeTrack().getLoadBase();
auto fees = app_.getValidations().fees(ledger->header().hash, base);
{
auto fees2 = app_.getValidations().fees(ledger->header().parentHash, base);
fees.reserve(fees.size() + fees2.size());
std::ranges::copy(fees2, std::back_inserter(fees));
}
std::uint32_t fee = 0;
if (!fees.empty())
{
std::ranges::sort(fees);
if (auto stream = journal_.debug())
{
std::stringstream s;
s << "Received fees from validations: (" << fees.size() << ") ";
for (auto const fee1 : fees)
{
s << " " << fee1;
}
stream << s.str();
}
fee = fees[fees.size() / 2]; // median
}
else
{
fee = base;
}
tryAdvance();
app_.getFeeTrack().setRemoteFee(fee);
tryAdvance();
}
if (ledger->seq() % 256 == 0)
{

View File

@@ -1973,10 +1973,12 @@ PeerImp::onMessage(std::shared_ptr<protocol::TMProposeSet> const& m)
{
using namespace telemetry;
// root: inbound peer message entry point (kConsumer); must not inherit
// any span left active on this peer thread.
auto span =
// any span left active on this peer thread. Named after the span it holds,
// peer.proposal.receive, to keep it distinct from `proposalSpan` below,
// which holds the consensus-level span handed to the job worker.
auto proposalReceiveSpan =
ScopedSpanGuard::freshRoot(TraceCategory::Peer, seg::peer, peer_span::op::proposalReceive);
span.setAttribute(peer_span::attr::peerId, static_cast<int64_t>(id_));
proposalReceiveSpan.setAttribute(peer_span::attr::peerId, static_cast<int64_t>(id_));
protocol::TMProposeSet const& set = *m;
@@ -2004,7 +2006,7 @@ PeerImp::onMessage(std::shared_ptr<protocol::TMProposeSet> const& m)
// every time a spam packet is received
PublicKey const publicKey{makeSlice(set.nodepubkey())};
auto const isTrusted = app_.getValidators().trusted(publicKey);
span.setAttribute(peer_span::attr::proposalTrusted, isTrusted);
proposalReceiveSpan.setAttribute(peer_span::attr::proposalTrusted, isTrusted);
// If the operator has specified that untrusted proposals be dropped then
// this happens here I.e. before further wasting CPU verifying the signature
@@ -2578,10 +2580,12 @@ PeerImp::onMessage(std::shared_ptr<protocol::TMValidation> const& m)
{
using namespace telemetry;
// root: inbound peer message entry point (kConsumer); must not inherit
// any span left active on this peer thread.
auto valSpan = ScopedSpanGuard::freshRoot(
// any span left active on this peer thread. Named after the span it holds,
// peer.validation.receive, to keep it distinct from the consensus-level
// span handed to the job worker below.
auto validationReceiveSpan = ScopedSpanGuard::freshRoot(
TraceCategory::Peer, seg::peer, peer_span::op::validationReceive);
valSpan.setAttribute(peer_span::attr::peerId, static_cast<int64_t>(id_));
validationReceiveSpan.setAttribute(peer_span::attr::peerId, static_cast<int64_t>(id_));
if (m->validation().size() < 50)
{
@@ -2622,11 +2626,11 @@ PeerImp::onMessage(std::shared_ptr<protocol::TMValidation> const& m)
// false when telemetry is compiled out, switched off in the config, or
// the Peer trace category is disabled; a span that exists but was
// sampled out still pays.
if (valSpan)
if (validationReceiveSpan)
{
valSpan.setAttribute(
validationReceiveSpan.setAttribute(
peer_span::attr::ledgerHash, to_string(val->getLedgerHash()).c_str());
valSpan.setAttribute(peer_span::attr::fullValidation, val->isFull());
validationReceiveSpan.setAttribute(peer_span::attr::fullValidation, val->isFull());
}
if (!isCurrent(
@@ -2644,7 +2648,7 @@ PeerImp::onMessage(std::shared_ptr<protocol::TMValidation> const& m)
// suppression for 30 seconds to avoid doing a relatively expensive
// lookup every time a spam packet is received
auto const isTrusted = app_.getValidators().trusted(val->getSignerPublic());
valSpan.setAttribute(peer_span::attr::validationTrusted, isTrusted);
validationReceiveSpan.setAttribute(peer_span::attr::validationTrusted, isTrusted);
// If the operator has specified that untrusted validations be
// dropped then this happens here I.e. before further wasting CPU
@@ -2713,12 +2717,29 @@ PeerImp::onMessage(std::shared_ptr<protocol::TMValidation> const& m)
static_cast<int64_t>(val->getSignTime().time_since_epoch().count()));
}
// validation_status is set once on each exit below, not as a default
// here, to avoid OTel SDK attribute duplication. It is what separates
// the microsecond drop paths from the queued path, which also covers
// job wait and checkValidation.
if (!isTrusted && (tracking_.load() == Tracking::Diverged))
{
if (span && *span)
{
span->setAttribute(
telemetry::consensus::span::attr::validationStatus,
telemetry::consensus::span::val::validationDroppedDiverged);
}
JLOG(pJournal_.debug()) << "Dropping untrusted validation from diverged peer";
}
else if (isTrusted || !app_.getFeeTrack().isLoadedLocal())
{
// Set before the handle is moved into the job below.
if (span && *span)
{
span->setAttribute(
telemetry::consensus::span::attr::validationStatus,
telemetry::consensus::span::val::validationQueued);
}
std::string const name = isTrusted ? "ChkTrust" : "ChkUntrust";
std::weak_ptr<PeerImp> const weak = shared_from_this();
@@ -2732,6 +2753,12 @@ PeerImp::onMessage(std::shared_ptr<protocol::TMValidation> const& m)
}
else
{
if (span && *span)
{
span->setAttribute(
telemetry::consensus::span::attr::validationStatus,
telemetry::consensus::span::val::validationDroppedLoad);
}
JLOG(pJournal_.debug()) << "Dropping untrusted validation for load";
}
}

View File

@@ -704,10 +704,10 @@ private:
/**
* Record the OTel metrics for one completed `TMGetObjectByHash` request.
*
* Extracted from `processGetObjectByHash()` purely to keep that method
* within the 80-line limit; it holds no logic of its own beyond deriving
* the hit/miss split from `requested` and `found`. Called once per
* request, after the fetch loop and the `charge()` call.
* Called once per request from `processGetObjectByHash()`, after the fetch
* loop and the `charge()` call. A separate method so that one stays within
* the 80-line limit; it holds no logic of its own beyond deriving the
* hit/miss split from `requested` and `found`.
*
* Records `getobject_request_objects`, `getobject_lookup_us`,
* `getobject_charge`, and both label values of

View File

@@ -36,7 +36,14 @@ using ::xrpl::telemetry::attr::ledgerHash;
using ::xrpl::telemetry::attr::peerId;
/**
* Trust flag qualified by message type, shared with consensus.*.receive.
* Trust flag qualified by message type — whether the sending key is on this
* node's UNL.
*
* The literals match consensus::span::attr::proposalTrusted and
* ::validationTrusted, so the peer and consensus receive spans report on one
* spanmetrics dimension instead of two. Unlike the constants above these are
* declared here rather than re-exported from SpanNames.h, so the two spellings
* are only kept equal by hand: change one and the dimension splits silently.
*/
inline constexpr auto proposalTrusted = makeStr("proposal_trusted");
inline constexpr auto validationTrusted = makeStr("validation_trusted");

View File

@@ -596,8 +596,8 @@ PathRequest::findPaths(
// One `pathfind.discover` span wraps the entire per-source-asset loop so
// that a single RPC call produces one discover span instead of N (one per
// candidate source asset). Trade-off: per-asset discovery/ranking timing
// is no longer split into individual spans — span count and Tempo storage
// are bounded per RPC at the cost of per-asset visibility.
// is not measured separately — span count and Tempo storage are bounded
// per RPC at the cost of per-asset visibility.
//
// This is an unscoped guard: it takes the ambient span as its own parent,
// but does not itself become the ambient parent. Adding per-asset child

View File

@@ -30,11 +30,12 @@
// The app and overlay includes below are why
// .github/scripts/levelization/results/loops.txt records
// `xrpld.app <-> xrpld.telemetry` and `xrpld.overlay <-> xrpld.telemetry`, where
// ordering.txt previously had telemetry strictly below both. The observable
// gauges are pull-model: their callbacks sample live state when the reader
// thread fires, so they need the concrete types to call getJqTransOverflow(),
// size(), getPeerDisconnectCharges(), foreach() and txMetrics().
// `xrpld.app <-> xrpld.telemetry` and `xrpld.overlay <-> xrpld.telemetry` as
// cycles, rather than an acyclic ordering.txt entry placing telemetry strictly
// below both. The observable gauges are pull-model: their callbacks sample live
// state when the reader thread fires, so they need the concrete types to call
// getJqTransOverflow(), size(), getPeerDisconnectCharges(), foreach() and
// txMetrics().
//
// The cycle is confined to this translation unit. No telemetry header includes
// app or overlay (MetricsRegistry.h forward-declares what it needs and takes a
@@ -198,9 +199,9 @@ MetricsRegistry::~MetricsRegistry()
void
MetricsRegistry::start(
std::string const& endpoint,
std::string const& instanceId,
std::string const& nodeId)
[[maybe_unused]] std::string const& endpoint,
[[maybe_unused]] std::string const& instanceId,
[[maybe_unused]] std::string const& nodeId)
{
#ifdef XRPL_ENABLE_TELEMETRY
if (!enabled_)
@@ -221,11 +222,6 @@ MetricsRegistry::start(
initSyncInstruments();
JLOG(journal_.info()) << "MetricsRegistry: provider and instruments ready";
#else
(void)endpoint;
(void)instanceId;
(void)nodeId;
(void)enabled_;
#endif // XRPL_ENABLE_TELEMETRY
}
@@ -248,8 +244,6 @@ MetricsRegistry::startAsyncGauges()
registerAsyncGauges();
JLOG(journal_.info()) << "MetricsRegistry: started successfully";
#else
(void)enabled_;
#endif // XRPL_ENABLE_TELEMETRY
}
@@ -409,20 +403,19 @@ MetricsRegistry::stop()
// -----------------------------------------------------------------
void
MetricsRegistry::recordRpcStarted(std::string_view method)
MetricsRegistry::recordRpcStarted([[maybe_unused]] std::string_view method)
{
#ifdef XRPL_ENABLE_TELEMETRY
if (!enabled_ || !rpcStartedCounter_)
return;
rpcStartedCounter_->Add(1, {{"method", std::string(method)}});
#else
(void)method;
(void)enabled_;
#endif
}
void
MetricsRegistry::recordRpcFinished(std::string_view method, std::int64_t durationUs)
MetricsRegistry::recordRpcFinished(
[[maybe_unused]] std::string_view method,
[[maybe_unused]] std::int64_t durationUs)
{
#ifdef XRPL_ENABLE_TELEMETRY
if (!enabled_ || !rpcFinishedCounter_)
@@ -435,15 +428,13 @@ MetricsRegistry::recordRpcFinished(std::string_view method, std::int64_t duratio
{{"method", std::string(method)}},
opentelemetry::context::Context{});
}
#else
(void)method;
(void)durationUs;
(void)enabled_;
#endif
}
void
MetricsRegistry::recordRpcErrored(std::string_view method, std::int64_t durationUs)
MetricsRegistry::recordRpcErrored(
[[maybe_unused]] std::string_view method,
[[maybe_unused]] std::int64_t durationUs)
{
#ifdef XRPL_ENABLE_TELEMETRY
if (!enabled_ || !rpcErroredCounter_)
@@ -456,10 +447,6 @@ MetricsRegistry::recordRpcErrored(std::string_view method, std::int64_t duration
{{"method", std::string(method)}},
opentelemetry::context::Context{});
}
#else
(void)method;
(void)durationUs;
(void)enabled_;
#endif
}
@@ -468,7 +455,9 @@ MetricsRegistry::recordRpcErrored(std::string_view method, std::int64_t duration
// -----------------------------------------------------------------
void
MetricsRegistry::recordJobQueued(std::string_view jobType, std::string_view jobName)
MetricsRegistry::recordJobQueued(
[[maybe_unused]] std::string_view jobType,
[[maybe_unused]] std::string_view jobName)
{
#ifdef XRPL_ENABLE_TELEMETRY
if (!enabled_ || !jobQueuedCounter_)
@@ -477,18 +466,14 @@ MetricsRegistry::recordJobQueued(std::string_view jobType, std::string_view jobN
1,
{{kJobTypeLabel, std::string(jobType)},
{kHandlerLabel, std::string(sanitiseHandler(jobName))}});
#else
(void)jobType;
(void)jobName;
(void)enabled_;
#endif
}
void
MetricsRegistry::recordJobStarted(
std::string_view jobType,
std::string_view jobName,
std::int64_t queuedDurUs)
[[maybe_unused]] std::string_view jobType,
[[maybe_unused]] std::string_view jobName,
[[maybe_unused]] std::int64_t queuedDurUs)
{
#ifdef XRPL_ENABLE_TELEMETRY
if (!enabled_ || !jobStartedCounter_)
@@ -508,19 +493,14 @@ MetricsRegistry::recordJobStarted(
{{kJobTypeLabel, std::string(jobType)}, {kHandlerLabel, handler}},
opentelemetry::context::Context{});
}
#else
(void)jobType;
(void)jobName;
(void)queuedDurUs;
(void)enabled_;
#endif
}
void
MetricsRegistry::recordJobFinished(
std::string_view jobType,
std::string_view jobName,
std::int64_t runningDurUs)
[[maybe_unused]] std::string_view jobType,
[[maybe_unused]] std::string_view jobName,
[[maybe_unused]] std::int64_t runningDurUs)
{
#ifdef XRPL_ENABLE_TELEMETRY
if (!enabled_ || !jobFinishedCounter_)
@@ -534,11 +514,6 @@ MetricsRegistry::recordJobFinished(
{{kJobTypeLabel, std::string(jobType)}, {kHandlerLabel, handler}},
opentelemetry::context::Context{});
}
#else
(void)jobType;
(void)jobName;
(void)runningDurUs;
(void)enabled_;
#endif
}
@@ -1095,32 +1070,30 @@ MetricsRegistry::registerCompleteLedgersGauge()
return;
// Parse comma-separated ranges like
// "32570-50000,50005-75891421".
// "32570-50000,50005-75891421". A range of one ledger arrives
// as a bare sequence number, so parseLedgerRange() decides what
// a segment is; only genuinely unreadable ones are skipped.
std::size_t rangeIndex = 0;
std::istringstream stream(rangeStr);
std::string segment;
while (std::getline(stream, segment, ','))
{
auto const dashPos = segment.find('-');
if (dashPos == std::string::npos || dashPos == 0 ||
dashPos == segment.size() - 1)
auto const range = MetricsRegistry::parseLedgerRange(segment);
if (!range)
continue;
auto const startStr = segment.substr(0, dashPos);
auto const endStr = segment.substr(dashPos + 1);
auto const idxStr = std::to_string(rangeIndex);
opentelemetry::nostd::get<opentelemetry::nostd::shared_ptr<
opentelemetry::metrics::ObserverResultT<int64_t>>>(result)
->Observe(
static_cast<int64_t>(std::stoll(startStr)),
static_cast<int64_t>(range->first),
{{"bound", "start"}, {"index", idxStr}});
opentelemetry::nostd::get<opentelemetry::nostd::shared_ptr<
opentelemetry::metrics::ObserverResultT<int64_t>>>(result)
->Observe(
static_cast<int64_t>(std::stoll(endStr)),
static_cast<int64_t>(range->second),
{{"bound", "end"}, {"index", idxStr}});
++rangeIndex;
@@ -1462,7 +1435,7 @@ MetricsRegistry::registerStateTrackingGauge()
// State value: 0-4 from OperatingMode, 5=validating, 6=proposing.
auto const mode = app.getOPs().getOperatingMode();
auto stateValue = static_cast<double>(mode);
auto stateValue = static_cast<double>(std::to_underlying(mode));
// If FULL, refine using consensus info for validating/proposing.
if (mode == OperatingMode::FULL)

View File

@@ -99,7 +99,9 @@
* // before any metric-emitting code:
* metricsRegistry_ = std::make_unique<telemetry::MetricsRegistry>(
* telemetry_->isEnabled(), app, journal);
* metricsRegistry_->start(setup.exporterEndpoint);
* // The endpoint comes from [telemetry] metrics_endpoint, read directly in
* // Application::setup() rather than through Telemetry::Setup.
* metricsRegistry_->start(endpoint, instanceId, nodeId);
*
* // Later in setup(), once overlay_ exists (the last of the services the
* // callbacks read). Phase 2 registers the observable instruments:
@@ -147,11 +149,13 @@
#include <xrpl/beast/utility/Journal.h>
#include <algorithm>
#include <charconv>
#include <cstdint>
#include <limits>
#include <optional>
#include <string>
#include <string_view>
#include <utility>
#ifdef XRPL_ENABLE_TELEMETRY
#include <opentelemetry/metrics/meter.h>
@@ -308,11 +312,11 @@ public:
* phase. Mostly ObservableGauges, plus the ObservableCounters whose
* source value is already cumulative.
*
* Split from `start()` because the two halves have different
* prerequisites. `start()` needs only config strings; these callbacks
* read live Application services, so this half must run later.
* A separate entry point from `start()` because the two halves have
* different prerequisites. `start()` needs only config strings; these
* callbacks read live Application services, so this half must run later.
* Registering an observable also arms the reader thread to invoke its
* callback on the next tick, which is why the split is about ordering
* callback on the next tick, which is why the separation is about ordering
* and not just tidiness.
*
* @pre `start()` has already run (the meter exists). If it has not,
@@ -557,6 +561,73 @@ public:
return static_cast<std::int64_t>(scaled + fraction);
}
/**
* Read one comma-separated segment of a complete-ledger range string.
*
* The producer is xrpl::to_string(RangeSet), documented in
* xrpl/basics/RangeSet.h. It renders an interval as `first-last`, and an
* interval whose first equals its last as a bare sequence number. A segment
* with no dash is therefore a range of one ledger, not a malformed one.
*
* Defined inline for the same reason as sanitiseHandler(): in a
* telemetry-enabled build MetricsRegistry.cpp is not compiled into the
* unit-test binary, so an out-of-line definition would be untestable.
*
* @param segment One segment, already split on ','. Leading or trailing
* whitespace is rejected, because the producer emits none.
* @return The inclusive first and last sequence of the range. The two are
* equal for a single-ledger range. std::nullopt when @p segment is not
* something this producer can emit.
*
* @note Pure and reentrant: holds no state, performs no I/O, and is safe to
* call concurrently from any thread.
* @note Reports malformed input instead of throwing, so one unreadable
* segment costs its own range and not every range after it.
* @note A reversed range such as "9-4" is returned as given. RangeSet
* cannot emit one.
*
* Example:
* @code
* parseLedgerRange("32570-50000"); // {32570, 50000}
* parseLedgerRange("5000"); // {5000, 5000} -- one ledger
* parseLedgerRange("5-"); // nullopt
* @endcode
*/
[[nodiscard]] static std::optional<std::pair<std::uint32_t, std::uint32_t>>
parseLedgerRange(std::string_view segment) noexcept
{
auto const parseSeq = [](std::string_view text) -> std::optional<std::uint32_t> {
std::uint32_t value = 0;
auto const* const begin = text.data();
auto const* const end = begin + text.size();
auto const [ptr, ec] = std::from_chars(begin, end, value);
// from_chars stops at the first character it cannot use, so the
// whole segment counts as read only when it consumed all of it.
if (ec != std::errc{} || ptr != end)
return std::nullopt;
return value;
};
auto const dash = segment.find('-');
if (dash == std::string_view::npos)
{
auto const only = parseSeq(segment);
if (!only)
return std::nullopt;
return std::pair{*only, *only};
}
auto const first = parseSeq(segment.substr(0, dash));
auto const last = parseSeq(segment.substr(dash + 1));
if (!first || !last)
return std::nullopt;
return std::pair{*first, *last};
}
/**
* Record a job enqueued event.
* @param jobType The job type name (e.g. "ledgerData").
@@ -670,7 +741,7 @@ public:
* into it is not free: each call takes its lock and inserts an entry.
* @return Reference to the internal ValidationTracker instance.
*/
ValidationTracker&
[[nodiscard]] ValidationTracker&
getValidationTracker()
{
return validationTracker_;
@@ -684,7 +755,7 @@ public:
* start() has run or when disabled.
* @return The shared Meter, or empty if not yet started.
*/
opentelemetry::nostd::shared_ptr<opentelemetry::metrics::Meter>
[[nodiscard]] opentelemetry::nostd::shared_ptr<opentelemetry::metrics::Meter>
meter() const noexcept
{
return meter_;

View File

@@ -32,8 +32,12 @@ namespace xrpl::telemetry {
* trace_id is derived from txID[0:16]. If the incoming message carries
* a protobuf TraceContext with a valid span_id, it is used as the
* parent to preserve relay ordering.
* @param txID Transaction id; its first 16 bytes become the trace_id.
* @param msg The received message, read only for its trace context.
* @return An active guard, or a null guard when the Transactions category
* is disabled. Bind it: a discarded guard ends the span immediately.
*/
inline SpanGuard
[[nodiscard]] inline SpanGuard
txReceiveSpan(uint256 const& txID, [[maybe_unused]] protocol::TMTransaction const& msg)
{
#ifdef XRPL_ENABLE_TELEMETRY
@@ -63,8 +67,11 @@ txReceiveSpan(uint256 const& txID, [[maybe_unused]] protocol::TMTransaction cons
/**
* Create a "tx.process" span for transaction processing in NetworkOPs.
* trace_id is derived from txID[0:16].
* @param txID Transaction id; its first 16 bytes become the trace_id.
* @return An active guard, or a null guard when the Transactions category
* is disabled. Bind it: a discarded guard ends the span immediately.
*/
inline SpanGuard
[[nodiscard]] inline SpanGuard
txProcessSpan(uint256 const& txID)
{
return SpanGuard::hashSpan(

View File

@@ -138,21 +138,21 @@ public:
* Agreement percentage over the last 1 hour.
* @return Percentage [0.0, 100.0], or 0.0 if no data.
*/
double
[[nodiscard]] double
agreementPct1h() const;
/**
* Agreement percentage over the last 24 hours.
* @return Percentage [0.0, 100.0], or 0.0 if no data.
*/
double
[[nodiscard]] double
agreementPct24h() const;
/**
* Agreement percentage over the last 7 days.
* @return Percentage [0.0, 100.0], or 0.0 if no data.
*/
double
[[nodiscard]] double
agreementPct7d() const;
/** @} */
@@ -165,37 +165,37 @@ public:
/**
* Number of agreements in the 1-hour window.
*/
uint64_t
[[nodiscard]] uint64_t
agreements1h() const;
/**
* Number of misses in the 1-hour window.
*/
uint64_t
[[nodiscard]] uint64_t
missed1h() const;
/**
* Number of agreements in the 24-hour window.
*/
uint64_t
[[nodiscard]] uint64_t
agreements24h() const;
/**
* Number of misses in the 24-hour window.
*/
uint64_t
[[nodiscard]] uint64_t
missed24h() const;
/**
* Number of agreements in the 7-day window.
*/
uint64_t
[[nodiscard]] uint64_t
agreements7d() const;
/**
* Number of misses in the 7-day window.
*/
uint64_t
[[nodiscard]] uint64_t
missed7d() const;
/** @} */
@@ -208,13 +208,13 @@ public:
/**
* Total agreements since process start.
*/
uint64_t
[[nodiscard]] uint64_t
totalAgreements() const;
/**
* Total misses since process start.
*/
uint64_t
[[nodiscard]] uint64_t
totalMissed() const;
/**
@@ -226,7 +226,7 @@ public:
* counter validation_agreements_total. See the counting-semantics
* note in detail/ValidationTracker.cpp.
*/
uint64_t
[[nodiscard]] uint64_t
totalAgreementsEver() const;
/**
@@ -238,19 +238,19 @@ public:
* counter validation_missed_total. See the counting-semantics note
* in detail/ValidationTracker.cpp.
*/
uint64_t
[[nodiscard]] uint64_t
totalMissedEver() const;
/**
* Total validations this node sent.
*/
uint64_t
[[nodiscard]] uint64_t
totalValidationsSent() const;
/**
* Total network validations observed for comparison.
*/
uint64_t
[[nodiscard]] uint64_t
totalValidationsChecked() const;
/**

View File

@@ -180,19 +180,11 @@ void
ValidationTracker::evictOldPending(TimePoint now)
{
auto const cutoff = now - kLateRepairWindow;
for (auto it = pending_.begin(); it != pending_.end();)
{
if (it->second.reconciled && it->second.recordTime < cutoff)
{
it = pending_.erase(it);
}
else
{
++it;
}
}
std::erase_if(pending_, [cutoff](auto const& entry) {
return entry.second.reconciled && entry.second.recordTime < cutoff;
});
// Hard trim if still over limit. The loop above already removed every
// Hard trim if still over limit. The pass above already removed every
// reconciled entry older than the late-repair window, so here we drop
// any remaining reconciled entry as a last resort.
if (pending_.size() > kMaxPendingEvents)
@@ -219,7 +211,7 @@ ValidationTracker::agreementPct1h() const
if (window1h_.empty())
return 0.0;
auto const agreed = static_cast<double>(
std::count_if(window1h_.begin(), window1h_.end(), [](auto const& e) { return e.agreed; }));
std::ranges::count_if(window1h_, [](auto const& e) { return e.agreed; }));
return (agreed / static_cast<double>(window1h_.size())) * 100.0;
}
@@ -229,8 +221,8 @@ ValidationTracker::agreementPct24h() const
std::scoped_lock const lock(mutex_);
if (window24h_.empty())
return 0.0;
auto const agreed = static_cast<double>(std::count_if(
window24h_.begin(), window24h_.end(), [](auto const& e) { return e.agreed; }));
auto const agreed = static_cast<double>(
std::ranges::count_if(window24h_, [](auto const& e) { return e.agreed; }));
return (agreed / static_cast<double>(window24h_.size())) * 100.0;
}
@@ -239,7 +231,7 @@ ValidationTracker::agreements1h() const
{
std::scoped_lock const lock(mutex_);
return static_cast<uint64_t>(
std::count_if(window1h_.begin(), window1h_.end(), [](auto const& e) { return e.agreed; }));
std::ranges::count_if(window1h_, [](auto const& e) { return e.agreed; }));
}
uint64_t
@@ -247,23 +239,23 @@ ValidationTracker::missed1h() const
{
std::scoped_lock const lock(mutex_);
return static_cast<uint64_t>(
std::count_if(window1h_.begin(), window1h_.end(), [](auto const& e) { return !e.agreed; }));
std::ranges::count_if(window1h_, [](auto const& e) { return !e.agreed; }));
}
uint64_t
ValidationTracker::agreements24h() const
{
std::scoped_lock const lock(mutex_);
return static_cast<uint64_t>(std::count_if(
window24h_.begin(), window24h_.end(), [](auto const& e) { return e.agreed; }));
return static_cast<uint64_t>(
std::ranges::count_if(window24h_, [](auto const& e) { return e.agreed; }));
}
uint64_t
ValidationTracker::missed24h() const
{
std::scoped_lock const lock(mutex_);
return static_cast<uint64_t>(std::count_if(
window24h_.begin(), window24h_.end(), [](auto const& e) { return !e.agreed; }));
return static_cast<uint64_t>(
std::ranges::count_if(window24h_, [](auto const& e) { return !e.agreed; }));
}
double
@@ -273,7 +265,7 @@ ValidationTracker::agreementPct7d() const
if (window7d_.empty())
return 0.0;
auto const agreed = static_cast<double>(
std::count_if(window7d_.begin(), window7d_.end(), [](auto const& e) { return e.agreed; }));
std::ranges::count_if(window7d_, [](auto const& e) { return e.agreed; }));
return (agreed / static_cast<double>(window7d_.size())) * 100.0;
}
@@ -282,7 +274,7 @@ ValidationTracker::agreements7d() const
{
std::scoped_lock const lock(mutex_);
return static_cast<uint64_t>(
std::count_if(window7d_.begin(), window7d_.end(), [](auto const& e) { return e.agreed; }));
std::ranges::count_if(window7d_, [](auto const& e) { return e.agreed; }));
}
uint64_t
@@ -290,7 +282,7 @@ ValidationTracker::missed7d() const
{
std::scoped_lock const lock(mutex_);
return static_cast<uint64_t>(
std::count_if(window7d_.begin(), window7d_.end(), [](auto const& e) { return !e.agreed; }));
std::ranges::count_if(window7d_, [](auto const& e) { return !e.agreed; }));
}
uint64_t