mirror of
https://github.com/XRPLF/rippled.git
synced 2026-08-21 14:20:56 +00:00
The bucket edges for the OTel histograms lived as file-local `namespace {}`
constants, unreachable from any test, and they drifted from the collector's
spanmetrics ladder they were specified to match. The millisecond ladder
stayed capped at 5 s after the collector side was extended to 30 s, so any
quantile above 5 s read back as a flat 5000 -- Prometheus returns the
second-highest edge for a quantile in the `+Inf` bucket, which looks like a
measurement rather than an error.
Adds include/xrpl/telemetry/HistogramBuckets.h as the single owner of the
ladders, with a constexpr validator plus static_asserts so a descending or
duplicated edge cannot compile, and gtest coverage that pins the floor and
ceiling against the measured distributions:
- kMillisecondBuckets carries every representable collector edge and extends
to 120 s, because the updatepaths job type averages ~60 s and a 30 s
ceiling would censor it exactly as 5 s does today. Sub-millisecond
collector edges are omitted: beast::insight::Event rounds durations up to
whole milliseconds, so they would collect nothing.
- kByteBuckets is new, for Events whose samples are sizes rather than
durations. Edges follow the measured RPC response distribution (mean
2131 B, half under 1 kB, tail mean bounded at 7538 B) rather than a guess,
so the resolution sits between 512 B and 64 kB.
No behaviour change yet -- nothing consumes the header until the views are
rewired.
180 lines
6.8 KiB
C++
180 lines
6.8 KiB
C++
#pragma once
|
|
|
|
#include <array>
|
|
#include <cstddef>
|
|
#include <span>
|
|
#include <vector>
|
|
|
|
namespace xrpl::telemetry::buckets {
|
|
|
|
/**
|
|
* @file HistogramBuckets.h
|
|
* @brief Explicit histogram bucket edges for xrpld's OTel instruments.
|
|
*
|
|
* One header owns every ladder so a reviewer sees all of them at once and a
|
|
* test can assert their invariants. Before this existed the edges lived as
|
|
* file-local `namespace {}` constants, unreachable from any test, and they
|
|
* drifted apart.
|
|
*
|
|
* Why a ladder is worth this much care: when a quantile falls in the `+Inf`
|
|
* bucket, Prometheus returns the *second-highest* edge, not `+Inf`. A
|
|
* saturated histogram therefore reports a believable constant instead of an
|
|
* obvious error. The same trap exists at the bottom -- if nearly every
|
|
* sample lands in bucket 0, `histogram_quantile` interpolates inside it and
|
|
* invents a value. A ladder is correct only when its floor sits below the
|
|
* mass of the distribution and its ceiling above the tail.
|
|
*
|
|
* sample --> [ SDK lower_bound over edges ] --> per-bucket counter
|
|
* | |
|
|
* edges come from v
|
|
* THIS header OTLP export
|
|
* |
|
|
* v
|
|
* histogram_quantile() in Grafana
|
|
*
|
|
* Ladders are `std::array<double, N>` so they are constant-initialised and
|
|
* usable in a `static_assert`. The OTel SDK wants `std::vector<double>` in
|
|
* its aggregation config, so call toVector() at the registration site
|
|
* rather than storing vectors here.
|
|
*
|
|
* Example -- register a view with the millisecond ladder:
|
|
* @code
|
|
* auto config = std::make_shared<HistogramAggregationConfig>();
|
|
* config->boundaries_ = buckets::toVector(buckets::kMillisecondBuckets);
|
|
* @endcode
|
|
*
|
|
* Example -- the edge case that motivated a second ladder. An Event whose
|
|
* samples are sizes rather than durations must not borrow a latency ladder,
|
|
* or a quarter of its samples land in `+Inf` and every quantile reads back
|
|
* as the top edge:
|
|
* @code
|
|
* config->boundaries_ = buckets::toVector(buckets::kByteBuckets);
|
|
* @endcode
|
|
*
|
|
* @note Thread safety: every member is `constexpr` and immutable, so
|
|
* reading them from any thread is safe. toVector() allocates and is
|
|
* meant for start-up registration paths, never for a record path.
|
|
* @note Limitation: changing a ladder changes the exported series count and
|
|
* ends bucket comparability across the change -- existing series keep
|
|
* their old `le` values, so panels show a break at restart. Grafana
|
|
* Cloud bills per series, so re-measure the series count after any
|
|
* edit here.
|
|
*/
|
|
|
|
/**
|
|
* Bucket edges, in milliseconds, for whole-millisecond `beast::insight`
|
|
* Events: job queue wait and run times, io latency, RPC time, pathfinding.
|
|
*
|
|
* **This list must contain every representable edge of the collector's
|
|
* spanmetrics ladder, and may extend above it.** Agreement over the shared
|
|
* range is deliberate: it lets a span-derived latency panel and a native
|
|
* histogram panel be read on the same scale. It was specified that way
|
|
* originally, then silently broken when the collector ladder alone was
|
|
* extended, which left this side capped at 5 s while spans reached 30 s and
|
|
* censored every quantile above 5 s. `check_bucket_parity.py` now enforces
|
|
* the containment -- add a collector edge, add it here too.
|
|
*
|
|
* The sub-millisecond edges the collector carries (0.01 to 0.5 ms) are
|
|
* deliberately absent. `beast::insight::Event` rounds every duration up to
|
|
* a whole millisecond before it reaches the histogram, so those edges would
|
|
* collect nothing. Metrics that genuinely need finer resolution belong on
|
|
* the microsecond ladder, on the OTel-native path.
|
|
*
|
|
* The 60 s and 120 s edges exceed the collector's 30 s top on purpose,
|
|
* because jobs outlive spans: the updatepaths job type was measured
|
|
* averaging about 60 s, so a 30 s ceiling would censor its quantiles just
|
|
* as 5 s censors them today. All these Events share one ladder, so its
|
|
* ceiling has to cover the slowest member rather than the typical one.
|
|
*
|
|
* The 2, 3 and 4 s edges resolve second-scale work that previously had to
|
|
* interpolate across a single four-second-wide bucket.
|
|
*/
|
|
inline constexpr std::array kMillisecondBuckets{
|
|
1.0,
|
|
5.0,
|
|
10.0,
|
|
25.0,
|
|
50.0,
|
|
100.0,
|
|
250.0,
|
|
500.0,
|
|
1'000.0,
|
|
2'000.0,
|
|
3'000.0,
|
|
4'000.0,
|
|
5'000.0,
|
|
10'000.0,
|
|
30'000.0,
|
|
60'000.0,
|
|
120'000.0};
|
|
|
|
/**
|
|
* Bucket edges, in bytes, for `beast::insight` Events whose samples are
|
|
* sizes rather than durations. Currently only the RPC response size.
|
|
*
|
|
* Placed from the measured distribution rather than from a guess about how
|
|
* large a response could theoretically be. Measured over 24 h: mean 2131 B,
|
|
* half of all responses under 1 kB, three quarters under 5 kB. The tail
|
|
* above 5 kB has a mean of at most 7538 B, which bounds p99 near 80 kB and
|
|
* p99.75 below 256 kB.
|
|
*
|
|
* So the resolution belongs between 512 B and 64 kB, where the
|
|
* distribution actually turns, and two further edges are ample headroom.
|
|
* Spending edges at the megabyte scale would cost cardinality on a range
|
|
* nothing measured occupies. If a genuinely multi-megabyte response ever
|
|
* shows up in the top bucket, extend this -- but extend it on evidence.
|
|
*/
|
|
inline constexpr std::array kByteBuckets{
|
|
512.0,
|
|
1'024.0,
|
|
2'048.0,
|
|
4'096.0,
|
|
8'192.0,
|
|
16'384.0,
|
|
32'768.0,
|
|
65'536.0,
|
|
262'144.0,
|
|
1'048'576.0};
|
|
|
|
/**
|
|
* @brief Check that a ladder is strictly ascending and non-negative.
|
|
*
|
|
* The SDK places a sample with `std::lower_bound` over the edges, which
|
|
* silently misbuckets when edges repeat or descend. Checking at compile
|
|
* time makes that class of typo impossible to ship.
|
|
*
|
|
* @param ladder Bucket upper bounds to check.
|
|
* @return true when the ladder is non-empty, starts at or above zero, and
|
|
* every later edge is strictly greater than its predecessor.
|
|
*/
|
|
constexpr bool
|
|
isAscendingNonNegative(std::span<double const> ladder) noexcept
|
|
{
|
|
if (ladder.empty() || ladder.front() < 0.0)
|
|
return false;
|
|
|
|
for (std::size_t i = 1; i < ladder.size(); ++i)
|
|
{
|
|
if (!(ladder[i] > ladder[i - 1]))
|
|
return false;
|
|
}
|
|
return true;
|
|
}
|
|
|
|
static_assert(isAscendingNonNegative(kMillisecondBuckets));
|
|
static_assert(isAscendingNonNegative(kByteBuckets));
|
|
|
|
/**
|
|
* @brief Copy a ladder into the `std::vector<double>` the OTel SDK wants.
|
|
*
|
|
* @param ladder Bucket upper bounds.
|
|
* @return A vector holding the same edges in the same order.
|
|
*/
|
|
inline std::vector<double>
|
|
toVector(std::span<double const> ladder)
|
|
{
|
|
return std::vector<double>(ladder.begin(), ladder.end());
|
|
}
|
|
|
|
} // namespace xrpl::telemetry::buckets
|