mirror of
https://github.com/XRPLF/rippled.git
synced 2026-09-27 15:28:03 +00:00
fix(telemetry): Stop unl_expiry_days wrapping, and count only unfinished sweep evictions
unl_expiry_days subtracted two NetClock time points, whose rep is uint32_t, so the subtraction wrapped before the duration_cast ran. A list expired by one day read about +49709 days. The panel is green above 30 while its own description promises red at expiry, so an expired validator list rendered healthy. daysUntil() widens both endpoints to int64_t first, which makes the wrap impossible rather than checked for. It deliberately does not clamp at zero: a negative reading is the signal that expiry has passed. A config-listed list, which uses time_point::max(), now reports positive infinity, because any finite sentinel could not be told apart from the wrap this removes. -1 keeps its existing meaning of no published list fetched. The sweep counter told a second story it could not support. It counted every entry the 1-minute sweep evicted, including acquisitions that had already completed or failed and were merely still in the map. Those were counted when they ended, so the metric buried the wasteful case in ordinary cleanup while the runbook, the reference doc and the panel description all described only the unfinished population. It now counts what those three already claimed. isComplete()/isFailed() are used rather than isDone(), which is protected on TimeoutCounter and not callable here.
This commit is contained in:
@@ -1371,7 +1371,7 @@ concurrency cap. Observed in `AppMetricGauges::observeNodeStoreTotals()`,
|
||||
| `nodestore_state{metric="acquire_aborts"}` | Gauge | `metric` | Acquisitions destroyed before finishing |
|
||||
| `nodestore_state{metric="acquire_aborts_partial"}` | Gauge | `metric` | Subset of aborts that discarded partly built maps |
|
||||
| `nodestore_state{metric="acquire_completions"}` | Gauge | `metric` | Acquisitions that finished successfully |
|
||||
| `nodestore_state{metric="acquire_sweep_evictions"}` | Gauge | `metric` | Acquisitions evicted by the 1-minute sweep |
|
||||
| `nodestore_state{metric="acquire_sweep_evictions"}` | Gauge | `metric` | Unfinished acquisitions evicted by the 1-minute sweep |
|
||||
|
||||
**Three properties to know before querying these.**
|
||||
|
||||
|
||||
@@ -2227,7 +2227,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Discarded Acquire Work (Sweeps & Partial Aborts)",
|
||||
"description": "###### What this is:\n*Work that was fetched and then thrown away. Sweep Evictions are acquisitions removed by the one-minute idle sweep. Partial Aborts are the subset of aborted acquisitions that had already built part of a map when they were destroyed, so the bytes fetched for them were wasted.*\n\n###### How it's computed:\n*rate() over the cumulative nodestore_state{metric=\"acquire_sweep_evictions\"} and acquire_aborts_partial counters.*\n\n###### Reading it:\n*Sweep evictions are the sweeper reclaiming acquisitions that stopped making progress, so a sustained non-zero rate means acquisitions are going idle rather than finishing -- it is the sweeper cleaning up after the stall on the adjacent panels, not a cause of its own. Partial aborts quantify the waste: each one is fetch bandwidth and nodestore writes spent on a map that was discarded, which then has to be fetched again.*\n\n###### Healthy range:\n*Both at or near zero.*\n\n###### Watch for:\n*A sweep-eviction rate that persists after the sync should have settled -- acquisitions are being started and abandoned in a loop, and each cycle re-pays the fetch cost. Two measured runs of the same duration differed sharply here, 127 sweeps in the write-bound case against 38 in the read-bound one, so a high sweep count is itself weak evidence for the write-bound mode. Confirm with read latency and writer depth before acting on it.*\n\n###### Keywords:\n- **NodeStore** *(per node)* \u2014 the key-value object store holding ledger data (tree nodes), backed by NuDB.\n\n###### Computation boundary:\n*Result: Per node \u2014 each series is one server's own value.*\n*Recorded in xrpld code as a native metric (beast::insight); the collector only forwards it; the Grafana query selects and aggregates it.*\n\n###### Source:\n[app/ledger/InboundLedgers.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/app/ledger/detail/InboundLedgers.cpp)\n\n###### Function:\n`InboundLedgers::sweep / AcquireStats`\n\n###### References:\n[Telemetry glossary](https://github.com/XRPLF/rippled/blob/develop/docs/telemetry-glossary.md#nodestore)",
|
||||
"description": "###### What this is:\n*Work that was fetched and then thrown away. Sweep Evictions are unfinished acquisitions removed by the one-minute idle sweep; an acquisition that had already completed or failed waits for the same sweep but is not counted here. Partial Aborts are the subset of aborted acquisitions that had already built part of a map when they were destroyed, so the bytes fetched for them were wasted.*\n\n###### How it's computed:\n*rate() over the cumulative nodestore_state{metric=\"acquire_sweep_evictions\"} and acquire_aborts_partial counters.*\n\n###### Reading it:\n*Sweep evictions are the sweeper reclaiming acquisitions that stopped making progress, so a sustained non-zero rate means acquisitions are going idle rather than finishing -- it is the sweeper cleaning up after the stall on the adjacent panels, not a cause of its own. Partial aborts quantify the waste: each one is fetch bandwidth and nodestore writes spent on a map that was discarded, which then has to be fetched again.*\n\n###### Healthy range:\n*Both at or near zero.*\n\n###### Watch for:\n*A sweep-eviction rate that persists after the sync should have settled -- acquisitions are being started and abandoned in a loop, and each cycle re-pays the fetch cost. Two measured runs of the same duration differed sharply here, 127 sweeps in the write-bound case against 38 in the read-bound one, so a high sweep count is itself weak evidence for the write-bound mode. Confirm with read latency and writer depth before acting on it.*\n\n###### Keywords:\n- **NodeStore** *(per node)* \u2014 the key-value object store holding ledger data (tree nodes), backed by NuDB.\n\n###### Computation boundary:\n*Result: Per node \u2014 each series is one server's own value.*\n*Recorded in xrpld code as a native metric (beast::insight); the collector only forwards it; the Grafana query selects and aggregates it.*\n\n###### Source:\n[app/ledger/InboundLedgers.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/app/ledger/detail/InboundLedgers.cpp)\n\n###### Function:\n`InboundLedgers::sweep / AcquireStats`\n\n###### References:\n[Telemetry glossary](https://github.com/XRPLF/rippled/blob/develop/docs/telemetry-glossary.md#nodestore)",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 10,
|
||||
|
||||
@@ -511,7 +511,7 @@
|
||||
},
|
||||
{
|
||||
"title": "UNL Expiry (days) [$xrpl_network_type]",
|
||||
"description": "###### What this is:\n*Days remaining until the current UNL (trusted validator list) expires.*\n\n###### How it's computed:\n*Instantaneous gauge reading of days-to-expiry.*\n\n###### Reading it:\n*Higher is safer; green at 30+, yellow under 7, red at expiry.*\n\n###### Healthy range:\n*30+ days.*\n\n###### Watch for:\n*Fewer than 7 days, after which the node loses its trusted validator set if not renewed.*\n\n###### Keywords:\n- **UNL (Unique Node List)** *(per node)* \u2014 the list of validators a node trusts not to collude; the basis for its consensus and quorum.\n\n###### Computation boundary:\n*Result: Per node \u2014 each series is one server's own value.*\n*Computed in xrpld code (MetricsRegistry, OpenTelemetry SDK) and exported as a metric; the collector only forwards it; the Grafana query selects and aggregates it.*\n\n###### Source:\n[AppMetricGauges.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/telemetry/AppMetricGauges.cpp)\n\n###### Function:\n`registerValidatorHealthGauge`\n\n###### References:\n[UNL (Unique Node List)](https://xrpl.org/docs/concepts/consensus-protocol/unl) \u00b7 [Telemetry glossary](https://github.com/XRPLF/rippled/blob/develop/docs/telemetry-glossary.md#unl-unique-node-list)",
|
||||
"description": "###### What this is:\n*Days remaining until the current UNL (trusted validator list) expires.*\n\n###### How it's computed:\n*Instantaneous gauge reading of days-to-expiry. It goes negative once the list has expired, reads -1 while no published list has been fetched at all, and reads infinity for a config-listed set that has no expiry date.*\n\n###### Reading it:\n*Higher is safer; green at 30+, yellow under 7, red at expiry.*\n\n###### Healthy range:\n*30+ days.*\n\n###### Watch for:\n*Fewer than 7 days, after which the node loses its trusted validator set if not renewed.*\n\n###### Keywords:\n- **UNL (Unique Node List)** *(per node)* \u2014 the list of validators a node trusts not to collude; the basis for its consensus and quorum.\n\n###### Computation boundary:\n*Result: Per node \u2014 each series is one server's own value.*\n*Computed in xrpld code (MetricsRegistry, OpenTelemetry SDK) and exported as a metric; the collector only forwards it; the Grafana query selects and aggregates it.*\n\n###### Source:\n[AppMetricGauges.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/telemetry/AppMetricGauges.cpp)\n\n###### Function:\n`registerValidatorHealthGauge`\n\n###### References:\n[UNL (Unique Node List)](https://xrpl.org/docs/concepts/consensus-protocol/unl) \u00b7 [Telemetry glossary](https://github.com/XRPLF/rippled/blob/develop/docs/telemetry-glossary.md#unl-unique-node-list)",
|
||||
"type": "stat",
|
||||
"gridPos": {
|
||||
"h": 10,
|
||||
|
||||
@@ -1797,7 +1797,7 @@ label values rather than reporting them as zero.
|
||||
| `nodestore_state{metric="acquire_aborts"}` | MetricsRegistry.cpp | Acquisitions destroyed before finishing |
|
||||
| `nodestore_state{metric="acquire_aborts_partial"}` | MetricsRegistry.cpp | Subset of aborts that discarded partly built maps |
|
||||
| `nodestore_state{metric="acquire_completions"}` | MetricsRegistry.cpp | Acquisitions that finished successfully |
|
||||
| `nodestore_state{metric="acquire_sweep_evictions"}` | MetricsRegistry.cpp | Acquisitions evicted by the 1-minute sweep |
|
||||
| `nodestore_state{metric="acquire_sweep_evictions"}` | MetricsRegistry.cpp | Unfinished acquisitions evicted by the 1-minute sweep |
|
||||
|
||||
`nudb_writer_depth_x100` is fixed-point: divide by 100 to read it. The depth sits
|
||||
just above 1.0 even under load, so an integer gauge would truncate the whole
|
||||
@@ -3452,8 +3452,10 @@ Two more pairs from the same family:
|
||||
- `acquire_sweep_evictions` rising while `acquire_completions` stays at zero →
|
||||
partial work is being discarded and redone. The sweep drops any acquisition
|
||||
idle for more than one minute
|
||||
(`src/xrpld/app/ledger/detail/InboundLedgers.cpp:400`), taking whatever it had
|
||||
built with it.
|
||||
(`src/xrpld/app/ledger/detail/InboundLedgers.cpp:402`), taking whatever it had
|
||||
built with it. Only the ones that had not finished are counted: a completed or
|
||||
failed acquisition also waits in the map for the sweep, and counting those
|
||||
would make this rate track ordinary cleanup instead of wasted work.
|
||||
- `acquire_aborts_partial` rising → the expensive form of an abort, where partly
|
||||
built maps were thrown away. `acquire_aborts` alone does not separate the cheap
|
||||
case from this one.
|
||||
|
||||
@@ -108,10 +108,12 @@
|
||||
#include <xrpl/telemetry/ValidationTracker.h>
|
||||
#endif
|
||||
|
||||
#include <xrpl/basics/chrono.h>
|
||||
#include <xrpl/beast/utility/Journal.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <charconv>
|
||||
#include <chrono>
|
||||
#include <cstdint>
|
||||
#include <limits>
|
||||
#include <optional>
|
||||
@@ -638,6 +640,62 @@ public:
|
||||
return std::pair{*first, *last};
|
||||
}
|
||||
|
||||
/**
|
||||
* Days from @p now until @p deadline, signed, for a NetClock deadline.
|
||||
*
|
||||
* NetClock's rep is std::uint32_t, so subtracting two of its time_points
|
||||
* in the clock's own duration wraps as soon as the deadline is in the
|
||||
* past: a deadline one day gone reads as about +49,709 days. The
|
||||
* subtraction here happens in std::int64_t, which holds the whole
|
||||
* NetClock range on either side of zero, so no case can wrap and no
|
||||
* range check is needed.
|
||||
*
|
||||
* The result is deliberately **not** clamped at zero. A negative reading
|
||||
* is the only signal that the deadline has already passed, and clamping
|
||||
* would make an expired deadline look like one that expires today.
|
||||
*
|
||||
* NetClock::time_point::max() is the sentinel for "no deadline" - the
|
||||
* value ValidatorList gives a list loaded from the config file. It is
|
||||
* reported as positive infinity rather than as a count of days, because
|
||||
* a finite reading of ~40,000 days is indistinguishable from the wrap
|
||||
* this helper exists to prevent, while infinity is below no threshold
|
||||
* and above every real deadline.
|
||||
*
|
||||
* Defined inline for the same reason as sanitiseHandler().
|
||||
*
|
||||
* @param deadline The moment being counted down to, on NetClock.
|
||||
* @param now The reference time, on the same clock.
|
||||
* @return Days remaining, negative once @p deadline has passed, or
|
||||
* positive infinity when @p deadline is the no-deadline sentinel.
|
||||
*
|
||||
* @note Pure and reentrant: holds no state and performs no I/O.
|
||||
* @note Fractional, not truncated: half a day before the deadline reads
|
||||
* 0.5 and half a day after reads -0.5.
|
||||
*
|
||||
* Example:
|
||||
* @code
|
||||
* // The offsets are built from NetClock::duration so that adding one
|
||||
* // keeps the clock's own time_point type.
|
||||
* auto const week = NetClock::duration{7 * 86'400};
|
||||
* auto const day = NetClock::duration{86'400};
|
||||
* daysUntil(now + week, now); // 7.0
|
||||
* daysUntil(now - day, now); // -1.0 -- already expired
|
||||
* daysUntil(NetClock::time_point::max(), now); // +inf -- never expires
|
||||
* @endcode
|
||||
*/
|
||||
[[nodiscard]] static constexpr double
|
||||
daysUntil(NetClock::time_point deadline, NetClock::time_point now) noexcept
|
||||
{
|
||||
if (deadline == NetClock::time_point::max())
|
||||
return std::numeric_limits<double>::infinity();
|
||||
|
||||
auto const deadlineSecs = static_cast<std::int64_t>(deadline.time_since_epoch().count());
|
||||
auto const nowSecs = static_cast<std::int64_t>(now.time_since_epoch().count());
|
||||
|
||||
using FractionalDays = std::chrono::duration<double, std::chrono::days::period>;
|
||||
return FractionalDays{std::chrono::seconds{deadlineSecs - nowSecs}}.count();
|
||||
}
|
||||
|
||||
/**
|
||||
* Record a job enqueued event.
|
||||
* @param jobType The job type name (e.g. "ledgerData").
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
/**
|
||||
* GTest unit tests for MetricsRegistry.
|
||||
*
|
||||
* Four groups. The first three drive the pure static helpers, which are
|
||||
* Five groups. The first four drive the pure static helpers, which are
|
||||
* constexpr inline in the header and so need nothing on the link line. The
|
||||
* fourth drives a real registry object.
|
||||
* fifth drives a real registry object.
|
||||
*
|
||||
* 1. sanitiseHandler() — the `handler` label sanitiser.
|
||||
*
|
||||
@@ -15,14 +15,19 @@
|
||||
* real producer, xrpl::to_string(RangeSet), rather than restating its
|
||||
* format.
|
||||
*
|
||||
* 4. The registry lifecycle — construction, stop(), and the record and
|
||||
* 4. daysUntil() — the signed deadline arithmetic behind unl_expiry_days.
|
||||
* NetClock counts seconds in a uint32_t, so the interesting cases are the
|
||||
* ones on the far side of the deadline, where an unsigned subtraction
|
||||
* wraps to about 49,700 days instead of going negative.
|
||||
*
|
||||
* 5. The registry lifecycle — construction, stop(), and the record and
|
||||
* increment methods. Every test here runs in **both** builds: the core
|
||||
* is compiled into xrpl.libxrpl, which this binary links either way, so
|
||||
* with telemetry on these tests drive a real OTel pipeline and with it
|
||||
* off they drive the no-op stubs. An assertion that holds in only one
|
||||
* build carries its own #ifdef and says which build it pins.
|
||||
*
|
||||
* What group 4 pins about stop(), and what it does not:
|
||||
* What group 5 pins about stop(), and what it does not:
|
||||
*
|
||||
* stop() stores Phase::Stopped before it destroys the SDK provider, and every
|
||||
* record method reads that phase through recording() first. Without the store,
|
||||
@@ -34,9 +39,14 @@
|
||||
* what would catch a regression in the memory itself.
|
||||
*/
|
||||
|
||||
// cspell:ignore Wmissing
|
||||
// A comment below names the compiler flag -Wmissing-designated-field-initializers.
|
||||
// cspell's compound-word splitter emits the subword "Wmissing"; ignore it here.
|
||||
|
||||
#include <xrpl/telemetry/MetricsRegistry.h>
|
||||
|
||||
#include <xrpl/basics/RangeSet.h>
|
||||
#include <xrpl/basics/chrono.h>
|
||||
|
||||
#include <gtest/gtest.h>
|
||||
|
||||
@@ -591,7 +601,134 @@ TEST(MetricsRegistryParseLedgerRange, reads_back_what_the_real_producer_wrote)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 4. The registry lifecycle.
|
||||
// 4. daysUntil() — the signed deadline arithmetic behind unl_expiry_days.
|
||||
//
|
||||
// The property under test is the sign, not the magnitude. NetClock's rep is
|
||||
// std::uint32_t, so subtracting two of its time_points in the clock's own
|
||||
// duration wraps as soon as the deadline is in the past: a list that expired
|
||||
// yesterday reports about +49,709 days, which reads as the healthiest possible
|
||||
// value on a panel whose thresholds colour high numbers green. Every case
|
||||
// below therefore pins an exact value on the far side of the deadline, plus
|
||||
// the "never expires" sentinel, which must stay above every finite reading.
|
||||
//
|
||||
// Each expected value is a hand-worked literal with an exact binary
|
||||
// representation, so EXPECT_EQ on a double is sound here and no case restates
|
||||
// the implementation's own expression.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
namespace {
|
||||
|
||||
/**
|
||||
* Build a NetClock time_point from a count of seconds since the XRPL epoch.
|
||||
*/
|
||||
constexpr xrpl::NetClock::time_point
|
||||
netTime(std::uint32_t seconds)
|
||||
{
|
||||
return xrpl::NetClock::time_point{xrpl::NetClock::duration{seconds}};
|
||||
}
|
||||
|
||||
// An arbitrary "now", far enough from both ends of uint32_t that a case may
|
||||
// move the deadline either side of it.
|
||||
constexpr auto kNow = netTime(800'000'000);
|
||||
|
||||
constexpr auto kDay = 86'400U;
|
||||
|
||||
// Ahead of the deadline: ordinary positive readings.
|
||||
static_assert(Registry::daysUntil(netTime(800'000'000 + 7 * kDay), kNow) == 7.0);
|
||||
static_assert(Registry::daysUntil(netTime(800'000'000 + kDay / 2), kNow) == 0.5);
|
||||
static_assert(Registry::daysUntil(kNow, kNow) == 0.0);
|
||||
|
||||
// Past the deadline: the readings the unsigned subtraction used to wrap.
|
||||
static_assert(Registry::daysUntil(netTime(800'000'000 - kDay), kNow) == -1.0);
|
||||
static_assert(Registry::daysUntil(netTime(800'000'000 - kDay / 2), kNow) == -0.5);
|
||||
static_assert(Registry::daysUntil(netTime(800'000'000 - 30 * kDay), kNow) == -30.0);
|
||||
|
||||
// The sentinel the config path sets for a list that never expires.
|
||||
static_assert(
|
||||
Registry::daysUntil(xrpl::NetClock::time_point::max(), kNow) ==
|
||||
std::numeric_limits<double>::infinity());
|
||||
|
||||
} // namespace
|
||||
|
||||
TEST(MetricsRegistryDaysUntil, a_passed_deadline_reads_negative_not_wrapped)
|
||||
{
|
||||
// The defect this helper exists for. Subtracting in NetClock's own
|
||||
// unsigned duration turns "expired a day ago" into "+49,709 days left",
|
||||
// so a threshold watching for a small number never trips.
|
||||
EXPECT_EQ(Registry::daysUntil(netTime(800'000'000 - kDay), kNow), -1.0);
|
||||
EXPECT_LT(Registry::daysUntil(netTime(800'000'000 - kDay), kNow), 0.0);
|
||||
|
||||
// One second past the deadline is already negative, so the sign flips at
|
||||
// the deadline rather than somewhere inside the first day.
|
||||
EXPECT_LT(Registry::daysUntil(netTime(800'000'000 - 1), kNow), 0.0);
|
||||
EXPECT_EQ(Registry::daysUntil(kNow, kNow), 0.0);
|
||||
}
|
||||
|
||||
TEST(MetricsRegistryDaysUntil, readings_decrease_monotonically_as_the_deadline_passes)
|
||||
{
|
||||
// Walks the deadline from a week out to a week gone in hour steps and
|
||||
// requires every reading to be strictly below the one before it. A wrap
|
||||
// anywhere in that walk is a jump upward, so this fails on the exact
|
||||
// second the old arithmetic went wrong rather than only on sampled points.
|
||||
constexpr auto kHour = 3'600U;
|
||||
auto previous = Registry::daysUntil(netTime(800'000'000 + 7 * kDay), kNow);
|
||||
|
||||
for (std::uint32_t offset = 7 * kDay - kHour; offset > 0; offset -= kHour)
|
||||
{
|
||||
auto const ahead = Registry::daysUntil(netTime(800'000'000 + offset), kNow);
|
||||
EXPECT_LT(ahead, previous) << "ahead of deadline by " << offset << "s";
|
||||
previous = ahead;
|
||||
}
|
||||
|
||||
for (std::uint32_t offset = 0; offset <= 7 * kDay; offset += kHour)
|
||||
{
|
||||
auto const behind = Registry::daysUntil(netTime(800'000'000 - offset), kNow);
|
||||
EXPECT_LT(behind, previous) << "past deadline by " << offset << "s";
|
||||
previous = behind;
|
||||
}
|
||||
|
||||
// Ends a full week past the deadline, which is where the wrap was largest.
|
||||
EXPECT_EQ(Registry::daysUntil(netTime(800'000'000 - 7 * kDay), kNow), -7.0);
|
||||
}
|
||||
|
||||
TEST(MetricsRegistryDaysUntil, never_expires_sentinel_outranks_every_finite_reading)
|
||||
{
|
||||
// A list loaded from the config file carries time_point::max() rather than
|
||||
// a date. Reported as a number it would be about 40,000 days, which is
|
||||
// indistinguishable from the wrap above; infinity says "no deadline" and
|
||||
// cannot be mistaken for a count of days.
|
||||
auto const never = Registry::daysUntil(xrpl::NetClock::time_point::max(), kNow);
|
||||
|
||||
EXPECT_EQ(never, std::numeric_limits<double>::infinity());
|
||||
EXPECT_GT(never, Registry::daysUntil(netTime(4'000'000'000U), kNow));
|
||||
|
||||
// The reason the value matters: every "expiring soon" comparison must be
|
||||
// false for it, and every "already expired" comparison too.
|
||||
EXPECT_FALSE(never < 0.0);
|
||||
EXPECT_FALSE(never < std::numeric_limits<double>::max());
|
||||
}
|
||||
|
||||
TEST(MetricsRegistryDaysUntil, the_sentinel_is_the_one_the_validator_list_sets)
|
||||
{
|
||||
// Pins the sentinel to the type's own maximum rather than to a literal, so
|
||||
// a change to NetClock's width cannot leave this test agreeing with a
|
||||
// value the config path no longer produces.
|
||||
constexpr auto kMaxSeconds = std::numeric_limits<xrpl::NetClock::rep>::max();
|
||||
|
||||
EXPECT_EQ(xrpl::NetClock::time_point::max().time_since_epoch().count(), kMaxSeconds);
|
||||
EXPECT_EQ(
|
||||
Registry::daysUntil(netTime(kMaxSeconds), kNow), std::numeric_limits<double>::infinity());
|
||||
|
||||
// One second below the sentinel is a real date, so it reports a number of
|
||||
// days rather than the sentinel. Without this the sentinel branch could be
|
||||
// swallowing a whole range of legitimate deadlines.
|
||||
EXPECT_LT(
|
||||
Registry::daysUntil(netTime(kMaxSeconds - 1), kNow),
|
||||
std::numeric_limits<double>::infinity());
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 5. The registry lifecycle.
|
||||
//
|
||||
// The core is compiled into xrpl.libxrpl, which this binary links in both
|
||||
// builds, so every test below runs in both. The headers here serve only this
|
||||
|
||||
@@ -154,7 +154,12 @@ public:
|
||||
}
|
||||
|
||||
/**
|
||||
* Record that an idle acquisition was evicted by the sweep.
|
||||
* Record that the sweep evicted an acquisition that had not finished.
|
||||
*
|
||||
* An acquisition that already completed or failed sits in the map only
|
||||
* until the next sweep, and was counted by recordCompletion() or
|
||||
* recordGiveUp() when it ended. Callers must exclude those, or this count
|
||||
* measures map cleanup instead of discarded work.
|
||||
*/
|
||||
void
|
||||
recordSweepEviction()
|
||||
@@ -245,7 +250,10 @@ public:
|
||||
}
|
||||
|
||||
/**
|
||||
* Return the number of idle acquisitions evicted by the sweep.
|
||||
* Return the number of unfinished acquisitions evicted by the sweep.
|
||||
*
|
||||
* Every one of these threw away whatever had been fetched so far, so this
|
||||
* counts wasted work rather than how many entries the sweep removed.
|
||||
*/
|
||||
[[nodiscard]] std::uint64_t
|
||||
getSweepEvictions() const
|
||||
@@ -295,7 +303,7 @@ private:
|
||||
telemetry::Counter<> completions_;
|
||||
|
||||
/**
|
||||
* Idle acquisitions evicted by the sweep.
|
||||
* Unfinished acquisitions evicted by the sweep.
|
||||
*/
|
||||
telemetry::Counter<> sweepEvictions_;
|
||||
};
|
||||
|
||||
@@ -393,11 +393,15 @@ public:
|
||||
}
|
||||
else if ((la + std::chrono::minutes(1)) < start)
|
||||
{
|
||||
// Only an unfinished acquisition loses work here, and its
|
||||
// work then restarts. One that already completed or
|
||||
// failed was counted when it ended, so counting it again
|
||||
// would bury the wasteful case in ordinary map cleanup.
|
||||
auto const& inbound = *it->second;
|
||||
if (!inbound.isComplete() && !inbound.isFailed())
|
||||
app_.getAcquireStats().recordSweepEviction();
|
||||
|
||||
stuffToSweep.push_back(it->second);
|
||||
// An eviction here discards whatever the acquisition had
|
||||
// built, so the work restarts. Counted to tell that apart
|
||||
// from an acquisition that ended on its own.
|
||||
app_.getAcquireStats().recordSweepEviction();
|
||||
// shouldn't cause the actual final delete
|
||||
// since we are holding a reference in the vector.
|
||||
it = ledgers_.erase(it);
|
||||
|
||||
@@ -817,14 +817,15 @@ AppMetricGauges::registerValidatorHealthGauge()
|
||||
observe("unl_blocked", app.getOPs().isUNLBlocked() ? 1.0 : 0.0);
|
||||
observe("validation_quorum", static_cast<double>(app.getValidators().quorum()));
|
||||
|
||||
// Days until UNL list expiry (-1 if no expiry known).
|
||||
// Days until UNL list expiry. Negative once the list has
|
||||
// expired, +inf for a config-listed list that never expires,
|
||||
// and -1 when no published list has been fetched at all.
|
||||
auto const expiry = app.getValidators().expires();
|
||||
if (expiry)
|
||||
{
|
||||
auto const now = app.getTimeKeeper().closeTime();
|
||||
auto const diffHours =
|
||||
std::chrono::duration_cast<std::chrono::hours>(*expiry - now).count();
|
||||
observe("unl_expiry_days", static_cast<double>(diffHours) / 24.0);
|
||||
observe(
|
||||
"unl_expiry_days",
|
||||
MetricsRegistry::daysUntil(*expiry, app.getTimeKeeper().closeTime()));
|
||||
}
|
||||
else
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user