mirror of
https://github.com/XRPLF/rippled.git
synced 2026-08-22 14:50:54 +00:00
fix(telemetry): address the PR review findings
Four defects from the automated review on PR #7875, each verified against the current tree before fixing (one further comment, the row-63 dashboard overlap, was already fixed by an earlier commit and needed nothing). 1. Rule J could not detect an instrument-kind mismatch. instrument_kinds() wrote `kinds[wire] = ...`, so a wire name created through two different factories kept only the kind visited last and whichever emit site the file walk reached last silently decided the verdict. It now collects a set per name and reports the conflict itself -- one name exporting two instruments is the defect, and no suffix can be correct for both. Added a regression test that builds a name as both a counter and an observable gauge and asserts the message names both. 2. A duplicate connection was reported as `tls_fail`. The TLS handshake had in fact succeeded; PeerFinder simply already held a slot for that address, which is ordinary churn on a healthy node. Conflating the two made a rising `tls_fail` unreadable -- it could mean unreachable peers or merely a busy PeerFinder, and those need opposite responses. Added a distinct `duplicate` outcome and carried the widened vocabulary through every place that enumerates it: the panel description, both filter descriptions, the runbook branch table, the runbook outcome list and the expected_spans note. The `dial_outcome` template variable is a label_values() query, so it picks the new value up on its own. 3. ConnectAttempt::onShutdown had no `operation_aborted` guard, unlike the five other handlers in the same file. A clean teardown was therefore counted as `upgrade_fail`, inflating that outcome on any node shutting down with dials in flight. 4. ValidatorSite used the raw configured URI as a Prometheus label. [validator_list_sites] accepts credentials in the URI and ParsedUrl keeps them in username/password, so a configured `https://user:pass@host` would have copied the secret into a metric label and on into the collector, Prometheus and every dashboard. The label is now rebuilt from scheme, host, port and path -- everything needed to tell one site apart, and nothing more. Verified: naming checker exits 0 with Rule J still passing all 40 real instrument names; its unit tests now number 139 and all pass; 15 dashboards validate; both workload JSON files parse; clang-tidy over the full compile database reports no finding on either changed .cpp; pre-commit passes. Not verified: not compiled. Item 4 introduces string concatenation and item 2 a new constexpr, so CI's build is the first real check on both. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -393,12 +393,24 @@ ValidatorSite::reportFetchOutcome(
|
||||
// it is always non-null. Unlike activeResource (reset once a fetch
|
||||
// completes) it also keeps the URI exactly as configured, which keeps
|
||||
// the time series stable when a site redirects.
|
||||
//
|
||||
// The label is rebuilt from the parsed parts rather than using the raw
|
||||
// configured URI: [validator_list_sites] accepts credentials in the URI,
|
||||
// and ParsedUrl keeps them in username/password. Emitting the raw string
|
||||
// would copy them into a metric label, from which they would reach the
|
||||
// collector, Prometheus and every dashboard. Scheme, host, port and path
|
||||
// are all a reader needs to tell one site from another.
|
||||
auto const& url = sites_[siteIdx].loadedResource->pUrl;
|
||||
std::string siteLabel = url.scheme + "://" + url.domain;
|
||||
if (url.port)
|
||||
siteLabel += ":" + std::to_string(*url.port);
|
||||
siteLabel += url.path;
|
||||
|
||||
XRPL_METRIC_COUNTER_INC_LABELED(
|
||||
app_,
|
||||
telemetry::metric::unlFetchTotal,
|
||||
"Validator list fetch attempts, by site and outcome",
|
||||
{{telemetry::label::site, std::string(sites_[siteIdx].loadedResource->uri)},
|
||||
{telemetry::label::outcome, std::string(outcome)}});
|
||||
{{telemetry::label::site, siteLabel}, {telemetry::label::outcome, std::string(outcome)}});
|
||||
}
|
||||
|
||||
void
|
||||
|
||||
@@ -354,7 +354,10 @@ ConnectAttempt::onHandshake(error_code ec)
|
||||
if (!overlay_.peerFinder().onConnected(
|
||||
slot_, beast::IPAddressConversion::fromAsio(localEndpoint)))
|
||||
{
|
||||
reportOutcome(telemetry::peer_span::val::tlsFail);
|
||||
// Not a TLS failure: the handshake succeeded and PeerFinder simply
|
||||
// already holds a slot for this address. Reporting it as tls_fail
|
||||
// conflated ordinary dial churn with peers we cannot speak to.
|
||||
reportOutcome(telemetry::peer_span::val::duplicate);
|
||||
fail("Duplicate connection");
|
||||
return;
|
||||
}
|
||||
@@ -460,6 +463,16 @@ ConnectAttempt::onShutdown(error_code ec)
|
||||
return;
|
||||
}
|
||||
|
||||
// A cancelled shutdown is us tearing the attempt down, not the peer failing
|
||||
// it. Every other handler here guards this; without the same guard an
|
||||
// ordinary overlay stop was counted as upgrade_fail, inflating that outcome
|
||||
// on any node that shuts down while dials are in flight.
|
||||
if (ec == boost::asio::error::operation_aborted)
|
||||
{
|
||||
close();
|
||||
return;
|
||||
}
|
||||
|
||||
if (ec != boost::asio::error::eof)
|
||||
{
|
||||
reportOutcome(telemetry::peer_span::val::upgradeFail);
|
||||
|
||||
@@ -80,21 +80,31 @@ namespace val {
|
||||
/**
|
||||
* peer.dial outcome values.
|
||||
*
|
||||
* The identical five slugs `ConnectAttempt::reportOutcome` already passes to
|
||||
* the `overlay_connect_total` counter, defined here so the span and the
|
||||
* counter cannot drift apart: the dial state machine names its outcome once
|
||||
* and both signals receive that same value.
|
||||
* The identical slugs `ConnectAttempt::reportOutcome` passes to the
|
||||
* `overlay_connect_total` counter, defined here so the span and the counter
|
||||
* cannot drift apart: the dial state machine names its outcome once and both
|
||||
* signals receive that same value.
|
||||
*
|
||||
* - connected: the peer was activated and added to the overlay.
|
||||
* - tcp_fail: the TCP connect or local-endpoint read failed.
|
||||
* - tls_fail: the TLS handshake, slot check or shared value failed.
|
||||
* - tls_fail: the TLS handshake or the shared-value exchange failed.
|
||||
* - duplicate: TLS succeeded but PeerFinder already holds a slot for this
|
||||
* address, so the attempt was redundant rather than faulty.
|
||||
* - upgrade_fail: TLS succeeded but the HTTP upgrade, protocol negotiation
|
||||
* or activation was rejected.
|
||||
* - timeout: the attempt never reached any terminal state in time.
|
||||
*
|
||||
* `duplicate` is separate from `tls_fail` on purpose. Dialling an address we
|
||||
* are already connected to is normal churn on a healthy node, while a TLS
|
||||
* failure means the peer could not be spoken to at all. Reporting both as
|
||||
* `tls_fail` made a rising TLS-failure count unreadable: it could equally mean
|
||||
* broken peers or merely a busy PeerFinder, and the two need opposite
|
||||
* responses.
|
||||
*/
|
||||
inline constexpr auto connected = makeStr("connected");
|
||||
inline constexpr auto tcpFail = makeStr("tcp_fail");
|
||||
inline constexpr auto tlsFail = makeStr("tls_fail");
|
||||
inline constexpr auto duplicate = makeStr("duplicate");
|
||||
inline constexpr auto upgradeFail = makeStr("upgrade_fail");
|
||||
inline constexpr auto timeout = makeStr("timeout");
|
||||
} // namespace val
|
||||
|
||||
Reference in New Issue
Block a user