fix(telemetry): address the PR review findings

Four defects from the automated review on PR #7875, each verified against the
current tree before fixing (one further comment, the row-63 dashboard overlap,
was already fixed by an earlier commit and needed nothing).

1. Rule J could not detect an instrument-kind mismatch. instrument_kinds() wrote
   `kinds[wire] = ...`, so a wire name created through two different factories
   kept only the kind visited last and whichever emit site the file walk reached
   last silently decided the verdict. It now collects a set per name and reports
   the conflict itself -- one name exporting two instruments is the defect, and
   no suffix can be correct for both. Added a regression test that builds a name
   as both a counter and an observable gauge and asserts the message names both.

2. A duplicate connection was reported as `tls_fail`. The TLS handshake had in
   fact succeeded; PeerFinder simply already held a slot for that address, which
   is ordinary churn on a healthy node. Conflating the two made a rising
   `tls_fail` unreadable -- it could mean unreachable peers or merely a busy
   PeerFinder, and those need opposite responses. Added a distinct `duplicate`
   outcome and carried the widened vocabulary through every place that
   enumerates it: the panel description, both filter descriptions, the runbook
   branch table, the runbook outcome list and the expected_spans note. The
   `dial_outcome` template variable is a label_values() query, so it picks the
   new value up on its own.

3. ConnectAttempt::onShutdown had no `operation_aborted` guard, unlike the five
   other handlers in the same file. A clean teardown was therefore counted as
   `upgrade_fail`, inflating that outcome on any node shutting down with dials in
   flight.

4. ValidatorSite used the raw configured URI as a Prometheus label.
   [validator_list_sites] accepts credentials in the URI and ParsedUrl keeps them
   in username/password, so a configured `https://user:pass@host` would have
   copied the secret into a metric label and on into the collector, Prometheus
   and every dashboard. The label is now rebuilt from scheme, host, port and
   path -- everything needed to tell one site apart, and nothing more.

Verified: naming checker exits 0 with Rule J still passing all 40 real
instrument names; its unit tests now number 139 and all pass; 15 dashboards
validate; both workload JSON files parse; clang-tidy over the full compile
database reports no finding on either changed .cpp; pre-commit passes.

Not verified: not compiled. Item 4 introduces string concatenation and item 2 a
new constexpr, so CI's build is the first real check on both.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Pratik Mankawde
2026-07-28 17:04:28 +01:00
parent 0ddb4e2686
commit f649670ef7
8 changed files with 96 additions and 29 deletions

View File

@@ -393,12 +393,24 @@ ValidatorSite::reportFetchOutcome(
// it is always non-null. Unlike activeResource (reset once a fetch
// completes) it also keeps the URI exactly as configured, which keeps
// the time series stable when a site redirects.
//
// The label is rebuilt from the parsed parts rather than using the raw
// configured URI: [validator_list_sites] accepts credentials in the URI,
// and ParsedUrl keeps them in username/password. Emitting the raw string
// would copy them into a metric label, from which they would reach the
// collector, Prometheus and every dashboard. Scheme, host, port and path
// are all a reader needs to tell one site from another.
auto const& url = sites_[siteIdx].loadedResource->pUrl;
std::string siteLabel = url.scheme + "://" + url.domain;
if (url.port)
siteLabel += ":" + std::to_string(*url.port);
siteLabel += url.path;
XRPL_METRIC_COUNTER_INC_LABELED(
app_,
telemetry::metric::unlFetchTotal,
"Validator list fetch attempts, by site and outcome",
{{telemetry::label::site, std::string(sites_[siteIdx].loadedResource->uri)},
{telemetry::label::outcome, std::string(outcome)}});
{{telemetry::label::site, siteLabel}, {telemetry::label::outcome, std::string(outcome)}});
}
void

View File

@@ -354,7 +354,10 @@ ConnectAttempt::onHandshake(error_code ec)
if (!overlay_.peerFinder().onConnected(
slot_, beast::IPAddressConversion::fromAsio(localEndpoint)))
{
reportOutcome(telemetry::peer_span::val::tlsFail);
// Not a TLS failure: the handshake succeeded and PeerFinder simply
// already holds a slot for this address. Reporting it as tls_fail
// conflated ordinary dial churn with peers we cannot speak to.
reportOutcome(telemetry::peer_span::val::duplicate);
fail("Duplicate connection");
return;
}
@@ -460,6 +463,16 @@ ConnectAttempt::onShutdown(error_code ec)
return;
}
// A cancelled shutdown is us tearing the attempt down, not the peer failing
// it. Every other handler here guards this; without the same guard an
// ordinary overlay stop was counted as upgrade_fail, inflating that outcome
// on any node that shuts down while dials are in flight.
if (ec == boost::asio::error::operation_aborted)
{
close();
return;
}
if (ec != boost::asio::error::eof)
{
reportOutcome(telemetry::peer_span::val::upgradeFail);

View File

@@ -80,21 +80,31 @@ namespace val {
/**
* peer.dial outcome values.
*
* The identical five slugs `ConnectAttempt::reportOutcome` already passes to
* the `overlay_connect_total` counter, defined here so the span and the
* counter cannot drift apart: the dial state machine names its outcome once
* and both signals receive that same value.
* The identical slugs `ConnectAttempt::reportOutcome` passes to the
* `overlay_connect_total` counter, defined here so the span and the counter
* cannot drift apart: the dial state machine names its outcome once and both
* signals receive that same value.
*
* - connected: the peer was activated and added to the overlay.
* - tcp_fail: the TCP connect or local-endpoint read failed.
* - tls_fail: the TLS handshake, slot check or shared value failed.
* - tls_fail: the TLS handshake or the shared-value exchange failed.
* - duplicate: TLS succeeded but PeerFinder already holds a slot for this
* address, so the attempt was redundant rather than faulty.
* - upgrade_fail: TLS succeeded but the HTTP upgrade, protocol negotiation
* or activation was rejected.
* - timeout: the attempt never reached any terminal state in time.
*
* `duplicate` is separate from `tls_fail` on purpose. Dialling an address we
* are already connected to is normal churn on a healthy node, while a TLS
* failure means the peer could not be spoken to at all. Reporting both as
* `tls_fail` made a rising TLS-failure count unreadable: it could equally mean
* broken peers or merely a busy PeerFinder, and the two need opposite
* responses.
*/
inline constexpr auto connected = makeStr("connected");
inline constexpr auto tcpFail = makeStr("tcp_fail");
inline constexpr auto tlsFail = makeStr("tls_fail");
inline constexpr auto duplicate = makeStr("duplicate");
inline constexpr auto upgradeFail = makeStr("upgrade_fail");
inline constexpr auto timeout = makeStr("timeout");
} // namespace val