fix(telemetry): correct RPC and gRPC span status reporting

Six related defects in the RPC/gRPC span surface, all cases where a failure
was recorded as success or an attribute was missing on an error path.

GRPCServer: the non-exception branch set the span Ok unconditionally, then
sent a possibly-failed grpc::Status. The handler can return a non-OK status
without throwing, so every failed call traced as successful. Status now
follows result.second, with the error message as the span description.

ServerHandler: eight per-item error branches appended an error reply without
recording that the request failed. Batch responses and ripplerpc < 3.0 always
carry HTTP 200, so those failures were invisible and an entirely failed batch
ended its span as successful. Added an appendItemError() helper next to the
existing httpReplyError() lambda and routed all eight sites through it, so the
flag cannot be forgotten at a new call site.

ServerHandler: the early-return validation paths set the span error but not the
rpc_status attribute. Added it to httpReplyError() so every such path gets it.

RPCHandler: the fillHandler error path set only command and rpc_status, while
callMethod sets command, version and rpc_role. Error spans were therefore not
filterable by API version or role. The error path now mirrors that set.

RPCHandler: resolveCommandSpanName() checked only that command/method were
present, not that they agreed, while fillHandler rejects a mismatch as
rpcUNKNOWN_COMMAND. A request supplying both with different values was labelled
with one of the two names, misattributing the error to a command that never
dispatched. It now mirrors fillHandler's rule and collapses to "unknown".

ServerHandler: processRequest returned bool solely so the caller could set its
span status. Telemetry should read state, not shape the signature of the code it
observes, so the signature returns to void and rpc.process sets its own status
from spanHadError. The enclosing rpc.http_request span now leaves status unset:
the OTel spec has instrumentation leave status unset unless the operation itself
errored, and reserves Ok for an operator asserting verified success.
This commit is contained in:
Pratik Mankawde
2026-07-29 16:02:19 +01:00
parent 646ff1bc5a
commit 3715b7a2a3
4 changed files with 67 additions and 36 deletions

View File

@@ -243,8 +243,19 @@ GRPCServerImpl::CallData<Request, Response>::process(std::shared_ptr<JobQueue::C
{
std::pair<Response, grpc::Status> result = handler_(context);
setIsUnlimited(result.first, isUnlimited);
span.setAttribute(grpc_span::attr::grpcStatus, grpc_span::val::success);
span.setOk();
// The handler can return a non-OK status without throwing, so
// the span status must follow result.second rather than assume
// success — otherwise every failed call traces as OK.
if (result.second.ok())
{
span.setAttribute(grpc_span::attr::grpcStatus, grpc_span::val::success);
span.setOk();
}
else
{
span.setAttribute(grpc_span::attr::grpcStatus, grpc_span::val::error);
span.setError(result.second.error_message());
}
responder_.Finish(result.first, result.second, this);
}
}