Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics

Phase-10 independently instrumented the peer object-fetch path while this
branch instrumented fresh-node sync, so the two overlapped in three places.
Resolved by keeping each side's stronger implementation rather than shipping
both.

Per-job-type waiting/running/deferred existed twice. Phase-10's version
survives: it publishes per-type gauges from JobQueue::collect(), which
snapshots under the queue lock and publishes after releasing it, a
deliberate lock-order fix against the collector's own lock. This branch's
jobq_backlog gauge and the JobQueue::getJobTypeCounts() accessor that fed it
are removed, along with their panels, assertions and reference rows.
jobq_saturation stays: it reports the whole worker pool, which phase-10 has
no equivalent for.

The histogram view helper also existed twice with identical bodies under two
names; one survives, and the microsecond ladder is now the named array
rather than boundaries repeated inline. The job_type label was declared
twice, once as a file-local constant invisible to the naming check; both it
and handler now come from the constants header.

Two things phase-10 adds are complementary, not duplicates, and are kept as
they are: the handler label, which separates the two request kinds that both
report as the same job type, and getobject_rejected_total, which counts
malformed requests where this branch's serve_refused_total counts requests
this node declined to serve.

Also fixes two naming-check failures that pre-date this merge on phase-10.
The check derived label keys only from namespaced constants, so it could not
see the per-subsystem headers' flat k-prefixed style and rejected dashboards
querying labels the code really emits. It now reads both styles, with the
enforcement rules unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

.github/scripts/otel-naming/check_otel_naming.py

@@ -914,8 +914,17 @@ def metric_constants(root: Path) -> Tuple[Set[str], Set[str], Set[str]]:
     comment cannot seed the authoritative set (same reasoning as
     `strip_comments` for L1 spans).
 
-    A constant in none of the three namespaces is ignored rather than guessed
-    at, keeping the derivation conservative in the same direction as L1."""
+    Two header styles are recognised, because both are in use:
+
+    * Namespaced: constants sit inside `namespace metric` / `label` / `lval`,
+      and the enclosing namespace decides the bucket.
+    * Flat `k`-prefixed: a header with no such namespaces names the role in the
+      identifier instead -- `kLabelFoo` is a label key, `kResultFoo` and
+      `kReasonFoo` are label values. Used by the per-subsystem headers.
+
+    A constant that neither sits in one of the three namespaces nor carries a
+    recognised prefix is ignored rather than guessed at, keeping the derivation
+    conservative in the same direction as L1."""
     names: Set[str] = set()
     keys: Set[str] = set()
     values: Set[str] = set()
@@ -929,6 +938,14 @@ def metric_constants(root: Path) -> Tuple[Set[str], Set[str], Set[str]]:
             for block in namespace_spans(text, ns):
                 for m in METRIC_CONST_DEF.finditer(block):
                     bucket.add(m.group(2))
+        # Flat style: classify by identifier prefix. Only constants outside the
+        # namespaced blocks reach here, so a namespaced header is unaffected.
+        for m in METRIC_CONST_DEF.finditer(text):
+            ident, value = m.group(1), m.group(2)
+            if ident.startswith("kLabel"):
+                keys.add(value)
+            elif ident.startswith(("kResult", "kReason")):
+                values.add(value)
     return names, keys, values
 
 

.github/scripts/otel-naming/test_check_otel_naming.py

@@ -1359,7 +1359,7 @@ class InstrumentKindClassification(unittest.TestCase):
 class MetricPrefixFamilies(unittest.TestCase):
     def test_first_segment_is_the_family(self):
         self.assertEqual(
-            chk.metric_prefixes({"sync_state", "jobq_backlog", "unl_quorum"}),
+            chk.metric_prefixes({"sync_state", "jobq_saturation", "unl_quorum"}),
             {"sync_", "jobq_", "unl_"},
         )