rippled/docker/telemetry/workload/baselines/README.md

Performance Baselines

This directory holds the committed baseline file used by the OTel-driven regression gate.

How the gate works

After the validation suite runs, capture_timings.py queries Prometheus for the timings declared in ../regression-metrics.json and writes a timings.json. Then compare_to_baseline.py reads baseline-timings.json, ../regression-thresholds.json, and the captured timings.json. The comparator picks one of two modes automatically:

• Placeholder baseline ("placeholder": true or empty metrics): the comparator prints the captured timings JSON in exactly the format expected for this file, then exits 0 without gating. This is how we bootstrap the baseline. It prints that block only when the capture was complete — see An incomplete capture cannot seed a baseline.
• Populated baseline: the comparator diffs per-metric, enforces the thresholds (regression = current exceeds baseline on BOTH the percentage AND absolute bound), and exits non-zero on any regression. The single exception is a baseline that is not positive: the percentage change is undefined there, so the absolute bound decides alone. Without that fallback the AND gate would be unreachable and a 0 ms → 500 ms jump would be reported as "within bounds".

The regression gate runs against whatever workload profile run-full-validation.sh was invoked with. Capture and comparison are profile-agnostic — they only read Prometheus — so all existing profiles (full-validation, quick-smoke, stress) continue to work unchanged.

Current state: 19 metrics gate, on a baseline captured 2026-09-10

baseline-timings.json holds real captured values for the 19 keys the harness gates. Every value is the median of three clean CI runs — 34495527952, 34505215266 and 34507425933 — taken at a0385c53cb, profile full-validation, window 3m. The file records that provenance itself, in its source_runs and statistic fields.

The median of three is the point, not a detail. A single-run baseline is what disqualified five of the six excluded keys below: one sample carries no information about spread, and the bound is derived from that one sample alone.

It replaced a capture taken 2026-08-26, before the account-funding race was fixed. Phases whose funding silently failed submitted little or no traffic, so that capture recorded artificially low ledger and transaction timings. Once funding worked, span.ledger.build.p99 read 29.00 ms against its 9.109 ms baseline and turned the gate red on a run whose 200 span and metric checks all passed — which is why that key is now excluded.

Two earlier generations stay retrievable from this file's git history: a 2026-08-24 capture, and before it entries captured 2026-06-05 that were voided into a placeholder because they were captured against the spanmetrics ladder's old 1 ms floor, which made every sub-millisecond quantile in that capture bucket-edge arithmetic rather than a latency (a p95 of 0.95 ms is 0.95 × 1 ms). Because the comparator only flags a metric when the current value exceeds the baseline, a stale-high baseline passes everything silently, so those entries had to be dropped rather than left in place.

A placeholder must not outlive one run. CI stays green the whole time one stands, so an un-copied block is not a failure anyone will notice — it is a silent loss of regression coverage that looks identical to a passing gate. Voiding a baseline is the one hand edit that needs no CI block behind it. Setting one comes from a printed CI block, with a single documented exception — combining several runs into a median, which nothing automates yet (see Schema).

Absolute bounds are derived per metric, from the ladder

../regression-thresholds.json gives every gated key its own max_abs_increase_*, equal to hi_next − baseline: locate the baseline in the half-open bucket (lo, hi] of its ladder, take hi_next as the next edge above hi, and the bound is the distance from the baseline to hi_next. The trip point is therefore exactly hi_next — the gate fires only once the reading clears the bucket above the baseline's own.

That is what buys the guarantee. histogram_quantile returns a value interpolated inside whichever bucket the true quantile falls in, so any reading taken while the quantile is still in the baseline's bucket, or anywhere in the one immediately above, is at most hi_next and cannot fire. Firing needs the quantile to have moved at least two buckets up. A multiple of the enclosing bucket's width cannot deliver this, because once the quantile crosses hi the interpolation happens across the next bucket, which on this ladder is up to 8x wider — (0.5, 1] is 0.5 ms wide and (1, 5] is 4 ms wide. The full derivation, both ladders, and a per-key table of the arithmetic are in that file's _absolute_bound_derivation, _bucket_note (both ladders) and _derivation_table.

Two earlier generations of this bound were wrong, in opposite directions:

generation	bound	10x regression caught	single-crossing false positive reachable
flat	10 ms p50/p95, 15 ms p99, 20000 us job	5 / 28 keys	2 / 25 keys
2 × enclosing bucket width	per metric	28 / 28 keys	21 / 25 keys
hi_next − baseline (current)	per metric	19 / 19 keys	0 / 19 keys

The first two rows were measured when 28 and 25 keys were gated, by injecting a 10x regression into each gated key in turn against a real CI timings.json. The current row is derived, not sampled, and holds for every key on the 2026-09-10 baseline. A key's detection floor is exactly trip point ÷ baseline, because the gate fires when the reading exceeds the trip point and a kx regression reads k × baseline. So a 10x regression is caught precisely when the floor is under 10x, and the weakest floor on this baseline is 7.41x — see weakly guarded below.

That is a real improvement over the 2026-08-26 baseline, where job.acceptLedger.running.p95 had a 16.28x floor and was the one key a 10x regression missed. Its baseline rose from 6142.9 us to 15967.7 us while hi_next stayed at 100000 us, which pulled its floor down to 6.26x. Note the direction this can move in: floors are a property of where each baseline lands on the ladder, so a refresh changes sensitivity without anything about the code changing. Re-derive this table on every refresh.

The zero in the last column is by construction rather than by sampling: rule C in check_regression_bounds.py fails the build unless every trip point is exactly hi_next, and a trip point at a bucket edge cannot be crossed by interpolation inside that bucket.

The flat bound was calibrated for a 5-25 ms band the spans do not occupy: 18 of the 28 quantiles gated at the time sat below 1 ms, so it sat 1.15x to 2000x above the metric it guarded, and because the rule is an AND the percentage bound could never carry a regression alone. A 100x regression injected into span.ledger.store.p95 reported 0 regressions, exit 0. The second generation fixed the magnitude but kept an assumption that does not hold — that the reading's excursion is bounded by the enclosing bucket's width — which put 21 of 25 trip points inside the adjacent bucket, so a single legitimate bucket crossing could turn CI red.

Refreshing the baseline means re-deriving the bounds, because a refreshed value can land in a different bucket and so get a different hi_next. This is no longer a documentation-only rule: .github/scripts/telemetry/check_regression_bounds.py fails CI when a bound is not the one its own baseline implies, when a gated key has no override, when a baseline key is not declared by ../regression-metrics.json (or the reverse), when the percentage bound would become the operative one, and when a baseline carries the ladder-floor signature described below.

Which keys are only weakly guarded

The guarantee costs sensitivity where the ladder is coarse: the detection floor is hi_next / baseline, so a baseline sitting just above an edge is guarded loosely. Over the 2026-09-10 baseline the floor ranges 2.00x to 7.41x. Do not read these six as guarded:

key	baseline	fires at	floor	limiting ladder step
span.consensus.ledger_close.p95	0.6750 ms	5 ms	7.41x	1 ms → 5 ms
span.consensus.accept.p50	1.4364 ms	10 ms	6.96x	5 ms → 10 ms
job.acceptLedger.running.p95	15967.7 us	100000 us	6.26x	25000 us → 100000 us
span.rpc.ws_message.p95	0.8122 ms	5 ms	6.16x	1 ms → 5 ms
span.rpc.ws_message.p99	0.9873 ms	5 ms	5.06x	1 ms → 5 ms
span.tx.process.p99	0.9940 ms	5 ms	5.03x	1 ms → 5 ms

Four of the six are limited by the same 1 ms → 5 ms step, which is where this ladder is coarsest relative to how the spans actually behave. All six stay gated; the weak floor is recorded here so it is visible rather than surprising.

None of the six fires on an observed run of this workload — but the qualifier is load-bearing, and there is now a measurement behind it. Changing one line of the generated node config from [ips] to [ips_fixed], which holds peer connections open instead of treating the list as a discovery hint, moved span.consensus.ledger_close.p95 from 0.57 ms to 6.43 ms and tripped this gate, while every transaction-path metric fell. Nothing else in that commit touched the consensus path. So a weak floor is not the only way one of these keys reddens: a change to the cluster's topology is enough on its own, which is exactly why Refreshing the baseline treats a workload change as requiring a new baseline.

The fix is a 2 ms edge (ideally 3 ms as well) in the collector's spanmetrics buckets list plus the matching entries in kMillisecondBuckets, and 2000 us plus 50000 us edges in kMicrosecondBuckets. That work belongs to the branch that owns the ladders.

job.transaction.running.p95 and span.tx.process.p95 were on this list against the 2026-08-26 baseline, at 8.33x and 8.20x, and both dropped off it in the refresh — 2.65x and 2.00x now. span.tx.process.p95 is the tightest gated key on this baseline — 0.50 on baseline over trip point, and 0.76 on the observed maximum across the three source runs — so it is the first to re-measure if the gate reddens. The two ratios have different numerators; neither is the other.

span.tx.apply.p50 is absent from this table because it is no longer gated at all — see what all six excluded keys have in common. Beyond its variance it had a second, independent problem: its baseline of 0.00597 ms sat inside the ladder's first bucket (0, 0.01], so the reported figure was interpolation across that bucket, tracking the fraction of applies finishing under 10 us rather than a latency — the same mechanism that disqualified ledger.store below. Rule E did not flag it, correctly: the value is not quantile × first_edge exactly, so some mass does sit above 0.01 ms. Restoring the key therefore needs a finer low-end ladder as well as a spread-aware baseline.

Known exclusion: ledger.store is below the ladder's resolution

span.ledger.store is not gated. The 2026-08-24 capture returned p50/p95/p99 of exactly 0.005 / 0.0095 / 0.0099 ms, which is 0.5 / 0.95 / 0.99 × 0.01 ms — the ladder's first edge times the quantile, the signature of every sample landing in the first bucket. Those numbers are interpolation arithmetic on the bucket floor, not latencies. It is physically plausible: LedgerMaster.cpp:470 wraps an in-memory ledgerHistory_.insert, which completes in single-digit microseconds.

While all the mass stays under 10 us the reported quantile cannot move materially, so no absolute bound can gate this key — every ledger.store slowing from 2 us to 9 us, a 4.5x regression, leaves the reported value unchanged. Three keys that read as covered but cannot fire are worse than no keys, the same argument that excluded rpc.process, so they were removed from ../regression-metrics.json rather than left in with a bound that looks derived.

Restoring the key needs sub-10 us edges on the collector's spanmetrics ladder (for example 0.001ms and 0.005ms) plus the matching entries in HistogramBuckets.h. ledger.store presence is still asserted by ../expected_spans.json and docker/telemetry/integration-test.sh, and its rate is still on the ledger-operations dashboard; only the latency gate drops it. check_regression_bounds.py rule E fails the build if a key with this signature is gated again.

Known exclusion: ledger.validate p95 and p99 vary more than any bound can absorb

span.ledger.validate.p95 and .p99 are not gated. p50 still is. They are the first exclusion at quantile rather than span granularity, which is why ../regression-metrics.json grew an excluded_keys map — spans.names lists span names and _quantiles is shared across all of them, so removing two quantiles of one span cannot be expressed by deleting a name.

Measured across four CI runs, against the baseline in force when the two keys were excluded. The p50 row is the only one still gated. Its baseline was 0.0779 ms then and is 0.0598 ms now, with the same 0.25 ms trip point either way, so the argument is unchanged:

key	baseline at exclusion	trip point	observed min	observed max	spread
span.ledger.validate.p50 (kept)	0.0779 ms	0.25 ms	0.0484 ms	0.0778 ms	1.6x
span.ledger.validate.p95	0.2404 ms	0.5 ms	0.1281 ms	0.7500 ms	5.9x
span.ledger.validate.p99	1.0600 ms	10 ms	0.3875 ms	25.8750 ms	66.8x

Both excluded quantiles reach past their trip point on an ordinary run, so CI reddened twice with no code change: run 32867433073 read p95 = 0.7500 ms (+212%) and run 32862589645 read p99 = 25.8750 ms (+2341%). The two failures landed on different quantiles in different runs while the other quantile stayed well inside its bound in the same run — the signature of variance, not of a regression.

The mechanism is arrival timing, not slow code. The span opens only once a quorum-completing validation arrives (LedgerMaster.cpp:1003, inside checkAccept, past the tvc < minVal early return) and wraps the promotion work that follows — setValidated, setFull, setValidLedger, pendSaveValidated. Its duration therefore tracks when peer validations arrive in a 5-node cluster and what promotion then schedules, so a single slow consensus round dominates the tail of a 3 m rate window, and which round that is differs every run.

Widening the bound is not an option and must not be attempted. Tolerating 25.8750 ms against a 1.0600 ms baseline needs a bound of ~24.8 ms, i.e. a gate that fires at nothing a regression could plausibly reach. A bound that admits every healthy run's worst case admits every regression too. check_regression_bounds.py rule F fails the build if either key is re-gated with a bound while still listed in excluded_keys, and the per-key reasons in that map record this in full.

The general rule this exposed

hi_next − baseline is derived from the ladder, so it budgets for quantization noise — one bucket of interpolation headroom — and for nothing else. It knows nothing about how far the metric itself moves between runs on identical code. Where run-to-run workload variance is the larger term, the bound is simply the wrong size and the gate reddens on a healthy run.

Before gating any key, check its observed maximum across several runs against its trip point (baseline + bound), and gate it only if the maximum stays below that with margin. Spread alone proves nothing; it is spread relative to the trip point that decides. And because the trip point is derived from the baseline, a baseline that lands at the low end of a metric's own range shrinks that trip point without anything about the metric having changed.

That is what happened to three p50 keys on the 2026-08-26 baseline, and all three are excluded — this rule being applied, not a new exception. Measured across the three CI runs 32862589645, 32867433073 and 32964262700 (the last of which produced that baseline):

key	bound	trip point	observed max	max ÷ trip	spread
span.tx.apply.p50	0.0440 ms	0.05 ms	2.3378 ms	46.76x	391.8x
span.ledger.build.p50	0.3849 ms	0.5 ms	2.3826 ms	4.77x	20.7x
span.consensus.ledger_close.p50	0.0613 ms	0.1 ms	0.2377 ms	2.38x	6.1x

Before the exclusion, replaying either older run against that baseline reported exactly those three and nothing else — and run 32867433073 carries the same post-path-finding-removal workload as the baseline itself, so the movement was metric variance, not a workload difference. Those two runs are what would have reddened CI. After the exclusion both replay clean.

The evidence that settles it is span.tx.apply.p50's own history. It read 0.7917 ms in the 2026-08-24 baseline and 0.00597 ms in the 2026-08-26 one — a 132x difference between two runs of the same workload. At the old value the identical hi_next − baseline rule produced a 4.21 ms bound whose 5 ms trip point absorbed the entire range; at the new value it produces 0.0440 ms and cannot. Nothing about the metric changed. Whether the gate functioned was decided by where in its own distribution the captured run happened to land — which is not a threshold that needs tuning, it is a key that cannot be gated from a single-run baseline at all.

So the remedy is the excluded_keys entry with the measurement behind it, exactly as ledger.validate p95 and p99 got — not a widened bound, and not re-baselining until a run lands favourably. A key that fails this test is never fixed by widening its bound. The remaining 19 gated keys sit between 0.14 and 0.50 of their baseline over their trip point, the tightest being span.tx.process.p95 at 0.50. That ratio is derivable from the two committed JSON files, so it is checkable; a headroom figure against each key's observed maximum is not, because no per-run timings.json is committed.

What all six excluded keys have in common

key	trip point	observed max	mechanism
span.tx.apply.p50	0.05 ms	2.3378 ms	baseline in the ladder's first bucket
span.consensus.ledger_close.p50	0.1 ms	0.2377 ms	baseline in a low bucket
span.ledger.build.p50	0.5 ms	2.3826 ms	baseline in a low bucket
span.ledger.validate.p95	0.5 ms	0.7500 ms	baseline in a low bucket
span.ledger.validate.p99	10 ms	25.8750 ms	spread too large for any bound
span.ledger.build.p99	25 ms	29.0000 ms	spread too large for any bound

One invariant covers all six: the observed maximum exceeds baseline + bound, so an ordinary run clears the trip point with nothing having regressed. Two mechanisms produce it. Four of the six have a baseline sitting low in the ladder, where the derived bound is tiny because the bound is the distance to the next edge up. The other two fail despite generous bounds: ledger.validate.p99 has 8.94 ms and a 66.8x spread that reaches 25.875 ms against a 10 ms trip point, and ledger.build.p99 has 16.056 ms and a 4.11x spread whose maximum is 1.16x its trip point.

span.ledger.build.p99 is the newest of the six and the clearest illustration of the rule, because the previous baseline hid it: at 9.109 ms the same rule also gave a 25 ms trip point, and the key read as gated only because both the capture and the comparison runs happened to land low. Ledger construction keeps coverage through span.ledger.build.p95, whose baseline sits at 0.48 of its trip point.

The follow-up that would restore coverage, stated rather than left implied: a bound derived from the ladder alone cannot support these keys, because it carries no information about spread. What would let them be gated again is a bound sized against observed variance — a spread measurement captured alongside the baseline, rather than the ladder distance only. The 2026-09-10 baseline is already a median of three runs, which is the raw material for that; using the spread to size bounds is the part that is not implemented, and it is the design change these six exclusions are waiting on.

Bootstrapping the baseline

1. Merge a CI run with a "placeholder": true baseline. The telemetry-validation workflow runs, fails no gate, and prints the captured timings block to the workflow Step Summary under the heading ### Paste into baselines/baseline-timings.json.
2. Open a new PR. Copy the full JSON block from the Step Summary (or download the timings.json artifact) into this file, replacing the placeholder contents. The JSON is emitted in the exact byte-for-byte format this file expects — sorted keys, 2-space indent, trailing newline.
3. The committed baseline PR needs reviewer approval just like any other code change. This is the primary audit point for "who moved the performance bar."

An incomplete capture cannot seed a baseline

capture_timings.py writes timings.json before it enforces --min-capture-ratio, so a run that reached too little of Prometheus still leaves a file behind — one that exists, parses, and carries every declared key, some of them null. Nothing about it reads as degraded, and the obvious reaction to a red gate is to refresh the baseline, so this is exactly the file a person is most likely to paste.

Every capture therefore records its own verdict in a capture block (see Schema), and complete there is exactly the condition capture_timings.py exits 0 on. Both routes to a baseline read that flag and print nothing to paste unless it is true:

• the workflow's Step Summary heading becomes "Baseline NOT refreshable from this run", carrying the captured/declared counts and an ::error:: annotation;
• compare_to_baseline.py writes the same explanation to stderr, leaves stdout empty so a > redirect cannot produce a plausible-looking file, and exits 2.

An artifact with no capture block — one produced before this existed — counts as not complete. Completeness has to be proven, not assumed.

This only guards the paste. Against a populated baseline a thin capture still compares normally and its uncaptured keys are reported as not captured in current run, which is the pre-existing behaviour described under Schema below.

Refreshing the baseline

Refresh when a legitimate performance change lands on develop (for example, a deliberate rewrite that changes a span's structure). The process is identical to bootstrapping: run CI with the current baseline, inspect the delta, and if the new numbers should become the norm, open a PR pasting the fresh timings into baseline-timings.json. The reviewer decides whether the new baseline is acceptable.

Do not edit baseline-timings.json by hand outside of this process — every entry should trace back to a real CI run so variance characteristics are preserved.

Refreshing the baseline also obliges you to re-derive the absolute bounds in ../regression-thresholds.json, per Absolute bounds are derived per metric. A value that moves into a different bucket needs a different bound, and a bound left behind either stops catching regressions or starts firing on quantization noise.

It also obliges you to re-check each key's run-to-run spread against its new trip point, per The general rule this exposed. A refreshed baseline can land in a bucket whose hi_next no longer clears the metric's own variance, which turns the key into a recurring false positive — the failure that excluded ledger.validate p95 and p99.

The baseline is only valid at the log level it was captured at

Every timing here is coupled to the log_level that run-full-validation.sh writes into each node's [rpc_startup] stanza. Logging is synchronous, and several of the gated spans contain log statements, so the configured level is part of the measurement:

• ledger.build contains BuildLedger.cpp:81 (debug).
• consensus.accept contains RCLConsensus.cpp:683/687/698/715 (debug) — :715 logs once per transaction in the canonical set.
• tx.apply and the other spans.names entries in ../regression-metrics.json are affected the same way.

Raising the level admits more of those statements and inflates the p50/p95/p99 of the very spans the gate measures; lowering it deflates them. Neither shows up as a regression, because the baseline moves with it — the gate simply starts measuring a different configuration.

Changing the workload log level therefore invalidates this baseline and requires re-capturing it. Treat it exactly like a deliberate performance change: follow Refreshing the baseline, and note the level change in the PR so the reviewer knows why the numbers moved. In particular, do not capture a baseline while the harness is running at debug — see the runbook's "Why not debug" note; if you need debug-level detail, enable it per partition after the baseline exists.

Schema

{
  "schema_version": 1,
  "captured_at": "2026-04-24T17:30:00Z",
  "window": "3m",
  "git_sha": "<SHA of the commit that produced these numbers>",
  "profile": "<workload profile used>",
  "source_runs": [34495527952, 34505215266, 34507425933],
  "statistic": "median of three clean full-validation runs",
  "capture": {
    "declared": 19,
    "captured": 19,
    "min_ratio": 0.5,
    "complete": true
  },
  "metrics": {
    "span.tx.process.p99": { "value": 12.4, "unit": "ms" },
    "job.transaction.queued.p95": { "value": 1500.0, "unit": "us" }
  }
}

source_runs and statistic record how the numbers were arrived at, and the committed baseline carries both. They matter because the bound-derivation rule reads the baseline as a single number: if statistic says the values are a median of several runs, a reviewer knows the spread was observable, and if it is absent the baseline came from one run and every key on it is exposed to the single-run problem described under The general rule this exposed. Neither field is read by any script; they are provenance, like git_sha.

capture_timings.py emits neither field, and no script in this directory combines several runs, so a multi-run baseline is currently assembled by hand and these two fields are how that is declared. That is a gap, not a workflow: it sits outside the paste-from-CI rule the rest of this file describes, so the median and the run ids are only as trustworthy as the PR that introduced them. Automating the combination — and having it write both fields — is part of the multi-run baseline work the exclusions are waiting on.

capture describes the capture that produced the file, not the metrics in it: declared is how many keys the surface asked for, captured how many came back with a value, min_ratio the bar they were judged against, and complete the verdict. It is a sibling of metrics, never an entry inside it, so it is neither a metric key nor a gated entry — check_regression_bounds.py and compare_to_baseline.py both iterate metrics alone and never see it. Because a committed baseline is a verbatim copy of a capture, the block lands here too; it is metadata about provenance, exactly like git_sha. Entries committed before it existed simply do not carry it.

Keys follow {category}.{name}.p{quantile}. Only two categories are actually produced today — span.* and job.* — because build_query_plan() in prom_queries.py reads the spans and job_queue groups of regression-metrics.json, and that file defines only those two.

Placeholder baselines additionally include "placeholder": true. The comparator detects this field (or an empty metrics object) to switch into "populate" mode instead of enforcing thresholds. Remove the placeholder key when pasting real captured timings.

Missing metrics (value null) in a captured run do not count as regressions. In regression-report.json, summary.missing_in_current is a count only; the identities are in the metrics[] array, as the entries whose note is "not captured in current run". Filter for those to see which keys went missing:

jq -r '.metrics[] | select(.note == "not captured in current run") | .key' \
    /tmp/xrpld-validation/reports/regression-report.json

This keeps the gate robust when a profile doesn't exercise every span on every run.

Known gap: no rpc.* metric can gate (FU-4)

Per-RPC-method timings are not gated, and would not gate even if they were captured. Two independent blockers:

1. Nothing emits an rpc.* key. build_query_plan() in prom_queries.py builds rpc.* entries from cfg.get("rpc_methods", {}), and regression-metrics.json has no rpc_methods block — so the group resolves to empty and no rpc.* key ever reaches timings.json or this baseline.
2. Even a captured rpc.* key would silently not gate. resolve_thresholds() in compare_to_baseline.py maps the rpc category to the threshold group rpc_method, but regression-thresholds.json defines only defaults.span and defaults.job_queue. With no rpc_method block the lookup returns (None, None), which the comparator treats as "no threshold configured" — the metric is reported but can never fail the build.

Closing this needs both an rpc_methods group in regression-metrics.json and a defaults.rpc_method block in regression-thresholds.json. Adding only the first produces metrics that look gated in the report but are not.

Known exclusion: rpc.process is not captured

rpc.process is deliberately absent from the spans.names list in regression-metrics.json, so no span.rpc.process.* key appears in this baseline. The span is created only in ServerHandler::processRequest() (src/xrpld/rpc/detail/ServerHandler.cpp:718), which is reached only from the HTTP/JSON-RPC session path. The harness load generator is WebSocket-only and that path never calls processRequest, so the span is never emitted under any workload profile here — expected_spans.json marks it "optional": true for the same reason.

While it was listed, the three quantiles were captured as null on every run and the comparator short-circuited them as "new metric (not in baseline)" — so a 9999 ms value would still have reported regressed: false. Three keys that can never gate are worse than no keys: they inflate summary.total and read as covered.

If per-request HTTP timings are wanted, the fix is to give the harness an HTTP/JSON-RPC load path first, then re-add rpc.process and bootstrap a real baseline for it.