Files
rippled/docker/telemetry/workload/regression-thresholds.json

27 lines
3.0 KiB
JSON
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{
"_description": "Per-metric regression thresholds. A metric regresses when current - baseline exceeds BOTH the percentage and absolute bounds (AND, not OR — this tolerates small-value noise). Defaults apply unless a per-metric override exists.",
"_bucket_note": "SpanMetrics latency histograms use explicit buckets [0.01,0.05,0.1,0.25,0.5,1,5,10,25,50,100,250,500]ms then [1,2,3,4,5,10,30]s (20 edges; docker/telemetry/otel-collector-config.yaml is the authoritative list). An earlier version of this note claimed 15 edges starting at 1ms and justified the 10ms absolute span bound as \"~2 low-end bucket widths\" — that derivation is void, because the sub-millisecond edges make the low-end bucket width 0.01ms, not 5ms. The 10ms bound is retained on its own merit: it is roughly two bucket widths in the 5-25ms band where most span quantiles actually sit, so it still absorbs single-bucket quantization jitter while catching multi-bucket regressions. Second-scale consensus spans have 2s/3s/4s boundaries, so their quantiles quantize to ~1s widths there. The job_queue running bound is widened similarly — per-ledger apply work scales with TxQ burst load. NOTE: BOTH ladders were re-cut, and a baseline captured before its own ladder changed is an interpolation artefact, not a latency. The native job_queue histograms are microsecond-valued and their floor moved 100us → 1us. The span ladder was re-cut too, on 2026-08-04 in 3860c93db2, moving the floor 1ms → 0.01ms; so any sub-millisecond span quantile captured before that date is equally void — a p95 reading 0.95ms is 0.95 × the old 1ms first edge, not a measurement. An earlier note asserted that the surviving span baselines were unaffected by the ladder work; that is wrong for every span quantile below 1ms. Only the band from 1ms to 1s is safe: those edges are byte-identical across the two ladders. The re-cut also ADDED edges above 1s (2s/3s/4s/10s/30s), so a span whose quantiles land in the second-scale range — consensus.round ~3.9s, consensus.establish ~1.9s, the ledger.acquire tail — is distorted just as much, and any pre-2026-08-04 baseline for it is equally void. Do not read this note as licensing a stale second-scale baseline.",
"defaults": {
"span": {
"p50": { "max_pct_increase": 50.0, "max_abs_increase_ms": 10.0 },
"p95": { "max_pct_increase": 50.0, "max_abs_increase_ms": 10.0 },
"p99": { "max_pct_increase": 50.0, "max_abs_increase_ms": 15.0 }
},
"job_queue": {
"p95": { "max_pct_increase": 50.0, "max_abs_increase_us": 20000.0 }
}
},
"overrides": {
"span.consensus.ledger_close": {
"p50": { "max_pct_increase": 5.0, "max_abs_increase_ms": 200.0 },
"p95": { "max_pct_increase": 5.0, "max_abs_increase_ms": 500.0 },
"p99": { "max_pct_increase": 5.0, "max_abs_increase_ms": 1000.0 }
},
"span.consensus.accept": {
"p50": { "max_pct_increase": 5.0, "max_abs_increase_ms": 200.0 },
"p95": { "max_pct_increase": 5.0, "max_abs_increase_ms": 500.0 },
"p99": { "max_pct_increase": 5.0, "max_abs_increase_ms": 1000.0 }
}
}
}