ARCHIVE/rippled
1
0
Fork 0
mirror of https://github.com/XRPLF/rippled.git synced 2026-04-29 15:37:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: address review findings in regression gate

Browse Source 
- capture_timings.py: fail when captured/total ratio < 50%
  (--min-capture-ratio). Prevents silent pass on unreachable Prometheus.
- run-full-validation.sh: set REGRESSION_EXIT=2 on capture failure so
  the final exit code reflects it. Update exit code docs in header.
- compare_to_baseline.py: extract _skip_delta helper to bring
  compute_delta under 80 lines. Fix 0.0-as-falsy bug in abs_bound
  resolution (use explicit None check instead of `or`). Remove dead
  variable override_prefix_key.
- prom_queries.py: extract _build_simple_entries and _build_job_entries
  to bring build_query_plan under 80 lines. Fix module docstring return
  type example. Use aiohttp.ClientTimeout instead of bare int.
- telemetry-validation.yml: add set -euo pipefail to regression summary
  step; guard jq calls with -e flag and fallback; fail on missing
  baseline file; emit ::warning annotation when timings.json missing.
- baselines/README.md: document the placeholder field.
This commit is contained in:
Pratik Mankawde
2026-04-24 19:36:15 +01:00
parent df79d5e74b
commit 577d1f8a21
6 changed files with 139 additions and 109 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
.github/workflows/telemetry-validation.yml vendored
View File

@@ -238,16 +238,27 @@ jobs:
- name: Print regression summary
if: always()
run: |
set -euo pipefail
TIMINGS="/tmp/xrpld-validation/reports/timings.json"
REGRESSION="/tmp/xrpld-validation/reports/regression-report.json"
BASELINE="docker/telemetry/workload/baselines/baseline-timings.json"
if [ ! -f "$TIMINGS" ]; then
echo "## Regression Gate: no timings captured" >> "$GITHUB_STEP_SUMMARY"
echo "::warning::capture_timings.py did not produce timings.json — regression gate was not evaluated."
exit 0
fi
IS_PLACEHOLDER=$(jq -r '.placeholder == true or (.metrics | length == 0)' "$BASELINE")
if [ ! -f "$BASELINE" ]; then
echo "## Regression Gate: baseline file missing" >> "$GITHUB_STEP_SUMMARY"
echo "::error::baselines/baseline-timings.json not found in checkout"
exit 1
fi
IS_PLACEHOLDER=$(jq -e -r '.placeholder == true or (.metrics | length == 0)' "$BASELINE") || {
echo "::error::Failed to parse baseline JSON"
exit 1
}
echo "## OTel Timings Regression Gate" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
@@ -263,9 +274,9 @@ jobs:
cat "$TIMINGS" >> "$GITHUB_STEP_SUMMARY"
echo '```' >> "$GITHUB_STEP_SUMMARY"
elif [ -f "$REGRESSION" ]; then
REGR_COUNT=$(jq '.summary.regressions' "$REGRESSION")
IMPR_COUNT=$(jq '.summary.improvements' "$REGRESSION")
TOTAL=$(jq '.summary.total' "$REGRESSION")
REGR_COUNT=$(jq -e '.summary.regressions' "$REGRESSION") || REGR_COUNT=0
IMPR_COUNT=$(jq -e '.summary.improvements' "$REGRESSION") || IMPR_COUNT=0
TOTAL=$(jq -e '.summary.total' "$REGRESSION") || TOTAL=0
echo "| Stat | Count |" >> "$GITHUB_STEP_SUMMARY"
echo "|------|-------|" >> "$GITHUB_STEP_SUMMARY"
echo "| Metrics compared | $TOTAL |" >> "$GITHUB_STEP_SUMMARY"