ARCHIVE/rippled
1
0
Fork 0
mirror of https://github.com/XRPLF/rippled.git synced 2026-08-21 14:20:56 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics

Browse Source 
Two conflicts, both additive-vs-additive; each resolution keeps both sides.

check_otel_naming.py -- phase-10 taught the L6 label extractor to match the
label MAP first and to resolve a key hoisted into a `k...Label` constant,
scanning headers as well as sources. Our side had added the two-regex
first/subsequent literal scan and the `metric_constants(root)[1]` union that
covers the `namespace label` header style.

Kept phase-10's mechanism whole: METRIC_LABEL_MAP + the `(?:^|\{)` key regex
already subsumes what METRIC_LABEL_NEXT did, since matching inside the map body
makes every pair after the first open with a single `{`. So METRIC_LABEL_NEXT is
dropped as genuinely redundant rather than kept as a duplicate scan, and the
reason it existed is folded into METRIC_LABEL's comment. Re-added our
`metric_constants(root)[1]` union on top: LABEL_CONST_DEF only matches
`k`-prefixed identifiers, so it cannot see MetricNames.h's `label::jobType`
style, and without that union Rule D would reject dashboards querying labels
Rule I forced into constants. The two derivations are complementary and both
are now documented as such.

MetricsRegistry.cpp -- both sides added a new sibling view-registration helper
next to addMicrosecondHistogramView, and both added a registration call in
initExporterAndProvider(). Kept all four helpers
(addHistogramView/Microsecond/RoundDuration/SubMillisecond) and every
registration: phase-10's addSubMillisecondHistogramView + kNodeStoreReadUs
alongside our addRoundDurationHistogramView, sweepMallocTrimUs and the two
millisecond dial/resolve ladders.

phase-10's nodestore_read_us histogram does not duplicate our work. The
nodestore_latency gauge that would have overlapped it was retired in c4e434d520
before this merge, and the surviving nodestore_state gauge is complementary
rather than duplicative: both read the same fetch measurement, but the gauge
publishes only a since-boot mean via scaledMean() and cannot yield a
percentile -- the consequence observeNodeStoreTotals' own docs state plainly --
while the histogram buckets each fetch and can. The histogram also splits by
fetch_type and found, which the gauge cannot. phase-10 registered its
explicit-bucket View, so it does not inherit the SDK default ladder.

Each file keeps its own existing naming style: phase-10's k-prefixed constants
are left as-is, ours stay namespaced.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Pratik Mankawde
2026-07-28 14:09:05 +01:00
parent 206d3620c7 972c279253
commit fddf78567d
15 changed files with 1845 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.github/scripts/levelization/results/ordering.txt vendored
View File

@@ -140,7 +140,6 @@ test.nodestore > xrpl.basics
test.nodestore > xrpl.config
test.nodestore > xrpld.core
test.nodestore > xrpl.nodestore
test.nodestore > xrpl.protocol
test.nodestore > xrpl.rdb
test.overlay > test.jtx
test.overlay > test.unit_test

84
.github/scripts/otel-naming/check_otel_naming.py vendored
View File

@@ -183,18 +183,32 @@ BLOCK_COMMENT = re.compile(r"/\*.*?\*/", re.DOTALL)
# Dashboards reference span attributes in TraceQL as `span.<attr>`; the bare
# attribute is what must exist in L1, so strip the scope before validating.
TRACEQL_SCOPE = re.compile(r"^(?:span|resource|event|link|instrumentation_scope)\.")
# An OTel metric label key as emitted in C++: `Add(.., {{"label", ...}})` /
# `{{"label", value}}` instrument calls in MetricsRegistry.
# An OTel metric label map as emitted in C++: the `{{...}}` argument of an
# instrument call, e.g. `counter->Add(1, {{"job_type", a}, {"handler", b}})`.
# Matching the whole map first, rather than scanning the file for `{"key",`,
# keeps ordinary brace initializers (`{"http", "https", ...}`) out of the
# label set — they are not labels and must not license a dashboard filter.
METRIC_LABEL_MAP = re.compile(r"\{\{(.*?)\}\}", re.DOTALL)
# One key inside such a map: a string literal, or a `kFooLabel` constant name
# resolved through LABEL_CONST_DEF. A label set is a nested initializer list
# (`{{"a", x}, {"b", y}}`), so inside the map each pair opens with `{` except
# the first, whose `{` was consumed by the map's own `{{` — hence the `^`
# alternative. Matching only the doubled-brace form would derive just the first
# label of every multi-label instrument, silently under-deriving the L6 key set
# and making Rule D reject a dashboard that queries a label the code genuinely
# emits.
METRIC_LABEL = re.compile(r'(?:^|\{)\s*"([a-z_][a-z0-9_]*)"\s*,')
METRIC_LABEL_CONST = re.compile(r"(?:^|\{)\s*(k[A-Za-z0-9_]*)\s*,")
# A label-key constant definition, with or without `inline`:
# `constexpr char kHandlerLabel[] = "handler";`
#
# Two patterns are needed because a label set is a nested initializer list:
# `{{"a", x}, {"b", y}}`. The FIRST label is preceded by the doubled brace that
# opens both the set and the pair, while every SUBSEQUENT label is preceded by
# `}, {` closing the previous pair and opening the next. Matching only the
# doubled-brace form would derive just the first label of every multi-label
# instrument, silently under-deriving the L6 key set and making Rule D reject a
# dashboard that queries a label the code genuinely emits.
METRIC_LABEL = re.compile(r'\{\{\s*"([a-z_][a-z0-9_]*)"\s*,')
METRIC_LABEL_NEXT = re.compile(r'\}\s*,\s*\{\s*"([a-z_][a-z0-9_]*)"\s*,')
# This resolves the flat `k`-prefixed per-subsystem header style. The
# `namespace label` style used by MetricNames.h is covered separately, by
# metric_constants(), because those identifiers are not `k`-prefixed and are
# referenced at call sites as `label::jobType` — see metric_label_names().
LABEL_CONST_DEF = re.compile(
r'(?:inline\s+)?constexpr\s+char\s+(k[A-Za-z0-9_]*)\s*\[\s*\]\s*=\s*"([a-z_][a-z0-9_]*)"'
)
def strip_comments(text: str) -> str:
@@ -861,21 +875,43 @@ def metric_label_names(root: Path) -> Set[str]:
multi-label instrument such as `{{"from", a}, {"to", b}}` contributes ALL
of its keys.
Includes the keys declared as constants in `*MetricNames.h` (L1-metrics),
because Rule I requires call sites to reference a constant rather than a
literal — so once a subsystem is converted, its label keys no longer appear
as literals anywhere and a literal-only scan would under-derive the set and
make Rule D reject a dashboard querying a label the code really emits."""
A label key may be written inline as a string literal or hoisted into a
named constant (`constexpr char kHandlerLabel[] = "handler";`, then
`{{kHandlerLabel, value}}`). Both forms are collected, and the constants
may be declared in a header, so headers are scanned as well as sources.
Also includes the keys declared as constants in `*MetricNames.h`
(L1-metrics), because Rule I requires call sites to reference a constant
rather than a literal — so once a subsystem is converted, its label keys no
longer appear as literals anywhere and a literal-only scan would
under-derive the set and make Rule D reject a dashboard querying a label
the code really emits. That covers the `namespace label` header style,
whose identifiers are not `k`-prefixed and so are invisible to
LABEL_CONST_DEF; the two derivations are complementary."""
labels: Set[str] = set()
# Constant name -> label string, gathered across every scanned file so a
# `{{kFooLabel, ...}}` call site resolves even when the definition lives in
# a different file from the instrument call.
constants: Dict[str, str] = {}
used_constants: Set[str] = set()
for base in ("src", "include"):
for p in (root / base).rglob("*.cpp"):
if not p.is_file():
continue
text = read_source(p)
if "MetricsRegistry" not in p.name and "metric" not in text.lower():
continue
labels |= set(METRIC_LABEL.findall(text))
labels |= set(METRIC_LABEL_NEXT.findall(text))
for pattern in ("*.cpp", "*.h"):
for p in (root / base).rglob(pattern):
if not p.is_file():
continue
# Test code passes arbitrary literal pairs to exercise APIs
# (`signers("alice", 1, {{"alice", 1}, {"bob", 2}})`). Those are
# not metric labels, and must not license a dashboard filter.
if is_test_path(p):
continue
text = read_source(p)
if "MetricsRegistry" not in p.name and "metric" not in text.lower():
continue
constants.update(dict(LABEL_CONST_DEF.findall(text)))
for label_map in METRIC_LABEL_MAP.findall(text):
labels |= set(METRIC_LABEL.findall(label_map))
used_constants |= set(METRIC_LABEL_CONST.findall(label_map))
labels |= {constants[name] for name in used_constants if name in constants}
labels |= metric_constants(root)[1]
return labels

63
.github/scripts/otel-naming/test_check_otel_naming.py vendored
View File

@@ -839,6 +839,69 @@ class MetricLabelExtraction(unittest.TestCase):
finally:
shutil.rmtree(d)
def test_extracts_second_label_of_a_pair(self):
"""A multi-label map opens the first pair with `{{` and later ones with
a single `{`; every key must be collected, not just the first."""
d = Path(tempfile.mkdtemp())
try:
_write(
d / "src" / "xrpld" / "telemetry" / "MetricsRegistry.cpp",
'h->Record(v, {{"job_type", std::string(t)}, {"handler", h2}});\n',
)
self.assertEqual(chk.metric_label_names(d), {"job_type", "handler"})
finally:
shutil.rmtree(d)
def test_resolves_label_key_constants(self):
"""A key hoisted into a `constexpr char k...[]` constant resolves to its
string, including when the constant is declared in a header."""
d = Path(tempfile.mkdtemp())
try:
_write(
d / "include" / "xrpl" / "telemetry" / "GetObjectMetricNames.h",
"// metric name constants\n"
'inline constexpr char kLabelResult[] = "result";\n',
)
_write(
d / "src" / "xrpld" / "telemetry" / "MetricsRegistry.cpp",
'constexpr char kHandlerLabel[] = "handler";\n'
"counter->Add(1, {{kLabelResult, std::string(r)}});\n"
'c2->Add(1, {{"job_type", std::string(t)}, {kHandlerLabel, h}});\n',
)
self.assertEqual(
chk.metric_label_names(d), {"result", "handler", "job_type"}
)
finally:
shutil.rmtree(d)
def test_ignores_plain_brace_initializers(self):
"""Ordinary brace lists are not label maps: a bare `{"http", "https"}`
array must not license a dashboard filter on `http`."""
d = Path(tempfile.mkdtemp())
try:
_write(
d / "src" / "xrpld" / "telemetry" / "MetricsRegistry.cpp",
'std::array schemes{"http", "https", "ws"};\n'
'for (auto const* k : {"read_request_bundle", "read_threads"})\n'
" use(k);\n"
'counter->Add(1, {{"job_type", std::string(t)}});\n',
)
self.assertEqual(chk.metric_label_names(d), {"job_type"})
finally:
shutil.rmtree(d)
def test_ignores_test_code_literals(self):
"""Test fixtures pass arbitrary literal pairs; they define no labels."""
d = Path(tempfile.mkdtemp())
try:
_write(
d / "src" / "test" / "jtx" / "Env_test.cpp",
"// metric\n" 'env(signers("alice", 1, {{"alice", 1}, {"bob", 2}}));\n',
)
self.assertEqual(chk.metric_label_names(d), set())
finally:
shutil.rmtree(d)
class ReportExitContract(unittest.TestCase):
@staticmethod