ARCHIVE/rippled
1
0
Fork 0
mirror of https://github.com/XRPLF/rippled.git synced 2026-07-23 15:10:34 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill

Browse Source
This commit is contained in:
Pratik Mankawde
2026-07-20 12:19:45 +01:00
parent 71f0aa03a8 6203e46e8d
commit de49319b31
889 changed files with 39814 additions and 16964 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.github/scripts/levelization/results/ordering.txt vendored
View File

@@ -1,3 +1,6 @@
benchmarks.libxrpl > xrpl.basics
benchmarks.libxrpl > xrpl.config
benchmarks.libxrpl > xrpl.nodestore
libxrpl.basics > xrpl.basics
libxrpl.conditions > xrpl.basics
libxrpl.conditions > xrpl.conditions
@@ -165,6 +168,7 @@ test.peerfinder > xrpl.protocol
test.protocol > test.jtx
test.protocol > test.unit_test
test.protocol > xrpl.basics
test.protocol > xrpld.core
test.protocol > xrpl.json
test.protocol > xrpl.protocol
test.rpc > test.jtx

18
.github/scripts/otel-naming/README.md vendored
View File

@@ -38,15 +38,15 @@ hardcoded allowlist:
### Rules (each fails the build, when its inputs are present)
| Rule | Check |
| ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| A | No stray dotted span-attribute key (only the derived resource keys may be dotted). |
| G | Attribute keys are `lower_snake_case` (`^[a-z][a-z0-9_]*$` per dot-segment) — no camelCase, UPPERCASE, or spaces. |
| F | No string literals as attribute keys or span-name arguments in `setAttribute`/`addEvent`/`span`/`childSpan`. Attribute _values_ are exempt (runtime data); `*SpanNames.h` definitions and test files are exempt. |
| B | Every collector `spanmetrics.dimensions` name exists in the L1 key set. |
| C | Every Tempo span-filter tag exists in the L1 key set. |
| D | Every dashboard label resolves to an L1 span attribute, a native-metric label (L6, emitted by MetricsRegistry), or a Prometheus/Grafana builtin. TraceQL scope prefixes (`span.`/`resource.`/…) are stripped before the L1 lookup. |
| E | No dotted `xrpl.<domain>.<field>` attribute key in the runbook (only the L1 resource attrs `xrpl.network.*` may be dotted). Span names, filenames, OTel-standard keys, and metric labels are not flagged. |
| Rule | Check |
| ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| A | No stray dotted span-attribute key (only the derived resource keys may be dotted). |
| G | Attribute keys are `lower_snake_case` (`^[a-z][a-z0-9_]*$` per dot-segment) — no camelCase, UPPERCASE, or spaces. |
| F | No string literals as attribute keys or span-name arguments in `setAttribute`/`addEvent`/`span`/`rootSpan`/`childSpan` (`rootSpan` shares `span`'s `(cat, prefix, name)` signature). Attribute _values_ are exempt (runtime data); `*SpanNames.h` definitions and test files are exempt. |
| B | Every collector `spanmetrics.dimensions` name exists in the L1 key set. |
| C | Every Tempo span-filter tag exists in the L1 key set. |
| D | Every dashboard label resolves to an L1 span attribute, a native-metric label (L6, emitted by MetricsRegistry), or a Prometheus/Grafana builtin. TraceQL scope prefixes (`span.`/`resource.`/…) are stripped before the L1 lookup. |
| E | No dotted `xrpl.<domain>.<field>` attribute key in the runbook (only the L1 resource attrs `xrpl.network.*` may be dotted). Span names, filenames, OTel-standard keys, and metric labels are not flagged. |
Rule F runs **unconditionally** (it is a purely syntactic check on the
call-sites and needs no `*SpanNames.h`), so a code path that calls

27
.github/scripts/otel-naming/check_otel_naming.py vendored
View File

@@ -31,7 +31,8 @@ Layers
------
L1 code : src/**/*SpanNames.h, include/**/*SpanNames.h (ground truth)
L1 resource : src/libxrpl/telemetry/Telemetry.cpp (dotted allowlist)
L1 callsites : setAttribute/addEvent/span/childSpan in src/**, include/**
L1 callsites : setAttribute/addEvent/span/rootSpan/childSpan in src/**,
include/**
L2 collector : docker/telemetry/otel-collector-config.yaml (spanmetrics dims)
L3 tempo : docker/telemetry/tempo.yaml (span filter tags)
L4 dashboards: docker/telemetry/grafana/dashboards/*.json (PromQL labels)
@@ -46,9 +47,9 @@ Rules (each FAILS the build, when its inputs are present)
G Attribute keys must be lower_snake_case (^[a-z][a-z0-9_]*$ per segment).
Flags camelCase, UPPERCASE, spaces, and other stray characters.
F No string literals as attribute keys or span-name arguments. The
setAttribute/addEvent key and the span/childSpan prefix/name args must
reference a *SpanNames.h constant, never a "literal". Attribute VALUES are
exempt (runtime data). Definitions inside *SpanNames.h are exempt, and
setAttribute/addEvent key and the span/rootSpan/childSpan prefix/name args
must reference a *SpanNames.h constant, never a "literal". Attribute VALUES
are exempt (runtime data). Definitions inside *SpanNames.h are exempt, and
test files are exempt (they pass arbitrary literals to exercise the API).
B Every collector spanmetrics dimension exists in the L1 key set.
C Every tempo span-filter tag exists in the L1 key set.
@@ -123,12 +124,14 @@ USING_DECL = re.compile(r"using\s+(?:::)?[\w:]*::(\w+)\s*;")
# Telemetry call-sites whose string arguments must be constants, not literals.
# Require a receiver so we match real SpanGuard calls, not std::span / a math
# `span(...)` / a bare method declaration:
# - `SpanGuard::span(` / `SpanGuard::childSpan(` (static factory)
# - `SpanGuard::span(` / `SpanGuard::rootSpan(` / `SpanGuard::childSpan(`
# (static factories)
# - `<obj>.span(` / `<obj>->setAttribute(` etc. (member call)
# `span`/`childSpan` additionally require the `SpanGuard`/`.`/`->` receiver;
# `setAttribute`/`addEvent` only ever exist on a guard, so a `.`/`->` suffices.
# `span`/`rootSpan`/`childSpan` additionally require the `SpanGuard`/`.`/`->`
# receiver; `setAttribute`/`addEvent` only ever exist on a guard, so a `.`/`->`
# suffices. `rootSpan` shares `span`'s (cat, prefix, name) signature.
CALLSITE = re.compile(
r"(?:SpanGuard::|\.|->)\s*(setAttribute|addEvent|span|childSpan)\s*\("
r"(?:SpanGuard::|\.|->)\s*(setAttribute|addEvent|span|rootSpan|childSpan)\s*\("
)
# A C++ string literal (used to flag literals inside call-site argument lists).
STRING_LITERAL = re.compile(r'"((?:[^"\\]|\\.)*)"')
@@ -562,11 +565,13 @@ def run_rule_g(keys_by_header: Dict[Path, Set[str]], report: Report) -> None:
# setAttribute(key, value) -> check arg 0 (key); value (arg 1) exempt
# addEvent(name[, attrs]) -> check arg 0 (event name)
# span(category, prefix, name) -> check args 1,2 (prefix + span-name leaf)
# rootSpan(category, prefix, name)-> check args 1,2 (same signature as span)
# childSpan(name[, parentCtx]) -> check arg 0 (span-name leaf)
CONSTANT_ARG_POSITIONS: Dict[str, Set[int]] = {
"setAttribute": {0},
"addEvent": {0},
"span": {1, 2},
"rootSpan": {1, 2}, # same signature as span(cat, prefix, name)
"childSpan": {0},
}
@@ -597,7 +602,8 @@ def spanname_symbol_names(headers: List[Path]) -> Set[str]:
def run_rule_f(root: Path, report: Report, header_symbols: Set[str]) -> None:
"""Walk every telemetry call-site (non-test, non-*SpanNames.h) and check the
constant-only argument positions of setAttribute/addEvent/span/childSpan:
constant-only argument positions of
setAttribute/addEvent/span/rootSpan/childSpan:
Rule F (FAIL): a string literal in a key / span-name position. Attribute
VALUES are exempt (runtime data).
@@ -664,7 +670,8 @@ def run_rule_f(root: Path, report: Report, header_symbols: Set[str]) -> None:
def iter_calls(text: str):
"""Yield (call_name, raw_arglist, lineno) for each setAttribute/addEvent/
span/childSpan invocation, spanning multiple physical lines if needed."""
span/rootSpan/childSpan invocation, spanning multiple physical lines if
needed."""
for m in CALLSITE.finditer(text):
name = m.group(1)
# Walk from the opening paren, balancing nesting to find the close.

23
.github/scripts/otel-naming/test_check_otel_naming.py vendored
View File

@@ -559,6 +559,29 @@ class RuleFAndH(unittest.TestCase):
v, _ = self._run("src/Foo.cpp", 'SpanGuard::span(cat, "rpc", "command");\n')
self.assertEqual(v, ['span arg1 "rpc"', 'span arg2 "command"'])
def test_rootspan_literal_flagged_by_rule_f(self):
# rootSpan(cat, prefix, name) shares span()'s signature, so a string
# literal in the prefix/name position must FAIL rule F exactly as it
# does for span() — otherwise a call switched to rootSpan silently
# escapes span-name validation.
v, _ = self._run(
"src/Foo.cpp",
'SpanGuard::rootSpan(cat, "peer", "validation.receive");\n',
)
self.assertEqual(
v, ['rootSpan arg1 "peer"', 'rootSpan arg2 "validation.receive"']
)
def test_rootspan_constant_args_accepted(self):
# Constant references in the prefix/name position are accepted (no
# rule F), mirroring span()'s constant-arg handling.
v, _ = self._run(
"src/Foo.cpp",
"SpanGuard::rootSpan(TraceCategory::Peer, seg::peer, "
"peer_span::op::validationReceive);\n",
)
self.assertEqual(v, [])
def test_test_path_exempt(self):
v, _ = self._run("src/test/Foo.cpp", 'g.setAttribute("lit_key", v);\n')
self.assertEqual(v, [])

62
.github/scripts/strategy-matrix/generate.py vendored
View File

@@ -25,24 +25,16 @@ def get_cmake_args(build_type: str, extra_args: str) -> str:
return " ".join(args)
def runs_on_event(exclude_event_types: list[str], event: str | None) -> bool:
"""Whether a config should run for the current event.
'exclude_event_types' is a list of GitHub event names (e.g.
["pull_request"]) on which the config should NOT run; an empty list means
the config runs on every event. When no event is given (event is None), no
filtering is applied.
"""
if event is None:
return True
return event not in exclude_event_types
# ---------------------------------------------------------------------------
# Input types — shapes of the JSON config files
# ---------------------------------------------------------------------------
# Every config must declare 'minimal'. Minimal configs form the reduced matrix
# built for pull requests by default; the full matrix adds the rest. Packaging
# configs declare it too, but packaging is gated in the workflow, not by it.
@dataclasses.dataclass
class LinuxConfig:
"""One entry in linux.json's 'configs' or 'package_configs' arrays."""
@@ -50,13 +42,11 @@ class LinuxConfig:
compiler: list[str]
build_type: list[str]
arch: list[str]
minimal: bool
sanitizers: list[str] = dataclasses.field(default_factory=list)
suffix: str = ""
extra_cmake_args: str = ""
image: str = "" # only used by package_configs entries
# List of GitHub event names (e.g. "pull_request") on which this config
# should NOT run. Empty means it runs on every event.
exclude_event_types: list[str] = dataclasses.field(default_factory=list)
@dataclasses.dataclass
@@ -89,11 +79,9 @@ class PlatformConfig:
"""One entry in macos.json's or windows.json's 'configs' array."""
build_type: list[str]
minimal: bool
build_only: bool = False # if true, skip tests (e.g. macos/Windows Debug)
extra_cmake_args: str = ""
# List of GitHub event names (e.g. "pull_request") on which this config
# should NOT run. Empty means it runs on every event.
exclude_event_types: list[str] = dataclasses.field(default_factory=list)
def __post_init__(self) -> None:
if isinstance(self.build_type, str):
@@ -168,20 +156,18 @@ _ARCHS: dict[str, Architecture] = {
}
def expand_linux_matrix(
linux: LinuxFile, event: str | None = None
) -> list[MatrixEntry]:
def expand_linux_matrix(linux: LinuxFile, minimal: bool) -> list[MatrixEntry]:
"""Expand a LinuxFile into a flat list of matrix entries.
Each config entry is expanded over the cross-product of its
compiler, build_type, sanitizers, and architecture lists. Configs that
exclude the current event are skipped.
compiler, build_type, sanitizers, and architecture lists. When 'minimal' is
true, only configs flagged as minimal are included.
"""
entries: list[MatrixEntry] = []
for distro, configs in linux.configs.items():
for cfg in configs:
if not runs_on_event(cfg.exclude_event_types, event):
if minimal and not cfg.minimal:
continue
# An empty sanitizers list means "one entry with no sanitizer".
effective_sanitizers = cfg.sanitizers or [""]
@@ -240,19 +226,17 @@ def expand_linux_packaging(linux: LinuxFile) -> list[PackagingEntry]:
return entries
def expand_platform_matrix(
pf: PlatformFile, event: str | None = None
) -> list[MatrixEntry]:
def expand_platform_matrix(pf: PlatformFile, minimal: bool) -> list[MatrixEntry]:
"""Expand a PlatformFile (macOS or Windows) into matrix entries.
Configs that exclude the current event are skipped.
When 'minimal' is true, only configs flagged as minimal are included.
"""
platform_name, arch = pf.platform.split("/")
is_windows = platform_name == "windows"
entries: list[MatrixEntry] = []
for cfg in pf.configs:
if not runs_on_event(cfg.exclude_event_types, event):
if minimal and not cfg.minimal:
continue
for build_type in cfg.build_type:
entries.append(
@@ -292,12 +276,12 @@ if __name__ == "__main__":
action="store_true",
)
parser.add_argument(
"-e",
"--event",
help="The GitHub event name that triggered the workflow (e.g. 'push', "
"'pull_request'). Configs are filtered by their 'event_type'. If "
"omitted, no filtering is applied.",
default=None,
"-m",
"--minimal",
help="Emit only the minimal matrix (the configs flagged 'minimal'), "
"used for pull requests by default. If omitted, the full matrix is "
"emitted.",
action="store_true",
)
args = parser.parse_args()
@@ -308,15 +292,15 @@ if __name__ == "__main__":
else:
if args.config in ("linux", None):
matrix += expand_linux_matrix(
LinuxFile.load(THIS_DIR / "linux.json"), args.event
LinuxFile.load(THIS_DIR / "linux.json"), args.minimal
)
if args.config in ("macos", None):
matrix += expand_platform_matrix(
PlatformFile.load(THIS_DIR / "macos.json"), args.event
PlatformFile.load(THIS_DIR / "macos.json"), args.minimal
)
if args.config in ("windows", None):
matrix += expand_platform_matrix(
PlatformFile.load(THIS_DIR / "windows.json"), args.event
PlatformFile.load(THIS_DIR / "windows.json"), args.minimal
)
print(f"matrix={json.dumps({'include': [dataclasses.asdict(e) for e in matrix]})}")

33
.github/scripts/strategy-matrix/linux.json vendored
View File

@@ -1,17 +1,31 @@
{
"image_tag": "sha-e29b523",
"image_tag": "sha-2e25435",
"configs": {
"ubuntu": [
{
"compiler": ["clang"],
"build_type": ["Release"],
"arch": ["amd64"],
"minimal": true
},
{
"compiler": ["gcc"],
"build_type": ["Release"],
"arch": ["amd64"],
"minimal": false
},
{
"compiler": ["gcc", "clang"],
"build_type": ["Debug", "Release"],
"arch": ["amd64", "arm64"]
"arch": ["arm64"],
"minimal": false
},
{
"compiler": ["gcc", "clang"],
"build_type": ["Debug", "Release"],
"arch": ["amd64"],
"minimal": false,
"sanitizers": ["address", "undefinedbehavior"]
},
@@ -19,6 +33,7 @@
"compiler": ["gcc"],
"build_type": ["Debug"],
"arch": ["amd64"],
"minimal": true,
"suffix": "coverage",
"extra_cmake_args": "-DUNIT_TEST_REFERENCE_FEE=500 -Dcoverage=ON -Dcoverage_format=xml -DCODE_COVERAGE_VERBOSE=ON -DCMAKE_C_FLAGS=-O0 -DCMAKE_CXX_FLAGS=-O0"
},
@@ -26,6 +41,7 @@
"compiler": ["clang"],
"build_type": ["Debug"],
"arch": ["amd64"],
"minimal": false,
"suffix": "voidstar",
"extra_cmake_args": "-Dvoidstar=ON"
},
@@ -33,6 +49,7 @@
"compiler": ["clang"],
"build_type": ["Release"],
"arch": ["amd64"],
"minimal": false,
"suffix": "reffee",
"extra_cmake_args": "-DUNIT_TEST_REFERENCE_FEE=1000"
},
@@ -40,9 +57,9 @@
"compiler": ["gcc"],
"build_type": ["Debug"],
"arch": ["amd64"],
"minimal": false,
"suffix": "unity",
"extra_cmake_args": "-Dunity=ON",
"exclude_event_types": ["pull_request"]
"extra_cmake_args": "-Dunity=ON"
}
],
@@ -50,7 +67,8 @@
{
"compiler": ["gcc"],
"build_type": ["Release"],
"arch": ["amd64"]
"arch": ["amd64"],
"minimal": false
}
],
@@ -58,7 +76,8 @@
{
"compiler": ["gcc"],
"build_type": ["Release"],
"arch": ["amd64"]
"arch": ["amd64"],
"minimal": false
}
]
},
@@ -68,6 +87,7 @@
"compiler": ["gcc"],
"build_type": ["Release"],
"arch": ["amd64"],
"minimal": false,
"image": "ghcr.io/xrplf/xrpld/packaging-debian:sha-577d745"
}
],
@@ -77,6 +97,7 @@
"compiler": ["gcc"],
"build_type": ["Release"],
"arch": ["amd64"],
"minimal": false,
"image": "ghcr.io/xrplf/xrpld/packaging-rhel:sha-577d745"
}
]

5
.github/scripts/strategy-matrix/macos.json vendored
View File

@@ -4,13 +4,14 @@
"configs": [
{
"build_type": "Release",
"extra_cmake_args": "-DCMAKE_POLICY_VERSION_MINIMUM=3.5"
"extra_cmake_args": "-DCMAKE_POLICY_VERSION_MINIMUM=3.5",
"minimal": true
},
{
"build_type": "Debug",
"extra_cmake_args": "-DCMAKE_POLICY_VERSION_MINIMUM=3.5",
"build_only": true,
"exclude_event_types": ["pull_request"]
"minimal": false
}
]
}

4
.github/scripts/strategy-matrix/windows.json vendored
View File

@@ -2,11 +2,11 @@
"platform": "windows/amd64",
"runner": ["self-hosted", "Windows", "dev-box-windows-2026"],
"configs": [
{ "build_type": "Release" },
{ "build_type": "Release", "minimal": true },
{
"build_type": "Debug",
"build_only": true,
"exclude_event_types": ["pull_request"]
"minimal": false
}
]
}