3860c93db2
These changes were developed on the phase-10 branch but belong to content this branch and its upstreams introduced. Carrying them on phase-10 made its PR diff report churn in files phase-10 does not own, and left each PR claiming a scope that did not match its contents. Moved here from phase-10 (identical content, no functional change): - Dashboards: all 14 existing boards plus the new log-derived-insights board. - Docs: telemetry-runbook.md (minus the workload/benchmark sections, which describe phase-10 tooling) and the new telemetry-glossary.md. - Grafana Cloud + Alloy export path: collector config, compose override, the two .env examples and alloy/config.alloy. - Local stack: otel-collector-config.yaml gains sub-millisecond and second-scale spanmetrics buckets, pins unit=ms, and promotes close_time_correct; integration-test.sh and TESTING.md follow. - Node configs: exported_instance -> service_instance_id in comments; the mainnet sample now logs at warning to bound log volume. - Metrics code: Telemetry.cpp builds the metrics pipeline in the constructor via initMetrics() so the global MeterProvider is published before any subsystem creates a beast::insight instrument, and the histogram view keeps each instrument's own name instead of collapsing them under one series. MetricsRegistry gains a last_close_time gauge and skips negative job-queue durations. OTelCollector drops an unused accessor. - Naming CI: xrpl_work_item joins EXTERNAL_INFRA_LABELS and Rule E accepts the dotted perf-iac resource-attribute form. This must travel with the dashboards and runbook that reference those labels, or the rules fail. - Doxygen input glob no longer recurses dot-directories. Sections describing phase-10 tooling stay on phase-10 and keep their "Future Enhancement" / "Planned, not yet implemented" markers here; phase-10 removes those markers when it lands the tooling.
OTel naming-consistency check
check_otel_naming.py enforces the OpenTelemetry span-attribute naming
convention documented in
CONTRIBUTING.md
across every layer of the telemetry pipeline. The *SpanNames.h constants are
the single source of truth (L1); every other layer must agree with them.
Running locally
python .github/scripts/otel-naming/check_otel_naming.py
It takes no arguments, can be run from any directory inside the repo, and uses
only the Python standard library (no pip install, matching the levelization
check). A non-zero exit code means a violation was found; the output lists each
violation as RULE | location | token | expected.
What it checks
The valid key set is derived dynamically from the OTel code — there is no hardcoded allowlist:
- L1 keys come from the
namespace attr { ... }blocks of every*SpanNames.h, resolving themakeStr("x")/join(seg::a, seg::b)DSL (cross-file, sojoin(seg::rpc, ...)resolvesseg::rpcfrom the baseSpanNames.h). Each constant is resolved against its own header, so two headers that define a same-named constant (e.g. a baseattr::ledgerHashand a domainattr::ledgerHash) each contribute their real wire key — a later header cannot clobber an earlier one's value in a flat table. - Legitimate dotted keys = ONLY the keys the code actually sets as resource
attributes, i.e. the entries inside
Telemetry.cpp'sResource::Create({...})call: thesemconv::service::*keys (service.*) plus anyattr::<name>constants passed there (xrpl.network.*). A dotted key that is declared in a header but never set as a resource attr is a span attribute in resource clothing — a Rule-A violation, even if it lives in the baseSpanNames.h.
Rules (each fails the build, when its inputs are present)
| Rule | Check |
|---|---|
| A | No stray dotted span-attribute key (only the derived resource keys may be dotted). |
| G | Attribute keys are lower_snake_case (^[a-z][a-z0-9_]*$ per dot-segment) — no camelCase, UPPERCASE, or spaces. |
| F | No string literals as attribute keys or span-name arguments in setAttribute/addEvent/span/rootSpan/childSpan (rootSpan shares span's (cat, prefix, name) signature). Attribute values are exempt (runtime data); *SpanNames.h definitions and test files are exempt. |
| B | Every collector spanmetrics.dimensions name exists in the L1 key set. |
| C | Every Tempo span-filter tag exists in the L1 key set. |
| D | Every dashboard label resolves to an L1 span attribute, a native-metric label (L6, emitted by MetricsRegistry), or a Prometheus/Grafana builtin. TraceQL scope prefixes (span./resource./…) are stripped before the L1 lookup. |
| E | No dotted xrpl.<domain>.<field> attribute key in the runbook (only the L1 resource attrs xrpl.network.* may be dotted). Span names, filenames, OTel-standard keys, and metric labels are not flagged. |
Rule F runs unconditionally (it is a purely syntactic check on the
call-sites and needs no *SpanNames.h), so a code path that calls
SpanGuard::span/setAttribute directly without ever defining a header is
still caught.
Warnings (printed, never fail the build)
| Rule | Check |
|---|---|
| H | A namespace-qualified constant (e.g. foo::bar::myKey) used at a telemetry call-site is not defined in any *SpanNames.h. The constant should live in the proper header; defining it in-place bypasses rules A/G/F. Warns rather than fails — the argument may be a legitimately dynamic value, and the header may live on a later branch. Bare locals and std:: names are not warned. |
Presence-gated
Every rule runs only when the source files it needs are present in the tree
and is otherwise skipped (printed as SKIP: <rule> — <reason>), never failed.
This keeps the check correct no matter how telemetry work is split across PRs —
a stacked chain, one large PR, or independent per-stage PRs where (for example)
the collector config lands before the dashboards. The collector/Tempo/dashboard/
runbook layers are introduced in later phases; on a branch without them, only
the L1-intrinsic rules (A, G, F) run.