Commit Graph
230 Commits
Author SHA1 Message Date
Pratik Mankawde 2feb874df2 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Conflict resolved in PeerImp.cpp, proposal receive handler: keep this
branch's proposalSpan name, and take the arriving change that
allocates the holder only for a live span.
2026-09-30 23:05:38 +01:00
Pratik Mankawde 99c27c5a65 docs(telemetry): Correct the txq.enqueue and txq.cleanup runbook rows
txq.enqueue is the admission decision on every TxQ::apply call, with
the outcome in txq_status. On a rebuild in doAccept it is a child of
consensus.accept.apply, not a root.

txq.cleanup runs once per closed ledger, even when nothing expired. It
updates the fee metrics, resizes the queue and expires old entries.
2026-09-30 19:47:47 +01:00
Pratik Mankawde 74df02b9d7 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-25 14:52:46 +01:00
Pratik Mankawde c5db8e9d7d Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-25 14:48:01 +01:00
Pratik Mankawde e8dffdf678 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-25 14:48:01 +01:00
Pratik Mankawde 77e998668b Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-25 14:48:01 +01:00
Pratik Mankawde f5bdd11d60 docs(telemetry): List transactions among the traced work
The overview said only RPC requests are traced. This branch adds spans
for the transaction lifecycle, so name it too.
2026-09-25 14:42:39 +01:00
Pratik Mankawde 4e1d62f52d docs(telemetry): Explain how to run the local trace stack
The build guide said it covers setting up a backend to view traces, but
never mentioned docker/telemetry. Add the steps from build to Grafana,
linking the compose file and the example config for ports and keys
rather than repeating them, and a hop-by-hop check for missing traces.
2026-09-25 14:42:29 +01:00
Pratik Mankawde 4d7ae04751 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-25 14:38:07 +01:00
Pratik Mankawde 1c59adb110 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-25 14:37:43 +01:00
Pratik Mankawde 2c457b466f Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-25 14:26:21 +01:00
Pratik Mankawde 8730235eb1 fix(telemetry): Check peer trace context on parse
A peer's TraceContext was only checked when choosing a span parent.
Check it once, right after a P2P message is parsed, so every later
reader of the message sees a well-formed context:

- trace_id must be 16 bytes and span_id 8 bytes, neither all zero.
- trace_flags must fit 8 bits. Only the sampled and random bits are
  kept, the same mask the OpenTelemetry SDK applies.
- A context that fails any check is dropped as a whole. The message
  itself is kept.

The check has no OpenTelemetry dependency, so it also runs in builds
without telemetry. static_asserts tie its limits to the OpenTelemetry
types. traceFlagsByte() replaces the repeated trace_flags cast.

GTest cases cover every rule, and Beast cases send messages through
the real parser, including a compressed frame.
2026-09-25 14:22:32 +01:00
Pratik Mankawde 9edf2ea560 docs(telemetry): List rpc.startup in the runbook span tables
Adds the span to the RPC span table and to the span, metric and
dashboard summary. It has no dashboard panel.
2026-09-25 14:01:55 +01:00
Pratik Mankawde bf0670991a Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Conflicts: docker/telemetry/otel-collector-config.yaml (kept phase-6's
config, removed the attributes/hash processor as phase-2 did) and
docs/telemetry-runbook.md (kept phase-6's span table, applied phase-5's
tx.process attribute additions to its row).
2026-09-23 18:04:35 +01:00
Pratik Mankawde 021f20b311 docs(telemetry): Path-find spans carry raw account addresses 2026-09-23 18:03:36 +01:00
Pratik Mankawde c13e83d7f4 docs(telemetry): List the per-role account attributes in the runbook span table 2026-09-23 18:03:18 +01:00
Pratik Mankawde 243913250b Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-21 12:45:59 +01:00
Pratik Mankawde c8a8da6fd6 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-21 12:45:55 +01:00
Pratik Mankawde e5d28ccbc4 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-21 12:45:54 +01:00
Pratik Mankawde 870367d71e merge: bring develop forward into phase-3 tx tracing
Resolved src/libxrpl/tx/applySteps.cpp. calculateBaseFee now returns
std::expected<XRPAmount, TER>, so invokePreclaim rejects the transaction when
the fee cannot be computed. Kept that inside this branch's preclaimTer wrapper,
so the returned error also reaches the span's ter_result attribute.
2026-09-21 12:45:45 +01:00
Pratik Mankawde f48a2b4871 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-21 12:43:44 +01:00
Pratik Mankawde 675dffdc71 docs(telemetry): drop suppressed from the tx.receive attribute list
The span is created after the duplicate check, so it never carries the
attribute, and the row now says which transactions it covers.
2026-09-18 09:31:23 +01:00
Pratik Mankawde c1086e63a2 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-17 17:01:20 +01:00
Pratik Mankawde aea4431807 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-17 17:01:20 +01:00
Pratik Mankawde de3e376944 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-17 17:01:20 +01:00
Pratik Mankawde 94bdd19608 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-17 17:01:20 +01:00
Pratik Mankawde e52f235356 build: make the Conan option the only telemetry switch
CMakeLists declared option(telemetry) with a default of its own, so the setting
had two homes and they were free to disagree. The Conan option already reaches
CMake without it: conanfile.py forwards the option into the generated toolchain,
which sets the variable this file reads, and every build here goes through Conan.

Drop the option() call and keep the if(telemetry) test. docs/build/telemetry.md
told readers a CMake option had to be set as well, so that goes with it.
2026-09-17 16:54:03 +01:00
Ayaz Salikhov 295b74da1c build: Add assert-enabled builds and packages 2026-09-17 13:33:05 +02:00
Pratik Mankawde c622da5a76 refactor(telemetry): use the current name for the span metrics connector
The pinned collector warns on every start that "spanmetrics" is a deprecated
alias for "span_metrics". Rename the connector, its pipeline references and
the prose that names it.

The derived metric names are untouched. They come from the connector's
`namespace` setting rather than its component name, so occurrences inside a
metric name such as traces_spanmetrics_calls_total are deliberately left as
they are; renaming those would break every span panel. The rename is applied
only to the bare word, never where it is joined to a metric name by
underscores.

This branch introduces the connector, so the change belongs here.
2026-09-09 15:40:11 +01:00
Pratik Mankawde e612e5d9fa merge: bring the CI fixes forward from phase5-docs-deployment 2026-09-08 17:14:06 +01:00
Pratik Mankawde 9c9cb9d091 fix(telemetry): check TLS paths and fix the runbook build steps
requireReadableFile proved a path readable with getFileContents, which
loads the whole file into a std::string and then drops it. One of the
three paths it checks is tls_client_key, so a private key was loaded to
answer a question that does not need its contents. It now stats the
path, rejects anything that is not a regular file, and opens it without
reading. The message shape is unchanged:
"[telemetry] <key> cannot be read: <path> - <reason>".

A path naming a directory used to escape as an ios failure from the
stream buffer, naming neither the config key nor the path. It is now
rejected as "not a regular file" with both named. The new test covers
that case; it fails against the old implementation and against a copy
with the file-type branch removed.

The runbook's quick start and disable sections both told the reader to
run "cmake --preset default". No presets file is tracked, and the only
preset Conan generates is conan-release, so each of those steps failed
on its first command. Replaced with the flow BUILD.md documents, and
noted that telemetry is the current default while still passing the
flags.
2026-09-08 16:54:30 +01:00
Pratik Mankawde 9977810c6d merge: bring the review fixes forward from phase5-docs-deployment
One conflict, in docs/telemetry-runbook.md: both sides had independently
corrected the same consensus_round_id example. This branch kept the pipe form,
which Tempo rejects as a parse error; upstream moved the predicate inside the
braces, which parses and returns data. Upstream's query is kept, with this
branch's note that the value is the previous ledger sequence plus one.
2026-09-08 15:33:02 +01:00
Pratik Mankawde 039c2768ba fix(telemetry): require an https endpoint when a client certificate is set
The OTLP/HTTP exporter selects TLS from the endpoint URL scheme alone
(HttpSslOptions in the pinned SDK matches "https:" exactly), so a client
certificate handed to it alongside an http:// traces_endpoint is loaded and
never presented. The parser checked the cert/key pairing, use_tls and file
readability, but never the scheme, and the default traces_endpoint is plain
HTTP. makeTelemetrySetup() now requires traces_endpoint to start with
"https://" whenever tls_client_cert is set, including when the key is left
at its default.

Nothing asserted the client options reaching the exporter, so a swapped
certificate and key would have passed every test. Move the options mapping
into makeTraceExporterOptions() and assert it at that boundary with
distinct certificate and key paths, plus a one-way-TLS control and a
use_tls=0 control. One case runs the whole path from a [telemetry] section.

Runbook and example-config fixes:

- tx.included is emitted per transaction of the agreed consensus set,
  before buildLCL() applies anything, so it is a superset of the accepted
  ledger rather than proof of inclusion.
- the dispute.resolve query used the descendant operator, but the event is
  on the consensus.update_positions span itself, so it matched nothing.
- the exhausted-retries query asked for txq_status="retried" with
  retries_remaining=0, which cannot occur: the attribute is stamped before
  the attempt and the retried branch only runs while retries are left.
  Exhaustion is txq_status="failed" with a zero count.
- consensus_round_id is an int64, so the two queries comparing it to a
  quoted string matched nothing.
- note that consensus_trace_strategy=random is experimental and not used.
- note that a trailing "| attr = value" is rejected by current Tempo;
  attribute filters belong inside the braces.
2026-09-08 14:41:39 +01:00
Pratik Mankawde 981323f071 merge: bring the close-time attr doc fixes forward from phase5-docs-deployment 2026-09-04 12:40:21 +01:00
Pratik Mankawde ca22f57919 docs(telemetry): follow the close-time attr rename in the data reference
The emitted keys carry the unit and epoch suffix. Update the consensus
and ledger attribute tables and the ledger.build span row to match.

The Close Time Drift panel row is left alone: phase-7 removes that whole
table, so editing it here would only conflict on the way forward.
2026-09-04 12:37:19 +01:00
Pratik Mankawde 1afa35d54d docs(telemetry): follow the close-time attr rename in the runbook
The consensus.accept.apply row listed close_time, parent_close_time and
close_time_self. The emitted keys carry the unit and epoch suffix, so
update the row to match.
2026-09-04 12:35:59 +01:00
Pratik Mankawde 91c7137373 merge: bring the traces_endpoint rename forward from phase-5 2026-09-03 15:20:30 +01:00
Pratik Mankawde f87dff3eb6 docs(telemetry): follow the traces_endpoint rename in the runbook
The rename arrived from phase-1b by merge; the runbook still told
operators to set `endpoint`, which the parser no longer reads. Updates
the Quick Start ini block and the Configuration Reference row.

No metrics_endpoint row is added: this branch exports no metrics, so
documenting the key here would describe something the code ignores.
2026-09-03 15:16:39 +01:00
Pratik Mankawde 5f53745fbf docs(telemetry): correct the TESTING guide and the span attribute tables
Review feedback on the testing guide:

- rm -rf targeted data/, but this config writes under docker/telemetry/data/,
  so teardown did nothing and a second run reused the old NuDB and SQLite
  state. Corrected at both sites, including the Test 2 keygen node, which
  launches with the same config.
- The standalone span table said consensus.* does not fire. It does:
  ledger_accept drives a simulated round, so consensus.round, .phase.open,
  .ledger_close, .accept and .accept.apply all appear. Only .establish,
  .update_positions, .check, .proposal.* , .validation.receive and
  .mode_change cannot. The test intro claimed the same thing and now agrees
  with the table.
- Three blocks duplicated content the file already had. Test 1 now points at
  the shared Verification Queries section as Test 2 already did, and the
  Test 2 submit block checks engine_result like Test 1 does.
- The numbered step list was a copy of the script's own Step N headers and had
  drifted by four entries, so it now points at those headers instead.

Also corrects the runbook's ledger and peer span tables against the code:
ledger.build was credited with tx_count and tx_failed, which tx.apply sets,
and was missing its three close-time attributes; peer.validation.receive was
missing ledger_hash and full_validation. The five source line numbers in those
two tables were stale, so they now name the file only, as the other nineteen
rows do.
2026-09-03 10:52:04 +01:00
Pratik Mankawde 676c19b838 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-08-27 16:04:36 +01:00
Pratik Mankawde f293f65bd0 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-08-27 16:01:02 +01:00
Pratik Mankawde b060c76a76 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-08-27 16:01:01 +01:00
Pratik Mankawde cc24101629 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-08-27 16:01:01 +01:00
Pratik Mankawde fff4124d9b Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-08-27 16:00:25 +01:00
Ayaz Salikhov 3967ed6d54 build: Update release-info to get better pkg_release (#8131) 2026-08-27 13:34:38 +00:00
Pratik Mankawde 6a01f96877 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-08-27 14:04:50 +01:00
Pratik Mankawde e8635f3452 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-08-27 14:04:50 +01:00
Pratik Mankawde e85957da95 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-08-27 14:04:50 +01:00
Pratik Mankawde 295e147214 docs(telemetry): list the injection subsection in the contents
The table of contents in this file indexes third- and fourth-level headings,
so a new subsection that is absent from it is a gap rather than a style
choice.
2026-08-27 14:04:45 +01:00
Pratik Mankawde 8db0ded57b docs(telemetry): document the four states of current-context injection
Compiled out, compiled in and tracing, compiled in with no active span, and
compiled in but disabled by config all have to produce the right wire bytes,
and only two of them are obvious. Tabulate them, and record why the predicate
reads the context directly instead of calling GetSpan(), which allocates a
DefaultSpan in the no-span case.
2026-08-27 14:02:14 +01:00