Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill

Conflict resolutions:

- docker/telemetry/xrpld-telemetry.cfg: relocation conflict. phase-9 had
  already moved [insight] to the end of the file with server=otel, so the
  incoming block was dropped rather than inserted. Keeping both would have
  produced two [insight] sections, which merge last-wins into a single
  effective section, silently reviving the bug this branch just fixed.
  phase-9's per-branch service_instance_id=xrpld-devnet is preserved.

- OpenTelemetryPlan/06-implementation-phases.md: kept both corrections.
  phase-9's "Tempo" is right (no Jaeger anywhere in the stack) and
  phase-8's "active, sampled span" is right: Log.cpp:328 injects only
  when spanCtx.IsValid() && spanCtx.IsSampled().

- OpenTelemetryPlan/09-data-collection-reference.md and
  docs/telemetry-runbook.md: kept phase-9's structured-metadata LogQL.
  The collector's filelog regex_parser already extracts partition,
  severity, trace_id and span_id, so phase-8's inline regexp forms are
  redundant, and a line filter matches the literal text in a message body.

OpenTelemetryPlan/06-implementation-phases.md

@@ -627,11 +627,11 @@ See [Phase7_taskList.md](./Phase7_taskList.md) for detailed per-task breakdown.
 
 ---
 
-## 6.9 Phase 8: Log-Trace Correlation and Centralized Log Ingestion (Week 13)
+## 6.8.1 Phase 8: Log-Trace Correlation and Centralized Log Ingestion (Week 13)
 
 ### Motivation
 
-xrpld's `beast::Journal` logs and OpenTelemetry traces are currently two disjoint observability signals. When investigating an issue, operators must manually correlate timestamps between log files and Tempo traces. Phase 8 bridges this gap by injecting trace context (`trace_id`, `span_id`) into every log line emitted within an active span, and ingesting those logs into Grafana Loki via the OTel Collector's filelog receiver.
+xrpld's `beast::Journal` logs and OpenTelemetry traces are currently two disjoint observability signals. When investigating an issue, operators must manually correlate timestamps between log files and Tempo traces. Phase 8 bridges this gap by injecting trace context (`trace_id`, `span_id`) into every log line emitted within an active, sampled span, and ingesting those logs into Grafana Loki via the OTel Collector's filelog receiver.
 
 #### Gains
 
@@ -649,7 +649,7 @@ xrpld's `beast::Journal` logs and OpenTelemetry traces are currently two disjoin
 
 #### Decision
 
-The correlation value far outweighs the risks. The log format change is backward-compatible (fields are appended only when a span is active), and the filelog receiver regex is straightforward to maintain.
+The correlation value far outweighs the risks. The log format change is backward-compatible (fields are appended only when a sampled span is active), and the filelog receiver regex is straightforward to maintain.
 
 ### Architecture
 

OpenTelemetryPlan/09-data-collection-reference.md

@@ -943,7 +943,7 @@ state_accounting_full_duration
 > **Plan details**: [06-implementation-phases.md §6.8.1](./06-implementation-phases.md) — motivation, architecture, Mermaid diagrams
 > **Task breakdown**: [Phase8_taskList.md](./Phase8_taskList.md) — per-task implementation details
 
-Phase 8 injects OTel trace context into xrpld's `Logs::format()` output, enabling log-trace correlation. When a log line is emitted within an active OTel span, the trace and span identifiers are automatically appended after the severity field:
+Phase 8 injects OTel trace context into xrpld's `Logs::format()` output, enabling log-trace correlation. When a log line is emitted within an active, sampled OTel span, the trace and span identifiers are automatically appended after the severity field:
 
 ### Log Format
 
@@ -959,7 +959,7 @@ Example:
 
 - **`trace_id=<hex32>`** — 32-character lowercase hex trace identifier. Links to the distributed trace in Tempo.
 - **`span_id=<hex16>`** — 16-character lowercase hex span identifier. Identifies the specific span within the trace.
-- **Only present** when the log is emitted within an active OTel span. Log lines outside of traced code paths have no trace context fields.
+- **Only present** when the log is emitted within an active OTel span whose context is sampled. Log lines outside of traced code paths, and lines inside a span the sampler dropped, have no trace context fields. A dropped span still carries its parent's identifiers, so emitting them would point at a trace that was never exported.
 
 ### Implementation
 

OpenTelemetryPlan/Phase8_taskList.md

@@ -19,7 +19,7 @@
 
 ## Task 8.1: Inject trace_id into Logs::format()
 
-**Objective**: Add OTel trace context to every log line that is emitted within an active span.
+**Objective**: Add OTel trace context to every log line that is emitted within an active, sampled span. The sampled flag matters because a span dropped by the `ParentBasedSampler` still carries its parent's ids, so emitting them would advertise a trace that was never exported.
 
 **What to do**:
 
@@ -36,7 +36,7 @@
             auto span = opentelemetry::nostd::get<
                 opentelemetry::nostd::shared_ptr<opentelemetry::trace::Span>>(spanValue);
             auto spanCtx = span->GetContext();
-            if (spanCtx.IsValid())
+            if (spanCtx.IsValid() && spanCtx.IsSampled())
             {
                 char traceId[32], spanId[16];
                 spanCtx.trace_id().ToLowerBase16(
@@ -62,7 +62,7 @@
 
 - `src/libxrpl/basics/Log.cpp`
 
-**Performance note**: The implementation checks the thread-local context value directly (avoiding the heap allocation that `GetSpan()` performs on the no-span path). On threads without an active span (~99% of log lines), the cost is a thread-local read + variant type check (~15-20ns). On the active-span path, an additional shared_ptr copy + `GetContext()` + `IsValid()` adds ~50ns total. Overhead is negligible at typical logging rates.
+**Performance note**: The implementation checks the thread-local context value directly (avoiding the heap allocation that `GetSpan()` performs on the no-span path). On threads without an active span (~99% of log lines), the cost is a thread-local read + variant type check (~15-20ns). On the active-span path, an additional shared_ptr copy + `GetContext()` + `IsValid()`/`IsSampled()` adds ~50ns total. Overhead is negligible at typical logging rates.
 
 ---
 
@@ -230,7 +230,7 @@
 
 **Exit Criteria** (from [06-implementation-phases.md §6.8.1](./06-implementation-phases.md)):
 
-- [ ] Log lines within active spans contain `trace_id=<hex> span_id=<hex>`
+- [ ] Log lines within active, sampled spans contain `trace_id=<hex> span_id=<hex>`
 - [ ] Log lines outside spans have no trace context (no empty fields)
 - [ ] Loki ingests xrpld logs via OTel Collector filelog receiver
 - [ ] Grafana Tempo -> Loki one-click correlation works