fix(telemetry): keep filelog offset storage out of the shared collector config

The file_storage extension was added to otel-collector-config.yaml, which
every stack mounts. That made the extension mandatory: the collector image
runs as 10001:10001 and ships no writable directory, so any stack without a
prepared volume would fail to start rather than merely lose offsets. The
workload-validation stack mounts this same config and has no such volume.

Offset persistence is only useful where logs outlive a restart. The workload
harness creates a fresh log directory per run, so it has nothing to resume
from. Move the extension, the receiver's storage reference and the extended
service.extensions list into otel-collector-filestorage.yaml, layered as a
second --config by the developer stack alone. The base config keeps
start_at: beginning, which is what actually fixes the reported defect, and
stays self-sufficient for every other stack.

Verified against the pinned collector image: the base config validates and
runs on its own with no volume mounted and still ingests a line written
before startup; base plus overlay validates, preserves the base receiver's
operators through the merge, and re-ingests that line zero times on a second
run against the same volume.
This commit is contained in:
Pratik Mankawde
2026-08-17 11:44:30 +01:00
parent de0fefc95b
commit 301149d131
4 changed files with 46 additions and 17 deletions

View File

@@ -44,7 +44,14 @@ services:
# batches them for efficiency, and forwards to Tempo for storage.
otel-collector:
image: otel/opentelemetry-collector-contrib:0.158.0
command: ["--config=/etc/otel-collector-config.yaml"]
# Second --config layers filelog offset persistence on top of the shared
# base config; the collector deep-merges them. Only this stack keeps its
# logs across restarts, so only this stack needs it.
command:
[
"--config=/etc/otel-collector-config.yaml",
"--config=/etc/otel-collector-filestorage.yaml",
]
ports:
- "4317:4317" # OTLP gRPC
- "4318:4318" # OTLP HTTP (traces + native OTel metrics)
@@ -55,6 +62,8 @@ services:
volumes:
# Mount collector pipeline config (receivers → processors → exporters)
- ./otel-collector-config.yaml:/etc/otel-collector-config.yaml:ro
# Dev-only overlay: persist filelog read offsets across restarts
- ./otel-collector-filestorage.yaml:/etc/otel-collector-filestorage.yaml:ro
# Mount the xrpld log root for the filelog receiver. The telemetry
# configs write to docker/telemetry/data/logs/<network>/debug.log, so
# the default source is the repo-relative ./data/logs — user-owned and