ARCHIVE/rippled
1
0
Fork 0
mirror of https://github.com/XRPLF/rippled.git synced 2026-07-29 18:10:34 +00:00
Code Issues Packages Projects Releases Wiki Activity

Rename nudb_bytes metric to stored_object_bytes

Browse Source 
The nudb_bytes label value on the storage_detail gauge named something the
code never measured. It observes Database::getStoreSize(), which returns the
storeSz_ accumulator: the cumulative payload bytes of objects this process
handed to the NodeStore. That is not a NuDB file size. It excludes NuDB's
keys, bucket padding and log, and it resets with the process while the files
on disk do not.

The name caused two concrete errors. It invited sizing the store on disk from
a number that cannot do it, and it invited a write-amplification ratio against
node_written_bytes -- which reads the same accessor at MetricsRegistry.cpp:836,
so that ratio is a constant 1.0 and measures nothing.

The nudb_ prefix was wrong too. storeSz_ is written only by
Database::storeStats(), called from DatabaseNodeImp, DatabaseRotatingImp and
Database itself. No backend code touches it, so the value reads the same on
RocksDB. That distinguishes it from the real nudb_* family
(nudb_writers_in_flight and friends), which come from getWriteStats() and are
absent entirely on a non-NuDB backend.

stored_object_bytes says what the value is and claims nothing about the
filesystem. Docs already described the value correctly; they keep that
explanation and now also record the old name, so a query pinned to it can be
traced. Neither Backend nor Database exposes an on-disk size accessor and none
was added -- no metric reports the store's on-disk size today.

Updates the node-health panel title, description and PromQL, and the four docs
that name the label value.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Pratik Mankawde
2026-07-28 15:46:24 +01:00
parent 8ade2b43d9
commit 7ef8c07cf9
6 changed files with 60 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
OpenTelemetryPlan/06-implementation-phases.md
View File

@@ -1680,18 +1680,22 @@ xrpld's `OperatingMode` enum maps 0-4 (DISCONNECTED through FULL). The external
**Task 7.13: Storage Detail Observable Gauge**
| Gauge Name | Label `metric=` | Type | Source |
| ---------------------- | --------------- | ----- | ---------------------------------------------------- |
| `xrpld_storage_detail` | `nudb_bytes` | int64 | `Database::getStoreSize()` — cumulative object bytes |
| Gauge Name | Label `metric=` | Type | Source |
| ---------------------- | --------------------- | ----- | ---------------------------------------------------- |
| `xrpld_storage_detail` | `stored_object_bytes` | int64 | `Database::getStoreSize()` — cumulative object bytes |
Despite the name, this is not a filesystem measurement. `getStoreSize()` sums the
object payloads this process has written, so it excludes NuDB's keys, bucket padding
and log, and it resets with the process while the files on disk do not. It is the
same accessor `node_written_bytes` uses, so the two series are equal by construction
and any write-amplification ratio built from the pair is a constant 1.0. There is no
This is not a filesystem measurement. `getStoreSize()` sums the object payloads this
process has written, so it excludes NuDB's keys, bucket padding and log, and it
resets with the process while the files on disk do not. It is the same accessor
`node_written_bytes` uses, so the two series are equal by construction and any
write-amplification ratio built from the pair is a constant 1.0. There is no
file-size accessor on `Backend` or `Database`, so no metric reports the store's
on-disk size today.
The label value was `nudb_bytes` through Phase 8 and was renamed in Phase 9: the
value is read from `Database`, not from the NuDB backend, so a backend prefix
misdescribed it and the old name implied an on-disk size it never reported.
**File**: `src/xrpld/telemetry/MetricsRegistry.cpp`
**Exit Criteria**:
@@ -1847,7 +1851,7 @@ Add checks to `validate_telemetry.py` for all new span attributes and metrics.
| `xrpld_ledgers_closed_total` |
| `xrpld_validations_sent_total` |
| `xrpld_state_changes_total` |
| `xrpld_storage_detail{metric="nudb_bytes"}` |
| `xrpld_storage_detail{metric="stored_object_bytes"}` |
**New dashboard load checks (~3)**:

27
OpenTelemetryPlan/09-data-collection-reference.md
View File

@@ -1565,18 +1565,23 @@ State value encoding: 0=disconnected, 1=connected, 2=syncing, 3=tracking, 4=full
#### Storage Detail (Observable Gauge — `storage_detail`)
| Prometheus Metric | Type | Labels | Description |
| ------------------------------------- | ----- | -------- | ---------------------------------------------------------- |
| `storage_detail{metric="nudb_bytes"}` | Int64 | `metric` | Cumulative object-payload bytes written (not on-disk size) |
| Prometheus Metric | Type | Labels | Description |
| ---------------------------------------------- | ----- | -------- | ---------------------------------------------------------- |
| `storage_detail{metric="stored_object_bytes"}` | Int64 | `metric` | Cumulative object-payload bytes written (not on-disk size) |
> **`nudb_bytes` is not a file size, despite the name.** It observes
> `getStoreSize()` (`src/xrpld/telemetry/MetricsRegistry.cpp:1507`), which sums the
> object payloads this process has written. It therefore excludes NuDB's keys,
> bucket padding and log, and it resets when the process restarts while the files
> on disk do not. `node_written_bytes` on the `nodestore_state` gauge calls the same
> accessor (`MetricsRegistry.cpp:836`), so the two series are equal by construction
> and any write-amplification ratio built from the pair is a constant 1.0. To size
> the store on disk, stat the backend's files; no metric reports it today.
> **`stored_object_bytes` is not a file size.** It observes `getStoreSize()`
> (`src/xrpld/telemetry/MetricsRegistry.cpp:1511`), which sums the object payloads
> this process has written. It therefore excludes NuDB's keys, bucket padding and
> log, and it resets when the process restarts while the files on disk do not.
> `node_written_bytes` on the `nodestore_state` gauge calls the same accessor
> (`MetricsRegistry.cpp:836`), so the two series are equal by construction and any
> write-amplification ratio built from the pair is a constant 1.0. To size the store
> on disk, stat the backend's files; no metric reports it today.
>
> This label value was called `nudb_bytes` before Phase 9. The value comes from
> `node_store::Database`, not from the NuDB backend, so it reads the same on
> RocksDB and carries no backend prefix. Queries and dashboards pinned to the old
> name return no data.
#### Synchronous Counters (Phase 7+)

21
OpenTelemetryPlan/Phase7_taskList.md
View File

@@ -482,22 +482,25 @@ validatorParticipationGauge_ = meter_->CreateInt64ObservableGauge(
> **Source**: [External Dashboard Parity](./06-implementation-phases.md#appendix-external-dashboard-parity)
**Objective**: Export NuDB-specific stored-bytes total and initial sync duration.
**Objective**: Export the NodeStore stored-bytes total and initial sync duration.
**Gauge label values**:
| Gauge Name | Label `metric=` | Type | Source |
| ---------------------- | ------------------------------- | ------ | ---------------------------------------------------- |
| `xrpld_storage_detail` | `nudb_bytes` | int64 | `Database::getStoreSize()` — cumulative object bytes |
| `xrpld_storage_detail` | `stored_object_bytes` | int64 | `Database::getStoreSize()` — cumulative object bytes |
| `xrpld_sync_info` | `initial_sync_duration_seconds` | double | Time from start to first FULL |
Despite the name, `nudb_bytes` is not a file size. `getStoreSize()` sums the object
payloads this process has written, so it excludes NuDB's keys, bucket padding and
log, and it resets when the process restarts while the files on disk do not. It is
the same accessor `node_written_bytes` uses, so the two series are equal by
construction and any write-amplification ratio built from the pair is a constant 1.0.
Neither `Backend` nor `Database` exposes a file-size accessor, so no metric reports
the store's on-disk size today.
`stored_object_bytes` is not a file size. `getStoreSize()` sums the object payloads
this process has written, so it excludes NuDB's keys, bucket padding and log, and it
resets when the process restarts while the files on disk do not. It is the same
accessor `node_written_bytes` uses, so the two series are equal by construction and
any write-amplification ratio built from the pair is a constant 1.0. Neither
`Backend` nor `Database` exposes a file-size accessor, so no metric reports the
store's on-disk size today.
This label value was `nudb_bytes` when Phase 7 shipped it and was renamed in Phase 9,
because the value comes from `Database` rather than the NuDB backend.
**Key modified files**: `src/xrpld/telemetry/MetricsRegistry.h/.cpp`

6
docker/telemetry/grafana/dashboards/node-health.json
View File

@@ -1347,8 +1347,8 @@
}
},
{
"title": "NuDB Stored Bytes",
"description": "###### What this is:\n*Cumulative object-payload bytes this process has written to the NuDB object-store back end. Despite the metric name, this is not the size of the store on disk.*\n\n###### How it's computed:\n*Current value of the nudb_bytes gauge, plotted over time. It observes getStoreSize(), the same accessor node_written_bytes uses, so the two series are equal and their ratio is a constant 1.0 rather than a write-amplification measure.*\n\n###### Reading it:\n*A smoothly growing line is normal; the slope is the write rate. It excludes NuDB's keys, bucket padding and log, and it restarts from zero with the process while the files on disk do not.*\n\n###### Healthy range:\n*Gradual growth consistent with ledger data being stored.*\n\n###### Watch for:\n*A sudden jump in growth rate can indicate runaway storage or an unexpected back-fill. Do not use this panel to size the store on disk or to plan disk capacity; no metric reports on-disk size today, so check the filesystem directly.*\n\n###### Source:\n[MetricsRegistry.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/telemetry/MetricsRegistry.cpp)\n\n###### Function:\n`registerStorageDetailGauge`",
"title": "Stored Object Bytes",
"description": "###### What this is:\n*Cumulative object-payload bytes this process has written to the NodeStore object-store back end. This is not the size of the store on disk.*\n\n###### How it's computed:\n*Current value of the stored_object_bytes gauge, plotted over time. It observes getStoreSize(), the same accessor node_written_bytes uses, so the two series are equal and their ratio is a constant 1.0 rather than a write-amplification measure.*\n\n###### Reading it:\n*A smoothly growing line is normal; the slope is the write rate. It excludes NuDB's keys, bucket padding and log, and it restarts from zero with the process while the files on disk do not.*\n\n###### Healthy range:\n*Gradual growth consistent with ledger data being stored.*\n\n###### Watch for:\n*A sudden jump in growth rate can indicate runaway storage or an unexpected back-fill. Do not use this panel to size the store on disk or to plan disk capacity; no metric reports on-disk size today, so check the filesystem directly.*\n\n###### Source:\n[MetricsRegistry.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/telemetry/MetricsRegistry.cpp)\n\n###### Function:\n`registerStorageDetailGauge`",
"type": "timeseries",
"gridPos": {
"h": 8,
@@ -1368,7 +1368,7 @@
"datasource": {
"type": "prometheus"
},
"expr": "label_replace(storage_detail{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", xrpl_work_item=~\"$xrpl_work_item\", xrpl_branch=~\"$xrpl_branch\", xrpl_node_role=~\"$xrpl_node_role\", metric=\"nudb_bytes\"}, \"series\", \"NuDB Stored Bytes\", \"\", \"\")"
"expr": "label_replace(storage_detail{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", xrpl_work_item=~\"$xrpl_work_item\", xrpl_branch=~\"$xrpl_branch\", xrpl_node_role=~\"$xrpl_node_role\", metric=\"stored_object_bytes\"}, \"series\", \"Stored Object Bytes\", \"\", \"\")"
}
],
"fieldConfig": {

8
docs/telemetry-runbook.md
View File

@@ -1803,11 +1803,13 @@ Two more pairs from the same family:
quantity, the larger of the recorded load and the pending batch size
(`src/libxrpl/nodestore/BatchWriter.cpp:47-53`), so it is a batch-queue length
rather than a thread count.
- **`nudb_bytes` is not the size of the store on disk**, despite the name. It reports
the cumulative object-payload bytes this process has written — the same value as
- **`stored_object_bytes` is not the size of the store on disk.** It reports the
cumulative object-payload bytes this process has written — the same value as
`node_written_bytes`, from the same accessor — so it excludes keys, padding and
the log, and it resets with the process. A ratio of the two is a constant 1.0 and
measures nothing.
measures nothing. This label value was called `nudb_bytes` before Phase 9; it
comes from `node_store::Database` rather than the NuDB backend, so it is not part
of the `nudb_*` family above and reads the same on RocksDB.
- These gauges are sampled on the `MetricsRegistry` reader's 10 s cadence, while
the `jobq_*` lane gauges are sampled at 1 s by a different provider. Widen the
window when correlating them rather than reading a single scrape; see the caveat

18
src/xrpld/telemetry/MetricsRegistry.cpp
View File

@@ -1472,7 +1472,8 @@ MetricsRegistry::registerStorageDetailGauge()
// --- Task 7.13: Storage detail gauges ---
// Reports the cumulative payload bytes handed to the NodeStore. See the
// note at the observe() call below: this is logical bytes stored, not
// on-disk file size, because no accessor for the latter exists.
// on-disk file size, because no accessor for the latter exists. The label
// value names it that way so it is not read as a filesystem measurement.
storageDetailGauge_ =
meter_->CreateInt64ObservableGauge("storage_detail", "Storage detail metrics");
storageDetailGauge_->AddCallback(
@@ -1490,11 +1491,13 @@ MetricsRegistry::registerStorageDetailGauge()
->Observe(value, {{"metric", name}});
};
// Cumulative payload bytes handed to the NodeStore -- NOT
// on-disk file size, despite the name. getStoreSize() sums
// the object payloads this process has written, so it
// excludes NuDB's keys, bucket padding and log, and it
// resets with the process while the files do not.
// Cumulative payload bytes handed to the NodeStore. This is
// not an on-disk file size: getStoreSize() sums the object
// payloads this process has written, so it excludes NuDB's
// keys, bucket padding and log, and it resets with the
// process while the files do not. The value comes from
// Database, not from any backend, so it carries no nudb_
// prefix -- it reads the same on RocksDB.
//
// This is the same call node_written_bytes makes on the
// nodestore_state gauge, so the two series are equal by
@@ -1504,7 +1507,8 @@ MetricsRegistry::registerStorageDetailGauge()
// computing one would mean stat()ing the backend's files
// from the reader thread, which needs a new Backend method
// rather than a change at this call site.
observe("nudb_bytes", static_cast<int64_t>(app.getNodeStore().getStoreSize()));
observe(
"stored_object_bytes", static_cast<int64_t>(app.getNodeStore().getStoreSize()));
}
catch (...) // NOLINT(bugprone-empty-catch)
{