The NuDB database backend is a high performance key/value store presented
as an alternative to RocksDB on Mac and Linux deployments, and the preferred
backend option for Windows deployments. The LevelDB backend is deprecated for
all platforms.
This includes these changes:
* Add Backend::verify API for doing consistency checks
* Add Database::close so caller can catch exceptions
* Improved Timing test for NodeStore creates a simulated workload
NuDB is a high performance key/value database optimized for insert-only
workloads, with these features:
* Low memory footprint
* Values are immutable
* Value sizes from 1 2^48 bytes (281TB)
* All keys are the same size
* Performance independent of growth
* Optimized for concurrent fetch
* Key file can be rebuilt if needed
* Inserts are atomic and consistent
* Data file may be iterated, index rebuilt.
* Key and data files may be on different volumes
* Hardened against algorithmic complexity attacks
* Header-only, nothing to build or link
In normal operation, InboundLedgers::findCreate never returns null, but
during system shutdown, it will return null.
Since this only happens in system shutdown, when findCreate returns null
the calling function stops what it was doing and returns.
During testing, an issue where destroying the application object
and creating a new one caused problems with a static PathTable. This table
is now cleared when re-initialized.
This changes expect and unexpected to receive the reason text as a
template argument, allowing the std::string conversion of char const*
parameters to take place only if the condition evaluates to false. This
cuts all calls to malloc and free on tests that pass.
* Add Backend::verify API for doing consistency checks
* Add Database::close so caller can catch exceptions
* Improved Timing test for NodeStore creates a simulated workload
Unit tests that wish to spawn threads for testing concurrency may now do so
by using unit_test::thread as a replacement for std::thread. These threads
propagate unhandled exceptions to the unit test, and work with the abort on
failure feature.
This removes the old default configuration for the "rocksdb" backend and
replaces it with the configuration that was formerly available using
the experimental backend "rocksdbquick".
The new configuration setting improves the performance of the key/value
database by changing the compaction style and tuning the size parameters for
the typical rippled workload. Testing shows a decrease in I/O spikes for both
reading and writing.
An alternative to the unity build, the classic build compiles each
translation unit individually. This adds more modules to the classic build:
* Remove unity header app.h
* Add missing includes as needed
* Remove obsolete NodeStore backend code
* Add app/, core/, crypto/, json/, net/, overlay/, peerfinder/ to classic build
The SHAMap class is refactored into a separate module where each translation
unit compiles separate without errors. Dependencies on higher level business
logic are removed. SHAMap now depends only on basics, crypto, nodestore,
and protocol:
* Inject NodeStore::Database& to SHAMap
* Move sync filter instances to app/ledger/
* Move shamap to its own module
* Move FullBelowCache to shamap/
* Move private code to shamap/impl/
* Refactor SHAMap treatment of missing node handler
* Inject and use Journal for logging in SHAMap
