The buffers_type::iterator could hold a pointer to a buffers_type that
was destroyed. This changes buffers_type::iterator to point to the
original streambuf instead, which always outlives the iterator.
Legacy workarounds for Visual Studio non thread-safe initialization
of function local objects with static storage duration are removed:
* Remove LeakChecked
* Remove StaticObject
* Remove SharedSingleton
This introduces changes in nudb to improve speed, reduce database size,
and enhance correctness. The most significant change is to store hashes
rather than entire keys in the key file. The output of the hash function
is reduced to 48 bits, and stored directly in buckets.
The API is also modified to introduce a Codec parameter allowing for
compression and decompression to be supported in the implementation
itself rather than callers.
THe data file no longer contains a salt, as the salt is applicable
only to the key and log files. This allows a data file to have multiple
key files with different salt values. To distinguish physical files
belonging to the same logical database, a new field UID is introduced.
The UID is a 64-bit random value generated once on creation and stored
in all three files.
Buckets are zero filled to the end of each block, this is a security
measure to prevent unintended contents of memory getting stored to
disk. NuDB offers the varint integer type, this is identical to
the varint described by Google.
* Add varint
* Add Codec template argument
* Add "api" convenience traits
* Store hash in buckets
* istream can throw short read errors
* Support std::uint8_t format in streams
* Make file classes part of the public interface
* Remove buffers pessimization, replace with buffer
* Consolidate creation utility functions to the same header
* Zero fill unused areas of buckets on disk
* More coverage and improvements to the recover test
* Fix file read/write to loop until all bytes processed
* Add verify_fast, faster verify for large databases
The database version number is incremented to 2; older databases can
no longer be opened and should be deleted.
Insert now blocks when the size of the memory pool exceeds a predefined
threshold. This solves the problem where sustained insertions cause the
memory pool to grow without bound.
NuDB is a high performance key/value database optimized for insert-only
workloads, with these features:
* Low memory footprint
* Values are immutable
* Value sizes from 1 2^48 bytes (281TB)
* All keys are the same size
* Performance independent of growth
* Optimized for concurrent fetch
* Key file can be rebuilt if needed
* Inserts are atomic and consistent
* Data file may be iterated, index rebuilt.
* Key and data files may be on different volumes
* Hardened against algorithmic complexity attacks
* Header-only, nothing to build or link
This changes expect and unexpected to receive the reason text as a
template argument, allowing the std::string conversion of char const*
parameters to take place only if the condition evaluates to false. This
cuts all calls to malloc and free on tests that pass.
Unit tests that wish to spawn threads for testing concurrency may now do so
by using unit_test::thread as a replacement for std::thread. These threads
propagate unhandled exceptions to the unit test, and work with the abort on
failure feature.
The SHAMap class is refactored into a separate module where each translation
unit compiles separate without errors. Dependencies on higher level business
logic are removed. SHAMap now depends only on basics, crypto, nodestore,
and protocol:
* Inject NodeStore::Database& to SHAMap
* Move sync filter instances to app/ledger/
* Move shamap to its own module
* Move FullBelowCache to shamap/
* Move private code to shamap/impl/
* Refactor SHAMap treatment of missing node handler
* Inject and use Journal for logging in SHAMap
* Fix to_string conversion
* Fix assert on debug invariant checks
* Fix the treatment of the output position when the entire output is committed.
* Add unit test
The abstract_clock is now templated on a type meeting the requirements of
the Clock concept. It inherits the nested types of the Clock on which it
is based. This resolves a problem with the original design which broke the
type-safety of time_point from different abstract clocks.
This transforms a ConstBufferSequence into a new ConstBufferSequence whose
data is encoded according to the Content transfer encoding rules of RFC2616.
The implementation does not copy any memory.
* Use signal_set as cross platform way of handling SIGINT
* Remove polling on main thread for shutdown.
* Add extra logging for received signal.
* Clean up exit handling on error in setup routines.
* Reuse isStopped() from Stoppable for status (could be isStopping() instead).
* Ctrl-C should now work for standalone mode as well on Windows.
Also small fixes to Resolver:
* Add Resolver prefix to logging.
* Fix AsyncObject::removeReference() logic.
* Fix work remaining logic.
